Category Archives: Examinations

Fairness and mutant algorithms

Posted on by

Back in 2014, I wrote two blogs (part 1 & part 2) about examinations and asked if they were fit for purpose. The conclusion – they provide students with a clear objective to work towards, the process is scalable and the resulting qualification is a transferable measure of competency. They are of course far from perfect, exams do not always test what is most needed or valued and when results are presented in league tables, they give a too simplistic measure of success.

However, I didn’t ask if examinations were fair, that is treating students equally without favouritism or discrimination.

In the last two weeks the question around fairness has been in the headlines following the government’s decision to cancel all A level and GCSE examinations in order to reduce the risk of spreading Covid-19. Whilst many agreed with this it did cause a problem, how could we fairly assess student performance without an examination?

Are examinations fair?

This is not a question about the fairness of an exam as a type of assessment, there are for example other ways of testing ability, course work, observations etc. Its asking if the system of which an examination is part treats all students equally, without bias.

In the world of assessment exams are not considered sufficiently well designed if they aren’t both reliable and valid. It might be interesting to use this as a framework to consider the fairness of the exam system.  

  • Validity – the extent to which it measures what it was designed to measure e.g. add 2+2 to assess mathematical ability.
  • Reliability – the extent to which it consistently and accurately measures learning. The test needs to give the same results when repeated. e.g. adding 2+2 is just as reliable as adding 2+3. The better students will get them both right and the weaker students both wrong.

The examining bodies will be very familiar with these requirements and have controls in place to ensure the questions they set are both valid and reliable. But even with sophisticated statistical controls, writing questions and producing an exam of the same standard over time is incredibly difficult.  Every year the same questions are asked, have students performed better or is it just grade inflation, were A levels in 1951 easier or harder than today? It’s the reliability of the process that is most questionable.

If we step away from the design of the exam to consider the broader process, there are more problems. Because there are several awarding bodies, AQA, OCR, Edexcel to name but three, students are by definition sitting different examinations. And although this is recognised and partly dealt with by adjusting the grade boundaries, it’s not possible to completely eliminate bias. It would be much better to have one single body setting the same exam for all students.

There is also the question of comparability between subjects, is for example A level maths the same as A level General studies? Research conducted by Durham University in 2006 concluded that a pupil would be likely to get a pass two grades higher in “softer” subjects than harder ones. They added that “from a moral perspective, it is clear this is unfair”. The implication being that students could miss out on university because they have chosen a harder subject.

In summary, exams are not fair, there is bias and we haven’t even mentioned the impact of the school you go to or the increased chances of success the private sector can offer. However, many of these issues have been known for some time and a considerable amount effort goes into trying to resolve them. Examinations also have one other big advantage, they are accepted and to a certain extent the trusted norm and as long as you don’t look too closely, they work or at least appear to. Kylie might be right, “it’s better the devil you know”….. than the devil you don’t.

The mutant algorithm

Boris Johnson is well known for his descriptive language, this time suggesting that the A level problem was the result of a mutant algorithm. But it was left to Gavin Williamson the Secretary of State for Education to make the announcement that the government’s planned method of allocating grades would need to change.

“We now believe it is better to offer young people and parents’ certainty by moving to teacher assessed grades for both A and AS level and GCSE results”

The government has come in for a lot of criticism and even their most ardent supporters can’t claim that this was handled well.

But was it ever going to be possible to replace an exam with something that everyone would think fair?

Clarification on grading

To help answer this question we should start with an understanding of the different methods of assessing performance.

  1. Predicted Grades (PG) – predicted by the school based on what they believe the individual is likely to achieve in positive circumstances. They are used by universities and colleges as part of the admissions process. There is no detailed official guidance as to how these should be calculated and in general are overestimated. Research from UCL showed that the vast majority, that is 75% of grades were over-predicted.
  2. Centre Assessed Grades (CAG) – These are the grades which schools and colleges believed students were most likely to achieve, if the exams hadn’t gone ahead. They were the original data source for Ofqual’s algorithm. It was based on a range of evidence including mock exams, non-exam assessment, homework assignments and any other record of student performance over the course of study.  In addition, a rank order of all students within each grade for every subject was produced in order to provide a relative measure. These are now also being referred to as Teacher Assessed Grades (TAG)
  3. Calculated grades (CG) – an important difference is that these are referred to as “calculated” rather than predicted! These are the grades awarded based on Ofqual’s algorithm. They use the CAG’s but adjusts them to ensure they are more in line with prior year performance from that school. It is this that creates one of the main problems with the algorithm…

it effectively locks the performance of an individual student this year into the performance of students from the same school over the previous three years.

Ofqual claimed that if this standardisation had not taken place, we would have seen the percentage of A* grades at A-levels go up from 7.7 % in 2019 to 13.9 % this year. The overall impact was that the algorithm downgraded 39 % of the A-level grades predicted by teachers using their CAG’s. Click here to read more about how the grading works.

Following the outcry by students and teachers Gavin Williamson announced on the 17th of August that the Calculated Grades would no longer be used, instead the Centres Assessed Grades would form the basis for assessing student performance.  But was this any fairer, well maybe a little, but it almost certainly resulted in some students getting higher grades than they should whilst others received lower, and that’s not fair.

Better the devil you know

The Government could certainly have improved the way these changes were communicated and having developed a method of allocating grades scenario stress tested their proposal. Changing their mind so quickly at the first sign of criticism suggests they had not done this. It has also left the public and students with a belief that algorithms dont work or at the very least should not to be trusted.

Perhaps the easiest thing to have done would have been to get all the students to sit the exam in September or October. The Universities would then have started in January, effectively everything would move by three months, and no one would have complained about that would they?

Double entry bookkeeping replaced by internet

Posted on by

There is an interesting question being asked at the moment, given that fact-based knowledge is so accessible using the internet, is there a case for not teaching facts at all?

According to Don Tapscott, a consultant and speaker, who specialises in organisations and technology, memorising facts and figures is a waste of time because such information is readily available. It would be far better to teach students to think creatively so that they can learn to interpret and apply the knowledge they discover online.

“Teachers are no longer the fountain of knowledge, the internet is”
Don Tapscott

Is this the solution for educators with an over full curriculum, the result of having to continually add new content to ensure their qualification remains relevant and topical? Perhaps they can remove facts and focus on skills development? After all its skills that matter, knowing is useful but it’s the ability to apply that really matters …right?

What makes you an accountant

When you start to learn about finance, you will be taught a number of underpinning foundational subjects including, law, economics, costing and of course basic accounting. Sat stubbornly within the accounting section will be double entry bookkeeping. This axiom is fiercely protected by the finance community such that if anyone questions its value or challenges its relevance they will be met with pure contempt. And yet, is the knowledge as to how you move numbers around following a hugely simple rule i.e. put a number on one side and an equivalent on the other of any use in a world where most accounting is performed by computers and sophisticated algorithms? I am sure there will be similar examples from other professions and industries. The challenge being, do doctors really need to understand basic anatomy or lawyers read cases dating back to 1892?

“Everyone is entitled to his own opinion, but not to his own facts”
Daniel Patrick Moynihan

But Knowledge is power

Daniel T. Willingham is a psychologist at the University of Virginia and the author of a number of books including, why students don’t like school. His early research was on the brain, learning and memory but more recently he has focused on the application of cognitive psychology in K-16 education.

Willingham argues that knowledge is not only cumulative, it grows exponentially. In addition, factual knowledge enhances cognitive processes like problem solving and reasoning. How knowledge Helps.

Knowledge is cumulative – the more you know the more you can learn. Individual chunks of knowledge will stick to new knowledge because what you already know provides context and so aids comprehension. For example, knowing the definition of a bond ‘a fixed income instrument that represents a loan made by an investor to a borrower (prior knowledge), enables the student to grasp the idea that anything fixed has to be paid by the company (the lender) regardless of its profitability and this is the reason debt is considered risky. (new knowledge)

Knowledge helps you remember – the elaboration effect has featured in a previous blog. In essence it suggests that the brain finds it easier to remember something if it can be associated with existing information. Using the same example from above, it is easier to remember that bonds are risky if you already knew what a bond was.

Knowledge improves thinking – there are two reasons for this, firstly it helps with problem solving. Imagine you have a problem to solve, if you don’t have sufficient background knowledge, understanding the problem can consume most of your working memory leaving no space for you to consider solutions. This argument is based on the understanding that we have limited capacity in working memory (magic number 7) and so to occupy it with grasping the problem at best slows down the problem-solving process, but at worse might result in walking away with no solution. Secondly knowledge helps speed up problem solving and thinking. People with prior knowledge are better at drawing analogies as they gain experience in a domain. Research by Bruce Burns in 2004 compared the performance of top chess players at normal and blitz tournaments. He found that what was making some players better than others is differences in the speed of recognition, not faster processing skills. Players who had knowledge of prior games where far quicker in coming up with moves than those who were effectively solving the problem from first principle. Chess speed at least has a lot to do with the brain recognising pre learned patterns.

Skills are domain specific – not transferable

There is one other important lesson from an understanding of knowledge – skills are domain specific. The implication being that teaching “transferable skills” e.g. skills that can be used in different areas, communication, critical thinking etc doesn’t work. A skill (Merriam Webster) is the ability to use one’s knowledge effectively and readily in execution or performance. The argument being that in order to use knowledge effectively, it needs to be in a specific domain.
In July 2016 the Education Endowment Foundation in the UK released the results of a two-year study involving almost 100 schools that wanted to find out if playing chess would improve maths. The hypothesis was that the logical and systematic processes involved in being a good chess player would help students better understand maths i.e. the skills would transfer. The conclusion however found there were no significant differences in mathematical achievement between those having regular chess classes and the control group.

Long live double entry bookkeeping

This is an interesting topic and open to some degree of interpretation and debate but it highlights the difficult path curriculum designers have to tread when it comes to removing the old to make space for the new. In addition there is a strong argument to suggest that core principles and foundational knowledge are essential prerequisites for efficient learning.
But whatever happens, we need to keep double entry bookkeeping, not because knowing that every debit has a credit is important but it helps structure a way of thinking and problem solving that has enabled finance professional to navigate significant complexity and change since Luca Pacioli allegedly invented it in 1494.

And the case from 1893 – Carlill v Carbolic Smoke Ball Company

Sensemaking, humility and the humanities

Posted on by

human-being-girl-picture

For a variety of reasons, I have been engaged this month in thinking not so much about examinations but what subjects should be examined.

Whilst the news has been dominated by terrorism, Trump and Brexit, we may be facing a far bigger problem, of which these news stories are a good example, how can we be sure of making the right decisions in a world of mass information, complexity and change.

People voted Brexit for a whole variety of reasons, many “facts” were presented in simple terms, we will save £350m a week and this money will go into the NHS, immigration will be reduced as we gain control over our borders. Yet these facts are far too simplistic, any level of analysis, critical thinking and challenge would have revealed the difficulty of delivering them, and in many instances they won’t be delivered. If this is the case, did people vote to leave, or stay not on the facts as presented but using other criteria, maybe they were just naive and placed far too much trust in Politicians or perhaps they had never been taught about sensemaking, humility or studied the humanities.

Sensemaking

An interesting article caught my eye earlier in the month, “Silicon Valley needs to get schooled”. it was by Christian Madsbjeg, author of the book Sensemaking and senior partner in ReD, a strategy consulting company based on the human sciences. In the article Madsbjeg argues that the reason for a lack of new and exciting products from Silicon Valley is not because of a shortage of ideas but a complete failure to understand people.

In the book Sensemaking he expands on the problem. In order to cope with complexity, we look to science, logic and the algorithm (a rules based process) for a solution. On the face of it crunching big data so that it spews out the correct answer seems perfect, but, and this is a quote from the book, Madsbjeg makes a very important point, he says “we stop seeing numbers and models as a representation of the world and we start to see them as the truth – the only truth”.  We are in fact looking at the numbers without the context of the world from which they came or a sufficiently deep understanding of the behaviours we are measuring.

We rely on science and the scientific method for so much of what we do but where people are involved we need a different approach. To put it another way “When human beings enter the equation, things go non-linear” Neil deGrasse.

Sensemaking is “how we make sense of the world so we can perform better in it”. It recognises that situations are complex and information ambiguous. It requires people to make a continuous effort to understand the connectivity that exists between people, places, and events in order to anticipate their trajectories and act accordingly.

Humility

trumpwillwin-notextIntellectual humility as defined by the authors of a recent paper entitled, Cognitive and Interpersonal Features of Intellectual Humility is the opposite of intellectual arrogance or conceit. It is in effect, recognising that you could be wrong. One of the findings from the research was that people who displayed intellectual humility were better than the control group at evaluating the quality of evidence they had been presented with. A very useful skill indeed, given the world of false news in which we currently find ourselves.

Humanities

And what job will you get after studying History for three years……

The humanities (English, History, Philosophy etc) have been given a bad press in recent times. Overshadowed by the drive to develop coding skills and with the constant chanting of STEM (Science, Technology, Engineering and Mathematics) in the background, it’s not surprising that less people are studying them. They were at an all-time low in 2014 at 6.1% of all bachelor’s degrees, a long way of the 1967 record of 17.2%.

But it is generally recognised that the humanities can teach us a lot. In another reference from Christian Madsbjeg’s book, Sensemaking, he suggests the humanities can teach us, one that other worlds exist, two that they are different and three, we learn how to imagine other worlds that in turn helps us better understand our own.

As with sensemaking and humility, are these not the types of skills we need to learn?

Examinations – what to examine?

What subjects should be examined depends to a large extent on what job you would like to do. But with the claim that 60% of 11 year olds will leave school to do jobs which have not yet been invented it’s hard to know the answer. What we do know is that the world is unlikely to slow down, change not happen, data become less available and complexity give way to simplicity. As a result, we need to teach people and so examine the skills that will help them better navigate this world. Maybe when those primary school children go onto higher education they will be studying sensemaking, humility and the humanities.

Even though the ink is barely dry on the letter sent by Theresa May bringing about our formal negotiations to exit Europe, the interesting thing is we will never know if this was a good or bad decision. Because post Brexit people will behave differently, some will work hard to make the impossible possible whilst others will continue to frustrate the process, and none of that could have been foreseen at the time.

So, let’s hope the basis for the original decision to leave was not because of the headline – We will save £350m a week and this money will go into the NHS!

Teaching to the test – another point of view

Posted on by

point-of-veiw-2A point of view is a programme on radio 4 that allows certain well-read, highly educated individuals, usually with large vocabularies to express an opinion. It lasts 10 minutes and is often thought provoking, concluding with a rhetorical question that has no answer.

This week Will Self the novelist and journalist gave his point of view on teaching to the test, as you might imagine it caught my attention. Self starts by telling a story about the life of a “good student,” and how it would unfold. He describes the way in which their concentration intensifies when the teacher states that what they are going to learn next is important and often examined. The story continues, as a result of their diligence and technique, the “good student” gets the necessary grades to go to University. They don’t however select the University on the basis of the course of study or on what they passionately wish to learn, no its based on the Universities credibility in league tables.

Upon successfully gaining a degree the student, now an employee gets a desk job that rewards a similar style of rubric mentality. As an employee, they are assessed against targets, performing well only on the ones that promise promotion and a pay rise. Eventually they retire and die.

Self concludes that this ordinary, dull, uninspiring life started back in the classroom all those years ago, when the teacher failed to educate and inspire, and simply taught to the test.

Over egging the pudding

There is a logic to this story, and it sounds ever the more inevitable as Self narrates it in his black and grey voice. But that’s all it is, a story. It avoids detail and colour, offering little regard as to the individual’s ability to reflect at some point in their life and ask searching and probing questions. It is as if somehow because the teacher highlighted the importance of one piece of knowledge it somehow stifled the student’s capacity to one day think for themselves.  Self is how they say, overegging the pudding, taking an interesting question as to the impact teaching to the test might have and serving up an omelette.

Teaching to the test is not bad

Brunel university asked a question as to what makes an unmissable lecture. In addition to many arguably more commendable answers, including the passion of the tutor and because they wanted to learn, the likelihood of the subject being taught having a high probability of being in the exam was key. Suggesting that a specific topic might be on the exam paper firstly, ensured a good attendance and secondly guaranteed the student listened intently.

Attention is important but even for the diligent student focus is vital. Learning everything is simply not possible, faced with 20 chapters, the student needs some clue as to where they need to direct their energy and time. Of course, the educationist will say that everything is important, but saying that will not make it so. Knowing that something is examinable at least gives a starting point and helps guide the student through the material quickly and efficiently. It’s also worth adding that It does not exclude the need to be inquisitive, in fact by making the student read a particular topic it may inspire them to find out more.

Exams and exam answers also provide examples of what is expected and the standard the student must reach if they are to be successful, no amount of narrative in the student handbook or curriculum guidance will do this as effectively.

The type of assessment matters

Of course, in Selfs world, teaching to the test removes the need to do anything more than learn about what will be in the exam. He suggests that students need to think outside the box rather than simply tick them. I have to admit I like that sentence.

But he does have a point, if the test is so narrow that it only assesses memory or a very small part of the syllabus then that is all the student will focus on. But that is just a bad test, this is of course where I am in danger of becoming idealistic and painting a picture that is not a true reflection as to what is happening. Not all tests are good, and undoubtedly some students will pass with limited thought and little more than good memory skills. Yet with changes in technology it becomes ever more possible to build tests and simulations that asses the student ability to perform in real world situations, and for that matter think outside the box.

Teaching to the test has become a term used to describe bad teaching and poor assessment and no one would agree that either of these are desirable. But it is not the process that’s problematic, it’s the application. Testing in its many forms is part of learning but it needs done well and thoughtfully.

In conclusion

Having now read the blog I would encourage you to listen to Will Self – click. It is of course not for me to say who presents the right point of view, you need to make up your own mind. For those however who were taught to the test no matter how long ago, you probably won’t understand even what I am asking because to the best of my knowledge this question has never been tested before……………?

 

Test obsession and Test Anxiety

Posted on by

Tests anxiety

“We live in a test conscious and test giving culture in which the lives of people are in part determined by their test performance”

Sarason, Davidson, & Lighthall

What’s interesting about this statement is, it was first published in 1960 and was based on students in the US, yet would not seem out of place in describing the situation in the UK.  The UK, as with so many other things has unfortunately caught up with the US and become a nation that tests and measures…everything.

Where a person’s worth is judged only by the tests they have passed it is perhaps not surprising that examination success has become so important and test anxiety increased.

But it’s not just the UK, this is a global obsession, take China for example where the pressure to succeed has become so intense that cheating in the Gaokao, the nation’s A-Chinese-invigilator-sca-010university entrance exam is a major problem. The government has not been slow to react and for the first time anyone found cheating will face a possible seven year jail sentence. In Ruijin, east China’s Jiangxi Province, invigilators use instruments to scan students’ shoes before they entered the exam hall, while devices to block wireless signals are also used to reduce the opportunity to cheat.

Test anxiety or stress

Stress is a broad term that is experienced when you find yourself in adverse or demanding circumstances, sitting an exam perhaps. Test anxiety is a situation specific type of stress, experienced by people who find examinations threatening. Recently, there has been an increased interest in exam stress and test anxiety in the UK and a need for it to be given closer academic scrutiny.

The research so far shows that test anxiety can actually impair learning and hurt test performance. And this is the issue, are students underperforming in examinations, which as stated above can have a significant impact on their lives not because of their lack of knowledge or even their ability to apply knowledge, but simply because the medium used to assess them is an exam.

In simple terms test anxiety effects exams results and exam results play a major part in people’s lives.

There are three components of test anxiety (Zeidner 1998)

  • Cognitive – the negative thoughts you can have during tests e.g. “if I fail this I will fail all my examinations” and the performance limiting difficulties experienced as a result of anxiety e.g. inability to read questions clearly or solve problems.
  • Affective – physical symptoms e.g. trembling, tension etc.
  • Behavioural – test anxiety creates an environment that encourages students to avoid studying or best delay it.

The reason people develop test anxiety is thought to be rooted in certain social issues e.g. how you are judged by others and the fear of failure in the public domain. It may also be related to the type of anxiety people experience when they have to make a best man’s speech, for example. Another aspect is that it is not always what others think, but what you think of yourself that is the issue and so the expectation of exam failure could impact on an individual’s ego and self-esteem.

Phase one is OK BUT

I think in the UK we are through the worst part of this, let’s call it phase one, and by that I mean we know that examinations and testing are not the answer, and that people are not their exam result. We have learned this the hard way by producing groups of exam qualified students, releasing them into the world of work, ill prepared to cope with the demands of the workplace. In addition, we have developed helpful techniques that enable people to better cope with test anxiety. Some of these I have discussed in previous blogs, Stress or Pressure – Don’t let the bridge collapse, Exam stress – Mindfulness and the “7/11” to name but two.

BUT………we still have some way to go with phase two, which involves answering the question, what do we replace exams with if they are so bad? And until we solve that, helping good people perform in the system we have just now is the best we can do.

 

Currywurst , apples and the £33,000 a year boarding school

Posted on by

currywurst

Currywurst fast becoming the most popular dish in Berlin

I have three relatively unrelated stories this month.

Berlin

The first follows a trip to Berlin this weekend and the obligatory city tour. Just of Unter den Linden you will find a sunken glass plate between the pavement that provides a view into a room full of empty bookshelves, sufficient to house the 20,000 books that were destroyed on this site on the 10th of May 1933. Some 70,000 people, many German university students gathered to burn books with “unGerman” ideas. Joseph Goebbels joined the students at the bonfire and declared: ” The future German man will not just be a man of books, but a man of character.” “It is to this end that we want to educate you.”

This was not only symbolic but practical. Without these books or to be precise only having the ones that remained, it might just be possible to create a nation of people who all think the same? Who all share the same ideals and live life by the same values. And yet of course this is not how you create an educated society, a society able to think for itself, the result will be compliance and idealism. Which was more likely what Goebbels had in mind.

Goodwill Hunting

When I got home one of my favourite movies had already started. Good Will Hunting tells the story of a maths genius, Will Hunting who struggles to come to terms with his past and as a result is unable to make a commitment for fear of being let down. But for me the best scene is where Will confronts a first year grad student who has been belittling his friend Chuckie.

Will: “Wood drastically underestimates the impact of social distinctions predicated upon wealth, especially inherited wealth…” You got that from Vickers. “Work in Essex County,” Page 98, right? Yeah I read that too. Were you gonna plagiarize the whole thing for us, you have any thoughts of your own on this matter? Or is that your thing, you come into a bar, you read some obscure passage and then you pretend, you pawn it off as your own, your own idea just to impress some girls? Embarrass my friend?

Will: See the sad thing about a guy like you, is in about 50 years you’re gonna start doin’ some thinkin’ on your own and you’re gonna come up with the fact that there are two certainties in life. One, don’t do that. And two, you dropped a hundred and fifty grand on a fxxxin’ education you coulda’ got for a dollar fifty in late charges at the Public Library.

Clark: Yeah, but I will have a degree, and you’ll be serving my kids fries at a drive-thru on our way to a skiing trip. Will: [smiles] Yeah, maybe. But at least I won’t be unoriginal.

This of course challenges the idea that knowledge is the same as thinking. Knowing something is not the same as having manipulated the information in your head, questioned and explored the arguments from many angles, the result should be an opinion, not someone else’s but your own.

Wellington college

And finally to bring my trilogy up to date. Only this week Wellington College who boast George Orwell and Sebastian Faulks as past students announced that have introduced a six-hour interview process to help teachers see beyond children’s over tutoring. They encourage families to spend more quality time instead of having their children tutored, Julian Thomas, Wellington’s head master, said: “When parents ask how should they prepare their children for our selection process, we tell them ‘have supper together and talk as a family; go to plays and good films and discuss them. Help them to think critically about the world around them; enjoy interesting conversations go out for walks and see the world”.

What Wellington are trying to do is encourage behaviours that will help children become thinking adults.

In conclusion

Knowledge like information is becoming increasingly accessible, unlike Germany in 1933 we have no restrictions, the internet has given us that power. But learning is more than knowledge you have to take the facts and make them your own, stitching them together carefully and thoughtfully, and that takes time and a considerable amount of effort. On the whole absorbing information will probably not come as easy to you as it did to Will Hunting, yet even with his genius he only knew what he had read. His learning had in fact only just begun as he went looking for the real life experiences that would test if he really understood anything at all.

Examinations are moving more towards tests that require you to think, asking that you provide an opinion or make recommendations given a set of circumstances, case studies are a good example. Of course you need knowledge as a foundation but the ability to sift quickly through the vast amount of information available may in time become more important than knowledge itself .

Ps apples is taken from Good Will Hunting – “How ya like those apples”

 

 

The tip of the iceberg – exam tipping is becoming obsolete

Posted on by

tip1

Assessment is changing, there was a time when all examinations were sat in a room, the answers would be hand written on a piece of paper and a retired English teacher would stand at the front reading out instructions as to what you could and couldn’t do in the next three hours.

Not any more…….you request a date that is convenient, turn up at the exam center, no longer is this a sports hall, it might be a driving school test center or the college you studied at. Then you log onto the PC and answer questions on the computer screen in front of you. The results may be immediate; it depends on if it is “human marked” or computer marked.

But in some ways these changes are only the tip of the iceberg!

What no past exam papers.

As examinations move into the digital world we are seeing other changes as well. There is a move towards objective testing, scaled scoring and examining bodies no longer providing past exam papers, what did you say, no past exam papers……!

This is partly down to the nature of the test i.e. you can’t provide an exact replica of a past exam question if it is an objective test. Remember objective test questions are randomly selected from a pool, and are different for each student. But there is also a shift towards some examining bodies only providing an example of the type of questions that could be set rather that a continuous flow of, the last exam papers.

If the test changes – how you study (and teach) has to change

Now for someone who has advocated that students analyse past exam questions in order to identify key areas so as to better direct their studies, this is a bit of a blow. It has also been the method I have used in the past to focus my own delivery in class and on line. Of course using past exam questions has always been much more than just spotting key areas, it is about focus, providing a place to start, showing content in the right context, helping with writing style etc.

There will still be past questions, sample questions will be provided. What we don’t know is how representative they will be of the examination. Or will it be as we have seen in the past with pilot and specimen papers, they change over time, drifting away from the original in terms of style and emphasis. Although I can see the logic in examining bodies not releasing papers, I hope they will continue to keep the sample papers fresh, in keeping with current thinking about the subject and how it will be examined.

What to do?

Students and tutors still need focus, there has to be emphases on key areas in order to chunk the content so that it can be more easily learned, it’s just that we won’t be able to use past questions or at least as much as we have in the past. That emphasis will now have to come from articles written by the examiners, examiner reports and syllabus weightings. If faced with a new subject where there is only one sample paper, it will be necessary to read the guidance from the body closely, noting reference to “this being a key part of the subject” or “one the examiner thought was answered badly in the past.” These together with the syllabus weightings and specific learning outcomes will have to be your guide. It is of course possible that the subject has not changed much from before and so some of the older past question can be used. As far as questions style is concerned then that will have to come from the questions and answers that are published, it may not be ideal but it’s the best we can do.

The overall impact of these changes is that students will have to know more, something that is hard to argue with. Students and tutors alike will have to devote far more time to the subject, which is fine if students have the time and can afford the extra costs involved in longer periods of study.

But it’s not all bad news, new technologies can help students make the most of dead time, studying on the train using their mobile phone for example. Also knowledge is more freely available than ever before as many top institutions provide a huge amount of free easy to access content online.

One final thought, examinations may change and they may not be fair but on the whole they are equal, everyone as before is in the same boat, and someone will always pass, wont they!

The future – Sitting the exam at home?

On line exams

An online student, all be it a mature one shows his ID to the online assessor

And maybe even the exam room will become obsolete. Proctur U is a US based company that also has a presence in the UK offering online invigilation. Watch this video to see how it works and judge for yourself

 

 

Video killed my teacher – metaphorically speaking

Posted on by

Video killed the radio star

What did you do the last time you needed to repair, cook or dare I say learn something? Did you google it and follow the link to YouTube? If so you are no different to the over one billion people who actively use YouTube every month.

This blog is not actually about YouTube but the medium of video and the increasingly important role it plays in our daily life and how we use it to learn.

 

 

Social learning and the bobo doll

Albert Bandura is the Professor Emeritus at Stanford University and is widely regarded as one of the greatest living psychologists. He is perhaps most famous for his Social learning theory, the theory of how people learn by observing others, and the bobo doll experiment. Click here to listen to Bandura himself explain.

Behavioral theories of learning suggested that all learning was the result of associations formed by conditioning, reinforcement, and even punishment, see Skinner and Pavlov. Bandura’s social learning theory proposed that learning can also occur simply by observing the actions of others. And that is where the true value of video lies, it is in the ability of people to watch what others do and learn from them.

What makes a good learning video?

Firstly, as with any form of delivery it needs to planned and structured. What is the purpose of the video, why use video and not send an email? Think about the audience, why will they want to watch it, what makes it relevant for them? Break it into three sections, a beginning, where you tell the audience what you are going to tell them, the middle, where you actually tell them, and the end where you tell them what you have told them.

Secondly It has to be relatively short, 10 minutes is a maximum. Even 6 minutes of good video takes a lot of planning, equally it wont test concentration levels too much. This does not mean you can’t record many hours of video, it just needs to be chunked, labeled and structured so it can be easily followed.

And lastly think about your delivery. Pace, tone of voice and body language all help the learner. This is where you manage the mood of your audience, if your happy they will be happy. Generally, speak more slowly than you would normally but be careful toooo slowww can be boring, vary how you say something depending on what you’re saying. Also think about the visuals and if it would be better to show an image rather than talk. But don’t go mad and put too much on screen all at the same time, it gets confusing.

Examples of good video 

But of course the best way to explain the power of video in learning is to show the videos.

1.The queen of cooking Delia is also the expert of slow deliberate, perfectly planned presenting. Here she explains how to cook an omelet, notice the attention to detail.  Ps Delia left school at 16 without a single GCE O-level. 3.43 minutes in length.

2.Here is someone who breaks the presenting rules, certainly the one that says don’t talk too fast. However, CGP Grey is great at using visuals, his dialogue is fast but incredibly informative, its packed with information, and it’s funny. If you are confused by the US elections, you won’t be after watching this.  5.19 minutes in length.

3.Crash course is a little like Khan academy which I have written before, what makes it different is the humour and how it is shot to camera using powerful visuals. Watch this clip if you want to learn about supply and demand. This pushes the boundaries time wise at 10.21 minutes.

4.This is the big one certainly as far as hits are concerned. James Stevens, Vsauce, is watched by 19 million people. This one answers the questions as to, what would happen if everyone jumped at once? 7.12 minutes in length.

Why cramming works and making stuff up is okay

Posted on by

Will making stuff up

Will making stuff up

To a certain extent I have spent much of my career making things up. When I was a student that was not the case, I listened and learned and so when I spoke, I spoke with confidence that what I was saying was correct, because someone had just told me it was. Yet knowing is only the start, and in some ways a poor relative of the “figuring it out for yourself” technique.  I am reminded of quote from the film Good Will Hunting, which along with Dead Poets capture some really magical moments in learning.

Will Hunting – “See, the sad thing about a guy like you is in 50 years you’re gonna start doing some thinkin on your own and you’re gonna come up with the fact that there are two certainties in life.” “One, don’t do that.” “And Two, you dropped a hundred and fifty grand on a f***in education you coulda got for a dollah fifty in late charges at the public library.”

 Question practice – the secret to exam success

Having had no real formal educational training I have been exploring ideas as to why some techniques work and others don’t, why it is that student A passes yet student B who did exactly the same, failed. One clear observation from over twenty years in the high stakes exam world is that the most important activity that a student can engage in is, question practice. As a lecturer I would make statements, explain them using real world examples, get students to laugh, and maybe even enjoy the subject. But, the very best learning seemed to happen when the student was required to do a question. So it was with great interest that I read of some research that came out of the US in 2011, it’s called Retrieval Practice.

 Retrieval practice – the power of cramming

Retrieval practice is simply the process of retrieving something from memory.  So for example if I asked you, who was the Prime Minister that took us into the European Economic Community in 1973, you might say, on reflection Edward Heath. You already knew the answer but were forced to recall it. If however you were not sure who it was and were subsequently told (given feedback) it was Edward Heath and that Harold Wilson in 1975 held the first referendum, you are likely to remember both. But the most interesting and perhaps surprising aspect of this research is that not only can you recall the facts, it also leads to a deeper learning in so much that you can answer questions on related information. This in some ways gives credence to the idea that cramming information, maybe not at the last minute could be beneficial, not simply because you will remember it for a few hours’ but that it will lead to deeper learning.

Mark McDaniel is a psychology professor at Washington University in St. Louis

“We think of tests as a kind of dipstick that we insert into a student’s head, an indicator that tells us how high the level of knowledge has risen in there when in fact, every time a student calls up knowledge from memory, that memory changes.” “Its mental representation becomes stronger, more stable and more accessible.”

Jeffrey Karpicke, a professor of cognitive psychology at Purdue University

“Retrieving is the principal way learning happens.” “Recalling information we’ve already stored in memory is a more powerful learning event than storing that information in the first place,” he says. “Retrieval is ultimately the process that makes new memories stick.” “Not only does retrieval practice help students remember the specific information they retrieved, it also improves retention for related information that was not directly tested.”

Final thoughts

And so I am pleased to say that what I have observed in the classroom, that question practice improves exam results might be a little simplistic and that not only does it help students pass exams they might actually have been learning something at the same time 🙂

If you want to read more follow these links

To Really Learn, Quit Studying and Take a Test (New York Times)

Researchers Find That Frequent Tests Can Boost Learning (Scientific American)

Election special – who has had the best education?

Posted on by

The direction in which education starts a man will determine his future in life

Plato

uk-politiciansInspired by the election and the personalities that have been thrust upon us during the run up to next Thursdays vote, I thought it might be interesting to look at the academic backgrounds of those that have ambition to be Prime Minster. It might even give you some help in choosing who to vote for.

The contenders

8761f28.jpgDavid Cameron – born 1966 – David William Donald Cameron was bought up in Peasemore, Berkshire his father was a stockbroker and mother a retired Justice of the Peace. He attended Heatherdown Preparatory School (12 O levels – 4 A’s, 5 B’s, a C and 2 below C) before moving to Eton, the school that can boast 19 past British Prime Ministers. After A levels (3 A grades) he went to Brasenose College, Oxford, from which he graduated in 1988 with a first in philosophy, politics and economics.

 

Ed MilibandEd Miliband – born 1969 – Edward Samuel Miliband went to Haverstock Comprehensive School (Labours Eton), North London. His father, a Polish Jew, was one of the leading Marxist theorists of his generation. His mother, Marion Kozak, is a long-standing human rights campaigner. After completing his A Levels (A,A,B,B) he read Philosophy, Politics and Economics at Corpus Christi College, Oxford, gaining a 2:1 and later Economics at the London School of Economics where he received his MSc.

 

The King makers

Nick CleggNick Clegg – born 1967 – Nicholas William Peter Clegg was raised in Buckinghamshire by his Dutch mother, a special needs teacher and his half Russian father, Nicholas P Clegg CBE, the chairman of united trust bank. Clegg was educated at two independent schools, Caldecott School in Farnham Royal in South Buckinghamshire, and later at Westminster School in Central London. He went on to study Social Anthropology at Cambridge (2:1) and continued post graduate studies at University of Minnesota through a scholarship where he wrote a thesis on the political philosophy of the Deep Green movement. He then went to the College of Europe in Bruges for his Masters.

Nigel FarageNigel Farage – born 1964 – Nigel Paul Farage grew up in the Kent village of Downe. His father, Guy Justus Oscar Farage, was a stockbroker and heavy drinker left the family home when he was five. He was educated at Greenhayes School for Boys in West Wickham, then Eden park before joining  Dulwich College (The Dulwich experiment)  an independent public school in south London. After A levels he decided not to go to university, but to work in the City, trading commodities at the London Metal Exchange.

Nicola SturgeonNicola Sturgeon – born 1970 – Nicola Ferguson Sturgeon is the eldest of three daughters born to Robert Sturgeon, an electrician, and Joan Kerr Sturgeon a dental nurse. She grew up in Prestwick and Dreghorn and attended Dreghorn Primary School and then Greenwood Academy. She later studied at the University of Glasgow, where she read Law and graduated with a Bachelor of Laws (Hons) and a Diploma in Legal Practice.

 

And the winner is………

By most people’s standards all our candidates had a very good education, progressed academically and enjoyed school life. In addition, with the exception of Nigel Farage whose father left when he was 5, all have had full parental support.

Some statistics-In 1981 around the time David Cameron was taking his O levels only 25% of students got 5+ GCSE O levels, and around 10% went on to gain 3 A level passes. In 2014 (admittedly many years later – but it will give us an idea) Oxford received 17,000 applications for 3,200 places, that’s a 5% chance of being accepted.  So Cameron, Miliband and Clegg all stand out. Farage and Sturgeon are clearly well educated but do not have such a strong academic background.

Not a political statement but

Overall for me the person who has had the “best” education is Nick Clegg. David Cameron and to a lesser extent Ed Miliband have had a rather insular academic journey. But Clegg went to Cambridge, then travelled to the US and Europe to study, which must have broadened his views, exposing him to the opinions and cultures of other students, many with different backgrounds . And in second place I would probably go for Nigel Farage, yes I did say Nigel Farage, not the man you understand, but as a good A level student he went into the world of work, the university of life argument.

PS – The former Conservative Prime Minister John Major famously left his comprehensive school, Rutlish, in London with three O-Levels: history, English language and English literature.