The tip of the iceberg – exam tipping is becoming obsolete

tip1

Assessment is changing, there was a time when all examinations were sat in a room, the answers would be hand written on a piece of paper and a retired English teacher would stand at the front reading out instructions as to what you could and couldn’t do in the next three hours.

Not any more…….you request a date that is convenient, turn up at the exam center, no longer is this a sports hall, it might be a driving school test center or the college you studied at. Then you log onto the PC and answer questions on the computer screen in front of you. The results may be immediate; it depends on if it is “human marked” or computer marked.

But in some ways these changes are only the tip of the iceberg!

What no past exam papers.

As examinations move into the digital world we are seeing other changes as well. There is a move towards objective testing, scaled scoring and examining bodies no longer providing past exam papers, what did you say, no past exam papers……!

This is partly down to the nature of the test i.e. you can’t provide an exact replica of a past exam question if it is an objective test. Remember objective test questions are randomly selected from a pool, and are different for each student. But there is also a shift towards some examining bodies only providing an example of the type of questions that could be set rather that a continuous flow of, the last exam papers.

If the test changes – how you study (and teach) has to change

Now for someone who has advocated that students analyse past exam questions in order to identify key areas so as to better direct their studies, this is a bit of a blow. It has also been the method I have used in the past to focus my own delivery in class and on line. Of course using past exam questions has always been much more than just spotting key areas, it is about focus, providing a place to start, showing content in the right context, helping with writing style etc.

There will still be past questions, sample questions will be provided. What we don’t know is how representative they will be of the examination. Or will it be as we have seen in the past with pilot and specimen papers, they change over time, drifting away from the original in terms of style and emphasis. Although I can see the logic in examining bodies not releasing papers, I hope they will continue to keep the sample papers fresh, in keeping with current thinking about the subject and how it will be examined.

What to do?

Students and tutors still need focus, there has to be emphases on key areas in order to chunk the content so that it can be more easily learned, it’s just that we won’t be able to use past questions or at least as much as we have in the past. That emphasis will now have to come from articles written by the examiners, examiner reports and syllabus weightings. If faced with a new subject where there is only one sample paper, it will be necessary to read the guidance from the body closely, noting reference to “this being a key part of the subject” or “one the examiner thought was answered badly in the past.” These together with the syllabus weightings and specific learning outcomes will have to be your guide. It is of course possible that the subject has not changed much from before and so some of the older past question can be used. As far as questions style is concerned then that will have to come from the questions and answers that are published, it may not be ideal but it’s the best we can do.

The overall impact of these changes is that students will have to know more, something that is hard to argue with. Students and tutors alike will have to devote far more time to the subject, which is fine if students have the time and can afford the extra costs involved in longer periods of study.

But it’s not all bad news, new technologies can help students make the most of dead time, studying on the train using their mobile phone for example. Also knowledge is more freely available than ever before as many top institutions provide a huge amount of free easy to access content online.

One final thought, examinations may change and they may not be fair but on the whole they are equal, everyone as before is in the same boat, and someone will always pass, wont they!

The future – Sitting the exam at home?

On line exams

An online student, all be it a mature one shows his ID to the online assessor

And maybe even the exam room will become obsolete. Proctur U is a US based company that also has a presence in the UK offering online invigilation. Watch this video to see how it works and judge for yourself

 

 

Why cramming works and making stuff up is okay

Will making stuff up

Will making stuff up

To a certain extent I have spent much of my career making things up. When I was a student that was not the case, I listened and learned and so when I spoke, I spoke with confidence that what I was saying was correct, because someone had just told me it was. Yet knowing is only the start, and in some ways a poor relative of the “figuring it out for yourself” technique.  I am reminded of quote from the film Good Will Hunting, which along with Dead Poets capture some really magical moments in learning.

Will Hunting – “See, the sad thing about a guy like you is in 50 years you’re gonna start doing some thinkin on your own and you’re gonna come up with the fact that there are two certainties in life.” “One, don’t do that.” “And Two, you dropped a hundred and fifty grand on a f***in education you coulda got for a dollah fifty in late charges at the public library.”

 Question practice – the secret to exam success

Having had no real formal educational training I have been exploring ideas as to why some techniques work and others don’t, why it is that student A passes yet student B who did exactly the same, failed. One clear observation from over twenty years in the high stakes exam world is that the most important activity that a student can engage in is, question practice. As a lecturer I would make statements, explain them using real world examples, get students to laugh, and maybe even enjoy the subject. But, the very best learning seemed to happen when the student was required to do a question. So it was with great interest that I read of some research that came out of the US in 2011, it’s called Retrieval Practice.

 Retrieval practice – the power of cramming

Retrieval practice is simply the process of retrieving something from memory.  So for example if I asked you, who was the Prime Minister that took us into the European Economic Community in 1973, you might say, on reflection Edward Heath. You already knew the answer but were forced to recall it. If however you were not sure who it was and were subsequently told (given feedback) it was Edward Heath and that Harold Wilson in 1975 held the first referendum, you are likely to remember both. But the most interesting and perhaps surprising aspect of this research is that not only can you recall the facts, it also leads to a deeper learning in so much that you can answer questions on related information. This in some ways gives credence to the idea that cramming information, maybe not at the last minute could be beneficial, not simply because you will remember it for a few hours’ but that it will lead to deeper learning.

Mark McDaniel is a psychology professor at Washington University in St. Louis

“We think of tests as a kind of dipstick that we insert into a student’s head, an indicator that tells us how high the level of knowledge has risen in there when in fact, every time a student calls up knowledge from memory, that memory changes.” “Its mental representation becomes stronger, more stable and more accessible.”

Jeffrey Karpicke, a professor of cognitive psychology at Purdue University

“Retrieving is the principal way learning happens.” “Recalling information we’ve already stored in memory is a more powerful learning event than storing that information in the first place,” he says. “Retrieval is ultimately the process that makes new memories stick.” “Not only does retrieval practice help students remember the specific information they retrieved, it also improves retention for related information that was not directly tested.”

Final thoughts

And so I am pleased to say that what I have observed in the classroom, that question practice improves exam results might be a little simplistic and that not only does it help students pass exams they might actually have been learning something at the same time 🙂

If you want to read more follow these links

To Really Learn, Quit Studying and Take a Test (New York Times)

Researchers Find That Frequent Tests Can Boost Learning (Scientific American)

Teaching to the test – Interesting research but the fat lady is still in good voice

Fat lady still singingThis week researchers from the University of East Anglia released some very interesting findings that resulted from testing 594 bio-science students in their first week of term at five universities.

The students selected would be considered by many more than competent in their subject, almost all had a grade A*, A or B in biology at A-level. Yet when they were given 50 minutes to answer 38 multiple choice questions on cells, genetics, biochemistry and physiology from their A level core syllabus, they only got 40% correct. The period of time between the students sitting their A levels and the test was three months.

Lead researcher for the study, Dr Harriet Jones, said: “What our research shows is that students are arriving at university with fantastic A-level grades, but having forgotten much of what they actually learned for their exams.”  She went on to say that the trend to teach to the test, to ensure good results for schools’ reputations, was the problem.

The schools are to blame then

The facts of the research are clear, students who had successfully passed a test, were unable to pass a similar test three months later. The conclusion reached is that the students did not understand (see my blog on understanding) their subject well enough and passed their A levels probably using little more than memory. And who is to blame, the schools of course, for teaching to the test. Why the school do this is worthy of further debate, but government pressure and the impact of league tables will certainly be in the mix.

But do employers not accuse Universities of delivering up similar ill prepared students. The test is different but from the employers perspective the result is the same. A University student who professes to know something but when tested “in the real world” doesn’t.

Does this mean that Universities are also teaching to the test!

It’s about the test etc

The problem is not in teaching to the test; the problem is with the test, the pass mark and possibly the marking. If the test was more Testing but for what!aligned to what the student needs to know/do at a fundamental level, the pass mark sufficiently high and the marker having some degree of autonomy to form judgements, then the results would probably be different. It could of course be that the exams are easier – Exam chief: ‘you don’t have to teach a lot’ for our tests.

The big criticism of teaching to the test is, it results in a narrowness of understanding, little in the way of depth and does not push students to think in abstract and creative ways. But if the test, which incidentally does not have to be in the exam hall or on paper/PC was able to “test” for these qualities then teaching towards it would perhaps be more acceptable.

Bottom line

Teaching to the test is unlikely to change, in fact given the popularity of league tables  in education just now it may well increase, but with more effective testing the results might be better students, happy Universities and even happier employers.

 

 

50 shades of Grade…..Measuring students worth

I don’t often write specifically about the market that I am most closely involved, that of accountancy training and education. But there does seem to be an anomaly in the way the accountancy world measure success that isn’t the case in many other professions and examinations.

To become a qualified accountant in the UK (ACCA, ICAEW,CIMA etc,  yes there are more) you have to pass a number of demanding examinations and submit evidence of practical experience. The exams are taken over 3 years and everything rests on a number of exams of 100 marks each, of which you need to get 50%. The pass rate for accountancy students varies from paper to paper but is around 60% In training and education terms these exams are often referred to as High stake exams. For some failure is not simply a setback in terms of time, it could result in a lost job.

50% good – 49% bad

The purpose of such exams (including the practical experience) is of course to ensure that those that pass are competent. But how competent….?  It seems that you are either competent or not, if you get 49% you are not competent, if you get 50% then you are. This makes the margin for error very high, many a career has taken a dramatic turn based on 1 mark. Now of course in any exam there has to be a point where someone succeeds and another fails but does competence not have many shades to it, is it not a continuous process rather than a discreet one.

Shades of Grade

You may be one of thousands of students (>300,000 actually) who have just had your A level results. But of course you don’t just pass or fail, in fact this has not been the case since 1963, even before then there was an indication of the mark. You are given a grade from A through to E and all are considered a pass, 98.1% passed their A levels this year.

And the grades refer to the marks e.g. E is 40%, a D is 50%, a C is 60%, a B is 70% and an A is 80%. A similar grading exists for Degrees, 2.1, 2.2 etc and for Law exams, Pass, Commendation, Distinction.

No pressure for excellence

So why is this not the case in accountancy? To be honest I am not sure. But by introducing grades above a pass you not only provide employers with a better indication of competency, you encourage colleges to teach to a higher and deeper level, motivated by the students who want to learn more. And that for me is the most important point; you create an incentive for students to try harder. At the moment if you are scoring 55% then why put in more work, a pass is a pass, but a distinction well that will make you stand out from the crowd.

Of course the market will adjust its perception of what a pass means, for example employers may only want students with a distinction rather than a simple pass, but they have always looked and will continue to look for differentiation, at least this is a meaningful one.

What it does not do is fix the 49% you fail problem, and careers will still hinge on 1 mark, but that unfortunately is the way with exams and the reason exam technique is so important.

So to all the accountancy examining bodies why not introduce grades – grey is good and isn’t it an accountants favourite colour?

Thank you for the music – listening to music when revising

May and June are the traditional months when students around the world lock themselves away to revise for their exams.

In China for example over 9 million students will be sitting the university entrance exams.

Last May (2012) Teenagers at Xiaogang school in Hubei province were pictured hooked up to bags of intravenous fluids hanging from the classroom ceiling to boost energy levels during the revision period. An extreme action by anyone’s standards, but perhaps an example of how much pressure students feel this time of year.

As I mentioned in last month’s blog my daughter is currently caught up in this May/June exam frenzy. So once again I found myself looking to her for inspiration. What was she doing, how did she revise? This is not because she is a perfect example of a revision student, in many ways she is not, but I do think she is typical of many.

What does Beth do?

  • Makes notes from her notes – This is a standard exam skill, reducing content down into measurable and personal chunks. She does use mind maps (possibly my influence) but not exclusively.
  • Prepares as if she has to teach someone else – this I find interesting and has certainly not come from me. She writes on a white board the key points as if she was going to teach that subject. I like this idea, as many teachers and lecturers will tell you nothing focuses the mind nor motivates you more than having to teach it to others.
  • Practices past exam questions…of course!
  • Studies while listening to music – now this is the one that intrigued me and as a result I have devoted the rest of the blog to answering the question …..

Is it a good idea to revise whilst listening to music?

As ever the science needs much interpretation.

It’s a bad idea

Researchers from the University of Wales, tasked 25 students with memorising lists of consonants. Some were shown the letters while sitting in silence, others while listening to music by their favourite bands or by groups they had a strong aversion to. The conclusion was that listening to music, hampered their recall.

So it’s good then

Scientists at Stanford University, in California, believe there is a molecular basis for music known as the “Mozart Effect“. It was discovered that rats, like humans, perform better on learning and memory tests after listening to a specific Mozart Sonata in D.

But then there is the evidence that suggests that switching attention when trying to learn as might be the case with listening to music slows down the cognitive process.

Yet you cannot ignore the research that clearly shows music has the ability to alter your brain and induce relaxation which in turn helps create an ideal state for learning.

Watch what happens to your brain when you listen to music.

Hopefully you get the idea.

Conclusions

  • Listening to music puts you into a more relaxed frame of mind and that is of course good for learning. So listening to music before or after revising can help.
  • If you do want to listen to music, avoid music that requires you to shift your attention. This would suggest you should not listen to  music with lyrics  as it can mean you need to think about what is being said nor should you listen to something new that you may not have heard before. This is one of the reasons classical, in particular baroque music is the preferred choice of many students. Also don’t play the music too loud, keep it as background noise.
  • If there are specific facts that you simply need to know, then avoid listening to music completely, give it your full attention. But you can’t concentrate at this level all day, only for short periods.
  • On the whole be consistent don’t keep changing the type of music, you need familiarity.

Music to help you study

The internet has many websites that offer relaxing and helpful music, here are a few that might help.

Exams in the headlines – but for the wrong reasons again!

Well it has certainly been an interesting time in the exam world! Here are just a few of the headlines.

Behind all of these headlines are personal stories, for example students who can’t get into sixth form colleges because they didn’t get the necessary grades. The argument being, that if the grade boundaries had not changed (by 10%) resulting in them getting a D and not a C then they would have got into their colleges of choice.

Stacey Cole chief executive of Ofqual said “the grades are right “

Although the impact on individuals is considerable, statistically the change was small. The proportion of test papers awarded at least an A fell by 0.8 percentage points to 22.4 per cent (this was 8.6 per cent in 1988!) the first annual drop since GCSE exams were first sat in 1988. A* grades also fell but only by 0.5 percentage points to 7.3 per cent.

However, when something goes up, it must come down so perhaps it was inevitable that the ever increasing improvement in student grades had to reverse or at least plateau.

What has changed/gone wrong? 

  • Teachers are encouraged to prepare students for the exam, because parents, employers and educational institutions measure success at least partly (exclusively?) by the results.
  •  Students are better at exam skills than ever before. After all, this blog is about how to pass exams and the skills needed to help.
  • Universities and employers seem equally unimpressed with the quality of students and candidates they get, complaining they don’t have basic levels of numeracy, literacy and common sense!

Answers to some of the questions

Have students been getting better each year?

I think the answer is yes, the results prove they have. But maybe they have been getting better at passingexams. And not improving on some of the more difficult to measure skills like, attitude, common sense, being thoughtful. Exams don’t give you time to be thoughtful! This might explain why employers are so unhappy.

But they may just be getting better…..
On the 6th of May 1954 Roger Banister ran the 4 minute mile, it is now the standard of all male middle distance runners. Does this mean the mile is now shorter…….or maybe runners have improved?

Why did someone not say something?

The pressure to succeed, measured by exam results has been so great on teachers, examining bodies and students that no one was willing nor was it in their best interest to say, “this just doesn’t make sense.”

Why do we have exams, to test knowledge/competence or to separate the best from the rest?

They should probably be to asses’ knowledge but are mostly used to try and pick the best people.

Have exams been made easier, the dumbing down argument?

This is tricky, and although you can compare exam papers it’s a bit like comparing Wayne Rooney with George Best. You can debate the pros and cons but I am not sure it is conclusive; things were just different in the past.

But it’s not fair

What does seem clear in this whole debacle is that raising the grade required
half way through the year is not an example of exam rigour, it’s an example of being unfair and that is the one thing exams should never be.

Also see my blog what do exams prove

Back to more exam tips next month….

Are you taught in the most efficient & effective way? – The flipped classroom

To pass any exam you need to learn in the most efficient and effective way but only part of that is down to you the student, much is about the way in which you are taught.

Little has changed in the way knowledge has been transferred over the years. Initially teaching was one person talking to a few people; let’s call this the Neanderthal approach, the first classroom? Then we had a major innovation the printing press, around 1450, which made it possible for knowledge to flow around the world, all be it slowly, no email in those days. But with the advent of the computer in 1837 (Charles Babbage) or 1942 (first digital computer) or the first PC, IBM 1953, the ability to store and transfer information to the masses was possible. Yet little changed in the way we were taught, until now.

Now we have both online and classroom learning. But what is best way to learn, what is the most efficient and effective way to transfer knowledge?

The lecture

If you think of a traditional lecture, the lecturer stands at the front and talks, the student makes notes and then goes away to read and perhaps actually learn something about the subject.

The online approach

A mass of content is available online, you can watch the very best lecturers from all around the world, stopping and starting the presentation to suite the speed in which you learn. Or perhaps you have a sophisticated e learning course where activities are carefully constructed to explain what you need to know. But this can be lonely and despite the claim that online is the same as being in a classroom, from my experience it is not, it is different.

The blend and the flip

But is the best method to have a blend of both online and Classroom / Lecture Theater. This is the idea behind the blended course. Attend the lecture and then watch some online instruction later in the day perhaps. However much thought needs to be given to how this works, directing students to random You Tube clips is far from ideal. Yet it feels like a move in the right direction.

And now the Flip – The flipped classroom (2004 ish) takes its name from the idea that content has historically been taught in the day and students expected to apply that content in their own time for homework.  But is making notes in class the best use of that face to face time? Should students not watch the lecture on the evening, benefiting from studying at their own pace and then attend class the next day, where they can ask questions? Can we flip the process?

In class the lecturer or teacher can then bring the subject to life using real world examples, answer student questions and deal with individuals on a one to one basis.

It’s not perfect, but it is better

Okay it is far from perfect and it has yet to be proven over time. But it does seem to make a lot of sense and for the first time we may be seeing the power of the PC complementing rather than competing with the classroom.

Oh and it has some powerful supporters Sal Khan & Bill Gates

Watch this for a great explanation why flipping the class is a good idea