Teaching to the test – another point of view

point-of-veiw-2A point of view is a programme on radio 4 that allows certain well-read, highly educated individuals, usually with large vocabularies to express an opinion. It lasts 10 minutes and is often thought provoking, concluding with a rhetorical question that has no answer.

This week Will Self the novelist and journalist gave his point of view on teaching to the test, as you might imagine it caught my attention. Self starts by telling a story about the life of a “good student,” and how it would unfold. He describes the way in which their concentration intensifies when the teacher states that what they are going to learn next is important and often examined. The story continues, as a result of their diligence and technique, the “good student” gets the necessary grades to go to University. They don’t however select the University on the basis of the course of study or on what they passionately wish to learn, no its based on the Universities credibility in league tables.

Upon successfully gaining a degree the student, now an employee gets a desk job that rewards a similar style of rubric mentality. As an employee, they are assessed against targets, performing well only on the ones that promise promotion and a pay rise. Eventually they retire and die.

Self concludes that this ordinary, dull, uninspiring life started back in the classroom all those years ago, when the teacher failed to educate and inspire, and simply taught to the test.

Over egging the pudding

There is a logic to this story, and it sounds ever the more inevitable as Self narrates it in his black and grey voice. But that’s all it is, a story. It avoids detail and colour, offering little regard as to the individual’s ability to reflect at some point in their life and ask searching and probing questions. It is as if somehow because the teacher highlighted the importance of one piece of knowledge it somehow stifled the student’s capacity to one day think for themselves.  Self is how they say, overegging the pudding, taking an interesting question as to the impact teaching to the test might have and serving up an omelette.

Teaching to the test is not bad

Brunel university asked a question as to what makes an unmissable lecture. In addition to many arguably more commendable answers, including the passion of the tutor and because they wanted to learn, the likelihood of the subject being taught having a high probability of being in the exam was key. Suggesting that a specific topic might be on the exam paper firstly, ensured a good attendance and secondly guaranteed the student listened intently.

Attention is important but even for the diligent student focus is vital. Learning everything is simply not possible, faced with 20 chapters, the student needs some clue as to where they need to direct their energy and time. Of course, the educationist will say that everything is important, but saying that will not make it so. Knowing that something is examinable at least gives a starting point and helps guide the student through the material quickly and efficiently. It’s also worth adding that It does not exclude the need to be inquisitive, in fact by making the student read a particular topic it may inspire them to find out more.

Exams and exam answers also provide examples of what is expected and the standard the student must reach if they are to be successful, no amount of narrative in the student handbook or curriculum guidance will do this as effectively.

The type of assessment matters

Of course, in Selfs world, teaching to the test removes the need to do anything more than learn about what will be in the exam. He suggests that students need to think outside the box rather than simply tick them. I have to admit I like that sentence.

But he does have a point, if the test is so narrow that it only assesses memory or a very small part of the syllabus then that is all the student will focus on. But that is just a bad test, this is of course where I am in danger of becoming idealistic and painting a picture that is not a true reflection as to what is happening. Not all tests are good, and undoubtedly some students will pass with limited thought and little more than good memory skills. Yet with changes in technology it becomes ever more possible to build tests and simulations that asses the student ability to perform in real world situations, and for that matter think outside the box.

Teaching to the test has become a term used to describe bad teaching and poor assessment and no one would agree that either of these are desirable. But it is not the process that’s problematic, it’s the application. Testing in its many forms is part of learning but it needs done well and thoughtfully.

In conclusion

Having now read the blog I would encourage you to listen to Will Self – click. It is of course not for me to say who presents the right point of view, you need to make up your own mind. For those however who were taught to the test no matter how long ago, you probably won’t understand even what I am asking because to the best of my knowledge this question has never been tested before……………?

 

The Price is right? – Value for Money Education

The Price is RightThe idea behind the price is right was that you had to guess the price of an everyday object. Not a complicated game I admit but by game show standards a successful one, it ran from 1984 to 2007. But how do you know if something is the right price? What does “right price” even mean?

 

Easier in a market where there are many similar products all providing a similar service or experience, not so easy when assessing the value is subjective, comparability difficult  and getting it wrong  expensive. This is exactly the situation you might find when trying to choose a course provider, a college or university. How do you know if you’re getting value for money, if one provider is more expensive than another is it extra profit, inefficiency or a measure of quality and so value?

What do you want?

The first question to ask is, what do you want from the course provider and how will you measure success? On the face of it the answer may seem obvious – it’s to pass the exam or get as high a grade as possible. But learning is about so much more than the exam result, isn’t it?

What about the skills you develop and the knowledge you acquire, what about the people you will meet and the inspiration, motivation and direction you will receive? These are difficult to measure and are often ignored yet in the long run are probably far more valuable than the passing of an exam.  Also would you be happy with knowing just enough to pass and then afterwards forgetting everything, is that what you pay for, is that value for money?

A high quality provider will teach content and explain concepts so that you not only retain knowledge but develop a deeper understanding of the subject matter.

So it’s all about passing?

You may want to pass but what method of study will give you the best chance of passing? How much contact time do you The price of everythingexpect? Are you sufficiently self motivated that you need little or no help, do you want the convenience of studying online or the discipline of having to go to lectures? You may be aware of the method you prefer but many students aren’t. The better provider will know enough about you as an individual, and if they don’t they will ask before making any recommendations as to what method of study is best.

Risk and cost of failure

How important to you is passing, or passing as soon as possible? It could be a false economy to go with the provider who suggests the course can be covered in 10 weeks at a cost of £1,000 when another suggests 15 weeks at £1,500. The longer course with the higher price could well be good value for money if you pass first time. Equally when the stakes are high you don’t want to make a mistake,  consider something like eye surgery, would you go to the cheapest provider where the risk of getting it wrong could be life changing.  Of course expensive does not always mean value for money. You will need to do some homework first. Below is some advice as to what to look for when choosing a provider before you part with any money.

What to look for in a course provider

Fundamentally it’s about trust and confidence in the provider. Education is not a commodity, it’s not homogenous, it’s personal and too important to get wrong

Here are a few things to look out for and questions to ask.

  • Experience and quality of the Teachers/Lecturers – having a stable and experienced lecture team is an indication of quality. Ask how the college ensures their staff are up to date, do they have a formal training scheme? What research credentials do they have?
  • Long term player – How long has the organisation been in existence, ask them what their long term strategy is for learning or at least what they think the future might hold.
  • Where do they rank in league tables – maybe they have industry awards or accreditation by external bodies.
  • Investment in the future and level of innovation – what do the premises look like, are they well maintained? What technologies have they introduced recently?
  • What are the range of different study options (length of course/F2F contact time etc) and levels of personalisation – for you to have the best course the provider should be able to offer some degree of personalised learning.
  • Ask if you can try before you buy – What have you got to lose they can only say no. Oh and ask them how easy it is to transfer to other courses and get your money back if you’re not happy.
  • And finally one of the most useful ways in making any decision is to ask friends/colleagues what they think or experiences have been, and don’t forget to check them out on the social media sites.

Conclusion

There is a lot more to this debate and the topic is certainly worthy of another blog. Value for money is a big question in education. I have not for example even mentioned the cost of education in the context of employability and student debt. Nor which subjects have the highest employability statistics etc.

The purpose of this blog was to highlight the complexity of choosing a provider and to give some advice as to what to look for.

In summary, clarify exactly what you want from your course provider before you start looking, ask some of the questions above and dependant on the answers you get make your decision. And if all goes to plan you will you end up with the right provider, at the right price and so great value for money.

PS Happy New Year everyone – I think as far as learning and exams are concerned 2015 is going to be as interesting and as uncertain as 2014. I am looking forward to it.

 

Teaching to the test – Interesting research but the fat lady is still in good voice

Fat lady still singingThis week researchers from the University of East Anglia released some very interesting findings that resulted from testing 594 bio-science students in their first week of term at five universities.

The students selected would be considered by many more than competent in their subject, almost all had a grade A*, A or B in biology at A-level. Yet when they were given 50 minutes to answer 38 multiple choice questions on cells, genetics, biochemistry and physiology from their A level core syllabus, they only got 40% correct. The period of time between the students sitting their A levels and the test was three months.

Lead researcher for the study, Dr Harriet Jones, said: “What our research shows is that students are arriving at university with fantastic A-level grades, but having forgotten much of what they actually learned for their exams.”  She went on to say that the trend to teach to the test, to ensure good results for schools’ reputations, was the problem.

The schools are to blame then

The facts of the research are clear, students who had successfully passed a test, were unable to pass a similar test three months later. The conclusion reached is that the students did not understand (see my blog on understanding) their subject well enough and passed their A levels probably using little more than memory. And who is to blame, the schools of course, for teaching to the test. Why the school do this is worthy of further debate, but government pressure and the impact of league tables will certainly be in the mix.

But do employers not accuse Universities of delivering up similar ill prepared students. The test is different but from the employers perspective the result is the same. A University student who professes to know something but when tested “in the real world” doesn’t.

Does this mean that Universities are also teaching to the test!

It’s about the test etc

The problem is not in teaching to the test; the problem is with the test, the pass mark and possibly the marking. If the test was more Testing but for what!aligned to what the student needs to know/do at a fundamental level, the pass mark sufficiently high and the marker having some degree of autonomy to form judgements, then the results would probably be different. It could of course be that the exams are easier – Exam chief: ‘you don’t have to teach a lot’ for our tests.

The big criticism of teaching to the test is, it results in a narrowness of understanding, little in the way of depth and does not push students to think in abstract and creative ways. But if the test, which incidentally does not have to be in the exam hall or on paper/PC was able to “test” for these qualities then teaching towards it would perhaps be more acceptable.

Bottom line

Teaching to the test is unlikely to change, in fact given the popularity of league tables  in education just now it may well increase, but with more effective testing the results might be better students, happy Universities and even happier employers.

 

 

50 Shades Darker – How much to test

How much to test

In August last year I wrote how Professional accountancy bodies believe that competency can be measured by a candidate scoring 50% and failure if scoring 49%. This all or nothing approach seems neither fair nor useful; hence the idea that grades of competence could be introduced e.g. 50% pass, 65% commendation, 75% distinction etc.

But the mark a student gets in an exam is only part of the story when it comes to measuring competence. Can a student be considered competent if the exam they pass only includes questions from say 75% of the syllabus? Yes the whole syllabus might be covered in an 18 month period but in any one exam 25% is not tested. Equally the 75% is often considered core and so examined every sitting, this means that a student need only focus on the 75%. Admittedly if the pass mark is 50% they need to score 50% out of 75% (67%) but with practice this is possible. One final observation, it is unlikely the student will score 0% on the non-core part of the syllabus. They may get say 5% or even 10% out of 25%. Not a great score but the 67% pass mark now becomes 58%. This logic sits at the heart of the exam driven approach.

Objective testing might be the answer?

Objective tests (OT) – test that are relatively short and can be unambiguously marked, are considered by some to be a weaker form of assessment, they are part of the dumbing down of examinations. The beauty of an OT question is that the marking is completely accurate, no marking bias at all. This is often ignored in traditional exams and not seen as a problem largely because it’s not that visible. But the OT does not solve the “how much to test” problem, in fact it makes it worse. If you are asking for less then you are examining less. So if the only benefit is the avoidance of marker bias why are more examining bodies using OT style exams, is it just about saving money…….?

An example

Imagine that you have 4,500 OT style questions that cover every aspect of the syllabus, let’s also assume that the student only has to answer 50. The 50 questions are randomly picked from the 4,500 in the question bank. Is it fair that a student is considered competent if they are only being tested on 1% of what they need to know?

I think the answer is Yes, because in order to be sufficiently prepared to answer 50 questions from a bank of 4,500 when there are no core topics you have to have be capable of getting all 50 correct (assuming a 100% pass mark) and because you don’t know which 50 are coming up this effectively means you have to be able to get all 4,500 correct. The examining body can of course control  the effectiveness and level of difficulty by changing the pass mark.

All OTs would be OTT

This is not an argument to suggest that all examinations should be assessed using OT type questions, they should not. For example they are not Less-is-Morevery effective at measuring a student’s ability to communicate or evaluate complex and ambiguous situations, but they should be considered part of the tool kit that examining bodies have in assessing competence.

So on the face of it OT’s may look like a soft option, anyone can tick a box but they are certainly not easy to get right. Maybe less is actually more…..

50 shades of Grade…..Measuring students worth

I don’t often write specifically about the market that I am most closely involved, that of accountancy training and education. But there does seem to be an anomaly in the way the accountancy world measure success that isn’t the case in many other professions and examinations.

To become a qualified accountant in the UK (ACCA, ICAEW,CIMA etc,  yes there are more) you have to pass a number of demanding examinations and submit evidence of practical experience. The exams are taken over 3 years and everything rests on a number of exams of 100 marks each, of which you need to get 50%. The pass rate for accountancy students varies from paper to paper but is around 60% In training and education terms these exams are often referred to as High stake exams. For some failure is not simply a setback in terms of time, it could result in a lost job.

50% good – 49% bad

The purpose of such exams (including the practical experience) is of course to ensure that those that pass are competent. But how competent….?  It seems that you are either competent or not, if you get 49% you are not competent, if you get 50% then you are. This makes the margin for error very high, many a career has taken a dramatic turn based on 1 mark. Now of course in any exam there has to be a point where someone succeeds and another fails but does competence not have many shades to it, is it not a continuous process rather than a discreet one.

Shades of Grade

You may be one of thousands of students (>300,000 actually) who have just had your A level results. But of course you don’t just pass or fail, in fact this has not been the case since 1963, even before then there was an indication of the mark. You are given a grade from A through to E and all are considered a pass, 98.1% passed their A levels this year.

And the grades refer to the marks e.g. E is 40%, a D is 50%, a C is 60%, a B is 70% and an A is 80%. A similar grading exists for Degrees, 2.1, 2.2 etc and for Law exams, Pass, Commendation, Distinction.

No pressure for excellence

So why is this not the case in accountancy? To be honest I am not sure. But by introducing grades above a pass you not only provide employers with a better indication of competency, you encourage colleges to teach to a higher and deeper level, motivated by the students who want to learn more. And that for me is the most important point; you create an incentive for students to try harder. At the moment if you are scoring 55% then why put in more work, a pass is a pass, but a distinction well that will make you stand out from the crowd.

Of course the market will adjust its perception of what a pass means, for example employers may only want students with a distinction rather than a simple pass, but they have always looked and will continue to look for differentiation, at least this is a meaningful one.

What it does not do is fix the 49% you fail problem, and careers will still hinge on 1 mark, but that unfortunately is the way with exams and the reason exam technique is so important.

So to all the accountancy examining bodies why not introduce grades – grey is good and isn’t it an accountants favourite colour?

Exams in the headlines – but for the wrong reasons again!

Well it has certainly been an interesting time in the exam world! Here are just a few of the headlines.

Behind all of these headlines are personal stories, for example students who can’t get into sixth form colleges because they didn’t get the necessary grades. The argument being, that if the grade boundaries had not changed (by 10%) resulting in them getting a D and not a C then they would have got into their colleges of choice.

Stacey Cole chief executive of Ofqual said “the grades are right “

Although the impact on individuals is considerable, statistically the change was small. The proportion of test papers awarded at least an A fell by 0.8 percentage points to 22.4 per cent (this was 8.6 per cent in 1988!) the first annual drop since GCSE exams were first sat in 1988. A* grades also fell but only by 0.5 percentage points to 7.3 per cent.

However, when something goes up, it must come down so perhaps it was inevitable that the ever increasing improvement in student grades had to reverse or at least plateau.

What has changed/gone wrong? 

  • Teachers are encouraged to prepare students for the exam, because parents, employers and educational institutions measure success at least partly (exclusively?) by the results.
  •  Students are better at exam skills than ever before. After all, this blog is about how to pass exams and the skills needed to help.
  • Universities and employers seem equally unimpressed with the quality of students and candidates they get, complaining they don’t have basic levels of numeracy, literacy and common sense!

Answers to some of the questions

Have students been getting better each year?

I think the answer is yes, the results prove they have. But maybe they have been getting better at passingexams. And not improving on some of the more difficult to measure skills like, attitude, common sense, being thoughtful. Exams don’t give you time to be thoughtful! This might explain why employers are so unhappy.

But they may just be getting better…..
On the 6th of May 1954 Roger Banister ran the 4 minute mile, it is now the standard of all male middle distance runners. Does this mean the mile is now shorter…….or maybe runners have improved?

Why did someone not say something?

The pressure to succeed, measured by exam results has been so great on teachers, examining bodies and students that no one was willing nor was it in their best interest to say, “this just doesn’t make sense.”

Why do we have exams, to test knowledge/competence or to separate the best from the rest?

They should probably be to asses’ knowledge but are mostly used to try and pick the best people.

Have exams been made easier, the dumbing down argument?

This is tricky, and although you can compare exam papers it’s a bit like comparing Wayne Rooney with George Best. You can debate the pros and cons but I am not sure it is conclusive; things were just different in the past.

But it’s not fair

What does seem clear in this whole debacle is that raising the grade required
half way through the year is not an example of exam rigour, it’s an example of being unfair and that is the one thing exams should never be.

Also see my blog what do exams prove

Back to more exam tips next month….

What’s the point of exams – what do they prove?

With many students in the middle of exams right now, working long hours, making huge personal sacrifices and putting themselves under considerable pressure, perhaps we should stop, take a moment to reflect and ask ……….what’s the point of exams?

Why are you doing this, what will it prove when you do pass, what will passing give you that you don’t have now?

 

 

It’s not about knowledge

If you pass an exam, you have proved that you knew the answers to questions set by the examiner at a particular point in time. To be precise you have only really proved you knew enough answers to get a pass mark, in some instances this might be less than half! But you have not proved that you understand everything about the subject or that you could work unsupervised in practice, knowing what to do is not quite the same as doing it.

This is not to say that examinations are easy, they are not or to underestimate their importance, it is just to be a little clearer on what exam success means.

Higher level skills

By passing an exam you are demonstrating many other skills, for example;

Motivation – You have proved that when you set your mind to something you can achieve it.

Concentration – For some people, concentration comes easily for others it might involve removing all distractions by locking themselves in a room. Whatever method you used, you have learned how to cut out distractions and focus on the task in hand.

Prioritisation and Time management – Undoubtedly you have had too much to learn and too little time to learn it. But if you pass the exam you have proved that you got the balance between an endless, or at least what appeared endless set of demands and the overall objective just right.

But most of all exams give you….

A great sense of achievement – You set yourself a target and achieved it.  It is a statement to others that you worked hard and have succeeded. It will remain a tangible and permanent reminder of success that can never be taken away.

Self confidence – It will build self esteem and help you develop a type of confidence that only comes from being successful in a chosen field. Others will congratulate you and as a result, treat you differently.

Choices – it will open doors to opportunities that simply would not have been possible without the piece of paper that says “Congratulations, you have passed”. Exam success will give you choices, it will change how others look at you but perhaps more importantly it will change how you feel about yourself…

So if you have been working hard keep at it, you may not be proving you are the greatest mathematician in the world, but if all goes to plan the end result will make up for all the pain you are going through right now, honest!

But just in case…..everything doesn’t work out check out these Famous A-level flunkers