Category Archives: The future of learning

Blooms 1984 – Getting an A instead of a C

Posted on by

When people see the year 1984 most think of George Orwell’s book about a dystopian future, but a few other things happened that year. Dynasty and Dallas were the most popular TV programs and one of my favorite movies, Amadeus won best picture at the Oscars. You can be excused for missing the publication of what has become known as the two Sigma problem by Benjamin Bloom, of Blooms taxonomy fame. He provided the answer to a question that both teachers and students have been asking for some time – how can you significantly improve student performance?  

One of the reasons this is still being talked about nearly 40 years later is because Bloom demonstrated that most students have the potential to achieve mastery of a given topic. The implication is that it’s the teaching at fault rather than the students inherent lack of ability. It’s worth adding that this might equally apply to the method of learning, it’s not you but the way you’re studying.

The two-sigma problem
Two of Bloom’s doctoral students (J. Anania and A.J. Burke) compared how people learned in three different situations:

  1. A conventional lecture with 30 students and one teacher. The students listened to the lectures and were periodically tested on the material.
  2. Mastery learning – this was the conventional lecture with the same testing however students were given formative style feedback and guidance, effectively correcting misunderstandings before re-testing to find out the extent of the mastery.
  3. Tutoring – this was the same as for mastery learning but with one teacher per student.

The results were significant and showed that mastery learning increased student performance by approximately one standard deviation/sigma, the equivalent of an increase in grading from a B to an A. However, if this was combined with one-to-one teaching, the performance improved by two standard deviations, the equivalent of moving from a C to an A. Interestingly the need to correct students work was relatively small.

Bloom then set up the challenge that became known as the two-sigma problem.

“Can researchers and teachers devise teaching/learning conditions that will enable the majority of students under group instruction to attain levels of achievement that can at present be reached only under good tutoring conditions?”

In other words, how can you do this in the “real world” at scale where it’s not possible to provide this type of formative feedback and one to one tuition because it would be too expensive.

Mastery learning – To answer this question you probably need to understand a little more about mastery learning. Firstly, content has to be broken down into small chunks, each with a specific learning outcome. The process is very similar to direct instruction that I have written about before. The next stage is important, learners have to demonstrate mastery of each chunk of content, normally by passing a test scoring around 80% before moving onto new material. If not, the student is given extra support, perhaps in the form of additional teaching or homework. Learners then continue the cycle of studying and testing until the mastery criteria are met.

Why does it work?
Bloom was of the opinion that the results were so strong because of the corrective feedback which was targeted at the very area the student didn’t understand. The one to one also helped because the teacher had time to explain in a different way and encourage the student to participate in their own learning which in turn helped with motivation. As you might imagine mastery is particularly effective in situations where one subject builds on another, for example, introduction to economics is followed by economics in business.

Of course, there are always problems, students may have mastered something to the desired level but forget what they have learned due to lack of use. It’s easy to set a test but relatively difficult to assess mastery, for example do you have sufficient coverage at the right level, is 80% the right cut score? And finally, how long should you allow someone to study in order to reach the mastery level and what happens in practice when time runs out and they don’t?

The Artificial Intelligence (AI) solution
When Bloom set the challenge, he was right, it was far too expensive to offer personalised tuition, however it is almost as if AI was invented to solve the problem. AI can offer an adaptive pathway tracking the student’s progression and harnessing what it gleans to serve up a learning experience designed specifically for the individual. Add to this instructionally designed online content that can be watched by the student at their own pace until mastery is achieved and you are getting close to what Bloom envisaged. However, although much of this is technically possible, questions remain. For example, was the improvement in performance the result of the ‘personal relationship’ between the teacher and student and the advise given or the clarity in explaining the topic. Can this really be replicated by a machine?

In the meantime, how does this help?
What Bloom identified was that in most situations it’s not the learner who is at fault but the method of learning or instruction. Be careful however, this cannot be used as an excuse for lack of effort, “its not my fault, it’s because the teacher isn’t doing it right”.

How to use Blooms principles.

  • Change the instruction/content – if you are finding a particular topic difficult to understand, ask questions such as, do I need to look at this differently, maybe watching a video or studying from another book. Providing yourself with an alternative way of exploring the problem.
  • Mastery of questions – at the end of most text books there are a number of different questions, don’t ignore them, test yourself and even if you get them wrong spend some time understanding why before moving on. You might also use the 80% rule, the point being you don’t need to get everything right

In conclusion – It’s interesting that in 1985 Bloom came up with a solution to a problem we are still struggling to implement. What we can say is that personalisation is now high on the agenda for many organisations because they recognise that one size does not fit all. Although AI provides a glimmer of hope, for now at least Blooms 2 Sigma problem remains unsolved.

Listen to Sal Khan on TED – Let’s teach for mastery, not test scores

Artificial Intelligence in education (AIEd)

Posted on by

robot learning or solving problems

The original Blade Runner was released in 1982. It depicts a future in which synthetic humans known as replicants are bioengineered by a powerful Corporation to work on off-world colonies. The final scene stands out because of the “tears in rain” speech given by Roy, the dying replicant.

I’ve seen things you people wouldn’t believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die.

This was the moment in which the artificial human had begun to think for himself. But what makes this so relevant is that the film is predicting what life will be like in 2019. And with 2018 only a few days away, 2019 is no longer science fiction, and neither is Artificial Intelligence (AI).

Artificial Intelligence and machine learning

There is no one single agreed upon definition for AI, “machine learning” on the other hand is a field of computer science that enables computers to learn without being explicitly programmed. The way it does this is by analysing large amounts of data in order to make accurate predictions, for example regression analysis does something very similar when using data to produce a line of best fit.

The problem with the term artificial intelligence is the word intelligence, defining this is key. If intelligence is, the ability to learn, understand, and make judgments or have opinions based on reason, then you can see how difficult deciding if a computer has intelligence might be. So, for the time being think of it like this:

AI is the intelligence; machine learning is the enabler making the machine smarter i.e. it helps the computer behave as if it is making intelligent decisions.

AI in education

As with many industries AI is already having an impact in education but given the right amount of investment it could do much more, for example

Teaching – Freeing teachers from routine and time-consuming tasks like marking and basic content delivery. This will give them time to develop greater class engagement and address behavioural issues and higher-level skill development. These being far more valued by employers, as industries themselves become less reliant on knowledge but dependant on those who can apply it to solve real word problems. In some ways AI could be thought of as a technological teaching assistant. In addition the quality and quantity of feedback the teacher will have available to them will not only be greatly improved with AI but be far more detailed and personalised.

Learning – Personalised learning can become a reality by using AI to deliver a truly adaptive experience. AI will be able to present the student with a personalised pathway based on data gathered from their past activities and those of other students. It can scaffold the learning, allowing the students to make mistakes sufficient that they will gain a better understanding.  AI is also an incredibly patient teacher, helping the student learn from constant repetition, trial and error.

Assessment and feedback – The feedback can also become rich, personalised and most importantly timely. Offering commentary as to what the individual student should do to improve rather than the bland comments often left on scripts e.g. “see model answer” and “must try harder.” Although some teachers will almost certainly mark “better” than an AI driven system would be capable of, the consistency of marking for ALL students would be considerably improved.

Chatbots are a relatively new development that use AI.  In the Autumn of 2015 Professor Ashok Goel built an AI teaching assistant called Jill Watson using IBM’s Watson platform. Jill was developed specifically to handle the high number of forum posts, over 10,000 by students enrolled on an online course. The students were unable to tell the difference between Jill and a “real” teacher. Watch and listen to Professor Goel talk about how Jill Watson was built.

Pearson has produced an excellent report on AIEd – click to download.

Back on earth

AI still has some way to go, and as with many technologies although there is much talk, getting it into the mainstream takes time and most importantly money. Although investors will happily finance driverless cars, they are less likely to do the same to improve education.

The good news is that Los Angeles is still more like La La Land than the dystopian vision created by Ridely Scott, and although we have embraced many new technologies, we have avoided many of the pitfalls predicated by the sci-fi writers of the past, so far at least.

But we have to be careful watch this, it’s a robot developed by AI specialist David Hanson named “Sophia” and has made history by becoming the first ever robot to be granted a full Saudi Arabian citizenship, honestly…..

 

What’s the use of lectures?

Posted on by

Robot lecturerThe title of this month’s blog is not mine but taken from what many would consider a classic book about what can realistically be achieved by someone stood at the front of a classroom or lecture theatre, simply talking. Written some 25 years ago but updated recently Donald A. Bligh’s book takes 346 pages to answer the question, what’s the use of lectures?

What makes this book interesting is the amount of research it brings to bear on a topic some consider an art form and so not easily measured or assessed.

With many in Higher education questioning what they get for their £9,250 per annum, and contact time being one way of measuring value, it’s as important a question as ever.

For clarity, we should define what we mean by lecturing, as ever Wikipedia can help –  A lecture (from the French meaning ‘reading’) is an oral presentation intended to deliver information or teach people about a particular subject.

What should happen in a lecture?

If you’re a student attending a lecture you would hope to learn something, however as many of my past blogs have discussed, learning is a complicated process and so we may need to break this question down a little further by asking, what should a lecture actually achieve?

A lecture should….

  • Transmit information
  • Promote thought,
  • Maybe change opinion or attitude
  • Inspire and motivate
  • Help you be able to do something i.e. develop a behavioural skill

Well here is the bad news, according to Mr Bligh, a lecture is only really good for one of the above, to transmit information. And it’s not even better than many other methods e.g. reading, it’s simply as effective, but no more.

Promoting thought, changing opinions

Lectures are relatively passive whereas a discussion requires that people listen, translate what is said into their own words, check if it makes sense with what is already understood, construct a sentence in response etc. In effect, a discussion is far more effective than a lecture in developing thought.

In addition, putting the student in a situation where they have to think is important, for example by giving them a problem or asking a question as is the case when you have to answer a past exam question for example. A discussion can also help change opinions, especially where you can hear other people’s views, often different to your own. It has a longer-term impact when the group comes to a consensus.

Inspiration and motivation

Bligh also argues that on the whole lectures are not an effective means of inspiring or motivating. He suggests that it should certainly be the objective of the lecturer to try, it’s just they rarely succeed. I find myself slightly disagreeing, lecturers can be inspirational, and yet maybe this is just my personal bias from having watched Sir Ken deliver his “do schools kill creativity“  or the last lecture delivered by Randy Paush.

But perhaps, these are just the exceptions that prove the rule.

Developing skills

And finally, if you want to help people become good at a particular behaviour, you don’t tell them how to do it, you get them to practice, over and over again, with good feedback.

The end of the lecture?

I don’t think this is the end of the lecture, these criticisms have been around for many years. But I can’t help thinking that with new technologies and online learning, lectures are going to have to get a whole lot better in the future.

And what will Universities point to as value for money then?

 

 

The 5 top EdTech trends – summer of 2017

Posted on by

Glastonbury a marginally more interesting gathering….but only just.

We are in the season when many learning and technology leaders gather to discuss what’s new and what’s trending in the world of education. And at two recent conferences, Learning Technologies and EdTechXEurope there was plenty to see. Generally, the role of technology in learning seems to have found its place with many acknowledging it should support learning not drive it. However it’s still very easy to look at the latest shiny new offerings and think, this is great how can I use it, rather than, what learning problem does it solve.

Here are a few of the most notable developments.

1. Video is getting even better – fuelled by the YouTube generation of learners, those who would rather watch a video than read a book as a means to consume knowledge, we have some new developments.

Firstly, using video to deliver micro learning.  Not just small chunks of video but untethered, JIT, 3 minute courses that offer the learner digestible easy to remember information. Think of micro learning as a series of very short courses that could be linked to each other or not, and can even include assessment.

Secondly, interactive video. TV is no longer the all commanding medium it once was, it like other technologies has had to evolve. In recent years the shift has been towards better engagement, offering spin off programmes where there is a live audience, web sites that showcase the backstory to the characters and programmes that require the audience to vote and so influence events. Now we have interactive video, where the individual can choose what they would do and so change the future. Check out this amazing example, used by Deloitte to attract new talent.

2. Gamification is becoming better understood. For the uninitiated gamification is the use of game based principles to improve motivation, concentration and more effective learning. Gamification uses Points (P) as a measure of reward, Badges (B) as a visual record of success, and leader boards (L) to create competition.

We now believe Dopamine, the pleasure induced neurotransmitter (chemical) is not created as a result of a reward e.g. by being given a badge, it is the challenge and subsequent achievement that releases the dopamine which in turn leads to pleasure. This might seem obvious, with hindsight, no one gets pleasure from being top of a leader board, if they did nothing to get there.  In addition, dopamine is released when you have a new experience, so think about changing pathways, setting different questions and tasks, it’s certainly not very motivational to go over the same content again.

3. Information overload is leading to a need for Knowledge Curation – we are living in an age where  information is abundant. You can learn anything from the internet. But there lies the problem, we have too much information, we suffer from information overload. Curation is the collecting and sorting of meaningful content around a theme, and it is now in some instances being thought of as more valuable than the content itself.

Arguably curation is not so much about what you curate and share but what you don’t share. In addition to the organisation of content the curators need to have an expertise in the subject and an understanding of their audience and what they want.

Steven Rosenbaum in his book Curation Nation, offers up a good summary. “Curation replaces noise with clarity. And it’s the clarity of your choosing; it’s the things that people you trust help you find.”

4. The market is becoming more accepting of user generated content (UGC) – organisations are beginning to see the benefits of UGC for a whole host of reasons. It’s a very fast way of generating content, there is a lot of expertise that can be uncovered by allowing individuals to share what they know, it’s often user friendly, and importantly its cheap. It is of course not perfect, and there are concerns about quality, but by allowing the users to rate the content, the quality might just look after itself.

5. Virtual reality (VR), Augmented Reality (AR) and Artificial intelligence (AI) – not that these are all related, but just a simple way of me summarising three areas to keep an eye on in the not too distant future. All of these technologies are becoming cheaper, largely because of the investment made and experience being gained in the gaming industry.

By way of a footnote Google have released an open source software called Tensorflow which can help with machine learning, something that they believe will help drive new initiatives in AI.

Sensemaking, humility and the humanities

Posted on by

human-being-girl-picture

For a variety of reasons, I have been engaged this month in thinking not so much about examinations but what subjects should be examined.

Whilst the news has been dominated by terrorism, Trump and Brexit, we may be facing a far bigger problem, of which these news stories are a good example, how can we be sure of making the right decisions in a world of mass information, complexity and change.

People voted Brexit for a whole variety of reasons, many “facts” were presented in simple terms, we will save £350m a week and this money will go into the NHS, immigration will be reduced as we gain control over our borders. Yet these facts are far too simplistic, any level of analysis, critical thinking and challenge would have revealed the difficulty of delivering them, and in many instances they won’t be delivered. If this is the case, did people vote to leave, or stay not on the facts as presented but using other criteria, maybe they were just naive and placed far too much trust in Politicians or perhaps they had never been taught about sensemaking, humility or studied the humanities.

Sensemaking

An interesting article caught my eye earlier in the month, “Silicon Valley needs to get schooled”. it was by Christian Madsbjeg, author of the book Sensemaking and senior partner in ReD, a strategy consulting company based on the human sciences. In the article Madsbjeg argues that the reason for a lack of new and exciting products from Silicon Valley is not because of a shortage of ideas but a complete failure to understand people.

In the book Sensemaking he expands on the problem. In order to cope with complexity, we look to science, logic and the algorithm (a rules based process) for a solution. On the face of it crunching big data so that it spews out the correct answer seems perfect, but, and this is a quote from the book, Madsbjeg makes a very important point, he says “we stop seeing numbers and models as a representation of the world and we start to see them as the truth – the only truth”.  We are in fact looking at the numbers without the context of the world from which they came or a sufficiently deep understanding of the behaviours we are measuring.

We rely on science and the scientific method for so much of what we do but where people are involved we need a different approach. To put it another way “When human beings enter the equation, things go non-linear” Neil deGrasse.

Sensemaking is “how we make sense of the world so we can perform better in it”. It recognises that situations are complex and information ambiguous. It requires people to make a continuous effort to understand the connectivity that exists between people, places, and events in order to anticipate their trajectories and act accordingly.

Humility

trumpwillwin-notextIntellectual humility as defined by the authors of a recent paper entitled, Cognitive and Interpersonal Features of Intellectual Humility is the opposite of intellectual arrogance or conceit. It is in effect, recognising that you could be wrong. One of the findings from the research was that people who displayed intellectual humility were better than the control group at evaluating the quality of evidence they had been presented with. A very useful skill indeed, given the world of false news in which we currently find ourselves.

Humanities

And what job will you get after studying History for three years……

The humanities (English, History, Philosophy etc) have been given a bad press in recent times. Overshadowed by the drive to develop coding skills and with the constant chanting of STEM (Science, Technology, Engineering and Mathematics) in the background, it’s not surprising that less people are studying them. They were at an all-time low in 2014 at 6.1% of all bachelor’s degrees, a long way of the 1967 record of 17.2%.

But it is generally recognised that the humanities can teach us a lot. In another reference from Christian Madsbjeg’s book, Sensemaking, he suggests the humanities can teach us, one that other worlds exist, two that they are different and three, we learn how to imagine other worlds that in turn helps us better understand our own.

As with sensemaking and humility, are these not the types of skills we need to learn?

Examinations – what to examine?

What subjects should be examined depends to a large extent on what job you would like to do. But with the claim that 60% of 11 year olds will leave school to do jobs which have not yet been invented it’s hard to know the answer. What we do know is that the world is unlikely to slow down, change not happen, data become less available and complexity give way to simplicity. As a result, we need to teach people and so examine the skills that will help them better navigate this world. Maybe when those primary school children go onto higher education they will be studying sensemaking, humility and the humanities.

Even though the ink is barely dry on the letter sent by Theresa May bringing about our formal negotiations to exit Europe, the interesting thing is we will never know if this was a good or bad decision. Because post Brexit people will behave differently, some will work hard to make the impossible possible whilst others will continue to frustrate the process, and none of that could have been foreseen at the time.

So, let’s hope the basis for the original decision to leave was not because of the headline – We will save £350m a week and this money will go into the NHS!

So you want to be an astronaut – assessment for astronauts

Posted on by

 

Astronaut-Tim-Peake-Dials-Wrong-Phone-Number-from-Space

Your there by yourself. There’s no doctor, there’s no computer engineer – so you have to learn all of these skills. Tim Peake

What is the purpose of assessment?

In order to gain a better understanding of the assessment process for astronauts, let’s ask a more basic question first, what is an assessment and what does it prove?

At one level it is simply a measurement of performance benchmarked against a given outcome or standard. The results can, help a teacher identify progress so they can adapt the next lesson, or give assurance an individual is capable of performing a particular task. This is probably the most important type of assessment for astronauts.   Modern high stakes “examinations however play another role; they offer the student a transferable badge of honour that can help open doors to better career prospects and increased salary. If you doubt the importance of this last point, click this link to read about the level of corruption in India and the lengths people will go in order to obtain a certificate saying they have passed an exam.

Employability gap – so what’s the problem?

However examinations on their own do not provide a guarantee that the individual who passes will be able to do the job.  And it is here where the disconnect becomes clear. Often the method of assessment is not based on what the student will be doing in the work place, and even if it is, it is not set in the same context. Many believe this gap has become ever wider as more and more students come out of school/university lacking the skills required by the employer.

But should the exam/assessment be changed to narrow the gap or is it the role of the employer to provide the necessary “on the job” education in the work place? However the problem with the employer being given the responsibility, it implies that what you study doesn’t matter, only that you do. This just seems wasteful, wasteful of time, money and effort. Assessment must get closer to measuring the skills required for one simple reason, what gets measured gets achieved, a cliché for sure but a true one.

Assessment for astronauts

Costs – The costs involved in training astronauts is a bit unclear, NASA will pay Russia $70m for a seat on Russian space craft in 2017 that includes training, figures of around $25m are also cited, Tim Peaks training was quoted as £16m so very similar.

Basic requirements – Mission Specialist (non pilots – most are pilots with a military background) include the following:

  1. Bachelor’s degree from an accredited institution in engineering, biological science, physical science, or mathematics. The degree must be followed by at least three years of related, progressively responsible, professional experience.
  2. Ability to pass a NASA space physical, which is similar to a military or civilian flight physical.
  3. Height between 58.5 and 76 inches.

Training – Tim Peak’s education started in 2008 when he was selected by ESA (European space agency) from 8,413 applicants. Tests at this stage included, intelligence, spatial awareness and concentration. Then for the 10 selected another 18 months of intense training followed, this time on topics as wide ranging as space law, rocket propulsion, spaceflight engineering and the hardest for Tim, learning to speak Russian. He was also subject to one week’s caving in Sardinia, Italy, with five other astronauts to simulate what it would be like in space, this was to build teamwork, problem solving and cope with poor hygiene facilities. Then it was one year of advanced training, including working underwater to simulate the lack of gravity. In 2013 it was time to go into space for the first time, this was the last test before being selected and then finally in 2015 Tim was allowed to do the job for real. In all it had taken 6/7 years.

What have we learned?

Clearly you need some underpinning academic skills i.e. a degree but a relevant one. The training is provided by the state, the argument being that it is for the greater good of society so the costs should be met by the tax payer. An interesting point for those that believe this should be the case for all education. But most importantly the assessments were all built around making sure that when Tim got into the space he could do the job. Very few of the tests required a piece of paper and a desk, and many were simulations of what he would meet in the real world.

Given the advances in technology the time is now right to introduce more simulations into the exam room, not so we can all become astronauts but to help prepare the next generation for the work place.

And just imagine your badge of honour when going for that next job – “what exams have you passed” “oh a few, but did I mention I was an astronaut…..”

 

A Nostradamus moment – Predicting learning in the future

Posted on by

Top-5-Nostradamus-Predictions-That-Came-TrueLast year along with a colleague I looked into some of the key trends that were shaping the world of Professional Education. The result was the production of a Learning Strategy completed in December 2014. The document highlighted some of what we thought might impact our organisation in the future. Taking the technologies, attitudes and resources of today and guessing how they might change is certainly brave, possibly foolish, but reading what others think is always interesting.

Tomorrow I will be visiting the Digital Education show in Earls Court; guest speakers include Sugata Mitra, Richard Gerver and Sir Ken Robinson. The topics up for debate are current, wide reaching and of course equally prophetic.  Add to this the publication of the 2015 Horizon higher education report and you get an irresistible mix of views on the future, some of which I have highlighted below.

The measurement of learning will increase

Good teachers have always tracked student performance. It may have been in their head, summarised at the end of term in the form of a report but measuring student performance is certainly not new. The difference is now the results are more public, displayed in league tables showing winners and losers. In addition we have data not just on one student but thousands and once you have data, big data in fact, you analyse it, learning analytics is born. This allows you to make recommendations for improvement and predictions based on the observed trends and patterns. Given that new technologies make the gathering of data relatively easy, measuring student performance and the methods by which they learn will only increase.

All classroom courses will become a blend

The genie is out of the bottle, learning in a classroom complimented by the use of instructionally sound online resources offers so many benefits. It enables a more personalised learning experience, makes effective use of student time outside the classroom and is often mobile resulting in greater convenience.  It is hard to see why you would ever have just classroom only courses again. Yet not all courses are a blend or to be precise although they have online resources they are little more than a classroom course with some online PDF’s or links that are not used due to poor quality, relevance or support from the teacher, so we still have some way to go.

Informal learning will emerge from the shadows

So wrong!

So wrong!

Informal learning or as it sometimes called student/curiosity led learning has always existed but is now more easily recognised. Teachers and educators are also beginning to invest time into using it more effectively.  Once again technology is playing an important role by making knowledge more accessible and facilitating greater collaboration via online forums and social networks.

Video is an incredibly successful example of social learning, it’s hard to imagine but YouTube didn’t exist before 2005, that’s only 10 years ago. It now has 1 billion users, 300 hours’ of video are uploaded every minute and is available in 61 languages. A very practical example of learning with video can be found by clicking the banana – trust me you won’t be disappointed.

And that’s just three

I could equally have mentioned:

  • More money will be spent on personalisation and adaptive technologies
  • Greater acceptance of BYOD – your own devise that you take from home to class to work
  • Wearable learning technologies – think how wearable sports devices have expanded recently
  • Internet of things (IoT) – a network of connected objects that link the physical world with the world of information through the web could provide a wealth of new ways of learning

How did I do?

A similar blog from 2011