Reality bites

Evidence-based practice is the darling of Learning & Development geeks.

And with good reason. Amongst all the myths and barrow pushing, not to mention the myriad ways to approach any given problem, empiricism injects a sorely needed dose of confidence into what we do.

Friends of my blog will already know that my undergraduate degree was in science, and although I graduated a thousand years ago, it forever equipped me with a lens through which I see the world. Suffice to say my search for statistical significance is serious.

But it is through this same lens that I also see the limitations of research. Hence I urge caution when using it to inform our decisions.

For instance, I recently called out the “captive audience” that characterises many of the experiments undertaken in the educational domain. Subjects who are compelled to participate in these activities may not behave in the same manner as when they are not; which could, for example, complicate your informal learning strategy.

Hence studies undertaken in the K12 or higher education sectors may not translate so well in the corporate sector, where the dynamics of the environment are different. But even when a study is set among more authentic or relevant conditions, the results are unlikely to be universally representative. By design it locks down the variables, making it challenging to compare apples to apples.

In short, all organisations are different. They have different budgets, systems, policies, processes, and most consequentially, cultures. So if a study or even a meta-analysis demonstrates the efficacy of the flipped classroom approach to workplace training, it mightn’t replicate at your company because no one does the pre-work.

Essentially it’s a question of probability. Subscribing to the research theoretically shifts the odds of success in your favour, but it’s not a sure thing. You never know where along the normal distribution you’re gonna be.

My argument thus far has assumed that quality research exists. Yet despite the plethora of journals and books and reports and articles, we’ve all struggled to find coverage of that one specific topic. Perhaps it’s behind a paywall, or the experimental design is flawed, or the study simply hasn’t been done.

So while we may agree in principle that evidence-based practice is imperative, the reality of business dictates that in fact much of that evidence is lacking. In its absence, we have no choice but to rely on logic, experience, and sometimes gut instinct; informed by observation, conversation, and innovation.

In your capacity as an L&D professional, you need to run your own experiments within the construct of your own organisation. By all means take your cues from the available research, but do so with a critical mindset, and fill in the gaps with action.

Find out what works in your world.

Time pilot

There was an aspect I omitted from my previous post, Space invaders, in which I covered the spacing effect as a means of maximising long-term retention.

Zsolt Olah picked it: interleaving and – ironically! – I judged it a bridge too far to include in the same post and so decided to split it out across a second one.

You’ll see why…

Interleaving

Before defining interleaving, I feel it’s helpful to review the opposite: blocked presentation or blocking. This approach is equivalent to massed presentation whereby the content is consumed successively:

An alternative approach is to split each block into leaves and vary their presentation. This approach is known as interleaved presentation or varied presentation or mixed presentation:

The illustration above maintains the topical sequence, in contrast to random interleaved presentation whereby the sequence is shuffled:

Spaced interleaving

Just as the spacing effect can be employed to produce spaced block presentation:

…so too can it be employed to produce spaced interleaved presentation (grouped):

…or spaced random interleaved presentation (grouped):

I use the qualifier “grouped” because, while that’s the traditional way of illustrating interleaving – I suspect due to the formal paradigm of education by which you fill up the periods assigned to you – I see no reason why each leaf couldn’t be separated over time; hence spaced interleaved presentation (separated):

…and spaced random interleaved presentation (separated):

As you can see, it gets complicated real quick, and that’s before employing reinforcement or the testing effect!

Dynamic interleaving

When I look at the various permutations of spacing and interleaving, I wonder if an optimal timing or sequencing regime exists. Full disclosure: I’ve done zero research to find out; if you have, I’d love to play my lazy web card and ask you to share your findings via a comment below.

I also wonder about the efficacy of a bespoke regime that sequences the leaves according to a dynamic pattern – test latest > test previous > present new – leading to a final summative test:

I call this regime spaced dynamic interleaved retrieval (quite a mouthful!) or dynamic interleaving for short. Again, if you’re aware of anything out there that’s similar, please let me know.

Higher Assessment

I find it strange when a blogger doesn’t approve my comment.

I consider comments the life blood of my own blog, and whether they be positive or negative, classy or rude, they all add to the diversity of the conversation. If your fragile ego can’t handle that, don’t blog.

I recently submitted a constructive comment to a particular blog post, twice, and it never eventuated. A later comment by someone else has.

Right, rather than waste my thought bubble, I’ve decided to reproduce the thrust of it here…

The OP was about the future of Higher Education being modular and flexible, which I agreed with. However something that caught my eye was the author’s observation about the assessment of prior learning via an essay or exam defeating the point of documentary evidence of previous course content or work experience.

Yet I feel that assessment via an essay or exam or some other means is the point. We needn’t rely so much on the bureaucracy if we could simply demonstrate what we know – regardless of how we came to know it.

When accrediting prior learning, a university needn’t get bogged down with evaluating myriad external permutations that may be worthy of credit, because what matters is the outcome of those permutations.

Similarly from the student’s point of view, it wouldn’t matter if they’ve done a mooc but not paid for the certificate, or if they did a course many years ago and worked in the field thereafter. What matters is the knowledge they can demonstrate now.

As a bastion of education, the university is losing ground to external competitors. Yet it maintains a certain gravitas that I suggest can be channelled into more of an assessment-driven role for society, whereby it validates knowledge at a certain standard and awards its qualifications accordingly.

Its role in teaching and learning is retained, of course, to fill in the gaps; powered by research to keep it at the forefront of the science.

25 more real-world examples of Virtual Reality

A couple of years ago I started up Virtual Reality Working Out Loud Week to promote real-world applications of virtual reality.

The inaugural #VRwolweek unearthed 20 real-world examples of the emerging technology, and the enduring popularity of that blog post tells me that we are hungry for more.

Loath to disappoint, I hereby present 25 more real-world examples of virtual reality, drawn from this year’s and last year’s events. I thank everyone who contributed to the following list.

• Kicking off with the Colonel, it would be remiss of me to omit KFC’s virtual escape room The Hard Way. Widely criticised for its evil genius paradigm, I urge us to appreciate the game’s otherwise authenticity. If used as a primer for training in real life, then it’s an engaging example of setting up an employee for success.
• Anchor Construction uses virtual reality to train its construction workers, while UPS uses it to train its truck drivers.
• South Wales Fire and Rescue uses interactive 360° video to train its new recruits on extricating a casualty from a road traffic incident.
• The Dutch Fire Department uses 360° video to teach the public how to react in case of an emergency, while on the other side of the flames in Australia FLAIM Trainer combines VR with haptics and heat-generating clothing to immerse firefighters in realistic situations.
• In Africa, Meet the Soldier aims to increase empathy among warring cattle farmers, while Cisco and Dimension Data are helping save the rhino.
• This charming Kiwi uses 360° video to record pov tutorials for mobile productivity apps. “See the apps and devices in action, in the context of where we work, live and play.”
• A group of middle school students has used 360° photos to create a virtual tour of Fort Vancouver, while the Chatham Historic Dockyard Trust uses 360° video to take you on a tour of their Age of Sail galleries.
• This Australian agency creates virtual tours and visualisations for the mining, architecture and tourism industries.
• Have you ever wondered how a self-driving car senses the world around it? Wonder no more with the Waymo 360° experience.
• Emmy Award winner for Outstanding Innovation in Interactive Storytelling, Pearl is a beautiful 360° animation that heralds the future of narrative.
• Virtual reality isn’t new to gamers, but now it’s social. Check out Evasion and Poker VR.
• I’m continually amazed at what can be achieved with CoSpaces Edu, such as the Virtual Reality Learning Lab’s uber cool reboot of Frogger. And while we’re going retro, have a laugh at Mario in real life.
• Topshop allows their customers to ride a virtual waterslide over the black cabs and double-deckers of central London.
• SeaWorld hybridises the real world with the virtual. Patrons of The Kraken Unleashed ride a rollercoaster while wearing VR headsets that plunge them into the abyss.It reminds me of Batman Adventure at Australia’s Movie World back in 1992, when we all sat on moveable seats in front of a big screen simulating the batplane screaming through Gotham City.A rollercoaster ramps the immersion up a few notches, to say the least, and I can see why it’s the perfect vehicle for a pre-recorded experience because the timing is precise.
• In Norway, Audi lets you test-drive their new Q5 in a giant virtual sandbox. It took me a while to work out the prospective customer would dig the racetrack in a real sandbox, which was then scanned and transformed into virtual reality. It’s a modern-day twist on Daytona USA presumably intended to attract the Amazon generation in-store to be worked over by the sales reps.Incidentally, I see the clever Scandi’s have now moved on to Augmented Reality with the Quattro Coaster app, which lets you build a road and drive a mini car on it in your living room.
• VR needn’t have an Audi-sized budget to be effective for marketing. A product manager in the medical industry created a WebVR experience to promote the hi-tech material in her range of surgical gowns. Given her name you may deduce I know this person, so I can tell you this impressive project was done on a shoestring.
• Finally, these other examples of virtual reality in healthcare – for autism, disability and pain management – must surely turn the most ardent of sceptics.

Oh how far we’ve come since Hugo Gernsback strapped on his teleyeglasses back in 1968. Long may this wonderful technology continue to evolve!

Virtual Duality

Something struck me during this year’s Virtual Reality Working Out Loud Week.

Billed as an event for “anyone who is working with or experimenting with virtual reality, whether that be at home, at school or at work”, this was the second time I had run it. Again I was keen for our peers in L&D and other industries to share what they are doing with this emerging technology.

At the time of writing this blog post, the #vrwolweek hashtag achieved 612,836 impressions on Twitter with an estimated reach of 350,292 accounts. Impressive indeed. Less impressive, however, is the fact that barely a dozen people shared an experience.

And this is what struck me… There is a gulf between those who talk about VR and those using it, and it appears this gulf is widening.

As last year’s 20 real-world examples of virtual reality attests, the technology is being applied by pioneers in various industries. This year unearthed additional examples in healthcare, transport, firefighting, education, special needs, gaming and tourism.

This year also highlighted folks such as Robert Ibisch, Flemming Funch, Lorraine Minister, M. Lovecraft, Simon Dueckert and Arun Pradhan who are actively experimenting with VR.

So that leaves approximately 350,000 people who have nothing to share. Is that because they can’t or because they won’t…? In any case, they didn’t.

As with so many other examples of technology, there is a division between the haves and the have nots. Yet among those who own a smartphone and can afford $20 for an entry-level headset, VR polarises the doers and the do nots.