Scaling up

Posted 1 December 2020 by Ryan Tracey
Categories: assessment, capability framework, evaluation

Tags: , , , , , , , , , , , , ,

In Roses are red, I proposed definitions for oft-used yet ambiguous terms such as “competency” and “capability”.

Not only did I suggest a competency be considered a task, but also that its measurement be binary: competent or not yet competent.

As a more general construct, a capability is not so readily measured in a binary fashion. For instance, the question is unlikely to be whether you can analyse data, but the degree to which you can do so. Hence capabilities are preferably measured via a proficiency scale.

Feet on scales

Of course numerous proficiency scales exist. For example:

No doubt each of these scales aligns to the purpose for which it was defined. So I wonder if a scale for the purpose of organisational development might align to the Kirkpatrick Model of Evaluation:

 Level  Label  Evidence 
0 Not Yet Assessed  None
1 Self Rater Self rated
2 Knower Passes an assessment
3 Doer Observed by others
4 Performer Meets relevant KPIs
5 Collaborator Teaches others

Table 1. Tracey Proficiency Scale (CC BY-NC-SA)

I contend that such a scale simplifies the measurement of proficiency for L&D professionals, and is presented in a language that is clear and self-evident for our target audience.

Hence it is ahem scalable across the organisation.

Roses are red

Posted 16 November 2020 by Ryan Tracey
Categories: capability framework, competency framework

Tags: , , , , , , , , , , , ,

It seems like overnight the L&D profession has started to struggle with the definition of terms such as “capability”, “competency” and “skill”.

Some of our peers consider them synonyms – and hence interchangeable – but I do not.

Indeed I recognise subtle but powerful distinctions among them, so here’s my 2-cents’ worth to try to cut through the confusion.

Old style botanical drawing of a rose and violets

Competency

From the get-go, the difference between the terms may be most clearly distinguished when we consider a competency a task. It is something that is performed.

Our friends in vocational education have already this figured out. For example, if we refer to the Tap furnaces unit of competency documented by the Australian Department of Education, Skills and Employment, we see elements such as Plan and prepare for furnace tapping and Tap molten metal from furnace.

Importantly, we also see performance criteria, evidence and assessment conditions. Meeting a competency therefore is binary: either you can perform the task successfully (you are “competent”) or you can not (in the positive parlance of educationalists, you are “not yet competent”).

Capability

Given a competency is a task, a capability is a personal attribute you draw upon to perform it.

An attribute may be knowledge (something you know, eg tax law), a skill (something you can do, eg speak Japanese), or a mindset (a state of being, eg agile).

I consider capability an umbrella term for all these attributes; they combine with one another to empower the behaviour that meets the competency.

Capability is an umbrella term for the attributes that empower the behaviour that meets a competency.

Frameworks

According to the definitions I’ve outlined above, we frequently see in the workplace that “capability frameworks” are mislabelled “competency frameworks” and vice versa.

Terms such as Decision Making and Data Analysis are capabilities – not competencies – and moreover they are skills. Hence, not only would I prefer they be referred to as such, but also that they adopt an active voice (Make Decisions, Analyse Data).

I also suggest they be complemented by knowledge and mindsets, otherwise the collection isn’t so much a capability framework as a “skills framework”; which is fine, but self-limiting.

Deployment

I have previously argued in favour of the L&D team deploying a capability framework as a strategic imperative, but now the question that begs to be asked is: should we deploy a capability framework or a competency framework?

My typical answer to a false dichotomy like this is both.

Since capabilities represent a higher level of abstraction, they are scalable across the whole organisation and are transferable from role to role and gig to gig. They also tend to be generic, which means they can be procured in bulk from a third party, and their low volatility makes them sustainable. The value they offer is a no-brainer.

In contrast, competencies are granular. They’re bespoke creations specific to particular roles, which makes them laborious to build and demanding to maintain. Having said that, their level of personalised value is sky high, so I advise they be deployed where they are warranted – targeting popular roles and pivotal roles, for example.

Semantics

A rose by any other name would smell as sweet.

Yet a rose is not a violet.

In a similar manner I maintain that capabilities and competencies are, by definition, different.

In any case, if we neglect them, the next term we’ll struggle to define is “service offering”.

Academic deflation

Posted 3 November 2020 by Ryan Tracey
Categories: higher education

Tags: , , , , , , , , ,

I once had a conversation with a man who scared me.

It was over 20 years ago, and I remember him being quite a lovely fellow. He was simply proud of his son earning his PhD.

What scared me was his conviction that in order to keep up in the imminent future, an undergraduate qualification would not be enough; one must earn a postgraduate qualification. Not to get ahead, but to remain competitive.

With the ink still drying on my Bachelor of Science degree, my heart sank at the prospect of several years’ more slog.

The man was describing academic inflation and upon reflection, I realised I had experienced it already. Only a few years prior, as I was studying my high school diploma, “going to uni” was all the rage. Accounting was inexplicably popular, and competition for the trendiest major – Communication – drove its course entry score to the dizzying heights of Medicine and Law.

Not going to uni was seriously uncool, which no doubt contributed to a shortage of tradies such as plumbers, who to this day can charge a fortune for changing a washer.

The parallels to Dr. Seuss’s The Sneetches are uncanny. This is a story about creatures called Sneetches, some of whom have a green star on their belly, some of whom do not. Of course the latter group covets those little status symbols, and their envy is duly exploited by an entrepreneur who invents a Star-On machine.

After the plain-bellies eagerly pay to get their own stars, the elitism of the original star-bellies wanes, and so the entrepreneur invents a Star-Off machine. The former elites pay an inflated price to reclaim their privilege, and so the pendulum swings back and forth with stars going on and off bellies until all the Sneetches run out of money.

Star confetti

Back in the real world which recently declared Australia’s most educated generation faces the worst job prospects in decades, I wonder: is academic inflation undergoing a Sneetch-like reversal?

The UK office of Ernst & Young ruffled a few feathers when they dropped the degree requirement for their entry-level jobs, while Elon Musk famously maintains that you don’t need a degree to work at Tesla.

I admit to not taking too much notice of this trend until Google launched its Career Certificates. Their courses can be completed online over several months, and they cover red hot topics such as data analytics and UX design. Their website says it all:

Learn job-ready skills to start or advance your career in high-demand fields. These certificates developed by Google connect you to top national employers who are hiring for related roles.

Furthermore, the tech giant’s SVP of Global Affairs claims we will consider our new career certificates as the equivalent of a four-year degree for related entry-level roles.

Wow… if it’s good enough for Google, then “top national employers” isn’t an empty promise. Suddenly that little green star doesn’t shine so bright.

Perhaps after years of academic inflation, the pendulum is swinging back towards academic deflation. The prospect should sound alarm bells for a sector that’s already reeling from the impact of COVID-19.

If the trend towards short, practical, employer-sanctioned courses continues, one day of course we’ll collectively realise that the way to get ahead of the pack will be to embark on a longer, deeper dive that leads to a qualification with scholarly gravitas.

Hence the next generation of students going to university may no longer be doe-eyed teenagers craving a foot in the door, but experienced operators seeking to enhance their careers.

Which will in turn attract the young ones back when they realise they’re missing out on all the best jobs.

That is until everyone, once again, has stars upon thars.

More than just a pretty face

Posted 6 October 2020 by Ryan Tracey
Categories: assessment, badges

Tags: , , , , , , , , , , , ,

I’ve blogged in favour of digital badges in the past, not because they’re colourful motivators – which arguably they are, at least for some people – but because they represent an achievement.

While the robustness of the criteria for earning a badge may be challenged, as may be the assessment of meeting said criteria, the concept holds true: a badge must be earned by demonstrating that you have done something.

What that something is is a variable to be defined. Some badges such as the ones that are popular among IT geeks are earned by completing a training program or by passing an exam. I call these “certification badges”.

However I maintain a stronger implementation of the idea emerges when we earn the badge by successfully executing a task (or a suite of tasks). I call these “practitioner badges”.

Assorted badges, including one stating Qualified Dog-Petter

For example, you might complete a 40-hour course and pass a massive multiple-choice quiz to earn an XYZ-issued “Project Management” badge. That’s quite an achievement.

But I’d be more impressed (and more confident as an employer) if you were to demonstrate how you’ve applied the XYZ-endorsed principles to a real project in the real world, thereby earning a “Project Manager” badge. To me, that’s a greater achievement because it shifts the focus of the exercise from the activity (learning) to its outcome (performance).

In an organisational context, I see opportunities to blend the tasks to enrich the experience. For example, one task may be to apply a principle to your current project, while the next task is to share your reflection of doing so on the enterprise social network; thereby facilitating not only metacognition and expert feedback, but also peer-to-peer knowledge sharing.

Celebrating the latest cohort of people who’ve earned badges in the same forum may also generate a bit of FOMO.

In any case, my point is a badge should be more than just a pretty face. I propose we distinguish between two types of badge – namely a certification badge and a practitioner badge – with the latter representing an achievement above and beyond the former.

Transformers

Posted 1 September 2020 by Ryan Tracey
Categories: instructional design, training

Tags: , , , , , , , , , , , , , , , , , , , , , , , ,

It seems like everyone’s spruiking the “new normal” of work.

The COVID-19 pandemic is keeping millions of previously office-bound employees at home, forcing L&D professionals to turn on a dime.

Under pressure to maintain business continuity, our profession has been widely congratulated for its herculean effort in adapting to change.

I’m not so generous.

Our typical response to the changing circumstances appears to have been to lift and shift our classroom sessions over to webinars.

In The next normal, which I published relatively early during lockdown, several of my peers and I recognised the knee-jerk nature of this response.

And that’s not really something that ought to be congratulated.

Who led the digital transformation of your company? The CEO (incorrect), The CTO (incorrect), COVID-19 (correct)

For starters, the virus exposed a shocking lack of risk management on our part. Digital technology is hardly novel, and our neglect in embracing it left us unprepared for when we suddenly needed it.

Look no further than the Higher Education sector for a prime example. They’re suffering a free-fall in income from international students, despite the consensus that people can access the Internet from other countries.

Beyond our misgivings with technology, moreover, the virus has also shone a light on our pedagogy. The broadcast approach that we deliver virtually today is largely a continuation of our practice pre-pandemic. It wasn’t quite right then, and it isn’t quite right now. In fact, isolation, digital distractions and Zoom fatigue probably make it worse.

I feel this is important to point out because the genie is out of the bottle. Employee surveys reveal that the majority of us either don’t want to return to the office, or we’ll want to split our working week at home. That means while in-person classes can resume, remote learning will remain the staple.

So now is our moment of opportunity. In the midst of the crisis, we have the moral authority to mature our service offering. To innovate our way out of the underwhelming “new normal” and usher in the modern “next normal”.

In some cases that will mean pivoting away from training in favour of more progressive methodologies. While I advocate these, I also maintain that direct instruction is warranted under some circumstances. So instead of joining the rallying cry against training per se, I propose transforming it so that it becomes more efficient, engaging and effective in our brave new world.

Transformer-style toy robot

Good things come in small packages

To begin, I suggest we go micro.

So-called “bite sized” pieces of content have the dual benefit of not only being easier to process from a cognitive load perspective, but also more responsive to the busy working week.

For example, if we were charged with upskilling our colleagues across the business in Design Thinking, we might kick off by sharing Chris Nodder’s 1.5-minute video clip in which he breaks the news that “you are not your users”.

This short but sweet piece of content piques the curiosity of the learner, while introducing the concept of Empathize in the d.school’s 5-stage model.

We’re all in this together

Next, I suggest we go social.

Posting the video clip to the enterprise social network seeds a discussion, by which anyone and everyone can share their experiences and insights, and thus learn from one another.

It’s important to note that facilitating the discussion demands a new skillset from the trainer, as they shift their role from “sage on the stage” to “guide on the side”.

It’s also important to note that the learning process shifts from synchronous to asynchronous – or perhaps more accurately, semi-synchronous – empowering the learner to consume the content at a time that is most convenient for them (rather than for the L&D department).

There is no try

Next, I suggest we go practical.

If the raison d’être of learning & development is to improve performance, then our newly acquired knowledge needs to be converted into action.

Follow-up posts on the social network shift from the “what” to the “how”, while a synchronous session in the virtual classroom enables the learner to practise the latter in a safe environment.

Returning to our Design Thinking example, we might post content such as sample questions to ask prospective users, active listening techniques, or an observation checklist. The point of the synchronous session then is to use these resources – to stumble and bumble, receive feedback, tweak and repeat; to push through the uncomfortable process we call “learning” towards mastery.

It’s important to recognise the class has been flipped. While time off the floor will indeed be required to attend it, it has become a shorter yet value-added activity focusing on the application of the knowledge rather than its transmission.

Again, it’s also important to note that facilitating the flipped class demands a new skillset from the trainer.

A journey of a thousand miles

Next, I suggest we go experiential.

Learning is redundant if it fails to transfer into the real world, so my suggestion is to set tasks or challenges for the learner to do back on the job.

Returning to our Design Thinking example, we might charge the learner with empathising with a certain number of end users in their current project, and report back their reflections via the social network.

In this way our return on investment begins immediately, prior to moving on to the next stage in the model.

Pics or it didn’t happen

Finally, I suggest we go evidential.

I have long argued in favour of informalising learning and formalising its assessment. Bums on seats misses the point of training which, let’s remind ourselves again, is to improve performance.

How you learned something is way less interesting to me than if you learned it – and the way to measure that is via assessment.

Returning to our Design Thinking example, we need a way to demonstrate the learner’s mastery of the methodology in a real-world context, and I maintain the past tense of open badges fits the bill.

In addition to the other benefits that badges offer corporates, the crux of the matter is that a badge must be earned.

Informalise learning. Formalise its assessment.

I am cognisant of the fact that my proposal may be considered heretical in certain quarters.

The consumption of content on the social network, for example, may be difficult to track and report. But my reply is “so what” – we don’t really need to record activity so why hide it behind the walls of an LMS?

If the openness of the training means that our colleagues outside of the cohort learn something too, great! Besides, they’ll have their own stories to tell and insights to share, thereby enriching the learning experience for everyone.

Instead it is the outcome we need to focus on, and that’s formalised by the assessment. Measure what matters, and record that in the LMS.

In other words, the disruptive force of the COVID-19 pandemic is an impetus for us to reflect on our habits. The way it has always been done is no substitute for the way it can be done better.

Our moment has arrived to transform our way out of mode lock.