Posted tagged ‘statistics’

The sum of us

10 July 2017

What is the definition of the term “data scientist”…?

In my previous post, Painting by numbers, I offered a shorthand definition of data science based on what I could synthesise from the interwebs. Namely, it is the combination of statistics, computer programming, and domain expertise to generate insight. It follows, then, that the definition of data scientist is someone who has those skill sets.

Fat chance!

In this post I intended to articulate my observation that in the real world, incredibly few people could be considered masters of all three disciplines. I was then going to suggest that rather than seeking out these unicorns, employers should build data science teams comprising experts with complementary talents. I say “was” because I subsequently read this CIO article by Thor Olavsrud in which he quotes Bob Rogers saying, well… that.

Given Thor and Bob have stolen my thunder (18 months ago!) I think the only value I can add now is to draw a parallel with pop culture. So I will do so with the geeky HBO sitcom Silicon Valley.

The cast of Silicon Valley: Dinesh, Gilfoyle, Richard, Jared and Erlich.

If you aren’t familiar with this series, the plot revolves around the trials and tribulations of a start-up called Pied Piper. Richard is the awkward brainiac behind a revolutionary data compression algorithm, and he employs a sardonic network engineer, Gilfoyle, and another nerdy coder, Dinesh, to help bring it to market. The other team members are the ostentatious Erlich – in whose incubator (house) the group can work rent-free in exchange for a 10% stake – and Jared, a mild-mannered economics graduate who could have been plucked from the set of Leave It to Beaver.

The three code monkeys are gifted computer scientists, but they have zero business acumen. They are entirely dependent on Jared to write up their budgets and forecasts and all the other tickets required to play in the big end of town. Gilfoyle and Dinesh’s one attempt at a SWOT analysis is self-serving and, to be generous, NSFW.

Conversely, Jared would struggle to spell HTML.

Arguably the court jester, Erlich, is the smartest guy in the room. Despite his OTT bravado and general buffoonery, he proves his programming ability when he rolls up his sleeves and smashes out code to rescue the start-up from imploding, and he repeatedly uses his savvy to shepherd the fledgling business through the corporate jungle.

Despite the problems and challenges the start-up encounters throughout the series, it succeeds not because it is a team of unicorns, but because it comprises specialists and a generalist who work together as a team.

Unicorn silhouette

And so the art of Silicon Valley shows us how unlikely we would be in real-life to recruit an expert statistician / computer programmer / business strategist. Each is a career in its own right that demands years of education and practice to develop. A jack-of-all-trades will inevitably be a master of none.

That is not to say a statistician can’t code, or a programmer will be clueless about the business. My point is, a statistician will excel at statistics, a computer programmer will excel at coding, while a business strategist will excel at business strategy. And I’m not suggesting the jack-of-all-trades is useless; on the contrary, he or she will be the glue that holds the specialists together.

So that begs the question… which one is the data scientist?

Since each is using data to inform business decisions, I say they all are.

Advertisements

Painting by numbers

3 June 2017

A lifetime ago I graduated as an environmental biologist.

I was one of those kids who did well in school, but had no idea what his vocation was. As a pimply teenager with minimal life experience, how was I to know even half the jobs that existed?

After much dilly dallying, I eventually drew upon my nerdy interest in science and my idealistic zeal for conservation and applied for a BSc. And while I eventually left the science industry, I consider myself extremely fortunate to have studied the discipline because it has been the backbone of my career.

Science taught me to think about the world in a logical, systematic manner. It’s a way of thinking that is founded on statistics, and I maintain it should inform the activities we undertake in other sectors of society such as Learning & Development.

The lectures I attended and the exams I crammed for faded into a distant memory, until the emergence of learning analytics rekindled the fire.

Successive realisations have rapidly dawned on me that I love maths and stats, I’ve floated away from them over time, the world is finally waking up to the importance of scientific method, and it is high time I refocused my attention onto it.

So it is in this context that I have started to review the principles of statistics and its contemporary manifestation, analytics. My exploration has been accompanied by several niggling queries: what’s the difference between statistics and analytics? Is the latter just a fancy name for the former? If not, how not?

Overlaying the post-modern notion of data science, what are the differences among the three? Is a data scientist, as Sean Owen jokingly attests, a statistician who lives in San Francisco?

The DIKW Pyramid

My journey of re-discovery started with the DIKW Pyramid. This beguilingly simple triangle models successive orders of epistemology, which is quite a complex concept. Here’s my take on it…

The DIKW Pyramid, with Data at the base, Information a step higher, Knowledge another step higher, and Wisdom at the peak.

At the base of the pyramid, Data is a set of values of qualitative or quantitative variables. In other words, it is the collection of facts or numbers at your disposal that somehow represent your subject of study. For example, your data may be the weights of 10,000 people. While this data may be important, if you were to flick through the reams of numbers you wouldn’t glean much from them.

The next step up in the pyramid is Information. This refers to data that has been processed to make it intelligible. For example, if you were to calculate the average of those ten thousand weights, you’d have a comprehensible number that is inherently meaningful. Now you can do something useful with it.

The next step up in the pyramid is Knowledge. To avoid getting lost in a philosophical labyrinth, I’ll just say that knowledge represents understanding. For example, if you were to compare the average weight against a medical standard, you might determine these people are overweight.

The highest step in the pyramid is Wisdom. I’ll offer an example of wisdom later in my deliberation, but suffice it to say here that wisdom represents higher order thinking that synthesises various knowledge to generate insight. For example, the wise man or woman will not only know these people are overweight, but also recognise they are at risk of disease.

Some folks describe wisdom as future focused, and I like that because I see it being used to inform decisions.

Statistics

My shorthand definition of statistics is the analysis of numerical data.

In practice, this is done to describe a population or to compare populations – that is to say, infer significant differences between them.

For example, by calculating the average weight of 10,000 people in Town A, we describe the population of that town. And if we were to compare the weights of those 10,000 people with the weights of 10,000 people in Town B, we might infer the people in Town A weigh significantly more than the people in Town B do.

Similarly, if we were to compare the household incomes of the 10,000 people in Town A with the household incomes of the 10,000 people in Town B, we might infer the people in Town A earn significantly less than the people in Town B do.

Then if we were to correlate all the weights against their respective household incomes, we might demonstrate they are inversely proportional to one another.

The DIKW Pyramid, showing statistics converting data into information.

Thus, our statistical tests have used mathematics to convert our data into information. We have climbed a step up the DIKW Pyramid.

Analytics

My shorthand definition of analytics is the analysis of data to identify meaningful patterns.

So while analytics is often conflated with statistics, it is indeed a broader expression – not only in terms of the nature of the data that may be analysed, but also in terms of what is done with the results.

For example, if we were to analyse the results of our weight-related statistical tests, we might recognise an obesity problem in poor neighbourhoods.

The DIKW Pyramid, showing analytics converting data into knowledge.

Thus, our application of analytics has used statistics to convert our data into information, which we have then translated into knowledge. We have climbed another step higher in the DIKW Pyramid.

Data science

My shorthand definition of data science is the combination of statistics, computer programming, and domain expertise to generate insight. Or so I’m led to believe.

Given the powerful statistical software packages currently available, I don’t see why anyone would need to resort to hand coding in R or Python. At this early stage of my re-discovery, I can only assume the software isn’t sophisticated enough to compute the specific processes that people need.

Nonetheless, if we return to our obesity problem, we can combine our new-found knowledge with existing knowledge to inform strategic decisions. For example, given we know a healthy diet and regular exercise promote weight loss, we might seek to improve the health of our fellow citizens in poor neighbourhoods (and thereby lessen the burden on public healthcare) by building sports facilities there, or by subsidising salad lunches and fruit in school canteens.

The DIKW Pyramid, showing data science converting data into wisdom.

Thus, not only has our application of data science used statistics and analytics to convert data into information and then into knowledge, it has also converted that knowledge into actionable intelligence.

In other words, data science has converted our data into wisdom. We have reached the top of the DIKW Pyramid.

5 games every e-learning professional should play

3 April 2017

You can narrow down someone’s age by whether they include spaces in their file names. If they do, they’re under 40.

That is a sweeping declaration, and quite possibly true.

Here’s another one… Gamers are a sub-culture dominated by young men.

This declaration, however, is stone-cold wrong. In fact, 63% of American households are home to someone who plays video games regularly (hardly a sub-culture). Gamers are split 59% male / 41% female (approaching half / half) while 44% of them are over the age of 35 (not the pimply teenagers one might expect). [REF]

In other words, the playing of video games has normalised. As time marches on, not gaming is becoming abnormal.

Woman and man seated on a couch playing a video game.

So what does this trend mean for e-learning professionals? I don’t quite suggest that we start going to bed at 3 a.m.

What I do suggest is that we open our eyes to the immense power of games. As a profession, we need to investigate what is attracting and engaging so many of our colleagues, and consider how we can harness these forces for learning and development purposes.

And the best way to begin this journey of discovery is by playing games. Here are 5 that I contend have something worthwhile to teach us…

1. Lifesaver

Lifesaver immediately impressed me when I first played it.

The interactive film depicts real people in the real world, which maximises the authenticity of the learning environment, while the decision points at each stage gate prompt metacognition – which is geek speak for realising that you’re not quite as clever as you thought you were.

The branched scenario format empowers you to choose your own adventure. You experience the warm glow of wise decisions and the consequences of poor ones, and – importantly – you are prompted to revise your poor decisions so that the learning journey continues.

Some of the multiple-choice questions are unavoidably obvious; for example, do you “Check for danger and then help” or do you “Run to them now!”… Duh. However, the countdown timer at each decision point ramps up the urgency of your response, simulating the pressure cooker situation in which most people I suspect would not check for danger before rushing over to help.

Supplemented by extra content and links to further information, Lifesaver is my go-to example when recommending a game-based learning approach to instructional design.

2. PeaceMaker

Despite this game winning several prestigious awards, I hadn’t heard of PeaceMaker until Stacey Edmonds sang its praises.

This game simulates the Israeli-Palestinian conflict in which you choose to be the Israeli Prime Minister or the Palestinian President, charged with making peace in the troubled region.

While similar to Lifesaver with its branched scenario format, its non-linear pathway reflects the complexity of the situation. Surprisingly quickly, your hipsteresque smugness evaporates as you realise that whatever you decide to do, your decisions will enrage someone.

I found this game impossible to “win”. Insert aha moment here.

3. Diner Dash

This little gem is a sentimental favourite of mine.

The premise of Diner Dash is beguilingly simple. You play the role of a waitress in a busy restaurant, and your job is to serve the customers as they arrive. Of course, simplicity devolves into chaos as the customers pile in and you find yourself desperately trying to serve them all.

Like the two games already mentioned, this one is meant to be a single player experience. However, as I explain in Game-based learning on a shoestring, I recommend it be deployed as a team-building activity.

4. Keep Talking and Nobody Explodes

As its name suggests, Keep Talking and Nobody Explodes is a multi-player hoot. I thank Helen Blunden and David Kelly for drawing it to my attention.

In the virtual reality version of the game, the player wearing the headset is immersed in a room with a bomb. The other player(s) must relay the instructions in their bomb defusal manual to their friend so that he/she can defuse said bomb. The trouble is, the manual appears to have been written by a Bond villain.

It’s the type of thing at which engineers would annoyingly excel, while the rest of us infuriatingly fail. And yet it’s both fun and addictive.

As a corporate e-learning geek, I’m also impressed by the game’s rendition of the room. It underscores for me the potential of using virtual reality to simulate the office environment – which is typically dismissed as an unsuitable subject for this medium.

5. Battlefield 1

I could have listed any of the latest games released for Xbox or PlayStation, but as a history buff I’m drawn to Battlefield 1.

It’s brilliant. The graphics, the sounds, the historical context, the immersive realism, are nothing short of astonishing. We’ve come a long way since Activision’s Tennis.

Activision's Tennis video game on a vintage TV featuring two blocky players on court.

My point here is that the advancement of gaming technology is relentless. While we’ll never have the budget of Microsoft or Sony to build anything as sophisticated as Battlefield 1, it’s important we keep in touch with what’s going on in this space.

Not only can we be inspired by the big end of town and even pick up a few design tips, we need to familiarise ourselves with the world in which our target audience is living.

What other games do you recommend we play… and why?

Playing by numbers

23 April 2012

The theme of last week’s Learning Cafe in Sydney was How to Win Friends and Influence Learning Stakeholders.

Among the stakeholders considered was the “C-Level & Leadership”. This got me thinking, do the C-suite and lower rung managers expect different things from L&D?

There’s no shortage of advice out there telling us to learn the language of finance, because that’s what the CEO speaks. And that makes sense to me.

While some of my peers shudder at the term ROI, for example, I consider it perfectly reasonable for the one who’s footing the bill to demand something in return.

Show me the money.

Stack of Cash

But I also dare to suggest that the managers who occupy the lower levels of the organisational chart don’t give a flying fox about all that.

Of course they “care” about revenue, costs and savings – and they would vigorously say so if asked! – but it’s not what motivates them day to day. What they really care about is their team’s performance stats.

I’m referring to metrics such as:

• Number of widgets produced per hour
• Number of defects per thousand opportunities
• Number of policy renewals
• Number of new write-ups

In other words, whatever is on their dashboard. That’s what they are ultimately accountable for, so that’s what immediately concerns them.

Woman drawing a graph

The business savvy L&D consultant understands this dynamic and uses it to his or her advantage.

He or she appreciates the difference between what the client says they want, and what they really need.

He or she realises the client isn’t invested in the training activity, but rather in the outcome.

He or she doesn’t start with the solution (“How about a team-building workshop?”), but rather with the performance variable (“I see your conversion rate has fallen short of the target over the last 3 months”).

He or she knows that the numbers that really matter don’t necessarily have dollar signs in front of them.

The unscience of evaluation

29 November 2011

Evaluation is notoriously under done in the corporate sector.

And who can blame us?

With ever increasing pressure bearing down on L&D professionals to put out the next big fire, it’s no wonder we don’t have time to scratch ourselves before shifting our attention to something new – let alone measure what has already been and gone.

Alas, today’s working environment favours activity over outcome.

Pseudo echo

I’m not suggesting that evaluation is never done. Obviously some organisations do it more often than others, even if they don’t do it often enough.

However, a secondary concern I have with evaluation goes beyond the question of quantity: it’s a matter of quality.

As a scientist – yes, it’s true! – I’ve seen some dodgy pseudo science in my time. From political gamesmanship to biased TV and clueless newspaper reports, our world is bombarded with insidious half-truths and false conclusions.

The trained eye recognises the flaws (sometimes) but of course, most people are not science grads. They can fall for the con surprisingly easily.

The workplace is no exception. However, I don’t see it as employees trying to fool their colleagues with creative number crunching, so much as those employees unwittingly fooling themselves.

If a tree falls in the forest

The big challenge I see with evaluating learning in the workplace is how to demonstrate causality – ie the link between cause and effect.

Suppose a special training program is implemented to improve an organisation’s flagging culture metric. When the employee engagement survey is run again later, the metric goes up.

Graph

Congratulations to the L&D team for a job well done, right?

Not quite.

What actually caused the metric to go up? Sure, it could have been the training, or it could have been something else. Perhaps a raft of unhappy campers left the organisation and were replaced by eager beavers. Perhaps the CEO approved a special bonus to all staff. Perhaps the company opened an onsite crèche. Or perhaps it was a combination of factors.

If a tree falls in the forest and nobody hears it, did it make a sound? Well, if a few hundred employees undertook training but nobody measured its effect, did it make a difference?

Without a proper experimental design, the answer remains unclear.

Evaluation by design

To determine with some level of confidence whether a particular training activity was effective, the following eight factors must be considered…

Scientist

1. Isolation – The effect of the training in a particular situation must be isolated from all other factors in that situation. Then, the metric attributed to the staff who undertook the training can be compared to the metric attributed to the staff who did not undertake the training.

In other words, everything except participation in the training program must be more-or-less the same between the two groups.

2. Placebo – It’s well known in the pharmaceutical industry that patients in a clinical trial who are given a sugar pill rather than the drug being tested sometimes get better. The power of the mind can be so strong that, despite the pill having no medicinal qualities whatsoever, the patient believes they are doing something effective and so their body responds in kind.

As far as I’m aware, this fact has never been applied to the evaluation of corporate training. If it were, the group of employees who were not undertaking the special training would still need to leave their desks and sit in the classroom for three 4-hour stints over three weeks.

Why?

Because it might not be the content that makes the difference! It could be escaping the emails and phone calls and constant interruptions. It could be the opportunity to network with colleagues and have a good ol’ chat. It might be seizing the moment to think and reflect. Or it could simply be an appreciation of being trained in something, anything.

3. Randomisation – Putting the actuaries through the training and then comparing their culture metric to everyone else’s sounds like a great idea, but it will skew the results. Sure, the stats will give you an insight into how the actuaries are feeling, but it won’t be representative of the whole organisation.

Maybe the actuaries have a range of perks and a great boss; or conversely, maybe they’ve just gone through a restructure and a bunch of their mates were made redundant. To minimise these effects, staff from different teams in the organisation should be randomly assigned to the training program. That way, any localised factors will be evened out across the board.

4. Sample size – Several people (even if they’re randomised) can not be expected to represent an organisation of hundreds or thousands. So testing five or six employees is unlikely to produce useful results.

5. Validity – Calculating a few averages and generating a bar graph is a sure-fire way to go down the rabbit hole. When comparing numbers, statistically valid methods such as Analysis of Variance are required to get significant results.

6. Replication – Even if you were to demonstrate a significant effect of the training for one group, that doesn’t guarantee the same effect for the next group. You need to do the test more than once to establish a pattern and negate the suspicion of a one-off.

7. Subsets – Variations among subsets of the population may exist. For example, the parents of young children might feel aggrieved for some reason, or older employees might feel like they’re being ignored. So it’s important to analyse subsets to see if any clusters exist.

8. Time and space – Just because you demonstrated the positive effect of the training program on culture in the Sydney office, doesn’t mean it will have the same effect in New York or Tokyo. Nor does it mean it will have the same effect in Sydney next year.

Weird science

Don’t get me wrong: I’m not suggesting you need a PhD to evaluate your training activity. On the contrary, I believe that any evaluation – however informal – is better than none.

What I am saying, though, is for your results to be more meaningful, a little bit of know-how goes a long way.

For organisations that are serious about training outcomes, I go so far as to propose employing a Training Evaluation Officer – someone who is charged not only with getting evaluation done, but with getting it done right.

_______________

This post was originally published at the Learning Cafe on 14 November 2011.