Archive for the ‘augmented reality’ category

20 real-world examples of Augmented Reality

26 January 2019

The 2019 instalment of Virtual Reality Working Out Loud Week kicks off next month. For a couple of previous VR WOL Weeks I’ve collated lists of the examples that the participants unearthed – see 20 real-world examples of Virtual Reality and 25 more real-world examples of Virtual Reality.

This time however I’ve decided to do something different. I’ve decided to focus my attention on Augmented Reality.

As with VR, there’s plenty of talk out there about how wonderful AR is and the incredible potential it offers us. There’s also a lot of talk about how much better it is than VR, which I find comically absolutist. Surely it’s circumstantial?

In any case, I’m interested in what people in the real world are currently doing with this emerging technology, so I’ve collated the following examples.

Mixed reality jet engine

In the post-Pokémon Go era, an increasing number of AR games pepper the market. While one game replaces Pikachu with robots, another goes a step further by leveraging the background environment as you ward off alien invaders.

For its part, Niantic is continuing to evolve Pokémon Go with occlusion, by which the augmentation integrates with the background environment. (For example, a monster pops its head up from behind your couch.) Word on the street is this kind of next level immersion will feature in their upcoming Harry Potter spinoff.

The trail being blazed by the entertainment industry is being quickly followed by workplace educators. For example, Japan Airlines uses AR to train its engine technicians, while Deakin University uses it to teach its healthcare students. In terms of performance support, doctors use AR to perform target guided surgery, while the Royal Navy uses it to aid the officer of the watch.

Some companies are also using AR to educate their customers. In my previous blog post Educate everyone I praised Hyundai’s virtual guide which helps Sonata owners maintain their vehicle. Another car maker, MINI, has glasses that not only provide its drivers with navigational prompts, but also lets them see through the car.

In the world of retail furniture, Ikea lets you see how their products look in your home and another app walks you through assembling them.

Customer education blurs with marketing, and prime examples of the latter are ModiFace’s hair colour and nail polish previewers. In a similar vein, Kinect lets you try on clothes, Lacoste lets you try on shoes, Shop 4 Rings lets you try on jewellery, while Speqs completes your look with glasses.

In regard to the point-and-play type of AR, one of the most impressive I’ve seen is that of winemaker 19 Crimes who brings the convicts on their labels to life. The eerie black-and-white treatment reminds me of Koko tormenting a cat in The Clown’s Little Brother – which was released back in 1920!

A cartoon clown riding a real cat.

My coverage of augmented reality here is by no means exhaustive; it simply represents the instances I’ve stumbled upon recently. If you are aware of another real-world example, please share it via a comment below.

For more virtual reality, follow the #VRwolweek hashtag on Twitter and I encourage you to participate yourself.

In the meantime, who fancies running an #ARwolweek…?

Advertisements

Pokémon whoa!

9 August 2016

When I was a teenager I was addicted to video games. Not stay-awake-for-three-days-and-die-of-starvation kind of addicted, but I would spend every conceivable moment playing.

While I enjoyed car games and classic shoot’em ups, my favourites were the more strategic kinds of games in which you could explore, think, and discover. There was always somewhere else to reach, something fascinating to find, another stratagem to try.

I ended up imposing a ban on myself in order to focus on my studies, and I did not lift a video gaming finger for years afterwards.

Then along came Ingress.

Ingress screenshot

Ingress is a mobile game developed by Niantic (an offshoot of Google). You choose to join one of two factions that are fighting for world supremacy by capturing virtual portals – which are represented by public landmarks in the real world such as statues and fountains.

Ingress is oft described as an augmented reality game, but I disagree. It’s an alternate reality game. And it’s addictive. There’s something about covering a swathe of land in deep green (or blue, if you’re one of them) which elicits that sweet dopamine of achievement.

The plot isn’t real, of course, but that doesn’t matter. The emotion is.

Eventually I had to give up Ingress because I could feel my dubious history with video games flooding back, and I was chewing through my data like a hot knife through butter.

Zubat over Sydney Harbour

It is in this light that I look on with amusement at Niantic’s latest offering: Pokémon Go.

I was vaguely aware of this game since I maintain a general interest in augmented reality. Then one day it exploded. Suddenly everyone was playing it!

I hadn’t planned to play Pokémon Go myself, but given its sheer popularity I felt it would be remiss of me as an e-learning professional to forgo an informed opinion. So I gave it a go.

Frankly, it isn’t my cup of tea. Indeed I find the game a bit childish and pointless. When you catch one Pokémon, you catch another, and then another… ad infinitum. So what?

Still, I can see why others enjoy it. All the key game mechanics are there: it’s intuitive to play; your first Pokémon is easy to catch; you level up quickly; and the Pokédex is an interesting element that taps into the collector’s mentality.

You can also earn ownership of a gym (reminiscent of becoming a mayor in Foursquare) and you can incubate and hatch eggs (reminiscent of the parenting responsibilities for Tamagotchi).

Pokémon Go players congregating undercover by the forecourt of the Sydney Opera House.

Predictably, Pokémon Go has provoked a torrent of criticism from non-players, and I say some of it is warranted. From public nuisances to privacy concerns, the game has also fuelled discourtesy, disrespect, and self harm.

Personally I don’t think that you are stupid just because you play Pokémon Go. However, I do think the game is a vehicle for stupid people to express their stupidity.

I also think the decision to assign churches as gyms was ill advised, though I realise there is a technical reason for this and thankfully some churches are using their lemons to make lemonade.

Pokemon Go screenshot

On the plus side, Pokémon Go gets people walking – that’s unusual for a video game, even a mobile one – and it has single-handedly introduced millions of people around the world to the concept of AR.

And if we needed it, it’s yet another reminder to L&D folks that people love games.

A way with the fairies

20 April 2015

One of the greatest hoaxes to be perpetrated last century was that of the Cottingley Fairies.

In 1917, 9-year-old Frances Griffiths and her 16-year-old cousin, Elsie Wright, borrowed Elsie’s father’s camera to take a photograph of the fairies they claimed lived down by the creek. Sure enough, when he developed the plate, Elsie’s father saw several fairies frolicking in front of Frances’s face.

The first of the five photographs, taken by Elsie Wright in 1917, shows Frances Griffiths with the alleged fairies.

A couple of months later the girls took another photograph, this time showing Elsie playing with a gnome. While her father immediately suspected a prank, her mother wasn’t so sure and she took the photos to a local spiritualist meet-up. From there the photos, and three others taken subsequently by the girls, eventually hit the press and became a worldwide sensation.

Although the photos were quite real, the fairies of course were fake. In 1983, the cousins (by now old ladies) finally confessed they were cardboard cut-outs.

Not everyone – it must be said – had been fooled. In fact, most probably weren’t. However one who took the bait hook, line and sinker was none other than Sir Arthur Conan Doyle, whose involvement helped catapult the photographs to public attention in the first place.

It must also be said that as a devout spiritualist, Sir Arthur wanted to believe.

Disney Fairies Trail screenshot of Tinker Bell flying in front of a garden bed.

I was reminded of the Cottingley affair when I stumbled upon the Disney Fairies Trail app.

Developed in association with the Botanic Gardens Trust, the app promised to bring to life Tinker Bell and her flighty friends using our beautiful public spaces as the back drop.

Despite not being a member of the app’s target audience, I am an augmented reality advocate, so I downloaded the app and installed it on my iPad.

Its instructions were simple:

  • Choose a botanic garden.
  • Start the trail.
  • Use the map to help find all the fairy locations.
  • Your device will vibrate when there is a fairy nearby.
  • Use your device to find the fairy.
  • Tap the fairy to reveal their [sic] secrets.

I wanted to believe, so I hotfooted my way to the Royal Botanic Garden in Sydney to give it a go. Unfortunately the experience was less than optimal.

Disney Fairies Trail map

From the get-go, the map was so high level it was effectively useless.

After wandering semi-randomly around the park, I stumbled upon a tiny sign with an arrow promising that fairies live over there. Yet after crisscrossing my way all over the vicinity, my device never vibrated.

So I moved on. After a while I stumbled upon another tiny sign promising that the fairy trail continued this way, but after a short distance the path split out into multiple alternatives, none of which were sign posted.

After some more rambling, I finally stumbled upon a nice big sign declaring that fairies live here. Alas, still no vibrating – but I had a thought… perhaps the app doesn’t like my iPad? So I whipped out my Android smartphone and downloaded the app to it. It wouldn’t be as fun on the smaller screen, but I was determined to see a friggin fairy.

But it didn’t work on my smartphone either.

Deflated sports mascot

I should have read the customer reviews on the App Store before going to so much trouble. I clearly wasn’t the only one who had trouble with the app. And this bewilders me.

Unfortunately it was no surprise in retrospect when the real-life aspect of the experience didn’t work. To put my opinion into context, the Botanic Gardens Trust is the organisation that allows hordes of boot campers to bully tourists and locals alike off the park’s footpaths; and whose own staff choose the peak CBD lunch hour to drive their trucks and trailers along said paths.

But Disney! How could Disney fail to bring Tinker Bell & Co to life? The makers of masterpieces such as Toy Story and The Lion King evidently couldn’t engineer a dinky little augmented reality app that would work on my device of choice – or even on my device of second choice.

It just goes to show, if you want something done right, you have to do it yourself. So I did.

Green fairy

I discovered the Aurasma app a while ago, and I’ve been toying around with it to get a sense of how it works.

The app allows you to upload an image or a video, which appears or plays when you scan a real-world trigger with your device’s camera, hence augmenting reality.

I haven’t yet encountered a burning need to use it for an educational purpose at my workplace, but when I had trouble with the Fairies Trail app, I decided to see if I could replicate the intended experience.

So I downloaded Green Fairy 3 by TexelGirl, uploaded her to Aurasma, and associated her with a rose in the garden. When I scanned my iPad’s camera over the rose… Voila! She appeared.

Green fairy appearing in the garden

As you can see via my screenshot, Aurasma supports the binary transparency of PNG files; with the background of the fairy image invisible, the real background shines through. The app also supports the partial transparency of PNG files; if I were to make the fairy 50% transparent, the real background would be partially visible through her.

My fairy was a static image, so she wasn’t moving. While Aurasma doesn’t seem to support animated gifs, it does support video. However there appears to be a conspicuous problem. My understanding is that MP4 format does not support transparency, and while FLV does, it won’t run on the iPad. I tweeted the Aurasma folks asking them to clarify this, but they are yet to respond.

Nevertheless, I discovered a work-around which is to duplicate the real background in the background of your video clip. That way when the video launches, it appears that its background is the real background. (I suspect this is how Dewars brought Robert Burns to life.)

Of course, this means the experience is no longer augmented reality, but rather an illusion of it. Though I wouldn’t go so far as to call it a hoax. Fauxmented reality, perhaps?

Which won’t be a problem, if we want to believe.

The paradox of augmented reality

27 May 2013

Sydneysider Scott O’Brien is back in town after an extended stint in San Francisco. Scott is the Co-founder & CMO of Explore Engage, a digital media company that is attracting serious attention for its augmented reality eyewear.

I caught up with Scott in the harbour city and asked him the following questions…

  • What are your favourite examples of augmented reality? (0:08)
  • What are you working on at Explore Engage? (2:20)
  • How does your eyewear differ from Google Glass? (3:05)
  • Is augmented reality worth the hype? (3:54)
  • What opportunities exist for the finance sector? (5:04)
  • What opportunities exist for workplace education? (6:14)

I was impressed with the examples of augmented reality cited by Scott.

Medical education has long been the poster boy of salivatingly engaging content, and the tradition continues with this emerging technology. Daqri’s 4D Anatomy app showcases the visualisation capabilities of the medium, while the Australian Defence Force not only targets a real-world need with their Mobile Medic app, but also incorporates it into their recruitment process.

Ingress Enlightened logo

Google’s Ingress is an augmented reality MMOG that exemplifies the gamification capability of the medium. Two factions fight for control over the real world by capturing virtual “portals” that are represented by public landmarks such as statues and fountains.

The fact that Ingress was developed by Google’s internal startup, Niantic Labs, is enlightening (excuse the pun). Augmented reality is still an emerging technology in which experiments must be undertaken and failures borne. It is by learning from the results, and responding to them via adaptation, that you increase the probability of break-through success.

I am also fascinated by Google’s marketing strategy with Ingress. The game is in “closed beta” mode, which means you need an invitation to play it. Reminiscent of Studio 54, only the members of the “in” crowd have the privilege of enjoying that which is denied to others. Google deepens the mystique by seemingly neglecting to promote the product – instead relying on organic growth of the subculture.

On the subject of Google, I think Scott’s differentiation between Google Glass and Explore Engage’s Augmented Reality Eyewear is an important one. While Google Glass has augmented reality capability, it is essentially a wearable computer with which digital information is conveniently presented in front of the wearer’s eye. In contrast, the Explore Engage eyewear is specifically designed to integrate digital information with the real-world background. There is no better example of the latter concept than BMW’s Augmented Reality Glasses – which aren’t Explore Engage’s by the way, but are oh so sexy all the same.

While I’m on my definitions soapbox, I’ll take this opportunity to point the finger at Star Chart. This is a wonderful (and free) app, but its so-called “augmented reality mode” is no such thing; it does not lay its stellar information over the night sky! In contrast, Sun Seeker lays the sun’s trajectory over the real background. In other words, it augments reality.

Money

In terms of ROI, 2.5 million downloads of Transformers 3’s Defend the Earth speaks for itself. The return on Audi’s Virtual Q3 is less obvious, but that’s because it’s less about car sales and more about engaging consumers and associating the brand with innovation. How do you evaluate that? By analysing car sales of course, after the Q3 finally lands on Aussie shores.

While the Commonwealth Bank should be applauded for their Property Guide app, which combines geolocation with big data to provide something truly useful to their prospective customers, I must say as someone in the financial services industry: the general lack of financially oriented augmented reality apps represents a typical lack of imagination in the sector. Worse still, the examples highlighted by Infosys’s whitepaper are almost exclusively home finders and ATM locators, which means they’ve merely copied each other. Yawn.

As I am concurrently in the education profession, however, I must also recognise that the potential for augmented reality remains largely untapped. Scott’s examples attest to the power of the medium in terms of visualisation, gamification and performance support – which are factors that make education in the workplace engaging and effective. So what are we waiting for?

I think the mobility of the technology also remains under exploited. For example, how about an architecture tour of your local city in which details of buildings are highlighted when you point your mobile device at them? Or even better, when you look at them through your AR-enabled glasses?

And Scott’s mention of avatars adds more fuel to the fire of possibility. I imagine learning interventions in dangerous environments (such as mining sites) in which training can be undertaken in context, minus the threat to life or limb. Unlike in a simulator or a virtual world, the training is done at the workplace.

Therein lies the paradox of augmented reality. By complementing the real world with artificiality, it makes the learning experience more authentic.

When augmented reality isn’t

29 August 2011

I’m a big fan of the Powerhouse Museum.

In a world in which everyone loves to bang on about emerging technology, relatively few ever do anything about it. The PhM, however, has the guts to give it a go.

So I was excited to stumble upon their Augmented Reality browsing of Powerhouse Museum around Sydney app for Apple and Android.

Website of Augmented Reality browsing of Powerhouse Museum around Sydney

I love history, I love augmented reality, and I own an iPhone – so a combination of all three proved irresistible.

Unfortunately, though, I was a little bit disappointed.

Here’s why…

1. The title is meh

Exciting initiatives should have a catchy yet self-evident title to attract users like bears to a honey pot. However, Augmented Reality browsing of Powerhouse Museum around Sydney is boring and clunky.

I’d prefer something like Pocket Time Machine: An augmented reality tour of Old Sydney. A bit cheesy, I know, but a lot more interesting.

2. The app focuses on south CBD and the inner west

As the first European settlement on the continent – with a rich indigenous history – Sydney is teeming with sites of historical significance. However the app conspicuously misses the most obvious ones (eg Sydney Harbour Bridge, Sydney Opera House and the AMP Building).

Sydney Harbour Bridge under construction

Of course you have to start somewhere and the PhM website does promise a new version, but it refers to contemporary photography and gamification. I’d rather they expand their range into The Rocks and Circular Quay.

3. The app barely augments reality

Since the app is built on the Layar platform, it connects to Google Maps. Select the “i” icon at the relevant location and a photo pops up from the museum’s collection showing you what it looked like 100 years ago. This functionality is excellent, and frankly it could stand alone.

Screenshots of Augmented Reality browsing of Powerhouse Museum around Sydney

The augmented reality component comprises those floating “i” icons, which you’re supposed to select as you hold your device in front of you. Plainly speaking, they’re annoying:King Street on Augmented Reality browsing of Powerhouse Museum around Sydney

There are too many of them – which is confusing;

They are difficult to select – which is frustrating; and,

They have a tendency to get in the way – which defeats the purpose!

In short, the augmented reality component is redundant.

All is not lost

Of course, there is an alternative to abandoning augmented reality.

I suggest PhM follows the lead of the Museum of London and leverages the technology more fully. How? By laying the old photos over the real background.

Screenshot of Streetmuseum

This is what edtech is all about: transforming the educational experience.

Put a map on a smartphone? A crumpled tourist map is just as good; Plug in some photos? Nice touch, but those can be printed too; Lay century-old photos over the modern world in real time? Now that’s novel.

Even better, why not complement the visual with narration to provide a richer multimedia experience?

Who dares wins

As you would have gathered earlier, it is not my intention to pick on PhM. On the contrary, I salute them for having a red-hot go at something new.

Having taken the first step, they have earned the right to sit back and evaluate their app, with a view to making it even better the next time around.

The two types of augmented reality

20 July 2010

My favourite example of augmented reality is now a couple of years old:

While it might not be as flash as the xkcd enthusiasts might demand from this emerging technology, it remains practical and – gasp! – useful in the workplace.

And in one way at least, it is similar to this other famous example:

In both cases, artificial imagery is layered over the real world.

In the BMW example, the real world is on the other side of his glasses. In the Layar example, the real world is on the other side of his (or her?) smartphone.

Compare that with the wicked promo GE did for its Smart Grid:

Ryan watches the plane fly in The Sunday Telegraph's Night at the Museum 2 promo.I tried a similar thing at home when my local newspaper promoted Night At The Museum 2. I put the paper up to my webcam, and like magic a dinosaur skeleton came to life, a giant squid flailed its tentacles, and an aeroplane buzzed around my head.

But are these two latter examples really augmented reality?

By projecting both the digital imagery and the real background onto a computer screen, I would argue they are not actually augmenting reality. Instead, they are augmenting a representation of reality.

It’s just like adding cartoons to a movie set like they did in Who Framed Roger Rabbit, using CGI like they did in Star Wars, or even scribbling a moustache and devil horns onto someone’s photo.

Bob Hoskins & Jessica Rabbit, Jar Jar Binks, and a mistreated basketball coach.

In all these examples, the background isn’t real. It’s film, or light, or paper. In other words, a copy of reality.

Rewind

This insight was genius – at least in my own mind – until I realised that a smartphone doesn’t actually show reality on the other side of itself as do goggles or the viewfinder of an old camera. Instead, the device digitises the image and represents it as pixels on the screen, like a modern camera.

With that in mind, the Layar example is closer to the GE example than it is to the BMW example. Damn!

This was bugging me, and after a period of reflection I think I’ve identified why.

New criteria

To me, the exciting emergent form of augmented reality has the following characteristics…

1. It adopts the user’s personal POV.

When a webcam captures reality and projects it onto a computer screen, it’s not real in the sense that you don’t look at the background in that way (unless you constantly carry a mirror around with you).

A smartphone similarly projects the background onto its screen, but because you are mobile and pointing the device in front of you, it is for all intents and purposes real.

2. It is live.

We don’t live our lives by watching a recording of it. We live it here and now.

Reality is in real-time.

The two types

In light of the above criteria, I recognise two types of augmented reality:

  • Type I Augmented Reality (AR1), whereby the artificial imagery is layered over the background from the personal POV in real-time;

    and

  • Type II Augmented Reality (AR2), whereby the artificial imagery is layered over the background from an impersonal POV or not in real-time.

So this is an example of AR2

…because while the background is certainly real and the POV is personal, it’s not in real-time. It’s a recording.

Compare it to this example of AR1:

So what?

I know I’m being really pedantic, but for workplace learning purposes, it helps to be clear on what we’re talking about.

I think Type I Augmented Reality has amazing untapped potential because we see our workplace from our personal POV in real-time.

Type II Augmented Reality certainly has fantastic uses, but Type I is so much more authentic.

I’m sure we’ll see more AR2, and I hope we do.

However, I’m really looking forward to more AR1!