10 ways augmented reality is changing the world
Taking ‘try before you buy’ to the next level, bringing the restaurant experience to our homes, and revolutionising the way people interact with brands — the layer of reality introduced by AR is changing everything.
According to Think With Google:
Augmented reality content saw a 94% higher conversion rate than products without that content
43% of people who shop via their phones expect beauty brands to use augmented reality for ‘try-on’ experiences
47% of shoppers who use their phones to research and purchase cars expect automotive retailers to provide AR tools that enable them to ‘see’ the vehicles in their driveways and peek inside
50% of people who use their phones to shop perceive brands and retailers which offer augmented reality experiences as being more innovative than those which don’t
Below are ten of our favourite uses of augmented reality to date. Some you can try today. Some are from the future.
IKEA: rearrange your furniture before buying
One of the first modern uses of augmented reality many of us will have witnessed came from IKEA. Their ‘Place’ app was front and center of Apple’s first ‘ARKit’ showcase. You can use it to visualize how IKEA’s furniture will look in your home before buying.
By using the same LiDAR scanner that iPhones utilize for the iOS ‘Measure’ app, IKEA Place can accurately show whether furnishings will fit within a space. It eliminates the issue of shoppers liking furniture on the showroom floor but not feeling the same way when they see a piece in the context of their home.
Niantic: Pokémon GO and other AR games
The summer of 2016 will forever be remembered as the summer of Pokémon GO. It’s rare for something to bring strangers together in public all across the globe, but the developers at Niantic achieved just that through their work with Nintendo.
Pokémon GO added a layer of gameplay to real-world maps and players’ cameras, turning street corners into virtual battlegrounds and placing the franchise’s iconic creatures into our reality.
In 2019, Niantic created Harry Potter: Wizards Unite with Warner Bros. Then, in March 2021, they announced a new partnership with Nintendo to bring more iconic video game franchises to the ‘real world’ through AR, starting with the Pikmin series.
Instagram Filters: your face becomes the billboard
First, we were vomiting rainbows. Then we turned into dogs that licked the inside of our phone screens. Snapchat set the AR face filter trend in motion with their ‘Lenses’ feature in 2015, and hundreds of brands and artists have fuelled the hype ever since. You rarely see Instagram Stories without them nowadays.
I had the opportunity to talk to Instagram filter creator and Spark AR partner Jerzy Pilch about how brands are using augmented reality to engage audiences, what AR might look like five years from now, and how restaurants can use the tech to recreate their in-house experience in delivery customer’s homes.
Two of my favourite filters by Jerzy are the randomizer he created for Greggs (‘what’s for lunch?’) and the ‘try this lipstick’ experience he built for Marionnaud Italia and Superdrug.
Watch our conversation here:
Feet Filters: fashion that only exists in AR
Fashion house Gucci first used augmented reality in 2019 to let customers ‘try on’ shoes before buying. Two years later, in March 2021, they released their first digital-only sneakers.
The brand’s creative director, Alessandro Michele, designed the ‘Virtual 25’ kicks in collaboration with AR fashion platform ‘Wanna’. The non-physical shoes cost $12 (about £9). Customers can take photos with the sneakers ‘on’, and download an in-game version of them for their VRChat and Roblox avatars.
Wanna’s co-founder, Sergey Arkhangelskiy, said: "In five or maybe 10 years a relatively big chunk of fashion brands revenue will come from digital products.”
Directions: giant arrows now, giant chickens soon
In my interview with augmented reality designer Jerzy Pilch (see the video above), he mentions the concept of a giant chicken appearing in AR to guide people to the nearest Nando’s! A similar idea already exists in the Google Maps app: virtual arrows and street names laid on top of your phone camera’s view.
Once we’re all walking around with AR glasses on our faces, there’s no doubt businesses will be able to pay to have their shops and venues highlighted in our heads-up displays (HUDs).
HUDs with real-time information and navigation cues will also inevitably come to our car windshields. WayRay’s holographic displays can show drivers details about the turns they need to make, flash up warnings about obstacles (e.g. other cars reversing into your path), and highlight parking fees while in an ‘empty space search’ mode. There’s plenty of safety legislature to get through, but it’s coming.
Translation: AR is the new Babelfish
One of my earliest experiences with augmented reality came through Word Lens, a smartphone app I first used in 2010. You held your camera up to any text — e.g. a menu, sign, or poster — and it would translate the words into a language of your choice, in real-time. It even tried to match the color/font of the text!
Google bought the developers of Word Lens, Quest Visual, in 2014, shuttered the standalone app in 2015, and baked the functionality directly into the Google Translate app.
Google Lens: understanding the world
Google also baked Word Lens functionality directly into ‘Google Lens’, the AR system built into the main Google app. But Google Lens also does a whole lot more.
Open Google Lens and hold it up to just about anything, and little dots will appear which you can tap to get information about the world around you. Shopping? Scan a shirt or a piece of furniture and you’ll see prices in other stores. Dining out? Scan the menu and you’ll see Google Maps reviews which show popular and recommended dishes. Sightseeing? Scan landmarks to read up on their history, and to check opening hours.
Google Lens can identify dog breeds, solve written math equations, copy text to your clipboard, add event details from posters to your calendar — so many possibilities! And it’s always growing and learning, tapping into new information and adding new connections between the physical and digital worlds.
NFTs and virtual art galleries
The tech world is ablaze with discussions about NFTs. This is a tricky one to get your head around, but bear with me. A digital artist named Beeple just made $69 million by selling NFTs, so they could be a big deal — or it might just be a bubble of hype.
‘NFT’ stands for ‘non-fungible token’. ‘Non-fungible’ means something is totally unique and can’t be replaced with something else. In terms of digital art, an NFT could be compared to an incredibly rare trading card: the high value and scarceness only exists because people agree it does.
An artist can create any digital piece — a picture, song, 3D model, etc. — and then ‘mint’ that artwork as an NFT. A buyer can then bid on that NFT, much like a collector in a traditional art auction house, and become the owner.
This is where many people start raising their eyebrows: much of the time, you’ll essentially be buying a JPEG image. Anyone can go download a copy of the artwork for themselves. Buying an NFT is pretty much just buying a certificate that confirms you bought it. There’s no physical artwork.
You could compare it to buying the Mona Lisa: only one person can own the original painting, but anyone can buy prints of the piece. The value is in the eye of the beholder.
With so many artists and brands banking on NFTs and other formats of digital-only work, people are starting to figure out ways to display them in the real world. A few companies have scrambled to build screens — more or less fancy digital photo frames.
Entrepreneur Mark Cuban has taken a different approach. He’s launched ‘Lazy’, an online ‘gallery space’ where buyers and collectors can show off their NFTs. Before long, these digital art galleries will be implemented in AR. NFT illustrations and sculptures hanging on your walls and floating around your kitchen in augmented reality. Get your friends round, put on some AR glasses, and admire art that’s not really there.
Still a bit confused by NFTs? This video featuring Beeple is a great place to start:
The packaging experience
In my interview with Jerzy, we talked about how delivery-only food brands which operate from dark kitchens will need to find innovative ways to make the at-home dining experience more exciting. Jerzy gave the example of an AR jungle ‘blooming’ around you as you unpack your meal.
Pizza Hut recently launched limited-edition boxes which, when scanned with a smartphone, allow customers to play an AR version of Pac-Man. It’s a throwback to the era of arcade cabinets in Pizza Huts. The most skillful players will have the chance to win one of those old-school cabinets as part of the campaign!
Neural signals: this is your brain on AR
Most of the augmented reality uses I’ve listed here rely on phones for interaction. If we want to get to the kind of tech Tony Stark plays around with in Iron Man, we’ll need to link our brain’s neural signals to AR glasses. Facebook Reality Labs recently showcased a prototype that somewhat achieves this.
With a wristband that reads physical/neural triggers and glasses or contact lenses which provide the AR display, Facebook Reality Labs claim they’re getting close to creating virtual interfaces which can be navigated and controlled without handheld devices.
In theory, this is the first step towards us being able to interact with digital interfaces by just thinking about what we want to do. So be sure to keep a clean mind!
More of Kitty’s thoughts about the future:
