Connected Worlds: Digital, physical, immersive storytelling
Connected Worlds is the first in a series of highlights from my time in the US, and it’s a cracker. This immersive, Dr Seuss-like fantasy world, more than 200 square metres in size, was recently launched at the New York Hall of Science (NYSCI). It’s a digital game, but one grounded in the very real theme of environmental sustainability. For a quick overview, here’s the marketing video:
You’re faced with four ecosystems: desert, plains, rainforest, swamp. A large waterfall courses down the back wall, and there are two other water sources on each side: river and reservoir. Your mission? Keep the world in balance. Ensure that each ecosystem has enough water to survive.
The Explainer (love the Hunger Games-esque name) orientates you before you begin.
To get planting, hold your hands out in the air. Seeds materialise like fairy dust.
Lower your arms to let the seed pods pop and drop.
To direct the water, drag large silver logs into position.
Cut down dead plants by swiping your arm. But be careful – while you’re working on one ecosystem, another may start to fail. You need to work as a team.
Social, physical, real, magical
Connected Worlds is a fantastic example of a digital experience that’s also:
- social – if you can’t communicate and cooperate, you fail
- physical – only by moving your body do things happen
- real – in this case, I mean a real purpose rather than real objects: to keep the world in balance
- magical – you have power, you get to play God … plus everything looks really cool.
It’s virtual reality that connects rather than isolates (in the way that VR headsets currently do). For science and tech museums wanting to retain hands-on engagement, it makes a lot of sense to go big, social, and immersive over small, individual, and wearable/mobile.
I was reminded of an experience we used to have at Te Papa: The Wall – a digital, social space where you could upload images and move them around with a wand. It was radical for its time, but Connected Worlds pretty much knocks it out of the park, particularly with its level of interactivity and its real, relevant purpose.
Failing to cooperate = learning to fail
One 30-minute session wasn’t enough for my kids. We sat out the next one and then did it again.
My eldest daughter, Luca (10), was scowling the second time: ‘That boy was hogging the whole waterfall! He wasn’t letting any water go anywhere!’ But that’s just it. The learning is in failing as much as succeeding. The fact that everyone has an impact is even clearer, and the need for cooperation even more glaring. Here’s Luca working with some boys in a more collaborative moment.
I saw mums and dads working with their kids, pulling each other around, calling for logs, cracking themselves up. I saw kids captivated by the power in their hands. I overheard others plotting sabotage, hauling logs to form a dam. Whatever the case, everyone was fully engaged. And you could certainly see the beginnings of that holy grail of engagement – connection between visitors in the gallery.
Intuitive versus instructive
There’s no text in the play space, but information panels are placed nearby.
The intuitive nature of the experience makes it more powerful. The more you play, the more you discover, like how the slug can block off the water supply, and the way certain animals migrate if the conditions are right. (My kids noticed more than me.)
And the hosts are always there to help – real people in this unreal world. They’re all pretty young (volunteer college students rather than retirees), so the overall vibe is youthful.
A large computer screen just outside the space gives you an overview of what’s going on.
I wondered whether it should be closer, but we were there in the school holidays, so it wasn’t really being used. In an educational or more mediated session, an individual or group could be assigned to direct, with others taking specific roles in the game. Sessions can also be paused and restarted to allow for discussion. Huge potential.
Michael Cosaboom, Director of Exhibit Services, showed me the back-end interface that controls the game. I was expecting a handful of levels for different audiences, but numerous variables can be customised: availability of water, rate of evaporation, rate at which plants wilt … mindboggling stuff. The higher the number, the harder it is. Take this variable: how many clouds clustered together make a rainstorm. To make the water cycle more visible for novice players and learners, you’d lower the number for that variable and the rain would fall more readily.
Settings are saved, and those settings can then be applied to different groups depending on their level.
The possibilities are endless, and the scripting can happen on the go. A really engaged teacher could potentially adjust the settings to explore specific aspects relevant to their students – feedback loops, for example.
Fantasy versus reality
Part of me wanted more of a human presence in the game. Cities that guzzle water. Cars and factories that pump out fumes. Something more closely resembling our current reality. But simplicity is key and, as a player in the game, you are the human presence.
Michael explained that prototyping (and they did a lot) led them to take the fantasy angle. Initially, the graphics were more realistic, but people brought preconceived notions to the game, expecting the plants and animals to behave in too narrowly defined ways. Making it more fantastical freed them up.
It’s a hugely impressive technical undertaking: 15 projectors rigged above the space (6 for the walls, 2 for the waterfall, 7 for the floor), 12 Kinects to detect gestures, 8 Mac Pros, 3 infrared cameras, a special floor surface, all operating on a closed network.
I was amazed at how stable the technology was, and how well the gesture-recognition worked. Mike confirmed that they’ve had very few glitches. Design I/O have created similar interactives elsewhere, including Funky Forest at Singapore Art Museum, so they’ve had a chance to iron out the kinks. And a key component is that the graphics are code, which means no large bitmap images to shuffle around.
An advisory board and scientists from Columbia University, New York University, and Yale also played vital roles in the development. I can just imagine the difficult conversations and debates. Simplicity versus accuracy. Fiction versus reality. The stuff that makes our jobs fascinating and exasperating all at once.
What was the hardest part? Simplifying the experience according to Michael (as it so often is) so that the key messages could come through: 1) that everything is connected, and 2) that your actions have consequences. Within that, the prevailing sense is still one of discovery and wonder. Impressive.
The only thing that confused me was finding the experience. The museum map doesn’t yet mention it, and the route through is like this …
Right now, Connected Worlds feels like a hidden gem, but that’s intentional in part. There was no opportunity to test its full potential during the school holidays. With summer over, that’s about to change. Whenever possible, Michael is in there with anyone who’ll play, testing the potential of all the variables. I’ll also be fascinated to see what any evaluation reveals.
The word really needs to get out.
Thanks to Seb Chan and his kids for recommending this one.
Six big ticks
Emotional connection, social connection, physicality, less is more, orientation, real stuff