Future Interfaces, Science Fiction and Idea Transmission — Can Our Imagination Outpace Reality?
Design of the Times
VR is a short acronym with a lot of history, a lot of baggage and a lot of dreams, be they manifested or broken. Whether it comes from its constant dance with the military-industrial complex; broken promises of holodecks and anti-gravity skateboards from ’90s futurism; the many dystopian visions afforded us by Lynch (Wild Palms), Cronenberg (eXistenz) and Leonard (Lawnmower Man), Tad Williams’ Otherland among countless others, or the entries that assume that it will come and form the foundation of our virtual beingness like Stephenson’s Snow Crash, Vinge’s Rainbow’s End, Kline’s Ready Player One and Jeff Noon’s VURT, William Shatner’s TV series TekWar (1994)** or Kathryn Bigelow’s Strange Days (1995), the exploration of life with VR never comes in moderation. It is depicted as a pandemic, a soma, a panacea, a global construct.
As we see it appear more frequently and in larger-scale productions, what is so fascinating is the freedom and scope of imagination required to stay ahead of the manifest reality. From Altered Carbon and Philip K. Dick’s Electric Dreams to Black Mirror and Star Trek, Virtual and Augmented Reality are increasingly depicted in high quality, big budget mainstream fare.
The other day (Q1 2020), I was at a projection mapping enthusiasts’ meetup in Los Angeles and ran into Todd A. Marks from imagesonscreen.com, who for over a quarter-century has been handling customized, story-specific playback graphics that are designed to mesh with the visual style of the project, while providing an authentic “look and feel”. This has led to managing and working with collaborators on the computer screen UIs for films like The Net, Deep Impact, Soderbergh’s Solaris, Anchorman 2 and its 1970’s/80’s CRT monitors and broadcast bays, Constantine, Venom and Star Trek: Nemesis among many others. The conversation led to advancements in playback technology, like moving from CRTs stowed behind smoked plexiglass on the USS Enterprise to the argument to use LCD monitors which were flat and could be placed flush with the console. But more importantly, we discussed the challenges and creativity required to devise what is actually on the screen.
This serendipity was kind of amazing because the YouTube algo had just thrown before me a takedown of Hackers in Movies a few days prior. So I could tell I was in a flow state freq.
Todd told me “One of the things that I do on a film/production is help introduce them to new display technologies and help them think about different methods to interact and interface with them.”
Mise En Place
Though it is easy to do a takedown on of the realism of computer screens operated by movie hackers as opposed to real-life Linux consoles, the audience needs to understand what is actually being explained in a visual manner that advances the narrative. Marks calls it VIPAL: “Visually Interesting, Plausible and Logical.”
I found this notion exhilarating. How do you strike the balance between reality and fiction? Between getting the point across and being literal. Heck, the dripping green digits in The Matrix — that presumably contained all the code for reality as we know it — turned out to be little more than a Japanese food menu. How can you implement the ideal narrative interface in situ in immersive contexts? This will require substantial play and experimentation, a liberation of the imagination and the capability to deploy it smoothly.
Vox.com took a look at this ideation and production pipeline in the following video:
Get On Board
About a year ago, I took my lifelong casual interest in board game design much more seriously. I spend a lot of time on my own in Hollywood, as many do, and was tired of looking at the social media scrolling down my 6-inch screen. I wanted to give my eyes a break, do something with my hands, get my ideas on the table in front of me, analyze game mechanics, systems, UI and the solo automata for running campaigns with paper and dice “AI.” It has proven to be an exceedingly rewarding and educational journey.
A board game is an interesting conundrum — immutable in its organic form of printed and molded materials, yet constantly interpreted, reinterpreted, modulated and reconsidered by its users. The ideal is color-blind-proof, language-proof, generation-proof, easy to pick up and organize, easy to set up and tear down, quick to learn, but rich in emergent play and complexity and thematic by design, even when abstract. Something that can be understood by sight, touch, voice, silence, relational dynamics, deduction and risk-taking.
What does this have to do with computer screens in movies? How we interface with data is more than just design and science, more than just practicality and narrative. It is a shortcut for the transmission of complex ideas.
Transmission Efficiency
Just as a story is highly efficient means of transmitting complex ideas, history, context, an interface is doing so to access mechanisms that can assist anyone (unless it is designed of course to obfuscate its applications to all but those granted access), as a point of leverage to influence larger more complex systems using logical switches, symbolic visual or tactile feedback and input methods, and interconnected control schemes.
How can we use these examinations to design better-spatialized interfaces--what I call diegetic UI — in other words, things that feel intrinsic to their contexts and as readily available as the same? How can we address and include from the design phase the myriad considerations for accessibility, a key consideration that can and will become increasingly mainstream in the coming days, months, years. Something that begs for guidelines, best practices and even SDKs for those not typically versed with these considerations and design challenges, and these should come from those who know — and this underscores the point about how to think beyond what we know, and assume, in interface design.
Thematically, this holds whether we are figuring out the scanners on an X-Wing Fighter that was designed in the 1970s for a movie filming 40 years later, to the new futuristic worlds on Star Trek series being made in 2020 like Picard. How can we retain the style and cultural effects of an interface while also considering its longevity and ability to transcend linguistic frameworks, all the while managing the widest subset of possible scenarios and data? How do they help us to tell so much story through these design considerations while still remaining plausible from a storytelling perspective and usable from a design perspective that can transcend time and culture?
Likely they can’t, and that reveals another thing: that these interface also encode our understanding of the world within which they are operating from the time in which they were created. Not even Syd Mead’s vision of a future Los Angeles, remained tethered to the reality for more than 20 years, though many of his ideas influenced and informed the interfaces that did come to be.
Digital Divide
In the immersive industry this question looms larger than ever as the very nature of its interface is under design. Some interesting experiments are being done now that Leap Motion’s mature hand tracking technology is now essentially incorporated into the very popular and tetherless Oculus Quest head mounted display device.
Behold just some of the early experiments taking place in the side-loaded, handtracking gallery at SideQuest
When we can become an octopus, or a bird in VR, what is the interface for that? When we are disembodied thought clouds, what is the appropriate interface? How does it work? What does it control? What makes it intuitive? How does it help to understand the world, the mechanism, the constraints and the potential of the system?
I will leave it as food for thought: How do we free our imagination and expand it to contain not only the data the future will provide us, but the new ways we’ll engage with it? What will the practical and aesthetic, communicative and symbolic or pan-linguistic, accessibility-cognizant considerations inform about their nature, and about who we are going to become?
**full disclosure: in the series I play the role of an underground VR hacker/dealer named Mustapha. My short stint on this 1990s series belies how possible future becomes alternate past so very quickly.
Special thanks to Matthew Johnson for his assistance with this article. The header image is a publicity photo from the 1995 film Hackers.
This story originally appeared at the author’s company site ConstantChangeMedia.com and has been revised and expanded for Medium:
