Bitching about Transparent Screens

I’ve been tagged a number of times on Twitter from people are asking me to weigh in on the following comic by beloved Parisian comic artist Boulet.

Since folks are asking (and it warms my robotic heart that you do), here’s my take on this issue. Boulet, this is for you.

Sci-fi serves different masters

Interaction and interface design answers to one set of masters: User feedback sessions, long-term user loyalty, competition, procurement channels, app reviews, security, regulation, product management tradeoffs of custom-built vs. off-the-shelf, and, ideally, how well it helps the user achieve their goals.

But technology in movies and television shows don’t have to answer to any of these things. The cause-and-effect is scripted. It could be the most unusable piece of junk tech in that universe and it will still do exactly what it is supposed to do. Hell, it’s entirely likely that the actor was “interacting” with a blank screen on set and the interface painted on afterward (in “post”). Sci-fi interfaces answer to the masters of story, worldbuilding, and often, spectacle.

I have even interviewed one of the darlings of the FUI world about their artistic motivations, and was told explicitly that they got into the business because they hated having to deal with the pesky constraints of usability. (Don’t bother looking for it, I have not published that interview because I could not see how to do so without lambasting it.) Most of these things are pointedly baroque where usability is a luxury priority.

So for goodness’ sake, get rid of the notion that the interfaces in sci-fi are a model for usability. They are not.

They are technology in narrative

We can understand how they became a trope by looking at things from the makers’ perspective. (In this case “maker” means the people who make the sci-fi.)

thankthemaker.gif

Not this Maker.

Transparent screens provide two major benefits to screen sci-fi makers.

First, they quickly inform the audience that this is a high-tech world, simply because we don’t have transparent screens in our everyday lives. Sci-fi makers have to choose very carefully how many new things they want to introduce and explain to the audience over the course of a show. (A pattern that, in the past, I have called What You Know +1.) No one wants to sit through lengthy exposition about how the world works. We want to get to the action.

buckrogers

With some notable exceptions.

So what mostly gets budgeted-for-reimagining and budgeted-for-explanation in a script are technologies that are a) important to the diegesis or b) pivotal to the plot. The display hardware is rarely, if ever, either. Everything else usually falls to trope, because tropes don’t require pausing the action to explain.

Secondly (and moreover) transparent screens allow a cinematographer to show the on-screen action and the actor’s face simultaneously, giving us both the emotional frame of the shot as well as an advancement of plot. The technology is speculative anyway, why would the cinematographer focus on it? Why cut back and forth from opaque screen to an actor’s face? Better to give audiences a single combined shot that subordinates the interface to the actors’ faces.

minrep-155

We should not get any more bent out of shape for this narrative convention than any of these others.

  • My god, these beings, who, though they lived a long time ago and in a galaxy far, far away look identical to humans! What frozen evolution or panspermia resulted in this?
  • They’re speaking languages that are identical to some on modern Earth! How?
  • Hasn’t anyone noticed the insane coincidence that these characters from the future happen to look exactly like certain modern actors?
  • How are there cameras everywhere that capture these events as they unfold? Who is controlling them? Why aren’t the villains smashing them?
  • Where the hell is that orchestra music coming from?
  • This happens in the future, how are we learning about it here in their past?

The Matter of Believability

It could be, that what we are actually complaining about is not usability, but believability. It may be that the problems of eye strain, privacy, and orientation are so obvious that it takes us out of the story. Breaking immersion is a cardinal sin in narrative. But it’s pretty easy (and fun) to write some simple apologetics to explain away these particular concerns.

eye-strain

Why is eye strain not a problem? Maybe the screens actually do go opaque when seen from a human eye, we just never see them that way because we see them from the POV of the camera.

privacy

Why is privacy not a problem? Maybe the loss of privacy is a feature, not a bug, for the fascist society being depicted; a way to keep citizens in line. Or maybe there is an opaque mode, we just don’t see any scenes where characters send dick pics, or browse porn, and would thereby need it. Or maybe characters have other, opaque devices at home specifically designed for the private stuff.

orientation

Why isn’t orientation a problem? Tech would only require face recognition for such an object to automatically orient itself correctly no matter how it is being picked up or held. The Appel Maman would only present itself downwards to the table if it was broken.

So it’s not a given that transparent screens just won’t work. Admittedly, this is some pretty heavy backworlding. But they could work.

But let’s address the other side of believability. Sci-fi makers are in a continual second-guess dance with their audience’s evolving technological literacy. It may be that Boulet’s cartoon is a bellwether, a signal that non-technological audiences are becoming so familiar with the real-world challenges of this trope that is it time for either some replacement, or some palliative hints as to why the issues he illustrates aren’t actually issues. As audience members—instead of makers—we just have to wait and see.

Sci-fi is not a usability manual.

It never was. If you look to sci-fi for what is “good” design for the real-world, you will cause frustration, maybe suffering, maybe the end of all good in the ’verse. Please see the talk I gave at the Reaktor conference a few years ago for examples, presented in increasing degrees of catastrophe.

I would say—to pointedly use the French—that the “raison d’être” of this site is exactly this. Sci-fi is so pervasive, so spectacular, so “cool,” that designers must build up a skeptical immunity to prevent its undue influence on their work.

I hope you join me on that journey. There’s sci-fi and popcorn in it for everyone.

Jasper’s Music Player

ChildrenofMen-player03

After Jasper tells a white lie to Theo, Miriam, and Kee to get them to escape the advancing gang of Fishes, he returns indoors. To set a mood, he picks up a remote control and presses a button on it while pointing it at a display.

ChildrenofMen-player02

He watches a small transparent square that rests atop some things in a nook. (It’s that decimeter-square, purplish thing on the left of the image, just under the lampshade.) The display initially shows an album queue, with thumbnails of the album covers and two bright words, unreadably small. In response to his button press, the thumbnail for Franco Battiato’s album FLEURs slides from the right to the left. A full song list for the album appears beneath the thumbnail. Then track two, the cover of Ruby Tuesday, begins to play. A small thumbnail to the right of the album cover appears, featuring some white text on a dark background and a cycling, animated border. Theo puts the remote control down, picks up the Quietus box, and walks over to Janice. *sniff*

This small bit of speculative consumer electronics gets around 17 seconds of screen time, but we see enough to consider the design. 

Persistent display

One very nice thing about it is that it is persistently visible. As Marshall McLuhan famously noted, we are simply not equipped with earlids. This means that when music is playing in a space, you can’t really just turn away from it to stop listening. You’ll still hear it. In UX parlance, sound is non-modal.

Yet with digital music players, the visual displays that tell you about what’s being played, or the related interfaces that help you know what you can do with the music are often hidden behind modes. Want to know what that song you can’t stop hearing is? Find your device, wake it up, enter a password, find the app, and even then you may have to root around to find the software to find what you’re looking for.

But a persistent object means that non-modal sound is accompanied by (mostly) non-modal visuals. This little box is always somewhere, glowing, and telling you what’s playing, what just played, and what’s next.

Remote control

Finding the remote is a different problem, of course, and if your household is like my household, it is a thing which seems to want to be lost. To keep that non-modality of sound matched by the controls, it would be better to have the device or the environment know when Jasper is looking at the display, and enable loose gestural or voice controls to control it.

Imagine the scene if he grabs the Quietus box, looks up to the display, and says, “Play…” then pause while he considers his options, and says “…‘Ruby Tuesday’…the Battiato one.” We would have known that his selection has deep personal meaning. If Cuarón wanted to convey that this moment has been planned for a while, Jasper could even have said, “Play her goodbye song.”

Visual layout

The visual design of the display is, like most of the technology, meant to be a peripheral thing, accepting attention but not asking for it. In this sense it works. The text is so small the audience is not tempted to read it. The thumbnails are so small it is only if you already knew the music that it would refresh your memory. But if this was a real product meant to live in the home, I would redesign the display to be usable at the 3–6 meter distance, which would require vastly reducing the number of elements, increasing their size, and perhaps overlaying text on image.

ChildrenofMen-player03

Brain Scanning

The second half of the film is all about retrieving the data from Johnny’s implant without the full set of access codes. Johnny needs to get the data downloaded soon or he will die from the “synaptic seepage” caused by squeezing 320G of data into a system with 160G capacity. The bad guys would prefer to remove his head and cryogenically freeze it, allowing them to take their time over retrieval.

1 of 3: Spider’s Scanners

The implant cable interface won’t allow access to the data without the codes. To bypass this protection requires three increasingly complicated brain scanners, two of them medical systems and the final a LoTek hacking device. Although the implant stores data, not human memories, all of these brain scanners work in the same way as the Non-invasive, “Reading from the brain” interfaces described in Chapter 7 of Make It So.

The first system is owned by Spider, a Newark body modification
specialist. Johnny sits in a chair, with an open metal framework
surrounding his head. There’s a bright strobing light, switching on
and off several times a second.

jm-20-spider-scan-a

Nearby a monitor shows a large rotating image of his head and skull, and three smaller images on the left labelled as Scans 1 to 3. Continue reading

Brain Upload

Once Johnny has installed his motion detector on the door, the brain upload can begin.

3. Building it

Johnny starts by opening his briefcase and removing various components, which he connects together into the complete upload system. Some of the parts are disguised, and the whole sequence is similar to an assassin in a thriller film assembling a gun out of harmless looking pieces.

jm-6-uploader-kit-a

It looks strange today to see a computer system with so many external devices connected by cables. We’ve become accustomed to one piece computing devices with integrated functionality, and keyboards, mice, cameras, printers, and headphones that connect wirelessly.

Cables and other connections are not always considered as interfaces, but “all parts of a thing which enable its use” is the definition according to Chris. In the early to mid 1990s most computer user were well aware of the potential for confusion and frustration in such interfaces. A personal computer could have connections to monitor, keyboard, mouse, modem, CD drive, and joystick – and every single device would use a different type of cable. USB, while not perfect, is one of the greatest ever improvements in user interfaces. Continue reading

Iron Man HUD: 1st person view

In the prior post we catalogued the functions in the Iron HUD. Today we examine the 1st-person display.

When we first see the HUD, Tony is donning the Iron Man mask. Tony asks, “JARVIS, “You there?”” To which JARVIS replies, ““At your service sir.”” Tony tells him to “Engage the heads-up display,” and we see the HUD initialize. It is a dizzying mixture of blue wireframe motion graphics. Some imply system functions, such as the reticle that pinpoints Tony’s eye. Most are small dashboard-like gauges that remain small and in Tony’s peripheral vision while the information is not needed, and become larger and more central when needed. These features are catalogued in another post, but we learn about them through two points-of-view: a first-person view, which shows us what Tony’s sees as if we were there, donning the mask in his stead, and second-person view, which shows us Tony’s face overlaid against a dark background with floating graphics.

This post is about that first-person view. Specifically it’s about the visual design and the four awarenesses it displays.

Avengers-missile-fetching04

In the Augmented Reality chapter of Make It So, I identified four types of awareness seen in the survey for Augmented Reality displays:

  1. Sensor display
  2. Location awareness
  3. Context awareness
  4. Goal awareness

The Iron Man HUD illustrates all four and is a useful framework for describing and critiquing the 1st-person view. Continue reading