Satellite Lamps is a project that reveals one of the most significant contemporary technology infrastructures, the Global Positioning System (GPS). Made with Einar Sneve Martinussen and Jørn Knutsen as part of the Yourban research project at AHO, it continues our project of revealing the materials of technologies that started in 2009 with RFID and WiFi.
GPS is widely used yet it’s invisible and few of us really have any idea of how it works or how it inhabits our everyday environments. We wanted to explore the cultural and material realities of GPS technology, and to develop new understandings about it through design.
“Satellite Lamps shows that GPS is not a seamless blanket of efficient positioning technology; it is a negotiation between radio waves, earth-orbit geometry and the urban environment. GPS is a truly impressive technology, but it also has inherent seams and edges.”
We created a series of lamps that change brightness according to the accuracy of received GPS signals, and when we photograph them as timelapse films, we start to build a picture of how these signals behave in actual urban spaces.
The project is documented in an extensive article that thoroughly details how it was made and why. You can read more on how we explored GPS technology, how the visualisations were made, and about the popular cultural history of GPS.
Posted in Film, Photography, Project, Research, Technology, Urbanism4 Comments on Satellite LampsMy PhD thesis called ‘Making Visible’ was submitted in December 2013 and successfully defended on 12 June 2014. The thesis reflects upon the design material exploration research from the Touch and Yourban projects. It uses these explorations to situate design research with technology as a cultural, material and mediational practice:
In Making Visible I outline how interaction design may engage in the material and mediation of new interface technologies. Drawing upon a design project called Touch, that investigated an emerging interface technology called Radio Frequency Identification or RFID, I show how interaction design research can explore technology through material and mediational approaches. I demonstrate and analyse how this research addresses the inter-related issues of invisibility, seamlessness and materiality that have become central issues in the design of contemporary interfaces. These issues are analysed and developed through three intertwined approaches of research by design: 1. a socio- and techno-cultural approach to understanding emerging technologies, 2. through material exploration and 3. through communication and mediation. When taken together these approaches form a communicative mode of interaction design research that engages directly with the exploration, understanding and discussion of emerging interface technologies.
The thesis is made up of four peer-reviewed journal articles accompanied by a ‘meta-reflection’ document that reflects upon and situates these publications through theory, concepts and models.
This document develops the concept of mediational material that focuses attention on the material and communicative practices in interaction design. These are used to explore, develop and share knowledge of technologies as design materials.
I’ve made it available in a number of different digital formats:
It will also be available through AHO’s open-access archive ADORA.
The four included articles have been published in peer-reviewed journals.
This article takes up the issues of so-called ‘immaterial’ and ‘seamless’ technologies and asks how designers might explore them in order to consider them as design materials. It situates interaction design as a sociocultural practice that is concerned with culture, critical approaches and with engaging the technocultural imagination. It concludes by analysing its mediational strategies, such as the use of documentary formats, online film and weblog writing, and the ways in which new material perspectives have been shared, discussed and developed by others.
This article explores how visual signage may make aspects of ubiquitous computing technologies visible and how digital tools and platforms impact that visual design and semiosis. It looks at a range of ‘identification’ technologies such as barcodes and rfid, that only become ‘visible’ or ‘interactional’ through a designer’s intervention in physical or visual expression. It finds that designers should emphasize the bindings and distinctions between design processes and visual mediations, and symbols and signs, in engaging with emerging technologies as material for creative and communicative composition.
Satellite lamps is a project about using design to investigate and reveal one of the fundamental constructs of the networked city: GPS – the Global Positioning System. It extends the concepts of ‘mediational materials’ to an investigation of the ways in which GPS technology inhabits urban spaces. The article takes up how a discursive and reflexive interaction design practice can contribute to new perspectives on networked city life. Importantly, Satellite Lamps is a multimediational web text, involving different media (film, media, notebooks and a host of images) allowing for the richness of the work to come to the surface in a way that would not have been possible in traditional means of academic publishing.
This article is about the role of film in interaction and product design research with technology, and the use of film in exploring and explaining emerging technologies in multiple contexts. It concludes by looking towards the potentials for a discursive design practice, where the object of design and analysis is the discourse that is catalysed by new artefacts, and the emphasis of design research is on communication.
Posted in Interaction design, Research3 Comments on Making VisibleInternet machine is a multi-screen film about the invisible infrastructures of the internet. The film reveals the hidden materiality of our data by exploring some of the machines through which ‘the cloud’ is transmitted and transformed.
Internet machine (showing now at Big Bang Data or watch the trailer) documents one of the largest, most secure and ‘fault-tolerant’ data-centres in the world, run by Telefonica in Alcalá, Spain. The film explores these hidden architectures with a wide, slowly moving camera. The subtle changes in perspective encourage contemplative reflection on the spaces where internet data and connectivity are being managed.
In this film I wanted to look beyond the childish myth of ‘the cloud’, to investigate what the infrastructures of the internet actually look like. It felt important to be able to see and hear the energy that goes into powering these machines, and the associated systems for securing, cooling and maintaining them.
What we find, after being led through layers of identification and security far higher than any airport, are deafeningly noisy rooms cocooning racks of servers and routers. In these spaces you are buffeted by hot and cold air that blusters through everything.
Server rooms are kept cool through quiet, airy ‘plenary’ corridors that divide the overall space. There are fibre optic connections routed through multiple, redundant, paths across the building. In the labyrinthine corridors of the basement, these cables connect to the wider internet through holes in rough concrete walls.
Power is supplied not only through the mains, but backed up with warm caverns of lead batteries, managed by gently buzzing cabinets of relays and switches.
These are backed up in turn by rows of yellow generators, supplied by diesel storage tanks and contracts with fuel supply companies so that the data centre can run indefinitely until power returns.
The outside of the building is a facade of enormous stainless steel water tanks, containing tens of thousands of litres of cool water, sitting there in case of fire.
And up on the roof, to the sound of birdsong, is a football-pitch sized array of shiny aluminium ‘chillers’ that filter and cool the air going into the building.
In experiencing these machines at work, we start to understand that the internet is not a weightless, immaterial, invisible cloud, and instead to appreciate it as a very distinct physical, architectural and material system.
This was a particularly exciting project, a chance for an ambitious and experimental location shoot in a complex environment. Telefónica were particularly accommodating and allowed unprecedented access to shoot across the entire building, not just in the ‘spectacular’ server rooms. Thirty two locations were shot inside the data centre over the course of two days, followed by five weeks of post-production.
I had to invent some new production methods to create a three-screen installation, based on some techniques I developed over ten years ago. The film was shot using both video and stills, using a panoramic head and a Canon 5D mkIII. The video was shot using the Magic Lantern RAW module on the 5D, while the RAW stills were processed in Lightroom and stitched together using Photoshop and Hugin.
The footage was then converted into 3D scenes using camera mapping techniques, recreating the perspective by hand (a kind of low-tech, traditional photogrammetry) so that entirely new camera movements could be created by animating a virtual three-camera rig within this new virtual space. The final multi-screen installation is played out in 4K projected across three screens.
There are more photos available at Flickr.
Internet machine is part of BIG BANG DATA, open from 9 May 2014 until 26 October 2014 at CCCB (Barcelona) and from February-May 2015 at Fundación Telefónica (Madrid).
Internet Machine is produced by Timo Arnall, Centre de Cultura Contemporània de Barcelona – CCCB, and Fundación Telefónica. Thanks to José Luis de Vicente, Olga Subiros, Cira Pérez and María Paula Baylac.
Posted in Exhibition, Film, Photography, Project, Research, Technology78 Comments on Internet machineThe Immaterials project is concerned with the increasing invisibility of interfaces and infrastructures. The systems we interact with everyday such as WiFi and 3G networks have a profound impact on how we experience the world. As Adam Greenfield says:
the complex technologies the networked city relies upon to produce its effects remain distressingly opaque, even to those exposed to them on a daily basis. […] it’s hard to be appropriately critical and to make sound choices in a world where we don’t understand the objects around us.
And as James Bridle has eloquently and disturbingly observed:
Those who cannot perceive the network cannot act effectively within it, and are powerless.
The project set out to expose some of the phenomena and mechanisms of technological infrastructures through visual, photographic, narrative, animated and cinematic techniques. Over the last five years I have worked with Einar Sneve Martinussen, Jørn Knutsen, Jack Schulze and Matt Jones towards a body of work that is now brought together in an exhibition for the first time.
From 2004–2008 I speculated about the ways in which wireless interactions inhabited physical space, through my work on a Graphic language for touch, and also through films such as Wireless in the world. Some of my students made beautiful but fictional speculations about the physical qualities of different kinds of radio.
Jack Schulze and I also made a short, playful film called Nearness about action at a distance. In the film, a series of simple reactions are set off by immaterial phenomena, such as radio waves, mobile networks, light, magnetism and wind.
In 2006 we ran a Touch workshop with BERG where we became concerned about the invisibility of RFID technology, and the effect that had on our ability to design with it. We found it extraordinary that a technology that was defined as a proximity or ‘touch’-based interface, was so opaque in terms of its physical, spatial, gestural materiality. How do we as designers make these materials visible, so we can have reflective conversations with them?
We developed Experiments in Field Drawing as a method of revealing, literally drawing, the physical presence of RFID interactions. We revealed these fields in a much richer, multi-dimensional way using photography, animation and light painting in the film Immaterials: Ghost in the Field.
Matt Jones coined the term immaterials to describe the project and gave a great talk about some ways of understanding the immaterials of interaction design. Matt and I also looked at machine vision, another phenomena that increasingly becomes a material for design in Robot Readable World.
In 2011 at AHO, as part of a research project called Yourban, we extended the investigations to WiFi, using similar light painting techniques we revealed the enormous scale and pervasiveness of ad-hoc WiFi networks in urban spaces in Immaterials: Light Painting WiFi.
Finally, over the last two years we’ve become increasingly interested in the Global Positioning System (GPS), that has become a central part of both the vision and the implementation of contemporary interfaces.
We have built a series of Satellite Lamps that sense the presence of the 24 GPS satellites in orbit. The lamps change brightness according to the strength of GPS signals they receive, showing how the technology itself is messy and unpredictable, and revealing how GPS is a negotiation between radio waves, earth-orbit geometry and the urban environment.
Satellite Lamps has so far been exhibited at Lighthouse and Dread, you can watch the film and read the extensive article detailing our process as well as a cultural history of GPS.
The visual languages that we’ve developed have ended up in advertising, on the BBC and Discovery Channel, and the techniques have been extended in research at MIT and CIID, and by many designers, enthusiasts and hackers. It’s exciting that both the subject and the methods are being taken up and used broadly by other people, and we’re looking forward to seeing more.
the truly pressing need is for translators: people capable of opening these occult systems up, demystifying them, explaining their implications to the people whose neighborhoods and choices and very lives are increasingly conditioned by them. — Adam Greenfield (2009)
The Immaterials project emerged from the humble preoccupations of a few designers dealing with some of the invisible, immaterial, intangible stuff we had in front of us. These small experiments led to larger and more visually and narratively communicative work. In the end what I think we’ve developed is an approach to technology that revolves around material exploration, explanation and communication. Because images and language, as well as materials, form our understandings of technology, Immaterials has shown how we can use ‘design and playful explorations to shape or stir the popular imagination’.
All the Immaterials projects are on display at Lighthouse in Brighton from 5 September until 13 October 2013.
Satellite Lamps and Robot Readable World are on display at Dread in Amsterdam from 7 September until 24 November 2013.
Posted in Interaction design, Research11 Comments on The Immaterials project‘The best design is invisible‘ is the interaction design phrase of the moment. The images above are from my ever-expanding collection of quotes about how design and technology will ‘disappear‘, become ‘invisible‘ or how the ‘best interface is no interface‘.
The Verge has recently given both Oliver Reichenstein and Golden Krishna a platform to talk about this. This has spawned manifestos, films, talks, books, #NoUI hashtags and some debates about what it might mean. I’ll call this cluster of things ‘invisible design’.
I agree with some of the reasons driving this movement; that design’s current infatuation with touchscreens is really problematic. I’ve spent the last eight years rallying against glowing rectangles, studying our obsession with screens and the ways in which this has become a cultural phenomena. In response I have been researching and inventing interfaces for taking interaction out from under the glass.
But I also take issue with much of the thinking for a few reasons that I’ll outline below.
We already have plenty of thinking that celebrates the invisibility and seamlessness of technology. We are overloaded with childish mythologies like ‘the cloud’; a soft, fuzzy metaphor for enormous infrastructural projects of undersea cables and power-hungry data farms. This mythology can be harmful and is often just plain wrong. Networks go down, hard disks fail, sensors fail to sense, processors overheat and batteries die.
Computing systems are suffused through and through with the constraints of their materiality. – Jean-François Blanchette
Invisible design propogates the myth that technology will ‘disappear’ or ‘just get out of the way’ rather than addressing the qualities of interface technologies that can make them difficult or delightful.
Intentionally hiding the phenomena and materiality of interfaces, smoothing over the natural edges, seams and transitions that constitute all technical systems, entails a loss of understanding and agency for both designers and users of computing. Lack of understanding leads to uncertainty and folk-theories that hinder our ability to use technical systems, and clouds the critique of technological developments.
As systems increasingly record our personal activity and data, invisibility is exactly the wrong model.
By removing our knowledge of the glue that holds the systems that make up the infrastructure together, it becomes much more difficult, if not impossible, to begin to understand how we are constructed as subjects, what types of systems are brought into place (legal, technical, social, etc.) and where the possibilities for transformation exist. – Matt Ratto (2007)
In other words, as both users and designers of interface technology, we are disenfranchised by the concepts of invisibility and disappearance.
The movement tells us to ‘embrace natural processes’ and talks about the ‘incredibly intuitive’ Mercedes car interface. This language is a trap (we should ban the use of natural and intuitive btw) that doesn’t give us any insight into how complex products might actually become simple or familiar.
Invisible design leads us towards the horrors of Reality Clippy. Does my refrigerator light really go off? Why was my car unlocked this morning? How did my phone go silent all of a sudden? Without highly legible systems for managing and understanding all of this ‘smartness’ we are going to get very lost and highly frustrated. The tricky business of push notifications and the Facebook privacy train wreck is just the tip of the iceberg.
The example of the Nest thermostat invisibly ‘learning’ your habits to control your home temperature is a good one. But the Nest has a highly visible interface that reassures you as to its status, tells you when it is learning, and a large dial for adjusting temperature. Beautiful, legible microinteractions. A Nest without these visual and direct manipulation interfaces would be useless, uncanny and frustrating. Nest wants UI.
The discussion around invisible design often talks about using sensors and tangible interfaces instead of visual interfaces. But these systems are not inherently simpler or more familiar. They have their own material qualities with edges and ‘grain’ that need to be understood and learnt. Their literal invisibility can cause confusion, even fear, and they often increase unpredictability and failure.
In our work with interface technologies such as RFID and computer vision, we’ve discovered that it takes a lot of work to make sense of the technologies as design materials. So it’s not useful to say that UI is ‘disappearing’ into sensing, algorithms and tangible interfaces, when we don’t fully understand them as UI yet.
Interfaces are the dominant cultural form of our time. So much of contemporary culture takes place through interfaces and inside UI. Interfaces are part of cultural expression and participation, skeuomorphism is evidence that interfaces are more than chrome around content, and more than tools to solve problems. To declare interfaces ‘invisible’ is to deny them a cultural form or medium. Could we say ‘the best TV is no TV’, the ‘best typography is no typography’ or ‘the best buildings are no architecture’?
Much of our work at BERG is not just about solving problems, but about cultural invention:
We’re not interested in this idea of the invisible technology in a modernist sense. Tech won’t be visible but only if it’s embedded into the culture that it exists within. By foregrounding the culture, you background the technology. It’s the difference between grinding your way through menus on an old Nokia, trying to do something very simple, and inhabiting the bright bouncy bubbly universe of iOS. The technology is there, of course, but it’s effectively invisible as the culture is foregrounded.” – Jack Schulze (in Domus 965 / January 2013)
We should be able to simultaneously celebrate the fantastic explosion of diversity in UI, and develop healthy critique around the use of interfaces like touch screens. But by calling for UI to disappear altogether so that things can be more efficient, we remain in the same utilitarian and rational mindset that produces inert technological visions like this, rather than seeing interfaces as part of the cultural landscape.
The movement ignores at least thirty years of thinking in design and technology. A few examples:
Much of the recent invisible design discussions repeat the thinking in Jared Spool’s ‘Great Designs Should Be Experienced and Not Seen‘ and Donald Norman’s ‘Invisible Computer. But a better reference point would be Don Norman’s earlier book, The design of everyday things, where he instead talks about the ‘problems caused by inadequate attention to visibility’ and supporting or managing our mental models of systems. We need a lot more thinking about our mental models of algorithms in particular.
Adam Greenfield has investigated the social and ethical issues around the development of ubiquitous computing systems, and is particularly concerned by its disappearance:
“Ubiquitous systems must contain provisions for immediate and transparent querying of their ownership, use, and capabilities. Everyware must, in other words, be self-disclosing. Whether such disclosures are made graphically, or otherwise, they ensure that you are empowered to make informed decisions as to the level of exposure you wish to entertain.” – Adam Greenfield (2006)
Some designers have talked about the actual qualities they want from ubiquitous computing interfaces, such as polite, pertinent and pretty:
“The vast quantities of information that personal informatics generate need not only to be clear and understandable to create legibility and literacy in this new world, but I’d argue in this first wave also seductive, in order to encourage play, trial and adoption” Matt Jones & Tom Coates (2008)
Matthew Chalmers has, more than anyone else, revealed the history of seamlessness. Seamlessness is ‘the deliberate “making invisible” of the variety of technical systems, artifacts, individuals and organizations that make up an information infrastructure. This work actively disguises the moments of transition and boundary crossing between these various parts in order to present a solid and seemingly coherent interface to users.’ (Ratto 2007). Although Mark Weiser is often thought of as an advocate of seamless systems, Chalmers found that:
Weiser describes seamlessness as a misleading or misguided concept. In his invited talks to UIST94 and USENIX95 he suggested that making things seamless amounts to making everything the same, reducing components, tools and systems to their ‘lowest common denominator’. He advocated seamful systems (with “beautiful seams”) as a goal. Around Xerox PARC, where many researchers worked on document tools, Weiser used an example of seamful integration of a paint tool and a text editor (Weiser, personal communication). He complained that seamless integration of such tools often meant that the user was forced to use only one of them. One tool would be chosen as primary and the others reduced and simplified to conform to it, or they would be crudely patched together with ugly seams. Seamfully integrated tools would maintain the unique characteristics of each tool, through transformations that retained their individual characteristics. This would let the user brush some characters with the paint tool in some artful way, then use the text editor to ‘search and replace’ some of the brushstroked characters, and then paint over the result with colour washes. Interaction would be seamless as the features of each tool were “literally visible, effectively invisible”. Seamful integration is hard, but the quality of interaction can be improved if we let each tool ‘be itself’. – Matthew Chalmers (2003)
Matt Ratto investigates the darker side of this drive towards invisibility, revealing that seamlessness encourages:
“a particular kind of passivity and lack of engagement between people and their actions and between people and their social and material environment” and that we must “critique the clean, orderly, and homogenous future that is at the heart of these modernist visions” – Matt Ratto (2007)
And Anne Galloway suggests that it is in the seams where the design work can be done:
“Although seamlessness may remain a powerful and effective metaphor to guide particular projects, when it comes to actually getting the work done—and the challenges of having to do it with people who can be very different from each other—then I suggest it is in everyone’s best interests to recognise the importance of seams and scars in marking places where interventions can be made, or where potential can be found and acted upon.” – Anne Galloway (2007)
In interaction design we need to look at the long history of Durrell Bishop‘s work, one of the strongest advocates for self-evident design, whether it is physical or virtual, through his teaching and design practice. Durrell’s ‘Platform 12’ in the RCA Design Products course attempts to see design as:
“a celebration of a model for how things work, where once again we can treat function as beauty, instead of merely treating design as form and image.”
Durrell’s work on the Marble Answering Machine (1992) is a brilliant piece of self-evident design, and remains a touchstone for all interaction design work.
Designers also need to look at the first four chapters of ‘Where the action is‘ by Paul Dourish which give a coherent account of the relationships between human abilities and computer interfaces over the last 50-60 years. Dourish shows how interfaces are not becoming invisible, but how they are increasingly social and tangible.
And finally, from a design perspective, there is a long tradition of making complex products legible and understandable. Industrial designer Konstantin Grcic talks about the relationship between the technologies and the use of an object:
“A machine is beautiful when it’s legible, when its form describes how it works. It isn’t simply a matter of covering the technical components with an outer skin, but finding the correct balance between the architecture of the machine… and an expressive approach that is born out of the idea of interaction with those using the object.” – Konstantin Grcic (2007)
And perhaps more famously, Dieter Rams has always talked of honesty and understanding in his product design practice. Making a product understandable is one of his Ten Principles of “Good Design”.
This drive for understanding needs to go further than physical form (as it has done at Apple) and start to inform the design of systems and UI.
We must abandon invisibility as a goal for interfaces; it’s misleading, unhelpful and ultimately dishonest. It unleashes so much potential for unusable, harmful and frustrating interfaces, and systems that gradually erode users and designers agency. Invisibility might seem an attractive concept at first glance, but it ignores the real, thorny, difficult issues of designing and using complex interfaces and systems.
We might be better off instead taking our language from typography, and for instance talk about legibility and readability without denying that typography can call attention to itself in beautiful and spectacular ways. Our goal should be to ‘place as much control as possible in the hands of the end-user by making interfaces evident‘.
Of course the interfaces we design may become normalised in use, effectively invisible over time, but that will only happen if we design them to be legible, readable, understandable and to foreground culture over technology. To build trust and confidence in an interface in the first place, enough that it can comfortably recede into the background.
Posted in Interaction design, Ubicomp124 Comments on No to NoUIThe film Immaterials: Light painting WiFi made with Einar Sneve Martinussen and Jørn Knutsen as part of the Yourban project is a finalist at the Vimeo Awards 2012.
The awards ceremony is on the 7 June at 8PM in New York City. The Vimeo Festival also looks great!
Posted in FilmThis is a short film, an experiment in machine-vision footage. It uses found-footage from computer vision research to explore how machines are making sense of the world.
As robots begin to inhabit the world alongside us, how do they see and gather meaning from our streets, cities, media and from us? Machines have a tiny, fractional view of our environment, that sometimes echoes our own human vision and often doesn’t.
Read more about the film and have a look at Matt Jones’ talk of the same title.
Posted in Research5 Comments on Robot readable worldI have five works in MoMA‘s latest exhibition ‘Talk to Me‘ in New York. The works are Nearness, Immaterials, Media Surfaces, The Journey and Suwappu.
I have written more about the exhibition and the works at the Touch and BERG weblogs. The exhibition has also been reviewed by CNN, the New York Times, Fast Company and the Wall Street Journal amongst others.
Posted in Film, Interaction designI’ve been quite taken with the films of Adam Lisagor for a while.
I make small, palatable videos, like commercials, for companies involved in tech, to figure out how to convey the essence of their products in concise, accessible ways.
I like to think that I’m able to do this because I think slowly enough to notice the exact points while using a product at which I respond with the most delight. And if I can reproduce those moments on-screen, without explicitly saying that they’re delightful, an audience will intuitively understand the delight they might feel themselves.
He’s good at surfacing the joy and pleasure in some of the smallest interactions, particularly evident in this ad for the Jambox by Jawbone.
Posted in Film, videoA truth that releases a waterfall of emotion. It is this energy that propels us through The Tree of Life. A voluptuous, bulging energy shaped and encouraged by sweeping camera movement, ultra wide lenses, lyrical blocking, the safe-harbor of Jessica Chastain’s face, and the vacillation in Hunter McCracken’s. These combine to create scenes that perfectly capture the rapturous feelings of childhood. Sensations evoked when light & dark entwine, and our instinctual knowledge that these things are the same.
And on how to approach the film:
A moment long enough for me to relax, and I was suddenly taken by a feeling of great tenderness and calm. I don’t completely understand why I felt this, but the inclusion of these CGI dinosaurs struck me as an particularly affectionate and loving decision. Terrence Malick believes in his audiences, and has faith that we also can believe. It’s the feeling of your mother brushing the hair off your forehead as she tells you a bedtime story. You protest because she’s changed a part of the usual tale, or it’s not the way you want it to be, but smiling, she says “Shhh shhh. Just listen.”
From the brilliant Kartina Richardson.
Posted in Film, Photography