Interfaces

Design / Prototype / Code

This portfolio is a glimpse at the way I design and prototype in code. Design something familiar, program it to do something unexpected, make sure people feel something.

User interfaces (UIs) are the connections between us — soft, adaptable humans — and the hard, deterministic world of data and machines.

Historically, these interfaces have been rigid, representing the deterministic logic of the machine and relying on the user to translate the hard logic into a soft, meaningful form. With the rise of LLM capabilities, our technology is newly better at acting soft and squishy. Which brings the question: how should our interfaces handle the transition from the hard logic of machines into the soft logic of humans?

...Hopefully this framework of segmenting at different scales or gradually transitioning across different dimensions can help organize our thinking.

This book celebrates escapes from flatland, rendering several hundred superb displays of complex data. Revealed here are design strategies for enhancing the dimensionality and density of portrayals of information—techniques exemplified in maps, the manuscripts of Galileo, timetables, notation describing dance movements, aerial photographs, the Vietnam Veterans Memorial, electrocardiograms, drawings of Calder and Klee, computer visualizations, and a textbook of Euclid's geometry.

Our investigation yields general principles that have specific visual consequences, governing the design, editing, analysis, and critique of data representations. These principles help to identify and to explain design excellence—why some displays are better than others.

On the visual weariness of the web.

95% of the information on the web is written language. It is only logical to say that a web designer should get good training in the main discipline of shaping written information, in other words: Typography.

By highlighting text and “starring” your selection, you can create a personal marker to a passage.

What you save is stored only on your specific browser locally, and is never sent to the server. Other visitors will not see your highlights, and you will not see your previously saved highlights when visiting the site through a different browser.

Unfolding functionality
MouseOver
MouseDragging instead of Value-Slider
Memory Dots / Five Favorites
Transparency & Shadows
Full Screen Mode, Rooms Metaphor
Workspace – Desktop
MetaWindow

A name retroactively applied to a landmark computer demonstration, presented by Douglas Engelbart on December 9, 1968. The 90-minute presentation essentially demonstrated almost all the fundamental elements of modern personal computing:

• windows,
• hypertext,
• graphics,
• efficient navigation and command input,
• video conferencing,
• the computer mouse,
• word processing,
• dynamic file linking,
• revision control,
• and a collaborative real-time editor

Two studs wide and angled at 45°, the ubiquitous “2x2 decorated slope” is a LEGO minifigure’s interface to the world. These iconic, low-resolution designs are the perfect tool to learn the basics of physical interface design.

Let me argue that the actual dawn of user interface design first happened when computer designers finally noticed, not just that end users had functioning minds, but that a better understanding of how those minds worked could completely shift the paradigm of interaction.

Even among front-end developers, few people actually grok this mental model of a user. It's why most React(-like) apps in the wild are spaghetti, and why most blog posts about React gripes continue to miss the bigger picture. Doing React (and UI) well requires you to unlearn old habits and actually design your types and data flow so it uses potentially invalid input as its single source of truth. That way, a one-way data flow can enforce the necessary constraints on the fly.

The way Stanley likes to encode and mutate his data is how programmers think about their own program: it should be bug-free and not crash. The mistake is to think that this should also apply to any sort of creative process that program is meant to enable. It would be like making an IDE that only allows you to save a file if the code compiles and passes all the tests.

Stripe partners with millions of the world’s most innovative businesses. These businesses are the result of many different inputs. Perhaps the most important ingredient is “ideas.”

Stripe Press highlights ideas that we think can be broadly useful. Some books contain entirely new material, some are collections of existing work reimagined, and others are republications of previous works that have remained relevant over time or have renewed relevance today.

Adopting a new app is the only way for most people to get new interfaces and features into their personal computing environments. This presents users with a funny challenge: surveying pre-made apps for the ones that, hopefully, match the way they think and work. But this is a surprisingly arduous process because apps often operate on their own silo of data, and they present a new, separate environment which must be learned and adopted. When that doesn’t work, people are stuck trying to use software that doesn’t think or work the way they do.

Alexander has been exploring the concept of an “itemized” OS, one that allows users to construct their own environment. In such an OS all of your digital things — emails, todos, notes, reminders, webpages, podcast episodes, and so forth — are “items” that sit in one, local graph. Users are then able to create new interfaces with the items that matter in their lives and work, or introduce items that bring new functionality to their overall environment (such as having a reminder item which can be added to any other item).

There are some new apps appearing that offer alternative ways of browsing the web...This canvas-based approach adds spatial dimension to the web browsing experience; they allow us to arrange browser windows above, below, to the left, and right of other browser windows.

The same way we're able to put an open book next to a piece of paper and below a row of sticky notes in meatspace. Arranging objects in space to create groupings, indicate relationships, and build hierarchies is one of those classical human skills that never goes out of style.

Great software starts on the command line. Warp is the terminal reimagined with AI and collaborative tools for better productivity.

A worst-practice UI experiment.

What would it look like to support discussion around edits in a very low-formality way, without any need for branching layers or suggestion mode? This prototype was our attempt. The key ideas are:

1. Contributors just edit directly on the doc. No branches or drafts, no suggested changes.
2. After an edit session, you can retroactively supply rationale for particular edits, including grouping related edits together.
3. The primary author of the doc doesn’t need to approve or reject edits. But the rationale and grouping, plus the diff view, makes it easy to see what changed and revert things they don’t agree with.

Although groups are often spatially colocated (for example, a section rewritten) they might also be spread out across the document (for example, a term search-and-replaced, or addition of Oxford commas throughout the doc).

From using this prototype, we’ve found that seeing recent edits and being able to easily revert them feels very useful—it’s a form of undo that’s not limited to the time order that edits happened.

There's more room for spatial concepts to become part of our web browsing experience.

One example is an idea I call "trails." It's based on the story of Hansel and Gretel walking through the forest and leaving a trail of breadcrumbs behind them, so that they could find their way back later. What if you could do this on the web?

A breadcrumb in this case is a single pixel that you can place in a precise location on a webpage. Placing a breadcrumb could be as simple as Option + click. While navigating the web, you could leave breadcrumbs on different pages you find interesting over the course of a browsing session. When you're done, that sequential "trail of breadcrumbs" would be saved. You could then jump back into the trail and navigate "forward" and "backward" through the things you found interesting in that browsing session. Or share the trail with a friend, and they could step through your spatial path of navigating the web.

If you have ever encountered a piece of highly domain-specific business software, you may have noticed that it was largely a graphical front-end to some relational database. You may also, in fact, have avoided using the system at all—studies show that information workers prefer to dump their data into spreadsheets, a general and more familiar tool which, unfortunately, is poorly suited for many standard database tasks. It is time that we stop streamlining the process of creating a new application for every schema, and that we instead develop the visual query languages that will let endusers access the full power of relational database management systems from a simple and unified interface. Once information workers can create, manage, and query real databases with the same ease as they routinely manipulate spreadsheets today, they will never return to their schema-dependent, consultant-made, and oddly-colored Microsoft Access applications.

Sound holds an immense power to elevate any experience—including the most boring of software.

Sound in software isn’t inherently bad. It’s just been really badly designed.

We use sound in every !Boring app, and many have called it out as one of their favorite aspects of our apps. We’ve learned a few things about when to use sound, how to design it, and how to implement it. When done right, sound unlocks a path to much richer software experiences

In the demo, an AI bot for a style guide makes changes as another collaborator in the document. It puts changes on a branch, which you can choose to partially or completely merge—just like you would suggestions from a human coauthor.

The changes are highlighted in a diff to help you review. Once you accept edits from a bot, the history timeline also shows which edits came from the bot.

One more thing: since the bots are simply text prompts, you can create your own by adding a bot prompt into your document collection!

Bots can be shared, edited, and versioned just like regular documents—for example, you can view the history of a prompt, or use a branch to try a new variant of a prompt.

Find everything humans know about...

Dynamic history is thus:

• Auto-save every character typed into a doc
• Offer flexible views of the history depending on the task
• Allow users to mark important milestones

It is fun to revisit memories this way, a digital stamp in my weather passport, where everything can be contained in a forecast and Stockholm sits between Vilnius and London by sheer chance. It has also been a way to feel close to people I love while traveling, to know whether it is raining where they are.

As with most technology, this is artistry by committee. There is no Thomas Cole waiting in the wings. But someone has to animate the stars, to decide when to streak one across the screen–to play god in our pockets.

When I was a product designer, people would ask what I did for a living, and sometimes I’d answer “I draw pictures of websites.”

Sure, I could just say “I design websites.” That’s true. The end result of my work is (hopefully) that a website looks better, works better, or results in better outcomes.

But most of my day isn’t spent looking at the website, or working on the code of the website, or manipulating the website directly in some way. It’s spent in Figma or Sketch, drawing pictures of how I think the website should look and work.

Through some kind of alchemy, the pictures I draw have an impact on the finished website. But they’re not all the same.

So we all know Super Mario, right? And if you think about when you’re collecting coins in Super Mario, it doesn’t stop the game and pop up an alert dialogue and say, “You have just collected ten points, OK, Cancel”, right? It just does it. It does it in the background, but it does provide you with a feedback mechanism.

The feedback you get in Super Mario is about the number of points you’ve just gained. When you collect an item that gives you more points, the number of points you’ve gained appears where the item was …and then drifts upwards as it disappears. It’s unobtrusive enough that it won’t distract you from the gameplay you’re concentrating on but it gives you the reassurance that, yes, you have just gained points.

Currently, dev teams need to spend time implementing inline comments specifically for their app, even though it’s not a unique differentiator. What if instead there was a universal comments layer across all apps, to save developers work and give users a more uniform experience?

...What’s unique to each app is the things you can comment on: a range of cells in a spreadsheet, a span of text in a Markdown doc, a set of shapes in a drawing. In our system, we allow developers to describe the things that can be commented on in their specific app using the concept of pointers.

Pointers can refer to part of a spreadsheet, a Markdown doc, or a drawing.

Once the developer has specified a pointer type for a given type of data, all that’s left is to implement a few simple UI affordances to integrate with the Patchwork commenting UI.

...[And] beyond their usefulness for comments, we’ve found that pointers are a useful primitive in other contexts.

One of the great failures of modern computing is how it has largely ignored the presence of the human body beyond the slightest acknowledgement that humans have a pair of eyeballs and a few fingertips.

...Consider, for instance, our hands. Human hands are capable of so much more than typing and swiping. Human finger sensitivity is astonishingly precise, capable of detecting wrinkles on a smooth surface with a wavelengths as small as 10nm, features so small they are invisible to the eye. Hands can both manipulate and communicate, have effect on the world and perceive its most minuscule features.

Enthusiasm around large language models (LLMs) and text-to-image systems, however, has made it easy to forget about the impressive facility of our hands and the astonishing capabilities of our vision systems, leading instead to proclamations about the pending death of the graphical user interface as an input system. Instead of veering towards a future where computers make greater use of our bodies’ expressiveness, LLMs compel us to diverge further from such a course. Why manipulate anything on a computer with gesture or point and click when you can type or speak?

Students traditionally learn art and design by studying the masters, analyzing, sketching and interpreting the grand visions of the past. In doing this, they get to understand the ideas, concepts and motivations behind the visual form.

In user interface design, this practice is curiously absent.

For others, perhaps especially those artists who worked with light and transparency and were involved in the birth of the Light and Space Movement, an immaculate surface is a prerequisite. Helen Pashgian explained this very clearly:

“On any of these works, if there is a scratch... that’s all you see. The point of it is not the finish at all – the point is being able to interact with the piece, whether it is inside or outside, to see into it, to see through it, to relate to it in those ways. But that’s why we need to deal with the finish, so we can deal with the piece on a much deeper level”.

The importance of a pristine surface calls for a very low tolerance to damage by the artists. The feeling is shared by Larry Bell:

“I don’t want you to see stains on the glass. I don’t want you to see fingerprints on the glass... I don’t want you to see anything except the light that’s reflected, absorbed, or transmitted”

That computers should be easy to learn and use is a rarely-questioned tenet of user interface design. But what do we gain from prioritising usability and learnability, and what do we lose? I explore how simplicity is not an inevitable truth of user interface design, but rather contingent on a series of events in the evolution of software. Not only does a rigid adherence to this doctrine place an artificial ceiling on the power and flexibility of software, but it is also culturally relative, privileging certain information cultures over others. I propose that for feature-rich software, negotiated complexity is a better target than simplicity, and we must revisit the ill-regarded relationship between learning, documentation, and software.

Software applications can utilize spatial interfaces to afford users powerful ways of thinking and interacting. Though often associated with gaming, spatial interfaces can be useful in any kind of software, even in less obvious domains like productivity tools or work applications. We will see spatial interfaces move into all verticals, starting with game-like interfaces for all kinds of social use-cases.

• the right interface can have an incredible power

  • making something that was previously extremely hard, available to everyone (switching from Roman to Arabic numerals as an interface to mathematics)
  • making invention possible (Feynman Diagrams)
  • making Solving Things Visually possible (Cartesian Coordinates)
    • Data Visualization can be thought of as an alternative, sometimes more powerful, interface to data
  • working with the amounts of data for which we don't have enough Pixel Space (Collection-Browsing Interfaces)

• new powerful notations "stick around" and infect human thinking for generations

Sometimes, when you don’t work on a feature idea right away, it loses steam and fades away. Yet, the idea of multi-edit remained surprisingly popular at Figma.

...Multi-edit is the feature which has had the longest gap between initial idea and launch. Over those years, we kept picking it up and putting it down, and we kept polishing and iterating on it. It feels like a rock that we have been tumbling in a rock tumbler until it is shiny and smooth.

One of the odd things about this process is that as we refined multi-edit, it almost started fading away. We’ve been living with multi-edit internally for months now, and we’ve kind of stopped noticing it—until we switch back to the production environment, and we realize how inefficient things were before!

I realized that — you know, that the mouse is antisocial. The GUI is antisocial. So what’s that mean? You have a problem to solve, and you solve it with the GUI. What do you have? A problem solved.

But when you solve it with a command line interface in a scripting environment, you have an artifact. And all of a sudden, that artifact can be shared with someone.

By the way, the way you did it can show cleverness. I’ve never seen anybody use a GUI in a clever way. Ever. There’s no cleverness to it. No, like, “Oh my god, you should see the way Adam clicked that mouse. Oh my god. Guys, guys, guys, guys, come on! Check it out: Adam’s going to click the button! Oh my god! That’s amazing!” It just doesn’t happen.

I’ve found myself looking at architecture to pick up hints on designing multiplayer apps. Or rather: the coming ecosystem of multiplayer apps.

The web is social in lots of ways. Zoom calls are high-bandwidth group video; Google Docs are low-bandwidth group presence and chat. Discord has at-my-own-pace group interaction and live voice channels.

These islands of social interaction are joining up. Increasingly the web is going multiplayer (and yes yes the metaverse too) and we step between them – desktop to Slack to Zoom to Figma.

BUT – how should it feel to move between them?

Customers have trouble distinguishing between essential features and those that are just "nice to have." Examples of the latter class: those arbitrarily overlapping windows suggested by the uncritically but widely adopted desktop metaphor; and fancy icons decorating the screen display, such as antique mailboxes and garbage cans that are further enhanced by the visible movement of selected items toward their ultimate destination. These details are cute but not essential, and they have a hidden cost.

/

Increased complexity results in large part from our recent penchant for friendly user interaction. I've already mentioned windows and icons; color, gray-scales, shadows, pop-ups, pictures, and all kinds of gadgets can easily be added.

We simply can no longer design and develop only for “optimal” content or browsing conditions. Instead, we must embrace the inherent flexibility and unpredictability of the web, and build resilient components. Static mockups cannot cater to every scenario, so many design decisions fall to developers at build time. Like it or not, if you’re a UI developer, you are a designer — even if you don’t consider yourself one!

...Sometimes interpreting a design means asking the designer to further elaborate on their ideas (or even re-evaluate them). Other times, it means making design decisions on the fly or making recommendations based on our knowledge and experience.

Back in the days of Roman numerals, basic multiplication was considered this incredibly technical concept that only official mathematicians could handle, but then once Arabic numerals came around, you could actually do arithmetic on paper, and we found that 7-year-olds can understand multiplication.

It's not that multiplication itself was difficult. It was just that the representation of numbers — the interface — was wrong

This is the soul of what was lost in the jump from terminals to GUIs. Terminals, as the spiritual descendants of punchcard automation, invite you to marshal armies of automata. The GUI puts you at the center of the HCI loop and waits for each click or press—you turn the crank.

...If GUI is manual, and automation is automatic, maybe AI opens up the possibility of semi-automatic software?

The failure mode of automation is literalism and lack of reasonable limits on behavior = brittle. But LLMs are good at doing what you mean, not just what you say.

It’s kind of funny when you think about it: what’s worse than the technical problem of building and maintaining multiple native applications? The people problem of building and maintaining multiple native applications.

To belabor the point, I am reminded of Dave’s post about states. He enumerates a large (but not comprehensive) list of the many of the dimensions that affect the your application. I’m not good at math, but to Peter’s point about how these compound, imagine the math on Dave’s list of states. That’s got to be a very, very large number.

How can you know that you have correctly functioning code?

This is what kills you with trying to control complexity. All the small variances play off of each other to create an unknowably complicated environment.

Sometimes, you have to learn to let go of control.

It’s been 19 years since Pixar released Monsters, Inc. with all that CGI hair. Where are my hairy icons? Ones that get all long and knotted as the notifications number goes up.

Why can’t I feel my phone? I found that paper from 2010 (when I was complaining about keyboards) about using precision electrostatics to make artificial textures on touchscreens.

I should be able to run my thumb over my phone while it’s in my pocket and feel bumps for apps that want my attention. Touching an active element should feel rough. A scrollbar should *slip. Imagine the accessibility gains. But honestly I don’t even care if it’s useful: 1.5 billion smartphone screens are manufactured every year. For that number, I expect bells. I expect whistles.

I think there's a fair bit of truth to [Melanie's essay], and I really like this 'always already' terminology, which highlights the way that when programming we're always immersed in some pre-existing world of affordances, that all our actions take place on top of this ground.

Nevertheless I think I want to say that the distinction between 'programming' and 'using' is roughly in the right place, that clicking a button actually is different enough to pasting someone else's premade code in your click event handler for that same button that it makes sense to call the second one 'programming' and not the first. (I could change my mind on this.)

The Pursuit of Lossless Design-Development Handoffs.

In 2014, I wrote about my belief that design and engineering are best when tightly woven together. That’s truer now than ever.

If I’m feeling confident, I’ll jump right into my text editor…From here, more functionality is added and the code is tweaked until the feature looks and feels right to me. Whether it’s something simple like this, or prototyping a new interaction like multi-connect, there’s no substitute for designing with real code.

In rare cases when I have ideas or plans that I’m less confident about, it’s time to break out the paper, pens, and markers,

Because the Kinopio interface elements and aesthetic are full-grown, I almost never use traditional design software anymore.

Everyone who interacts with computers has in important ways always already been programming them.

Every time you make a folder or rename a file on your computer, the actions you take through moving your mouse and clicking on buttons, translate into text-based commands or scripts which eventually translate into binary.

Why are the common conceptions of what a programmer and user is so divorced from each other? The distinction between programmer and user is reinforced and maintained by a tech industry that benefits from a population rendered computationally passive. If we accept and adopt the role of less agency, we then make it harder for ourselves to come into more agency.

How do we choose which level of fidelity is appropriate for a project?

I think about it like this: The purpose of making sketches and mockups before coding is to gain confidence in what we plan to do. I’m trying to remove risk from the decision to build something by somehow “previewing” it in a cheaper form. There’s a trade-off here. The higher the fidelity of the mockup, the more confidence it gives me. But the longer it takes to create that mockup, the more time I’ve wasted on an intermediate step before building the real thing.

I like to look at that trade-off economically. Each method reduces risk by letting me preview the outcome at lower fidelity, at the cost of time spent on it. The cost/benefit of each type of mockup is going to vary depending on the fidelity of the simulation and the work involved in building the real thing.

Latency in modern computer interfaces, with modern OSes and modern applications, is terrible and getting worse. This applies to smartphones as well. At the same time, while UIs were much more responsible on computers of the past, those computers were also awful in many ways: new systems have changed our lives substantially. So, what gives?

URLs (Uniform Resource Locators) have rapidly become the standard method for specifying how to access information on the Internet. Although mostly used on the World Wide Web, URLs are also becoming more common for specifying locations for other distributed Internet services such as Gopher and anonymous FTP. Internet users see URLs both online and in print, and therefore URLs have visual interfaces. This paper gives an overview of many of the issues that concern the visual and user interfaces of URLs.

Humans love symmetry. We find symmetry to be very attractive. Our brains may even be hard-wired through evolution to process symmetrical data more efficiently. So, it's no surprise that, as designers, we try to build symmetry into our product interfaces and layouts. It makes them feel very pleasant to look at.

Unfortunately, data is not symmetrical…Once you release a product into "the real world", and users start to enter "real world data" into it, you immediately see that asymmetrical data, shoe-horned into a symmetrical design, can start to look terrible.

To fix this, we need to lean into an asymmetric reality. We need to embrace the fact that data is asymmetric and we need to design user interfaces that can expand and contract to work with the asymmetry, not against it. To borrow from Bruce Lee, we need to build user interfaces that act more like water:

“You must be shapeless, formless, like water. When you pour water in a cup, it becomes the cup. When you pour water in a bottle, it becomes the bottle. When you pour water in a teapot, it becomes the teapot. Water can drip and it can crash. Become like water my friend.” — Bruce Lee

After the first websites demonstrate the commercial and aesthetic potential of the web, the media industry floods the web with a surge of new content. Amateur webzines — which define and voice and tone unique to the web — are soon joined by traditional publishers. By the mid to late 90’s, most major companies will have a website, and the popularity of the web will begin to explore. Search engines emerge as one solution to cataloging the expanding universe of websites, but even they struggle to keep up. Brands soon begin to look for a way to stand out.

An ex-Apple interface designer’s 40-year plan to redesign not just the way we use computers, but the way we think with them.

Reminds me of a little feature I like in Notion where if you type dash-arrow (like ->) it turns into → — but intelligently — like it doesn’t do that with inline code or a code block.

How many layers of UI inconsistencies are in Windows 10?

We’ve all heard this riddle: if you dig down deep enough in Windows 10, you’ll find elements that date from Windows 3.x days. But is it actually true? In this article we’ll discover just how many UI layers are in Windows and when they were first introduced.

1. Nothing
2. Loading
3. None
4. One
5. Some
6. Too Many
7. Incorrect
8. Correct
9. Done

Let’s end the confusion between UI and UX. Let’s stop slapping sirens on children’s toys. Let’s prioritize personal goals over addiction. Let’s get our lives and our health back in balance by interacting with the real world instead of staring into a light, checking new notifications. Let’s think beyond screens.

The best result for any technology is to solve meaningful problems in impactful ways.

The best design reduces work.
The best computer is unseen.
The best interaction is natural.
The best interface is no interface.

Guidebook is a website dedicated to preserving and showcasing Graphical User Interfaces, as well as various materials related to them.

I’m finding that many people not only have lowered their standards with regard to the user interface, but more and more often when I bring up the subject, they seem to consider it a somewhat secondary aspect, something that’s only good for ‘geek talk’. The same kind of amused reaction laymen have to wine or coffee connoisseurs when they describe flavours and characteristics using specific lingo. Something that makes sense only to wine or coffee geeks but has little to no meaning or impact for the regular person.

The problem is that if an increasing number of people start viewing user interface design as an afterthought, or something that isn’t fundamental to the design of a product or experience — it’s all just ‘geek talk’ — then there is a reduced incentive to care about it on the part of the maker of the product.

What I see in Safari 15 is the first steps into a new design language for iOS, one prioritizing adaptive, contextual interfaces. Ever since the move to the new “all screen” iPhone X design, content has been king on iOS, and Apple has been removing more and more user chrome. This is the next step on that journey.

Adept Fuyu-Heavy is a new multimodal model designed specifically for digital agents.

People mix the terms UI and UX together. UX is tricky because it doesn’t refer to any one thing. Interface design, visual styling, code performance, uptime, and feature set all contribute to the user’s “experience.” Books on UX further complicate matters by including research methods and development methodologies. All of this makes the field confusing for people who want to understand the fundamentals.

That’s why I avoid teaching the term ’UX.’ It means too many things to too many different people. Instead I focus on individual skills. Once you understand the individual skills, you can assemble them into a composite system without blurring them together. For software design, the core skill among all user-facing concerns is user interface design.

When many of us participate in a ritual, we each become the like the ritual. The ritual is an interface, a common set of procedures. It can act as a stable point of coordination, a “source-of-truth”, in software terms, or a lego dot that helps us click together.

Second, an open platform is essential. Communication is just one part of what humans do on the job. The modern knowledge worker relies on dozens of different products for their daily work, and that number is constantly expanding. These critical business processes and workflows demand the best tools, regardless of vendor.