Tags: sound

Spotify – dConstruct 2022

If you were at dConstruct on Friday and you enjoyed the mood music during the breaks, this is what you were listening to.

Notes on synthetic speech - Tink - Léonie Watson

This is a fascinating deep dive by Léonie on the inner workings of speech synthesis. She has quite a conundrum: she wants fast playback, but she also wants a voice that doesn’t sound robotic. Unfortunately it’s the robotic-sounding voices that work best at speed.

If you’re interested in this topic, I highly recommend listening to (or reading) the accessibility episode of the Clearleft podcast which featured Léonie as a guest giving demos and explanations.

Centuries of Sound

An audio mix for every year of recorded sound, 1859 to the present.

Currently up to 1936.

Sight and Sound: The Cinema of Walter Murch on Vimeo

I enjoyed this documentary on legendary sound designer and editor Walter Murch. Kinda makes me want to rewatch The Conversation and The Godfather.

CTS - conserve the sound

An online museum of sounds—the recordings of analogue machines.

It’s day two of An Event Apart in Boston and Val is giving a new talk about building expressive products:

The products we design today must connect with customers across different screen sizes, contexts, and even voice or chat interfaces. As such, we create emotional expressiveness in our products not only through visual design and language choices, but also through design details such as how interface elements move, or the way they sound. By using every tool at our disposal, including audio and animation, we can create more expressive products that feel cohesive across all of today’s diverse media and social contexts. In this session, Val will show how to harness the design details from different media to build overarching themes—themes that persist across all screen sizes and user and interface contexts, creating a bigger emotional impact and connection with your audience.

I’m going to attempt to live blog her talk. Here goes…

This is about products that intentionally express personality. When you know what your product’s personality is, you can line up your design choices to express that personality intentionally (as opposed to leaving it to chance).

Tunnel Bear has a theme around a giant bear that will product you from all the bad things on the internet. It makes a technical product very friendly—very different from most VPN companies.

Mailchimp have been doing this for years, but with a monkey (ape, actually, Val), not a bear—Freddie. They’ve evolved and changed it over time, but it always has personality.

But you don’t need a cute animal to express personality. Authentic Weather is a sarcastic weather app. It’s quite sweary and that stands out. They use copy, bold colours, and giant type.

Personality can be more subtle, like with Stripe. They use slick animations and clear, concise design.

Being expressive means conveying personality through design. Type, colour, copy, layout, motion, and sound can all express personality. Val is going to focus on the last two: motion and sound.

Expressing personality with motion

Animation can be used to tell your story. We can do that through:

• Easing choices (ease-in, ease-out, bounce, etc.),
• Duration values, and offsets,
• The properties we animate.

Here are four personality types…

Calm, soft, reassuring

You can use opacity, soft blurs, small movements, and easing curves with gradual changes. You can use:

• fade,
• scale + fade,
• blur + fade,
• blur + scale + fade.

Pro tip for blurs: the end of blurs always looks weird. Fade out with opacity before your blur gets weird.

You can use Penner easing equations to do your easings. See them in action on easings.net. They’re motion graphs plotting animation against time. The flatter the curve, the more linear the motion. They have a lot more range than the defaults you get with CSS keyword values.

For calm, soft, and reassuring, you could use easeInQuad, easeOutQuad, or easeInOutQuad. But that’s like saying “you could use dark blue.” These will get you close, but you need to work on the detail.

Confident, stable, strong

You can use direct movements, straight lines, symmetrical ease-in-outs. You should avoid blurs, bounces, and overshoots. You can use:

• quick fade,
• scale + fade,
• direct start and stops.

You can use Penner equations like easeInCubic, easeOutCubic and easeInOutCubic.

Lively, energetic, friendly

You can use overshoots, anticipation, and “snappy” easing curves. You can use:

• overshoot,
• overshoot + scale,
• anticipation,
• anticipation + overshoot

To get the sense of overshoots and anticipations you can use easing curves like easeInBack, easeOutBack, and easeInOutBack. Those aren’t the only ones though. Anything that sticks out the bottom of the graph will give you anticipation. Anything that sticks out the top of the graph will give you overshoot.

If cubic bezier curves don’t get you quite what you’re going for, you can add keyframes to your animation. You could have keyframes for: 0%, 90%, and 100% where the 90% point is past the 100% point.

Stripe uses a touch of overshoot on their charts and diagrams; nice and subtle. Slack uses a bit of overshoot to create a sense of friendliness in their loader.

Playful, fun, lighthearted

You can use bounces, shape morphs, squashes and stretches. This is probably not the personality for a bank. But it could be for a game, or some other playful product. You can use:

• bounce,
• elastic,
• morph,
• squash and stretch (springs.

You can use easing equations for the first two, but for the others, they’re really hard to pull off with just CSS. You probably need JavaScript.

The easing curve for elastic movement is more complicated Penner equation that can’t be done in CSS. GreenSock will help you visual your elastic easings. For springs, you probably need a dedicated library for spring motions.

Expressing personality with sound

We don’t talk about sound much in web design. There are old angry blog posts about it. And not every website should use sound. But why don’t we even consider it on the web?

We were burnt by those terrible Flash sites with sound on every single button mouseover. And yet the Facebook native app does that today …but in a much more subtle way. The volume is mixed lower, and the sound is flatter; more like a haptic feel. And there’s more variation in the sounds. Just because we did sound badly in the past doesn’t mean we can’t do it well today.

People say they don’t want their computers making sound in an office environment. But isn’t responsive design all about how we don’t just use websites on our desktop computers?

Amber Case has a terrific book about designing products with sound, and she’s all about calm technology. She points out that the larger the display, the less important auditive and tactile feedback becomes. But on smaller screens, the need increases. Maybe that’s why we’re fine with mobile apps making sound but not with our desktop computers doing it?

People say that sound is annoying. That’s like saying siblings are annoying. Sound is annoying when it’s:

• not appropriate for the situation,
• played at the wrong time,
• too loud,
• lacks user control.

But all of those are design decisions that we can control.

So what can we do with sound?

Sound can enhance what we perceive from animation. The “breathe” mode in the Calm meditation app has some lovely animation, and some great sound to go with it. The animation is just a circle getting smaller and bigger—if you took the sound away, it wouldn’t be very impressive.

Sound can also set a mood. Sirin Labs has an extreme example for the Solarin device with futuristic sounds. It’s quite reminiscent of the Flash days, but now it’s all done with browser technologies.

Sound is a powerful brand differentiator. Val now plays sounds (without visuals) from:

• Slack,
• Outlook Calendar.

They have strong associations for us. These are earcons: icons for the ears. They can be designed to provoke specific emotions. There was a great explanation on the Blackberry website, of all places (they had a whole design system around their earcons).

Here are some uses of sounds…

Alerts and notifications

You have a new message. You have new email. Your timer is up. You might not be looking at the screen, waiting for those events.

Navigating space

Apple TV has layers of menus. You go “in” and “out” of the layers. As you travel “in” and “out”, the animation is reinforced with sound—an “in” sound and an “out” sound.

Confirming actions

When you buy with Apple Pay, you get auditory feedback. Twitter uses sound for the “pull to refresh” action. It gives you confirmation in a tactile way.

Marking positive moments

This is a great way of making a positive impact in your user’s minds—celebrate the accomplishments. Clear—by Realmac software—gives lovely rising auditory feedback as you tick things off your to-do list. Compare that to hardware products that only make sounds when something goes wrong—they don’t celebrate your accomplishments.

Here are some best practices for user interface sounds:

• UI sounds be short, less than 400ms.
• End on an ascending interval for positive feedback or beginnings.
• End on a descending interval for negative feedback, ending, or closing.
• Give the user controls to top or customise the sound.

When it comes to being expressive with sounds, different intervals can evoke different emotions:

• Consonant intervals feel pleasant and positive.
• Dissonant intervals feel strong, active, or negative.
• Large intervals feel powerful.
• Octaves convey lightheartedness.

People have made sounds for you if you don’t want to design your own. Octave is a free library of UI sounds. You can buy sounds from motionsound.io, targetted specifically at sounds to go with motions.

Let’s wrap up by exploring where to find your product’s personality:

• What is it trying to help users accomplish?
• What is it like? (its mood and disposition)

You can workshops to answer these questions. You can also do research with your users. You might have one idea about your product’s personality that’s different to your customer’s. You need to project a believable personality. Talk to your customers.

Designing for Emotion has some great exercises for finding personality. Conversational Design also has some great exercises in it. Once you have the words to describe your personality, it gets easier to design for it.

So have a think about using motion and sound to express your product’s personality. Be intentional about it. It will also make the web a more interesting place.

JavaScript Systems Music

A massively in-depth study of boundary-breaking music, recreated through the web audio API.

1. Steve Reich - It’s Gonna Rain (1965)
2. Brian Eno - Ambient 1: Music for Airports, 2/1 (1978)
3. Brian Eno - Discreet Music (1975)

You don’t have to be a musician or an expert in music theory to follow this guide. I’m neither of those things. I’m figuring things out as I go and it’s perfectly fine if you do too. I believe that this kind of stuff is well within reach for anyone who knows a bit of programming, and you can have a lot of fun with it even if you aren’t a musician.

One thing that definitely won’t hurt though is an interest in experimental music! This will get weird at times.

I’ve seen some lovely examples of the Web Audio API recently.

At the Material conference, Halldór Eldjárn demoed his Poco Apollo project. It generates music on the fly in the browser to match a random image from NASA’s Apollo archive on Flickr. Brian Eno, eat your heart out!

At Codebar Brighton a little while back, local developer Luke Twyman demoed some of his audio-visual work, including the gorgeous Solarbeat—an audio orrery.

The latest issue of the Clearleft newsletter has some links on sound design in interfaces:

• Why strong sound design is critical to successful products by Amber Case and Aaron Day,
• UI Sounds: From Zero To Hero by Roman Zimarev, and
• Form Validation with Web Audio by Ruth John.

I saw Ruth give a fantastic talk on the Web Audio API at CSS Day this year. It had just the right mixture of code and inspiration. I decided there and then that I’d have to find some opportunity to play around with web audio.

As ever, my own website is the perfect playground. I added an audio Easter egg to adactio.com a while back, and so far, no one has noticed. That’s good. It’s a very, very silly use of sound.

In her talk, Ruth emphasised that the Web Audio API is basically just about dealing with numbers. Lots of the examples of nice usage are the audio equivalent of data visualisation. Data sonification, if you will.

I’ve got little bits of dataviz on my website: sparklines. Each one is a self-contained SVG file. I added a script element to the SVG with a little bit of JavaScript that converts numbers into sound (I kind of wish that the script were scoped to the containing SVG but that’s not the way JavaScript in SVG works—it’s no different to putting a script element directly in the body). Clicking on the sparkline triggers the sound-playing function.

It sounds terrible. It’s like a theremin with hiccups.

Still, I kind of like it. I mean, I wish it sounded nicer (and I’m open to suggestions on how to achieve that—feel free to fork the code), but there’s something endearing about hearing a month’s worth of activity turned into a wobbling wave of sound. And it’s kind of fun to hear how a particular tag is used more frequently over time.

Anyway, it’s just a silly little thing, but anywhere you spot a sparkline on my site, you can tap it to hear it translated into sound.

Why strong sound design is critical to successful products - O’Reilly Media

From Designing Products with Sound by Amber Case and Aaron Day:

Sound eases cognitive burdens.

Sound is also a powerful brand differentiator.

Sound is emotional.

Finally, sound impacts productivity.

But also:

Not every product needs sound design.

UI Sounds: From Zero To Hero | Icons8 Blog

Following on from Ruth’s piece, here are some more thoughts on sound in UI from Roman Zimarev, the creator of icon sounds.

He makes a distinction between notification sounds and interaction sounds, as well as talking about sound identity in branding.

Form Validation with Web Audio | CSS-Tricks

An interesting idea from Ruth—using subtle sounds to augment inline form validation.

There aren’t any extremely established best practices for this stuff. The best we can do is make tasteful choices and do user research. Which is to say, the examples in this post are ideas, not gospel.

Poco Apollo

Here’s a beautiful use of the web audio API: Enoesque generative music composed right in your browser. Each piece is generated from one of the 14,226 photos in NASA’s Apollo archive. The darker and murkier the picture, the moodier the music.

SolarBeat

Luke just demoed this at Codebar. It’s a lovely audio/visualisation of the solar system—a sonic orrery that you can tweak and adjust.

Hypnotic.

ARP Observatory (@ArpObservatory) | Twitter

In July we started receiving audio signals from outside the solar system, and we’ve been studying them since.

Tweets contain sound samples on Soundcloud, data visualisations, and notes about life at the observatory …all generated by code.

ARP is a fictional radio telescope observatory, it’s a Twitter & SoundCloud bot which procedurally generates audio, data-visualisations, and the tweets (and occasionally long-exposure photography) of an astronomer/research scientist who works at ARP, who is obsessive over the audio messages, and who runs the observatory’s Twitter account.

Open Access at ESA

Images, videos, sounds, and 3D models are now available from the European Space Agency under a Creative Commons Attribution Share-alike license.

Hatnote Listen to Wikipedia

Listen to the sound of Wikipedia’s recent changes feed. Bells indicate additions and string plucks indicate subtractions. Pitch changes according to the size of the edit; the larger the edit, the deeper the note.

RadioEight

This is rather lovely: explore a network of nodes, each of which contains the audio of a child describing a dream.

Inspired by the concept of an 8th continent to which all children belong, RadioEight is an interactive soundscape dedicated to the hidden world of dreams.

THE WALK OF LIFE PROJECT

The thesis: any film is improved by playing Walk Of Life by Dire Straits over the ending.

The proof: this website.

(this is absorbing and brilliant)

Introducing Soundslice offline mode | The Soundslice Blog

Adrian documents how he’s using Service Workers on Soundslice. I could imagine doing something similar for The Session.

Museum of Endangered Sounds

Sounds from our collective technological past.

(I’ll look past the fact that the sound labelled “ZX Spectrum” is using an image of an Amstrad PCP 464)