Journal tags: headers

There’s a new addition to the latest version of Chrome called speculation rules. This already existed before with a different syntax, but the new version makes more sense to me.

Notice that I called this an addition, not a standard. This is not a web standard, though it may become one in the future. Or it may not. It may wither on the vine and disappear (like most things that come from Google).

The gist of it is that you give the browser one or more URLs that the user is likely to navigate to. The browser can then pre-fetch or even pre-render those links, making that navigation really snappy. It’s a replacement for the abandoned link rel="prerender".

Because this is a unilateral feature, I’m not keen on shipping the code to all browsers. The old version of the API required a script element with a type value of “speculationrules”. That doesn’t do any harm to browsers that don’t support it—it’s a progressive enhancement. But unlike other progressive enhancements, this isn’t something that will just start working in those other browsers one day. I mean, it might. But until this API is an actual web standard, there’s no guarantee.

That’s why I was pleased to see that the new version of the API allows you to use an external JSON file with your list of rules.

I say “rules”, but they’re really more like guidelines. The browser will make its own evaluation based on bandwidth, battery life, and other factors. This feature is more like srcset than source: you give the browser some options, but ultimately you can’t force it to do anything.

I’ve implemented this over on The Session. There’s a JSON file called speculationrules.js with the simplest of suggestions:

{
  "prerender": [{
    "where": {
        "href_matches": "/*"
    },
    "eagerness": "moderate"
  }]
}

The eagerness value of “moderate” says that any link can be pre-rendered if the user hovers over it for 200 milliseconds (the nuclear option would be to use a value of “immediate”).

I still need to point to that JSON file from my HTML. Usually this would be done with something like a link element, but for this particular API, I can send a response header instead:

Speculation-Rules: “/speculationrules.json"

I like that. The response header is being sent to every browser, regardless of whether they support speculation rules or not, but at least it’s just a few bytes. Those other browsers will ignore the header—they won’t download the JSON file.

Here’s the PHP I added to send that header:

header('Speculation-Rules: "/speculationrules.json"');

There’s one extra thing I had to do. The JSON file needs to be served with mime-type of “application/speculationrules+json”. Here’s how I set that up in the .conf file for The Session on Apache:

<IfModule mod_headers.c>
  <FilesMatch "speculationrules.json">
    Header set Content-type application/speculationrules+json
   </FilesMatch>
</IfModule>

A bit of a faff, that.

You can see it in action on The Session. Open up Chrome or Edge (same same but different), fire up the dev tools and keep the network tab open while you navigate around the site. Notice how hovering over a link will trigger a new network request. Clicking on that link will get you that page lickety-split.

Mind you, in the case of The Session, the navigations were already really fast—performance is a feature—so it’s hard to guage how much of a practical difference it makes in this case, but it still seems like a no-brainer to me: taking a few minutes to add this to your site is worth doing.

Oh, there’s one more thing to be aware of when you’re implementing speculation rules. You have the option of excluding URLs from being pre-fetched or pre-rendered. You might need to do this if you’ve got links for adding items to shopping carts, or logging the user out. But my advice would instead be: stop using GET requests for those actions!

Most of the examples given for unsafe speculative loading conditions are textbook cases of when not to use links. Links are for navigating. They’re indempotent. For everthing else, we’ve got forms.

I mentioned that I overhauled the JavaScript on The Session recently. That wasn’t just so that I could mess about with HTML web components. I’d been meaning to consolidate some scripts for a while.

Some of the pages on the site had inline scripts. These were usually one-off bits of functionality. But their presence meant that my content security policy wasn’t as tight as it could’ve been.

Being a community website, The Session accepts input from its users. Literally. I do everything I can to sanitise that input. It would be ideal if I could make sure that any JavaScript that slipped by wouldn’t execute. But as long as I had my own inline scripts, my content security policy had to allow them to be executed with script-src: unsafe-inline.

That’s why I wanted to refactor the JavaScript on my site and move everything to external JavaScript files.

In the end I got close, but there are still one or two pages with internal scripts. But that’s okay. I found a way to have my content security policy cake and eat it.

In my content security policy header I can specifiy that inline scripts are allowed, but only if they have a one-time token specified.

This one-time token is called a nonce. No, really. Stop sniggering. Naming things is hard. And occassionally unintentionally hilarious.

On the server, every time a page is requested it gets sent back with a header like this:

content-security-policy: script-src 'self' 'nonce-Cbb4kxOXIChJ45yXBeaq/w=='

That gobbledegook string is generated randomly every time. I’m using PHP to do this:

base64_encode(openssl_random_pseudo_bytes(16))

Then in the HTML I use the same string in any inline scripts on the page:

<script nonce="Cbb4kxOXIChJ45yXBeaq/w==">
…
</script>

Yes, HTML officially has an attribute called nonce.

It’s working a treat. The security headers for The Session are looking good. I have some more stuff in my content security policy—check out the details if you’re interested.

I initially thought I’d have to make an exception for the custom offline page on The Session. After all, that’s only going to be accessed when there is no server involved so I wouldn’t be able to generate a one-time token. And I definitely needed an inline script on that page in order to generate a list of previously-visited pages stored in a cache.

But then I realised that everything would be okay. When the offline page is cached, its headers are cached too. So the one-time token in the content security policy header still matches the one-time token used in the page.

Most pages on The Session don’t have any inline scripts. For a while, every page had an inline script in the head of the document like this:

<script nonce="Cbb4kxOXIChJ45yXBeaq/w==">
document.documentElement.classList.add('hasJS');
</script>

This is something I’ve been doing for years: using JavaScript to add a class to the HTML. Then I can use the presence or absence of that class to show or hide elements that require JavaScript. I have another class called requiresJS that I put on any elements that need JavaScript to work (like buttons for copying to the clipboard, for example).

Then in my CSS I’d write:

:not(.hasJS) .requiresJS {
 display: none;
}

If the hasJS class isn’t set, hide any elements with the requiresJS class.

I decided to switch over to using a scripting media query:

@media (scripting: none) {
  .requiresJS {
   display: none;
  }
}

This isn’t bulletproof by any means. It doesn’t account for browser extensions that disable JavaScript and it won’t get executed at all in older browsers. But I’m okay with that. I’ve put the destructive action in the more modern CSS:

I feel that the more risky action (hiding content) should belong to the more complex selector.

This means that there are situations where elements that require JavaScript will be visible, even if JavaScript isn’t available. But I’d rather that than the other way around: if those elements were hidden from browsers that could execute JavaScript, that would be worse.

I’m at day two of Indie Web Camp Brighton.

Day one was excellent. It was really hard to choose which sessions to go to because they all sounded interesting. That’s a good problem to have.

I ended up participating in:

• a session on POSSE,
• a session on NFC tags,
• a session on writing, and
• a session on testing your website that was hosted by Ros

In that testing session I shared some of the bookmarklets I use regularly.

Bookmarklets? They’re bookmarks that sit in the toolbar of your desktop browser. Just like any other bookmark, they’re links. The difference is that these links begin with javascript: rather than http. That means you can put programmatic instructions inside the link. Click the bookmark and the JavaScript gets executed.

In my mind, there are two different approaches to making a bookmarklet. One kind of bookmarklet contains lots of clever JavaScript—that’s where the smart stuff happens. The other kind of bookmarklet is deliberately dumb. All they do is take the URL of the current page and pass it to another service—that’s where the smart stuff happens.

I like that second kind of bookmarklet.

Here are some bookmarklets I’ve made. You can drag any of them up to the toolbar of your browser. Or you could create a folder called, say, “bookmarklets”, and drag these links up there.

Validation: This bookmarklet will validate the HTML of whatever page you’re on.

Validate HTML

Carbon: This bookmarklet will run the domain through the website carbon calculator.

Calculate carbon

Accessibility: This bookmarklet will run the current page through the Website Accessibility Evaluation Tools.

WAVE

Performance: This bookmarklet will take the current page and it run it through PageSpeed Insights, which includes a Lighthouse test.

PageSpeed

HTTPS: This bookmarklet will run your site through the SSL checker from SSL Labs.

SSL Report

Headers: This bookmarklet will test the security headers on your website.

Security Headers

Drag any of those links to your browser’s toolbar to “install” them. If you don’t like one, you can delete it the same way you can delete any other bookmark.

Harry wrote a really good article all about the performance measurement Time To First Byte. Time To First Byte: What It Is and Why It Matters:

While a good TTFB doesn’t necessarily mean you will have a fast website, a bad TTFB almost certainly guarantees a slow one.

Time To First Byte has been the chink in my armour over at thesession.org, especially on the home page. Every time I ran Lighthouse, or some other performance testing tool, I’d get a high score …with some points deducted for taking too long to get that first byte from the server.

Harry’s proposed solution is to set up some Server Timing headers:

With a little bit of extra work spent implementing the Server Timing API, we can begin to measure and surface intricate timings to the front-end, allowing web developers to identify and debug potential bottlenecks previously obscured from view.

I rememberd that Drew wrote an excellent article on Smashing Magazine last year called Measuring Performance With Server Timing:

The job of Server Timing is not to help you actually time activity on your server. You’ll need to do the timing yourself using whatever toolset your backend platform makes available to you. Rather, the purpose of Server Timing is to specify how those measurements can be communicated to the browser.

He even provides some PHP code, which I was able to take wholesale and drop into the codebase for thesession.org. Then I was able to put start/stop points in my code for measuring how long some operations were taking. Then I could output the results of these measurements into Server Timing headers that I could inspect in the “Network” tab of a browser’s dev tools (Chrome is particularly good for displaying Server Timing, so I used that while I was conducting this experiment).

I started with overall database requests. Sure enough, that was where most of the time in time-to-first-byte was being spent.

Then I got more granular. I put start/stop points around specific database calls. By doing this, I was able to zero in on which operations were particularly costly. Once I had done that, I had to figure out how to make the database calls go faster.

Spoiler: I did it by adding an extra index on one particular table. It’s almost always indexes, in my experience, that make the biggest difference to database performance.

I don’t know why it took me so long to get around to messing with Server Timing headers. It has paid off in spades. I wish I had done it sooner.

And now thesession.org is positively zipping along!

In Going Offline, I dive into the many different ways you can use a service worker to handle requests. You can filter by the URL, for example; treating requests for pages under /blog or /articles differently from other requests. Or you can filter by file type. That way, you can treat requests for, say, images very differently to requests for HTML pages.

One of the ways to check what kind of request you’re dealing with is to see what’s in the accept header. Here’s how I show the test for HTML pages:

if (request.headers.get('Accept').includes('text/html')) {
    // Handle your page requests here.
}

So, logically enough, I show the same technique for detecting image requests:

if (request.headers.get('Accept').includes('image')) {
    // Handle your image requests here.
}

That should catch any files that have image in the request’s accept header, like image/png or image/jpeg or image/svg+xml and so on.

But there’s a problem. Both Safari and Firefox now use a much broader accept header: */*

My if statement evaluates to false in those browsers. Sebastian Eberlein wrote about his workaround for this issue, which involves looking at file extensions instead:

if (request.url.match(/\.(jpe?g|png|gif|svg)$/)) {
    // Handle your image requests here.
}

So consider this post a patch for chapter five of Going Offline (page 68 specifically). Wherever you see:

if (request.headers.get('Accept').includes('image'))

Swap it out for:

if (request.url.match(/\.(jpe?g|png|gif|svg)$/))

And feel to add any other image file extensions (like webp) in there too.

I’m at Homebrew Website Club. I figured I’d use this time to document some tweaking I’ve been doing to the back end of my website.

securityheaders.io is a handy site for testing whether your website’s server is sending sensible headers. Think of it like SSL Test for a few nitty-gritty details.

adactio.com was initially scoring very low, but the accompanying guide to hardening your HTTP headers meant I was able to increase my ranking to acceptable level.

My site is running on an Apache server on an Ubuntu virtual machine on Digital Ocean. If you’ve got a similar set-up, this might be useful…

I ssh’d into my server and went to this folder in the Apache directory

cd /etc/apache2/sites-available

There’s a file called default-ssl.conf that I need to edit (my site is being served up over HTTPS; if your site isn’t, you should edit 000-default.conf instead). I type:

nano default-ssl.conf

Depending on your permissions, you might need to type:

sudo nano default-ssl.conf

Now I’m inside nano. It’s like any other text editor you might be used to using, if you imagined what it would be like to remove all the useful features from it.

Within the <Directory /var/www/> block, I add a few new lines:

<IfModule mod_headers.c>
  Header always set X-Xss-Protection "1; mode=block"
  Header always set X-Frame-Options "SAMEORIGIN"
  Header always set X-Content-Type-Options "nosniff"
</IfModule>

Those are all no-brainers:

• Enable protection against cross-site-scripting.
• Don’t allow your site to be put inside a frame.
• Don’t allow anyone to change the content-type headers of your files after they’ve been sent from the server.

If you’re serving your site over HTTPS, and you’re confident that you don’t have any mixed content (a mixture of HTTPS and HTTP), you can add this line as well:

Header always set Content-Security-Policy "default-src https: data: 'unsafe-inline' 'unsafe-eval'"

To really up your paranoia (and let’s face it, that’s what security is all about; justified paranoia), you can throw this in too:

Header unset Server
Header unset X-Powered-By

That means that your server will no longer broadcast its intimate details. Of course, I’ve completely reversed that benefit by revealing to you in this blog post that my site is running on Apache on Ubuntu.

I’ll tell you something else too: it’s powered by PHP. There’s some editing I did there too. But before I get to that, let’s just finish up that .conf file…

Hit ctrl and o, then press enter. That writes out the file you’ve edited. Now you can leave nano: press ctrl and x.

You’ll need to restart Apache for those changes to take effect. Type:

service apache2 restart

Or, if permission is denied:

sudo service apache2 restart

Now, about that PHP thing. Head over to a different directory:

cd /etc/php5/fpm

Time to edit the php.ini file. Type:

nano php.ini

Or, if you need more permissions:

sudo nano php.ini

It’s a long file, but you’re really only interested in one line. A shortcut to finding that line is to hit ctrl and w (for “where is?”), type expose, and hit enter. That will take you to the right paragraph. If you see a line that says:

expose_php = On

Change it to:

expose_php= Off

Save the file (ctrl and o, enter) then exit nano (ctrl and x).

Restart Apache:

service apache2 restart

Again, you might need to preface that with sudo.

Alright, head on back to securityheaders.io and see how your site is doing now. You should be seeing a much better score.

There’s one more thing I should be doing that’s preventing me from getting a perfect score. That’s Public Key Pinning. It sounds a bit too scary for a mere mortal like me to attempt. Or rather, the consequences of getting it wrong (which I probably would), sound too scary.