Tim Retout

seL4 Microkit Tutorial

Mon, 29 Apr 2024 22:02:25 +0100

Recently I revisited my previous interest in seL4 - a fast, highly assured operating system microkernel for building secure systems.

The seL4 Microkit tutorial uses a simple Wordle game example to teach the basics of seL4 Microkit (formerly known as the seL4 Core Platform), which is a framework for creating static embedded systems on top of the seL4 microkernel. Microkit is also at the core of LionsOS, a new project to make seL4 accessible to a wider audience.

The tutorial is easy to follow, needing few prerequisites beyond a QEMU emulator and an AArch64 cross-compiler toolchain (Microkit being limited to 64-bit ARM systems currently). Use of an emulator makes for a quick test-debug cycle with a couple of Makefile targets, so time is spent focusing on walking through the Microkit concepts rather than on tooling issues.

This is an unusually good learning experience, probably because of the academic origins of the project itself. The DiÃ¡taxis documentation framework would class this as truly a “tutorial” rather than a “how-to guide” - you do learn a lot by implementing the exercises.

Prevent DOM-XSS with Trusted Types - a smarter DevSecOps approach

Mon, 01 Jan 2024 12:46:22 +0000

It can be incredibly easy for a frontend developer to accidentally write a client-side cross-site-scripting (DOM-XSS) security issue, and yet these are hard for security teams to detect. Vulnerability scanners are slow, and suffer from false positives. Can smarter collaboration between development, operations and security teams provide a way to eliminate these problems altogether?

Google claims that Trusted Types has all but eliminated DOM-XSS exploits on those of their sites which have implemented it. Let’s find out how this can work!

DOM-XSS vulnerabilities are easy to write, but hard for security teams to catch

It is very easy to accidentally introduce a client-side XSS problem. As an example of what not to do, suppose you are setting an element’s text to the current URL, on the client side:

// Don't do this
para.innerHTML = location.href;

Unfortunately, an attacker can now manipulate the URL (and e.g. send this link in a phishing email), and any HTML tags they add will be interpreted by the user’s browser. This could potentially be used by the attacker to send private data to a different server.

Detecting DOM-XSS using vulnerability scanning tools is challenging - typically this requires crawling each page of the website and attempting to detect problems such as the one above, but there is a significant risk of false positives, especially as the complexity of the logic increases.

There are already ways to avoid these exploits â€” developers should validate untrusted input before making use of it. There are libraries such as DOMPurify which can help with sanitization.¹

However, if you are part of a security team with responsibility for preventing these issues, it can be complex to understand whether you are at risk. Different developer teams may be using different techniques and tools. It may be impossible for you to work closely with every developer â€” so how can you know that the frontend team have used these libraries correctly?

Trusted Types closes the DevSecOps feedback loop for DOM-XSS, by allowing Ops and Security to verify good Developer practices

Trusted Types enforces sanitization in the browser², by requiring the web developer to assign a particular kind of JavaScript object rather than a native string to .innerHTML and other dangerous properties. Provided these special types are created in an appropriate way, then they can be trusted not to expose XSS problems.

This approach will work with whichever tools the frontend developers have chosen to use, and detection of issues can be rolled out by infrastructure engineers without requiring frontend code changes.

Content Security Policy allows enforcement of security policies in the browser itself

Because enforcing this safer approach in the browser for all websites would break backwards-compatibility, each website must opt-in through Content Security Policy headers.

Content Security Policy (CSP) is a mechanism that allows web pages to restrict what actions a browser should execute on their page, and a way for the site to receive reports if the policy is violated.

Figure 1: Content-Security-Policy browser communication

This is revolutionary, because it allows servers to receive feedback in real time on errors that may be appearing in the browser’s console.

Trusted Types can be rolled out incrementally, with continuous feedback

Web.dev’s article on Trusted Types explains how to safely roll out the feature using the features of CSP itself:

Deploy a CSP collector if you haven’t already
Switch on CSP reports without enforcement (via Content-Security-Policy-Report-Only headers)
Iteratively review and fix the violations
Switch to enforcing mode when there are a low enough rate of reports

Static analysis in a continuous integration pipeline is also sensible â€” you want to prevent regressions shipping in new releases before they trigger a flood of CSP reports. This will also give you a chance of finding any low-traffic vulnerable pages.

Smart security teams will use techniques like Trusted Types to eliminate entire classes of bugs at a time

Rather than playing whack-a-mole with unreliable vulnerability scanning or bug bounties, techniques such as Trusted Types are truly in the spirit of ‘Secure by Design’ â€” build high quality in from the start of the engineering process, and do this in a way which closes the DevSecOps feedback loop between your Developer, Operations and Security teams.

Sanitization libraries are especially needed when the examples become more complex, e.g. if the application must manipulate the input. DOMPurify version 1.0.9 also added Trusted Types support, so can still be used to help developers adopt this feature. ↩︎
Trusted Types has existed in Chrome and Edge since 2020, and should soon be coming to Firefox as well. However, it’s not necessary to wait for Firefox or Safari to add support, because the large market share of Chrome and Edge will let you identify and fix your site’s DOM-XSS issues, even if you do not set enforcing mode, and users of all browsers will benefit. Even so, it is great that Mozilla is now on board. ↩︎

Nostalgia for my attention span

Wed, 06 Dec 2023 18:38:45 +0000

This post was possibly inspired by my daughter’s homework assignment to interview an old person about technology change. Guess who’s old now?

Sometimes I look back at how life used to be, and remember what it was like. There are two key nostalgia points for me: before internet, and before smartphones.

Before the internet

Before the internet, there were computers. There were always computers in my life, they were just less fancy, with mainly text and fewer graphics at first. These days we adapt our user interfaces to look more like DOS and call it ‘retro’, although claiming it’s more efficient that way. Sometimes it is. The written word is a fundamental mode of human communication â€” it resonates.

Some of my earliest programming memories were typing lines of BASIC code into some sort of BBC Micro, usually not successfully. Most of my computer learning was largely theoretical, through Usborne books (now available online! Thanks internet for reducing the marginal cost of publishing to approximately zero) but without the benefit of a computer of my own.

I went for a walk this morning, with a slight bite to the air, and bought a newspaper, like the old person I am (I’m leaning in). Imagine that - a physical paper full of news. When I was an inquisitive sixth-former, ‘TheGuardian’ cost 50 pence â€” today it was Â£2.80. This increase far outstrips the rate of inflation, which would bring it to around 90 pence. No, this must surely reflect the drop in circulation of the physical edition, and the rise in advertisement-filled web content.

Before smartphones

As the internet became mainstream, I remember dial-up, and Freeserve, and even the tail end of USENET, and IRC.

This was still a time when email was a thing you had to be at your desk to check. Feature phones were very useful, but it was still possible to get lost. My university halls had a shared land-line phone on each floor.

And I wrote on the internet. I remember commenting on blogs, and I think my writing was better back then; more free, less inhibited.

I do find myself giving less attention to news squeezed into the small form-factor of a mobile phone. Chat messages and notification pings. I’m not convinced this is great progress for humanity.

Data Diodes

Tue, 18 Apr 2023 23:05:36 +0100

At ArgoCon today, Thomas Fricke gave a nice talk on Cloud Native Deployments in Air Gapped Environments describing container vulnerability scanning in the German energy sector… and since he didn’t mention data diodes, and since some of my colleagues at Oakdoor/PA Consulting make data diodes for a living, I thought this might be interesting to write about!

It’s one thing to have an air-gapped system, but eventually in order to be useful you’re going to have to move data into it, and this is going to need something better than just plugging a USB stick into your critical system. Just ask Iran how well this goes.

Eight years after Stuxnet, the UK National Cyber Security Centre published the NCSC Safely Importing Data Pattern - but I found this a bit cryptic on first reading, because it’s not clear what type of systems the pattern applies to, and deliberately uses technology-neutral language. Also, this was published around the same time GDPR was being implemented, mentions “sensitive or personal data”, and claims to be aimed at “small to medium organisations” - but I don’t know how many small businesses implement a MILS security architecture. So without picking up on the mention of “data diode”, you can be left scratching your head about how to actually implement the pattern.

One answer using Oakdoor components:

PySISL, a Python library which you use to transform the data into a very simple format called SISL
an Oakdoorâ„¢ Import Diode, which can verify the syntax of SISL in hardware, and prevent any data moving back the other way
then some more PySISL code to validate the semantics of SISL on the high side and reconstruct the original format

The Oakdoor diodes themselves are quite interesting - they’re electrical rather than optical like most data diodes. The other thing I’d always wondered is how on earth you could even establish a TCP handshake across one - the answer is, you can’t, so you use a UDP-based protocol like TFTP for file transfer.

In this way, you build the transform/verify and protocol break that the NCSC pattern requires.

Congratulations, you can now import your documents to your otherwise air-gapped system without also importing malicious code, and without risking data exfiltration.

Note carefully that the Safely Importing Data pattern makes no guarantees about the integrity of your documents - they could be severely modified going through this process. For the same reason, I anticipate challenges applying this pattern to software binaries.

AlmaLinux and SBOMs

Sat, 04 Feb 2023 16:37:32 +0000

At CentOS Connect yesterday, Jack Aboutboul and Javier Hernandez presented a talk about AlmaLinux and SBOMs [video], where they are exploring a novel supply-chain security effort in the RHEL ecosystem.

Now, I have unfortunately ignored the Red Hat ecosystem for a long time, so if you are in a similar position to me: CentOS used to produce debranded rebuilds of RHEL; but Red Hat changed the project round so that CentOS Stream now sits in between Fedora Rawhide and RHEL releases, allowing the wider community to try out/contribute to RHEL builds before their release. This is credited with making early RHEL point releases more stable, but left a gap in the market for debranded rebuilds of RHEL; AlmaLinux and Rocky Linux are two distributions that aim to fill that gap.

Alma are generating and publishing Software Bill of Material (SBOM) files for every package; these are becoming a requirement for all software sold to the US federal government. What’s more, they are sending these SBOMs to a third party (CodeNotary) who store them in some sort of Merkle tree system to make it difficult for people to tamper with later. This should theoretically allow end users of the distribution to verify the supply chain of the packages they have installed?

I am currently unclear on the differences between CodeNotary/ImmuDB vs. Sigstore/Rekor, but there’s an SBOM devroom at FOSDEM tomorrow so maybe I’ll soon be learning that. This also makes me wonder if a Sigstore-based approach would be more likely to be adopted by Fedora/CentOS/RHEL, and whether someone should start a CentOS Software Supply Chain Security SIG to figure this out, or whether such an effort would need to live with the build system team to be properly integrated. It would be nice to understand the supply-chain story for CentOS and RHEL.

As I write this, I’m also reflecting that perhaps it would be helpful to explain what happens next in the SBOM consumption process; i.e. can this effort demonstrate tangible end user value, like enabling AlmaLinux to integrate with a vendor-neutral approach to vulnerability management? Aside from the value of being able to sell it to the US government!

Git internals and SHA-1

Wed, 29 Jun 2022 18:54:15 +0100

LWN reminds us that Git still uses SHA-1 by default. Commit or tag signing is not a mitigation, and to understand why you need to know a little about Git’s internal structure.

Git internally looks rather like a content-addressable filesystem, with four object types: tags, commits, trees and blobs.

Content-addressable means changing the content of an object changes the way you address or reference it, and this is achieved using a cryptographic hash function. Here is an illustration of the internal structure of an example repository I created, containing two files (./foo.txt and ./bar/bar.txt) committed separately, and then tagged:

You can see how ‘trees’ represent directories, ‘blobs’ represent files, and so on. Git can avoid internal duplication of files or directories which remain identical. The hash function allows very efficient lookup of each object within git’s on-disk storage.

Tag and commit signatures do not directly sign the files in the repository; that is, the input to the signature function is the content of the tag/commit object, rather than the files themselves. This is analogous to the way that GPG signatures actually sign a cryptographic hash of your email, and there was a time when this too defaulted to SHA-1. An attacker who can break that hash function can bypass the guarantees of the signature function.

A motivated attacker might be able to replace a blob, commit or tree in a git repository using a SHA-1 collision. Replacing a blob seems easier to me than a commit or tree, because there is no requirement that the content of the files must conform to any particular format.

There is one key technical mitigation to this in Git, which is the SHA-1DC algorithm; this aims to detect and prevent known collision attacks. However, I will have to leave the cryptanalysis of this to the cryptographers!

So, is this in your threat model? Do we need to lobby GitHub for SHA-256 support? Either way, I look forward to the future operational challenge of migrating the entire world’s git repositories across to SHA-256.

Exploring StackRox

Tue, 26 Apr 2022 21:07:05 +0100

At the end of March, the source code to StackRox was released, following the 2021 acquisition by Red Hat. StackRox is a Kubernetes security tool which is now badged as Red Hat Advanced Cluster Security (RHACS), offering features such as vulnerability management, validating cluster configurations against CIS benchmarks, and some runtime behaviour analysis. In fact, it’s such a diverse range of features that I have trouble getting my head round it from the product page or even the documentation.

Source code is available via the StackRox organisation on GitHub, and the most obviously interesting repositories seem to be:

stackrox/stackrox, containing the main application, written in Go
stackrox/scanner, the vulnerability scanner, also in Go. From a first glance at the go.mod file, it does not seem to share much code with Clair, which is interesting.
stackrox/collector, the runtime analysis component, in C++ but also with hooks into the kernel.

My initial curiosity has been around the ‘collector’, to better understand what runtime behaviour the tool can actually pick up. I was intrigued to find that the actual kernel component is a patched version of Falco’s kernel module/eBPF probes; a few features are disabled compared to Falco, e.g. page faults and signal events.

There’s a list of supported syscalls in driver/syscall_table.c, which seems to have drifted slightly or be slightly behind the upstream Falco version? In particular I note the absence of io_uring, but given RHACS is mainly deployed on Linux 4.18 at the moment (RHEL 8) this is probably a non-issue. (But relevant if anyone were to run it on newer kernels.)

That’s as far as I’ve got for now. Red Hat are making great efforts to reach out to the community; there’s a Slack channel, and office hours recordings, and a community hub to explore further. It’s great to see new free software projects created through acquisition in this way - I’m not sure I remember seeing a comparable example.

Reflections on OSSF London 2021

Thu, 07 Oct 2021 08:55:06 +0100

On Tuesday I attended the Open Source Strategy Forum in London, which is a meeting of the Fintech Open Source Foundation (FinOS), part of the Linux Foundation. (There is a New York version coming up in November for those across the pond.)

The morning keynotes included Gabriele Columbro introducing the day, then Russell Green highlighting the progress FinOS has made; Liz Rice of CNCF fame with an inspiring talk about contributing back to upstream; an interesting conversation between Nick Cook and Jane Gavronsky about innovations in financial regulation, and finally a presentation from Andrew Agerbak of BCG about how open source can help banks move to public cloud. (I disagreed with some of Andrew’s presentation; I would weight the regulatory requirements more strongly, but agree with the point that open source can help with cloud portability.)

In the afternoon there were two talks which stood out for me but were very sparsely attended:

Rob Knight from SUSE presented a short session on Rancher, which offers an open source, vendor-neutral way to manage Kubernetes clusters across multiple providers. I disagree with the view that modern Kubernetes demands smaller clusters, perhaps one per workload - to me this reduces utilization unnecessarily as discussed in the original Borg paper. But I’m excited about an approach to multi-cloud Kubernetes that does not rely on a commercial relationship with a single vendor.

Axel Simon from Red Hat gave a short talk about Sigstore, a new approach to open source supply chain security. Essentially this brings together several other tools (tbd, rekor, fulcio) and offers a free-to-use non-profit Lets-Encrypt-but-for-software-signatures CA. This seems quite promising.

To me these talks both cut to the heart of the big strategic challenges facing large financial services organisations today: how do you outsource to the cloud in a portable way, and how do you keep it all secure when you’re working at this scale?

Gabriele’s opening keynote included this quote:

…the financial services industry has pioneered things that later became popular in open source, like efficient messaging protocols, microservices, and event streaming, but the technologies were kept proprietary in banks and only became mainstream when other companies open sourced them. It would be beneficial to contribute more…

CTO, Investment Bank

It feels like there’s a real appetite within financial services to reclaim that spirit of innovation through open source that had been ceded in recent years to big tech. I came away from the day excited about the future of open source in financial services.

(Shout out to James McLeod of FinOS for inviting me along!)

GCP - Planning for the Worst

Mon, 04 Oct 2021 10:41:28 +0100

Last month, Google Cloud published Planning for the Worst: Reliability, Resilience, Exit and Stressed Exit in Financial Services. This happens to be a topic I have previously worked on, so I was very interested to hear the perspective that GCP would bring.

The wider industry context here is that regulators are very interested in potential risks to the financial system arising from the wholesale migration to cloud computing; in March 2021 the Prudential Regulation Authority in the UK published two supervisory statements closely related to the topic, including Outsourcing and third party risk management, which introduces the concept of a “stressed exit”. That is, if a Cloud Service Provider were to become insolvent, suffer a catastrophic technical failure, or (perhaps more likely) get banned from doing business in a particular geographical region… as a bank, what would you do if you have outsourced all your computing services to that provider?

I would rate the Google Cloud paper as “average” - it treads familiar ground and provides very little insight that could not be gathered from an afternoon of reading the regulations (which, however, are helpfully listed in the introduction). The equivalent Azure paper contains much more detailed advice on how to develop an exit plan and was published more than a year earlier. Still, there are a few interesting nuggets here:

Firstly, Google lists its commitment to Open Source as an advantage for exit planning; many of its products and services are available in open source versions. Two examples which come to mind are Kubernetes and Tensorflow; here GCP has adopted a strategy of creating new software categories that the other CSPs have embraced, which does make it easier to avoid vendor lock-in. Dataflow is available as Apache Beam. DataProc is really a managed Hadoop.

However, an obvious counterexample is listed slightly further down: BigQuery’s capabilities are substantially unique (I predict migrating large workloads to AWS Athena may be challenging, for example). Is there a globally-consistent competitor to Cloud Spanner yet?

The exit planning benefit of GCP’s open source commitments therefore largely depends on whether the workloads have been designed with exit requirements in mind.

Secondly, common standards for hosting applications in virtual machines or containers are also listed as an advantage; the benefit of these similarly depends on whether the workload is built against these interfaces.

Finally, Anthos (GCP’s multi-cloud management tool) is mentioned under both exit planning and stressed exit planning. It is true that this facilitates management of Kubernetes clusters in other clouds or on-premise; but if you are unexpectedly ending your commercial relationship with Google, how easy is it going to be to migrate those clusters to an alternative “single pane of glass”? If you have configured workload identity for your GKE-on-AWS clusters, and GCP disappears, how much trouble are you in? I have had a brief look, but have not yet found a white paper on this!

When discussing critically-important financial workloads which may well be essential to the stability of the entire financial system, I think the question asked by the regulators is legitimate: what new risks are you introducing when you add a technology outsourcing relationship to the mix? If this relationship suddenly breaks down, what happens?

But this train of thought needs to be taken to the logical conclusion: if one of the hyperscale cloud providers genuinely did collapse, or even if a bank just decided to end a commercial relationship, most of the exit plans in existence today would fail. When developing the plans, each workload is considered individually, and assumes the luxury of several months to execute rapid custom development of infrastructure on a new cloud. In reality, all workloads in the bank would be affected at once - there would be a massive shortage of cloud engineers within the organisation (and possibly across the industry depending on the scenario), and these would be some of the most rushed and risky projects of all time. This is concentration risk that is touched on only briefly at the end of the paper.

And in this regard the paper does hint at the right answers - the most critical types of workload need to be built against industry standard interfaces (e.g. containers are essentially an extension of the stable Linux kernel-userspace API boundary); avoid over-reliance on CSP-specific services such as BigQuery; use open source that you can run elsewhere (e.g. Kubernetes, complex as it is, is at least provided by multiple vendors) and replicate data across multiple suppliers to enable sufficiently swift recovery from disasters.

These engineering constraints are necessary because society cannot afford for these services to be locked-in to a single vendor; not for mere anti-competition reasons, but to ensure continuity of service.

Maglev Load Balancers

Tue, 28 Sep 2021 20:11:01 +0100

Maglev is the codename of Googleâ€™s Layer 4 network load balancer, which is referred to in GCP as External TCP/UDP Network Load Balancing. I read the 2016 Maglev paper to better understand various implementation details of Maglev with an emphasis on security (in particular as affects availability).

Maglev uses a scale-out approach, implemented within clusters built from commodity hardware achieving n+1 redundancy, providing greater tolerance to failure compared with traditional hardware load balancers deployed in pairs (only 1+1 redundancy). The collection of Maglev machines are in an active-active setup, with the router balancing across them via Equal Cost Multipath (ECMP) routing. This permits greater hardware utilization compared to an active-passive approach.

The paper discusses in detail the various techniques used to increase performance (userspace networking to avoid kernel switching, use of 5-tuple hashing to avoid needing to share data across threads, and pinning the threads to dedicated CPUs).

A local connection tracking table is used to ensure packets in the same connection are routed to the same backend endpoint consistently. However, two failure cases are discussed: during upgrades/failures the set of Maglev machines might change, or the connection table might simply run out of space.

The impact of these cases are reduced using Maglev hashing, a novel approach to connection tracking with consistent hashing that optimizes for balancing the load rather than guaranteeing connections will not shift to a different backend endpoint. The paper includes results from experiments to show that the typical impact of failure cases is low.

This was an interesting read, giving an insight into how modern highly-available load balancing works. The approach from this paper has subsequently been adopted by other implementations (e.g. is used in the open source project Cilium which is part of GKE Dataplane v2).

Google Workspace Super Admins

Sun, 19 Sep 2021 16:44:36 +0100

I recently had cause to remind myself of Google Workspace administrator account best practices. Briefly:

Set up separate admin accounts, e.g. [email protected] to exist side-by-side with [email protected]. Keep accounts individually identifiable, and ideally ensure there are multiple Super Admins in your organization.¹
Avoid using [email protected] for day-to-day use.
One of these Super Admin accounts must be set as the primary account contact, but (due to the previous point) you’re unlikely to be checking the emails very often. Set up a “Secondary email” for the organization to receive alerts and updates.
Enrol the admin account in Advanced Protection, which enforces 2SV with two physical security keys. Avoid losing the keys.

Interestingly the Super Admin will then have a personal email address and a personal phone linked to the account - I guess there’s some risk that those could be used as a vector for taking over the account, but presumably Advanced Protection makes this more challenging.

There is GCP guidance on this topic which contradicts the idea of keeping Super Admin accounts individually identifiable - i.e. “not specific to a particular user”. I suspect it’s outdated and have sent feedback on that page. ↩︎

Go Trie Benchmarks

Sun, 10 Jan 2021 22:02:15 +0000

After writing a trie I wanted to better understand its performance, so I wrote some benchmarks against various other Go implementations for storing UK postcodes.

At some point since the new year I entirely replaced the implementation from my last post with one that more closely matches the “pure” trie described at the start of TAOCP 6.3; i.e. a table of nodes, consisting of a list of entries, where each node entry can be either a link to another node, or a key (that is, an entire string stored in the trie).

Perhaps my chosen use case is unusual (postcodes take up less than a 64-bit pointer!) but I was pleased with the performance; admittedly the current implementation achieves this by reducing the alphabet size, but I think this could generalize fairly easy with the alphabet passed in at Trie construction time. There is room for optimization of the entry lookup, which currently just indexes into the alphabet string; a hash function would be faster, but I am unsure whether I want to think about hash collisions.

The biggest weakness of these benchmarks so far is that they do not look at prefix search - only testing whether a string exists in the trie. I did benchmark my trie against a plain Go map, and the trie manages to achieve faster access if you are looking for keys in insertion order, which I was quite impressed with - I believe this will be because the right nodes will be in the L1/L2 caches, whereas a map cannot make use of this. Random access is of course much faster with a map.

I enjoy this type of algorithm exploration, though I don’t have a serious use for a trie in the short term. Maybe something will come up!

Golang Trie

Thu, 31 Dec 2020 09:47:47 +0000

A trie (pronounced either “tree” or “try”) is a data structure typically used to store a set of strings in a way that allows looking up by prefix efficiently - i.e. unlike a hashmap where the keys are randomly ordered - this makes it a reasonable choice for an autocompletion system. A possible advantage over binary trees is that the keys are not stored in full in each node - so if you have a large number of strings which often have overlapping prefixes (e.g. “cat”, “cats”, “catastrophe”) then you may be able to save memory.

I looked around the internet for examples of tries written in Golang, but many make use of Go “maps” within the nodes; maps are a complex data structure, and this feels like cheating! More precisely, it is harder to understand the size/speed of the map and whether this outweighs the space savings of using a trie in the first place. Similarly with Go slices - slices require a pointer and two ints, so while using them might be idiomatic, each one costs 16 extra bytes (on 64 bit systems), which seems like it might not be necessary.

Therefore, I wrote my own Golang trie to understand more of the details.

De la Briandais tries

Grabbing my copy of TAOCP vol 3 from the dusty shelf, I looked up tries and implemented the linked-list version, which apparently is known as a “de la Briandais” trie.

I made an assumption that we would break strings into UTF-8 runes. My original node definition was as follows:

type node struct {
	next      *node // 8 bytes (on 64-bit systems)
	children  *node // 8 bytes
	character rune  // 4 bytes
}

Initially I used a separate node with a sentinel value rune(-1) to indicate that a node ended a key, but later I added a “value” field and used the zero value. This allowed mapping keys to integers, but means that you cannot store the zero value against a key - a boolean field can be added to avoid this limitation.

Lookup performance

I made no attempt to keep the linked-list in any particular order.

If there are a large number of keys starting with the same prefix (e.g. if we are storing random binary bytes, or a large number of Chinese words with an “alphabet” of around 3000 characters) then the lookup may need to scan through a large number of nodes and this may not be an effective data structure. However for small alphabets like English text or UK postcodes, there can be only a relatively small number of nodes at each level.

My arbitrary benchmarks on random alphanumeric strings suggest this trie outperforms some of the competitors in both lookup time and memory usage, but it is worth testing on your data set to know for sure.

Memory usage

When benchmarking, I was surprised to find that each node seemed to allocate 32 bytes of memory, rather than 20 bytes. I ruled out struct alignment issues, and realised instead that Golang allocates small objects in fixed size classes, so the memory usage gets rounded up to a power of two.

I think it might be possible to reduce this by storing all nodes in a single slice (which would be 8 byte aligned, so each node would use 24 bytes), but I have not yet tried this.

Expanding strings to runes means that each character takes four bytes - for English text, often each character requires only one byte when stored within a string (ignoring the 16 byte string header). It is quite possible that a map or binary tree or another data structure will use less memory than this trie on your data set (but changing the node definition to use bytes would have no effect due to the allocation issue above).

Conclusion

Tries are a relatively basic data structure, but many implementations found in the wild have unexpected properties when you look under the hood! Writing my own trie has already taught me more about how Go handles memory, and I may experiment some more with different implementations to explore the trade-offs.

Assume nothing - always benchmark, and ensure your choice of data structure is appropriate for your specific use case.

Focusing on Ingenuity

Mon, 28 Dec 2020 08:13:29 +0000

I am led to believe that New Year’s Resolutions seldom work, and that a more effective approach to goal setting is to choose a theme for the year. This makes sense to me, as in hindisght 2020 threw up a few surprises.

Last January, buoyed by my success in gaining various technology-related certifications, I wrote that I would focus on learning; at that time I intended to engage in more formal study, but subsequently rediscovered how little I enjoy essay writing! However, in the end I did learn a good deal this year, so perhaps I achieved my aim after all.

This year I am taking “ingenuity” as a theme; I want to recapture that joy in creation. I heard another idea recently - that we are made in the image of God, and therefore we love to create. I love this, as it legitimizes that feeling of play for me - and I think matches the best of the original hacker ethos. I am reevaluating my actions and commitments through this lens, and looking to spend more time being ingenious (or trying to be)!

Bin Calendar

Mon, 28 Dec 2020 00:24:37 +0000

Around this time each year it is especially useful to know when the rubbish is due to be collected by the local council, since the schedule is inevitably disrupted by the holidays until well into January. In fact where I live we have fortnightly collections, with different types of bin collected on alternate weeks, so I never find it easy to remember which bin is due to be put out.

This time last year I wrote a bin collections Twitter bot, mainly while exploring the Twitter API - but twelve months later, I have grown to accept that Twitter is not the most appropriate user experience for discovering important notifications.¹

The correct user experience for looking up scheduled events is of course a calendar. The council do publish a PDF calendar, but it requires some deciphering, as they choose to combine everyone’s calendar on one page and then let you look up the bin colour like an old train timetable, given that you know your collection day. Fortunately they have also created a web service which tells you the exact collections due in the next few weeks, tied (I believe) directly into their backend systems. This API is used by the main web page which allows everyone to check their bin day, and so it was even updated correctly when these were disrupted early in the pandemic.

I have adapted my Twitter bot code (which reads this API) to instead publish an ics file; I have then subscribed to this in my favoured calendaring application, and enabled notifications at 5pm on the day before the bins are due. This should work.

A word on the implementation (since I apparently said nothing on my blog about this a year ago): I wrote it as a Google Cloud Function in Golang, triggered by Cloud Scheduler via Pub/Sub. The ics file is output to a Cloud Storage bucket. This requires only a few seconds of CPU time per day, and fits well within the smallest 128MB RAM quota - so is extremely inexpensive, with no significant attack surface to worry about.

Although in an ideal world the council would publish and maintain these calendars themselves, I find some joy in creating hacks like this.

For one thing, I hardly ever look at Twitter now - in fact my bin bot is probably my most frequented Twitter feed - but even when I used Twitter more frequently, it was easy to lose important posts in a sea of news, jokes and arguments. I adopted a policy of following only people I knew in real life, which altered the feel of the network quite substantially; and of course ensuring a chronological feed rather than relying on the algorithm. But it didn’t make it feel right, and I’m happy to have social media fade out of my life. But I digress. ↩︎

seL4 on Raspberry Pi 3 in AArch64 mode

Mon, 11 May 2020 23:06:48 +0100

Several references exist that document how to run seL4 on the Raspberry Pi 3 in 32-bit mode. One annoying paper cut encountered when getting this working is the need for a custom u-boot - either a binary distributed by the authors of seL4, or reverting a particular commit in u-boot.

A recent release of seL4 mentioned AArch64 support for RPi3. I’ve got it running, and it appears to avoid the need for a custom u-boot. (N.B. you still need a GPIO serial cable!)

I started with a stock Raspbian Lite image to get a known-good filesystem (with start.elf and bootcode.bin), then added u-boot and seL4 on top.

After installing the appropriate toolchain, seL4test can be configured and built almost as normal:

mkdir sel4test && cd sel4test
repo init -u https://github.com/seL4/sel4test-manifest.git
repo sync
mkdir build-rpi3 && cd build-rpi3
# Note AARCH64 here
./init-build.sh -DPLATFORM=rpi3 -DAARCH64=1
ninja

Copy build-rpi3/images/sel4test-driver-image-arm-bcm2837 to the SD card.

U-Boot was compiled approximately as per https://a-delacruz.github.io/ubuntu/rpi3-setup-64bit-uboot.html - I chose to use the latest stable release tag, v2020.04.

make CROSS_COMPILE=aarch64-linux-gnu- rpi_3_defconfig
make -j -s CROSS_COMPILE=aarch64-linux-gnu-

Copy u-boot.bin to the SD card.

Edit config.txt to add:

enable_uart=1
kernel=u-boot.bin
# Enable 64 bit mode
arm_control=0x200

Additionally I add a boot.scr file that automates loading and booting the image. First create boot.txt with the commands from the seL4 RPi3 support page:

fatload mmc 0 0x10000000 sel4test-driver-image-arm-bcm2837
bootelf 0x10000000

Then use mkimage from the u-boot source tree:

mkimage -A arm -O linux -T script -C none -n boot.scr -d boot.txt boot.scr

Copy boot.scr to the SD card.

If all is well, your serial output should show the test suite running to a reassuring conclusion:

U-Boot 2020.04 (May 11 2020 - 22:37:52 +0100)                                   
                                                                                
DRAM:  948 MiB                                                                  
RPI 3 Model B (0xa02082)                                                        
MMC:   mmc@7e202000: 0, sdhci@7e300000: 1                                       
Loading Environment from FAT... *** Warning - bad CRC, using default environment
                                                                                
In:    serial                                                                   
Out:   vidconsole                                                               
Err:   vidconsole                                                               
Net:   No ethernet found.                                                       
starting USB...                                                                 
Bus usb@7e980000: scanning bus usb@7e980000 for devices... 3 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found            
Hit any key to stop autoboot:  0                                                
switch to partitions #0, OK                                                     
mmc0 is current device                                                          
Scanning mmc 0:1...                                                             
Found U-Boot script /boot.scr                                                   
151 bytes read in 5 ms (29.3 KiB/s)                                             
## Executing script at 02400000                                                 
3199560 bytes read in 141 ms (21.6 MiB/s)                                       
## Starting application at 0x00901000 ...                                       
                                                                                
ELF-loader started on CPU: ARM Ltd. Cortex-A53 r0p4                             
  paddr=[901000..c090f7]                                                        
No DTB passed in from boot loader.                                              
Looking for DTB in CPIO archive...found at a235c0.                              
Loaded DTB from a235c0.                                                         
   paddr=[238000..23bfff]                                                       
ELF-loading image 'kernel'                                                      
  paddr=[0..237fff]                                                             
  vaddr=[ffffff8000000000..ffffff8000237fff]                                    
  virt_entry=ffffff8000000000                                                   
ELF-loading image 'sel4test-driver'                                             
  paddr=[23c000..502fff]                                                        
  vaddr=[400000..6c6fff]                                                        
  virt_entry=40ecf8                                                             
Enabling MMU and paging                                                         
Jumping to kernel-image entry point... 
[...snip test output...]
Test suite passed. 128 tests passed. 41 tests disabled.                         
All is well in the universe

Lockdown

Wed, 06 May 2020 23:23:44 +0100

It seems right that I should mention the pandemic on my personal blog; I doubt very much that I will say anything original or interesting, but it would seem strange to look back on my writings from 2020 and see nothing about COVID-19.

I had to check this, but I have been staying at home since Monday 15th March, save for occasional walks around the park. (UK schools closed on 20th March, and we officially announced the lockdown on 23rd March, but our house started a bit early due to having a mild cough.) That’s 52 days at home for me so far, or 7 weeks and 3 days.

Yet, due to the miracle of modern technology, I have been working full-time over this period. This has not been stress-free.

We do not suddenly have copious free time; but, still, there have been some highlights. Recently I dug out a couple of Raspberry Pi boards and tackled some challenging (for me) projects - I got seL4 to run, and now I’m learning ARM assembler to debug it (but have concluded I will need a JTAG interface to make progress). At the weekend I programmed a set of red/amber/green LED traffic lights using Scratch while my daughter ignored my feeble attempts at home education.

This pleasant bread-making, home-schooling, online shopping, remote working life contrasts starkly with the reality of what has been happening in our hospitals and care homes. This is the paradox of lockdown - we are isolated, so don’t see the horror. The imagery of war has been overused already; but this crisis can too easily feel like one of those distant wars, in a far-off land that happens to look like our local hospital.

2020

Thu, 02 Jan 2020 12:59:07 +0000

Life comes at you fast. Since changing jobs three months ago, I have earned several cloud-related certifications, and started an assignment as a cloud security architect with a large financial services client.

Consulting is quite different to ordinary employment; a great deal of emphasis is placed on making connections, and building a personal brand. I’m enjoying the variety of work, and the opportunity to develop my skills.

My focus this year is on learning; I intend to spend more time reading and writing, especially as a way to distill and clarify my ideas. Who knows, I might even post more on this blog?

PA Consulting

Wed, 04 Sep 2019 17:37:46 +0100

In early October, I will be saying goodbye to my colleagues at CV-Library after 7.5 years, and joining PA Consulting in London as a Principal Consultant.

Over the course of my time at CV-Library I have got married, had a child, and moved from Southampton to Bedford. I am happy to have played a part in the growth of CV-Library as a leading recruitment brand in the UK, especially helping to make the site more reliable - I can tell more than a few war stories.

Most of all I will remember the people. I still have much to learn about management, but working with such an excellent team, the years passed very quickly. I am grateful to everyone, and wish them all every future success.

My Free Software Activities for Jan/Feb 2019

Thu, 28 Feb 2019 21:55:44 +0000

I have done a small amount of free software work! However, I’m going to cheat and list it since the start of the year.

First, the fun stuff:

I organised the first two meetings of the Bedford Linux User Group. Fire engines were observed on both occasions, but this was pure coincidence.
I sent pull requests adding a fancy map to the new lug.org.uk site. I need to follow up to make that mobile-friendly…

apt security

I sent PRs to whydoesaptnotusehttps.com adjusting the summary and providing instructions on using HTTPS.

I’m planning to extend this with a threat model for apt.

Firefox app mode

I wrote a patch for a bug I’ve been subscribed to for a while, requesting an “app” mode similar to Chrome:

https://bugzilla.mozilla.org/show_bug.cgi?id=1283670

I don’t have much hope that this will actually get reviewed or applied, because it conflicts with a new “earlyBlankFirstPaint” feature. Still, I had fun writing it, and if anyone can tell me how Mozilla development works, I’d be most grateful.

Libreoffice bugs

I filed a fun Writer bug:

https://bugs.documentfoundation.org/show_bug.cgi?id=122862

If you insert a tall enough image into a page header in LibreOffice Writer, you can get it to add pages indefinitely. Impressively, the application remains quite responsive while it tries to flow the text forever…

…no idea how to fix this one though!