Chris Krycho - Tech

Sympolymathesy, or: v5.chriskrycho.com

2019-11-18T20:00:00.000-07:00

Assumed Audience: literally every single subscriber of this blog!

(I apologize if you’re seeing this in multiple feeds, if you are one of the people subscribed to more than one sub-feed on this site. I needed to make sure *all* my subscribers saw this.)

I’ve just officially launched v5.chriskrycho.com, “Sympolymathesy”. As such, this is the final post on this site! For all the details, check out the relaunch post!

Test the Interface

2019-11-13T22:30:00-05:00

Assumed Audience: software developers interested in honing their craft—especially folks just trying to get a handle on good techniques for testing.

A fundamental principle of testing software is: test the interface. Failing to keep this principle in mind is at the root of the majority of the problems I see in automated tests (including quite a few of those I’ve written in the past!).

What do I mean by test the interface? I mean that when you are thinking about what kind of test to write, you can answer it by thinking about how the piece of code will be used. That’s the interface: the place that piece of code interacts with the rest of the world around it. The interaction might be between two functions, or it might be feeding data from a web API into your application to show users data, or any of a host of things in between. The point is: wherever that interaction is, that’s the interface, and that’s what you test.

To see what I mean, let’s define some different kinds of interfaces and how we might test them in the context of a JavaScript application. (I’m using this context because it’s the one I’m most familiar with these days—but the basic principles apply equally well in lots of other contexts.) When we’re writing our app, we have a bunch of different levels of abstraction we can deal with:

the entire application as the user experiences it
individual user interface elements within the application—UI components
functions and classes that manage the business logic of the application

This is actually pretty much it, though each of those covers an enormous amount of ground. Notice too that each of these layers of abstraction (each interface) is composed of lower levels of abstraction (smaller interfaces). However, you still want to test each interface on its own terms.

When you are trying to test the entire application as the user experiences it, you should be doing “end-to-end” style testing, preferably with some kind of testing tool that generates the same kinds of input (from the app’s perspective) as a user would. In web apps, we often use tools like Puppeteer or Webdriver to simulate a user clicking through our UI and filling in forms and so on. This is the right level of testing: we interact with the whole app and its interface the same way a user does!

What we shouldn’t do at this level is use our knowledge of the framework our app is using to go in and replace function calls, or swap out UI components. As soon as we do that, our test stops actually telling us the truth about the interface it’s testing. A user can’t reach in and swap out a function at runtime. If you do that in your tests, then your test tells you something about a fake world you’ve constructed—not the world your user lives in! How do you know that’s the right level to test at? Because that’s the level at which your app interacts with the user: in terms of clicks and form-filling and those kinds of events. Not in terms of function calls!

What about UI components? The same basic principle holds here. The public interface of a component in any modern web framework is its template—whether that’s JSX, Glimmer templates, Vue templates, Angular templates, or something else. How do you know that? Because that’s the level at which the rest of your codebase will use the component. So what you should test is that template invocation. This is the level of a “rendering” test (as we call them in Ember).

The rest of your codebase doesn’t have the liberty (and in most cases doesn’t have the ability) to reach in and change the behavior of the class or function for your component at runtime. All it can do is call that component with its arguments, and work with anything the component hands back to it. If, during your tests, you violate that—say, by reaching in and calling internal methods on the class that backs a component, rather than via the event handlers you set up to trigger those methods—you are no longer testing what you think you are. Again: you’re in a world of your own construction, not the world the rest of your app code lives in. Your test only tells you what happens when you do something manually behind the scenes with the internals of your component… not what happens when interacting with the component the way other code will.

The same basic principle applies for other classes used in your codebase. This is the layer for “unit” tests. For functions, you just pass in the various arguments allowed and check that you’re getting the results you expect. For classes, you set them up using their public constructors and call only their public methods and set only their public fields. In languages like JavaScript, Python, Ruby, and others, you can often poke at and use methods and data on the class which are really meant to be private.¹ That can be particularly tempting when you’re the author of the class: you know what these internal details are supposed to do, after all! It can seem faster and easier to just set up a class with some state ahead of time, or to swap out one of its methods for an easier one to test using monkey-patching or mocking.² If you do this, however, instead of using the documented public API, you’re once again testing something other that what the rest of your app will be using… and this means that once again your tests don’t actually tell you whether the rest of the app can actually use it correctly!

In each of these cases, we need to test the interface—the place where the rest of the world will interact with our code, not its internal mechanics.

This helps guarantee that what we are testing is what the rest of the world sees—whether the “world” in question is other functions or classes, or external APIs, or actual users. It also helps us when refactoring, which is making changes to the internals of some piece of code without changing its public interface. If we test the interface, we can safely refactor internally and know two things: if our tests break, we got our refactoring wrong; and we don’t have to change our tests in the process of refactoring! If we test the internals instead of the interface, though, we’ll often have to make changes to our tests when we’re trying to refactor, because we’ll be changing those “behind the scenes” details.

None of this is obvious or intuitive when you’re just starting out, but keeping the principle of test the interface in mind will help you pick the right kind of test: end-to-end, some kind of rendering/UI test for individual components, and unit tests for standalone “business logic” classes or functions. Hopefully this can help a few of you out there internalize this faster than I did!

JavaScript is getting private fields and methods soon, which will help a lot with this—but the basic principle here will remain important even then, because not everything that’s private in terms of API design can or should be private in terms of the implementation mechanics. This is a question I’d love to dig into… in a future post.↩︎
A related tip—if you find yourself wishing that the implementation were easier to test, and needing to mock or stub parts of it to make it testable, that’s often a sign that your design needs some work!

Note that I didn’t spend much time on functions here because it’s much harder to get yourself into these messes with functions. In most languages, you don’t have any way to reach in and mess with their internals, so you’re safe from a lot of these issues. Inputs and outputs are all you have to work with. This is one of the great advantages to working with a functional style where you can. Use of closures for managing state complicates this story a bit, but even there: less so than with most of the other things discussed here!↩︎

A Note on Attention

2019-11-04T22:00:00-05:00

Assumed Audience: written with an eye to folks willing to think a little bit harder about attention—something we all need to get better at.

Walking through the airport this morning, I found that my attention itself had caught my attention. I have had this bit from L. M. Sacasas bouncing around in my head for the last month or so:

The ability to take photographs expands (and limits) the hiker’s perceptive repertoire—it creates new possibilities, the landscape now appears differently to our hiker. Smartphone in hand, the hiker might now perceive the world as field of possible images. This may, for example, direct attention up from the path toward the horizon, causing even our experienced hiker to stumble. We may be tempted to say, consequently, that the hiker is no longer paying attention, that the device has distracted him. But this is, at best, only partly true. The hiker is still paying attention. In fact, he may be paying very close, directed attention, hunting for that perfect image. But his attention is of a very different sort than the “in the flow” attention of a hiker on the move. They are now missing some aspects of the surroundings, but picking up on others. Without the smartphone in hand, the hiker may not stumble, but they may not notice a particularly striking vista either.

Yesterday’s bit of history might give you to how I ended up walking through Denver International Airport with my camera in hand today, taking in the architecture with a rather different eye than I usually do. My normal path through the airport is merely utilitarian: get to the gate, fill my water bottle, hit the bathroom, grab a seat and knock out some work while I wait to board. Today, though, I found myself considering the structures of the facility, its décor, the little touches architects and designers had put in place to (or at least, to attempt to) humanize what might otherwise be a terribly forbidding structure.

(Airports are among the chief edifices of our modern, technological regime. They are not merely travel hubs, but miniature shopping malls. They require not only the massive infrastructure that supports aircraft themselves, but also the perhaps even greater infrastructure to support the cars we park there, the workers who cycle through, the food to be eaten, and so on. They are, I think, under-appreciated as signifiers of our default approach of technological mastery over the world. More on this, perhaps, another day.)

The very act of carrying a camera with me changed my perceptual relationship to the airport. And why? Because the device around my neck prompted (that is: my choice to carry it with me prompted) a shift in my attention. I left Sacasas’ words above just as he wrote them, focused on the way a smartphone shifts the attention of his hypothetical hiker. The smartphone, by dint of its ubiquity and its versatility, does indeed (and more especially than other devices) shift our attention. Not least, I think, because for most of us, there is no conscious choice to bring it with us, as there was for me with my camera this morning.

The phone both draws our attention and diffuses it. For all that having the smartphone in my pocket may increase my attentiveness to the beautiful vista on a hike, it may also tempt me to disengage entirely from the hike: not only to consider the environment in a different light, but to send my mind across the world to other places, other activities entirely. The contrast with a dedicated device like the camera I brought with me is illuminating. Certainly a camera shifts my relationship to a hike, just as it does to a stroll through the airport. But it does so by way of focusing my attention, and drawing it into aspects of the hike that I might otherwise miss. That focusing is a tradeoff: as Sacasas notes, it means there are things I notice which I might not have had I not had a camera in hand; but also that there are aspects of a hike that I do not enjoy as I would had I no camera to hand—precisely because my attention is focused by the device. But the difference between the way a single-purpose technology affects our attention and the way a device with greater versatility affects our attention is rather stark.

I have an increasing degree of affection for an interest in devices dedicated to a single job. Some of this is simply that, as good as smartphones are at many tasks, a purpose-built piece can often serve better than a jack-of-all-trades production. As my friend Stephen has often lamented: the iPod was a better portable music player—in the sense of doing that one specific job—than any iPhone! I could say the same of my Kobo… and far more for a paper book! I find the experience of using those task-tailored technologies much more enjoyable than our ubiquitous and flexible—and therefore ultimately distracting—smartphones. They shape my attention very differently, and (I think), better.

Apple, Your Developer Documentation is… Missing

2019-10-28T07:25:00-04:00

Assumed Audience: practitioners or interested lookers-on for software development—and Apple itself.

Edit: some folks rightly pointed out that my use of “garbage” suggests that the problem is the quality of the existing documentation; I’ve retitled the post to capture that the problem is the massive absence of documentation. You can see the original title by way of the slug.

Over the past few months, I have been trying to get up to speed on the Apple developer ecosystem, as part of working on my rewrite project. This means I have been learning Swift (again), SwiftUI, and (barely) the iOS and macOS APIs.

It has been terrible. The number of parts of this ecosystem which are entirely undocumented is frankly shocking to me.

Some context: I have spent the last five years working very actively in the JavaScript front-end application development world, working in first AngularJS and then Ember.js. Ember’s docs once had a reputation of being pretty bad, but in the ~4 years I’ve been working with it, they’ve gone from decent to really good. On the other hand, when I was working in AngularJS 5 years ago, I often threw up my hands in quiet despair at the utter lack of explanation (or, occasionally, the inane explanations) of core concepts. I thought that would have to be the absolute worst a massive tech company (in that case, Google) providing public APIs could possibly do.

I was wrong. The current state of Apple’s software documentation is the worst I’ve ever seen for any framework anywhere.

Swift itself is relatively well covered (courtesy of the well-written and well-maintained book). But that’s where the good news ends.¹ Most of SwiftUI is entirely undocumented—not even a single line explanation of what a given type or modifier does. Swift Package Manager has okay docs, but finding out the limits of what it can or can’t do from the official docs is difficult to impossible; I got my ground truth from Stack Overflow questions. I’ve repeatedly been reduced to searching through WWDC video transcripts to figure out where someone says something relevant to whatever I’m working on.²

This is, frankly, unacceptable. In the Ember ecosystem, we have a simple rule that code doesn’t get to ship unless it’s documented. The same goes in Rust (I should know: I wrote the RFC that made that official policy). Now, I understand that Apple’s API developers (often) work in a different context than these open source projects—especially in that they face crunches around releases which are tied to hardware products shipping.

But. At the end of the day, when you’re vending an API, it isn’t done until it’s documented. Full stop.

Given what I know of Apple’s approach to this, the problem is not individual engineers (who are not responsible for writing docs) or even the members of dedicated documentation teams (who are responsible for writing docs). But that does not make it any less a failure of Apple’s engineering organization. The job of an API engineering organization is to support those who will consume that API. I don’t doubt that many of Apples API engineers would love for all of these things to be documented. I likewise do not doubt that the documentation team is understaffed for the work they have to do. (If I’m wrong, if I should doubt that, because Apple’s engineering culture doesn’t value this, then that’s even worse an indictment of the engineering culture.) This kind of thing has to change at the level of the entire engineering organization.

Apple claims to be interested in building a platform that is accessible to everyone—from a brand new developer to the most experienced gray-haired folks who’ve been around since the NeXT days. Right now, they’re not even close. I have a decade of software development under my belt, a fair bit of it in languages with rich type systems and functional programming idioms, and some of this stuff is hard for me to figure out. I can’t imagine how mind-bogglingly terrible the experience would be for someone just starting in software, or coming over with experience only in languages like Ruby or Python or JavaScript. It would be completely impossible to learn.

Apple, if you want developers to love your platform—and you should, because good developers are your lifeblood—and if you don’t want them to flee for other platforms—and you should be worried about that, because the web is everywhere and Microsoft is coming for you—then you need to take this seriously. Adopt the mentality that has served other frameworks and languages so well: If it isn’t documented, it isn’t done.

I am not the only one who has noticed this. No Overview Available summarizes the extent of documentation in Apple’s APIs and… it’s not a good look. Hat tip to Lobste.rs user wink and my friend Jeremy Sherman, who both noted this.↩︎
Credit where credit is due: it is genuinely excellent from an accessibility and general usability standpoint that Apple has these transcripts. However, they’re not a substitute for documentation!↩︎

rewrite Dev Journal: How Progress Doesn’t Feel

2019-10-26T18:30:00-04:00

Assumed Audience: practitioners or interested lookers-on for software development—especially indies.

Today, I had about 3½ hours dedicated to working on rewrite and I did make progress… but it sure didn’t feel like it.

For the past month or so, I’ve been sidelined from working on the project by way of getting very sick and then being swamped with travel for a conference followed by a friend’s wedding. I resolved, however, to get some things done today. I enjoyed that sense of momentum I had in those first couple weeks I was working, and I want it back.

Today’s work, however… felt exceedingly unproductive. It wasn’t. I was learning. Specifically, I was learning how Swift Packages and the Swift Package Manager (SPM) work, how they work with Xcode, and what they can and cannot do. In general, learning like this is necessary and valuable, and this specific knowledge domain is particularly necessary and valuable for me. For one thing, the details of how I’m building the app will make this very applicable very quickly. (More on that in a future update.) For another, because I work best when I have a good understanding of how the whole system works together.

The net of it, though, was that I came out with a good idea of how to use Swift Packages… and zero new lines of code written for the app itself. I may yet make a little progress this evening after my little girls are in bed, but the big takeaway for me today was the need to remind myself that learning is a form of progress. You might think that I, academically-minded and hyper-nerdy fellow that I am, would find that easy to remember. It turns out, though, that it isn’t easy to remember in the context of the desire to actually ship something!

I hope this dev journal entry serves as an encouragement to others doing development work. There’s value in these days. They’re not failures. They’re an integral part of the way you get to the day when you actually ship.

I know this because I’ve been here before. There was a time when I had no idea how I was going to ship that PHP app and I spent whole afternoons figuring out something about SVN—afternoons that did not feel like part of shipping, but which ultimately led me to recover from accidentally deleting my entire codebase. There was a time when I had no idea how a JavaScript “single page application” (SPA) could actually work, and spent multiple afternoons reading about API-driven applications—afternoons which did not feel productive, but ultimately led to the point where I’m in a technical leadership role on one of the biggest SPAs in the world. So I can say fairly confidently that however unproductive this afternoon’s work loading up an accurate understanding of SPM will yield dividends. And sooner than it feels like today!

User Interfaces are API Boundaries

2019-09-12T09:00:00-04:00

Assumed Audience: software developers, especially those who work on user interfaces.

Domain-driven design, and its near neighbor the ports and adapters (hexagonal) architecture all emphasize the importance of distinguishing between your internal “business logic” and your interactions with the rest of the world. Much of the time, the “ports” that get discussed are API calls (e.g. over HTTP) or interacting with a database.

Yesterday, in the midst of a rollicking conversation about building forms in web apps,¹ I realized:

User interfaces are API boundaries, too!

One of the key insights of DDD and the ports-and-adapters model is that every interaction with the world outside your program is a place of uncertainty. The API might have changed and you might be getting back different responses than you expect. The database might have been corrupted. The network might be down—or worse, degraded so that you get partial messages through, and have to deal with incomplete or nonsensical data. Your software design has to account for this. If you isolate the complexity of dealing with that to well-defined, well-constrained boundaries for your application, everything in between can be much simpler.

And the most reliably unpredictable source of data we have for any application… is users! People are complicated and distracted, and our interfaces are always imperfect, often misleading or confusing in various ways (our best intentions notwithstanding). So we get “bad” data from our users. I scare-quote “bad” here because the data is not (necessarily) morally bad (though: see Twitter!) and it is (usually) a mistake rather than malice at root (though: see all sorts of hacking). But from the perspective of our app’s internals, the data has to be validated and transformed to our model of the world, just as data returned from an API or a database does.

If you’re familiar with how these architectures suggest handling sources of data external to your program, the implication for user interaction is obvious: you need to treat it like you would an API. You should have a clean separation between the data model of a form and the data model used within your application. Put in common OO parlance: your form model is a kind of data transfer object.

Notice that this holds whether you’re using a traditional web form which submits a POST request via HTTP, or building a rich SPA-style JavaScript app which will use the form data without ever sending it anywhere. You have to first validate the data to make sure it is complete and correct—presumably with a mechanism for letting the user know if it isn’t. You also normally need to transform the basically flat data you get back from your form into in a data structure which is appropriately rich for the domain you’re working with.

Again: all of this is bog standard for DDD and ports-and-adapters thinking. The point is that you should treat forms specifically and user interaction in general in much the same way as any other external data: because user interfaces are API boundaries.

In a future post, hopefully some time in the next week or two, I’ll trace out one of the implications of this for how I think about building forms in much more concrete terms!

about which conversation more another day!↩︎
I found it fruitful in two senses: it was generative for me as I reflected on it, and it produced some forward motion in a conversation about real-world software development.↩︎

rewrite Dev Journal: How I Started

2019-09-04T21:30:00-04:00

Assumed Audience: practitioners or interested lookers-on for software development—especially indies.

This week, I started the actually writing software phase of working on rewrite. As of yesterday evening, I actually have a bit of code that, although useless in every way, and not especially attractive, does in fact displays a single reference on an iPhone screen. I have an incredibly, impossibly long way to go.

It doesn’t matter: I started.

I had the day off on Monday courtesy of Labor Day, and I chose to spend the chunk of it that wasn’t focused on chores and housework and family time on actually getting this thing moving. I’ve been dreaming for over four years now. That time wasn’t wasted, by a long shot, but as I noted in my announcement post, you can plan forever… or you can just start building at some point.

The problem I had—as I noted back in July—is that I didn’t actually feel like I knew how to start. This a mammoth project, for one thing. For another, doing this kind of indie work requires dramatically improving my design and product chops. And of course, developing for native apps means learning whole new technology stacks as well!

I’ve known all along that the only way to get anywhere was to break the problem down. You know: the same as always in software development. Find something small to start on. I had already identified which part of the app to dig into first: reference management. This is a place where I could have something that is useful to me in reasonably short order (I want to be able to update my own reference library on my iPad!). Even that was daunting, though—still far too large.

So, Monday, I asked: What is the smallest possible reference-related functionality I could build? The answer, I concluded, was: just display an existing library of references. Then I went further, because “display an existing library of references” is surprisingly complicated. It involves, at a minimum:

defining how to display a reference
- …and therefore also designing the data structures representing references
actually building that display
defining how to display a list of references to the user—which things to include, which not to, etc.
actually building that list
making it so that tapping on an item in a list displays the detailed view defined previously
making it so you can get back out of the detail view to the list view again
loading a list of references from somewhere, e.g. a BibTeX file from iCloud Drive
parsing that list of references into the internal data structures designed to represent them

Now, for an experienced iOS or macOS developer, some of those things might feel trivial enough that they don’t even warrant their own bullet point. But I’m not an experienced iOS or macOS developer; I have no idea how to do any of this! What’s more breaking it down that way means that I have small, discrete tasks that I can go after in small blocks of time. Remember: this is a side project, which I am fitting in around my day job, my church commitments, and time with friends and family. I have to be able to make progress in small blocks of time! I have to feel that progress, at least a little, so that I can keep going.

Every bullet point on the list I came up with represents a discrete item of work. It not only may but must be small, so that I can do it in an evening or three. It needs to be done in such a way that later steps can build on it—but it doesn’t actually need to be anything like what those later steps will transform it into. It just needs to be enough to keep momentum moving, and done in such a way that later changes are not too difficult to make.¹

Monday evening, I took the items I had identified and turned them into GitHub Issues in a GitHub project, called Display a BibTeX Library. (I’ll talk about how I’m using GitHub projects a little at some point in the future; it’s a nice flow so far.) Every single one of those items, no matter how apparently small, got its own card. One of them, about Swift-Rust interop, got a note saying, roughly, “I’m pretty sure this will need to be its own whole sub-project.” And looking at that list… I felt a lot less daunted! Every one of the tasks involves something I don’t know how to do yet—but only one thing I don’t know how to do yet, not many.

Last night, I took the first item on the list: display a single reference item. Not parsing them, not even from a string hard-coded into the app, and certainly not loading a file or anything like that. Just hard-coding in a data structure and displaying it on the screen. And, over the course of a few hours, including lots of searching² for answers about specific SwiftUI error messages,³ I managed to implement that one thing.

As I said above: it’s useless in the strict sense. It’s not especially pretty, either. But it’s progress. I started. And now I have that little sense of momentum, and I can keep going!

If you think this sounds like the ideas current in the best parts of agile software development—working to make future changes easy; iterating rapidly; delivering the smallest possible chunks of value, and doing so continuously—you’re not wrong!↩︎
“Searching,” not “Googling,” because I only very rarely Google anything. I search, with Duck Duck Go as my starting point and its ! search commands as power tools for searches in specific places.↩︎
Wow could SwiftUI’s compiler diagnostics use some improvement! I sincerely hope that the Swift team in general and the SwiftUI folks in particular take a close look at Rust and Elm for inspiration here.↩︎

Ember Type-Defs Livestreaming

2019-07-20T15:05:00-04:00

Assumed Audience: People interested in Ember and/or TypeScript.

Over the course of the rest of this year, I’m going to be working regularly on expanding the set of available TypeScript type definitions in the Ember.js community. A huge part of that effort is enabling others to do the work. It’s completely unfeasible for me to be the only person (or even the Typed Ember team the only people) doing this work. There’s simply too much to be done!

One part of my work, then, is figuring out how to level up the community’s ability to take on these tasks. I’m going to be doing this in a few ways:pairing with people who want to convert their addons to TypeScript or write type definitions for them, creating much deeper guides on how to convert a library to TypeScript or write effective types for a third-party library, and—more notably for this post—live-streaming, and sharing recordings of, as many of those efforts as I can.

I’m hoping that this will prove a boon not only to the Ember community, but also to the TypeScript community at large. Many people are comfortable working in these spaces… but especially as TypeScript’s popularity grows, having more of this kind of advanced material will hopefully be a boon to lots of teams in lots of frameworks and contexts.

You can check out the recording of the first live-stream (working on types for a new module layout in Ember Data 3.11) here, and I created a dedicated YouTube playlist for all of this content, as well: Typing the Ember.js Ecosystem!

Here’s the first episode! (Note that I accidentally overlapped the tool I’m using to show keystrokes and the video—noob mistake for sure!)

I expect to be doing this on the following dates almost certainly:

August 16
September 20
October 18 or
November 15
December 20

I will also be doing it a number of other Fridays throughout the year—most likely beginning with August 9. To watch live, you can follow me on Twitch. I’ll also always announce those plans in the Ember Discord at least one day ahead of time, and I’ll shortly be setting up an ongoing thread in the Ember forums for discussion and feedback and the like.

Here’s hoping both Ember and non-Ember users find the materials helpful in picking up TypeScript!

Appearance: Corecursive #34

2019-07-15T16:45:00-04:00

I was delighted to spend a bit over an hour chatting with Adam Gordon Bell on the Corecursive podcast. I was there officially to talk about TypeScript, and I did a lot of that… but we also dug into Rust a bit, of course, as well as talking about my schedule and “productivity”.

I’ve been podcasting for a few years now, but this was only the second time I’ve ever been on someone else’s podcast—and it was a blast. Thanks so much to Adam for having me on!

Announcing rewrite

2019-07-06T21:50:00-04:00

Assumed Audience: people interested in note-taking, research, and long-form writing.

For the past four years, I have been dreaming of an absurdly ambitious application: a tool that can actually handle research writing well. Good research writing—whether it’s a college paper or a Ph.D. thesis, a journal publication or a magazine article, a scholarly blog or a big book—is a complex and challenging task. At a minimum, research writing includes:

finding resources, tracking references to them, and citing them correctly
taking, and making good use of, notes—including quotes from those references
writing large, complex, highly structured documents
publishing in a very specific format for everything from typeface to page margins to citation styles

There are tools out there which handle some of those pieces well—a few genuinely great notes apps in particular, and some solid contenders in the reference management space. They don’t play together well, though—whether it’s sync errors between apps, the disjointed experience of using tools which know almost nothing about each other, fighting with different journals’ styles, or the simple pain of trying to write in what is really a desktop publishing tool (looking at you, Microsoft Word).

And I would know: over the course of my M.Div., I wrote well over a hundred thousand words of papers; and never mind the many books’ worth of notes I took.

I eventually found a flow that worked for me during those years, pulling together Markdown, BibTeX, pandoc, and CSLs to generate Word documents. (I wrote about that here, if you’re curious.) It did what I wanted! But it was arcane: even after years of working that way, I could never remember the various script invocations involved without looking them up. In short: there was something wonderful there… but the user experience was terrible.

Even by the time I wrote that blog post, I was thinking: what would a genuinely great app in this space look like? I had sketched out the basics in a notebook early that month—

the first page of my working notebook

—and I haven’t stopped thinking about it ever since. How do you do reference management well? What makes for a good note-taking system? How do you author large, complex documents with easy reference to those notes? How do you pull all those all together for publishing? How do you make it a great app and make it cross-platform?

For the past four years, I have been doing two things in preparation for building rewrite (the working title for this app):

filling up that same notebook with design constraints, software architecture considerations, and business model ideas
developing the technical skills I’ll need to actually deliver on that dream

Even with those years of planning, though, I’d be lying if I said I feel ready to take this on: this project is huge. It will take years to get to a minimum viable product. But you can plan forever and never ship… or you can just get moving, whether you feel ready or not. I quietly launched a newsletter for the project back at the end of April, and over the past few weeks since getting back from vacation—and having at last wrapped up New Rustacean—I have been working on it in earnest. I started digging into SwiftUI, and have been sketching out more detailed UI ideas, and even decided on what to build first.¹

And so today, I’m publicly “launching” the project, by which I mean: giving it a little more fanfare than I have previously by blogging about it here and by giving it its own website: rewrite.software. For now, it’s just a simple landing page with a form to sign up for the mailing list, mostly there so I have a place to direct people to sign up for the project mailing list. Anything more than that would be a distraction from what I really need to be doing now: learning and building!

I hope some of you will follow along as I take on this absurdly ambitious project, and I expect to be blogging about my adventures in Swift and SwiftUI, Rust, and more in the years to come!

I’m going to try to ship a nice tool for managing references. It’ll be good to have the experience of actually shipping an app. More on that↩

My Final Round of URL Rewrites… Ever.

2019-07-05T10:45:00-04:00

Assumed Audience: web development nerds like me.

Those of you subscribed to my RSS feed most likely saw a bunch of posts again earlier this week. That’s because the canonical URLs for the posts on my site changed: from www.chriskrycho.com/<year>/<title slug> to v4.chriskrycho.com/<year>/<title slug>. So, for example, my announcement that I’m speaking at All Things Open 2019 moved from www.chriskrycho.com/2019/all-things-open-2019.html to v4.chriskrycho.com/2019/all-things-open-2019.html. I spent much of this past Wednesday working on getting this migration done, after spending a fair bit of time over the last week planning it. Over the course of the next few days, you’ll see v1 and v3 start working; v2 is already up as I write this.¹

But why, you ask? Because I now have—at last!—a stable URL design for my website, which will never have to change again. (“At last” I say because I’ve been thinking about doing this since 2015. It feels great to finally have it done.) I care about stable URLs. I want a link to my content to work just exactly as well in 10 years as it does today. Don’t break the web! Don’t break all the documents that aren’t on the web but which point to places on the web! Historically, that has meant that every time I launch a new website design, I have to do a bunch of work to move the previous version of the site and create redirects for it.

No more! From this point forward, my content will always live at a versioned URL. This site is v4.chriskrycho.com. When I launch the redesign I’ve been working on (very soon!), it’ll be v5.chriskrycho.com.² When I launch another redesign in 5 years, that’ll live at v6.chriskrycho.com—and so on. All I’ll have to do at that point is change where www and the root feed.xml redirect to, and everything else will just keep working.

The idea isn’t new to me—I got it originally from someone else; but I don’t remember who because it has been such a long time since I first saw the idea. I had done something somewhat similar when I launched the last version of my site, archiving the previous version at 2012-2013.chriskrycho.com, but I failed to start the new version at a similarly specific location. What this means is that I had to take and redirect every piece of content that lived on what is now v3.chriskrycho.com from www.chriskrycho.com to its new home. Now, as I’m preparing to do the v5 launch, I had to do the same again, but this time for what is now at v4!

I don’t want to do this again! Even with building a small tool to generate either file-based or Netlify redirect rules, getting it right is both time-consuming and error-prone, especially when also needing to do a DNS migration to create v4.chriskrycho.com and get myself off some old shared hosting and… it was a pain and a lot of manual work.³ The new approach means I will never have to do this again, and I cannot express just how happy that makes me.

So: v4 it is for now, and v5 coming soon. When that happens, you’ll see an announcement post in your feed, and then you’ll automatically be switched over to the new root feed on the v5 site, without having to do anything at all. 🎉

For very long-time readers: I also used this as an opportunity to get my old 52 Verses site off of Blogger’s infrastructure and into a purely-static-HTML setup as well. Happily, that one doesn’t involve any URL tweaking—just extracting the content from Blogger and pushing it to a static site host.↩
Feel free to watch that space as I iterate on it! It’s coming together nicely but still has a long way to go.↩
The final redirects file is here, if you’re curious.↩

#EmberJS2019, Part 2

2019-06-17T21:20:00-04:00

Over the last year, the Ember community has steadily delivered on the vision we all traced out in last year’s #EmberJS2018 and Roadmap RFC process, culminating with the shipping-very-soon-now Ember Octane Edition. (All the pieces are pretty much done and are either on stable or will be shortly; we just need another LTS release before we cut a full new edition!)

So… what should we tackle next? This year, I have only two parts, unlike last year’s four (and I’m sneaking them in just under the wire, as today is the deadline for entries!):

Part 1: Let’s finish modernizing the Ember programming model!
Part 2 (this post): Let’s make TypeScript a first-class citizen of the Ember ecosystem.

For the last two and a half years, I have been working off and on towards making TypeScript viable in Ember apps and addons. I was delighted when others came along to help pick up the load, and we’ve charted a course for what we’d like to do over the year ahead. The major priorities we identified all point at what I’ve wanted since I started down this road back at the very end of 2016: for TypeScript to be a first-class citizen in the Ember ecosystem. Here’s my roadmap for how we get there. (Note that here I’m speaking only for myself—neither for the Typed Ember team nor for LinkedIn!)

The Roadmap

1. Execute on Our Priorities

All of us want this to happen. It’s not yet clear what all of our priorities will be in our jobs over the back half of 2019—but if we can, we’d like to see those efforts across the line.

The TypeScript team has eased one of our heavy burdens, by investing in performance monitoring infrastructure over the course of this year and paying close attention to how their changes affect us as well as other TypeScript consumers. We’re deeply appreciative! But there’s still a lot of work to be done that just needs time to actually do the work—reducing churn in type definitions, building type-checked templates, and improving our documentation.

None of those are insurmountable by any stretch. But they’d also be far likelier to happen if they were concretely identified as priorities for Ember as a whole, and we had commitment from the community to help!

To briefly summarize those priorities again:

We need to make it so that consumers of our type definitions do not face breakage from updates to Ember’s types or TypeScript definitions. We already worked out a basic strategy to solve this problem, and I’ve done further exploration to validate that with key stakeholders of large apps (both TypeScript users and apps which want to use TypeScript) and core Ember contributors… but none of us have had time since EmberConf to write out the strategy as a Typed Ember RFC, much less to do the actual implementation work.
We need to make templates type-aware and type-safe. As fantastic as the experience of writing Glimmer components is—and I genuinely do love it!—it’ll be an order of magnitude better when you get autocomplete from your editor as soon as you type <SomeComponent @… and see the names of the valid arguments to a SomeComponent and the types of things you can pass to them. Everyone who has used TSX in a well-typed React app knows just how good this can be. We can make the experience equally great in Ember, while maintaining the authoring and performance advantages of separate templates. Again: we know how to do this (and the TypeScript team is also working on things which may make it even better for us). We just need the time allocated to take the work from prototype to ready-for-real-world-use.
We need to dramatically expand the documentation we provide. Right now, ember-cli-typescript provides a minimal (and very useful!) set of docs. It’s enough to get you started, and if you’re already comfortable with TypeScript you’ll do all right. However, we’d love to provide helpful guides that show people not just the mechanics of using TypeScript with an Ember app, but best practices and techniques and the happy path. There’s a world of difference between being able to run ember install ember-cli-typescript successfully and being able to author an app or addon in TypeScript successfully, and we need to bridge that gap for successful ecosystem-wide adoption!

2. Type the Ecosystem

We also need to do the work—and this simply cannot be done solely by the handful of us that make up the core team—to get types in place for the whole of the Ember ecosystem. Two years ago, I started drafting a blog post outlining a quest to type the ecosystem. I rewrote a good chunk of it last year. I even began working on a draft of the quest in our repository in 2018! But we haven’t actually done it, and while a handful of important addons do now have types, (a) most still don’t, and (b) many of those which do have type definitions could use further iteration to tighten them up to make them more reliable or more useful.

I hope to actually open that quest sometime during the third quarter of this year. If things go as I hope, I will be doing some of that work myself, and I will be building documentation and training materials for others so they can see how to do it, and I will be available for code reviews on conversion efforts. I cannot guarantee that by any stretch—but it is my fervent hope, and there is very good reason to think it may actually come to pass!

In many ways, this also hinges on our ability to provide a good story for reducing churn in type definitions. Just as it’s important that we make Ember’s own types stable for consumers, it’s also important that we help addon developers provide the same kinds of guarantees for their consumers. The entire Ember community takes SemVer seriously, and that means the tools have to support that.

3. Define Ember’s TypeScript Story

If we manage to execute on all the priorities outlined above, then there’s one last step for making TypeScript an official part of Ember’s story: an RFC defining exactly how Ember can officially support TypeScript—including two major commitments:

shipping its own type definitions, with a well-considered approach to SemVer and TypeScript
considering TypeScript a peer to JavaScript in API design decisions

I have an basic frame in mind for how we tackle both of those. The first is by far the more important of the two; the latter is already happening in an ad hoc way and merely requires the former before it can be formalized. But getting to the point where Ember can ship its own type definitions while upholding its semantic versioning commitments and taking advantage of the advances always happening with TypeScript itself is a large and non-trivial task.

It means a substantial amount of work within the Ember codebase. It means building new tooling for Ember’s core development process—so that the experience of working on Ember itself can remain ever-more productive even as we make sure that the types published for consumers are reliable, accurate, and stable. It means investing in both further education of the community and more static analysis tooling, to make sure that breaking type-level changes are not introduced accidentally.

These efforts are worth the investment they will require, but they are serious efforts.

Why It Matters

This is not just me speaking as TypeScript fanboy and promoter. These things matter for TypeScript consumers of Ember—and they do, profoundly. But I would not suggest even that RFC for official support merely to that end. The TypeScript user community in Ember has done extremely well to date without that kind of official commitment, and could continue to do so (as do many other communities in the broader JavaScript ecosystem).

Why, then, do I suggest we make this not just a commitment for our little informal team but for the Ember community as a whole? Because while TypeScript users will benefit the most from these improvements, JavaScript developers will also benefit from them.

When both Ember core and every major addon in the Ember ecosystem has top-notch types available, every other app and addon author will be able to take advantage of those types, courtesy of the integration offered by TypeScript in every major editor. Whether they’re using Vim or Visual Studio, Ember developers will be able to get rich documentation, suggestions, and inline errors—even for their templates! This can be a massive win for developer productivity throughout the ecosystem. Investing to make TypeScript a first-class citizen fo the Ember ecosystem will make the experience of authoring Ember apps and libraries better for everyone. So let’s do it!

#EmberJS2019, Part 1

2019-06-17T20:25:00-04:00

So… what should we tackle next? This year, I have only two parts, unlike last year’s four (and I’m sneaking them in just under the wire, as today is the deadline for entries!):

Part 1 (this post): Let’s finish modernizing the Ember programming model!
Part 2: Let’s make TypeScript a first-class citizen of the Ember ecosystem.

The Octane Edition represents the delivery of several years worth of work and experimentation. It represents a willingness to say “no” to many good efforts and things Ember needs to continue succeeding. All of that is very much to the good! It’s precisely what I and many others called for last year.

This year, it’s time to deliver on a number of other long-standing goals of the Ember effort. That means:

a modernized build-system, and with it the long-promised “svelte” builds, tree-shaking, and the ability to npm-install-your-way-to-Ember
a modernized routing system, leaving behind the final bits of cruft from the Ember 1.x era and fully-embracing the component-service architecture suggested last year

Modernized Build System

Others have covered the build system in some detail, and I largely agree with their assessments. We do need to focus on landing that and continuing to modernize our build pipeline, and the Embroider effort and everything it unlocks should absolutely be a core part of the roadmap. One of the biggest ones, for many potential adopters of Ember and many existing Ember users who have large codebases they’d like to migrate into Ember is that long-awaited npm-install-your-way-to-Ember story. Let’s make that happen! I’m confident that as long as we make that commitment, we’ll get it done.

Given that confidence, I’m going to focus for the rest of this post on the directional question with our routing system—both why we need a change and what I think the change should look like.

Modernized Routing System

The Ember Router was years ahead of its time, and it remains very solid and reliable; it’s a workhorse. Unfortunately, the routing system as a system is showing its age, and has entered something of a period of instability of just the sort Editions are meant to address. Today, routing concerns are spread across four different parts of the application: the route map, route classes, controller classes, and the router service. Over the next year, we should iteratively design and implement our way toward a future without controllers… and possibly some other simplifications, if we can manage them. We can make working with Ember simultaneously easier for newcomers and better for old hands.

“Controllers are dead!” is one of the great bogeymen of Ember lore at this point; but I hope quite sincerely that a year from now it’s basically true. Controllers are the single part of Ember today that shows very clearly the application’s SproutCore roots. When I started learning AppKit and UIKit early this year, I was struck by all the things that sounded like Ember 1.x—or rather, vice versa! And just as Apple itself is now moving aggressively toward a programming model without controllers, so should we!¹

It’s not that controllers are bad, exactly. It’s that they don’t fit with the rest of the framework at this point. They’re long-lived singletons (like services) but serve as backing classes for templates (like components). They are eagerly instantiated as soon as a LinkTo component with references them is instantiated, but not set up until the app transitions to the route associated with them. They’re required if you want to use query parameters, and query parameters don’t work with Ember’s “data down, actions up” paradigm… pretty much at all.

Controllers need to go, but they need a well-designed set of replacements—in particular, we need a good design for query parameter handling and for what template should be associated with a given route.

Query param handling is, I admit, mostly outside my wheelhouse. All of the situations where I’ve used it would be trivially solved by putting them on the router service, tracking changes on them with the @tracked decorator, and updating them with actions. However, I’m reliably informed that some of the more gnarly scenarios out there require a bit more than this, and I defer to the folks who know what they’re talking about there!

React and Vue simply solve the template problem by mounting components at given route locations. Ember should probably follow roughly the same path, while baking in good defaults along the way. Don’t call them “routable components” though! It’s not just that it’s too much baggage; it’s that a good design in this space should not require the components themselves to be anything special at all. Instead, whether it’s part of the route map or the router class grows a small bit of new, purely declarative API—e.g. static class properties specifying the relevant components for the loading, resolved, and error states of the route’s model—the route itself should be able to specify exactly what component to render.

If we put in the work to get a design that satisfies all these constraints, we can come out with a much simpler routing system—and Ember’s entirely programming model will be much more coherent as a result.² We’ll simply have components, services, and routes—and routes will simply be a mapping from URL to a particular set of data and a corresponding component to render it into. That in turn will take us most of the rest of the way toward the programming model Chris Garrett proposed a year ago: Ember as a Component-Service Architecture. This is the fitting conclusion to what we started in Octane: bringing the whole Ember programming model into coherence.

Bonus

I’d also like to strongly commend my friend Dustin Masters’ post, The Case for Embeddable Ember. Call this a stretch goal: if we ship all the build pipeline elements represented above, the extra work required to get to that point is relatively small—and extremely valuable for many teams who want to replace legacy applications written wholly with Backbone, jQuery etc., or who just want to see if Ember might have value to add without doing a full rewrite of their existing React/Vue/Angular/Aurelia apps.

Oh… and it turns out that the design constraints I suggested for a routing system that works well with components would lead fairly nicely and easily to Dustin’s proposal, and make for a straightforward path to fully adopt Ember and map its component tree to the router when you’re ready. Just saying.

You can expect to hear a lot more from me about Swift UI in this space, both in a general sense and as it relates to Ember. There are some fascinating points of contact between the two programming models!↩
There may also be opportunities for further simplification past this, along with more substantial rethinks of our router as we have it. But those are not necessary for the next year, and making these changes will unlock further experimentation in that direction while making Ember more usable in the meantime.↩

From My Sent Folder: On Mozilla and IRC

2019-04-30T08:25:00-04:00

Assumed Audience: anyone who cares about the success of free and open-source software.

A New Rustacean listener sent me a note lamenting the way Mozilla’s transition from IRC to Discord—i.e., from an open protocol to a proprietary service. For many advocates of free software, this is a deeply unsettling move. The kind listener who sent me an email pointed to Matrix, an open-protocol service, and noted: “perhaps the default web interface is slightly less slick, but there’s not much in it, and it’s open source.” I sympathize with them, but also had a few other thoughts. My reply is reproduced below.

Yeah, it’s complicated for sure. The last time I looked, Matrix’s mobile clients were so bad as to be effectively unusable; I know that was one of the considerations for the Rust move to Discord.

It seems to me that a lot of folks committed to free and open-source have ended up wanting free-as-in-beer as well as free-as-in-speech, and the combo of that with the existing market dynamics means that good is rarely open/free-as-in-speech, and vice versa. It’s a Gordian knot I don’t see an easy way through.

The specific dynamic you highlight with docs is a prime example of the other cultural challenges I see, too: devs don’t like writing docs, they like writing code… but it turns out that docs and other “soft” things—including, yes, UI polish!—are actually as or more important for adoption as things like protocols.

We’ll see this same cycle repeat until free and open source software advocates learn to prioritize the things users actually value as well as the things they (we!) believe they should value.

EmberConf 2019 Typed Ember Team Report

2019-03-26T15:30:00-04:00

Assumed Audience: People interested in Ember and TypeScript.

One of the real joys of working on ember-cli-typescript over the last few years has been the team that has grown up around it. When I started, it was just me—trying desperately to just make things work at all and blogging about it as a way of documenting what I had learned and maybe, just maybe, drumming up interest along the way. No longer! Over the last two years, those efforts have grown into a group of us—Mike North, Dan Freeman, James Davis, and me—steadily pushing forward the state of the art for TypeScript in Ember.¹ This week, the four of us were all at EmberConf 2019, and we ate dinner together the last evening of the conference—turning it into an impromptu discussion of the current state of affairs and what’s next on our plates.

Notes on Our Meeting

The following is a summary (which all of us signed off on as accurate) of what ended up being a very wide-ranging conversation about the state of TypeScript in the Ember ecosystem.

Reducing Churn

We have made very good progress getting the types to the generally very-good state they’re now in, including mostly insulating Ember users from the breakages in types that come up at times as TypeScript itself changes. However, the single most significant challenge for our efforts remains finding a way to change this story from mostly insulating users to genuine reliability of the sort Ember users are used to. An initial period of churn was to be expected given the amount of ground we had to cover. However, we are rapidly moving out of the early adopter phase for this tooling, and our responsibility to shield users from churn is increasing commensurate with that shift.

This is not a small task, because Ember and TypeScript have diametrically opposed views of the world when it comes to Semantic Versioning and tooling stability.

On the one hand, Ember is deeply committed to SemVer—more than nearly any other tool we’re aware of.² As a result, the Ember community expects tooling and libraries to have similar commitments to backwards compatibility, stability, and clear deprecation warnings and migration paths when there are breaking changes.

On the other hand, the TypeScript team wholly disavows the validity of SemVer for a compiler in particular: They take the view that any change to a compiler is effectively a breaking change, since even a bug fix will certainly cause someone’s code to stop compiling. (See especially these two comments: 1, 2.) As a team, we understand the TypeScript team’s position, but broadly land on Ember’s side in this discussion—no surprise there!³

Although we need to clearly articulate for our users the reasons for churn when churn does exist, reconciling these two opposed paradigms is not merely a matter of documentation. We also need (more) tooling to bridge the gap between them. A few examples of the kinds of things we need to address:

We need to be able to map type definitions to specific versions of Ember and TypeScript. For example, our types today do not have any way to distinguish between how computed properties resolve (and therefore can be accessed) in Ember <3.1 and Ember ≥3.1 for the traditional EmberObject model—that is, do you need to do this.get('someProperty') or can you simply do this.someProperty? We’d like to be able to offer that kind of granularity.
We need to guarantee that the Ember sub-packages (@ember/object, @ember/array, etc.) specify their own dependencies on each other correctly. In our current flow and structure with DefinitelyTyped, these occasionally get out of sync, with changes in the packages getting published in an order that makes it difficult for them to resolve each other correctly.
We need to help users avoid the problem of having multiple versions of the type definitions installed as a result of yarn’s and npm’s strategies for preventing unwanted transitive dependency changes—strategies which are basically correct for everything except type definitions, but which are the opposite of what type definitions require.⁴ Current strategies (mostly involving Yarn’s resolutions field in package.json) work but are fragile, error-prone, and difficult for end users to automate.
Perhaps most importantly, we need to be able to insulate end users from changes in TypeScript that break the existing type definitions. Minimally, this probably means specifying the versions of TypeScript with which they’re compatible. TS already provides some tooling for this—but we have not yet started using that tooling, and we may need some layer on top of it. Our end goal will be to make it possible for users to reliably know when they can safely upgrade between TypeScript versions and what version of the type definitions they will need.

We batted around a number of ideas as a group for how to address those issues, and we think we have ideas for how to solve all of them. You can expect to see a Typed Ember Roadmap RFC forthcoming in which we synthesize those thoughts into a coherent plan for solving these pain points, pending further discussion and an idea of when and how we can allocate our own time toward those efforts.

Typed Templates

The other major pain point for TypeScript adopters in Ember has been that no type-checking occurs in templates. This means that a substantial part of all apps and many addons is completely outside the TypeScript world and the benefits it brings—type checks around invocation, autocomplete, refactoring, etc. The same is true of everything except JSX/TSX, it turns out. People have written workarounds for Vue and Angular, but they are just that: workarounds, rather than first-class citizens of TypeScript the way TSX is.⁵ We know how to implement that kind of workaround ourselves—Dan implemented a basic prototype last year, in fact!—and we plan to work out the remaining details and do so… and more!

“Remaining details” is an important part of that, though.

We want to make sure that we build that in such a way that it can be generalized to other kinds of files (e.g. CSS for use with CSS Modules).
We will need to work out what the story will be for integrating with editor tooling.
We need to make sure that any solution we implement will integrate nicely with the recently proposed template imports and single-file components primitives.
We need to identify what they will and won’t be able to type-check effectively, and document those constraints.
Last but not least, we need to make sure the implementation will be relatively straightforward to to rework if TypeScript eventually exposes a first-class hook for us to provide it this kind of information.

These are obviously not trivial problems to solve. More details will be in the aforementioned roadmap RFC when it appears!

Improving Documentation

We also need to substantially expand our documentation in general. We made a decision not to ship ember-cli-typescript 2.0 without having a proper documentation site in place,⁶ and we finally got that foundation in place last week, with an ember-cli-addon-docs setup. We converted the existing (far-too-long) README, updated the pieces which were out of date, and added notes for upgrading from ember-cli-typescript 1.x. However, the documentation as its stands is a starting point. We need to expand it to cover (at least):

successful strategies both for starting new projects and for migrating existing projects
working with types effectively
writing type definitions effectively
managing your public API as a library/addon author
standard troubleshooting and debugging techniques
explanations for the mechanics of some of the unusual workarounds we have implemented

And no doubt that’s just the tip of the iceberg. If that sounds like a lot, that’s because it is! (If it sounds like a lot of that will be useful to the broader TypeScript ecosystem, that point isn’t lost on me, either.)

Other Concerns on our Radar

There are a number of other concerns on our radar which we didn’t explicitly talk about at our impromptu meetup but which we have discussed at other times over the last few months:

Performance: we have occasionally seen massive regressions in the performance of TypeScript against our apps and types, and we’d like to prevent that happening in the future. At the moment, we don’t even have good benchmarks to point the TypeScript team to—we’ll need to work out a good strategy here.
Education: as members of the Ember community get excited about adopting TypeScript and start adopting it, we want to help them be successful—avoiding common pitfalls, making libraries/addons easy for others to use safely, and so on. In particular, if you are converting an addon to TypeScript or adding type definitions for it, please involve us so we can help you stay on the happy path and author your types in a way that is stable and usable for the rest of the community!
Providing a type-safe story for Ember Concurrency: currently, the various workarounds available for using Ember Concurrency with TypeScript require you to throw away type safety in several places, because TypeScript does not understand the type-level transformation applied by decorators and cannot resolve the types correctly in the traditional .extend block. We’ve been working on a strategy to solve this with with Jan Buschtöns (@buschtoens), involving a bit of a clever hack with a Babel transform that none of us love, but which will get the job done.
Supporting Ember core’s adoption of TypeScript: much as with the community in general, we want to help make sure that as core Ember libraries adopt TypeScript, they’re able to do so successfully—both in terms of gaining utility to their own projects, and also so that someday we can see…
Ember shipping its own types: we would love to get to a point where Ember itself ships its own types natively. However, we all see many of the other points in this document as effectively being prerequisites for that to happen successfully—especially the issues around stability and semantic versioning.

Onward!

In many ways, the ember-cli-typescript 1.x era was the MVP era for Ember-with-TypeScript. We made it possible for teams to successfully adopt TypeScript in their Ember apps and addons, even in large codebases. Early adopters in this era have enjoyed many upsides—but have also had to eat higher costs as we tried to stabilize the foundational tooling and types for the ecosystem. In the 2.x era, we aim to eliminate that churn and make the experience best-in-class, in ways that benefit the rest of the Ember ecosystem whether or not users are adopting TypeScript.

Someday, we’d love for TypeScript support to be so good everywhere in Ember that our team is out of a job. There’s a long way to go to get there, though—we could use your help! Come find us on GitHub and in #e-typescript on the Ember Discord and help us make this happen!

We also want to mention—and thank!—Derek Wickern, who did a heroic amount of work to write types for Ember in 2017, and even though he is no longer working in Ember projects, he kindly still contributes especially by way of answering questions in Discord now and again.↩
The only comparable ecosystems I’m aware of are the Rust and Elm programming languages. Rust has an official RFC defining its commitment to and definition of SemVer for the language and packages within the ecosystem. Elm has a similar commitment for packages—enforcing its definition with integration between the package manager and the compiler—while still being in a 0.x mode for the language.↩
I may write more on this from my own perspective in the future; I leave that aside here as this post aims to accurately present of the team’s conversation and perspective, not just mine!↩
I added documentation with details on the current state of affairs around this specific issue to the docs site!↩
The same is true for Vue and Angular; we hope that the things we do here will help those communities as well.↩
I have long been of the mind that an open source project is simply not done until it has at least a reasonable baseline of documentation, and was delighted to learn that the rest of the team shares that mentality.↩

Announcing ember-cli-typescript v2.0

2019-03-13T16:00:00-04:00

Assumed Audience: People interested in Ember and TypeScript.

I’m pleased to announce that the unofficial Ember TypeScript team (Dan Freeman, Derek Wickern, James Davis, and Mike North, and I) have just published ember-cli-typescript v2.0.0!

Check out the upgrade instructions to get started with the new version!

What’s new?

There are just two changes, but they’re a big deal!

The addon now uses Babel 7’s TypeScript support to actually build your TypeScript, while continuing to use TypeScript itself to type-check your app. We added some fancy new build errors to go with that, too! This means your builds will be much faster and that tools like ember-auto-import will Just Work™ with TypeScript apps and addons now. (There are a few caveats that come with this, so please see the release notes!)
We added a documentation site! You can check out the documentation at typed-ember.github.io/ember-cli-typescript. Previously, the README was over 6,000 words long… and growing. Now, the README just has the basic stuff you need to get started, and documentation lives in… the docs! Thanks to the ember-cli-addon-docs crew for making it so easy to build such a nice documentation site!

We covered the Babel 7 changes in detail two earlier blog posts:

ember-cli-typescript v2 beta (me)
ember-cli-typescript v2 release candidate (Mike North)

Happily, nothing has changed in the implementation since the beta or RC releases except fixing some bugs!

For my part, thanks to Dan Freeman, who did the lion’s share of the work to get us working with Babel 7!

On behalf of the whole unofficial Ember TypeScript team, thanks to everyone who tested this before we released, identified gaps in the docs (not to mention catching errors and typos there!), and provided feedback along the way. There’s more good stuff coming in the months ahead!

My Current Setup

2019-01-19T09:00:00-05:00

Assumed Audience: people interested in productivity and work flow and app choice.

A friend was asking me the other day what my workflow looks like, and while I don’t generally spend a lot of time writing about my working setup, I figured I’d throw up a quick post representing my current list so that if people ask I can point them here.

Two important notes on this, though: First, this is just what I use. I make no particular claim that it’s the best. There are lots of things here that are very specific to me and the way I work, and even to my specific mental quirks. Second, it’s far more important to care about the work you do than about the tools you use to get it done. The tools matter: some people say they don’t and I don’t think that’s right at all. But they don’t matter as much as they might feel like they do, and tool fetishism is real. I think the happy point is finding tools which are good enough and fit comfortably enough into your workflow that they don’t distract you, and then get back to the work you do!

Software development

For software development, I currently use:

VS Code as my primary text editor; I also occasionally use Sublime Text or Vim for specific tasks where it makes sense. Code is incredibly fast, impressively low in memory usage given it’s an Electron app, and remarkably customizable. My only outstanding complaint is that there’s no way to actually make it look like a native macOS app. Happily, I can make it behave like a native macOS app in all the ways that matter to me. Its support for both TypeScript and Rust—the two languages I spend the most time with right now—is great. You’re welcome to see my full configuration; I keep it updated at that location via the Settings Sync plugin.
macOS’ built-in Terminal app, just using its tabs for individual tasks. I have spent a lot of time with alternatives, including iTerm 2 and kitty, and I’m comfortable in tmux – but at the end of the day I just like the way Terminal feels the best. It’s fast, light, and built-in macOS things all just work correctly out of the box.
Fork for a git GUI. I’ve also used Tower in the past, but I’ve found Fork to be lighter, faster, and a better fit for the way I think and how I expect things to behave (e.g. for interactive rebasing). I do a ton of work in git on the command line as well.
A mix, varying day by day, of Safari Tech Preview, Firefox, and Chrome for my test browsers. I substantially prefer Safari in nearly every way, but Chrome’s dev tools remain best in class for most things—with the exception of Grid, where Firefox is still the undisputed champion. (When I can, I do most of my JavaScript/TypeScript debugging in VS Code, though: it’s a much better experience.)
Dash for offline documentation access. There are other options out there, including some which are free, but Dash remains the best in my experience, and if nothing else it’s deeply integrated into my workflow and muscle memory.

That’s basically the whole list at the moment. I keep my software development workflow fairly light.

Research and writing

For research and writing, I use:

iA Writer for all my blogging and longer-form writing. This is a recent change; I had been a heavy user of Ulysses for the last few years. However, while Ulysses was the best option I had found for how I work, it’s never been quite right for me. For one, I’ve always found Ulysses’ “feel” to be a bit heavy: it’s just noticeably slower for me than most of the other writing apps, and I’m hypersensitive to input latency. I’ve also always disliked one of the things that makes many people love it: the way it abstracts Markdown into its “smart text objects.” iA Writer gives me the ability to manage a library in much the same way that Ulysses did, but in a way that feels more truly native on both macOS and iOS; it just uses normal Markdown (no fancy text objects); and it’s fast.¹ That combo makes it a better fit for me.
Bear for my note-taking. I’ve talked about this a fair bit here before, so I won’t belabor the details. It’s an excellent, lightweight note-taking app. The main way it falls down for me is that it does not really handle nested block content in Markdown documents (e.g. it won’t correctly display a block quote inside another block quote, or a code sample inside a list, etc.). I’d also love it if it stored its library in a way that made it easier for me to interact with from other apps, i.e. as plain text on the local drive. (You can export content from it easily enough, which is great, but it’s not quite as seamless as I’d like.) Those nits are just that, though: nits. I’m very happy with Bear for my note-taking at this point.
Unread on iOS for reading RSS, with Feedbin as my long-preferred RSS feed service backing it. (I’m a fan of paying for these kinds of services, so I’m happy to lay out the $5/month for Feedbin. Free alternatives exist, but I don’t love ad-driven models and avoid them where I can.)

You’ll note that there are no apps for reading longer material on that list. I could mention Apple Books as the place I read most ebooks I read, but that’s more a function of the alternatives not being meaningfully better in the ways I care about.

“Productivity”

For “productivity” concerns, I use:

Tadam for a Pomodoro timer—because it’s precisely as annoyingly obtrusive as I need it to be, which is very!—and Bear for tracking what I do each Pomodoro cycle, each day, each week, each month, and each year. That habit remains very helpful for me.
Things for a to-do app. Things hits the sweet spot for me in terms of its ability to manage everything from simple tasks and recurring to-do items around the house up to complex multi-month-long projects. I particularly like its distinction between when I want to be reminded about something and when that task is due. I’ve used OmniFocus in the past, but it never quite fit me; Things does. They’re very comparable in terms of features; it’s just that the way Things approaches those features works better for me.
Spark as my email client, mostly for its snooze feature, which I use when I know I need to see an email as an email sometime later, and its ability to integrate nicely with other apps. I have it connect to Things, and emails that require an action get put there instead of snoozed. The combination lets me keep my inbox at Zero by the end of every day. And its lovely “Inbox Zero” images are a really nice touch:

Inbox Zero in Spark

Podcasting

For podcast production, I use:

iZotope RX (I’m using v6 Standard) for audio cleanup, as I recently wrote about.
Logic Pro X for the actual editing work most of the time, occasionally using Ferrite. Logic is overkill for what I do, but I’m fast with it at this point, so I can’t see moving anytime soon, and there’s nothing else out there that I think is substantially better (though there are other apps that are comparably good).
Forecast for encoding and including chapter breaks.
Feeder for generating the RSS feeds, since all my podcasts are currently built in ways that don’t support RSS feed generation: Pelican for Winning Slowly and Mass Affection,² and rustdoc for New Rustacean. (If I ever manage to finish building my own site generator, it’ll have out-of-the-box support for custom RSS feed templates, so that I can have this stuff generated automatically for me!)
Netlify for serving the actual static site content (i.e. HTML, CSS, and JS), and Backblaze B2 for hosting the audio.
Transmit for actually uploading the audio files.

iA Writer seems to get randomly slow at times for reasons I haven’t yet identified, but at least for now, I’m taking that tradeoff over Ulysses’ habit of being a bit slow all the time. As I’ve often noted before: my ideal writing app doesn’t exist.↩
Mass Affection isn’t dead! I promise!↩

JavaScript is C

2018-12-20T18:45:00-05:00

Assumed Audience: software developers, especially those interested in modern, typed programming languages.

Earlier this week, I was working on a problem in the Ember app where I spend most of my day job, and realized: JavaScript is the same as C.

That probably doesn’t make any sense, so let’s back up. The scenario I was dealing with was one where there was a bit of invariant around a piece of data that I had to maintain for the application not to blow up in horrible ways, but had no good way to enforce with the language’s tools. This action on that piece of data was only valid if this condition held true… but even with the fully-type-checked TypeScript application we now have, the action (because of the entire application’s architecture and indeed the entire way that Ember apps are wired together!) could not be statically verified to be safe.

As I considered the best way to handle this—I ended up having the function involved in the action just throw an error if the invariant wasn’t properly maintained—I was reminded of the years I spent writing C. In C, it’s quite possible to write safe code around memory management. I managed it fine in the applications I worked on, by carefully documenting the invariants a given function required to be safe. This piece of data is allocated by that function and then released to the caller to manage. Even with every bit of static analysis I threw at those kinds of things, it was possible to get it wrong.

The exact same kinds of problems I had in C, I have in JavaScript or even TypeScript today. Experientially, JavaScript¹ is C, as far as having to deal with these kinds of invariants goes.²

Enter Rust: the kinds of management of memory that I was always having to keep track of in my head (or, better, with detailed documentation comments along the way—but with the same problem that it was easy to get wrong), I could now have statically guaranteed by a compiler. Given that I spent the first six years of my career managing and carefully tracking all of that by hand, it’s no wonder I fell in love with Rust. I could have the compiler guarantee the invariants I needed around memory management.

And it turns out, this same dynamic exists in the world of front-end web development. People sometimes wonder why (and colleagues are often bemused that) I get so excited by Elm. But the step from JavaScript (or even TypeScript) to Elm is just like the step from C to Rust. It’s a real and profound shift in what kinds of things you can know for certain about your program.

In a C application, try as hard as I may, at the end of the day I am always on my own, making sure the invariants I need for memory safety hold. In Rust, I can be 100% confident that I will not have memory-unsafe code. Not 98%-and-I’d-better-check-those-last-2%-really-closely. One hundred percent. That’s a game-changer.

In a JavaScript or TypeScript application, try as hard as I may, at the end of the day I am always on my own, making sure the invariants I need for state management hold. In Elm, I can be 100% confident that I will not have code which needs a given invariant about a piece of state to hold break the way it could in this TypeScript application. Because I can’t even apply the relevant transformations in question if it isn’t! That’s a game-changer.

Neither of those is a guarantee I won’t have bugs. (A compiler that could guarantee that would have to be sentient and far smarter than any human!) Neither of them means I can’t intentionally do stupid things that violate invariants in ways that get the program into broken states from the user’s point of view. But both of them give me the tools and the confidence that I can absolutely guarantee that certain, very important kinds of invariants hold. We’re not looking for an absence of all bugs or a system which can prevent us from making any kind of mistake. We’re looking to be able to spend our times on the things that matter, not on minutiae the computer can check for us.

So: I’m not going back to C, and I’m ready to move past JavaScript and TypeScript.

This goes for plenty of languages that aren’t JavaScript, too. It’s equally true of c^♯ or Python.↩
Obviously there are some kinds of things you don’t have to worry about in JS that you do in C: memory management, for one. The point is that the manual-verification-of-every-invariant-you-care-about is the same.↩

Internal and External Parameter Names in JavaScript and TypeScript

2018-11-26T20:25:00-05:00

Earlier this month I was working on a fairly thorny problem for work—taking a total value and splitting it into numbers which summed up to it, possibly including with a rule about what the split-up values had to be a multiple of. E.g. you want to order 50 Buffalo wings, and you have to choose the flavors for the wings in increments of 5.

I spent a lot of time thinking about the implementation of the algorithm for that, but I also spent a lot of time thinking about what its API should look like. Here, it’s the latter I want to dive into (the former is a little tricky but not all that interesting).

I started out with just simple parameters to the function:

function splitNicely(
  total: number, components: number, factor?: number
): number {
  // the implementation
}

This is nice enough to use internally. But calling it is pretty confusing:

const result = splitNicely(50, 5, 2);

Which number is what value here? Who knows!

So then I just exposed all of the items as an options hash:

interface SplitArgs {
  total: number;
  components: number;
  factor?: number;
}

function splitNicely(
  { total, components, factor }: SplitArgs
): number {
  // the implementation
}

This was a lot nicer to call:

splitNicely({ total: 50, components: 5, factor: 2 });

However, it was a bit verbose, and I realized that it’s fairly obvious that the first argument should be the value we’re splitting up, so I simplified a bit:

interface SplitArgs {
  components: number;
  factor?: number;
}

function splitNicely(
  total: number,
  { components, factor }: SplitArgs
): number {
  // the implementation
}

Now calling it read relatively well:

splitNicely(10, { components: 5, factor: 2 });

However, the names were not my favorite for invoking the function. Really, what I wanted was for the function invocation to describe what I was doing, when reading it from the outside—while having these useful names for operating on the implementation internally.

At this point, I remembered two things:

Swift and Objective-C have the nice notion of internal and external parameter names.
JavaScript (and thus TypeScript) let you rename values in “destructuring assignment.”

The second one lets us get the same basic effect in JavaScript or TypeScript as we get in Swift, if we’re using an options argument! Here’s how destructuring works in the function definition. Let’s see it first with just JavaScript. The object passed as a parameter has a key named of, which has a string value—but of is a bad name inside the function; there, we can just call it str and it’s perfectly clear.

function length({ of: str }) {
  return str.length;
}

console.log(length({ of: "waffles" }));  // 7

That’s the equivalent of a function that looks like this:

function length({ of }) {
  const str = of;
  return str.length
}

Here’s the same code but in TypeScript:

function length({ of: str }: { of: string }): number {
  return str.length;
}

console.log(length({ of: "waffles" }));  // 7

This is a big more annoying to write out in TypeScript, because we need to supply the type of the whole object after the object we’ve destructured, but the effect is the same once we get past the declaration. It’s also pretty silly to do this kind of thing at all in this example—but it becomes much more useful in more complicated functions, like the one that motivated me to explore this in the first place.

Recall that I liked having components and factor as the internal names. They weren’t great for calling the function, though. After some consideration, I decided invoking the function should look like this:

splitNicely(10, { into: 5, byMultiplesOf: 2 });

By using the destructuring technique, we can get exactly this, while keeping components and factor internally:

interface SplitArgs = {
  into: number;
  byMultiplesOf?: number;
}

function splitNicely(
  total: number,
  { into: components, byMultiplesOf: factor }: SplitArgs
): number {
  // the implementation
}

This is a great pattern to put in your toolbox. You can of course overdo it with this, as with any technique, but it’s a nice tool for these kinds of cases where you really want to make an expressive API for both callers and the internal implementation of a function.

ember-cli-typescript v2 beta

2018-11-19T21:30:00-05:00

A few weeks ago, the Typed Ember team published the first beta releases of ember-cli-typescript v2 (currently at beta 3). The new version brings much faster and more reliable builds, with nicer error reporting to boot.

In this post:

Testing the Upgrade
Under the Hood

Testing the Upgrade

I emphasize: this is a beta release. We have tested it in two large apps (including the one I work on every day) and a number of addons, and it seems to work correctly, but that does not mean it is ready for production. We need your help to get it ready for production. Accordingly, please do test it in your apps, and please do not run it in production! (If you do, and something breaks, we will not feel bad for you! You have been warned!)

This upgrade requires that you be using Babel 7. As such, we suggest you start by doing that upgrade:

ember install ember-cli-babel@7

There may be a couple things you find different with your app with Babel 7, so resolve those first. Then, and only then, upgrade ember-cli-typescript.

To test the ember-cli-typescript v2 beta in an app:

ember install ember-cli-typescript@beta

To test it in an addon:

ember install ember-cli-typescript@beta --save

(This will add the addon to your addon’s runtime dependencies, not just its development-time dependencies, which is necessary because of the changes we made to how the build pipeline works. See below for details.)

Then run your test suite and poke around! Note that on your first run, there are almost certainly going to be some things that break. In both of the large apps this has been tested in, there were a number of small changes we had to make to get everything working again. (We’re marking this as a breaking semver change for a reason!)

In general, most apps will only have a few places they need to make changes, but because a certain kind of imports are included in the things affected, you may see your test suite explode. Don’t panic! You will probably make a couple a couple changes and see everything go back to working again.

Here are the changes you need to know about:

We now build the application using Babel 7’s TypeScript plugin. This has a few important limitations—some of them bugs (linked below); others are conscious decisions on the part of Babel. The changes:
- const enum types are unsupported. You should switch to constants or regular enum types.
- trailing commas after rest function parameters (function foo(...bar[],) {}) are disallowed by the ECMAScript spec, so Babel also disallows them.
- re-exports of types have to be disambiguated to be types, rather than values. Neither of these will work:
```
export { FooType } from 'foo';
import { FooType } from 'foo';
export { FooType };
```
  In both cases, Babel attempts to emit a value export, not just a type export, and fails because there is no actual value to emit. You can do this instead as a workaround:
```
import * as Foo from 'foo';
export type FooType = Foo.FooType;
```
Other bugs you should be aware of:
- If an enum has a member with the same name as an imported type (babel/babel#8881), it will fail to compile.
- this types in ES5 getters and setters do not work (babel/babel#8069).
- Destructuring of parameters in function signatures currently do not work (babel/babel#8099).
ember-cli-typescript must be in dependencies instead of devDependencies for addons, since we now hook into the normal Broccoli + Babel build pipeline instead of doing an end-run around it.
Addons can no longer use .ts in app, because an addon’s app directory gets merged with and uses the host’s (i.e. the other addon or app’s) preprocessors, and we cannot guarantee the host has TS support. Note that everything will continue to work for in-repo addons because of the app build works with the host’s (i.e. the app’s, not the addon’s) preprocessors.
Apps need to use .js for addon overrides in the app directory, since the different file extension means apps no longer consistently “win” over addon versions (a limitation of how Babel + app merging interact)—note that this won’t be a problem with Module Unification apps.

That’s it. Again, please test it out in your app and report any issues you find!

Under the Hood

In the 1.x series of releases, we used TypeScript’s own build tooling, including its --watch setting, and then fed the results of that into Ember’s build pipeline. We made this work, but it was pretty fragile. Worse, it was slow, because we had to compile your code twice: once with TypeScript, and once with Babel in the normal Ember CLI pipeline.

In v2, we’re now able to use Babel to do the actual build of your TypeScript code, while using the TypeScript compiler in parallel to type-check it. This meant we were able to throw away most of that fragile custom code wiring TypeScript and Ember CLI’s file-watching and build pipelines together, so it’s much less fragile. And of course it’s much faster. What’s more, because Babel 7 is itself substantially faster than Babel 6 was, build times see an even larger speedup.

While we were at it, we added some extra nice error reporting for your type errors:

An example of TypeScript build errors in an Ember app

These changes are why the addon is now back to being part of the runtime dependencies for addons. In the 1.x series, we did a complicated dance to make sure you could use TypeScript in an addon without burdening consumers with the full size of each distinct version of the TypeScript compiler in use by addons they consumed. (For more on that, see my blog post on the 1.1 release.) We still want to keep that commitment, and we do: TypeScript is only a dev dependency. ember-cli-typescript, however, is a full dependency for addons, because we’re just part of the regular Babel/Ember CLI pipeline now! We play exactly the same role that other Babel transforms in the Ember ecosystem do, including e.g. the modules API polyfill.

In short, this update simplifies our internals and makes your experience as a developer better. That’s about as good an outcome as we could hope for. We as a team have been thinking through this for quite some time—ever since we learned that Babel 7 was adding TypeScript support—but the actual implementation was almost all Dan Freeman’s excellent work, so say thank you to him if you get a chance!

We’re eager to get it to a production-ready state, so please test it!

The Apple Magic Keyboard

2018-11-11T21:20:00-05:00

Assumed audience: nerds who care about keyboards, most of whom will probably think I’m crazy for my views in this post.

I was sitting at my kitchen counter this evening, doing a mishmash of things—from writing a little Rust, to sending some text messages to my family, to leaving a note on a programming forum. The whole time, I’ve just had this quiet sense of deep pleasure about the typing. I was slightly confused about why I was enjoying the feel of the typing specifically… until I realized what exactly I was typing on.

I’m sitting in front of a 2015 MacBook Pro… but I’m not typing on that keyboard. I’m typing on the Apple Magic Keyboard instead, which I brought upstairs from its normal perch on my desk and started using everywhere.¹

This keyboard is perfect for me. It’s not what other people love, I know—people who prefer “mechanical keyboards” find it far too shallow. But there is something about the feeling of typing on this particular keyboard that I simply love. For me, it’s a perfect bit of industrial design, because it simultaneously looks good and works exactly the way I want it to, right down to the feeling of every keystroke.

Mechanical keyboard lovers: you can keep your Cherry Blues. I’ll just stockpile Apple Magic Keyboards and be happy.

P.S. Apple should totally just figure out how to put exactly this keyboard in their next-generation MacBook Pros. They won’t. But they should.

What prompted it? I ordered a stand for it for all the times I’m not at my desk with its 5k monitor. And then just decided it was worth doing this way all the time.↩

Review: The Rust Programming Language

2018-11-07T07:00:00-05:00

I intended to publish this review months ago, but the single hardest month of my career punched me in the face repeatedly and I just found myself entirely unable to write for all of September and most of October. Here it is at last, with apologies to No Starch for the delay!

No Starch Press kindly provided me with a review copy of The Rust Programming Language, by Steve Klabnik and Carol Nichols, with contributions from the Rust community. A real-world, physical review copy. And it’s magnificent.

The Rust Programming Language

I’ve read the vast majority of this book (as well as the previous edition) online over the last couple years as Steve and Carol have worked on it; it has been an invaluable resource for many a New Rustacean episode. (Bonus: you can hear me talk with Carol about working on The Rust Programming Language in my interview with her back in 2016! And yes: writing the book took that long.) This is in some sense the authoritative book on Rust.

You might wonder why you’d pick up a physical copy of this book given that it is available online for free. There are a few reasons that come to mind:

The quality of the online book is in many ways a direct result of No Starch Press’ deep investment in the text. They’re not making any money from the online copy. They do recoup some of their costs when we buy ebook or physical copies. So that’s one good reason: a way of saying “thank you!” and investing in the continued existence of No Starch and projects like this.
I used the word magnificent above to describe this, and I mean it. This printing is a fabulous example of really excellent book design. I take typography and presentation seriously, and not a whit less for programming books than for copies of The Lord of the Rings. Everything in this printing is top-notch. Little details like the way that code listings are displayed—right down to the way some text is faded away to emphasize what’s new in a listing—make this one of the most readable programming texts I’ve ever seen.
As delightful and powerful as hypertext is, the physicality of a book is equally delightful and powerful, just in different ways. There is nothing quite like the tactile experience of flipping through a book. Nothing in digital text lets you forge connections to learning the way that scribbling notes in a margin does. And the sheer physicality of a volume this large gives you mental hooks to hang what you’re learning on: you can remember that it felt like this to have the book open to where you learned it.

(That last point is an argument in favor of printed books in general. You can expect me to come back to that theme time and again in this space, even while I continue to value digital spaces like this one for what they uniquely do.)

No Starch sent me this copy to review with no expectation of a positive review—but even if they’d been paying me, they couldn’t make me say that this book is great. But great it is. If you have any interest in Rust, you should grab a copy!

True Myth 2.2

2018-10-27T17:00:00-04:00

I just released v2.2¹ of True Myth, with two new pairs of helpers to deal with safe JavaScript object property lookup with Maybes and handling exception-throwing code with Results.

Safe JavaScript object property lookup

We often deal with optional properties on JavaScript objects, and by default JavaScript just gives us undefined if a property doesn’t exist on an object and we look it up:

type Person = {
  name?: string;
};

let me: Person = { name: 'Chris' };
console.log(me.name); // Chris

let anonymous: Person = {};
console.log(anonymous.name); // undefined

We can already work around that with Maybe.of, of course:

function printName(p: Person) {
  let name = Maybe.of(p.name);
  console.log(name.unwrapOr('<anonymous>'));
}

But this is a really common pattern! Maybe.property is a convenience method for dealing with this:

function printName(p: Person) {
  let name = Maybe.property('name', p);
  console.log(name.unwrapOr('<anonymous>'));
}

At first blush, this might be a head-scratcher: after all, it’s actually slightly longer than doing it with Maybe.of. However, it ends up showing its convenience when you’re using the curried form in a functional pipeline. For example, if we had a list of people, and wanted to get a list of just the people’s names (ignoring anonymous people), we might do this:

function justNames(people: Person[]): string[] {
  return people
    .map(Maybe.property('name'))
    .filter(Maybe.isJust)
    .map(Just.unwrap);
}

Another common scenario is dealing with the same kind of lookup, but in the context of a Maybe of an object. Prior to 2.2.0, we could do this with a combination of Maybe.of and andThen:

function getName(maybePerson: Maybe<Person>): string {
  return maybePerson.andThen(p => Maybe.of(p.name));
}

This is harder to compose than we might like, and we can’t really write it in a “point free” style, even if that’s more convenient. We also end up repeating the andThen invocation every time we go down a layer if we have a more deeply nested object than this. Accordingly, 2.2.0 also adds another convenience method for dealing with deeply nested lookups on objects in a type-safe way: Maybe.get (and the corresponding instance methods).

// Function version:
function getNameFn(maybePerson: Maybe<Person>): string {
  return Maybe.get('name', maybePerson);
}

// Method version
function getNameM(maybePerson: Maybe<Person>): string {
  return maybePerson.get('name');
}

Again, since the function version is curried, we can use this to create other little helper functions along the way:

const getName = Maybe.get('name');

function getAllNames(people: Maybe<Person>[]): string[] {
  return people
    .map(getName)
    .filter(Maybe.isJust)
    .map(Just.unwrap);
}

And if our object is a deeper type:

type ComplicatedPerson = {
  name?: {
    first?: string;
    last?: string;
  };
};

let none: Maybe<ComplicatedPerson> = Maybe.nothing();
console.log(none.get('name').toString());
// Nothing
console.log(none.get('name').get('first').toString());
// Nothing

let nameless: Maybe<ComplicatedPerson> = Maybe.just({});
console.log(nameless.get('name').toString());
// Just([object Object]);
console.log(nameless.get('name').get('first').toString());
// Nothing

let firstOnly: Maybe<ComplicatedPerson> = Maybe.just({
  name: {
    first: 'Chris',
  },
});
console.log(firstOnly.get('name').toString());
// Just([object Object]);
console.log(firstOnly.get('name').get('first').toString());
// Just(Chris);

Note that in these cases, since the type we’re dealing with is some kind of object with specific keys, if you try to pass in a key which doesn’t existing on the relevant object type, you’ll get a type error. (Or, if you’re using the curried version, if you try to pass an object which doesn’t have that key, you’ll get a type error.) However, we also often use JavaScript objects as dictionaries, mapping from a key to a value (most often, but not always, a string key to a specific value type). Maybe.property and Maybe.get both work with dictionary types as well.

type Dict<T> = { [key: string]: T };

let ages: Dict<number> = {
  'chris': 31,
};

console.log(Maybe.property('chris', ages)); // Just(31)
console.log(Maybe.property('joe', ages)); // Nothing

let maybeAges: Maybe<Dict<number>> = Maybe.of(ages);
console.log(ages.get('chris')); // Just(31)
console.log(ages.get('joe')); // Nothing

Hopefully you’ll find these helpful! I ran into the motivating concerns for them pretty regularly in the codebase I work with each day, so I’m looking forward to integrating them into that app!

Handling exception-throwing functions

The other big additions are the Result.tryOr and Result.tryOrElse functions. Both of these help us deal with functions which throw exceptions. Since JavaScript doesn’t have any native construct like Result, idiomatic JavaScript does often throw exceptions. And that can be frustrating you want to have a value type like a Result to deal with instead.

Sometimes, you don’t care what the exception was; you just want a default value (or a value constructed from the local state of your program, but either way just one value) you can use as the error to keep moving along through your program. In that case, you wrap a function which throws an error in Result.tryOr. Let’s assume we have a function either returns a number of throws an error, which we’ll just call badFunction because the details here don’t really matter.

const err = 'whoops! something went wrong!';
const result = Result.tryOr(err, badFunction());

The result value has the type Result<number, string>. If badFunction through an error, we have an Err with the value 'whoops! something went wrong!' in it. If it didn’t throw an error, we have an Ok with the number returned from badFunction in it. Handy!

Of course, we often want to do something with the exception that gets thrown. For example, we might want to log an error to a bug-tracking service, or display a nice message to the user, or any number of other things. In that case, we can use the Result.tryOrElse function. Let’s imagine we have a function throwsHelpfulErrors which returns a number or does just what it says on the tin: it throws a bunch of different kinds of errors, which are helpfully distinct and carry around useful information with them. Note that the type of the error-handling callback we pass in is (error: unknown) => E, because JS functions can throw anything as their error.

const handleErr = (e: unknown): string => {
  if (e instanceof Error) {
    return e.message;
  } else if (typeof e === 'string') {
    return e;
  } else if (typeof e === 'number') {
    return `Status code: ${e}`;
  } else {
    return 'Unknown error';
  }
}

const result = Result.tryOrElse(handleErr, throwsHelpfulErrors);

Here, result is once again a Result<number, string>, but the error side has whatever explanatory information the exception provided to us, plus some massaging we did ourselves. This is particularly handy for converting exceptions to Results when you have a library which uses exceptions extensively, but in a carefully structured way. (You could, in fact, just use an identity function to return whatever error the library throws—as long as you write your types carefully and accurately as a union of those error types for the E type parameter! However, doing that would require you to explicitly opt into the use of any to write it as a simple identity function, so I’m not sure I’d recommend it. If you go down that path, do it with care.)

And that’s it for True Myth 2.2! Enjoy, and of course please open an issue if you run into any bugs!

Thanks to Ben Makuh for implementing Result.tryOr and Result.tryOrElse. Thanks to Ben and also Frank Tan for helpful input on the Maybe.get and Maybe.property API design!

I published both 2.2.0 and 2.2.1, because once again I missed something along the way. This time it was making sure all the new functions were optionally curried to support partial application.↩

Scales of Feedback Time in Software Development

2018-10-22T21:15:00-04:00

Assumed Audience: fans of compiled languages with expressive type systems. I’m not trying to persuade fans of dynamic languages they should use a compiler here; I’m trying to surface something that often goes unstated in discussions among fans of compiled languages with expressive type systems, but hopefully it’s interesting beyond that. If you don’t like compiled languages, just skip the build step bits; the rest all still applies.

There are basically six stages of the development of any given software component where you can receive feedback on what you build:¹

compilers, static analysis tools, and/or pair programming
automated test suites
manual local testing
continuous integration (CI) test results
deploying to staging (or a similar test environment) for manual testing
deploying to live, i.e. when production traffic is meaningfully different from what you can test on staging

What’s interesting to note is that there are also, in my experience, roughly order-of-magnitude differences between each of those layers in terms of the cycle time between when you make a change and whether you know it is broken. That is, there seem to be rough factor-of-ten differences between the feedback you get from—

compilers, static analysis tools, and/or pair programming—all of which can show you feedback in near-real-time as you’re typing and saving your code, especially with a good language server or a fast compiler or a speedy linter
automated test suites, assuming they’re running on every build change and are reasonably speedy themselves, or scoped to the things impacted by the changes made
manual local testing, which you can repeat after every build, but which usually requires you to switch contexts to execute the program in some way
CI, presumably doing the automated equivalent of what you do in both layers 2 and 3, but requiring a push to some central location and a remote build and execution of the test suite, and often a much larger integration test suite than you’d run locally
deploying to staging, and repeating the same kinds of manual testing you might do locally in layer 2 in a more production-like environment
deploying to live, and repeating the same kinds of manual testing you might do locally in layers 2 or 5, as well as getting feedback from observability or monitoring systems using your real traffic

(Those last two might be comparable in the cycle time sense. However, the way most teams I’ve heard of work, any deploy to live is usually preceded by a deploy to staging. What’s more, with most changes that you can’t test until it’s live, it’s often the case that you’re not going to know if something is wrong until it has been live for at least a little while. Finally, some kinds of things you can really only test with production load and monitoring or observability systems, and those kinds of things are at least sometimes not to be visible immediately after deployment, but only in the system’s aggregate behavior or weird outliers that show up given enough scale.)

What all of this gets at is that stepping to a higher layer nearly always entails a dramatic increase in the cycle time for software development: that is, the amount of time between when I make a change and when I know whether it’s broken or not. If I can know that I have a problem because my compiler surfaces errors in my editor, that probably saves me a minute or two each day over only being able to see the same error in a test suite. By the same token, being able to surface an error in a test suite running on every build will likely save me anything from minutes to hours of cycle time compared to something I can only test in production.

At first blush, this looks like an argument for pushing everything to the lowest-numbered layer possible, and I think that’s kind of right. I (and probably many other people who end up in, say, Rust or Haskell or Elm or other languages with similarly rich type systems) tend to prefer putting as much as possible into layer 1 here precisely because we have so often been bitten by things that are at layer 2 in other languages or frameworks and take a lot of time to figure out why they broke at layer 2. This happened to me in a C^♯ server application just a couple weeks ago, and chasing it down was not fun.

However, my enthusiasm for rich type systems notwithstanding, I don’t think this observation about these layers of cycle time means we should put everything in the compiler all the time. Indeed, there are some things it is too expensive or difficult to test anywhere but production (all the way up at layer 6). What’s more–although this is often overlooked in these discussions–putting too much of this rich information in layer 1 can absolutely kill your compile times in many languages. In my experience, this is particularly true of many of the languages with rich enough type systems to make layer 1 handling genuinely viable in the first place!²

I do think, though, that being aware of the cost in cycle time is useful, as is being explicit about why we think it’s worth slotting a particular set of feedback into layer 2 vs. layer 1 (or layers 3, 4, 5, or 6). That goes for library development, of course.³ It goes equally for application development, though! It can be really helpful to make explicit both which of these layers you’re landing in and (just as important) why you’ve landed there for any given bit of feedback you want or need to get–making the tradeoffs explicit along the way.

Thanks to my friend Ben Makuh for looking over an earlier draft of this piece and providing really helpful feedback on it! Thanks as well to Greg Vaughn for noting shortly after I published it that pair programming also sits at the “immediate feedback” layer.

There’s some ongoing work in the Rust web working group to build an exemplar web framework, Tide. The most recent post tackled routing, and prompted an interesting discussion on the Rust internals forum. This post is a cleaned-up, better-articulated, more general version of a post I offered in that thread.↩
Right now I and a few others are trying to figure out why one particular type definition in the TypeScript definitions for Ember.js causes a build to take about 20× as long as the build without that type definition. It’s the difference between a 6.5-second build and a 2.5-minute build.↩
as in the example of a web server’s API for route handling which originally prompted this post↩

Why We Want Pattern-Matching in JavaScript

2018-09-23T13:00:00-04:00

I’ve often noted how much I want the JavaScript pattern-matching proposal to land. I noted in conversation with some people recently, though, that it’s not always obvious why it will be so helpful. Similarly, Dave Herman recently noted to me that DHH’s mantra of “Show me the code” is a really helpful tool for thinking about language design. (I tend to agree!) So with that in mind, here’s a piece of example code from the Ember app I work on today, very slightly modified to get at the pure essentials of this particular example.¹

The context is a UI component which shows the user their current discount, if any, and provides some nice interactivity if they try to switch to a different discount.

First, some types that we’ll use in the example, which I use in the actual component to avoid the problems that inevitably come with using string values for these kinds of things. Linters like ESLint or type systems like TypeScript or Flow will catch typos this way, and you’ll also get better errors at runtime even if you’re not using a linter or a type system!²

const DiscountTypes = {
  Offer: 'Offer',
  Coupon: 'Coupon',
  None: 'None',
};

const Change = {
  OfferToOffer: 'OfferToOffer',
  OfferToCoupon: 'OfferToCoupon',
  CouponToCoupon: 'CouponToCoupon',
  CouponToOffer: 'CouponToOffer',
};

Now, we set up a component which has a little bit of internal state to track the desired change before we submit it, which we display differently based on what the value of the ES5 getter for change is here:

class DiscountComponent {
  constructor(currentDiscountType) {
    this.currentDiscountType = currentDiscountType;
    this.newDiscountType = null;
  }

  changeDiscount(newDiscountType) {
    this.newDiscountType = newDiscountType;
  }

  submitChange() {
    // logic for talking to the server
  }

  get change() {
    const { currentDiscountType, newDiscountType } = this;

    if (currentDiscountType === DiscountTypes.Offer) {
      if (newDiscountType === DiscountTypes.Offer) {
        return Change.OfferToOffer;
      } else if (newDiscountType === DiscountTypes.Coupon) {
        return Change.OfferToCoupon;
      } else if (newDiscountType === DiscountTypes.None) {
        return null;
      } else {
        assertInDev(
          `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
        );
      }
    } else if (currentDiscountType === DiscountTypes.Coupon) {
      if (newDiscountType === DiscountTypes.Offer) {
        return Change.CouponToOffer;
      } else if (newDiscountType === DiscountTypes.Coupon) {
        return Change.CouponToCoupon;
      } else if (newDiscountType === DiscountTypes.None) {
        return null;
      } else {
        assertInDev(
          `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
        );
      }
    } else if (currentDiscountType === DiscountTypes.None) {
      return null;
    } else {
      assertInDev(
        `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
      );
    }
  }
}

Here’s the exact same semantics for computing the change value we’re interested, but with pattern matching:

class DiscountComponent {
  // ...snip

  get change() {
    case ([this.currentDiscountType, this.newDiscountType]) {
      when [DiscountTypes.Offer, DiscountTypes.Offer] ->
        return Change.OfferToOffer;
      when [DiscountTypes.Offer, DiscountTypes.Coupon] ->
        return Change.OfferToCoupon;
      when [DiscountTypes.Coupon, DiscountTypes.Offer] ->
        return Change.CouponToOffer;
      when [DiscountTypes.Coupon, DiscountTypes.Coupon] ->
        return Change.CouponToCoupon;
      when [DiscountTypes.None, ...] || [..., DiscountTypes.None] ->
        return null;
      when [...] ->
        assertInDev(
          `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
        );
    }
  }
}

The difference is stark. It’s not just that there are fewer lines of code, it’s that the actual intent of the code is dramatically clearer. (And while I’ve formatted it for nice display here, those are all one-liners in my normal 100-characters-per-line formatting.)

My preference would be for pattern-matching to have expression semantics, so you wouldn’t need all the return statements in the mix—and it’s possible, depending on how a number of proposals in flight right now shake out, that it still will. Even if pattern matching doesn’t ultimately end up with an expression-based syntax, though, we can still get a lot of those niceties if the do-expression proposal lands:

class DiscountComponent {
  // ...snip

  get change() {
    return do {
      case ([this.currentDiscountType, this.newDiscountType]) {
        when [DiscountTypes.Offer, DiscountTypes.Offer] ->
          Change.OfferToOffer;
        when [DiscountTypes.Offer, DiscountTypes.Coupon] ->
          Change.OfferToCoupon;
        when [DiscountTypes.Coupon, DiscountTypes.Offer] ->
          Change.CouponToOffer;
        when [DiscountTypes.Coupon, DiscountTypes.Coupon] ->
          Change.CouponToCoupon;
        when [DiscountTypes.None, ...] || [..., DiscountTypes.None] ->
          null;
        when [...] ->
          assertInDev(
            `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
          );
      }
    }
  }
}

Again, this is profoundly clearer about the intent of the code, and it’s far easier to be sure you haven’t missed a case.³

Edit: after some comments on Twitter, I thought I’d note how this is even nicer in pure functions. If we assume that it gets expression semantics (which, again, I’m hoping for), a pure functional version of the sample above would look like this:

const change = (currentType, newType) =>
  case ([currentType, newType]) {
    when [DiscountTypes.Offer, DiscountTypes.Offer] ->
      Change.OfferToOffer;
    when [DiscountTypes.Offer, DiscountTypes.Coupon] ->
      Change.OfferToCoupon;
    when [DiscountTypes.Coupon, DiscountTypes.Offer] ->
      Change.CouponToOffer;
    when [DiscountTypes.Coupon, DiscountTypes.Coupon] ->
      Change.CouponToCoupon;
    when [DiscountTypes.None, ...] || [..., DiscountTypes.None] ->
      null;
    when [...] ->
      assertInDev(
        `Missed a condition: ${currentDiscountType}, ${newDiscountType}`
      );
  };

This may not be quite as clear as the same thing in F^♯ or Elm or another language in that family… but it’s amazingly better than anything we’ve seen in JavaScript to date.

assertInDev looks a little different; we’re actually using the Maybe type from my True Myth library instead of returning null; it’s an Ember app; as such it uses a @computed decorator; and of course it’s all in TypeScript. I chose to write it with standard JavaScript to minimize the number of things you have to parse as a reader.↩
In the actual TypeScript, these are defined with an enum.↩
Fun fact: the original code actually had missed a number of cases, which I learned only because TypeScript’s strictNullChecks setting informed me.↩

“Zuckerberg’s Blindness and Ours” (L. M. Sacasas)

2018-09-17T08:20:00-04:00

Yesterday while talking with my wife as we drove to spend some time with extended family, I caught myself: tempted to describe a given response to a particular cultural ill as a solution. This is a turn of thinking that’s especially tempting for engineers—and perhaps the more so engineers with a physics background (like me!). In two of the fields to which I have applied myself (physics and software), knowledge often genuinely appears in the form of solutions to problems. But the extent to which science (and scientism) on the one hand and engineering disciplines (especially software) on the other have come to the fore in our culture—the degree to which they have achieved nearly unassailable authority for us—means that we now too often take solutions as coextant with knowledge more generally.

This is solutionism, and it is bad. I noted above that two of the fields to which I have applied myself share this feature of having solutions to problems as their predominant form of knowledge. But this is not so in two of the other fields I have studied in some depth: for neither theology nor music is a solution very often in demand. Very different modes of thought and of reasoning are in play in each of those, and appropriately so.

So it was with some particular appreciation that I read this piece by L. M. Sacasas, reflecting on the recent New Yorker profile of Mark Zuckerberg. Sacasas rightly highlights how mistaken this solutionist frame of knowledge is. From his conclusion (emphasis mine):

Reducing knowledge to know-how and doing away with thought leaves us trapped by an impulse to see the world merely as a field of problems to be solved by the application of the proper tool or technique, and this impulse is also compulsive because it cannot abide inaction. We can call this an ideology or we can simply call it a frame of mind, but either way it seems that this is closer to the truth about the mindset of Silicon Valley.

It is not a matter of stupidity or education, formally understood, or any kind of personal turpitude. Indeed, by most accounts, Zuckerberg is both earnest and, in his own way, thoughtful. Rather it is the case that one’s intelligence and one’s education, even if it were deeply humanistic, and one’s moral outlook, otherwise exemplary and decent, are framed by something more fundamental: a distinctive way of perceiving the world. This way of seeing the world, including the human being, as a field of problems to be solved by the application of tools and techniques, bends all of our faculties to its own ends. The solution is the truth, the solution is the good, the solution the beautiful. Nothing that is given is valued.

The trouble with this way of seeing the world is that it cannot quite imagine the possibility that some problems are not susceptible to merely technical solutions or, much less, that some problems are best abided. It is also plagued by hubris—often of the worst sort, the hubris of the powerful and well-intentioned—and, consequently, it is incapable of perceiving its own limits. As in the Greek tragedies, hubris generates blindness, a blindness born precisely out of one’s distinctive way of seeing. And that’s not the worst of it. That worst of it is that we are all, to some degree, now tempted and prone to see the world in just this way too.

A Real Victory

2018-09-05T21:45:00-04:00

On September 29, 2016, I started working on adding (Flow) types to the Ember app I had been working on since the start of the year. Throughout the rest of the year I worked on adding some basic Flow types to our app and for Ember. For my last commit in 2016, I switched us to TypeScript and began the rest of the long journey to fully type-checking our app. In early 2018, we made it to “the app type-checks”… in the loosest strictness settings.

And as of 6pm today—September 5, 2018, almost two full years later, and 21 months after we switched from Flow to TypeScript (more on this below)—we have a fully type-checked TypeScript Ember application, with the strictness notches dialed as strict as they will go.

It took almost two full years for us to get there, and I’m incredibly proud of that work.

It took almost two full years because it was a lot of work, and slow work to do at that, and it was rare that I could block out any large chunks of time for that work—we had to sneak in improvements between features we were working urgently on for our clients and our own internally goals. More, it wasn’t just the work of adding types to our application. It was also the work of writing types for Ember itself, and for the surrounding ecosystem—which thankfully I did not finish alone, but which I did have to start alone. It was the work of integrating (and reintegrating) TypeScript into Ember’s build pipeline.

Happily, I did not do most of that work alone, and even on our app I’ve had a ton of help getting the types in place. But it has been a massive task, and finishing it today was a real victory. It’s not perfect. We have 200-or-so instances of any in the application (most of them in semi-legitimate places, to be fair), and I wish it were more like 20. We have a number of places in the app with the ! “I promise this isn’t null or undefined here” operator on some nullable field, with long comments explaining why it’s not possible for it to be null there.¹

But it type-checks today, and type errors fail the builds, and that is a real victory.

You can consider this “part 1” of my thoughts on what feels to me like a pretty significant achievement. I’ll hopefully follow this up with some backstory sometime in the next few weeks.

See my recent post on thinking a lot about design decisions I would have made differently with TypeScript’s strict null checking available from day one!↩

True Myth 2.1.0 Released

2018-09-02T16:25:00-04:00

I’ve just released True Myth 2.1.0 (source, docs), which includes a handful of new utility functions for use with the Maybe types and arrays or tuples. Note that to make use of these you’ll need to be on at least TypeScript 3.0: they take advantage of the some of the shiny new features in the type system!

Edit: and, five minutes later, versions 2.1.1 and 2.1.2 are out with bugfixes consisting of “I forgot to export two functions. Now they’re exported.” Because that’s how this always works, right?

Here’s what’s new:

Maybe.find: for those times when you want to do Array.prototype.find and would love to not have to wrap up the result with a Maybe explicitly every time. As with most functions in True Myth, it’s curried so you can easily use it in a functional programming style.

import Maybe from 'true-myth/maybe';

let foundRegular = Maybe.find(n => n > 1, [1, 2, 3]);
console.log(foundRegular.toString());  // Just(2)

let notFound = Maybe.find(n = n < 1, [1, 2, 3]);
console.log(notFound.toString());  // Nothing

let findAtLeast5 = Maybe.find((n: number) => n > 5);
let foundCurried = findAtLeastFive([2, 4, 6, 8, 10]);
console.log(foundCurried.toString());  // Just(6)

Maybe.head (aliased as Maybe.first): for getting the first item of an array safely. Like lodash’s _.head (or someArray[0]) but it returns a Maybe instead of possibly giving you back undefined.

import Maybe from 'true-myth/maybe';

let empty = Maybe.head([]);
console.log(empty.toString());  // Nothing

let hasItems = Maybe.head([1, 2, 3]);
console.log(hasItems.toString());  // Just(1)

Maybe.last: the same as Maybe.head, but for getting the last element in an array.

import Maybe from 'true-myth/maybe';

let empty = Maybe.last([]);
console.log(empty.toString());  // Nothing

let hasItems = Maybe.last([1, 2, 3]);
console.log(hasItems.toString());  // Just(3)

Maybe.all: for converting an array of Maybes to a Maybe of an array. If you have an array whose contents are all Maybes, it’s sometimes useful to be able to flip that around so that if all of the items are Justs, you get back a single Just wrapping the array of the values which were wrapped in all the Justs in the array, but if any were Nothing, the whole thing is a single Nothing. This works for both heterogeneous and homogenous arrays, which is pretty cool. A code sample will make this a lot clearer:
```
import Maybe, { just, nothing } from 'true-myth/maybe';

let includesNothing = Maybe.all(just(2), nothing<string>());
console.log(includesNothing.toString());  // Nothing

let allJusts = Maybe.all(just(2), just('hi'), just([42]));
console.log(allJusts.toString());  // Just([2, 'hi', [42]]);
```
The resulting type of both includesNothing and allJusts here is Maybe<Array<string | number | Array<number>>>.
Maybe.tuple: just like Maybe.all except it works in tuples (preserving their types’ order) for up to five-item tuples. (As the docs I wrote say: if you’re doing a larger tuple than that I don’t want to know what you’re doing but I won’t help with it!)
```
import Maybe, { just, nothing } from 'true-myth/maybe';

type Tuple = [Maybe<number>, Maybe<string>, Maybe<number[]>];

let withNothing: Tuple = [just(2), nothing(), just([42])];
let withNothingResult = Maybe.tuple(withNothing);
console.log(withNothingResult.toString());  // Nothing

let allJusts: Tuple = [just(2), just('hi'), just([42])];
let allJustsResult = Maybe.tuple(allJusts);
console.log(allJustsResult.toString());  // Just([2, "hi", [42]])
```
These have the same output (i.e. the same underlying representation) as the array output, but a different type. The resulting type of both includesNothing and allJusts here is Maybe<[number, string, Array<number>]>.

Once TypeScript 3.1 is out, I should be able to collapse these into a single all, and tuple will just become an alias for it.

Going Offline

2018-09-01T09:00:00-04:00

Several times this week, I signed out of both my company Slack and the Ember Community Slack and just worked in “solitude” for about six and a half of the eight hours I was working. It was, in a word, bliss.

Over the past few years, I’ve come to enjoy a lot of chatting during the day while at work, and the social interaction is very much a must in a lot of ways for someone who works remotely. The flip side, however, is that chat is a huge time-suck, and more than that it’s a constant drain on attention. I’m far from the first person to see this or to say it, of course, and I’ve experienced it before. But it struck me much more forcefully this week than it usually has in the past.

Some of that, I suspect, is the effect of the burnout I’ve been experiencing: I’m much more taxed by social interaction and by shifts in attention right now than I have been in the past. More and more, though, I’ve also become aware of the effect the constant distraction of chat has on me. I am much less effective as a software developer when I am constantly switching modes.

To be sure: some tasks are more mentally demanding than others. Some days I get done plenty while still switching contexts often—less, perhaps, than I might if I were not being interrupted, but also perhaps more in the sense that enabling others to finish their tasks is also important. When I need to dive deep on something hard, though—or even when I just need to get after a large task—the kind of mental silence that signing out of chat affords is very helpful. I plan to make this a regular habit.

Once More Around The Wheel

2018-08-31T18:30:00-04:00

I very much wish my publishing needs were not so… complicated. Academic writing—though that’s done for a while—poetry, code, and a strong preference for semantic HTML to be generated by it along with an equally strong preference for plain text authoring…

Nothing works for me.

The only Markdown parser out there which does the right thing with all of those is Pandoc. The options for using Pandoc directly are not great. Shelling out to it via Pelican (my current strategy) works but is slow. The implemented-in-Rust Gutenberg generator looks like exactly what I want performance-wise, but its underlying Markdown engine doesn’t support citations or poetry.

I keep coming back to the conclusion that I’m basically going to have to build whatever I want myself, if I want my desired publishing flow to exist. I don’t really want to do that, though. It’s a boatload of work, even “just”—just!—to do something like (a) learn Haskell well enough to build on top of Pandoc directly or (b) build a good C-based API wrapper in Rust so that I can do it that way or (c) extend pulldown-cmark to support poetry and citation management.

For lots of reasons (c) is probably what I’ll ultimately end up doing; I want that for more than just blogging. More on that eventually, I hope.

But in the meantime I really just… want someone else to have the same weird needs I do and build this for me. I just know full well at this point that that’s not going to happen, and accordingly am basically resigned to muddling along with Pelican and Pandoc until such a time as I can actually buckle down and build what I want and need.¹ C’est la vie.

As an aside: the prospect of buckling down and building things like that in my spare time is much less appealing given my current struggles with burnout, and as I’ll write about at some point in the future I feel a (perhaps-surprising to you, my reader) lack of confidence about my ability to accomplish those things.↩

Type-Informed Design

2018-08-30T19:40:00-04:00

I’ve been working on getting the Ember app I work on fully type-checked in strict mode this week, and noticed something interesting along the way: there are a lot of design decisions—a few of them really core to the behavior of the app!—which we never, ever would have made if we had been using Typescript in the first place.

One of these is pervasive references to certain optional properties that appear in services throughout our app—the basket, for example. These can indeed be inset and at certain times they are. However, many of our components pull in this property from the service and simply assume it’s there to use. We’ve known for a while that this was a problem at times: Raygun has told us loud and clear. But it wasn’t obvious how pervasive this was—and how badly we were just assuming the presence of something that may well be absent all over the app!—until I saw the type errors from it. Dozens of them.

Some of them are places where we should have built the components differently: to take the item as an argument, for example, and to require it as an input, because the component just doesn’t make any sense without it, indeed lives in a part of the app such that it’s not even possible to render the component without it.

And sure, we could document that invariant and use TypeScript’s override tools to carry on. (What do you think I’m doing this week?)

But, and this is the thing that really caught my attention in thinking about all of this: it would be much better not to have to do that. Had we had TypeScript in place when we started, we simply would have designed large swaths of the app differently because we’d have seen these kinds of things when we were building it in the first place!

That’s a bit of wishing for the impossible in one sense: we literally couldn’t have done that when we started on the app, because TS didn’t have the necessary pieces to support typing the Ember libraries. My team helped build the TS and Ember story over the last 18 months! But at a minimum I have a pretty good idea how the process will be different next time around, with this tool available and providing this kind of helpful design feedback from the outset!

RSS Triage is Just Like Email Triage

2018-08-22T18:15:00-04:00

I’ve heard people say they find RSS overwhelming: the constant flow of new items. I find that managing RSS well is not actually all that hard though; it just takes the same kind of discipline as keeping an empty inbox does.

In short: keep my reading list restrained, and have a strategy to keep up to date on that list.

My strategy for keeping up to date is simple, too: for every time, either read it, send it to Pocket to read later (on my Kobo, via Kobo’s Pocket integration), or mark it as read. Then move on.

As for the Pocket items: clear those out after long enough, too: I go through my backlog every so often and if I haven’t read it yet and it has been in my to-read list for more than a month or two, I decide to read it now or just remove it. (At some point you just have to admit that you’re not going to read something, and that it’s okay. We all have limited time.)

With that simple strategy, RSS becomes more than merely “manageable” for me: I get to read a lot of things I actually want to read, and it’s focused in a way that Twitter or Apple News or whatever else isn’t.

Level up your `.filter` game

2018-08-18T10:00:00-04:00

Adam Giese’s “Level up your .filter game” does something really interesting and helpful: it introduces a bunch of fairly sophisticated functional programming concepts without ever mentioning functional programming and without ever using any of the jargon associated with those terms.

“Level up your .filter game” gives you a reason to use some standard FP tools—currying, higher-order functions, composition—in your ordinary work. It’s pitched at working JS developers. It gives a real-world example of wanting to filter search results based on user input. It shows the utility of defining a bunch of small functions which can fit together like LEGO.

Filters are an essential part of JavaScript development. Whether you’re sorting out bad data from an API response or responding to user interactions, there are countless times when you would want a subset of an array’s values. I hope this overview helped with ways that you can manipulate predicates to write more readable and maintainable code.

I commend the piece to you not so much for the explanation of how to use JavaScript’s Array.prototype.filter effectively (though it has some good suggestions that way!) but primarily as a great example of the kind of pedagogy we need a lot more of to demonstrate the value of functional programming in ordinary, day-to-day development work.

Stable Libraries

2018-08-14T19:45:00-04:00

True Myth has changed very little since I first released it, and although I have a few ideas for small additions I might make, I don’t really expect it to change much in the future. That’s okay.

There’s a strange idea in some parts of the software development ecosystem—a way of think I also find myself falling into from time—which takes a lack of changes to a library as a sign that the library is dead and shouldn’t be used. I call this idea “strange” because if you take a step back, it’s actually not necessarily very healthy for certain kinds of libraries to be changing all the time.

But if you’re in an ecosystem where rapid change in libraries is normal, you end up assuming that something which isn’t changing is unmaintained or not usable when in fact the opposite may be true. If someone opens a pull request or an issue for True Myth, I generally get to it in under a day, often under an hour if it’s in my normal working time. (That’s easy enough for me to do because it’s a small, simple library; I don’t have the scale problems that larger projects do.) The project isn’t dead. It’s just mostly done.

One of the things I’d like to see in the front-end/JavaScript community in particular is a growing embrace of the idea that some libraries can genuinely be finished. They might need a tweak here or there to work with a new packaging solution, or to fix some corner case bug that has been found. But the “churn” we all feel to varying degrees would be much diminished if maintainers didn’t feel a constant push to be changing for the sake of, well… change. The burden on maintainers would be lower, too. Maybe we’d all get to spend less time on small changes that just keep us “up to date” and more on solving bigger problems.

Don’t get me wrong: sometimes changing perspective warrants a rewrite. But in libraries as in apps, just as often you’ll end up with a bad case of second system syndrome; and rewrites are rarely—not never, but rarely—clean wins.

“Free Speech”

2018-08-11T10:35:00-04:00

Every time there is a major controversy about large platforms blocking or delisting some controversial figure, something like the following exchange follows:¹

Person 1: But what about free speech? You’re censoring this party!

Person 2: [Twitter/Facebook/Youtube/etc.] is a private platform! Free speech guarantees the right not to be jailed for what you say, not the right to have it on every platform you want.

So far as it goes, this is true. XKCD’s explanation is completely right on the legal merits:

But while this is all true in some sense, it also seems to me to be missing the larger and much more important point. Namely: the whole reason we have these arguments—and the reason people tend to think as they do about the “free speech” question in these situations, legally nonsensical or not—is that we have outsourced the vast majority of our public discourse to these private platforms.

Twitter and Facebook have become the de facto public fora of the 2010s, with Google’s search results and Wikipedia’s summaries taking similarly authoritative roles on what exists and what is true. Not that most people would put it that way, but it remains true: if something isn’t in Google search, it might as well not exist on the internet, and therefore for many people at all. Likewise with Wikipedia’s summaries: the admonitions of every college professor in the world notwithstanding, what Wikipedia says has an undeniable authority. And when someone is blacklisted from Twitter or Facebook, their ability to be heard at all by internet users as a block is dramatically curtailed.

This centralization of discussion and information into a few private platforms has a great many downsides. But perhaps chief among them is that we have ceded major aspects of our public and civic life to private platforms, and their interests are not the interests of the public good. They are driven almost entirely by the profit motive, or (possibly even worse at times) by nebulous and chimeric ideologies that treat “connecting people [digitally]” or “organizing the world’s information” as inherent and superlative goods. So when someone has their page removed from Facebook, or their website blacklisted from Google, there is a real sense in which they have been removed from public discourse and their speech “silenced”—even if not in an illegal sense.

For the purposes of this post, though, I could not care less what the major internet companies do or don’t show on their platforms. Instead, I worry about our practice both as individuals and also as communities-of-practice—churches, associations, and so on—of abdicating our responsibility to maintain real public and civic lives in our local places in favor of letting these corporate giants do the work for us. I worry about the costs of letting Google and Facebook replace genuine public fora in our lives. I worry about the long-term effect of letting supranational megacorporations driven by that toxic combination of profit motive and nonsensical ideologies set the terms of our lives. I worry about the whole set of underlying structural and systematic moves that have made delisting on one of those platforms seem like a violation of the ideal of free speech.

As I’ve said for many years in this space: we should work hard at reclaiming our lives from the tangle of the corporations. We should limit the way we both use and think about these platforms. We should read books, old and new, rather than simply rely on the Google results and Wikipedia summaries. We should have painful, awkward conversations and indeed arguments with neighbors and colleagues and family members rather than merely all-caps shouting at each other on Facebook or Twitter. We should carve out our own spaces on the internet, owning our own turf; but more than that we should remember that even that is no substitute for the thicker (and yes, more painful, frustrating, and awkward!) communities and interactions of a church or a neighborhood or a town hall meeting.

Given the context in which I’m writing this, it’s probably helpful to say that I think InfoWars is a font of demonic lies. I’m a Christian; I mean that literally.↩

Building Things

2018-08-06T21:15:00-04:00

I.

For almost three years, now, I have been more or less steadily—sometimes more, sometimes less!—putting out episodes of New Rustacean. It’s fairly popular. I’ve had really smart people tell me how helpful it was in getting them up to speed with the language. I have had the surprising and slightly weird (if also somewhat gratifying) experience of walking into a room and seeing people respond in recognition of my voice.

I’m grateful for the impact the podcast has had, and as I tell people often: this is far and away the most significant thing I could have done in the Rust ecosystem in the last three years. There are a lot of people better-equipped than I to write top-notch libraries and applications in the ecosystem. People well-equipped for podcasting by dint of already being active in the space, and well-equipped for teaching specifically by dint of background and training? There are a lot fewer of those. I don’t think there is anywhere at all I could have made a bigger dent in the same time for Rust.

And yet.

If I went and applied for a job today, where actual Rust experience was desired, the vast majority of my show’s listeners would have substantially more to show than me. A command line tool here, a little experiment there. My one real project has been on hold almost since I started it. Another project, my original inspiration for learning Rust at all, I’ve never even started. My actual lines of Rust code written in the last three years top out somewhere under 3,000. It’s a pittance. As well as I know the language’s ideas, and indeed as well as I can explain them… I actually haven’t gotten to build much of anything with it.

II.

The last few months at work, I’ve spent a lot of my time—and an increasingly large proportion of it—on mentoring, code reviews, and leading the team and effort I’m on. This is genuinely wonderful in a lot of ways. I love teaching, and it’s a pleasure to help shape the overall direction of a project and a codebase.¹ In many ways, I’m right in line with the goals I set explicitly with my manager at the beginning of the year.

That’s really good, and really important. I recently saw someone tweet the pithy remark that the definition of a senior engineer is that they are mentoring a more junior engineer. I don’t think that’s quite right—there is a lot of room for really outstanding technical contributors who don’t have the gift of teaching, but whose technical chops mean they genuinely are senior people on the team.² But the reasonable insight under the hyperbole is that enabling others can often be far more effective than merely doing work yourself.

And yet.

Over the last several months, the amount of code I have written myself has dropped substantially. Not to nothing, of course; I’m still doing the actual work of designing and implementing pieces of the application I work on a majority of the time. But I’m not sure how much more than 50% of my time it is on any given week at this point. As much as I’ve enjoyed helping drive this particular project forward,³ I haven’t actually gotten to build as much during this phase of it.

III.

These two things have a great deal in common, for all their superficial differences. Both are places where my most valuable contributions are not what I can build myself, but what I can enable others to build.

Thousands and thousands of people have listened to New Rustacean. For some non-trivial number of them, the podcast was an important part of their wrapping their heads around the language. I know this because they tell me, in emails and conversations and tweets that are genuinely my favorite parts of doing the show! I have done far, far more with the podcast than I possibly could have by building another library in Rust.

Similarly, albeit on a much smaller scale, my role in my team at Olo matters. I’ve been able to help set the overall technical direction of a number of our front-end initiatives at the company in important ways. I’ve been able to help more junior developers ramp up their skills. I have done far more in this kind of role than I could possibly have done by just quietly shipping features.

And yet.

Being a “force multiplier” (what a terrible phrase!) isn’t always what it’s cracked up to be. It can be both worth it and also profoundly frustrating and boring at times. I was drawn to software in no small part because of the joy of being able to make things—to start with nothing but an idea or a sketch and a few hours later have something people can interact with, that solves a problem for them. I still love that side of it, and it’s clear to me if nothing else that (for the foreseeable future, anyway) I have no desire whatsoever to go into management roles, “force multiplier” or not.

There’s a real trick here, because it’s not that I’m not building things in these roles. It’s just that building a team or a community is not quite the same thing—it does not scratch the same itch—as building a really elegant user interface component with an elegant and communicative animation. They’re both good; and they’re very, very different from each other.

I separate those on purpose: a project and a codebase are related, but they’re far from identical. A project can succeed—at least in the short term—with a terrible codebase; an excellent codebase is no guarantee of project success. Getting them aligned is rare, difficult, and rewarding.↩
This seems like a typical overcorrection: against the idea that teaching is unimportant, it now comes into vogue to say that teaching is the most important. Imagine if we simply noted that teaching is some people’s gift and vocation, and not others; and that we can complement one another’s strengths by sharing our own—that it is not a zero-sum game but one in which we are like hands and feet and elbows and ears, each one needing the other, none able to do without the others.↩
much of the time anyway; the burnout I’m experiencing is related to some of the dynamics of this particular project↩

Is Agile the Enemy of Good Design?

2018-07-29T16:15:00-04:00

In line with my recently stated desire to share out things I’m reading:

I just ran into a really excellent piece by John Cutler (who is also new to me), Is Agile the Enemy (of Good Design)?. The whole thing is worth your time, but a couple bits in particular stood out to me in light of some ongoing conversations Ben Makuh about wisdom and folly in startup culture.

In particular, these two bits from other designers Cutler cites sum up a huge amount of what’s wrong with a lot of what passes for “Agile” and indeed for “startup culture”:

The stuff you’re talking about rarely happens. It is all about “ship, ship, ship”. We don’t pivot. We don’t refine. The product owner just wants to mark it done in Jira. The MVPs are an excuse to get crappy stuff out the door. I guarantee that if I am methodical with my prototype testing, I can come up with something better because I will expose it to users. Not AS great as doing it the perfect Agile way, but better than nothing. I mean I struggle even to do usability testing. So you know…yes in theory all that is good, but it doesn’t happen.

The enemy of both actual agilistas and the UX/design community in 2018 is, as John points out, short-term, output-centric thinking driven by a focus on short-term financial results, and all the cultural ramifications of this mindset.

These things are antithetical to the original ideas of the Manifesto for Agile Software Development. But they’re also, well… pretty common. As Cutler puts it:

…Agile — like many other things in cut-throat business — is often no match for the universal threats of output fetishism, success theater, and cutting corners. Trust me… these predated Agile.

And this is some hot fire here:

So where does this leave us? Designers have a right to be concerned. At least with waterfall no one prematurely yells “ship it” in the middle of the project. Designers have time to work instead of trying to jump on and off the sprint conveyor belt. And because the “thing” is built in a big batch, they have time to tackle the design problem holistically right from the beginning. “Good” waterfall beats abused Agile any day.

He’s not wrong. You should read the whole thing.

Ember.js, TypeScript, and Class Properties

2018-07-10T20:00:00-04:00

A few months ago, I wrote a mostly-complete series describing the state of using TypeScript with Ember in 2018. I got one very important thing wrong in that series, and I’m back with the correction!¹

In that series, I showed an example of a component definition; it looked like this:

import Component from '@ember/component';
import { computed, get } from '@ember/object';
import Computed from '@ember/object/computed';
import { inject as service } from '@ember/service';
import { assert } from '@ember/debug';
import { isNone } from '@ember/utils';

import Session from 'my-app/services/session';
import Person from 'my-app/models/person';

export default class AnExample extends Component {
  // -- Component arguments -- //
  model: Person;      // required
  modifier?: string;  // optional, thus the `?`

  // -- Injections -- //
  session: Computed<Session> = service();

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel = computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${get(this.model, 'firstName')};`;
    }
  );

  aComputed = computed('aString', function(this: AnExample): number {
    return this.lookAString.length;
  });

  isLoggedIn = bool('session.user');
  savedUser: Computed<Person> = alias('session.user');

  actions = {
    addToCollection(this: AnExample, value: string) {
      const current = this.get('aCollection');
      this.set('aCollection', current.concat(value));
    }
  };

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.get('aCollection').includes('ahoy')) {
      this.set('aCollection', current.concat('ahoy'));
    }
  }
}

The problem here is all the computed property assignments and the actions hash assignments. The fact that this sample code ever worked at all was… an accident. It wasn’t supposed to work. I noted at the time that this way of doing things had a performance tradeoff because computed properties ended up installed on every instance rather than on the prototype… and as it turns out, that was never intended to work. Only the prototype installation was supposed to work. And as it turns out, the ES5 getters implementation of computed properties which landed in Ember 3.1 broke every computed property set up this way.

So if you can’t use class properties for this… how do you do it? There are two ways: the .extend() hack I mentioned previously, and decorators. (The Ember Decorators docs include a discussion of this topic as well—see their discussion of class fields.)

Note that throughout I’m assuming Ember 3.1+ and therefore ES5 getter syntax (this.property instead of this.get('property')).

`.extend()`

The first workaround uses .extend() in conjunction with a class definition. I originally wrote about this approach:

If you need the absolute best performance, you can continue to install them on the prototype by doing this instead…

As it turns out, it’s more like “If you want your app to work at all…”

Here’s how that would look with our full example from above. Note that there are three things which must go in the .extend() block with this approach: injections, computed properties, and the actions hash.

import Component from '@ember/component';
import { computed, get } from '@ember/object';
import { inject as service } from '@ember/service';
import { assert } from '@ember/debug';
import { isNone } from '@ember/utils';

import Person from 'my-app/models/person';

export default class AnExample extends Component.extend({
  // -- Injections -- //
  session: service('session'),

    // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel: computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${this.model.firstName};`;
    }
  ),

  aComputed: computed('aString', function(this: AnExample): number {
    return this.lookAString.length;
  }),

  isLoggedIn: bool('session.user'),
  savedUser: alias('session.user') as Person,

  actions: {
    addToCollection(this: AnExample, value: string) {
      this.set('aCollection', this.aCollection.concat(value));
    }
  },
}) {
  // -- Component arguments -- //
  model!: Person;     // required
  modifier?: string;  // optional, thus the `?`

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.aCollection.includes('ahoy')) {
      this.set('aCollection', this.aCollection.concat('ahoy'));
    }
  }
}

There are three main things to note here.

First, check out the session('service') injection. We need the name of the service being injected for TypeScript to be able to resolve the type correctly (which it does by using “type registries,” as discussed briefly in this footnote in my series earlier this year). The alternative is writing session: service() as Session—a type cast—which is fine but isn’t particularly idiomatic TypeScript.

Second, notice that we do have to use a type cast, as Person, for the savedUser definition. While many computed property macros and the computed helper itself can properly infer the type of the resulting computed property, macros which accept nested keys do not and cannot. Thus, bool can resolve its type to a boolean, but readOnly or alias have to resolve their type as any. The value passed to them could be a strangely shaped string key on the local object (['like.a.path']: true) or an actual path through multiple objects. (This is the same limitation that means we cannot do nested get lookups.)

Third, as I noted even when we were doing this the wrong way, with class field assignment, we need to explicitly specify the type of this for callback passed in to define the computed properties. In the context of a .extend() invocation, though, this sometimes falls down. You’ll see an error like this:

‘AnExample’ is referenced directly or indirectly in its own base expression.

This doesn’t happen for all computed properties, but it happens often enough to be very annoying—and it always happens with Ember Concurrency tasks. (More on this below.) This problem was actually the original motivation for my experimentation with assigning computed properties to class fields.

This set of problems with defining computed properties and injections in an .extend() invocation is a major motivator for my team in eagerly adopting decorators.

Decorators

The cleaner, but currently still experimental, way to do this is to use Ember Decorators.² To use these, you should run ember install ember-decorators and then set the experimentalDecorators compiler option to true in your tsconfig.json.

Once you’ve installed the decorators package, you can update your component. In general, the imports match exactly to the Ember module imports, just with @ember-decorators as the top-level package rather than @ember. Here’s how our component looks using decorators:

import Component from '@ember/component';
import { assert } from '@ember/debug';
import { isNone } from '@ember/utils';

import { action, computed } from '@ember-decorators/object';
import { alias, bool } from '@ember-decorators/object/computed';
import { service } from '@ember-decorators/service';

import Session from 'my-app/services/session';
import Person from 'my-app/models/person';

export default class AnExample extends Component {
  // -- Component arguments -- //
  model!: Person;     // required
  modifier?: string;  // optional, thus the `?`

  // -- Injections -- //
  @service session: Session;

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  @computed('model.firstName')
  get fromModel(): string {
    return `My name is ${this.model.firstName}`;
  }

  @computed('aString')
  get aComputed(): number {
    return this.aString.length;
  }

  @bool('session.user') isLoggedIn: boolean;
  @alias('session.user') savedUser: Person;

  @action
  addToCollection(this: AnExample, value: string) {
    this.set('aCollection', this.aCollection.concat(value));
  }

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.aCollection.includes('ahoy')) {
      this.set('aCollection', this.aCollection.concat('ahoy'));
    }
  }
}

First, notice that using decorators switches us to using actual ES5 getters. This is exactly the same thing that RFC #0281 specified, and which was implemented for Ember’s traditional computed property and injection functions in Ember 3.1 to unlock . What’s extra nice, though, is that decorators are backwards compatible all the way to Ember 1.11. (You won’t get the ES5 getters on versions prior to to Ember 3.1—there the decorators just install things on the prototype—but you will at least get the correct behavior.)

Second, note that we don’t get type inference for the computed property macros like @bool here. That’s because decorators are not currently allowed to modify the type of the thing they’re decorating from TypeScript’s perspective. Now, decorators can—and do!—modify the type of the thing they decorate at runtime; it’s just that TS doesn’t yet capture that. This means that all decorated fields will still require type annotations, not just a subset as in the .extend() world. It’s annoying—especially in the case of things like @bool, where it really seems like we ought to be able to just tell TypeScript that this means the thing is a boolean rather than writing @bool('dependentKey') someProp: boolean.

This leads us to our final point to notice: we also need the type annotations for service (or controller) injections—but we do not need the string keys for them service injections.³ The net of this is that the injections themselves roughly equally ergonomic.⁴

// the old way
session: service('session'),
// the new way
@service session: Session;

Ember Concurrency

One other thing I need to draw your attention to here: I and a few others have taken a stab at writing type definitions for Ember Concurrency. Unfortunately, typings that type-check run smack dab into the fact that as of 3.1 that style doesn’t work; and typings that work cannot be type-checked at present. You can’t even use decorators to push your way to a solution. Nor is there a lot of hope on the horizon for this reality to change.

You can see some of the discussion as to why starting here in one pull request for them; it all gets back to the limitation I mentioned above: TypeScript doesn’t let you change the types of things with decorators. Unfortunately, there’s no reason to believe that will change anytime soon. This is a fundamental conflict between the Ember Object model and modern JavaScript—and specifically TypeScript’s understanding of it.

I am still mulling over solutions to that problem (as are others), and we’ll be continuing to work on this idea in #-topic-typescript in the Ember Community Slack (and publicizing any good ideas we come up with there in a searchable location, of course). For today, the best thing you can do is explicitly set the this type to any for the task property generator function callback, and use type casts internally if you look up services or other properties from the containing object.

Summary: mea culpa

Sorry again to everyone I misled along the way with my earlier, very wrong advice! Hopefully this helps clear up the state of things and will help you keep from falling into this tar pit going forward!

I don’t feel too bad about having gotten in wrong: no one who read the posts noticed the problem at the time, and it was subtle and easy to miss… because, at the time, everything actually worked.↩
It’s experimental because decorators are still only at Stage 2 in the TC39 process. They may advance at this month’s meeting.↩

If you’re using a non-default name, like specialSession, for the name of the property, the usual rules apply for injections. In that case, you’d write the injection like this:

import Component from '@ember/component';
import { service } from '@ember-decorators/service';
import Session from 'my-app/services/session';

export default class AnExample extends Component {
  @service('session') specialSession: Session;
}

↩

Files do get an extra import in the decorator version… but as it happens, I’m more than okay with that; I’d actually prefer explicit imports of dependencies personally.↩

Client-Side Ideas for Server-Side Apps

2018-06-07T16:00:00-04:00

A quick note: I drafted this back in June, but forgot to actually publish it!

I’ve been working on the design of a particular website I maintain (not this one; keep your eyes open), and besides the fact that I have learned a lot about web design in general in the years since I originally built that site, I discovered that I desperately want to use a component-drive model for developing sites on the client.

In my day job, I’m used to breaking down my application into discrete components with their own responsibilities. I’ve gotten spoiled by the component-driven model that dominates the front-end web development world now. (My tool of choice is usually Ember, but you’d get the same with React or Vue or whatever else.) And on the server development side, I’m desperately missing those.

I’m using Pelican for this particular site because that’s what it’s been built on for the past few years and I have no desire to change it at the moment. And that means using Jinja2 for templating. And Jinja2 has no notion of components. Partials, yes—with all the implicit context you have to carry around in your head. It has a few different ways you can sort of hack your way to something sort of vaguely component-like using some of its fancy features. But without any kind of “argument” or “return value”/yielding (a la the ideas I discussed in this post). All of the solutions available in any of these server-side frameworks for breaking up pages are partial-style: which means they’re basically just dumb string includes!

There’s nothing like the way I solve this problem in an Ember app every single day: components. There’s no particular reason that the same component-based approach that has flourished on the client can’t be done on the client side. It just… hasn’t, mostly. Which is kind of weird.

Until this week, projects like Gatsby in the React world made no sense to me at all. It seemed like using a sledgehammer to kill a spider. But after this week, I’m suddenly very interested in it—and I might in fact experiment with some server-side component-driven approaches to this at some point in the future—because a couple of days mucking with Jinja2 has me desperately wishing for a good old Ember or React component.

As an aside: people talk about client-side development being overly complicated. I know some of what they mean, but the truth is that my experience hacking on this over the last week has actually served to remind me of just how great the tooling is in this world.

It’s true that there’s more complexity in many ways to building things with Ember or React or whatever other JS-powered client-side framework than with plain-old HTML. It’s more complex even than with something like Jinja2 or Liquid or whatever other server-side templating language you use. There’s good reason for that complexity, though: it comes with more power and more expressiveness. And the thing many critiquing the front-end seem to miss is that once you are used to having that power and expressiveness, it’s really painful to go back to not having it.

Sum Type Constructors in TypeScript

2018-05-31T07:00:00-04:00

A pretty common pattern I’ve seen is to have three basic states for some kind of HTTP request: loading, failure, and success. Since each of these has its own associated date, it’s a really good fit for a discriminated union or sum type. In a language like Elm (or F^♯ or Haskell or PureScript or…) you’d write that basically like this:

module Fetch exposing (State)

type alias HTTPStatusCode = Int
type alias ErrorData = { code: HTTPStatusCode, reason: String }

type State a
    = Loading
    | Failure ErrorData
    | Success a

Because I find that pattern extremely helpful, I’ve at times gone out of my way to replicate it in TypeScript. And what you get is… verbose. It’s a necessary evil, given what TypeScript is doing (layering on top of JavaScript), and so much so that I wouldn’t actually recommend this unless you’re already doing this kind of programming a lot and find it pretty natural. If you are, though, here’s how you get the equivalent of those four lines of Elm in TypeScript:

type HttpStatusCode = number;

export enum Type { Loading, Failure, Success }

export class Loading {
  readonly type: Type.Loading = Type.Loading;

  static new() {
    return new Loading();
  }
}

type ErrorData = { code: HttpStatusCode, reason: string };

export class Failure {
  readonly type: Type.Failure = Type.Failure;
  constructor(readonly value: ErrorData) {}

  static new(value: ErrorData) {
    return new Failure(value);
  }
}

export class Success<T> {
  readonly type: Type.Success = Type.Success;
  constructor(readonly value: T) {}

  static new<A>(value: A) {
    return new Success(value);
  }
}

export type FetchState<T> = Loading | Failure | Success<T>;
export const FetchState = {
  Type,
  Loading,
  Failure,
  Success,
};

export default FetchState;

That’s a lot more code to do the same thing. Even if you dropped the static constructors—which you really don’t want to do, because then you can’t use them in a functional style but have to use new Loading() or whatever to construct them.

You can make this work. And I do. And honestly, it’s amazing that TypeScript can do this at all—a real testament to the sophistication of the TypeScript type system and the ingenuity that has gone into it.

But have I mentioned recently that I’d really prefer to be writing something like F^♯ or Elm than TypeScript?

#EmberJS2018, Part 4

2018-05-29T07:45:00-04:00

Following the example of the Rust community, the Ember.js team has called for blog posts as the first step in setting the 2018 roadmap (which will formally happen through the normal RFC process). This is my contribution.

There are three major themes I think should characterize the Ember.js community and project for the rest of 2018:

Finishing What We’ve Started
Doubling Down on Documentation
Defaulting to Public for Discussions
Embracing the Ecosystem (this post)

Over the last few weeks, I’ve talked about a few big ideas that I think the Ember.js community should go after in 2018 which will help the framework excel over the next few years. This last one (like Part 3 before it) is more a culture shift than a matter of things to build.

We need to shift from a posture of defensiveness about Ember.js to one of embracing the ecosystem, and embracing our role in the ecosystem.

It’s easy to end up in an us-vs.-them mentality when looking at different libraries and frameworks. It’s doubly easy to go there when you often hear “Isn’t Ember dead?” or variations on that theme. We should avoid that way of thinking anyway. And there are three big pieces to this: contributing outwards, smoothing the paths into Ember from other ecosystems, and embracing the rest of the ecosystem.

Contributing outwards

There is genuinely great stuff happening all over the place in the front-end, and many of the things we love about working with Ember today have come directly out of e.g. React—hello, “data-down-actions-up”! The same is true in reverse: Ember has contributed many important ideas to the broader front-end ecosystem, from its early emphasis on rigorously linking URLs and application state to helping pioneer and popularize the use of good command line tooling, to more recent emphasis on compilation as a way of solving certain classes of problems.

So as we build all of these things, one of the best things to do—and, I believe, one of the ways we help Ember grow!—is think about how our work can benefit the larger ecosystem. When you build a library, you should consider whether there are parts of it that don’t have to be Ember specific. For example, a colleague and I recently built out the foundation of a solution for well-rationalized form-handling.¹ We build it in two pieces, though: a core library in TypeScript that will work as well in Vue or React as in Ember, and an Ember component library that consumes that core functionality.

The more we can take that tack in general, the better. It’s the first piece of making the gap between people’s experience in other parts of the front-end ecosystem and the Ember part smaller. Ember will seem much more interesting if people find themselves often getting value out of things we’ve built.

Smoothing the paths in

The flip side of this is figuring out ways to make it easier for people coming into Ember.js to map patterns from their existing experience onto the framework’s tools and patterns. The simple reality is that there are far, far more developers familiar with React, Angular, and Vue than with modern Ember.js. Ember genuinely has a lot to offer there, but we need to make it easier for people to see that value and to recognize how it’s a lot like the good parts of what they already know!

This is primarily a communications effort; it means changes to the docs and to the homepage, but also to what we do in blog posts and tutorials and talks as a community!

At the highest level, I cannot recommend strongly enough the model suggested by Chris Garrett in his #EmberJS2018 post: treat Ember.js (both in the docs and also in our presentations and communications about it) as a component-service framework. This not only maps more easily to patterns people know from other communities, it has the really important effect of demystifying a lot of the “magic” that seems perplexing in the framework, especially around Ember Data—which is, after all, just a service you can inject!

When we write blog posts, we can accomplish a lot of this simply by being aware of the rest of the ecosystem and making analogies there. You can see an example of how I’ve started trying to do this in my recent blog post on higher-order components in Ember.js. It was just one little line:

In React, the [higher-order components] pattern as a whole is often known as the renderProps pattern, for the way you most often accomplish it. It’s all the same idea, though!

That’s not a lot of extra work, but it means that if someone searches for “renderProps Ember.js” there now exists a blog post which will help someone map there existing knowledge over! I wasn’t writing a “how to do React renderProps in Ember” post—but I still smoothed the path in just a little bit. We should be doing that everywhere we can. It’s usually not a lot of effort to make those kinds of moves in talks or blog posts, but the yield is high: Ember stops being some super weird foreign entity and starts looking like a variation on a theme.

There is also a much larger effort we do need to undertake to make that story clearer on the home page and in the documentation—an effort that I know is already very much in consideration from chatting with the really amazing crew in #-team-learning on Slack. In the how you can help bucket: seriously please go into that channel and start chipping away at small tasks! There’s (always!) way more work to be done than hands to do it.

I think this also means prioritizing technical work that eases this. The sooner we can land the Glimmer component model, the better. The sooner we can hash out a more cogent story on routes and controllers and components, the better. The sooner we can make “npm-install-your-way-to-Ember” an actually viable strategy, the better. Because each of those things makes Ember dramatically more accessible to people working in other ecosystems today; each lowers the barrier to entry in some substantial way; and the combination of them all makes it far more viable for someone to try Ember in an existing application.

Embracing the rest of the ecosystem

The final piece of this is actively embracing the best parts of the rest of the ecosystem.

We as a community need to avoid defensiveness and recognize that there’s a lot of good in the rest of the front-end space. I understand how it can be easy to feel defensive. Being dismissed, having people be surprised that the project even still exists, etc. gets really old after a while. But however reasonable that defensiveness is, it’s ultimately counterproductive. It makes us hold onto things we don’t need to hold onto, and it makes us ignore things that might benefit us, and as a result it can make us needlessly weird technically.

Needless weirdness is an important idea I’d love for us to keep in mind. Any time you’re willing to move more slowly, to let the “new shiny” bake for a while to see whether it’s genuinely worth investing in, you’re going to seem weird. Likewise when you strongly embrace stability, in a broader ecosystem which hasn’t. Likewise when you value convention over configuration, in a broader ecosystem which hasn’t. But it’s important to be able to distinguish between needful and needless weirdness.

We should have regular conversations as a community—through RFCs, through forum threads, through blog post arguments, etc.—about what’s needful weirdness, and what has become needless weirdness. (Because which weird things are needful change over time!) We should gleefully embrace the needful weirdness. But we should equally gleefully drop the needless weirdness.

What makes Ember special is, by and large, not the specific technical implementations we’ve landed on.² What makes Ember valuable is having a coherent top-to-bottom story and a rich community with a commitment to aggressively seeking out shared solutions, and an even deeper commitment to providing good migration paths forward when we change things.

But here’s the thing: those values are increasingly (if slowly) being embraced outside the Ember ecosystem as well. Ember can contribute and even lead in many ways here—but only if we start actively embracing the good of other parts of the front-end ecosystem.

For example: I’ve heard more times than I can count over the last few years that our use of Broccoli.js is really important for Ember, and the reality is… that isn’t true. We could have built on top of just about any solution, and it would have been fine. Broccoli does have some advantages; it also has some real disadvantages (one of which is that we’re the only ones using it!), and we should forthrightly acknowledge those. By the same token, if Webpack is working well for many people, let’s neither trash it in discussion nor ignore it in implementation. Instead, let’s make it easy for people to integrate Webpack into the Ember world.

That doesn’t oblige us to chuck out our existing build tooling! It just means making our own build pipelines robust enough to interoperate well with other packaging systems. And that’s precisely what the Ember CLI team has been doing! This needs to be our pattern across the board going forward.

It’s truly well and good to have made a call a few years ago, and to be going out of our way to mitigate the costs of churn. At the same time, we need to communicate—to a degree that probably feels like overcommunicating to the people who already understand all these decisions!—so that both the original rationales and the current status are accessible to all the people who weren’t there when the decisions were made.

Insofar as it’s true that Broccoli and Webpack solve different problems, explaining how Broccoli and Webpack actually solve meaningfully different problems —or at least, excel at solving different problems—is one of the most important things we can do as well. Props to Chris Thoburn (@runspired) for doing this in a few different contexts recently, but we need a lot more of it—because it’s one example I think most people both inside and outside the Ember community have just kind of scratched their heads at for a long time (me included).

Again: I take the Broccoli/Webpack example simply because it’s an obvious one. The broader point is that we need to find ways to embrace the shared solutions which emerge not only in the Ember community but in the front-end ecosystem as a whole, even as we also do the hard work to make our own shared solutions useful to the rest of the front-end ecosystem. That two-way exchange will benefit us, and smooth the paths in for newcomers, and benefit the rest of the ecosystem, too—and that’s a huge win. Because in a very real sense, we front-end developers are all in this together.

Keep your eyes open; you’ll see a blog post announcing that along with a full set of documentation for it sometime in the next month or so!↩
To be clear: many, though certainly not all, of those specific implementations I like, but that’s beside the point.↩

Higher-Order Components in Ember.js

2018-05-26T14:00:00-04:00

One of the most powerful patterns in programming is the idea of higher-order functions: functions which can take other functions as arguments or return them as their return values. If you’ve spent much time at all working in JavaScript, you’ve certainly encountered these—whether you’re using Array.map to transform the values in an array, or passing a function as an argument to an event handler.

The same pattern is incredibly useful in building components, and most modern front-end frameworks support it—including Ember.js! (In React, the pattern as a whole is often known as the renderProps pattern, for the way you most often accomplish it. It’s all the same idea, though!)

In this little post, I’ll show you how to build a small “higher-order component” in Ember.js, hopefully demystifying that term a little bit a long the way. (If you just want to see how the pieces fit together, you can see the finished app in this repo.)

I’m going to be using classes and decorators throughout. Both are very much ready-to-go in Ember, and I commend them to you! I’m also going to be using some of the new optional features available in Ember 3.1+ to use template-only components!

Note that one of the most important consequences of this is that arguments have to be referenced as @theArgumentName rather than just theArgumentName in templates. The reason is precisely that there is no backing JavaScript component. In old-school Ember.js components, {{theArgumentName}} is implicitly turned into {{this.argumentName}}, which does a lookup on the backing component. In Glimmer-style components—of which these are the first part—arguments live on a designated args property and are accessible in templates via @theArgumentName instead.

Higher-Order Components, What Are They

Just like with a “higher-order function,” all we mean when we talk about a “higher-order component” is a component which takes other components as arguments, returns other components itself (in Ember’s case via yield in a template), or both.

The thing we’re actually going to build here is a “modal” which accepts an optional button as an arguments, and which yields out a component for dividing the modal into sections visually so you can pass your own content in and have it look just right. This is closely based on a component my colleagues and I at Olo built recently, just with some of our specific details stripped away to get at the actually important bits. Here’s what it looks like in practice:

a modal with sectioned text and a close button

The goal for the button arguments is to let the modal be able to render the button the caller passes in, while not being concerned with the functionality of the button. Otherwise, we’d have to tie the “API” of the modal to the details of button behavior, bind more actions into it, etc.

The goal for the yielded sectioning component is for whatever is rendering the modal itself to be able to pass content in and get it chunked up however the modal decides is appropriate—the modal can display its own styles, etc.—without having to worry about the details of applying classes or sectioning up the content itself.

In short, we want to separate our concerns: the modal knows how to lay out its contents and where to put buttons, but it doesn’t want to have to know anything about what the buttons do. The most complicated interaction in the world could be going on, and the modal won’t have to care. Likewise, things using the modal can pass content and buttons into it, and let the modal manage its own layout and so on without having to be concerned with the details of that. So what does that look like in practice?

The approach I use here builds on the “contextual components” pattern in Ember.js. The main new idea is that the context includes components!

Implementing It

We have three components here:

a button
a modal
a modal section

Since Ember.js still (for now!) requires component names to be at least two words separated by a dash, we’ll just call these x-button, x-modal, and x-modal-section.

`x-button`

The button component, we’ll keep pretty simple: it’s just a button element with a given label and an action bound to it:

<button class={{@buttonClass}} type='button' {{action @onClick}}>
  {{@label}}
</button>

`x-modal`

The x-modal has the meat of the implementation.

<div class='modal-backdrop'></div>
<div class='modal'>
  <div class='modal-content'>
    {{yield (hash section=(component 'x-modal-section'))}}
  </div>

  {{#if @button}}
    {{component @button buttonClass='modal-button'}}
  {{/if}}
</div>

The two things two notice here are the yield and the component.

The yield statement yields a hash with one property: section. Yielding a hash is a convenient pattern in general. Here, we’re doing it to make the API nicer for users of this component. It means that if we name the yielded value |modal| when we invoke this, we’ll be able to write modal.section to name this particular yielded item. (You’ll see exactly this below.)

We use the component helper twice: once as the value of the section key in the yielded hash, and once for the button below. In both cases, the helper does the same thing: invokes a component! While the most common way to render a component is with its name, inline—like {{x-modal}}—you can always render it with the component helper and the name as a string: {{component 'x-modal'}}. This lets you render different components dynamically!

Let’s remember our initial analogy: the same way you can pass different functions to a higher-order function like Array.prototype.map, you can pass different components to a higher-order component like our x-modal here. And just like you can return a function from a higher-order function, we can yield a component from a higher-order component. Just like higher-order functions, the function passed in or returned just has to have the right shape.

For example, the argument to Array.prototype.map needs to be a function which performs an operation on a single item in the array (and maybe also the index) and hands back the result of that operation. Similarly, the button argument to our x-modal needs to accept a buttonClass component so that the modal can apply some styling to it. The same thing holds for the component being yielded back out: it has an API you should use to invoke it, just like any other.¹

All of this gets at something really important: you can think of components as just being pure functions: they take some input in the form of arguments, and give you the output of what they render and what they yield—and they always give you the same rendered HTML and the same yielded values for the same inputs. They’re just functions!

`x-modal-section`

The x-modal-section component is the simplest of all of these: it has no behavior, just some styling to actually chunk up the content:

<div class='modal-section'>
  {{yield}}
</div>

Application controller and template

Now, let’s use in the context of the application template, where we can see how the pieces all fit together. First, let’s see the application controller backing it—nothing unusual here, just a simple toggle to show or hide the modal.²

import Controller from "@ember/controller";
import { action } from "@ember-decorators/object";

export default class Application extends Controller {
  constructor() {
    super(...arguments);
    this.showModal = false;
  }

  @action
  showIt() {
    this.set("showModal", true);
  }

  @action
  hideIt() {
    this.set("showModal", false);
  }
}

Now for the interesting bit—the template where we invoke x-modal and use its higher-order-component functionality:

{{#if showModal}}
  {{#x-modal
      button=(component 'x-button'
        label='Close modal!'
        onClick=(action 'hideIt')
      )
      as |modal|
  }}
    {{#modal.section}}
      Here is some content!
    {{/modal.section}}

    {{#modal.section}}
      Here is some other content.
    {{/modal.section}}

    {{#modal.section}}
      <p>The content can have its own sections, as you'd expect!</p>
      <p>Nothing crazy going on here. Just a normal template!</p>
    {{/modal.section}}
  {{/x-modal}}
{{/if}}

<button class='button' {{action 'showIt'}}>Show modal</button>

<!-- some other content on the page -->

We invoke the block form of x-modal just like we would any block component, and we get back the thing it yields with as |modal|. However, one of the arguments we pass to it is a component. But modal is a hash (an object!) with a property named section, which is the x-modal-section component.

Again, you can think of this like calling a function with one function as an argument and getting another function back as its return value—that returned function being something we could call over and over again once we had it.

Here, we “call the function”—invoke the x-modal component—with component 'x-button' as its argument, and the returned modal.section is a component we can invoke like a normal component.³ We could even pass it into some other component itself if we so desired.

And that’s really all there is to it!

Summary

“Higher-order components” aren’t necessarily something you need all the time, but they’re really convenient and very powerful when you do need them. They’re also a lot less complicated than the name might seem! Components are just things you can pass around in the context of a component template—they’re the functions of Handlebars.⁴

Splitting things into components like this does increase complexity, and in particular it can increase the mental overhead of keeping track of how the pieces fit together. However, they also let us cleanly separate different pieces of functionality from each other. Doing it this way means that our modal can be concerned about positioning a button without needing to expose an API for all of the button’s own mechanics for handling clicks and performing whatever actions necessary. That makes our modal and our button way more reusable across our application. The button can be used wherever a button is useful, and the modal doesn’t need to know or care anything about it. Likewise, the button has no need to know anything about the context where it’s being used; from the button component’s perspective, it just gets wired up to some actions as usual. The same thing goes for the modal sections: they let us abstract over how the DOM is laid out, what classes are applied to it, and so on—they chunk up the modal, but the modal itself maintains responsibility for how that chunking up happens. And the caller doesn’t even have to use that; it’s just a tool that’s available for that purpose.

To sum it all up, I’ll just reiterate my earlier description: components are just like pure functions: the same inputs give you the same outputs—and, just like functions, those inputs and outputs can be other functions, that is, other components.

If you want a good way to document the things a component yields, check out ember-cli-addon-docs, which can read an @yield JSDoc annotation.↩
And it could just as well be a component; the top-level controller template is just where we put our main app functionality.↩

We could also simplify this since we’re only returning one component, and if we had the full Glimmer component story, this could look very nice:

<Modal @button={{component 'Button'}} as |Section|>
  <Section>
    Some content!
  </Section>
  <Section>
    Some more content!
  </Section>

  <Section>
    <p>The content can have its own sections, as you'd expect!</p>
    <p>Nothing crazy going on here. Just a normal template!</p>
  </Section>
</Modal>

↩

If you’re inclined to “well actually” me about helpers being the real functions of Handlebars templates: in the Glimmer VM world, helpers are just a kind of component.↩

#EmberJS2018, Part 3

2018-05-23T07:30:00-04:00

There are three major themes I think should characterize the Ember.js community and project for the rest of 2018:

Finishing What We’ve Started
Doubling Down on Documentation
Defaulting to Public for Discussions (this post)
Embracing the Ecosystem

One of the small changes I think would substantially improve the Ember.js ecosystem is: defaulting to public for discussions among the core team. Indeed: for any open-source project with community involvement like Ember.js has, that should be the default. Not the only option, just the default option.

There is plenty of value in having private channels for discussion in contexts like this. Sometimes you have to deal with something awkward or socially difficult. Sometimes you have already taken the community’s input and just have to come to a decision about what to do on something. Private channels are useful.

But: they shouldn’t be the default. They should be what you turn to when you’re in one of those particular kinds of situations which require it. The default should be public discussion and interaction.

Over the last year, the maintainer-ship (and therefore decision-making) of ember-cli-typescript and the surrounding TypeScript ecosystem has grown from being pretty much just me to being a small group of four of us: Derek Wickern, Dan Freeman, James Davis, and me. We have the “final say,” so to speak, on the things we’re doing with the addon and the typings and so on. (What that actually means in practice is mostly just we all try to shoulder the burden of staying on top of pull requests.) And we have a private channel for discussions as a “core team” for projects in the typed-ember organization.

But: it’s not the default. It’s what we turn to when we’re in one of those particular kinds of situations which require it. The default is public discussion and interaction.

And this isn’t just an unspoken norm or something. As a team, we all explicitly agreed that we default to public. Pretty much the only times we chat in our private channel is if we’re figuring out how to diffuse an awkward situation kindly, or if we’re adding someone else to the team. Otherwise, we try to have all our discussions in the GitHub issues for the projects or the #topic-typescript room in the Ember Community Slack.

This has a few major effects, as I see it:

No one should feel left out or in the dark about what we’re up to. Even if we’re hashing out crazy-seeming ideas for how to move stuff forward, it’s all there for everyone to see. This includes neat things like Dan Freeman’s proof-of-concept on type-checked templates, or our mad sprint (as a team!) to get some core improvements landed before I gave a workshop at EmberConf, or anything else we’re going after.
We’re obviously available for input on things as people have questions, because we’re interacting with each other in those public forums. And if we’d like to start moving some of the oft-repeated questions over to the Ember Discourse or to Stack Overflow, it’s still really helpful for people who are on the Slack to see that we’re there and available for help.
We get to see the regular pain points others run into. That often turns into issues, priorities, etc. for us as a group. The slowly growing issue tracking things we need to document is essentially a direct product of that constant cycle of interaction.
We get the benefit of input from others! If we’ve missed something, or simply failed to think of something, others in the community often haven’t. One prime example of this: the “registry” strategy we use for making things like Ember Data store, adapter, etc. lookups work came out of conversations with a community member (Maarten Veenstra) which happened many months before we were in a spot where we could land that kind of thing—and initially I was pretty skeptical of it, but they were totally right, and it’s now core to how Ember’s typings work!

I recommend—very strongly—that the Ember.js core team adopt the same strategy. Teams do need private channels sometimes. But they shouldn’t be the default. They should be for those particular circumstances which require it.

The biggest things I think could come out of this are:

A greater confidence from within the Ember.js community about what the core team is up to and where we’re going. Technical leadership seems to me to be about 10% technical brilliance and 90% clear communication. We have loads of technical brilliance; we need more communication!
More confidence in the trajectory of Ember.js from outside its existing community. Seeing that there is active leadership is essential for people to have confidence that choosing Ember.js is a good choice both today and for the medium-to-long-term.

And we need both of those—a lot—for Ember.js to continue to grow and thrive in the years ahead!

How To Bundle TypeScript Type Definitions

2018-05-21T07:00:00-04:00

One of the lessons that led to the True Myth 2.0.0 release was the difficulty of consuming the library under its original packaging strategy. There are a few things that are not obvious about how TypeScript type definitions get consumed when you’re first starting out, and a few things that seem like they should work don’t. This is my attempt to help you (and the people consuming your TypeScript libraries!) avoid the same pain I (and the people consuming mine) have felt.

The Problem

The problem is the result of the ways TypeScript resolves type definitions, and the kinds of type definition files it can (and cannot) generate for you.

TypeScript only properly resolves two kinds of type definition distributions automatically:

A single-file type definition, located anywhere in the package as long as package.json has a types key pointing to it.
Type definition module files in the root of the distributed package, mapping to the distributed modules of the package (wherever they live).

TypeScript will only generate a single-file type definition for the AMD and SystemJS standards—which cannot be imported with ES6 module imports. If you want to use an output mode which generates a JS file per originating TS file—Node, ES6, etc.—you will get individual TS module file type definitions as well. It is not that the type definition files themselves can’t be written to support Node or ES6-style module layouts in a single-file definition. To the contrary: hand-written definitions for libraries often do just that. It is just a matter of what the compiler supports generating.

The net of this is: if you want module type definitions to go with ES6 modules to import, they must live in the root of your distributed bundle.

However, most libraries I’m familiar with—because I work in the browser ecosystem, not the Node ecosystem—do not work with the root of their repository as the place where their source lives, or for the place where the output of their build process lives. It’s far more common to have a src directory and dist or build directory, the latter of which is where the build artifacts go.

The Solution

The solution—which we shipped for ember-cli-typescript some time ago, and which I switched to this past week for True Myth—is to have separate build artifacts for the type definitions and the JavaScript output. Put the JavaScript output in the dist or build directory as usual, without type declarations. Then, put the type definitions in the root of the repository.

In the case of both ember-cli-typescript and True Myth, we’re doing the type generation step in the prepublishOnly hook and cleaning it up in the postpublish hook. Your package.json might look like something like this, assuming your tsconfig.json is set to generate JavaScript artifacts in dist as your build directory.

{
  "scripts": {
    "ts:js": "tsc",
    "ts:defs": "tsc --declaration --outDir . --emitDeclarationOnly",
    "prepublishOnly": "yarn ts:js && yarn ts:defs",
    "postpublish": "rm -r *.d.ts dist"
  }
}

(If you have nested modules, your postpublish hook there should clean up the generated folders as well as the generated files.)

You can see the full setup I built for True Myth—which generates type defs along these lines, as well as both CommonJS and ES6 modules—in the repository:

package.json—note especially the "scripts" configuration
root tsconfig.json, with derivedCommonJS tsconfig.json and ES6 tsconfig.json files.

This isn’t an especially complicated thing, but the scenario leading to the need for this is common enough, and the dance frustrating enough and easy enough to get wrong, that I really wish the TypeScript team would make it possible to generate single-file type definitions for all kinds of JavaScript module systems.

Rust is Incredibly Productive for CLIs

2018-05-20T08:35:00-04:00

There are reasons I’m a Rust fanboy. One of them is the kind of thing I proved out to myself today—again, because I’ve had this experience before, albeit not with anything quite this “complicated.”

I built a little tool in Rust to convert Evernote exports (in their custom .enex XML format) to Markdown files with YAML metadata headers—mostly just to see how quickly and effectively I could do it, because I’ve never actually had an excuse to use Serde and I thought this might be a nice spot to try it.

There’s a lot this little library doesn’t do. (Like include the creation and modification timestamps in the header, for example.) But all of those things would be very straightforward to do. I built this functioning little “script” in about two hours. For context: I’ve taken multiple passes at this in Python—which in the way people normally think about these things should be way easier—and I’ve failed both times.

Rust’s compiler just helps you out so much along the way, not only with the type-checking but with the really amazing metaprogramming capabilities you get with it. Being able to slap #[derive(Deserialize)] on a struct and a couple attributes on struct fields and having it Just Work™ to deserialize XML into local types is mind-blowing. (The only thing I know of that’s playing the same game is F^♯ type-providers. I’d love to hear about similar capabilities in other languages!)

I’m basically at the point where if I need a small command-line tool, I write it in Rust, not in a conventional scripting language like Python, because the benefits I get more than outweigh whatever small extra amount of mental overhead there is. And there’s not much of that mental overhead anyway for this kind of thing! As you can see in the actual code, I make free and liberal use of expect for this kind of tool.

It’s also hard to oversell the ecosystem—even as relatively nascent as it is compared to some much older languages, the tools which exist are just really good. This project uses Serde for deserializing from XML and serializing to YAML; Regex; Clap for command line parsing; a nice little wrapper around pandoc; and, superpower even among superpowers, Rayon: free parallelization.

Rust is, in short, very productive for things in this space. Far more than you might expect from the billing. Yes, it’s a “systems programming language” and you can write operating systems with it. But it’s also just a really great tool for all sorts of domains, including little CLI tools like this one.

Destructuring with True Myth 1.3+

2018-05-19T12:20:00-04:00

I just realized a neat capability that True Myth 1.3+ unlocks: you can now use destructuring of the value property on Just and Result and the error property on Error instances.

With Maybe instances:

import Maybe, { just, nothing, isJust } from 'true-myth/maybe';

const maybeStrings: Maybe<string>[] =
  [just('hello'), nothing(), just('bye'), nothing()];

const lengths = maybeStrings
  .filter(Maybe.isJust)
  .map(({ value }) => value.length);

With Result instances:

import Result, { ok, err } from 'true-myth/result';

const results: Result<number, string>[] =
  [ok(12), err('wat'), err('oh teh noes'), ok(42)];

const okDoubles = results
  .filter(Result.isOk)
  .map(({ value }) => value * 2);

const errLengths = results
  .filter(Result.isErr)
  .map(({ error }) => error.length);

None of this is especially novel or anything. It was just a neat thing to realize after the fact, because it wasn’t something I had in mind when I was making these changes!¹

This was a very strange experience. There’s nothing quite like learning something about a library you wrote.↩

The Chinese Room Argument

2018-05-19T11:20:00-04:00

I took a bunch of half days last week, because goodness but I was tired. Too long running at full-throttle, and I’d been running out of steam as a result. And what did I do instead, that ended up being so effective in recharging? Well, mostly… read literature reviews on interesting topics in philosophy, at least for the first few days. Dear reader, I am a nerd. But I thought I’d share a few of the thoughts I jotted down in my notebook from that reading.¹

“The Chinese Room Argument”

This was an argument whose influence I’ve certainly encountered, but the actual content of which I was totally unfamiliar with.²

The argument, in exceedingly brief summary, is that “formal operations on symbols cannot produce thought”—that syntax is insufficient for conveying semantics. Searle made the argument by way of a thought experiment; a reference in a review on “Thought Experiments” I may post about later is how I found my way to the discussion. That thought experiment supposes a man being handed symbols (under a door, perhaps) which are questions in Chinese, who has a set of rules for constructing correct answers to those questions, also in Chinese—but the man himself, however much he gives every appearance of knowing Chinese to the person passing in questions by way of the answers he gives, does not in fact know Chinese. He simply has a set of rules that allow him to give the appearance of knowledge. The Chinese Room argument, in other words, is a(n attempted) refutation of the Turing Test as a metric for evaluating intelligence.

The rejoinders to this are varied, of course, and I encourage you simply to follow the link above and read the summary—it’s good.

There were two particularly interesting points to me in reading this summary: the Churchland response, and the Other Minds response. To these I’ll add a quick note of my own.

1: The Churchland response

Searle’s argument specifically addressed an approach to AI (and especially so-called “strong AI,” i.e. AI that is genuinely intelligent) that was very much in vogue when he wrote the article in the 1980s, but which is very much out of vogue now: rule-driven computation. One of the responses, which looks rather prescient in retrospect, was the Churchland reply that the brain is not a symbolic computation machine (i.e. a computer as we tend to think of it) but “a vector transformer”… which is a precise description of the “neural network”-based AI that is now dominating research into e.g. self-driving cars and so on.

The main point of interest here is not so much whether the Churchlands were correct in their description of the brain’s behavior, but in their point that any hypothesis about neural networks is not defeated by Searle’s thought experiment. Why not? Because neural networks are not performing symbolic computation.

2: The Other Minds response

The other, and perhaps the most challenging response for Searle’s argument, is the “other minds” argument. Whether in other humans, or in intelligent aliens should we encounter them, or—and this is the key—in a genuinely intelligent machine, we attribute the existence other minds intuitively. Nor do we (in general) doubt our initial conclusion that another mind exists merely because we come to have a greater understanding of the underlying neuro-mechanics. We understand far more about human brains and their relationship to the human mind than we did a hundred years ago; we do not therefore doubt the reality of a human mind. (Most of us, anyway! There are not many hard determinists of the sort who think consciousness is merely an illusion; and there are not many solipsists who think only their own minds exist.)

But the “other minds” objection runs into other intuitive problems all its own: supposing that we learned an apparently-conscious thing we interacted with were but a cleverly-arranged set of waterworks, we would certainly revise our opinion. Which intuition is to be trusted? Either, neither, or in some strange way both?

And this gets again at the difficulty of using thought experiments to reason to truth. What a thought experiment can genuinely be said to show is complicated at best. Yet their utility—at least in raising problems, but also in making genuine advances in understanding the world—seems clear.

Lowered standards

The other thing I think is worth noting in all these discussions is a point I first saw Alan Jacobs raise a few years ago, but which was only alluded to in this literature review. Jacobs cites Jaron Lanier’s You Are Not A Gadget. (I don’t have a copy of the book, so I’ll reproduce Jacobs’ quotation here.)

But the Turing test cuts both ways. You can’t tell if a machine has gotten smarter or if you’ve just lowered your own standards of intelligence to such a degree that the machine seems smart. If you can have a conversation with a simulated person presented by an AI program, can you tell how far you’ve let your sense of personhood degrade in order to make the illusion work for you?

This is one of the essential points often left aside. Is the test itself useful? Is “ability to fool a human into think you’re human” actually pointing at what it means to be intelligent? This is sort of the unspoken aspect of the “other minds” question. But it’s one we ought to speak when we’re talking about intelligence!

For those of you following along at home: I wrote all but the last 100 or so words of this a week ago and just hadn’t gotten around to publishing it. It’s not the even more absurd contradiction to yesterday’s post on writing plans than it seems. Really. I promise.↩
It’s occasionally frustrating to find that there is so much I’m unfamiliar with despite attempting to read broadly and, as best I can, deeply on subjects relevant to the things I’m talking about on Winning Slowly, in programming, etc. One of the great humility-drivers of the last few years is finding that, my best efforts to self-educate notwithstanding, I know very little even in the fields I care most about.↩

#EmberJS2018, Part 2

2018-05-18T22:00:00-04:00

There are three major themes I think should characterize the Ember.js community and project for the rest of 2018:

Finishing What We’ve Started
Doubling Down on Documentation (this post)
Defaulting to Public for Discussions
Embracing the Ecosystem

Part 2: Double down on docs

The best project in the world is useless without documentation. As such, my second major goal for Ember.js this year is to see our documentation story improve dramatically across a number of fronts. This is not just the kind of thing that’s important in principle or because we care about doing the right thing, though those alone are sufficient motivation. It’s also absolutely necessary for Ember to grow and thrive in the ways it deserves to in the years ahead.

To be clear: Ember’s story around documentation is pretty good and it continues to improve all the time. A few years ago, the base documentation was a mess and even figuring out where to start was hard. Today, Ember.js itself has great guides along with versioned-and-searchable API documentation. The gaps now are in the surrounding ecosystem and in the framework internals. That’s huge progress! But if we want Ember to excel, we need to go after both of these with gusto.

The surrounding ecosystem

Ember Data, Ember Engines, and perhaps most important Ember CLI and its core dependency Broccoli all desperately need documentation work just at the “how do you even use these things level.”

Broccoli.js in particular is core to pretty much everything in Ember’s ecosystem, and its docs today are in roughly the state Webpack’s were back in its sad 1.0 days. We should take a page out of our own history (and Webpack’s for that matter!) and make it easy for people to use Broccoli in whatever ways their apps need, and that mostly means documenting it!¹ Oli Griffith’s recent blog post series is an incredibly valuable first step in that direction. But we need really solid documentation for Broccoli itself, and also for the equally important plugin ecosystem which is the primary way people interact with it.
The docs for Ember CLI itself are decent, but they’re quite out of date and are about to be a lot more so because of the previously-mentioned packager bits. We need accurate and up-to-date guides and API docs for the CLI, and we also need clarity about the seams between Ember CLI and Broccoli—something I’ve only begun to become clear on after a year of hacking on ember-cli-typescript! This includes a number of kinds of documentation:
- up-to-date guides
- complete API documentation
- a “cookbook” of common patterns to use
The Ember Data docs need to be split into two parts: one for users of Ember Data, and one for people building Ember Data integrations and addons. Right now, all the docs are targeted squarely at implementors of Ember Data addons. This means that one of the pieces of the Ember ecosystem that’s in widest use (and is most distinct from the rest of the JS ecosystem!) is really, really hard to learn. This is the part of the framework I still struggle the most with, despite having worked full time on an Ember app for over two years now.
Ember Engines are really need for manually breaking up your app into discrete sections which can be worked on independently and even loaded dynamically as you need them, and they provide a different level of abstraction than route-splitting and other similar approaches. (Not necessarily better or worse, but different.) Unfortunately, most of the documentation hasn’t been touched in over a year. That means if you want to use Ember Engines, almost all of the information is in an example here and a Slack conversation there. We need to turn that sort of “tribal knowledge” into actual docs!

To be clear, the Ember docs team is doing great work and is already going after a lot of these areas; but there’s an enormous amount of ground to cover. They could use your help! Because if Ember is going to flourish in the year(s) ahead, we need good docs. And users are the people best-placed in all the world to help write docs.

So how you can help:

Open issues about things you don’t understand.
If you see an error in the documentation, open a pull request to fix it.
Volunteer to proofread or edit as new materials are produced. Yes, seriously: proofreading is incredibly valuable.
Volunteer to write documentation of things you do understand where you see gaps.

Framework internals

Every time I have started poking into Ember’s own codebase—to ship a fix for some small bug, or simply to understand the behavior of my own application—I have found myself stymied by a really serious issue. Almost nothing is documented. This is true of Ember proper, of Ember Data, of Ember CLI, of Broccoli’s internals… Everything I named above as being in need of user-facing documentation also desperately needs developer-facing documentation.

A lot of this happens naturally in projects developed organically by small teams. I’ve seen it in my own current job: the vast majority of our codebase is without any formal documentation, because it didn’t require it when we were a much smaller organization working on a much smaller codebase. But no project—whether private or open-source—can grow or thrive unless it becomes possible for new contributors to come in, understand the system as it exists, and start making changes effectively. “Tribal knowledge” is not a bad thing in some contexts, but it does not scale.

The Ember.js ecosystem needs developer documentation of several sorts, then:

Architecture documents: what are the pieces of the framework or library in question, and how do they fit together? This is often the hardest piece to maintain, simply because it changes organically over time, and unlike the next couple examples it doesn’t have an inherent attachment to the code. However, it’s also the piece that’s absolutely the most important, because it’s what gives anyone trying to dive in and contribute the orientation they need to be effective.
“Why” comments: The internals of the core libraries very often have good reasons for doing things even in apparently odd ways. However, the reasons for those are very rarely written down anywhere. This is precisely what comments are for! If some implementation actually can’t be simplified in the way it looks like it can, write it down right there in a comment! This will save both you and other developers lots of wasted time with false starts and useless pull requests and so on.
Documentation of private API: Much of the public-facing API for Ember is fairly clear (modulo caveats around completeness and accuracy). However, most internal API is essentially entirely undocumented. This makes it extremely difficult for someone to know how to use the internal APIs when working on internal code!

All of these things came home to me pretty sharply as I started poking at the Glimmer VM project to see where and how I can pull together my knowledge of both TypeScript and Rust to drive some of those efforts forward. The core team folks I’ve interacted with have all been extremely helpful—and that’s always been true all along the way!—but they’re also busy, and taking the time to write down something once ends up being a major “force multiplier”. You can explain the same thing to multiple different people via multiple different conversations, or you can write it down once and make it a resource that anyone can use to start working effectively in the system!

How you can help:

If you’re a current Ember developer in any part of the ecosystem: start writing down what you know. If a question comes up more than once, put it in a document somewhere. If nothing else, then you can link to it instead of typing it up one more time in Slack!
If you’re just getting started on developing core Ember functionality: write down what you learn. If you’re working through some section of the codebase, don’t understand it, and then come to understand it by way of asking questions, add documentation for that! You’ll help the next person coming along behind you!

In short: please write more things down! We need user-facing and developer-facing documentation; they need to be different and distinct from each other; and we need the whole range in both. That’s an enormous amount of work, and it’s very different from programming (and therefore harder for many of us).² But it’s also work that will pay equally enormous dividends in enabling the Ember community to grow in both the number and the effectiveness of its contributors—and that’s something we very much need!

Most of Webpack’s bad reputation is long-since undeserved: it was poorly documented… a few years ago. So was Ember!↩
I’ll let you draw your own conclusions about my own relationship to writing given the absurd number of words I put out on this site.↩

True Myth 1.3.0 and 2.0.0

2018-05-18T19:15:00-04:00

Today I released two versions of True Myth: 1.3.0 and 2.0.0. You can read the 1.0 announcement from last November for an overview of the library and a discussion of why you might want to use the library in the first place!

Since its initial release last November, True Myth has gone through a number of small feature and bug fix releases, each of which is more interesting in its own right than 2.0 is—because there are almost no new “features” here, and the changes to the functionality which are in 2.0 are purely additive and could readily have gone in 1.3 instead.

In fact, the act of writing that sentence made me realize that there really should be a 1.3 which people can trivially upgrade to and then take on the changes in 2.0 later.

– 1.3.0 –

There are a few very small changes in 1.3 that are just nice ergonomic wins. (You may also be interested in looking back at the list of other releases to see what else has landed since 1.0.)

Expose `value` and `error`

The value property in Maybe.Just and Result.Ok instances, and the error property in Result.Err instances, are now public, readonly properties instead of private properties. I made those private in the initial implementation because I thought it made more sense to expose them via methods, but experience showed that this is a relatively common pattern in practice:

function dealsWithAMaybe(couldBeAString: Maybe<string>) {
  if (couldBeAString.isJust()) {
    console.log(`It was! ${couldBeAString.unsafelyUnwrap()}`);
  }
}

This is a contrived example of course, but I and my colleagues found in practice that this is a scenario that comes up relatively often, especially when integrating with existing code rather than writing new code – control flow patterns there tend to assume early-return-on-null or similar instead.

So I made a change (leaning on TypeScript’s notion of “type narrowing”) so that you don’t have to use unsafelyUnwrap in this scenario anymore! You can use the method types, the standalone functions, or direct matching against the variants on the property

import Maybe from 'true-myth/maybe';

function dealsWithAMaybe(maybe: Maybe<string>) {
  if (maybe.isJust()) {
    console.log(`It was! ${maybe.value}`);
  }
}

In the Result case this is even nicer (notice that I’m using the variant, rather than a function, to discriminate between the two and narrow the types here):

import Result, { Variant } from 'true-myth/result';

function dealsWithAResult(result: Result<string, Error>) {
  if (result.variant === Variant.Ok) {
    console.log(`Huzzah: ${result.value}`);
  } else {
    console.log(`Alas: ${result.error.message}`);
  }
}

Basically: you now have more options for handling these scenarios, a nicer API, and—not that it should usually matter that much, but for whatever it’s worth—better performance by way of doing things with property lookups instead of function invocations in quite a few places.¹

Static helper methods

At my friend and collaborator Ben Makuh’s suggestion, I built a couple static helper methods to go with those. These helpers just give you nice abstractions to drop into functional pipelines. For example, you can lean on the type-narrowing capabilities described above while working through a list of Maybes to know that an item is a Just and use the new Just.unwrap static method in the pipeline:

import Maybe, { Just } from 'true-myth/maybe';

function justLengths(maybeStrings: Array<Maybe<string>>) {
  return maybeStrings
    .filter(Maybe.isJust)
.map(Just.unwrap)
    .map(s => s.length);
}

Analogous helpers exist for Result in the form of the Ok.unwrap and Err.unwrapErr methods. (Nothing has no analog for what I hope are obvious reasons!)

Tweaks to the `variant` properties

The variant property on both Maybe and Result has changed in two ways:

It is now readonly. This was an implicit invariant previously—you would break everything in the library if you changed the variant value—and I’ve just made it explicit in the type system.
It is now properly constrained with a literal type on the concrete instances. That is, the type of Just.variant is no longer Variant but specifically Variant.Just. (This is what enables you to use the variant for narrowing as demonstrated above. I should have done this in 1.0, and just forgot to!)

And that’s it for 1.3.0!

– 2.0.0 –

The 2.0 release is identical in features with the 1.3 release. However, it makes a breaking change to how consumers interact with the application, requiring updates to your tsconfig.json file and your bundler configuration, and removing support for Flow types.

Configuration file updates

Getting True Myth working nicely with consuming TypeScript packages has been a source of frustration for me and others. In short, requiring you to use the "paths" key in the "compilerOptions" section of the tsconfig.json made for an annoying amount of setup work, and it meant that using True Myth in a library required you to set it up in any consuming app. No good.

For type resolution to Just Work™, the types must be at the root of the distributed package.

As a result, I’ve stopped using libkit, which put the generated types in a reasonable-seeming but (in my experience) painful-to-use place, and have simplified the build layout substantially.

The types themselves are generated only when publishing an update to npm. They go in the root at that point, and they get cleaned up after publishing. (This is pretty much identical to the solution we came up in ember-cli-typescript.)
The other build files no longer get dropped in a nested src directory.
Since I was already at it, I renamed the two build directories from commonjs to cjs and from modules to es

So the distributed build now looks something like this:

/
  index.d.ts
  maybe.d.ts
  result.d.ts
  unit.d.ts
  utils.d.ts
  dist/
    cjs/
      index.js
      maybe.js
      result.js
      unit.js
      utils.js
    es/
      index.js
      maybe.js
      result.js
      unit.js
      utils.js

You’ll just need to completely remove the "paths" mapping for True Myth from your tsconfig.json and, if you’ve done anything unusual with it, update your bundler configuration to point to the new build location, i.e. dist/commonjs/src should now just be dist/cjs. Bundlers which respect the modules key in package.json will pick it up automatically, as will Ember CLI.

Removing Flow types

To my knowledge, no one is actually using the Flow types for the library. When I first started on it, my collaborator Ben Makuh was using Flow, but he ended up migrating to TypeScript in the intervening time, and there are no consumers I know of. I was always relatively unsure of their correctness, and I don’t have a good way to validate their correctness, and maintaining them involved doing manual work on every release to update the types by hand.

If you do use True Myth with Flow, and you’re missing the types, please let me know. I just can’t maintain them myself at this point!

And that’s it! We’ve been using True Myth in production at Olo for quite some time, and it’s proved to be a really valuable tool. Give it a spin and let me know how these latest versions work for you!

I’ve made some changes under the hood to take advantage of this as well, so the library should be faster. Probably trivially faster, but my philosophy around library code is very much be as fast as you can; it’s a way of considering the people using your code—not just the developers, but the end users.↩

Aesthetics and Programming Languages

2018-05-13T11:00:00-04:00

My distaste for the aesthetics of C^♯ are fairly well known to people I talk to about programming languages—perhaps equally as well known as my love of Rust. So much so that both are running jokes among some of my colleagues and friends. My hypersensitivity to aesthetics both in general and also specifically in programming languages and work environment is also so well-known as to be a gag.

But I was writing a bunch of Rust this weekend, and looking at it and thinking about it and wondering why it is that C^♯ drives me so up the wall aesthetically and experientially, while Rust doesn’t. On the surface, they don’t actually look all that different.

Here’s roughly equivalent code in each:

public class Person {
    public string Name { get; set; } = "Chris";

    public void greet() {
        Console.WriteLine($"Hello, {Name}");
    }
}

struct Person {
    name: String,
}

impl Person {
    pub fn new() -> Person {
        Person { name: String::from("Chris") }
    }

    pub fn greet(&self) {
        println!("Hello, {}", self.name);
    }
}

When you start tossing in generics and lifetimes, Rust can actually end up looking a lot messier than C^♯.

impl<'a, 'b, T, U> SomeTrait<'a, U> for SomeType<'b, U>
where
    T: SomeOtherTrait + YetAnotherTrait,
    U: OhWowSoManyTraits
{
    fn some_trait_method(&self) {
        // ...
    }
}

Nothing about that is what I would call aesthetically beautiful in a general sense! There’s a lot of syntax.

What I’ve concluded so far, though, is that my difference in feelings comes down to the way that syntax maps back to the underlying semantics, and my feelings about those underlying semantics. The basic language design approach C^♯ takes—i.e. everything is a class; mutation is both encouraged and implicit; don’t bother with value types—drives me batty. I don’t love the syntax, not least because it ends up being so verbose and noisy (you can express the same things in F^♯ much more briefly)—but also because I actively dislike the programming models it encourages (I don’t like the C^♯ programming model when I see in in F^♯ either!).

Rust, by contrast, matches the way I do and want to think about the world. Mutability is allowed but neither actively encouraged nor actively discouraged; more to the point it’s explicit. Insofar as “shared mutable state is the root of all evil,” Rust has two legs up on C^♯: it (a) doesn’t allow shared mutable state and (b) makes explicit where mutation is happening. It also separates data from behavior. It also has real value types. It also has sum types and pattern matching. In both cases, a lot of the syntactical noise is inessential, a holdover from the legacy of C; but in Rust’s case the way it maps onto a programming model that is more like OCaml than like C decreases the pain I feel from that noise.

This could be taken to validate the idea that syntax doesn’t matter, that the underlying semantics are everything, but that’s not the case. It’s not that I love Rust’s syntax. It’s that, although I dislike it at times, it doesn’t rise to the level of frustration I feel in C^♯ because it’s not coupled to a programming model that I loathe. The syntax matters; it’s just not the only thing that matters.

An interesting thing to consider: what Rust would look like in a world where it embraced its OCaml roots. (I don’t think Rust should have done this; spending its complexity budget on ideas instead of syntax was the right choice. But it’s still interesting.) The simplest level of translation might look something (very) roughly like this:

impl 'a 'b T U SomeTrait 'a T for SomeType 'b U
  where T : SomeOtherTrait + YetAnotherTrait

  some_trait_method :: &self -> void
  some_trait_method self =
    -- ...

This is obviously still a lot of syntax, but it’s all basically necessary given the things Rust is trying to express with lifetimes, ownership, etc.—and I did this off the top of my head with literally no consideration other than “what’s the most direct translation into roughly Haskell-ish syntax I can write?” It makes me genuinely curious where a language that aimed for Rust’s same kinds of guarantees but actively embracing the ML/Haskell family’s syntax might end up. I have a guess that I’d like it even better than I do Rust.

#EmberJS2018, Part 1

2018-05-11T09:30:00-04:00

There are three major themes I think should characterize the Ember.js community and project for the rest of 2018:

Finishing What We’ve Started (this post)
Doubling Down on Docs
Defaulting to Public for Discussions
Embracing the Ecosystem

Finishing What We’ve Started

What I want, more than any new feature anyone could come up with, is for this to be the year Ember.js commits to finishing what we have started. The last few years have seen the Ember team do a lot of really important exploratory work, including projects like Glimmer.js; and we have landed some of the initiatives we have started. But I think it’s fair to say that focus has not been our strong suit. It’s time for a year of shipping.

We need to land all the things we have in flight, and as much as possible avoid the temptation (much though I feel it myself!) to go haring off after interesting new ideas. As such, literally everything I list below is an effort already in progress. It’s just a matter of making concerted efforts as a community to land them.¹

And that way of putting it is important: we have to make concerted efforts as a community to land these things. Very, very few people are paid to work on Ember.js full time—far too few to accomplish all of this! If these things matter to you and your company, find a way to carve out time for it. Even if it’s just a few hours a week, even if it’s “just” (and there’s no “just” about these!) helping out with triage of open issues or answering questions in Slack or Discourse or Stack Overflow, even if it doesn’t feel like a lot… it adds up.

To be very clear, before I go any further: none of this is a knock on everything that the Ember core team and community have done in the last couple years. A lot of things that have landed along the way—dropping in the Glimmer rendering engine midway through the 2.x series, landing ES5 getters just weeks ago in Ember 3.1, and so on—are genuinely great! All that I mean is, a year where we land and polish everything would make everything that much more awesome (and make Ember that much more competitive a choice in the client-side framework world).

So: what do we need to ship this year?

Land Glimmer `<Component>`s in Ember.js proper

We’ve taken the first steps toward this already via a number of RFCs that were written late last year and merged since. We need to finish the implementation for these. That means getting the Glimmer Components in Ember quest across the finish line.

The whole story here will make Ember feel much more modern in a variety of ways, as well as enabling some great performance and programming model wins: Immutable component arguments! Auto-tracked class properties! <AngleBracketComponent> invocation! Clear semantic distinctions between arguments and local context! So many good things. We just need to land it! The quest needs to be moving forward, not stagnant.

How you can help:

Show up and volunteer to go after pieces of the quest. There are people willing to mentor you through the work that needs to be done!
Test it as it lands! You don’t have to commit to shipping things in your app to test them in your app.

Land a lot of Ember CLI efforts

There are a great many Ember CLI efforts in flight. Every last one of them should be on stable and in use before the end of the year.

Module Unification

The Module Unification RFC was opened in May 2016 and merged October 2016. There has been a lot of progress made, but we need to ship it—from where I stand, it’d be nice if it landed less than 2 years after we approved it! And we’re getting pretty close; you can actually use the Module Unification blueprint in an Ember application today. Some stuff doesn’t work quite right yet, but it’s getting close.

How you can help: try it out! Spin up new apps with the module unification blueprint flag, and try running the migrator codemod, and report back on what breaks.

Broccoli 1.0

We’re super close on this one—Oli Griffiths has done some heroic work on this since EmberConf—but we need to finish it. Ember CLI, for historical reasons, has been using a fork of Broccoli.js for quite some time. This divergence has caused all manner of trouble, including compatibility issues between Broccoli plugins and an inability to take advantage of the best things that have landed in Broccoli since the fork happened.

Perhaps the single most important example of that is that Broccoli 1.0 supports the use of the system tmp directory. That single change will improve the performance of Ember CLI dramatically, especially on Windows. It will also flat-out eliminate a number of bugs and odd behaviors that appear when trying to integrate Ember CLI with other file watching tools (e.g. TypeScript’s --watch invocation).

How you can help: once the Ember CLI team says it’s ready for testing, test your app and addons with it! Make sure that everything works as it should—specifically, that you’re not making any assumptions that depend on either the forked API or the location of the tmp directory used for intermediate build steps.

The new `Packager` setup, with tree-shaking and app-splitting

One of the current major pain points with Ember’s build pipeline is that it’s hard to extend, and not really documented at all. (I’ll have a lot more to say on the question of documentation in the next post!) However, work is in progress to change that, too!

The accepted-and-actively-being-worked-on Packaging Ember CLI RFC aims to fix both of these. Quoting from it:

The current application build process merges and concatenates input broccoli trees. This behaviour is not well documented and is a tribal knowledge. While the simplicity of this approach is nice, it doesn’t allow for extension. We can refactor our build process and provide more flexibility when desired.

A few of the things we can expect to be possible once that effort lands:

tree-shaking – we can lean on Rollup.js to get only the code we actually need, cutting shipped file size dramatically
app-splitting – lots of different strategies to explore, including route-based or “section”-based, etc.
static-build-asset-splitting – no reason to cache-bust your dependencies every time the app releases!
distinct app builds – you could ship one build of your app for browsers which support ES Modules and one for browsers which don’t (heeeeey, IE11) – letting you minimize the payload size for the ones that do

How you can help:

If you know Ember CLI internals: pop into #-dev-ember-cli and ask how you can help land the features
If you don’t know Ember CLI internals: also pop into #-dev-ember-cli, but ask instead how you can test the changes
Help document those internals (see the next post in this series)

Install-your-way-to-Ember

We need to finish splitting apart the Ember source from its current state of still being fairly monolith and get it turned into a true set of packages. The new Modules API which landed last year was a huge step toward this and made the experience on the developer side look like this should be possible—but that’s still a shim around the actual non-modularized Ember core code. The process of splitting it apart is happening, but we need to finish it.

The promise here is huge: Ember will be able to be the kind of thing you can progressively add to your existing applications and slowly convert them, rather than something that comes along all as a large bundle. It’s technically possible to do this today, but you cannot drop in just the view layer, for example, and that’s a huge value for people who want to try out the programming model or add it for just one feature in an existing application.

Making it possible for people to install Glimmer components, then the service layer, then the router, and so on as they need it will make adoption easier for people who are curious about the framework. But it will also be a huge boon to those of us already using Ember and wanting to migrate existing applications (often a tangled mix of server-side rendering and massive jQuery spaghetti files!) to Ember progressively. I’ve had multiple scenarios come up at my own job in just the last month where this would have been hugely useful.

How you can help: make it known that you’re willing to help work on breaking apart Ember into its constituent pieces, and as that effort lands (hopefully over the rest of this year!) test it in your own apps and addons, and find the pain points in the install-your-way-to-the-framework process.

Make TypeScript great everywhere

This one is near and dear to my heart… and it also really falls in no small part to me and the rest of the group working on ember-cli-typescript and type definitions for the Ember ecosystem!

There are two big wins we can land this year:

Built-in support in Ember.js itself.
Solid type definitions for the rest of the Ember.js ecosystem

If you don’t like TypeScript, don’t panic! The upshot here will actually be a better experience for all users of Ember.js.

1. Built-in support in Ember.js itself

One of my goals for this summer² is to finish an RFC making TypeScript a first-class citizen of the Ember.js ecosystem. To clarify what this will and won’t entail (assuming it’s accepted, assuming I ever manage to finish writing it!):

Ember will always be JS-first, and it will never require type metadata reflected to runtime, unlike e.g. Angular. No one will ever have a worse experience because they prefer JS to TS. The idea will be to make TypeScript an equally good experience, and to include it for consideration when thinking about design choices for new features.
Ember users, both JS and TS, will get the benefits of having good types available right out of the box: many editors and IDEs can use TypeScript type definitions to enable better docs, autocompletion, etc.—and we may even be able to leverage it for better validation of Handlebars templates!
We’ll have (because we’ll have to have!) a story on what we support in terms of backwards compatibility and SemVer for TypeScript and Ember and the type definitions. Necessarily, it has been the Wild West for the first year of concentrated effort here, trying to get our type definitions from “barely exist and not useful” to “full coverage and 99% right.” But as TypeScript becomes more widely used, we have to have a stability story, and we very soon will.

There’s also ongoing work to convert Ember’s own internals to TypeScript, and landing that will help guarantee that the type definitions for Ember are actually correct, which in turn will make the experience for everyone better. (Bad type definitions are worse than no type definitions!)

How you can help: engage in the RFC process once we get it started, and if you are up for it show up to help convert the Ember internals to TypeScript as well.

2. Solid type definitions for the rest of the Ember.js ecosystem

Closely related to making TypeScript a first-class citizen for Ember.js itself is getting the pieces in place for the rest of the ecosystem as well. That means we need type definitions for addons—a lot of them! The ember-cli-typescript team will (hopefully late this month or in early June) be launching a quest issue to get type definitions for the whole Ember ecosystem in place—by helping convert addons to TS if their authors desire it, or by adding type definitions to the addons if they’re up for it, or by getting them up on DefinitelyTyped if they’re totally disinterested. (And, as I’ll note again in that quest issue, it’s totally fine for people not to be interested: there is a maintenance burden there!) The goal, again, is that when you’re using any part of the Ember ecosystem it’ll be easy to get all the benefits of TypeScript—and indeed that in many cases you’ll get a fair number of those benefits as a JS user.

How you can help: participate in the quest issue once it’s live! We’ll help mentor you through the process of converting addons to TypeScript, writing type definitions and getting them well-validated, and so on!

That’s a lot to do. More than enough all by itself, and a lot of moving parts. As such, I’ll reiterate what I said at the start: we don’t need new features this year. It’s time for a year of shipping.

To put it in the terms the Rust community used for their similar push at the end of 2017, and which we have often used to describe the ongoing efforts in Rust to land the “Rust 2018 edition”: this is an “impl period”—a play on the Rust impl keyword, used to describe the implementation of the behavior associated with a given data type. You can think of this as the same: it’s the implementation of the good ideas we have.↩
Confession: it was a goal for the spring but I found myself utterly exhausted after EmberConf… and had a full month with another major talk given for internal purposes afterwards. I’m worn out.↩

A Humanist Frame

2018-05-01T07:00:00-04:00

A few thoughts this morning on technologies, community, and the need for a positive (as well as a negative) vision of technology, cities, and indeed liberty—taking the latest issue of Michael Sacasas’ newsletter The Convivial Society as a jumping off point.

Early in the newsletter, Sacasas offers this note on technological visionaries stretching back to the telegraph (at least):

It seems that none of these visionaries ever took into consideration the possibility that the moral frailties of human nature would only be amplified by their new technologies.

He shortly thereafter suggests why that vision proved so alluring—the too-readily amplified frailties of human nature notwithstanding:

The rise of communication technologies from the mid-19th century through today has roughly coincided with the dissolution and degradation of the traditional communities, broken and often cruel though they may have been, that provided individuals with a relatively integrated experience of place and self. In 1953, the sociologist Robert Nisbett could write of the “quest for community” as the “dominant social tendency of the twentieth century.” Framing a new technology as a source of community, in other words, trades on an unfulfilled desire for community.

What strikes me as most interesting here is that Sacasas notes, even if only as an aside, one of the most important things that most critics of our current techno/cultural milieu seem entirely content to skip over: that the traditional communities were “broken and often cruel.” One of the reasons that the social revolutions of the last 150 years have had such force is precisely this: that the traditional communities so casually valorized today (though not by Sacasas himself) may have helped people have “an integrated experience of place and self”—but that experience was, often as not, one of abuse: of ethnic minorities, of women, of anyone outside the gentry…

Sacasas’ description—“broken and often cruel”—is more right than is usually granted in these discussions. If we want to escape the shackles of atomistic individualism, we had best be thinking of something other than the glorious past, because the past was not glorious.

Third, and closely related to the above considerations: Sacasas closes the newsletter with a quote from Willa Cather’s O Pioneers!, adding his own emphasis. I’ll reproduce the quotation in full here as he gave it (so: emphasis his) and then comment below.

“You see,” he went on calmly, “measured by your standards here, I’m a failure. I couldn’t buy even one of your cornfields. I’ve enjoyed a great many things, but I’ve got nothing to show for it all.”

“But you show for it yourself, Carl. I’d rather have had your freedom than my land.”

Carl shook his head mournfully. “Freedom so often means that one isn’t needed anywhere. Here you are an individual, you have a background of your own, you would be missed. But off there in the cities there are thousands of rolling stones like me. We are all alike; we have no ties, we know nobody, we own nothing. When one of us dies, they scarcely know where to bury him. Our landlady and the delicatessen man are our mourners, and we leave nothing behind us but a frock-coat and a fiddle, or an easel, or a typewriter, or whatever tool we got our living by. All we have ever managed to do is to pay our rent, the exorbitant rent that one has to pay for a few square feet of space near the heart of things. We have no house, no place, no people of our own. We live in the streets, in the parks, in the theaters. We sit in restaurants and concert halls and look about at the hundreds of our own kind and shudder.”

Alexandra was silent. She sat looking at the silver spot the moon made on the surface of the pond down in the pasture. He knew that she understood what he meant. At last she said slowly, “And yet I would rather have Emil grow up like that than like his two brothers. We pay a high rent, too, though we pay differently. We grow hard and heavy here. We don’t move lightly and easily as you do, and our minds get stiff. If the world were no wider than my cornfields, if there were not something beside this, I wouldn’t feel that it was much worth while to work. No, I would rather have Emil like you than like them. I felt that as soon as you came.”

Sacasas—with many others who are rightly critical of the social situation we have made for ourselves here in late modernity, including many of my friends over at Mere Orthodoxy—calls out the ways that our unrestrained freedom has come at a great cost to us. These critics are right to do so. But the bit that caught my attention as equally worthy of notice in the section from Cather is Alexandra’s response (emphasis mine):

We pay a high rent, too, though we pay differently. We grow hard and heavy here. We don’t move lightly and easily as you do, and our minds get stiff. If the world were no wider than my cornfields, if there were not something beside this, I wouldn’t feel that it was much worth while to work.

There is something in this exchange of the tension between agrarianism and urbanism that seems to come up every time Wendell Berry is mentioned. It is true that cities (and late modernity!) often offer a kind of freedom that itself is slavery. But it is also true that the slavery of freedom is not the only kind of slavery.

It remains one of my chief concerns that few who are taking seriously the problems we have made for ourselves in modernity seem interested in finding solutions that work in cities. It is one thing to have a healthy suspicion of the kind of city-centrism and techno-centrism and indeed techno-fundamentalism that is largely the order of the day. It is something else entirely, however, to fail to imagine either city or technological milieu as possibly good. (To be clear, this does not seem to be the tack that Sacasas is taking; and it is not so much that someone like my friend Jake Meador is hostile to cities as that his own sympathies run more to rural life.) This is, I think, part of what Stephen Carradini has been getting at in his own more optimistic view in especially our most recent Winning Slowly episode.

We must reject techno-utopianism; we must repent of our worship of technique. But we must not make the usual conservative mistake and stop with the mere rejection of something bad. That tends not to go so well. Instead, we need to consciously develop a frame that situates technology as properly subordinate to the humane, and which sets cities and farms and small towns and moon colonies not in opposition to each other but as complements.

Exploring 4 Languages: Integrity and Consistency

2018-03-24T22:00:00-04:00

In chapter 6, Wlaschin turns to one of the most important aspects of “domain modeling”: keeping it consistent. It’s all well and good to set up a domain model, but if you don’t have a way to make sure that model is reliable everywhere you use it, well… you’ve done a lot of extra work and you’re not going to see a lot of results for all that effort! But as Wlaschin points out, we can actually use the type systems, and the types we wrote up in the previous chapter, to help us enforce the business rules for our domain (as well as the business shapes in the domain).

An important note: you can see the latest version of this code (along with history indicating some of my travails in getting there!) in this public repository on GitHub.

A simple example: `WidgetCode`

We’ll start with one of the simpler examples: validating that a WidgetCode is legitimate. A WidgetCode, in this domain, is valid if, and only if, it has a W followed by four digits.

The basic tack we’ll take, in all four languages, is to leverage the way the types work to make it so we have to use a function to create a valid instance of a WidgetCode. That’s a bit of extra work (though especially in the functional-first languages, it ends up not being a lot of extra work) but it lets us use Result types to handle invalid data up front.

The downside is that we can’t just get directly at the value inside our wrapper types using basic pattern matching. Instead, we need to be provide a function for “unwrapping” it. Tradeoffs!

We’ll go at this using the most appropriate tool from each language, but in every case we’ll end up with a create function that takes a string and returns a Result with the successful option being a WidgetCode and the error option being a string describing the error; and a value function to unwrap a valid code. Throughout, I also assume an essentially-identical implementation of a related GizmoCode type; I pull both in to show how they end up being used side by side.

Rust

We are using a tuple struct to wrap the string value here. Since there is no pub modifier in the wrapped String, it’s opaque from the perspective of the caller—and this is exactly what we want. We’ll pull in the Regex crate and validate the code passed to us on creation.

use regex::Regex;

pub struct WidgetCode(String);

impl WidgetCode {
    pub fn create(code: &str) -> Result<WidgetCode, String> {
        let re = Regex::new(r"W\d{4}").expect(r"W\d{4} is a valid regex");
        if re.is_match(code) {
            Ok(WidgetCode(String::from(code)))
        } else {
            Err(String::from(
                "`WidgetCode` must begin with a 'W' and be followed by 4 digits",
            ))
        }
    }

    pub fn value(&self) -> &str {
        &self.0
    }
}

This is fairly idiomatic Rust: we’re borrowing a reference to the code as a “string slice”, and creating a new, wrapped String instance to wrap up the code or return a new String as an error. When we get the value out, we return a reference to the string,¹ with &self.0: & to indicate a reference, .0 to indicate the first item of a tuple. Note as well that the final if block here is an expression. There’s no semicolon terminating it, and this whole if block ends up being the resulting value of the function.

One other point of interest here is that the creation of the regex itself is checked by the compiler for us! If we pass an invalid regular expression, this simply won’t compile.

This could also live in its own module, ordering/widget_code.rs, and in fact that’s how I would normally do this (and have in the repository where I’m working): every one of these small types would get its own module file within the containing Ordering module. It’s not necessary, but as the domain model grows, it becomes increasingly convenient in that you always know where to find things.²

Then we can import it and use it like this in ordering/mod.rs:

mod widget_code;
mod gizmo_code;

use widget_code::WidgetCode;
use gizmo_code::GizmoCode;

pub enum ProductCode {
    Widget(WidgetCode),
    Gizmo(GizmoCode),
}

fn demo_it() {
    let valid = WidgetCode::create("W1234");
    let invalid = WidgetCode::create("wat");

    let unwrapped = match valid {
        Ok(ref code) => code.value(),
        Err(_) => "",
    };
}

Notice that in Rust, the mod.rs file declares all child modules. If you had a widget_code.rs on the file system but no mod widget_code;, Rust would just ignore the declaration entirely. Then Rust also requires us to use widget_code; to access its contents. The distinction between declaring and using a given module makes some sense: by the time all is said and done with this exercise, we won’t be doing much of anything in this Ordering module; it’ll exist primarily as a grouping construct for all the other modules.

In this case, we go ahead and import the WidgetCode type from the module. We only have the one type there, with no standalone functions: everything is attached to the type via the impl block; so we can just call everything directly off of the type. This ends up feeling kind of like the way we’d do things in a traditional OOP language, but also really not, because we still have a separation between the data type and the implementation of functionality attached to it. It’s not obvious here, but we could write impl WidgetCode in some other module in the crate, and as long as there’s no conflict between the implementations, it’s fine! And then we could call whatever function we defined in that block “on” WidgetCode. This is on the one hand totally unlike what we’ll see in the other languages, and on the other hand weirdly analogous to them.

I’m going to pass over why we need ref code here, as it gets into details of Rust’s model of ownership and reference borrowing and it’s going to be unneeded because of improvements to Rust’s compiler fairly soon. The one thing to note here is that we get nice memory/allocation behavior, i.e. we’re not doing a bunch of separate heap string allocations here. This is one of the big upsides to Rust in general! It’s not quite as pretty as what we’ll see below, but the performance wins are awesome.

Elm

Elm introduces us to a pattern we’ll see in each of the more traditional “functional” languages: the use of modules for this kind of structure. First the code, then some comments on it:

-- src/ordering/WidgetCode.elm
module Ordering.WidgetCode exposing (WidgetCode, create, value)

import Regex exposing (contains, regex)


type WidgetCode
    = WidgetCode String


create : String -> Result String WidgetCode
create code =
    if contains (regex "W\\d{4}") code then
        Ok (WidgetCode code)
    else
        Err "`WidgetCode` must begin with a 'W' and be followed by 4 digits"


value : WidgetCode -> String
value (WidgetCode code) =
    code

Elm’s module system lets you choose exactly what to expose. In this case, we’re only exporting the type itself along the create and value functions—but, importantly, not the normal type constructors for the type.

You can import the things exposed both as a module and as individual items. Assume we implemented GizmoCode the same way. We’d import and use them in Ordering.elm like this:

-- Ordering.elm
import Ordering.WidgetCode as WidgetCode exposing (WidgetCode)
import Ordering.GizmoCode as GizmoCode exposing (GizmoCode)

type ProductCode
    = Widget WidgetCode
    | Gizmo GizmoCode
    
valid =
    WidgetCode.create "W1234"


invalid =
    WidgetCode.create "wat"


unwrapped =
    case valid of
        Result.Ok code ->
            WidgetCode.value (code)

        Result.Err _ ->
            ""

As with Rust, we can’t construct the type without using the provided function. As I’ve written the imports, you’d create a WidgetCode by writing WidgetCode.create "W1234". You could also import it directly, but that would have its own problems once you had the create function imported for GizmoCode as well.

Finally, notice the way we aliased the module name here with as on the import: we don’t have to write out the fully qualified path this way. And there’s no conflict between the aliased module name and the type name – they live in their own namespaces (as it should be!). Importing the type name distinctly is handy because it means we don’t have to write the body of the union type out as Widget WidgetCode.WidgetCode.

F^♯

The F^♯ code looks a lot like the Elm code. The main differences here have to do with their module systems.

namespace Ordering

type WidgetCode = private WidgetCode of string
module WidgetCode =
    let create code =
        if Regex.IsMatch(code, @"W\d{4}") then
            Ok (WidgetCode code)
        else
            Error "`WidgetCode` must begin with a 'W' and be followed by 4 digits"

    let value (WidgetCode code) = code

Here we declare that we’re in the namespace Ordering. Everything here will be publicly visible to everything else in the namespace Ordering. We could also make this a module, and in that case we’d need to explicitly open it in other modules. Because it’s part of the base namespace we’re using for Ordering, though, we get it for “free”. There’s a downside to this, though. More on that below.

Also notice that this means that we have yet one more “namespace” for names to live in: namespace names are different from module are different from type names! So here we declare a top-level module Ordering here so that we can actually write code that does something in the file – namespaces can only contain type definitions (including module definitions).

namespace Ordering

type ProductCode =
    | Widget of WidgetCode
    | Gizmo of GizmoCode

module DemoIt =
    let valid = WidgetCode.create "W1234"
    let invalid = WidgetCode.create "wat"
    
    let unwrapped =
        match valid with
        | Ok(code) -> WidgetCode.value code
        | Error(_) -> ""

The things to notice here as particularly different from the others:

We don’t have to explicitly import the module names, because we used the same namespace (Ordering) to group them. We could also have done namespace Ordering.WidgetCode and open Ordering.WidgetCode; that might actually make more or less sense in the context. I think this is probably more idiomatic, however, which is why I picked it.
Since we’re keeping the rest of the containing module in the same namespace, we do have to declare module DemoIt for functionality – not just types – to live in. This is true for both Ordering.fs and WidgetCode.fs and so on.

This way of structuring things works really well, but it has one major downside compared to Elm and Rust: where any given name comes from is not obvious from any given text file. Using modules instead of namespaces and using more fully qualified names could help here, but the reality is simply that F^♯ (like C^♯) basically leaves you out to dry here. My take is that this is basically what happens when you design a language assuming IDE-like tooling. But especially when looking at e.g. GitHub diff views, or just browsing source code in general, I strongly prefer the way Elm and Rust generally lead you to do explicit imports or fully qualified paths. (Both have an escape hatch: Rust’s use path::to::module::*; and Elm’s import Path.To.Module exposing (..), but both are actively discouraged as bad practice in most situations.)

Reason

Interestingly, Reason looks most like Rust but behaves most like F^♯. The biggest difference is that I need a separate interface file for Reason to get the privacy benefits that I’m getting in all the other languages.

We put the definition file at ordering/Ordering_WidgetCode.rei. (I’ll comment on the long name in a moment.)

type gizmoCode = pri | GizmoCode(string);

let create: string => Js.Result.t(widgetCode, string);

let value: widgetCode => string;

With that module definition in place, we can separately supply the implementation, in ordering/Ordering_WidgetCode.re.

type widgetCode =
  | WidgetCode(string);

let create = code => {
  let isMatch =
    Js.Re.fromString("W\\d{4}") |> Js.Re.exec(code) |> Js.Option.isSome;
  if (isMatch) {
    Js.Result.Ok(WidgetCode(code));
  } else {
    Js.Result.Error(
      "`WidgetCode` must begin with a 'W' and be followed by 4 digits"
    );
  };
};

let value = (WidgetCode(code)) => code;

Note that you could do the same thing with an interface file for F^♯. We’re also doing something that’s similar in principle to the use of private types in in F^♯, but unlike in F^♯ we have to use the module interface to make it work as far as I can tell. The interface can declare the type private, but in the actual implementation, the type has to be non-private to be constructable. (If I’m wrong, please send me a note to let me know! But that’s what I gathered from reading OCaml docs, as well as from command line error messages as I played around.) Also, the fact that Reason has landed on the keyword pri instead of OCaml and F^♯’s much saner private is super weird.

The interface file just defines the types, and has the .rei extension. type widgetCode here is an abstract type, which provides no information about what it contains. Note the function types are provided as well. Here I’m using specifically the Js.Result type; there is also a Result type in at least one of the OCaml standard libraries. This is one of the more complicated things about Reason compared to the others: there are… several standard libraries to choose from, which will or won’t work differently depending on what compile target you’re picking.

In any case, once we have both the module and the implementation defined, we can use it like this in ordering.re:

module WidgetCode = Ordering_WidgetCode;

module GizmoCode = Ordering_GizmoCode;

open WidgetCode;

open GizmoCode;

type productCode =
  | Widget(widgetCode)
  | Gizmo(gizmoCode);

let valid = WidgetCode.create("W1234");

let invalid = WidgetCode.create("wat");

let unwrapped =
  switch valid {
  | Js.Result.Ok(code) => WidgetCode.value(code)
  | Js.Result.Error(_) => ""
  };

We do this mapping from Ordering_WidgetCode to WidgetCode here because OCaml and therefore Reason has only a single global namespace for its module names as defined by the file system. You can nest modules, but only within files. The workaround is, well… Ordering_ and remapping the name as we have here. This lets you access the nested modules as Ordering.WidgetCode and so on elsewhere.

Then we open WidgetCode etc. so that we can write widgetCode instead of WidgetCode.widgetCode in the productCode definition. This is basically the same effect we get from just being in the same namespace in F^♯ (which, again, we could rewrite exactly this way), or from the kinds of imports we discussed above for Rust and Elm.

Numeric validation: `UnitQuantity`

So far, the showing tilts heavily in F^♯’s and Elm’s favor in terms of expressiveness and elegance. However, there’s a lot of variation depending on exactly what you’re doing. If, for example, you want to validate a range, well… then Rust actually has a pretty good approach! Once again, you’ll note that these all have a lot in common; the difference mostly comes down to the degree of syntactical noise required to express the same basic thing.

In this section, I’m not really going to spend a lot of time discussing the details and differences; I’m just leaving it here to show an interesting example where the languages’ design decisions end up have slightly different ergonomic tradeoffs.

Rust

// ordering/unit_quantity.rs
pub struct UnitQuantity(u32);

impl UnitQuantity {
    pub fn create(qty: u32) -> Result<UnitQuantity, String> {
        match qty {
            0 => Err(String::from("`UnitQuantity` cannot be less than 1")),
            1...1000 => Ok(UnitQuantity(qty)),
            _ => Err(String::from("`UnitQuantity` cannot be greater than 1000")),
        }
    }

    pub fn value(&self) -> u32 {
        self.0
    }

    pub fn minimum() -> UnitQuantity {
        UnitQuantity(1)
    }
}

Elm

-- ordering/UnitQuantity.elm
module Ordering.UnitQuantity exposing (UnitQuantity, create, value)


type UnitQuantity
    = UnitQuantity Int


create : Int -> Result String UnitQuantity
create qty =
    if qty < 1 then
        Err "`UnitQuantity` cannot be less than 1"
    else if qty > 1000 then
        Err "`UnitQuantity` cannot be greater than 1000"
    else
        Ok (UnitQuantity qty)


value : UnitQuantity -> Int
value (UnitQuantity qty) =
    qty

    
minimum : UnitQuantity
minimum = UnitQuantity 1

F^♯

// ordering/UnitQuantity.fs
namespace Ordering

type UnitQuantity = private UnitQuantity of uint32
module UnitQuantity =
    let create qty =
        if qty < 1u then
            Error "`UnitQuantity` cannot be less than 1"
        else if qty > 1000u then
            Error "`UnitQuantity` cannot be greater than 1000"
        else
            Ok (UnitQuantity qty)

    let value (UnitQuantity qty) = qty
    
    let minimum = UnitQuantity 1

Reason

/* ordering/Ordering_UnitQuantity.rei */
type unitQuantity = pri | UnitQuantity(int);

let create: int => Js.Result.t(unitQuantity, string);

let value: unitQuantity => int;

let minimum: unitQuantity;

/* ordering/Ordering_UnitQuantity.re */

type unitQuantity =
  | UnitQuantity(int);

let create = qty =>
  if (qty < 1) {
    Js.Result.Error("`UnitQuantity` cannot be less than 1");
  } else if (qty > 1000) {
    Js.Result.Error("`UnitQuantity` cannot be greater than 1000");
  } else {
    Js.Result.Ok(UnitQuantity(qty));
  };

let value = (UnitQuantity(qty)) => qty;

let minimum = UnitQuantity(1);

Aside: On Documentation

One thing that became extremely clear in the course of working all of this out is that the documentation stories for these languages are in vastly, vastly different places.

Figuring out how to write this private create/value approach was very straightforward in Rust, because it’s literally just right there in how impl blocks and the pub keyword work: things default to private, including the contents of a struct, and you always define the related functionality with pub fn declarations in the related impl block.

Elm and F^♯ were both slightly harder, in that I had to poke around a bit to figure out the right way to do it. But not that much harder. Both use module-level isolation to accomplish this; the main difference there was that F^♯ just lets you do it inline and Elm explicitly ties modules to files.

Reason… was very, very difficult to get sorted out. This is just a function of the state of the ecosystem. Reason is distinct syntax for OCaml, but it also leans on BuckleScript. That means that if you want to figure out how to do anything, you probably need to search in the docs for all of those, and if your answer turns out to come from OCaml then you have to figure out how to translate it back into Reason and BuckleScript! Ultimately, I was able to figure it out and get the project layout to how you see it in the repository, but… it took a lot more digging than with any of the other projects!

Summary

As with our previous foray, we can see a ton of similarities across these languages. All lean heavily on pattern-matching for dealing with different scenarios; all let us make use of a Result type for handling success or failure; all make heavy use of expression-bodied-ness; and all supply some way to make types constructable only in safe/controlled ways.

For Rust, that’s a matter of leaving the internals of a struct private and making pub fn helpers to do the construction and value retrieval. For Elm, F^♯, and Reason, that’s a matter of having the normal type constructors be private while exposing the types themselves normally. They do that in different ways (F^♯’s private type, Elm’s exposing, and Reason’s pri annotation on the type variant in a module interface file), but the effect is essentially identical, and functionally equivalent to what we see in Rust.

The main differences we see across Elm, F^♯, and Reason have to do with the nature of the various module systems. In a lot of ways, Reason’s is the least capable for this specific purpose, because it’s directly tied to OCaml’s module system, which substantially predates any of the others. (I say “in a lot of ways” because OCaml’s modules are surprisingly capable; they end up being their own kind of types and you can do some crazy things with them, all of which I’d like to actually come to understand… eventually.) Rust’s module system, meanwhile, has a lot of similarities to Elm’s in particular, but because we actually carry functions along with the types they impl (though they get defined separately, with all the power that entails), we have a bit less boilerplate we need to write just to get at the specific functions in play.

Next time (probably only a couple of weeks away because we’re working through the book at work in a book club!), I’ll be looking at Chapter 7: Modeling Workflows as Pipelines. I suspect this will be a place where the true functional orientation of Elm, F^♯, and Reason will much more sharply differentiate them from the sometimes-functionalish-but-not-actually-functional way we write things in Rust.

This reference will live and be valid as long as the underlying WidgetCode is. We could also return a String if we wanted that value to live independently of the WidgetCode instance backing it.↩
Putting it in its own module, whether in a separate does have implications for privacy, though we don’t much care about them in this case. Rust lets us set the privacy on a whole spectrum, from “visible everywhere” to “only visible in this specific module.”↩

Announcing ember-cli-typescript 1.1.0

2018-02-12T07:00:00-05:00

I’m delighted to announce the release of ember-cli-typescript 1.1.0. This first minor release since 1.0 includes the following shiny and awesome new features:

Generators
Support for developing addons in TypeScript
Incremental compilation (a.k.a. fast rebuilds in ember serve mode)

Generators

We’ve now added support for generating all standard Ember items as TypeScript files instead of JavaScript files. So now when you run ember generate component user-profile for example, you’ll get user-profile.ts, user-profile-test.ts, and user-profile.hbs. For most files, this is just a nicety—just two files you don’t have to rename!—but in the case of services, controllers, and Ember Data models, adapters, and serializers it will actually make a really big difference in your experience of using TypeScript in your app or addon.¹

Those generators are mostly identical with ones in Ember and Ember Data, just with .ts instead of .js for the extension. The only changes we have made are: (a) we’ve tweaked them to use classes where possible, and (b) we have customized the controller, service, and Ember Data model, adapter, and serializer generators so you get the most mileage out of TypeScript for the least effort we can manage today. So when you do ember generate service session, this is what you’ll see:

import Service from "@ember/service";

export default class Session extends Service.extend({
  // anything which *must* be merged on the prototype
}) {
  // normal class definition
}

// DO NOT DELETE: this is how TypeScript knows how to look up your services.
declare module "ember" {
  interface ServiceRegistry {
    session: Session;
  }
}

Courtesy of these generators, you can now write almost exactly what you’d write in vanilla Ember and get full support for autocompletion of properties and methods on the Session service, as well as type-checking for how you use those. Service and controller injections just require you to explicitly name the service or controller being injected:

import Component from "@ember/component";
import { inject as service } from "@ember/service";

export default class UserProfile extends Component {
  session = service("session");
  // note the string ^ naming the service explicitly
}

So, for example, if your session service had a login method on it:

import Service from "@ember/service";
import RSVP from "rsvp";

export default class Session extends Service {
  login(email: string, password: string): RSVP.Promise<string> {
    // some API call to log in
  }
}

// DO NOT DELETE: this is how TypeScript knows how to look up your services.
declare module "ember" {
  interface ServiceRegistry {
    session: Session;
  }
}

Then anywhere you injected and used it, you’ll get auto-complete suggestions and type checking:

autocompletion

type-checking

(You’ll see the same kinds of things in other editors, from Vim to IntelliJ IDEA. Visual Studio Code is just my current editor of choice.)

Addon development

As promised with the 1.0 release, 1.1 (though arriving much later than I hoped it would) includes support for developing addons with TypeScript.

Strictly speaking, of course, you could always develop addons using TypeScript, but there were two problems with it: (1) dependency management and (2) manual work required to deal with the dependency management problems.

1. Dependency management

In the normal Ember CLI workflow, TypeScript had to be a dependency—not a devDependency—of the addon, because the normal pattern with Ember CLI is to ship the uncompiled files and have the consumer compile them all together at build time.

This makes a certain amount of sense for Babel given the Ember community’s shared reliance on Babel: it’s just assumed to be part of every app build. In that case, it gives consumers control over their compilation target. If an app only needs to target evergreen browsers, it can do that and ship a smaller payload, because an addon won’t have pre-compiled in things like generator support, etc.

In the case of TypeScript, however, this makes a lot less sense: many (probably most) consumers of addons written in TypeScript will still be normal JavaScript consumers. We did not want to burden normal consumers with a TypeScript compile step. We also didn’t want to burden any consumers with the reality that TypeScript is a large install. TypeScript 2.6.2 is 32MB on disk for me. Even with some degree of deduplication by npm or yarn, if addons used a variety of versions of TypeScript for development—as they surely would!—the install cost for consumers would quickly spiral into a nasty spot. And again: that’s bad enough for someone who wants to use TypeScript in their app; it’s far worse for someone who just wants to consume the compiled JavaScript.

2. Manual workarounds

You could work around all of that by building the JavaScript (and TypeScript definitions) yourself. But as part of that, you had to do all the work of making sure both the JavaScript files and the type definitions you generated ended up in the right place for distribution and consumption. That was always possible, but it was also always going to be a lot of work. In practice, as far as I know, no one has done this.

Solution

We now support TypeScript as a devDependency and also manage the work of generating JavaScript and type definitions for you. All you have to do is install ember-cli-typescript into an addon, and then when you do your build step, we’ll automatically do the work (on prepublish) of generating TypeScript .d.ts files and JavaScript source for you.

Consumers of your addon, therefore, will (a) not know or care that the addon is written in TypeScript if they just want to consume it as normal JavaScript² or (b) will get the benefits of your having written the library in TypeScript without paying the penalty of having to have multiple versions of the TypeScript compiler downloaded to their own app.

One important caveat: we do not support TypeScript in an addon’s app directory. However, for most addons, we don’t think this should be a problem. It’s rare for addons to put actual implementation in the app directory; instead it has simply become conventional for the app directory simply to have re-exports for convenient access to the functionality supplied by the addon.

Also note that you can supply type definitions for your addon without developing the addon itself in TypeScript.³ You do not need ember-cli-typescript installed for that. You only need the addon if you actually want to take advantage of the opportunities TypeScript affords for developing your own addon.

Incremental compilation

Last but not least, we’ve managed—mostly through the hard work of both Dan Freeman (@dfreeman) and Derek Wickern (@dwickern—to get support for TypeScript’s --watch mode integrated.⁴ What this means in practice is: way faster iteration as you work.

Previously, every time you triggered any change in your app (even if it didn’t involve any TypeScript files at all), the TypeScript compiler would recompile all the TypeScript files in your application. We didn’t initially have a good way to make TypeScript and Broccoli (and therefore Ember CLI) communicate clearly about what had changed. Now, courtesy of Dan and Derek’s hard work (and my cheerleading, testing, and fixing a few corner pieces along the way), we do! So when you change a .hbs file or a .js file… the TypeScript compiler won’t do anything. And when you change a TypeScript file, the TypeScript compiler will only recompile that file.

On my own app (~35,000 lines of TypeScript across ~700 files), that’s the difference between rebuilds involving TypeScript taking 15–20 seconds and their taking 1–2 seconds. Literally an order of magnitude faster! Over the course of a day of development, that saves a huge amount of time.

The way we did it also solved an incredibly annoying problem we had in the previous pass: any change to your app was triggering tsc to rebuild the entire TypeScript tree of your app, even if you didn’t so much as look at .ts file. This was particularly annoying when combined with the long rebuild times: change a CSS file and wait for your TypeScript files to rebuild? Ugh. But not anymore!

Credit and Thanks

Massive credit goes to Dan Freeman (@dfreeman) and Derek Wickern (@dwickern), who did most of the heavy lifting on the internals for this release, and together unlocked both incremental compilation and addon support. Derek also did the lion’s share of the work on writing the types for Ember and Ember Data.

Thanks to Maarten Veenstra (@maerten) for the original inspiration (and a spike last summer) for using a type registry, and to Mike North (@mike-north) for some discussion and planning around the idea late in 2017. I may have implemented them, but the ideas came from the community!

Thanks to Frank Tan (@tansongyang) for doing a lot of the work on porting the generators from the Ember and Ember Data repositories to ember-cli-typescript, as well as converting them to TypeScript and to use the new formats. He also contributed the type definitions for the new (RFC #232) QUnit testing API.

Thanks to everyone who contributed to ember-cli-typescript or the Ember typings in any way since we released 1.0.0:

ember-cli-typescript contributors (note that I intentionally include here everyone who opened issues on the repository: that is not a small thing and has helped us immensely):
- Bryan Crotaz (@BryanCrotaz)
- Daniel Gratzl (@danielgratzl)
- Guangda Zhang (@inkless)
- @guangda-prosperworks
- Krati Ahuja (@kratiahuja)
- Martin Feckie (@mfeckie)
- Nikos Katsikanis (@QuantumInformation)
- Per Lundberg (@perlun)
- Prabhakar Poudel (@Prabhakar-Poudel)
- Ryan LaBouve (@ryanlabouve)
- Simon Ihmig (@simonihmig)
- Theron Cross (@theroncross)
- Thomas Gossman (@gossi)
- Vince Cipriani (@vcipriani)
Ember typings contributors:
- Adnan Chowdhury (@bttf)
- Derek Wickern (@dwickern)
- Frank Tan (@tansongyang)
- Guangda Zhang (@inkless)
- Ignacio Bona Piedrabuena (@igbopie)
- Leonard Thieu @leonard-thieu
- Logan Tegman @ltegman
- Martin Feckie (@mfeckie)
- Mike North (@mike-north)
- Nathan Jacobson (@natecj)
- Per Lundberg (@perlun)
- Robin Ward (@eviltrout)

Thanks to Rob Jackson (@rwjblue) and Tobias Bieniek (@Turbo87 on GitHub, @tbieniek in the Ember Slack) for answering tons of questions and putting up with regular pestering about Ember CLI.

And last but not least, thanks to everyone who’s popped into #topic-typescript on the Ember Community Slack with questions, comments, problem reports, and the occasional word of encouragement. It really does help.

For details on how this all works, see TypeScript and Ember.js Update: Part 4, where I discuss these changes in detail.↩
although they may actually get some benefits in a number of modern editors, since e.g. VS Code and the JetBrains IDEs will leverage types if they exist!↩
More on that in a post to be released in the next couple weeks—one I promised long ago, but which we’re now in a place to actually do: a plan and a roadmap for typing the whole Ember ecosystem!↩
And of course, right as we finally landed our support for it, by hacking around the --watch invocation in a lot of really weird ways, Microsoft shipped API-level support for it. We hope to switch to using that under the hood, but that shouldn’t make any difference at all to you as a consumer of the addon, except that if/when we land it at some point, you’ll just have a nicer experience.↩

TypeScript and Ember.js Update, Part 4

2018-02-08T07:30:00-05:00

You write Ember.js apps. You think TypeScript would be helpful in building a more robust app as it increases in size or has more people working on it. But you have questions about how to make it work.

This is the series for you! I’ll talk through everything: from the very basics of how to set up your Ember.js app to use TypeScript to how you can get the most out of TypeScript today—and I’ll be pretty clear about the current tradeoffs and limitations, too.

(See the rest of the series. →)

In the previous posts in this series, I introduced the big picture of how the story around TypeScript and Ember.js has improved over the last several months, walked through some important background on class properties, and dug deep on computed properties, actions, and mixins.

In today’s post, we’ll look at how to write Ember Data models so they work correctly throughout your codebase, and see some improvements to how we can do Service and Controller injections even from a few weeks ago.

Here’s the outline of this update sequence:

Overview, normal Ember objects, component arguments, and injections.
Class properties—some notes on how things differ from the Ember.Object world.
Computed properties, actions, mixins, and class methods.
Using Ember Data, and service and controller injections improvements. (this post)
Mixins and proxies; or: the really hard-to-type-check bits.

Ember Data

There remains one significant challenges to using Ember Data effectively with TypeScript today: Ember Data, for reasons I haven’t yet dug into myself, does not play nicely with ES6 classes. However, we need named class exports for the sake of being able to use them as types elsewhere in our programs. The hack to work around this is much the same as anywhere else we need named exports but have to get things back into the prototype:

import DS from "ember-data";

export default class Person extends DS.Model.extend({
  firstName: DS.attr("string"),
  lastName: DS.attr("string")
}) {}

You can still define other items of the class normally, but attributes have to be prototypally bound or you will have problems. Note that this only applies (as far as I can tell) to Ember Data Models specifically—Adapter and Serializer classes work just fine.

The other problem we’ve historically had was dealing with lookups—the situation was similar to that I described in Part 3 for service injection. However, as of this week, we’re landing a solution that means you can drop the type coercions and just do a lookup like you would normally, and it will Just Work™️.¹ Keep your eyes open for the ember-cli-typescript 1.1 release in the next couple days!

Once this release of both ember-cli-typescript and the updated typings land, when you generate an Ember Data model by doing ember generate model person firstName:string lastName:string, it will look like this:

import DS from "ember-data";

export default class Person extends DS.Model.extend({
  firstName: DS.attr("string"),
  lastName: DS.attr("string")
}) {
  // normal class body definition here
}

// DO NOT DELETE: this is how TypeScript knows how to look up your models.
declare module "ember-data" {
  interface ModelRegistry {
    person: Person;
  }
}

That module and interface declaration at the bottom merges the declaration for this model with the declarations for all the other models. You’ll see the same basic pattern for DS.Adapter and DS.Serializer instances. The result is that using a model will now look like this. In addition to the Person model definition just above, our adapter might be like this:

import DS from "ember-data";

export default class Person extends DS.Adapter {
  update(changes: { firstName?: string; lastName?: string }) {
    fetch("the-url-to-change-it", {
      method: "POST",
      body: JSON.stringify(changes)
    });
  }
}

declare module "ember-data" {
  interface ModelRegistry {
    person: Person;
  }
}

Then putting the pieces together, our component definition will just look like this:

import Component from "@ember/component";
import { inject as service } from "@ember/service";

export default class PersonCard extends Component {
  id: string | number;

  store = service("store");
  model = this.store.findRecord("person", this.id);

  actions = {
    savePerson(changes: { firstName?: string; lastName?: string }) {
      this.store.adapterFor("person").update(changes);
    }
  };
}

The type of model here is now Person & DS.PromiseObject<Person> (which is actually what Ember Data returns for these kinds of things!), and the this.store.adapterFor actually correctly returns the Person adapter as well, so the call to its update method type-checks as well (including guaranteeing that the arguments to it are correct). That also means you’ll get autocompletion for those, including for their types, if you’re using an editor configured for it. And, happily for everyone, if you mistype a string (preson instead of person, for example), you’ll get a compile-time error!

Notice as well that the service injection is much cleaner than it was in earlier examples in the series. That’s because we made the same “registry”-type changes—as I suggested we might back in Part 1!—for controller and service injections. Before, for this kind of thing:

export default class PersonCard extends Component {
  store: Computed<DS.Store> = service();
}

Now:

export default class PersonCard extends Component {
  store = service("store");
}

That’s not quite as minimalist as what you get in vanilla Ember (where the name of the property is used to do the lookup at runtime), but it’s pretty close, and a huge improvement! Not least since it’s exactly as type-checked, and therefore as friendly to autocomplete/IntelliSense/etc. as it was before.

Migrating existing items

Your path forward for using the new approach is straightforward and fairly mechanical:

Add the module-and-interface declaration for each Ember Data Model, Adapter, and Serializer; and also each Ember Service and Controller you have defined.
Remove any type coercions you’ve written out already for these.

1. Add declaration

`DS.Model`

Before:

import DS from "ember-data";

export default class Person extends DS.Model.extend({
  firstName: DS.attr("string"),
  lastName: DS.attr("string")
}) {}

Now:

import DS from "ember-data";

export default class Person extends DS.Model.extend({
  firstName: DS.attr("string"),
  lastName: DS.attr("string")
}) {}

declare module "ember-data" {
  interface ModelRegistry {
    person: Person;
  }
}

`DS.Adapter`

Before:

import DS from "ember-data";

export default class Person extends DS.Adapter {
  // customization
}

Now:

import DS from "ember-data";

export default class Person extends DS.Adapter {
  // customization
}

declare module "ember-data" {
  interface AdapterRegistry {
    person: Person;
  }
}

`DS.Serializer`

Before:

import DS from "ember-data";

export default class Person extends DS.Serializer {
  // customization
}

Now:

import DS from "ember-data";

export default class Person extends DS.Serializer {
  // customization
}

declare module "ember-data" {
  interface SerializerRegistry {
    person: Person;
  }
}

`Service`

Before:

import Service from "@ember/service";

export default class ExternalLogging extends Service {
  // implementation
}

Now:

import Service from "@ember/service";

export default class ExternalLogging extends Service {
  // implementation
}

declare module "ember" {
  interface ServiceRegistry {
    "external-logging": ExternalLogging;
  }
}

`Controller`

Before:

import Controller from "@ember/controller";

export default class Profile extends Controller {
  // implementation
}

Now:

import Controller from "@ember/controller";

export default class Profile extends Controller {
  // implementation
}

declare module "@ember/controller" {
  interface ControllerRegistry {
    profile: Profile;
  }
}

If you don’t do add the type registry declarations, you’ll just get back:

compiler errors for any use of a string key in your service and controller lookups
Service and Controller (the top-level classes we inherit from) instead of the specific class you created if you use the no-argument version of the inject helpers
compiler errors for DS.Model, DS.Adapter, and DS.Serializer lookups (since they always have a string key)

If you’re looking to allow your existing code to all just continue working while you slowly migrate to TypeScript, you can add this as a fallback somewhere in your own project (adapted to whichever of the registries you need):

declare module "ember-data" {
  interface ModelRegistry {
    [key: string]: DS.Model;
  }
}

This will lose you the type-checking if you type a key that doesn’t exist, but it means that models you haven’t yet added the type definition for won’t throw compile errors. (We’ve made this opt-in because otherwise you’d never be able to get that type-checking for using an invalid key.)

2. Remove any existing coercions

Now that we have the necessary updates to be able to do these lookups automatically in the compiler, we need to remove any existing type coercions.

`Service` and `Controller`

This change is really straightforward (and actually just simplifies things a lot!) for Service and Controller injections.

  import Component from '@ember/component';
  import { inject as service } from '@ember/service';
- import Computed from '@ember/object/computed';
-
- import ExternalLogging from 'my-app/services/external-logging';

  export default class UserProfile extends Component {
-   externalLogging: Computed<ExternalLogging> = service();
+   externalLogging = service('external-logging');
    // other implementation
  }

Ember Data

This looks slightly different for the Ember Data side.

If you’ve been using the type coercion forms we shipped as a stopgap, like this—

const person = this.store.findRecord<Person>("person", 123);

—you’ll need to drop the type coercion on findRecord<Person>, which will give you a type error:

[ts] Type ‘Person’ does not satisfy the constraint ‘string’.

This is because, behind the scenes, findRecord still takes a type parameter, but it’s now a string—the name of the model you’re looking up—not the model itself. As such, you should never supply that type parameter yourself; it’s taken care of automatically. As a result, your invocation should just be:

const person = this.store.findRecord("person", 123);

The full type of lookups

One last note on Ember Data: calls like findRecord('person', 123) actually return the type Person & DS.PromiseObject<Person> – i.e., a type that acts like both the model and a promise wrapping the model. This is, to be sure, weird, but it’s the reality, so that’s what our types give you.

If you find yourself needing to write out that type locally for some reason—e.g. because part of your app deals explicitly with the result of a lookup—you may find it convenient to define a global type alias like this:

type Loaded<T> = T & DS.PromiseObject<T>;
const person: Loaded<Person> = this.store.findRecord("person", 123);

Given the new support for getting that type automatically, you shouldn’t normally need that, but it’s convenient if or when you do need it. For example, if a component is passed the result of a Person lookup and needs to be able to treat it as a promise or the model, you could write it like this:

import Component from "@ember/component";

export default class PersonDisplay extends Component {
  model: Loaded<Person>; // instead of just `model: Person`
}

Preview: Mirage

As it turns out, Ember CLI Mirage’s approach is a lot like Ember Data’s (although it’s actually a lot more dynamic!), so I have a very similar approach working in our codebase for doing lookups with Mirage’s database. Sometime in February or March, we hope to get that completed and upstreamed into Mirage itself, so that you can get these exact same benefits when using Mirage to write your tests.

Conclusion

And that’s pretty much a wrap on Ember Data! The next post you can expect in this series will be a break from nitty-gritty “how to use TS in Ember” posts for a very exciting, closely related announcement—probably tomorrow or Monday! The post after that will be a deep dive into (mostly the limitations of!) writing types for mixins and proxies.

If you’re curious about the mechanics, we’re basically setting up a “type registry” which maps the string keys to the correct model, so that the type of e.g. store.createRecord('some-model', { ... }) will do a lookup in an interface which defines a mapping from model name, i.e. some-model here, to the model type, e.g. export default class SomeModel extends DS.Model.extend({ ... }) {}. I’ll write up a full blog post on the mechanics of that sometime soon.↩

TypeScript and Ember.js Update, Part 3

2018-01-25T07:00:00-05:00

(See the rest of the series. →)

Note: if you’re following along with this as I publish it in late January 2018, please go back and read the end of Part 2, which I updated substantially yesterday evening to include more material I missed in the first version of that post, but which belonged there and not here.

In the previous posts in this series, I introduced the big picture of how the story around TypeScript and Ember.js has improved over the last several months and walked through some important background on class properties. In this post, I’ll build on that foundation to look closely at computed properties, actions, and mixins.

Here’s the outline of this update sequence:

Overview, normal Ember objects, component arguments, and injections.
Class properties—some notes on how things differ from the Ember.Object world.
Computed properties, actions, mixins, and class methods (this post).
Using Ember Data, and service and controller injections improvements.
Mixins and proxies; or: the really hard-to-type-check bits.

A detailed example (cont’d.) – computed properties, mixins, actions, and class methods

Let’s start by recalling the example Component we’re working through:

import Component from '@ember/component';
import { computed, get } from '@ember/object';
import Computed from '@ember/object/computed';
import { inject as service } from '@ember/service';
import { assert } from '@ember/debug';
import { isNone } from '@ember/utils';

import Session from 'my-app/services/session';
import Person from 'my-app/models/person';

export default class AnExample extends Component {
  // -- Component arguments -- //
  model: Person;      // required
  modifier?: string;  // optional, thus the `?`

  // -- Injections -- //
  session: Computed<Session> = service();

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel = computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${get(this.model, 'firstName')};`;
    }
  );

  aComputed = computed('aString', function(this: AnExample): number {
    return this.lookAString.length;
  });

  isLoggedIn = bool('session.user');
  savedUser: Computed<Person> = alias('session.user');

  actions = {
    addToCollection(this: AnExample, value: string) {
      const current = this.get('aCollection');
      this.set('aCollection', current.concat(value));
    }
  };

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.get('aCollection').includes('ahoy')) {
      this.set('aCollection', current.concat('ahoy'));
    }
  }
}

Computed properties

We already covered component arguments and injections as well as basic class properties and the exceptions to normal class-property ways of doing things, in Parts 1 and 2. With that background out of the way, we can now turn to computed properties. I’m including the component arguments in this code sample because they’re referenced in the computed property. Assume Person is a pretty “person” representation, with a firstName and a lastNameand maybe a few other properties.

  // -- Component arguments -- //
  model: Person;      // required
  modifier?: string;  // optional, thus the `?`

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel = computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${get(this.model, 'firstName')};`;
    }
  );

  aComputed = computed('aString', function(this: AnExample): number {
    return this.lookAString.length;
  });

  isLoggedIn = bool('session.user');
  savedUser: Computed<Person> = alias('session.user');

`computed` properties

When using a computed property in the brave new world of ES6 classes, we normally just assign them as instance properties. As mentioned in the previous post, and in line with my comments above, this has some important tradeoffs around performance. If you need the absolute best performance, you can continue to install them on the prototype by doing this instead:

export default class MyComponent extends Component.extend({
  fromModel: computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${get(this.model, 'firstName')};`;
    }
  ),
}) {
  // other properties
}

Whichever way you do it, TypeScript will correctly infer the type of the computed property in question (here fromModel) as long as you explicitly annotate the return type of the callback passed to computed. Accordingly, in this case, the type of fromModel is ComputedProperty<string>. The fact that it’s a ComputedProperty means if you try to treat it as a plain string, without using Ember.get to unwrap it, TypeScript will complain at you.¹

// type checking error:
this.fromModel.length;

// type checking valid:
this.get('fromModel').length;

The other really important thing to note here is the use of this: MyComputed. By doing this, we’re telling TypeScript explicitly that the type of this in this particular function is the class context. We have to do this here, because we don’t have any way to tell the computed helper itself that the function inside it will be bound to the this context of the containing class. Put another way: we don’t have any other way to tell TypeScript that one of the things computed does is bind this appropriately to the function passed into it; but gladly we do have this way—otherwise we’d be out of luck entirely! (You’ll see the same thing below when we look at actions). The boilerplate is a bit annoying, admittedly—but it at least makes it type-check.

Computed property macros

Beyond computed, there are a lot of other computed property tools we use all the time. Some of them can (and therefore do) infer the type of the resulting computed property correctly. But there are a bunch of idiomatic things that TypeScript does not and cannot validate – a number of the computed property macros are in this bucket, because they tend to be used for nested keys, and as noted above, TypeScript does not and cannot validate nested keys like that.

We have a representative of each of these scenarios:

  isLoggedIn = bool('session.user');
  savedUser: Computed<Person> = alias('session.user');

In the case of isLoggedIn, the bool helper only ever returns a boolean, so the type of isLoggedIn is ComputedProperty<boolean>. In the case of savedUser, since TypeScript can’t figure out what the nested key means, we have to specify it explicitly, using Computed<Person>.² In these cases, you have to do the work yourself to check that the type you specify is the correct type. If you write down the wrong type here, TypeScript will believe you (it doesn’t have any other good option!) and you’ll be back to things blowing up unexpectedly at runtime.

The typings supply the concrete (non-any) return type for: and, bool, equal, empty, gt, gte, lt, lte, match, map, max, min, notEmpty, none, not, or, and sum.

On nested keys

As noted above, TypeScript cannot do a lookup for any place using nested keys—which means that this.get('some.nested.key') won’t type-check, sadly. This is an inherent limitation of the type system as it stands today, and for any future I can foresee. The problem is this: what exactly is 'some.nested.key'? It could be what we use it for in the usual scenario in Ember, of course: a string representing a lookup on a property of a property of a property of whatever this is. But it could equally well be a key named 'some.nested.key'. This is perfectly valid JavaScript, after all:

const foo = {
  ['some.nested.key']: 'Well, this is weird, but it works',
};

TypeScript does not today and presumably never will be able to do that lookup. The workaround is to do one of two things:

If you know you have a valid parent, you can do the (catastrophically ugly, but functional) nested Ember.get that now litters our codebase:
```
import { get } from '@ember/object';
const value = get(get(get(anObject, 'some'), 'nested'), 'key');
```
Yes, it’s a nightmare. But… it type-checks, and it works well enough in the interim until we get a decorators-based solution that lets us leverage RFC #281.
Use the // @ts-ignore to simply ignore the type-unsafety of the lookup. This approach is preferable when you don’t know if any of the keys might be missing. If, for example, either some or nested were undefined or null, the lookup example above in (1) would fail.
```
import { get } from '@ember/object';
// @ts-ignore -- deep lookup with possibly missing parents
const value = get(anObject, 'some.nested.key');
```

Actions

What about actions? As usual, these just become class instance properties in the current scheme.

  actions = {
    addToCollection(this: AnExample, value: string) {
      const current = this.get('aCollection');
      this.set('aCollection', current.concat(value));
    }
  };

As with computed properties, we need the this type declaration to tell TypeScript that this method is going to be automatically bound to the class instance. Otherwise, TypeScript thinks the this here is the actions hash, rather than the MyComponent class.³

Happily, that’s really all there is to it for actions: they’re quite straightforward other than needing the this type specification.

Types in `.extend({...})` blocks

By and large, you can get away with using the same this: MyComponent trick when hacking around prototypal extension problems, or performance problems, by putting computed properties in a .extend({...} block. However, you will sometimes see a type error indicating that the class is referenced in its own definition expression. In that case, you may need to judiciously apply any, if you can’t make it work by using normal class properties.

`constructor` and class methods

ES6 class constructors and class methods both work as you’d expect, though as we’ll see you’ll need an extra bit of boilerplate for methods, at least for now.

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample): void {
    if (!this.get('aCollection').includes('ahoy')) {
      this.set('aCollection', current.concat('ahoy'));
    }
  }

For the most part, you can just switch to using normal ES6 class constructors instead of the Ember init method. You can, if you so desire, also move existing init functions passed to a .extends({ ...}) hash to class methods, and they’ll work once you change this._super(...arguments) to super.init(...arguments). It’s worth pausing to understand the relationship between init and prototypal init and the constructor. An init in the .extends() hash runs first, then an init method on the class, then the normal constructor.⁴

Note that you do not need to (and cannot) annotate the constructor with this: MyComponent. Depending on the class you’re building, you may occasionally have type-checking problems that come up as a result of this. I’ve only ever seen that happen when using computed properties while defining a proxy,⁵ but it does come up. In that case, you can fall back to using init as a method, and set this: MyComponent on it, and things will generally fall out as working correctly at that point. When it comes up, this seems to be just a limitation of what this is understood to be in a constructor given Ember’s rather more-complex-than-normal-classes view of what a given item being constructed is.

Other class methods do also need the this type specified if they touch computed properties. (Normal property access is fine without it.) That’s because the lookups for ComputedProperty instances (using Ember.get or Ember.set) need to know what this is where they should do the lookup, and the full this context isn’t inferred correctly at present. You can either write that on every invocation of getand set, like (this as MyComponent).get(...), or you can do it once at the start of the method. Again, a bit boiler-platey, but it gets the job done and once you’re used to it it’s minimal hassle.⁶

One last note, which I didn’t include in the example: if you have a function (usually an action) passed into the component, you can define it most simply by just using onSomeAction: Function; in the class definition, right with other class arguments. However, it’s usually most helpful to define what the type should actually be, for your own sanity check if nothing else. As with e.g. model in this example, we don’t actually have a good way to type-check that what is passed is correct. We can, however, at least verify in the constructor that the caller passed in a function using assert, just as with other arguments.

Summary

So that’s a wrap on components (and controllers, which behave much the same way).

In the next post, I’ll look at the elephant in the room: Ember Data (and closely related concern Ember CLI Mirage). While you can make Ember Data stuff largely work today, it’s still a ways from Just Works™️, sadly, but we’ll cover how to work around the missing pieces—we’ve gotten there in our own codebase, so you can, too!

As mentioned in Part 2, this problem doesn’t go away until we get decorators, unless you’re putting them on the prototype via .extends()—but see below for the problems with that. The short version is, we need decorators for this to actually be nice. Once we get decorators, we will be able to combine them with the work done for RFC #281 and normal lookup will just work:
```
@computed('model.firstName')
get fromModel() {
  return `My name is ${this.model.firstName};`;
}
```
↩
I’ve used Computed<Person> and similar throughout here because it’s the most clear while still being reasonably concise. The actual type name in Ember’s own code is ComputedProperty, but ComputedProperty<Person> is long, and it wouldn’t have added any real clarity here. In my own codebase, we use CP (for “Computed Property”) for the sake of brevity—so here that would just be CP<Person>.↩

In the future, this problem will hopefully be solved neatly by decorators:

  @action
  addToCollection(value: string) {
    const current = this.get('aCollection');
    this.set('aCollection', current.concat(value));
  }

For today, however, specifying a this type is where it’s at.↩

You can see this for yourself in this Ember Twiddle—just open your developer tools and note the sequence.↩
Proxies, along with details of mixins, are a subject I’m leaving aside for Part 5, otherwise known as the “wow, this stuff is really weird to type” entry in the series.↩
Not no hassle, though, and I look forward to a future where we can drop it, as Ember moves more and more toward modern JavaScript ways of solving these same problems!↩

TypeScript and Ember.js Update, Part 2

2018-01-24T07:00:00-05:00

(See the rest of the series. →)

In the previous post in this series, I introduced the big picture of how the story around TypeScript and Ember.js has improved over the last several months. In this post, I’ll be pausing from TypeScript-specific to take a look at how things work with class properties, since they have some big implications for how we work, which then have ripple effects on computed properties, actions, etc.

Here’s the outline of this update sequence:

Overview, normal Ember objects, component arguments, and injections.
Class properties—some notes on how things differ from the Ember.Object world (this post).
Computed properties, actions, mixins, and class methods.
Using Ember Data, and service and controller injections improvements.
Mixins and proxies; or: the really hard-to-type-check bits.

A detailed example (cont’d.) – class properties

Let’s start by recalling the example Component we’re working through:

import Component from '@ember/component';
import { computed, get } from '@ember/object';
import Computed from '@ember/object/computed';
import { inject as service } from '@ember/service';
import { assert } from '@ember/debug';
import { isNone } from '@ember/utils';

import Session from 'my-app/services/session';
import Person from 'my-app/models/person';

export default class AnExample extends Component {
  // -- Component arguments -- //
  model: Person;      // required
  modifier?: string;  // optional, thus the `?`

  // -- Injections -- //
  session: Computed<Session> = service();

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel = computed(
    'model.firstName',
    function(this: AnExample): string {
      return `My name is ${get(this.model, 'firstName')};`;
    }
  );

  aComputed = computed('aString', function(this: AnExample): number {
    return this.lookAString.length;
  });

  isLoggedIn = bool('session.user');
  savedUser: Computed<Person> = alias('session.user');

  actions = {
    addToCollection(this: AnExample, value: string) {
      const current = this.get('aCollection');
      this.set('aCollection', current.concat(value));
    }
  };

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.get('aCollection').includes('ahoy')) {
      this.set('aCollection', current.concat('ahoy'));
    }
  }
}

Throughout, you’ll note that we’re using assignment to create these class properties—a big change from the key/value setup in the old .extends({ ... }) model:

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

Class properties like this are instance properties. These are compiled to, because they are equivalent to, assigning a property in the constructor. That is, these two ways of writing class property initialization are equivalent—

At the property definition site:

export default class AnExample extends Component {
  // snip...

  // -- Class properties -- //
  aString = 'this is fine';
  aCollection: string[] = [];

  // snip..

  constructor() {
    super();
    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  // snip...
}

In the constructor:

export default class AnExample extends Component {
  // snip...

  // -- Class properties -- //
  aString: string;
  aCollection: string[];

  constructor() {
    super();

    this.aString = 'this is fine';
    this.aCollection = [];

    assert('`model` is required', !isNone(this.model));

    this.includeAhoy();
  }

  // snip...
}

You can see why the first one is preferable: if you don’t need any input to the component to set the value, you can simply set the definition inline where the property is declared.

However, this is quite unlike using .extend, which installs the property on the prototype. Three very important differences from what you’re used to fall out of this, and none of them are specific to TypeScript.¹

1. Default values

Since class property setup runs during the constructor, if you want the caller to be able to override it, you must give it an explicit fallback that references what’s passed into the function. Something like this:

class AnyClass {
  aDefaultProp = this.aDefaultProp || 0;
}

Again, translated back into the constructor form:

class AnyClass {
  constructor() {
    this.aDefaultProp = this.aDefaultProp || 0;
  }
}

Here, you can see that if something has already set the aDefaultProp value (before the class constructor is called), we’ll use that value; otherwise, we’ll use the default. You can think of this as being something like default arguments to a function. In our codebase, we have started using _.defaultTo, which works quite nicely. In the old world of declaring props with their values in the .extends({ ... }) hash, we got this behavior “for free”—but without a lot of other benefits of classes, so not actually for free.

2. No more shared state

Because these are instance properties, not assigned on the prototype, you do not have to worry about the problem—well-known among experienced Ember.js developers, but prone to bite people new to the framework—where you assign an array or object in the .extend() method and then find that it’s shared between instances.

export default Component.extend({
  anArray: [],  // <- this *will* be shared between instances
});

We’ve long had to handle this by setting up those properties in our init() method instead, so that they are created during object instantiation, rather than on the prototype. This problem goes away entirely with classes, including in TypeScript.

export default class MyComponent extends Component {
  anArray = [];  // <- this will *not* be shared between instances
}

(Note that here, we don’t have a type for the array, so it’s of type any[]; we always need type annotations for empty arrays if we want them to be a “narrower,” or more specific, type than that.)

3. Performance changes

The flip-side of this is that the only way we currently have to create computed property instances (until decorators stabilize) is also as instance, not prototype, properties. I’ll look at computed properties (and their types) in more detail in the next post, so here mostly just note how the computed is set up on the class: by assignment, not as a prototypal property.

export default class MyComponent extends Component {
  aString = 'Hello, there!';

  itsLength = computed('aString', function(this: MyComponent): number {
    return this.aString.length;
  });
}

This does have a performance cost, which will be negligible in the ordinary case but pretty nasty if you’re rendering hundreds to thousands of these items onto the page. You can use this workaround for these as well as for any other properties which need to be prototypal (more on that in the next post as well):²

export default class MyComponent extends Component.extend({
  itsLength: computed('aString', function(this: MyComponent): number {
      return this.aString.length;
    }
  );
}) {
  aString = 'Hello, there!';
}

This looks really weird, but it works exactly as you’d expect.

Class property variants

There are two times when things will look different from basic class properties. Both have to do with setting up the prototype to work the way other parts of the Ember object ecosystem expect.

Variant 1: Prototypal/merged properties

The first is when you’re using properties that need to be merged with properties in the prototype chain, e.g. attributeBindings or classNameBindings, or which (because of details of how components are constructed) have to be set on the prototype rather than as instance properties, e.g. tagClass.

For those, we can just leverage .extend in conjunction with classes:

import Component from '@ember/component';

export default class MyListItem extends Component.extend({
  tagName: 'li',
  classNameBindings: ['itemClass']
}) {
  itemClass = 'this-be-a-list';

  // etc.
}

This is also how you’ll use mixins (on defining them, see below):

import Component from '@ember/component';
import MyMixin from 'my-app/mixins/my-mixin';

export default class AnExample extends Component.extend(MyMixin) {
  // the rest of the definition.
}

Note, however—and this is very important—that you cannot .extend an existing class implementation. As a result, deep inheritance hierarchies may make transitioning to classes in Ember painful. Most importantly: they may work some of the time in some ways, but will break when you least expect. So don’t do that! (This isn’t a TypeScript limitation; it’s a limitation of classes in Ember today.)³

Variant 2: Mixins

The other time you’ll have to take a different tack—and this falls directly out of the need for prototypal merging—is with Mixins, which don’t yet work properly with classes. Worse, it’s difficult (if not impossible) to get rigorous type-checking internally in Mixin definitions, because you cannot define them as classes: you have to use the old style throughout, because mixins are created with .create(), not .extend().

I’ll have a lot more to say about these in part 5 of this series, including a detailed example of how to carefully type-annotate one and use it in another class. For now, suffice it to say that you’ll still need to incorporate Mixins via .extend():

import Component from '@ember/component';
import MyMixin from 'my-app/mixins/my-mixin';

export default class SomeNewComponent extends Component.extend(MyMixin) {
  // normal class properties
}

Summary

Those are the biggest differences from Ember.Object that you need to be aware of when working with class properties in Ember.js today, at least in my experience working with them day to day. These are not the only differences with classes, though, especially when dealing with TypeScript, so in my next entry we’ll take a look at how classes work (and work well!) with most things in Ember.js and TypeScript together.

You can use this same feature on classes using Babel, with the class properties transform.↩
Even when Ember.js RFC #281 lands, this problem will not go away, at least under the current implementation, since these will not be transformed into getters on the prototype. We are waiting for decorators to solve this problem completely.↩

In the future, we’ll (hopefully and presumably 🤞🏼) have an escape hatch for those merged or prototypally-set properties via decorators. That’ll look something like this:

import Component from '@ember/component';
import { className, tagName } from 'ember-decorators/component';

@tagName("li")
export default class MyListItem extends Component {
  @className itemClass = 'this-be-a-list';

  @action
  sendAMessage(contents: string): void {

  }
  // etc.
}

↩

TypeScript and Ember.js Update, Part 1

2018-01-22T07:10:00-05:00

(See the rest of the series. →)

Back in July 2017, I wrote a post on how to using TypeScript in your Ember.js apps. At the time, we were still busy working on getting the typings more solid for Ember itself, and class syntax for Ember was apparently a long way away.

Things have gotten quite a bit better since then, so I thought I’d update that post with recommendations for using TypeScript in an app now with the updated typings, as well as with another six months of experience using TypeScript in our app at Olo (~20k lines of code in the app and another ~15k in tests).

Here’s how I expect this update series to go:

Overview, normal Ember objects, component arguments, and injections (this post).
Class properties—some notes on how things differ from the Ember.Object world.
Computed properties, actions, mixins, and class methods.
Using Ember Data, and service and controller injections improvements.
Mixins and proxies; or: the really hard-to-type-check bits.

Normal Ember objects

For normal Ember objects, things now mostly just work if you’re using class-based syntax, with a single (though very important) qualification I’ll get to in a minute. And you can use the class-based syntax today in Ember.js—all the way back to 1.13, as it turns out. If you want to learn more, you can read this RFC or this blog post, both by @pzuraq (Chris Garrett), who did most of the legwork to research this and flesh out the constraints, and who has also been doing a lot of work on Ember Decorators.

Accordingly, I’m assuming the use of ES6 class syntax throughout. The big reason for this is that things mostly just don’t work without it. And we’ll see (in a later post) some hacks to deal with places where parts of Ember’s ecosystem don’t yet support classes properly. In general, however, if you see an error like "Cannot use 'new' with an expression whose type lacks a call or construct signature.", the reason is almost certainly that you’ve done export default Component.extend({...}) rather than creating a class.

A detailed example

That means that every new bit of code I write today in our app looks roughly like this, with only the obvious modifications for services, routes, and controllers—I picked components because they’re far and away the most common things in our applications.

In order to explain all this clearly, I’m going to start by showing a whole component written in the new style. Then, over the rest of this post and the next post, I’ll zoom in on and explain specific parts of it.

import Component from "@ember/component";
import { computed, get } from "@ember/object";
import Computed from "@ember/object/computed";
import { inject as service } from "@ember/service";
import { assert } from "@ember/debug";
import { isNone } from "@ember/utils";

import Session from "my-app/services/session";
import Person from "my-app/models/person";

export default class AnExample extends Component {
  // -- Component arguments -- //
  model: Person; // required
  modifier?: string; // optional, thus the `?`

  // -- Injections -- //
  session: Computed<Session> = service();

  // -- Class properties -- //
  aString = "this is fine";
  aCollection: string[] = [];

  // -- Computed properties -- //
  // TS correctly infers computed property types when the callback has a
  // return type annotation.
  fromModel = computed("model.firstName", function(this: AnExample): string {
    return `My name is ${get(this.model, "firstName")};`;
  });

  aComputed = computed("aString", function(this: AnExample): number {
    return this.lookAString.length;
  });

  isLoggedIn = bool("session.user");
  savedUser: Computed<Person> = alias("session.user");

  actions = {
    addToCollection(this: AnExample, value: string) {
      const current = this.get("aCollection");
      this.set("aCollection", current.concat(value));
    }
  };

  constructor() {
    super();
    assert("`model` is required", !isNone(this.model));

    this.includeAhoy();
  }

  includeAhoy(this: AnExample) {
    if (!this.get("aCollection").includes("ahoy")) {
      this.set("aCollection", current.concat("ahoy"));
    }
  }
}

Component arguments

export default class AnExample extends Component {
  // Component arguments
  model: Person;      // required
  modifier?: string;  // optional, thus the `?`

I always put these first so that the “interface” of the object is clear and obvious. You can do the same thing on a controller instance; in that case you would export a Model from the corresponding Route class and import it into the Controller. It’s a bit of boilerplate, to be sure, but it lets you communicate your interface clearly to consumers of the Component or Controller.

An important note about these kind of arguments: you do not have to do this.get(...) (or, if you prefer, get(this, ...)) to access the properties themselves: they’re class instance properties. You can simply access them as normal properties: this.model, this.modifier, etc. That even goes for referencing them as computed properties, as we’ll see below.

For optional arguments, you use the ? operator to indicate they may be undefined. To get the most mileage out of this, you’ll want to enable strictNullChecks in the compiler options.¹ However, note that we don’t currently have any way to validate component argument invocation.[^ts-templates] The way I’ve been doing this is using Ember’s debug assert in the constructor:

assert("`model` is required", !isNone(this.model));

import Component from "@ember/component";
import { Maybe } from "true-myth";

export default class MyComponent extends Component {
  optionalArg?: string;
  optionalProperty = Maybe.of(this.optionalArg);
}

Then if you invoke the property without the argument, it’ll construct a Nothing; if you invoke it with the argument, it’ll be Just with the value. [^ts-templates]: A few of us have batted around some ideas for how to solve that particular problem, but if we manage those, it’ll probably be way, way later in 2018.

Edit, January 24, 2018: Starting in TypeScript 2.7, you can enable a flag, --strictPropertyInitialization, which requires that all declared, non-optional properties on a class be initialized in the constructor or with a class property assignment. (There’s more on class property assignment in part 2 of this series.) If you do that, all arguments to a component should be defined with the definite assignment assertion modifier, a ! after the name of the property, as on model here:

export default class AnExample extends Component {
  // Component arguments
  model!: Person;     // required
  modifier?: string;  // optional, thus the `?`

You should still combine that with use of assert so that any misses in template invocation will get caught in your tests.

Injections

  // -- Injections -- //
  session: Computed<Session> = service();

Here, the most important thing to note is the required type annotation. In principle, we could work around this by requiring you to explicitly name the service and using a “type registry” to look up what the service type is – more on that below in my discussion of using Ember Data – but I’m not yet persuaded that’s better than just writing the appropriate type annotation. Either way, there’s some duplication. 🤔 We (everyone working in the typed-ember project) would welcome feedback here, because the one thing we can’t do is get the proper type without one or the other of these.

Edit, February 5, 2018: see Part 4 for some updates to this—I actually went ahead and built and implemented that approach, and everything is much nicer now.

  // the current approach -- requires importing `Session` so you can define it
  // on the property here
  session: Computed<Session> = service();

  // the alternative approach I've considered -- requires writing boilerplate
  // elsewhere, similar to what you'll see below in the Ember Data section
  session = service('session');

One other thing to notice here is that because TypeScript is a structural type system, it doesn’t matter if what is injected is the actual Session service; it just needs to be something that matches the shape of the service – so your normal behavior around dependency injection, etc. is all still as expected.

That’s enough for one post, I think. In the next entry, we’ll pick up with how you handle class properties, including computed properties, and then talk about mixins as well. In the post after that, we’ll look at Ember Data and some related concerns.

This isn’t my preferred way of handling optional types; a Maybe type is. And you can, if you like, use Maybe here:↩

Exploring 4 Languages: Starting to Model the Domain

2018-01-14T09:00:00-05:00

In the first three chapters of Domain Modeling Made Functional, Wlaschin walks through the creation of a “domain model” for an order-taking system. (It’s well worth reading the book just for a bunch of the lessons in that section—I found them quite helpful!) Then, after spending a chapter introducing F^♯’s type system, he introduces the ways you can use those type mechanics to express the domain. In today’s post, I’ll show the idiomatic implementations of these types in each of Rust, Elm, F^♯, and ReasonML.

Simple values

Simple wrapper types let you take simple types like strings, numbers, etc. and use types to represent part of the business domain you’re dealing with—the basic idea being that a Customer ID may be a number, but it’s not interchangeable with other numbers such as Order IDs.

Here’s the most ergonomic and effective (and automatically-formatted in line with the language standards, where applicable!) way to do that in each of the languages:

Rust:

struct CustomerId(i32);

Elm:

type CustomerId
    = CustomerId Int

F^♯:

type CustomerId = CustomerId of int

ReasonML:

type customerId =
  | CustomerId(int);

Note how similar these all are! The Rust implementation is the most distinctive, though you can do it with the same kind of union type as the others. Here’s how that would look:

enum CustomerId {
  CustomerId(i32),
}

For performance reasons, you might also choose to implement the F^♯ type as a struct:

<Struct>
type CustomerId = CustomerId of int

Complex data

Wlaschin then moves on to showing how to model more complex data structures: types that “and” or “or” together other data. We “and” data together using record or struct types, and “or” data together using “union” or “enum” types. (Assume we’ve defined CustomerInfo, ShippingAddress, etc. types for all of these.)

Rust:

// "and"
struct Order {
    customer_info: CustomerInfo,
    shipping_address: ShippingAddress,
    billing_address: BillingAddress,
    order_lines: Vec<OrderLine>,
    billing_amount: BillingAmount,
}

// "or"
enum ProductCode {
    Widget(WidgetCode),
    Gizmo(GizmoCode),
}

Elm:

-- "and"
type alias Order =
    { customerInfo : CustomerInfo
    , shippingAddress : ShippingAddress
    , billingAddress : BillingAddress
    , orderLines : List OrderLine
    , billingAmount : BillingAmount
    }

-- "or"
type ProductCode
    = Widget WidgetCode
    | Gizmo GizmoCode

F^♯:

// "and"
type Order = {
    CustomerInfo : CustomerInfo
    ShippingAddress : ShippingAddress
    BillingAddress : BillingAddress
    OrderLines : OrderLine list
    AmountToBill: BillingAmount
}

// "or"
type ProductCode =
    | Widget of WidgetCode
    | Gizmo of GizmoCode

ReasonML—note that since we’re assuming we’ve already defined the other types here, you can write this without duplicating the name and type declaration, just like you can with JavaScript object properties.

/* "and" */
type order = {
  customerInfo,
  shippingAddress,
  billingAddress,
  orderLine,
  billingAmount
};

/* "or" */
type productCode =
  | Widget(widgetCode)
  | Gizmo(gizmoCode);

An interesting aside: unless you planned to reuse these types, you wouldn’t usually write these as standalone types with this many wrapper types in it in Rust in particular (even if the compiler would often recognize that it could squash them down for you).¹ Instead, you’d normally write only the base enum type to start, and refactor out the struct wrapper later only if you found you needed it elsewhere:

enum ProductCode {
-    Widget(WidgetCode),
+    Widget(String),
-    Gizmo(GizmoCode),
+    Gizmo(String),
}

That said: given how the book is tackling things, and the fact that you might want to validate these types… having them as these low-cost wrappers is probably worth it. (In fact, having read a bit further than I’ve managed to write out yet, I can guarantee it.)

We work through the rest of the basic types this way. But what about the types where we don’t yet have a good idea how we want to handle them?

Each of these languages gives us an out (or more than one) for how to say “I don’t know what to put here yet.”

Rust (which does not have a built-in Never type… yet; see below):

// Make an empty enum (which you by definition cannot construct)
enum Never {}

// Use it throughout where we don't know the type yet. It will fail to compile
// anywhere we try to *use* this, because you can't construct it.
type OrderId = Never;

Elm (which has a built-in Never type):

-- It will fail to compile anywhere we try to *use* this, because you cannot
-- construct `Never`.
type alias OrderId =
    Never

F^♯ (which sort of does):

// Make a convenience type for the `exn`/`System.Exception` type
type Undefined = exn

type OrderId = Undefined

Reason (which also sort of does—identically with F^♯):

/* Make a convenience type for the `exn`/`System.Exception` type */
type undefined = exn;

/*
  Use it throughout where we don't know the type yet. It will compile, but fail
  to run anywhere we try to *use* this.
 */
type orderId = undefined;

For both F^♯ and Reason, that’s following Wlaschin’s example. The main reason to do that is to make explicit that we’re not actually wanting an exception type in our domain model, but just something we haven’t yet defined. Anywhere we attempted to use it, we’d have to handle it like, well… an exception, instead of an actual type.

type OrderId = !;

Workflows and functions

Once we have the basic types themselves in place, we need to write down the ways we transform between them. In a functional style, we’re not going to implement instance methods—though as we’ll see in the next post, what we do in Rust will have some similarities to class methods—we’re going to implement standalone functions which take types and return other types.

Again, you’ll note that despite the common lineage, there is a fair amount of variation here. (Note that we’d also have defined the UnvalidatedOrder, ValidationError, and ValidatedOrder types for all of this; I’m mostly interested in showing new differences here.)

Rust (using the Futures library to represent eventual computation):

type ValidationResponse<T> = Future<Item = T, Error = ValidationError>;

fn validate_order(unvalidated: UnvalidatedOrder) -> Box<ValidationResponse<ValidatedOrder>> {
    unimplemented!()
}

Elm (using the built-in Task type for eventual computation; Tasks encapsulate both eventuality and the possibility of failure):

type ValidationResponse a
    = Task (List ValidationError) a

type alias ValidateOrder =
    UnvalidatedOrder -> ValidationResponse ValidatedOrder

F^♯ (using the built-in Async type for eventual computation):

type ValidationResponse<'a> = Async<Result<'a,ValidationError list>>

type ValidateOrder =
    UnvalidatedOrder -> ValidationResponse<ValidatedOrder>

Reason (using the built-in JavaScript-specific Js.Promise type—which is exactly what it sounds like—for eventual computation):

type validationResponse('a) = Js.Promise.t(Js.Result.t('a, list(validationError)));

type validateOrder = unvalidatedOrder => validationResponse(validatedOrder);

Once again Rust is much more different here from the others than they are from each other. The biggest difference between Elm, F^♯, and Reason is how they handle generics and type parameters.

You’ll note that in Elm, they just follow the name of the wrapping type. This is a kind of syntactic symmetry: the way you name a generic type like this is the same basic way you construct it. It’s quite elegant. And as it turns out, the same is true of Reason; it’s just that its authors have chosen to follow OCaml and use parentheses for them instead of following Haskell with spaces—a reasonable choice, given Reason is surface syntax for OCaml and not Haskell.

F^♯ uses angle brackets, I strongly suspect, because that’s what C^# uses for generics, and keeping them syntactically aligned in things like this is very helpful. Rust similarly uses angle brackets for similarity with other languages which have similar surface syntax—especially C++ (with its templates).

The way you name generic parameters differs between the languages as well. Elm, following Haskell, uses lowercase letters to name its generics (usually called type parameters in Elm). F^# and Reason both (unsurprisingly) follow OCaml in using lowercase letters preceded by an apostrophe to name generics—in F^#, TypeGenericOver<'a>; in Reason, typeGenericOver('a). Rust follows the convention from languages like C++, Java, and C^# and uses capital letters, TypeGenericOver<T>. The use of specific letters is conventional, not mandated by the language (unlike the casing). The ML family usually starts with a and moves through the alphabet; Rust and the languages it follows usually start with T (for type) and moves forward through the alphabet. (Sometimes you’ll also see different letters where it’s obviously a better fit for what’s contained.)

These languages also vary in the syntax for constructing a list of things. In F^# has convenience syntax for a few built-ins (the most common being the List and Option types), allowing you to write them either as e.g. List<ConcreteType> or ConcreteType list (as here in the example). Elm, Reason, and Rust all just use the standard syntax for generic types—List a, list('a), and Vec<T> respectively.

Finally, you’ll also note that we haven’t written out a type declaration here for Rust; we’ve actually written out a stub of a function, with the unimplemented!() macro. If you invoke this function, you’ll get a clear crash with an explanation of which function isn’t implemented.

Now, Rust also does let us write out the type of these functions as type aliases if we want:

type ValidateOrder =
    Fn(UnvalidatedOrder) -> Box<ValidationResponse<ValidatedOrder>>;

You just don’t use these very often in idiomatic Rust; it’s much more conventional to simply write out what I did above. However, the one time you might use a type alias like this is when you’re defining the type of a closure and you don’t want to write it inline. This is a pretty sharp difference between Rust and the other languages on display here, and it goes to the difference in their approaches.

Rust is not a functional-first language in the way that each of the others are, though it certainly draws heavily on ideas from functional programming throughout and makes quite a few affordances for a functional style. Instead, it’s a programming language first and foremost interested in combining the most screaming performance possible with true safety, and leaning on ideas from the ML family (among others!) as part of achieving that.

Among other things, this is why you don’t have currying or partial application in Rust: those essentially require you to have invisible heap-allocation to be ergonomic. We don’t have that in Rust, as we do in Elm, Reason, and F^♯. If we want to pass around a function, we have to explicitly wrap it in a pointer to hand it around if we construct it in another function. (I won’t go into more of the details of this here; I’ve covered it some on New Rustacean and some in my Rust and Swift comparison a couple years ago.)

That same underlying focus on performance and explicitness is the reason we have Box<ValidationResponse<ValidatedOrder>> in the Rust case: we’re explicitly returning a pointer to the type here. In Elm, F^♯, and Reason, that’s always the case. But in Rust, you can and often do return stack-allocated data and rely on “move” semantics to copy or alias it properly under the hood.

Summary

So: lots of similarities here at first blush. The biggest differences that show up at this point are purely syntactical, other than some mildly sharper differences with Rust because of its focus on performance. The fact that these languages share a common lineage means it’s not hard to read any of them if you’re familiar with the others, and it’s actually quite easy to switch between them at the levels of both syntax and semantics.

As usual, when dealing with languages in a relatively similar family, it’s most difficult to learn the library differences. The most obvious example of that here is Reason’s Js.Promise, Elm’s Task, F^♯’s Async, and Rust’s Future types: each of those has their own quirks, their own associated helper functions or methods, and their own ways of handling the same basic patterns.

Still, if you have played with any one of these, you could pretty easily pick up one of the others. It’s sort of like switching between Python and Ruby: there are some real differences there, but the similarities are greater than the differences. Indeed, if anything, these languages are more similar than those.

Next time I’ll dig into Wlaschin’s chapter on validating the domain model, and here some of the not-just-syntax-level differences in the languages will start to become more apparent.

I can’t speak to what’s idiomatic this way in any of the non-Rust languages, because I just haven’t used them enough yet.↩

Exploring 4 Languages: Project Setup

2018-01-01T13:00:00-05:00

In this post, I’m just going to briefly talk through the steps I needed to do to set up each of the languages and my editor setup for them. Gladly, it was pretty simple. At the end, I’ll offer a note on my thoughts on the setup processes. (Note that this isn’t “How to do this for anyone ever”—it’s “how I did it, with some notes where it might be relevant to you.”)

For context, I’m running macOS and using VS Code as my editor. Whenever I say “Install the VS Code extension,” you can do it either by opening the extension side panel and searching for <Extension Name>, or by typing ext install <extension label>—I’ll write it like <Extension Name>/<extension label>.

The source code as of what I’m describing in this post is at the project-setup tag in the repo.

Rust

Language installation: Install rustup: curl https://sh.rustup.rs -sSf | sh¹.
Editor setup: Installed the VS Code extension: Rust (rls)/rust.
Project setup: In the root of my repo, I ran cargo new rust.

Elm

Language installation: There are installers, but I just did npm i -g elm.
Editor setup: Installed the VS Code Elm extension: Elm/elm.²
Project setup:
- Install the create-elm-app tool: npm i -g create-elm-app
- In the root of the project, I ran create-elm-app elm.

F^♯

Language installation: Install mono: brew install mono (note installation instructions here).
Editor setup: Install the VS Code Ionide extension: Ionide-fsharp/ionide-fsharp. It’ll automatically install the associated Paket and FAKE extensions from the Ionide project as well, and those will install Paket and FAKE during installation.
Project setup:
- In the root of the repo, I created the fsharp directory.
- Then I opened a VS Code instance to to that directory, opened the command palette, and ran F#: New Project.
  - I chose console
  - I left the directory blank
  - I named the project dmmf (for Domain Modeling Made Functional).
  - Since F^♯ (like C^♯) prefers PascalCase names, I renamed the generated module DMMF.

ReasonML

Language installation: Following the setup instructions here, I ran npm install -g bs-platform.
Editor setup: following the official instructions—
- I ran npm install -g https://github.com/reasonml/reason-cli/archive/3.0.4-bin-darwin.tar.gz to install the dependencies for the editor configuration.
- I installed the VS Code extension: Reason/reasonml.
Project setup: In the root of the repo, I ran bsb -init reason -theme basic-reason.

Comments on the setup processes

Most of the languages have fairly straightforward processes to get up and running with a good-to-excellent tooling experience.

The best of them is Rust, which is extremely easy to get up and running with.³ Elm is roughly in the middle—it’s less straightforward than Rust in that create-elm-app is not an officially supported approach, unlike rustup and cargo, so you’re going to have a much less awesome experience if you don’t know about it.

Reason and F^♯ both have slightly larger negatives.

Reason requires you to npm install a large, gzipped file with multiple dependencies all bundled, instead of having a dedicated installer a la rustup. It also has the possibility for a not-so-great first-run experience in the editor, which I discovered all too quickly.

F^♯ essentially requires you to use an editor extension to get the language setup with Paket, which is a much better choice of package manager than the default .NET package manager NuGet. Command line tools exist and are improving rapidly, and you can get them working… but it’s harder than it needs to be. And that project setup wizard is fine, but it’s a lot noisier than just doing create-elm-app or especially cargo new.

In any case, though, I have them all up and running now! More soon!

If you’re uncomfortable with running that script, there are other options as well.↩
Note that the VS Code extension is not the best experience out there for Elm: the Atom extensions (language-elm and elmjutsu) are. I stuck with VS Code because it’s good enough and, more importantly, the Code extensions are arguably best in class for the other languages… and it’s what I use every day.↩
I’m not just saying that because I’m a Rust fanboy, either! If Rust were hard to use, I’d be complaining louder because of my enthusiasm for the language.↩

Exploring 4 Languages

2017-12-31T20:20:00-05:00

Today, as I hit the first of the implementation chapters in Domain Modeling Made Functional, I started thinking about how I wanted to implement it. As I’ve noted elsewhere in the past, very little of the book is truly specific to F^♯, though that’s the language Wlaschin uses in the book—and Wlaschin himself agrees:

Thanks! Yes, it’s true that you could easily use #ElmLang, #RustLang, #Scala, or especially #OCaml to work through the book. I use hardly any F# specific features.

So… I decided to try something a little bit bonkers. I’m going to implement these exercises in four different languages:

These languages are all related: they’re descended from Standard ML. ReasonML and F^♯ are like siblings: Reason is merely a custom syntax for OCaml; F^♯ is (originally) an implementation of OCaml on .NET (though the two languages have diverged since F^♯ came into existence). Elm and Rust are cousins of each other and of Reason and F^♯, though they’re both drawing on other languages besides OCaml as well. I also have some familiarity with Rust, Elm, and F^♯ already, and have read the docs for Reason a couple times. So this is a bit less crazy than it might otherwise be.

Why, though? Mostly because I think it’ll be interesting to compare the implementations of the domain model from the book side by side. It’ll look just a bit different in each language, and I expect to learn a bit more of the feel of each language by doing this. (That side by side comparison is something I’ve done before and found very profitable.) I’ll also turn it into blog posts, which hopefully will be interesting to others!

More to come, and soon.

Types are Small

2017-12-29T14:00:00-05:00

I’ve been reading through Scott Wlaschin’s really excellent book Domain Modeling Made Functional and this quote (from his chapter introducing the idea of types in typed functional programming) crystallized something for me:

A type in functional programming is not the same as a class in object-oriented programming. It is much simpler. In fact, a type is just the name given to the set of possible values that can be used as inputs or outputs of a function.

A lot of times when I’m trying to explain how I use types in a typed functional programming style, this is a serious point of confusion—both for the Java or C♯ OOP programmer and for the programmers coming from dynamic languages. When people think of “types” they tend to think of classes and interfaces and methods, oh my!¹

One is the big heavy class. The other is a nice little LEGO block. The difference is huge in my day to day experience, but I’ve never been able to express it so clearly as Wlaschin’s quote.

I suspect that when I’m talking to most people coming from dynamically typed languages or from the standard OOP languages, they hear “Write three interfaces and six classes” when I say “using types to help me with my program.” But what I mean is “Write three tiny little shapes, and then one more that shows how they snap together in a slightly bigger one.” Types aren’t big heavy things. They’re just the shapes I want to flow through my program, written down like documentation for later… that gets checked for me to make sure it stays up to date, and lets me know if I missed something in my description of the shape of the data, or tried to do something I didn’t mean to before.

You can write a language like F♯ or TypeScript or Elm like you would C♯, but it’s generally not going to be an especially happy experience (and it’ll be less happy the more “purely functional,” a la Elm, you go). But you don’t have to! Types are just tiny little descriptions of the shapes you plan to deal with in a particular spot—more concise and more dependable than writing a JSDoc or something like that.

Types are small. You can build big things with them, but types are small.

In fact, nearly every “I’m just not into types” or even “I think types are worse for most things” talks I’ve seen—including this recent and popular one by Rich Hickey—tend to conflate all type systems together. But the experience of writing TypeScript is very different from the experience of writing C♯. (You’ll note that in that talk, for example, Hickey freely jumps back and forth between Java-style types and Haskell-style types when it suits his purposes, and he entirely skips past the structural type systems currently having something of a heyday.) In many cases, I suspect this is simply a lack of deep experience with the whole variety of type systems out there (though I’d not attribute that to any specific individual).↩

I Want JSON Decoders

2017-12-25T19:20:00-05:00

This post was originally published at DailyDrip.com. They’re doing really great work over there, so I encourage you to check out their content and consider subscribing!

The other day, I got a report about the Ember.js app I’m working on: when a customer applied a coupon in the basket, they’d see an indication that the coupon was applied, but the basket total would still display as if it hadn’t been updated. Orders were placed correctly, but they wouldn’t render right. I dug around for a bit, and then discovered that it was one of the (many) places where undefined was biting us.

How did this happen? It turned out it was a perfect storm: a confusingly-designed API combined with a reasonable (but in this case, very unhelpful) assumption in our data layer. When the total on a given basket dropped to zero, our API simply didn’t send back a value on the payload at all. Instead of { total: 0, ... }, there was just, well, { ... } – no total field at all. Meanwhile, our data layer was designed to let a server send back only the fields which required updating. That way, you can send back partial records to indicate only what has changed, instead of having to send back the whole of what might be a very large record, or a very large collection of records.

The combination was terrible, though: because the server didn’t send back the total field at all when it dropped to 0, the client never updated the total it displayed to the user: as far as it was concerned, the server was saying “no change here!”

The first and most obvious solution here, of course, is the one we implemented: we had the API always send back a value, even if that value was 0. But it seems like there should be a better way.

Lots of languages have fairly nice facilities for parsing JavaScript. Several languages even have tools for automatically constructing local, strongly-typed data structures from the structure of a JSON response on an API. F♯’s type providers are like this and really fancy in the way they’ll automatically derive the type for you so you don’t even have to write it out as you would in everything from Haskell to C#. But for the most part in JavaScript, you have at most a way to map data to a local record in your data store – certainly none of those type safe guarantees. In TypeScript, you can write the types you receive out carefully – though, as I discovered in this case, probably not carefully enough unless you model everything as an optional field, and then you’re back to checking for null or undefined everywhere, and why isn’t this already a solved problem?

And it turns out, it is a solved problem – or at least, it is in Elm, via those JSON Decoders. I don’t get to write Elm at work right now (or any time in the foreseeable future) – but if I can’t write Elm, I can at least try to steal a bunch of its great ideas and push them back into my TypeScript.

So… what exactly are JSON Decoders and how would they have solved this problem? (And why, if you’re already familiar a little with Elm and possibly feeling frustrated with decoding, are they actually worth it?)

A JSON Decoder is just a way of guaranteeing that once you’re inside the boundary of your program, you always have a valid instance of the data type you’ve decoded it into, or an error which tells you why you don’t have a valid instance of the data. They’re composable, so you can stack them together and take smaller decoders to build bigger ones, so if you have a complex JSON structure, you can define repeated substructures in it, or decoders for dissimilar sibling items in it, and use them to put together a grand decoder for your whole final structure. The decoders use the Result type, and they hand back either Ok with the decoded value or Err with the reason for the failure – and if any piece of a decoded type doesn’t match with what you’ve specified, you’ll end up with an Err.

Now, initially that might sound like a recipe for disaster – JSON payloads can be formed in weird ways all the time! – but in fact it encourages you to think through the various ways your payloads can be formed and to account for them. Sometimes, if the payload doesn’t have what you expect, that really does mean something is wrong either in your request or in the server-side implementation. In that case, getting an Err is exactly what you want. Other times, the server might be perfectly legitimate in sending back a variety of shapes in its response, and your responsibility is to decide how to decode it to make sense in your app. Remember, the problem I had was that I received a payload which didn’t have the data. With Elm’s decoders, I would have had three choices:

I could have treated this as an error, and passed that along to be dealt with in some way.
I could have normalized it as a 0-value payload.
I could have treated it explicitly as a no-op, maintaining whatever previous state I had in the data store, i.e. the implicit behavior of my actual data store.

What I couldn’t do, though, is do any one of those accidentally. I could still support incomplete payloads (via option 3), but I’d be explicitly opting into that, and there would be an obvious place where that was the case. This would be particularly helpful in a scenario where I wasn’t also in charge of the API: if I couldn’t just go change it so the API itself had a more sensible behavior, I could enforce whichever desired behavior on my own end. More than that, with something modeled on the Elm JSON Decoders, I would have to: there would be no implicit consumption of raw JSON.

The first time I played with the Elm JSON Decoder approach, I thought it was a lot of work. I was used to just doing JSON.parse() in JS or json.loads() in Python. Now I needed to define a whole series of decode steps explicitly for every field in a response? Good grief! But it grew on me. More than that, I now actively miss it in my apps; I’d have been really happy not to have to spend a morning hunting down this particular bug.

Sometimes that explicitness can seem like quite a lot of boilerplate, and indeed it is: there’s a reason the Elm elm-decode-pipeline project exists. But even given the initial nicety of something like F♯ type providers, I think the Elm approach has a slight edge in the long-term for maintainability specifically. It’s one thing to be able to just get to work right away and have a type definition you know to conform to a given API response. It’s something else entirely to be able to know that you’ve accounted for all the varieties of responses you might get (and without throwing an exception for failed JSON decoding at that!).

Given all of this, I’ve started mentally teasing out what such a JSON decoding library for Ember.js might look like in TypeScript. It’s a long way off, but it’s the kind of thing that I really want to experiment with, and that I think would make for a big win for the maintainability of our apps. Keep your eyes peeled, because I suspect this is another thing JS will steal from Elm, and that’s great in my book.

Chrome is Not the Standard

2017-12-21T07:10:00-05:00

This got an enormous amount of play around the web, and as a result people have ended up translating it to other languages. If you have a translation, I’ll be happy to link it here!

Russian, translated by Vlad Brown (http://softdroid.net)
Uzbek, translated by Alisher
French, translated by Jeremy Lee Shields (http://jeremylee.sh)

The post

Over the past few years, I’ve increasingly seen articles with headlines that run something like, “New Feature Coming To the Web”—followed by content which described how Chrome had implemented an experimental new feature. “You’ll be able to use this soon!” has been the promise.

The reality is a bit more complicated. Sometimes, ideas the Chrome team pioneers make their way out to the rest of the browsers and become tools we can all use. Sometimes… they get shelved because none of the other browsers decide to implement them.

Many times, when this latter tack happens, developers grouse about the other browser makers who are “holding the web back.” But there is a fundamental problem in this way of looking at things: Chrome isn’t the standard. The fact that Chrome proposes something, and even the fact that a bunch of developers like it, does not a standard make. Nor does it impose an obligation to other browsers to prioritize it, or even to ship it.

As web developers, it can be easy to become focused on interesting new features for the platform we work on. That’s no different than the excitement Android and iOS developers have when Google and Apple release new SDKs for developing on their platforms. It’s healthy to be excited about possible new features, things that might make our jobs easier or enable us to do things we couldn’t do before.

But there is an important difference between those platforms and the web. Those platforms are the domain of a single vendor. The web is a shared platform. This is its unique benefit, and its unique cost. It uniquely allows us to write software that can actually run, and run reasonably well, everywhere. But it also means that a minimum of four companies—the major browser vendors—get a say in whether a feature is a feature or whether it’s just an interesting idea one of the teams had.

Let’s get concrete about an example that’s been extremely high-profile for the last couple years—and, to be clear, one I think is a good idea from Google: progressive web apps (hereafter PWA). They have been pitched by Google and other supporters as an unambiguous win for the user experience of complex web applications. And, as a web developer myself, I’m actually inclined to agree with that assessment! However, I have fairly regularly seen people getting angry at especially Apple for not prioritizing support for PWAs in (especially iOS) Safari—Apple is, in this view, “holding back the future of the web.”

Well… no. For any given idea Google pitches, Apple may or may not be sold on Google’s vision of the web, or they may even think it’s a good idea but not more important than other things they’re working on.¹

And this is what it means to be part of the web platform. No single company gets to dominate the others in terms of setting the agenda for the web. Not Firefox, with its development and advocacy of WebAssembly, dear to my heart though that is. Not Microsoft and the IE/Edge team, with its proposal of the CSS grid spec in 2011, sad though I am that it languished for as long as it did. Not Apple, with its pitch for concurrent JavaScript. And not—however good its developer relations team is—Chrome, with any of the many ideas it’s constantly trying out, including PWAs.

It’s also worth recognizing how these decisions aren’t, in almost any case, unalloyed pushes for “the future of the web.” They reflect business priorities, just like any other technical prioritization. Google cares about PWAs because Google makes its money from the web and wants people to spend more of their time on the web. Apple cares about things like the battery life implications and the sheer speed of its iOS JavaScript engine because it makes money from hardware and it wants people to be happy with their iPhones and iPads.

Does any one of those browser’s commitments map cleanly to all users’ (or even all developers’) priorities? Of course not! This is and always has been the beauty of a competitive browser landscape. I’m a web developer who wants PWA support everywhere—so I want Apple supporting it. But I’m also a smartphone user who wants those applications to scream on my device—not to crawl, like they do on Chrome on Android, which is still years behind iOS in performance. As an end user, not just a developer, it matters to me that running Safari on my laptop instead of Chrome can dramatically increase my battery life.

These are tradeoffs, plain and simple. Chrome ships new features fast, but they’re not always stable and they often have performance costs. Safari ships new features on a much slower cadence, but they’re usually solid and always perform incredibly well. These are both engineering and business tradeoffs, and the companies behind the browsers are making because of their own business and engineering priorities. Don’t valorize any of the browser vendors, and don’t act as if any of them is the standard, or a reliable predictor of the future. Instead, value what each brings to the table, but also value the interplay at the table, and the ways each of these vendors pushes the others and challenges the others’ assumptions of what is most important. That’s what makes the web so great, even when it makes things move more slowly. Sometimes—often, even!—moving more slowly not in the experimental phase but in the finalizing phase makes for a much better outcome overall.

In this case, it seems to have been the latter, since yesterday’s release of Safari Tech Preview enabled Service Workers, one of the major pieces of the PWA push.↩

Vexing Ironies

2017-12-17T20:50:00-05:00

One of the most vexing problems in thinking and responding well to problems of ethics in information technology specifically is the way that so much of thinking and responding about information technology takes place within the context of, well, information technology. Two examples of that challenge caught my attention this evening, ironies I noticed in reading the very same article.

In “How smartphones hijack our minds”, Nick Carr explores much of the evidence for ways that use of smartphones can have seriously negative effects on our thinking in ways that are both pernicious (because we usually do not notice them consciously) and pervasive (in that they happen simply by dint of the presence of the devices). It’s a good article, and I commend it you as a helpful summary of a lot of the most current research on attention, smartphones, and the like; you should read it and think about how you use your phone.¹

But the first irony was that I read that article… on my smartphone. As indeed I read many of Nick Carr’s articles. There’s something more than a little odd about considering the use of a smartphone by way of reading an article on a smartphone. But in a very real sense, there’s almost no way I could have read Carr’s article otherwise.

And accordingly, the second irony is that, although Carr, who has staked out a position as a popular-level writer tackling issues of how modern information technology affects us, certainly is published in hard copy, everything of his I’ve ever read has been in digital form. Indeed, although his books are important in their own ways, I think it’s fair to say that most people’s interaction with his ideas, including his critiques of the ways we use and indeed rely on the internet, have all happened via and only because of the internet.

I’m not entirely sure what to make of these observations. I don’t fault Carr for publishing a blog, exactly; and though I am increasingly chastened about my own at-times-unwise use of a smartphone, I don’t fault myself for having an RSS reader there. (Better than that a Twitter app, to be sure!) But there is something at a minimum odd and perhaps even something off about the ways that we tend to use the very tools we are critiquing as the medium for advancing our critiques. We implicate ourselves.

But what is the alternative? On the one hand Carr’s message—which is important!—has likely been heard and even internalized by a far broader audience because he has transmitted it digitally than it would have been had he conscientiously limited it to books (and perhaps print media articles). The efficacy of the medium for distribution is the internet’s greatest strength. Likewise, I would never have run into Carr’s writing in the first place apart from articles in my RSS feed which linked it; and I do a great deal of my RSS feed reading on my iPhone and iPad, both of which are much better reading environments than a laptop or a desktop computer. Indeed, much of what I find most helpful in my reading on technology and ethics I find my way to via articles in my RSS feed, and I often items for reading later by simply tapping a an interesting-looking link in an article I’m reading in that RSS feed.

What do we make of this tension, these ironies? Especially when our concerns begin to rise to the level not merely of prudential judgments (though that level alone is perhaps sufficient reason to do more than we let ourselves) but deep ethical worries—do we abandon the smartphone altogether, cease blogging for fear of how it only contributes to the Google-ified and Facebook-ified age we live in?

I don’t know.

One of my concerns in my ongoing project is to prompt people around me—friends and family, but perhaps also blog readers!—to consider how our use of technology forms us. More on that in a future post.↩

Becoming a Contributor

2017-11-02T07:00:00-04:00

Here is the full text of the talk I gave at Rust Belt Rust, as it was prepared; headings correspond to individual slides. You can see the slides as they were presented here. Note that I extemporize fairly freely when actually giving a talk, so this is not a word-for-word equivalent of the talk as delivered, but the gist is the same!

I’ll update this post with the video once it’s available!

family

Hello, everyone! It’s good to see all of you. We only have half an hour, and even if that’s ten to fifteen minutes longer than a normal New Rustacean episode, that’s still not much time, so let’s jump right in! Our theme is “Becoming a Contributor.” There are two prongs to this talk, two big ideas I hope you all walk away with.

Introduction: The Big Ideas

The first thing I hope all of you take away is that there is no reason you cannot contribute meaningfully to the success of Rust – or indeed any open-source project you care about. Anyone can be a contributor. And not “even you” but perhaps “especially you”. The fact that you’re an outsider, or new to programming, or new to systems programming: sometimes that makes you a better contributor. Because you don’t necessarily share the biases of – you’re not wearing the same blinders that – someone who’s been writing systems-level code for 20 years have. So the first idea: you can contribute.

The second idea I hope you take away is just how many ways there are to contribute meaningfully. It has almost become a cliche in the Rust community to say “code isn’t the only thing that matters,” but I want to show you today just how true that is. And I want to make that point again more forcefully, because for all that we often say that, the idea that shipping code is what really matters is the kind of pernicious lie that can come back and bite any of us. It certainly gets to me at times! But it’s a lie, and we’re going to see that in detail. That’s the second big idea: there are an astounding number of ways you can contribute.

Introduction: Why?

There are a lot of things to be passionate about in the world of software development. But at the end of the day, I care about software because I care about people. To borrow a label from Scott Wlaschin – a developer I admire enormously, mostly working over in the F# community – I am a humanist, not a technologist. The technologies are interesting in themselves to a degree; but I mostly care about the ways that technologies can help us serve people more effectively. As software developers, that takes a lot of shapes. But today I want to zoom in on just these two ideas about open-source software:

Introduction: The Big Ideas

So: why these two ideas? For one thing, because I think they are among the most applicable to everyone here. We have an enormous open-source focus. But for another, because they can also serve as windows into the ways we can – and should – think about software more generally. So: let’s talk about how you become a contributor.

Introduction: Outline

We’re going to take this on in the good old grammar-school fashion: who, what, when, where, why, and how. We’re not going to take them in that order though, and we might smash a few of them together.

Introduction
Why bother contributing?
Who is a contributor?
What is a contribution? How can you contribute?
- …so many things they won’t fit on this slide.
When and where to contribute?
Conclusion

Why bother contributing?

The first question we might be asking is: why contribute at all? Why should you be interested in becoming a contributor? And the best answer I can offer is: because there is more work than hands to do it. Always. Every open-source maintainer can tell you the truth of this.

Who is a contributor?

People define this differently, but I have a very simple definition: A contributor is anyone who improves a project.

Who is a contributor? Examples

For example:

submit a patch to fix a typo
add a small correction for a code sample in a project
file an issue instead of just suffering through a problem in silence
everything else we’re going to talk about today

Who is a contributor? Me!

That might sound overblown, but it’s really not. I am literally standing on this stage in front of you today because I submitted some small typo and code sample improvements to “Rust by Example” a few years ago, and realized: I can make a difference in this community. And that gave me the motivation I needed to keep contributing.

Who is a contributor?

I don’t imagine the story is all that different for most people who are open-source contributors in this room. Something got them over the hump, and it was probably something small, insignificant-seeming at the time. They might be particularly skilled in this thing or that thing, but in fact a lot of them are in those roles just because they saw a need and stepped up to fill it. And then kept at it for a long time. But it made them a contributor. And that feeling – of helping build something bigger than you can build on your own – is a good one. I’d go so far as to say it’s part of what humans are meant for. It’s part of us in a deep, deep way.

Who is a contributor?

If you’re inclined to quibble with that definition, I challenge you to ask why? I think, most often, it’s because we feel defensive about wanting to project our own particular kinds of contribution as the most important, or the most valuable. But I’m more of the mindset that, as I read recently, “anyone who would be first… must be last of all, and servant of all.” We should stop worrying about our own prestige and turf-marking, and start rejoicing in the many different ways people are able to make our projects better.

There’s no magic that makes you qualified to be a contributor. There’s just a willingness to serve where you see a need.

What & how can you contribute?

And that takes us into the “what” of all of this, the how. (Yes, I’m combining those two). What is a contribution? How can you contribute? Turns out, this is a long list.

What & how: code

Let’s get this right out of the way up front, because it’s the most obvious: you can write code. You can fix bugs or help implement new features. You can do that even if you’re not an expert – especially in the Rust community. Many Rust projects have gone out of their way to mark issues as good-first-issues, or easy-to-tackle, or mentorship-available. Maybe it’s your first contribution to an open-source project: that’s okay. You can take a stab at it, and the fact that it might not be good is okay. The whole point of these kinds of issues is that they give you a place where you can jump in safely.

That goes equally for everything from the Rust compiler itself to many of the other projects in the ecosystem. Look at the repository, for example! And it’s not just this project. Lots of projects in the Rust ecosystem are like this.

What & how: code – we’re kind here

And no one is going to swear at you or insult for making a mistake here. Not even if you’re working on something important, and not even if you’ve been doing it for a while. That is not. how. we. roll. here. Everyone makes mistakes!

Instead, we want people to show up, knowing nothing: we’re happy to help. Remember: we want people to contribute! So: try opening a PR and let people help you learn how to do it well! In fact, if you haven’t ever opened a PR on a Rust project, find one that looks interesting to you and has an issue tagged that way, and submit a PR before the weekend is out! You can do it!

What & how: code – a caveat

But code is not the only thing that makes you a contributor. I put it up front because I think it’s worth doing – but I also wanted to get it out of the way. In every software community, it’s easy to over-value the code. That might sound crazy, given that it’s open-source software, but the reality is that no one fails to value the code. We do often fail to value all the other things that make an open-source software project actually useful. It’s certainly true that there’s no project without the code. But it’s also the case that there’s no useful software without a lot of other things besides the code, and we often undervalue those.

Filing bugs

So let’s take one step away from code, and talk about what is probably the single easiest way anyone can contribute. File issues. If you’re using a binary and it doesn’t work, open a ticket. If you’re integrating a library and it seems like the API doesn’t do what it should, or if it seems like it’s missing some functionality… well, you can suffer in silence, or you can open a bug ticket! Many times, the author of the software doesn’t know there’s a problem. The only way they can fix it is if they know about it!

filing bugs

Docs

Perhaps the thing most of you will be most persuaded of the utility of is documentation. All of us have faced the difficulty of trying to figure out how to integrate some poorly-documented (or undocumented!) library into our own codebase. That experience, in word, sucks.

So working on documentation is one of the highest-value areas you can contribute to any project. It’s also really hard, in a bunch of ways – harder, in some ways, than writing the code is!

Docs: who?

One kind of documentation is explanation of how things work under the hood. The implementer is the most qualified there! That doesn’t mean they don’t still need help even with that, though! Some people are incredible implementors and terrible explainers; you can often do a great service by serving as an “interpreter” for them – taking their explanations and making the literary tweaks and cleanups and polish that they need.

Another kind of documentation, though, developers and maintainers are often really poorly equipped to write, and that’s introductory documentation. This is the problem of expertise: when you know exactly how something is meant to work, and especially when you’re the one who implemented it, there are things that seem obvious to you which simply aren’t obvious to someone approaching it for the first time. And as hard as you try, you can’t escape that entirely. You can imagine what it might be like not to know something, but there’s no substitute for actually not knowing something.

Docs – how?

What that means is that one of the most valuable things you can do as you learn a new library is write down the things you don’t understand from the docs as you go. And when you figure them out, write that down, too. If nothing else, writing up that experience – filing it as an issue on the bug tracker, or otherwise getting it in the hands of the maintainers – can help them make important changes to things like the order various concepts are introduced, or adding little notes to help people feel comfortable with not knowing something until it can be introduced later, and other things like that. It can help them recognize and fill in gaps in their docs – things they simply assumed but which they didn’t realize they were assuming – and fill those in. At the most extreme, you might even help them realize that some parts of the docs need full rewrites… and the work you’ve done in writing things down might just be the foundation or the actual content of those new docs.

Write down the things you don’t understand from the docs as you go.
When you figure them out, write that down, too.
Then: file an issue or write a PR to improve it!

Docs: varieties

So what kinds of things would we call documentation?

API documentation
READMEs
Tutorials
Books
The Rust Reference

Okay, books are a huge undertaking, but they can genuinely serve as documentation. Especially for large projects. In fact, several of the most important pieces of “documentation” the Rust project itself has are books: “The Rust Programming Language”, “Rust by Example”, and “The Rustonomicon”. But there are also important but totally unofficial books like Daniel Keep’s “A Practical Intro to Macros in Rust 1.0” and “The Little Book of Rust Macros”, or Jorge Aparicio’s book on microcontrollers with Rust.

The Rust Reference: This is a special category, and one that’s especially important to me. The Rust Reference is supposed to be an exhaustive guide to the language, and the value of that being complete and accurate is hard to overstate. It’s also wildly out of date today. I wrote an RFC last year that said, basically, “We need to actually document everything! That includes updating the Reference!” The trick is: it’s a huge undertaking, and while I and a few others made a good start on it earlier this year, that effort got bogged down by life, and it needs to be resuscitated. And it’s not just Rust which could use investment in that area. Other languages and frameworks have the same issue. It’s really important that there be an answer other than “dive into the source and try to figure out what its intent is” – the more central the component is in the ecosystem, the more important that is.

Docs: Translation

Another huge place you can contribute to documentation is translation. For good or ill, English has become the sort of primary language of programming, but that doesn’t mean we should treat it as the only language, or as more important than other languages. Translating documentation is amazing and very needed work, and it’s work that not everyone is really capable of! I’m fluent in English and… ancient Hebrew and ancient Greek. For some reason, there’s not much demand for technical writing in Greek from the era when Plato was alive. So I’m not much use at translation.

translation

But many of you out there are multilingual, and could take docs written in English and convert them for, say, Czech-speaking developers. Perhaps just as importantly, you can go the other direction, and help non-English-speaking maintainers reach a broader audience. Take an amazing project which only has documentation in Amharic (because its developers don’t feel comfortable enough in English to translate it themselves) and translate it to English: use the fact that English is the common language to increase the reach of non-Western developers!

Visual Design

One of the areas where you could move the ball down the field fastest in the Rust community is with visual design. (To be clear, the language design is great!) But our websites could sometimes use some work.

Visual design: it’s not just us

Systems programming language types have historically not spent a lot of time on the presentation of their tools. In part this is just a matter of what these kinds of languages have been oriented towards: if you spend all day hacking on kernel code, you’re likelier to be a person for whom user interface and visual design is less interesting than, say, optimizing memory performance or minimizing the number of cache misses a given approach has. But presentation does matter, and it matters especially as we want to enable more and more people to be able to write this kind of code.

Speaking frankly, though I’ve spent a large chunk of my career to date writing systems-level languages, I’ve found the way a lot of these tools are presented to be a huge turn-off, and at times a barrier even to getting them working for me locally. Perhaps the most egregious example of that was some of the “documentation” – I’m not sure I should even call it that! – for Fortran, when I was first getting started programming back in college. The presentation of the material was essentially hacker-ish in a bad way: no CSS, no attention to organization of the material, no structure to help you find your way through it.

Visual design: how

If you’re an expert or just a talented amateur, please pitch in

You can help here even if you’re not especially comfortable with visual design or even if you’re outright bad at it if you’re willing to spend just a little time on it! For example, you can simply help a team adopt something like Bootstrap. Yes, it’ll look like many other open-source projects out there. But it won’t be horribly, catastrophically ugly and unreadable! Or you can do use one of these simple starter kits:

So don’t think that just because you aren’t a design expert means you can’t help here.

Just as important as the visual design is thinking about and actively designing the information hierarchy of your content. What leads to what? Which pieces go together, and which pieces can be broken up into their own pages or sections within pages? Think about the content like an outline. Many sites don’t have any such structure to them; they’re kind of haphazardly cobbled together. If you can help the maintainers with the structure and organization of their content, that can make an enormous differences as well.

Blogging

One of the other big ways you can help a project may not even end up in the repository at all. You can blog.

I know blogging can seem intimidating, for many of the same reasons that writing documentation can. Technical writing is hard, and it’s a completely different skill from programming. But it doesn’t have to be amazing; it just has to get the information out there – and you’ll get better as you practice.

Blogging: “Easy Mode”

You can start on “easy mode”, too. I mentioned this earlier when talking about documentation, but “just write down what you’re learning” is an incredibly effective technique for generating content. If you look at a lot of the technical blogging I’ve done over the years, it has been nothing more complicated than “here is what I just learned.” And if you want a superb example of this which is very different from mine, take a look at the work that Julia Evans does on her blog! She regularly writes down, in an inimitable way, highly technical ideas she’s just learning. If you want someone to make arcane Linux command line tools seem amazing and approachable, her blog is your ticket.

Just write down what you’re learning.
—Me, just now

Blogging: good examples

But even beyond “what I just learned,” blogging is a superb way for teaching in general. Over the course of this year, for example, Vaidehi Joshi has been writing what is essentially a friendly introduction to computer science on her blog on Medium. This is a totally different style of content (as well as of presentation!) from the kind of “what I just learned” content that Julia Evans writes,but it’s also really effective, because she takes her knowledge and translates it into something others can pick up. That’s obviously more work than just writing down things you just learned, but it can also pay really high dividends as others are able to substantially deepen their knowledge.

Blogging: all the options!

In blogging, as in documentation, there is a whole spectrum of basic teaching content you can contribute! And communities need the whole spectrum for simple introductions to extremely thorough, advanced tutorials.

But blog posts can also be much more versatile than traditional documentation.

They can be one-offs, or series. You can give a topic as much depth, or as little depth, as you care about or think it deserves. I wrote an 18-part series comparing Rust and Swift, and it could have been 30 parts if I hadn’t eventually gotten derailed. That’s not documentation, but there’s a lot people can learn from those kinds of things.
They can introduce a technology, or dig deep into how to use it, or how it’s built. You’re not limited to just one particular tack when blogging. Is your interest in the specific implementation details of some corner of the compiler? Write about that! Is your interest in how a given Rust library solves a specific kind of problem you’ve run into with another library, or with a similar library in another language? Write about that! You get the idea.
They can critique or highlight problems with specific pieces of the ecosystem! A careful, well-articulated, critical blog post can do wonders for showing the problems with a given approach and can even sometimes help suggest the right solutions to those problems. I’ve repeatedly watched, for example, as people have blogged about their struggles getting their heads around the Tokio tooling; the result has been a lot of work by the Tokio team to respond to those problems. The more thoughtful and careful you are in that kind of criticism, the better! Good criticism is incredibly valuable. Because we all have blind spots, and someone else’s perspective can help jar us out of those.
They can show how to integrate different parts of the ecosystem. For example, as part of the “Increasing Rust’s Reach” initiative, Ryan Blecher recently wrote up a detailed walk-through on how to use the Diesel ORM and the Rocket web framework together to build a small blogging engine. That’s huge! It makes it that much easier for someone who’s just starting out with Rust, coming in from something like Python or Ruby, to dive in and get that intensely rewarding feeling of having built something in a relatively small amount of time. That’s also helpful because (almost) no one is building something with just Diesel, or just any crate. A huge part of what every software developer does is about fitting together other pieces of software.
They can invite feedback on your own projects. Talk about what you’re doing, what your stumbling blocks are, what you don’t understand. People will often show up and help you with comments and clarifications!

And that’s just scratching the surface. Blogs are incredibly versatile, and you should lean on that.

Audio and Video

Not just words! Noises and pictures, too!

Audio: podcasts

Not everyone learns the same way.
Lots of people have commutes.

Audio: but there are already podcasts

Everything I’ve talked about so far has been in written form. But audio and video media can also be really helpful. Not everyone learns best by reading. And not everyone has tons of time to sit down and read a book every day. One of the reasons I started the New Rustacean podcast is that it gives people a way to get up to speed on the language while on a daily commute. But there’s still a huge need for more audio and video content in this space!

One podcast is not enough!

New Rustacean

Two podcasts is not enough!

Request for Explanation

Seriously, not even three podcasts is enough!

Rusty Spike

So I’m laying down another challenge: there’s plenty of room for more, and more kinds, of audio content in this ecosystem.

Video

Again: people have different learning styles!

There’s also a huge opening for people to produce good video content. I’ve heard often from people that things like RailsCasts were essential in helping them learn the Ruby on Rails ecosystem. We need video tutorials which might look kind of like that, or like the kinds of things I’m doing on the podcast. If you have any skill that way, and any interest in teaching, you should make Rust videos – there aren’t many out there.

Video: what

There are lots of options here—not just live streaming!

Another, totally different tack you can take with video is live-streaming. Sean Griffin has done this at times, and I’ve actually done it just once, and it’s a ton of fun – and it can be incredibly illuminating for other people to see how you work and how you solve problems. You can also do like I did and live-pair on something. It’s a pain to set up, but it’s also a lot of fun.

And no doubt there are more ideas you have—please just go do them!

Talk to people

Just talking with people matters. And there are lots of places to do it:

IRC/Gitter/Slack/Discourse
Meetups
Conferences

You can also host or help with a local meet-up! For a lot of people, one of the major challenges of learning any new piece of technology is that – even with IRC and Gitter and Slack and so on – you can feel isolated and alone. And people can help you solve problems in person, and make you feel supported in person, in ways that even a great community can’t really manage online. So go to meet-ups, at a minimum. And help the organizers. And if there isn’t a meet-up in your community… you can start one! The #rust-community team has a ton of resources.

Physicality matters. Presence matters. (We know this! We’re at a conference!)

Being inviting

Last but not least in this list of how to be a contributor, I want to take a minute and talk about “being a contributor” to those of you who’ve been contributors for a long time. Some of you have been shipping open-source software for years – some of you even for decades. Much of what I’ve said so far is old hat for you. Maybe not the design bits quite so much! But you’ve been doing this for a long time, and you’re not trying to get over the hump of making your first contribution. You have other things to contribute here:

The most important thing you can do is practice welcome people. The Rust community does this well, in general, but it’s something we need to keep in front of us as a goal as the community grows. It’s easy to get frustrated with newcomers as your project grows, demands on your time increase, and your work as a maintainer seems less like fun and more like a second job. But continuing to actively welcome newcomers in is incredibly powerful. You can make it possible for people to go from zero to really making a difference. And remember: so once were you. None of us started out as magical wizards of Rust and open-source.
The second big thing you can do is mentoring. As I mentioned, I’m now the maintainer of one of the core pieces necessary to make Ember.js and TypeScript play nicely together. But while I’ve done some writing-of-code with that, a much larger part of my current and future work there is about helping other people learn TypeScript well enough to start using it in their apps and add-ons. But the flip-side of that is: even a fair bit of the code I have written, I was able to write because someone more comfortable with some of the infrastructure mentored me through its quirks and oddities.

When & where to contribute

The last thing I want to touch on is when and where to contribute. There are two things I’d suggest you should consider here:

When & where: you

Where are you in the process of becoming comfortable with contributing?

Just getting started?
Already comfortable?

If you’ve never done any open-source work at all before, that’s different than if you’ve gotten pretty comfortable with it in a different ecosystem and are just figuring out where to make yourself useful in this ecosystem.

When & where: if you’re just getting started

If you’re just getting started, I’d pick a big project with lots of those “Help Wanted” and “Mentoring” and “Easy” tags on issues, and let the size of the project help you out. Those are projects that are used to helping people make their first contributions. Crazy as it seems, something like Servo can actually be an easier place to start out that a much smaller project. Sure, the technical lift is higher, but there are also a lot more people actively invested in your success there.

Look for these!
Pick big projects!

When & where: if you’re experienced

On the other hand, if you’re already comfortable contributing and have some idea what you’re best at, you might look around and find smaller projects with fewer contributors which look interesting and could use the help. Because again, there’s always more work to do than hands to do it.

When & where: project lifecycles

The second consideration dovetails nicely with that: where is a given project at in its life-cycle? As enthusiastic as you might be about some project, if it’s a small project and it’s already in a “basically done” state, well… that’s probably a lot less useful a place to invest your time if you’re focusing on code. On the other hand, it’s often the case that projects are “done” in terms of code, but desperately need help with documentation, their web site, etc. Big projects, or projects just starting out, are often better places to dig in if you’re really looking to flex your coding muscles (but both of them also usually have huge needs in terms of all those non-code avenues we talked about).

Where is a given project at in its life-cycle?

small project, basically done?
need docs?
big project, a billion needs?
etc.

Think about those, and then see if you can pick a project that’s a good fit for your current skillset and comfort level and jump in!

Conclusion

And that’s a good place to wrap things up! I hope you’re feeling like you can do this. Because you can. Open-source a project of your own and see where it goes. Write a blog post. Add some docs. Open a PR. Record a podcast. Make some videos. Start a meet up. Become a contributor! And remember:

Anyone can contribute meaningfully.
People can contribute in a stunning variety of ways.

More info

https://www.rust-lang.org/en-US/contribute.html
https://blog.rust-lang.org/2017/09/18-impl-future-for-rust.html
https://internals.rust-lang.org/
#rust, #rust-community, #rust-internals, etc. on irc.mozilla.org

Announcing True Myth 1.0

2017-11-01T08:40:00-04:00

I’m pleased to announce the release of True Myth 1.0! True Myth is a library I’ve been working on over the last month or so, for saner programming in JavaScript, with first-class support for TypeScript (and Flow).

True Myth provides standard, type-safe wrappers and helper functions to help you with two extremely common cases in programming:

not having a value—which it solves with a Maybe type and associated helper functions and methods
having a result where you need to deal with either success or failure—which it solves with a Result type and associated helper functions and methods

You could implement all of these yourself – it’s not hard! – but it’s much easier to just have one extremely well-tested library you can use everywhere to solve this problem once and for all.

Even better to get one of these with no runtime overhead for using it other than the very small cost of some little container objects—which we get by leaning hard on the type systems in TypeScript or Flow!

Aside: If you’re familiar with Folktale or Sanctuary, this has a lot in common with them—its main differences are:

True Myth has a much smaller API surface than they do
True Myth aims to be much more approachable for people who aren’t already super familiar with functional programming concepts and jargon
True Myth does no runtime checking of your types, whereas both those libraries do by default—it relies on TypeScript or Flow instead

I really like both of those libraries, though, so you might check them out as well!

`Maybe`

Sometimes you don’t have a value. In JavaScript land, we usually represent that with either null or undefined, and then trying to program defensively in the places we think we might get null or undefined as arguments to our functions. For example, imagine an endpoint which returns a JSON payload shaped like this:

{
  "hopefullyAString": "Hello!"
}

But sometimes it might come over like this:

{
  "hopefullyAString": null
}

Or even like this:

{}

Assume we were doing something simple, like logging the length of whatever string was there or logging a default value if it was absent. In normal JavaScript we’d write something like this:

function logThatValue(thePayload) {
  const length = !!thePayload.hopefullyAString
    ? thePayload.hopefullyAString.length
    : 0;
  
  console.log(length);
}

fetch(someUrl)
  .then(response => response.json())
  .then(logThatValue);

This isn’t a big deal right here… but—and this is a big deal—we have to remember to do this everywhere we interact with this payload. hopefullyAString can always be undefined or null everywhere we interact with it, anywhere in our program. 😬

Maybe is our escape hatch. If, instead of just naively interacting with the payload, we do a very small amount of work up front to normalize the data and use a Maybe instead of passing around null or undefined values, we can operate safely on the data throughout our application. If we have something, we get an instance called Just—as in, “What’s in this field? Just a string” or “Just the string ‘hello’”. If there’s nothing there, we have an instance called Nothing. Just is a wrapper type that holds the actual value in it. Nothing is a wrapper type which has no value in it. But both of them are concrete types and you’ll never get an undefined is not an object error when trying to use them!

Both of them have all the same methods available on them, and the same static functions to work on them. And, importantly, you can do a bunch of neat things with a Maybe instance without checking whether it’s a Nothing or a Just. For example, if you want to double a number if it’s present and do nothing if it isn’t, you can use the Maybe.map function:

import Maybe from 'true-myth/maybe';
const hereIsANumber = Maybe.just(12);          // Just(12)
const noNumberHere = Maybe.nothing<number>();  // Nothing

const double = (n: number) => n * 2;
hereIsANumber.map(double);  // Just(24)
noNumberHere.map(double);   // Nothing

There are a lot of those helper functions and methods! Just about any way you would need to interact with a Maybe is there.

So now that we have a little idea what Maybe is for and how to use it, here’s that same example, but rewritten to normalize the payload using a Maybe instance. We’re using TypeScript, so we will get a compiler error if we don’t handle any of these cases right—or if we try to use the value at hopefullyAString directly after we’ve normalized it!

(Note that Maybe.of will construct either a Maybe.Just if the string is present, or Maybe.Nothing if the value supplied to it is null or undefined.)

import Maybe from 'true-myth/maybe';

type Payload = { hopefullyAString?: string };
type NormalizedPayload = { hopefullyAString: Maybe<string> };

function normalize(payload: Payload): NormalizedPayload {
  return {
    hopefullyAString: Maybe.of(payload.hopefullyAString)
  };
}

function logThatValue(payload: NormalizedPayload) {
  const length = payload.hopefullyAString.mapOr(0, s => s.length);
  console.log(length);
}

fetch(someUrl)
  .then(response => response.json())
  .then(normalize)
  .then(logThatValue);

Now, you might be thinking, Sure, but we could get the same effect by just supplying a default value when we deserialize the data. That’s true, you could! Here, for example, you could just normalize it to an empty string. And of course, if just supplying a default value at the API boundary is the right move, you can still do that. Maybe is another tool in your toolbox, not something you’re obligated to use everywhere you can.

However, sometimes there isn’t a single correct default value to use at the API boundary. You might need to handle that missing data in a variety of ways throughout your application. For example, what if you need to treat “no value” distinctly from “there’s a value present, and it’s an empty string”? That’s where Maybe comes in handy.

`Result`

Another common scenario we find ourselves in is dealing with operations which might fail. There are a couple patterns we often use to deal with this: callbacks and exceptions. There are major problems with both, especially around reusability and composability.

The callback pattern (as in e.g. Node) encourages a style where literally every function starts with the exact same code:

function getMeAValue(err, data) {
  if (err) {
    return handleErr(err);
  }
  
  // do whatever the *actual* point of the function is
}

There are two major problems with this:

It’s incredibly repetitive – the very opposite of “Don’t Repeat Yourself”. We wouldn’t do this with anything else in our codebase!
It puts the error-handling right up front and not in a good way. While we want to have a failure case in mind when designing the behavior of our functions, it’s not usually the point of most functions – things like handleErr in the above example being the exception and not the rule. The actual meat of the function is always after the error handling.

But if we’re not using some similar kind of callback pattern, we usually resort to exceptions. But exceptions are unpredictable: you can’t know whether a given function invocation is going to throw an exception until runtime as someone calling the function. No big deal if it’s a small application and one person wrote all the code, but with even a few thousand lines of code or two developers, it’s very easy to miss that. And then this happens:

// in one part of the codebase
function getMeAValue(url) {
  if (isMalformed(url)) {
    throw new Error(`The url `${url}` is malformed!`);
  }
  
  // do something else to load data from the URL
  return data;
}

function render(toRender) {
  // if toRender can't generate valid HTML, throw Error("invalid HTML");
  // if it can, theRenderedHTML;
}

function setDom(html) {
  /* magic to render into DOM */
}

// somewhere else in the codebase -- throws an exception
const badUrl = 'http:/www.google.com';  // missing a slash
const response = getMeAValue(badUrl);  // throws here

// we never get here, but it could throw too
const htmlForPage = render(value);

// so we definitely can't get here safely
setDom(htmlForPage);

Notice: there’s no way for the caller to know that the function will throw. Perhaps you’re very disciplined and write good docstrings for every function – and moreover, perhaps everyone’s editor shows it to them and they pay attention to that briefly-available popover. More likely, though, this exception throws at runtime and probably as a result of user-entered data – and then you’re chasing down the problem through error logs.

More, if you do want to account for the reality that any function anywhere in JavaScript might actually throw, you’re going to write something like this:

try {
  const badUrl = 'http:/www.google.com';  // missing a slash
  const response = getMeAValue(badUrl);  // throws here
  
  // we never get here, but it could throw too
  const htmlForPage = render(value);
  
  // so we definitely can't get here safely
  setDom(htmlForPage);
} catch (e) {
  handleErr(e);  // ends up here
}

This is like the Node example but even worse for repetition!

And TypeScript and Flow can’t help you here! They don’t have type signatures to say “This throws an exception!” (TypeScript’s never might come to mind, but it might mean lots of things, not just exception-throwing.)

Instead, we can use a Result to get us a container type, much like Maybe, to let us deal with this scenario. A Result is either an Ok wrapping around a value (like Just does) or an Err wrapping around some type defining what went wrong (not like Nothing, which has no contents). Both of them have the same sets of methods on them, and the same static functions which can operate on them.

import Result from 'true-myth/result';

type Payload = {/* details of the payload...*/}

function getMeAValue(url: string): Result<Payload, string> {
  if (isMalformed(url)) {
    return Result.err(`The url '${url}' is malformed`);
  }
  
  // do something else to load data from the url
  return Result.ok(data);
}

function render(toRender: string): Result<HTMLElement, string> {
  // if toRender can't generate valid HTML, return Err("invalid HTML");
  // if it can, return Ok(theRenderedHTML);
}

function setDom(html: HTMLElement) {
  
}

// somewhere else in the codebase -- no exception this time!
const badUrl = 'http:/www.google.com';  // missing a slash

// value = Err(The url '${http:/www.google.com}' is malformed)
const value = getMeAValue(badUrl);

// htmlForPage = the same error! or, if it was Ok, could be a different
// `Err` (because of how `andThen` works).
const htmlForPage = value.andThen(render);

// we can't just invoke `setDom` because it doesn't take a `Result`.
value.match({
  Ok: html => setDom(html);
  Err: reason => alert(`Something went seriously wrong here! ${reason}`);
})

When we have a Result instance, we can perform tons of operations on whether it’s Ok or Err, just as we could with Maybe.Just and Maybe.Nothing, until we need the value. Maybe that’s right away. Maybe we don’t need it until somewhere else deep in our application! Either way, we can deal with it easily enough, and have type safety throughout!

Conclusion

Give it a spin!

yarn add true-myth
npm install true-myth
You can even just ember install true-myth and use it if you’re using Ember (in which case I encourage you to also use ember-cli-typescript)

Let me know what you think – if there’s stuff missing, open issues! And if it’s just not to your taste, again, I encourage you to take a look at Folktale and Sanctuary, which are both excellent and land in very different design spaces in many ways.

Announcing ember-cli-typescript 1.0.0

2017-08-08T09:00:00-04:00

I’m extremely pleased to announce the release of ember-cli-typescript 1.0.0! You can get it the same way you do any Ember addon:

$ ember install ember-cli-typescript

For a detailed walkthrough of adding TypeScript to your projects, see:

So what are we shipping today, and what’s on the roadmap?

What’s In 1.0?

This release is intentionally relatively minimal: the goal here is provide stable foundation for building Ember.js applications with TypeScript in the toolchain. This means that in any app you can install the add-on and just start progressively converting your app over to TypeScript. However, we don’t expect to change the way you use the addon at all in the foreseeable future.

I’ll give you fair warning that there is one major challenge you will find as you work with ember-cli-typescript today: the lack of type definitions for most projects, and the limits of the existing type definitions for Ember.js itself. That’s not as bad as it sounds, though:

See the Roadmap below—we’re working on that, and you can help!
I’ve been using TypeScript successfully in the app I work on at my day job for the last nine months or so. While the lack of (good or any) typings has had its frustrations, TypeScript has already added a lot of value for us.

The Roadmap

We have a bunch of things we’re actively working on and which you can expect to land in the next few weeks to months.

1.1: A prepublish build process for addons
Community-driven work on typings

1.1: A prepublish build process for addons

The major priority for the 1.1 release is an npm prepublication step to generate JavaScript and typing files from add-ons which are using TypeScript. Currently, addons have to take TypeScript as a full dependency, not a dev dependency, because they currently just ship the .ts files up to npm and they have to be compiled in your app at build time.

We really don’t want to make any app developer who is using your addon download either the TypeScript files or especially the TypeScript compiler if we can avoid it. There are three reasons for this:

The fact that an add-on is developed in TypeScript really shouldn’t affect app developers. If they’re writing a plain-old JavaScript app, the fact that your addon is originally written in TypeScript is irrelevant to them.
TypeScript is large. The v2.4 installation I have in the app I’m working on right now weights 26MB. If I were using four add-ons which required TypeScript, my install cost could easily go up by a hundred megabytes. That’s not always a huge deal on a corporate network, but even where people do have good download speeds, it’s a hit to developer time. Every time someone has to reinstall all the dependencies, those 26MB have to come down again. If TypeScript becomes common, you might suddenly find yourself with addons using 2.4, 2.5, 2.6, etc.; it’s not hard to see that ballooning up the size of your installation in a really non-trivial way: 26MB × n versions of TypeScript = do not want.
The TypeScript compilation step takes time. Addons can do this once and save every consuming app build time. This isn’t the end of the world, but anything we can do to keep build times lower is a real win for developer productivity.

Accordingly the plan is to automatically add a build step which runs the TypeScript compiler on your addon and generates plain-old-JavaScript and the corresponding type definition files (.d.ts) prior to publishing to npm. That way, TypeScript can remain a dev dependency (rather than a full dependency) of each addon, and not be installed alongside the addon for consumers. Just-JavaScript consumers can just consume the normal JavaScript generated by the build. TypeScript consumers will get the full benefits of the types via the generated typing files.

This should hopefully land by late August or early September. Fingers crossed.

Community-driven work on typings

The process of getting type definitions in place for all of Ember.js and its ecosystem is way, way too big for any one person or even a small handful of people to manage alone. This is something we’re going to take on as a community.

New typings for Ember.js itself

We’re actively working on type definitions for Ember which will give us actually-useful-and-correct type checking for Ember’s custom object model. Today, if you use Ember.get or Ember.set, you get no help from the type system. When we finish, those will be type-checked by the compiler and will error if you try to assign the wrong values!

Importantly, the typings we’re shipping will be backwards compatible with the existing Ember API, but will also include support for the RFC #176 JavaScript Modules API. TypeScript’s module definition system will let us support both in parallel, and we will. Backwards compatibility and stability without stagnation are things we value for this addon just as much as the rest of the Ember.js ecosystem does.

This effort, led by Derek Wickern (@dwickern), is ongoing in the typed-ember/ember-typings repository. (If you’re wondering why we’re not just doing it in the DefinitelyTyped repository, see below.) We probably won’t be able to get to 100% of everything the Ember Object model does—Ember’s custom object model is incredibly sophisticated, and TypeScript actually still can’t totally express it—but Derek already has most of it working. This will be a huge step forward.

To be clear, we’re not forking the way you get types. We’ll upstream all of this work to DefinitelyTyped as soon as we have them working, but the DefinitelyTyped repo is huge and very busy; it’s not a great place to do this kind of substantial rework of existing types. And we really don’t need to have all the other type definitions DefinitelyTyped supplies in our way as we’re working, either. Having a separate repo gives us a place we can work on types, try them out as a community, etc. before creating PRs on DefinitelyTyped and publishing them officially.

Addon typings

We need to get type definitions in place for the addons in the ecosystem! That way when you’re using, say, ember-test-selectors, you’ll get an error if you try to use the functions it provides incorrectly. Right now, every addon out there is missing types entirely, so everything gets treated as taking the useless any type.

In a week or so, I’ll have a blog post with a fleshed-out quest issue for tackling it in detail, but here’s the short version: we’re going to try to get type definitions for all the top addons in the ecosystem so that it’s easy to use TypeScript in your Ember.js app. That blog post and quest issue will explain how to write good typings, and also how to contribute them to a project which may or may not be interested in using TypeScript itself.

Typing Your Ember, Part 4

2017-07-31T19:30:00-04:00

(See the rest of the series. →)

In the previous post in this series, I noted that one of the most effective current strategies for using TypeScript effectively in an Ember app is to push as much of your logic possible out of the Ember layer and into plain-old-TypeScript. Unsurprisingly, people had some questions about how to do this, so here’s a brief example.

As I suggested in that post, we now have a lib directory in our app, and all new business logic for the app lives there instead of directly on e.g. an Ember.Service instance. Our current directory structure looks like this:

app/
  adapters/
  components/
  config/
  controllers/
  helpers/
  initializers/
  instance-initializers
  lib/    <-- this is the one we care about
    billing/
    utilities/
      numeric.ts
  routes/
  serializers/
  services/
  templates/
  transforms
  app.ts
  router.ts
tests/
package.json
bower.json
// etc.

The main thing to notice here is that lib is just a directory in the app like any other, and its child directories likewise. This means that Ember CLI will resolve it just like normal, too—there’s no need to mess with the resolver or anything.

Say we had a set of numeric utilities in that numeric.ts file like this:

// Make text out of numbers, like "1st", "2nd", "3rd", etc.
export const withEnding = (val: number): string => {
  // boring implementation details elided
};

Then using it in an Ember component might look like this (where currentNumber is passed into the component):

import Component from '@ember/component';
import { get, set } from '@ember/object';
import * as Num from '../lib/utilities/numeric';

export default Component.extend({
  init() {
    const currentNumber = get(this, 'currentNumber');
    const displayNumber = Num.withEnding(currentNumber);
    set(this, 'displayNumber', displayNumber);
  },
})

You might wonder why we’d do this instead of using an Ember.Service. In the above example, I could of course make Num a service and inject it…

import Component from '@ember/component';
import { getProperties, set } from '@ember/object';
import { inject } from '@ember/service';

export default Component.extend({
  num: inject(),

  init() {
    const { currentNumber, num } =
      getProperties(this, 'currentNumber', 'num');

    const displayNumber = num.withEnding(currentNumber);
    set(this, 'displayNumber', displayNumber);
  },
})

…but that doesn’t actually gain me anything—the service here is just a way of exposing a function, after all—and it actually makes everything a bit more verbose. It also decreases the overall analyzability of this for things like tree-shaking: that module dependency is now something that Ember itself has to manage, instead of being statically analyzable at build time. Taking this approach also diminishes the reusability of any numeric helpers I put in there. If we couple them to an Ember.Service, instead of using an ES6 module, they would stop being things we can easily reuse in non-Ember projects. Instead, by using modules, we leave ourselves the ability to easily extract those numeric helpers, and publish them for either internal or external consumption.

Along those lines, we actually have a module to support BEM with Ember Components—and we plan to extract both the basic TypeScript library as well as a BemComponent Ember-specific wrapper as open-source libraries in the near future. Besides the Ember addon, anyone will be able to consume and use the underlying TypeScript library, whatever their framework or library of choice. Importantly, that includes us in our other codebases, which include lots of old jQuery and some new React, and might include some Glimmer.js in the future. Any or all of our utilities for these kinds of things become reusable if they’re just TypeScript.

Pragmatically, it’s also just easier to do and get good help from TypeScript by going this way. It also means that unit-testing requires no context from Ember whatsoever, which keeps those tests lighter and faster. Even though Ember’s unit tests are already super quick, when you have hundreds or thousands of unit tests, every little bit matters. It also, and probably even more importantly, means there are fewer places where you could mess things up when configuring tests—not that I have any experience messing up test configurations in Ember!

One important thing to note is that this all works best with Ember—by far—when your lib modules aren’t managing stateful objects, but rather defining data structures and functions which just transform those structures in some way. This approach is a great fit for us, because we’re increasingly writing a lot of our business and even UI logic in terms of pure functions which transform simple “record” types. That keeps each controller, route, component, or service doing relatively little work: they are responsible for getting and passing around data in the application, and for triggering actions—but they’re not responsible for understanding or manipulating that data. Meanwhile the module code doesn’t do any stateful work; there’s no mutation—just boring, input-to-output functions.¹ By contrast, if you’re dealing with stateful objects, you’re apt to end up running into places where you have lifecycle concerns, and that’s where Ember excels.

In summary: in this model, Ember handles all the lifecycle and view management, and is responsible for sending data in and out of the application. Plain old modules handle defining what the core internal data types are, and for manipulating, transforming, and creating data.

If you’re wondering: we’re not using anything like Redux or Immutable.js yet, but both ember-redux and seamless-immutable would be great fits for the way we’re building the app at this point, and it’s likely at least ember-redux will become part of our stack in the relatively near future.↩

Typing Your Ember, Part 3

2017-07-28T12:00:00-04:00

(See the rest of the series. →)

In the first of this series, I described how to set up a brand new Ember.js app to use TypeScript. In the second part, walked through adding TypeScript to an existing Ember.js app. In this part, I’m going to talk about using TypeScript effectively in a modern Ember.js app.

Heavy lifting, so-so results

Let’s get this out of the way up front: right now, using types in anything which extends Ember.Object is going to be a lot of work for a relatively low reward. Ember.Object laid the foundation for the modern JavaScript class system (and thus the TypeScript class system), but it has a huge downside: it’s string keys and referennces all the way down. This kind of thing is just normal Ember code—and note all the string keys:¹

import Component from '@ember/component';
import { computed, get } from '@ember/object';

export default Component.extend({
  someProperty: 'with a string value',
  someOther: computed('someProperty', function() {
    const someProperty = get(this, 'someProperty');
    return someProperty + ' that you can append to';
  }),
});

What this comes out to—even with a lot of the very helpful changes made to TypeScript itself in the 2.x series to help support object models like this one—is a lot of work adding types inline, and having to be really, really careful that your types are correct. If that property you’re Ember.get-ing can ever be undefined or null, you’d better write the type as string | void instead of just string. For example: this code is written with the correct types:

import Component from '@ember/component';
import { computed, get } from '@ember/object';

export default Component.extend({
  someProperty: 'with a string value',  // no type annotation
  someOther: computed('someProperty', function() {
    const someProperty: string = get(this, 'property');
    return someProperty + ' that you can append to';
  }),
});

Note two important things about it, however:

TypeScript does not (and, with the current typings for Ember, cannot) figure out the type of someProperty from this definition; get currently just hands back any as the type of these kinds of things. That type annotation is necessary for you to get any mileage out of TypeScript at all in a computed property like this.
If, anywhere in your code, you set the value of someProperty—including to undefined or null, or to { some: 'object' }—this could fail.

Unfortunately, this second point means that TypeScript actually can’t guarantee this the way we’d like. There’s hope coming for this in the future in several ways—more on that in a moment—but for now, I’ll summarize this by saying TypeScript is really helpful within a function, once you’ve correctly defined the types you’re using. That means that you have to continue to be very careful in what you’re doing in the context of any Ember.Object instance, including all the Ember types which descend from Object, and therefore also any types you define which extend those in turn.

Future niceties

In the future, we’ll be able to get away from a lot of these difficulties by way of two changes coming down the line: Ember embracing ES6 classes to replace its current custom object system, and embracing decorators as a way of replacing the current approach to computed properties. Let’s take those in turn.

`class` syntax

When Ember was birthed in the early 2010s (first as “SproutCore 2” and then “Amber.js” and finally “Ember.js”), the JavaScript world was a remarkably different place. The current pace of change year to year is nothing short of astounding for any language, but doubly so for one that sat languishing for so long. When Ember came around, something like today’s class syntax was unimaginable, and so essentially every framework had its own class system of some sort. Over the past few years, with the proposal and standardization of the class syntax as nice sugar for JavaScript’s prototypal inheritance, the need for a custom object and inheritance model has essentially gone away entirely. However, Ember doesn’t do breaking changes to its API just because; we as a community and the core team in particular have chosen to place a high priority on backwards compatibility. So any adoption of ES6 classes had to work in such a way that we got it without making everyone rewrite their code from scratch.

All of this impacts our story with TypeScript because, well, TypeScript for a long time couldn’t even begin to handle this kind of complexity (it’s a lot for a static type system to be able to express, given how very dynamic the types here can be). As of TS 2.3, it can express most of this object model, which is great… but it’s forever out of step with the rest of the JS/TS ecosystem, which is not so great. ES6 classes are first-class items in TypeScript and the support for getting types right within them is much, much stronger than the support for the mixin/extension style object model Ember currently uses. So moving over to ES6 classes will make it much easier for TS to do the work of telling you you’re doing it wrong with that class—and most importantly, it’ll be able to do that automatically, without needing the incredibly hairy type definition files that we’re still trying to write to get Ember’s current model represented. It Will Just Work. That means less maintenance work and fewer places for bugs to creep in.

Gladly, we’re getting there! Already today, in the most recent versions of Ember, you can write this, and it will work:

import Component from '@ember/component';

export default class MyComponent extends Component {
  theAnswer = 42;
  andTheQuestionIs =
    "What is the meaning of life, the universe, and everything?";
}

When I say “it will work,” I mean you can then turn around and write this in your my-component.hbs and it’ll be exactly what you would expect from the old Ember.Component.extend() approach:

{{andTheQuestionIs}} {{the Answer}}

There is one serious limitation of that today: you can’t do that with a class you need to extend further. So if, for example, you do like we do and customize the application route rinstance and then reuse that in a couple places, you’ll still have to use the old syntax:

import Route from '@ember/route';

export default Route.extend({
  // your customizations...
});

But everywhere you consume that, you can use the new declaration:

import ApplicationRoute from 'my-app/routes/application';

export default class JustSomeRoute extends ApplicationRoute {
  model() {
    // etc.
  }
}

There’s more work afoot here, too, to make it so that these restrictions can go away entirely… but those changes will undoubtedly be covered in considerable detail on the official Ember blog when they roll out.

Decorators

Now, that’s all well and good, but it doesn’t necessarily help with this scenario:

import Component from '@ember/component';
import { computed, get } from '@ember/object';

export default class MyComponent extends Component {
  someProperty = 'is just a string';

  someOtherProperty = computed('someProperty', function() {
    const someProperty = get(this, 'someProperty');
    return someProperty + ' and now I have appended to it';
  });
}

We’re back in the same spot of having unreliable types there. And again: some really careful work writing type definitions to make sure that computed and get both play nicely together with the class definition would help somewhat, but… well, it’d be nice if the types could just be determined automatically by TypeScript. (Also, there’s an open bug on the TypeScript repository for trying to deal with computed; suffice it to say that computed as it currently stands is a sufficiently complicated thing that even with all the incredible type machinery TS 2.1, 2.2, and 2.3 have brought to bear on exactly these kinds of problems… it still can’t actually model computed correctly.)

For several years now, Rob Jackson has maintained [a small library] that let you write computed properties with decorators. Up till recently, those were incompatible with TypeScript, because they used to work in the context of object literals rather than classes—and TypeScript never supported that. However, as of about a month ago as I’m writing this, they’ve been updated and they do work with ES6 classes. So, given the class syntax discussed above, you can now ember install ember-decorators and then do this:

import Component from '@ember/component';
import { computed } from 'ember-decorators/object';

export default class MyComponent extends Component {
  someProperty = 'with a string value';

  @computed('someProperty')
  someOther(someProperty: string) {
    return someProperty + ' that you can append to';
  }
}

Here, we can provide a type on the parameter to someOther, which at a minimum makes this enormously cleaner and less repetitive syntactically. More interestingly, however, we should (though no one has done it just yet, to my knowledge) be able to write a type definition for @computed such that TypeScript will already know that someProperty here is a string, because it’ll have the context of the class in which it’s operating. So that example will be even simpler:

import Component from '@ember/component';
import { computed } from 'ember-decorators/object';

export default class MyComponent extends Component {
  someProperty = 'with a string value';

  @computed('someProperty')
  someOther(someProperty) {
    return someProperty + ' that you can append to';
  }
}

And in that imagined, wonderful future world, if we tried to do something that isn’t a valid string operation—say, we tried someProperty / 3—TypeScript would complain to us, loudly.

Although this is still a future plan, rather than a present reality, it’s not that far off. We just need someone to write that type definition for the decorators, and we’ll be off to the races wherever we’re using the new ES6 class approach instead of the existing Ember.Object approach. So: soon. I don’t know how soon, but soon.

Current ameliorations

In the meantime, of course, many of us are maintaining large codebases. I just checked, and our app (between the app itself and the tests) has around 850 files and 34,000 lines of code. Even as those new abilities land, we’re not going to be converting all of them all at once. And we want to get some real mileage out of TypeScript in the meantime. One of the best ways I’ve found to do this is to take a step back and think about the pieces of the puzzle which Ember is solving for you, and which it isn’t. That is, Ember is really concerned with managing application state and lifecycle, and with rendering the UI. And it’s fabulous about those things. What it’s not particularly concerned with (and what it shouldn’t be) is the particulars of how your business logic is implemented. And there’s no particular reason, especially if most of that business logic is implemented in terms of a bunch of pure, straightforward, input-to-output functions that operate on well-defined data types, for all of your business logic to live in Ember.Object-descended classes.

Instead, we have increasingly chosen to write our business logic in bog-standard TypeScript files. These days, our app has a lib directory in it, with packages like utilities for commonly used tools… but also like billing, where we implement all of our client-side billing business logic. The display logic goes in the Ember.Controller and Ember.Component classes, and the routing and state management goes in the Ember.Route and Ember.Data pieces as you’d expect. But none of the business logic lives there. That means that we’re entirely free of the aforementioned constraints for the majority of the time dealing with that data. If we do a good job making sure the data is good at the boundaries—route loads, for example, and when we send it back to the server—then we can effectively treat everything else as just boring old (new?) TypeScript.

So far we’ve only taken that approach with about a quarter of our app, but it’s all the latest pieces of our app, and it has been incredibly effective. Even once we’re able to take advantage of all those shiny new features, we’re going to keep leaning heavily on this approach, because it lets Ember do what Ember is best at, and keeps us from coupling our business logic to the application state management or view rendering details.

Conclusion

So that’s the state of things in Ember with TypeScript today. Your best bet for getting real mileage out of TypeScript today is to use the new class syntax support and decorators wherever you can within Ember-specific code, and then to write as much of your business logic outside the Ember system as possible. Gladly, all of that points you right at the future (in the case of syntax) and just good practice (in the case of separating out your business logic). So: not too shabby overall. It’s working well for us, and I hope it does for you as well!

Next time: how we got here with the ember-cli-typescript compiler, and where we hope to go from here!

Note that here and throughout, I’m using the RFC #176 Module API, which you can use today via this polyfill.↩

The Book of F♯

2017-07-21T19:30:00-04:00

I keep my book review ratings simple—they’re either required, recommended, recommended with qualifications, or not recommended. If you want the TL;DR, this is it:

Recommended With Qualifications: This book is just okay, and at this point it’s a bit outdated—but if you’re in its fairly narrow target audience, it’s a decent way to get up to speed on F#.

The Book of F♯: Breaking Free With Managed Functional Programming is a No Starch Press publication by Dave Fancher, published in 2014. I read it over the course of the last four or so months, just plugging away in my spare cycles. A couple qualifications on the short list of observations that follow:

I don’t have any experience whatsoever writing production F♯ (though I have read a fair bit of it). I am interested because it’s a functional programming language on the .NET stack—which isn’t my own personal favorite stack, but is the stack at Olo. If we’re going to ship functional code on the server, it’ll be in F♯.¹
I am also not a C♯ developer. As such, I’m explicitly not the audience of this book. As Fancher put it in the intro:

I wrote this book for people like me: experienced .NET developers looking to break into functional programming while retaining the safety net of the tools and libraries they’re already using.

The net of that is that a lot of what frustrated me about the book is just a result of my not being the target audience.

Those qualifications aside, some assorted thoughts on the book:

First, as the intro and my above qualification suggest: this is really not interesting or useful as a general introduction to F♯. Throughout, it assumes a very high baseline of C♯ knowledge. In fact, the majority of the discussion of F♯, even in the section of the book which turns away from object oriented programming toward functional programming, focuses on comparing F♯ to C♯. This makes sense for the target audience, but this is not the book for you if you’re not a C♯ developer.

That said, if you are a C♯ developer, this could be a useful resource as you’re spinning up. It also might be a useful book to work through with a group of C♯ developers who want to learn F♯. The comparisons do generally work in F♯’s favor, even when doing exactly what you would be doing in the C♯, which makes it an easier “sell” in that regard.

Along the same lines, the book is structured as a very gradual introduction to functional programming ideas. Roughly the first half of the book emphasizes F♯’s object-oriented programming abilities, and only in the second half does Fancher turn to a functional style. Again, this is probably the right move given the audience, but it means the book spends a lot of time on kinds of F♯ you won’t actually be writing very often once you’re going. Idiomatic F♯ isn’t object-oriented. But as a way of helping someone make the transition, it’s not a bad plan: object-oriented F♯ is briefer and nicer in many ways than the exact same code in C♯. It meant that the first half of the book was completely uninteresting to me, though: I don’t want to write a line of object-oriented F♯.²

All of this had a pretty serious downside even for existing C♯ developers, though: the book often ends up seeming like it’s sort of apologizing for or defending F♯ against an expected audience of people asking “What’s wrong with C♯?” And even though there’s a real sense in which that’s true—that is what a lot of the audience is asking, no doubt—it became quite annoying rhetorically.³ It’s also unnecessary: if someone is picking up a book on F♯, you can assume that they’re alredy at least a little interested in the language and what it might offer! Along those lines, I much prefer the tack taken in what I’ve seen of Scott Wlaschin’s upcoming Domain Modeling Made Functional: Tackle Software Complexity with Domain-Driven Design and F♯ (The Pragmatic Bookshelf, expected in fall 2017)—which shows not how to do the same things as in C♯, just more briefly; but how to solve the same problems much more effectively.

Those problems aside, the book was… fine. I wouldn’t call it scintillating reading, but this kind of technical writing, especially at this length, is really hard work. Credit to Fancher for managing an introduction to an entire programming language in a relatively approachable way, and credit to him and his editors for making sure it remains lucid throughout. Still: I’d love to see the bar for programming books be higher. We need more books which are genuinely engaging in the world of programming language texts. These things are interesting; we don’t have to make them dry and dull! (And if you want a pretty good example of that: everything I’ve read of Edwin Brady’s Type-Driven Development with Idris hits the mark.)

A few other observations about the language itself from reading the book.

First, reading this highlighted a lot of strange things about F♯, all of which ultimately come down to the ways F♯’s development has been driven by concerns for interoperability with C♯. Worse, there are a lot of places where the influence of C♯ casts this shadow entirely unnecessarily. One particular expression of this which drove me crazy: F♯ far too often uses exceptions instead of Options. It’s one thing to make sure the language gracefully handle exceptions: you will have them coming from outside contexts. It is another entirely to design core parts of the language to throw exceptions where it doesn’t have to.

Perhaps the most prominent example is the List.head function. Its type signature is 'T list -> 'T, where I would expect it to be 'T list -> 'T option. If you call List.head on an empty list, you get an exception. It would make far more sense for it to return an Option and just give you None if there’s no item. Then you’re not worried about try expressions and the type system will actually help you! This is one of the most valuable parts of having a type system like F♯’s! I really don’t understand a lot of these decisions, not least since this isn’t for interop with C♯ collections—these are for native F♯ collections.

Second, the use of things like computation expressions instead of type machinery has an interesting effect: it makes it simpler to read when you first encounter it, but harder to compose, build, etc.—and it’s more syntax to remember. Computation expressions just end up being a way to do “monadic” transformations, from what I can tell. But as I noted often in my discussion of Rust and Swift, I profoundly prefer approaches that build on the same existing machinery—even in the surface syntax of the language—rather than constantly building new machinery. It makes it easier to deeply internalize new concepts and to understand the language (rather than just being able to use) the language. It also seems (from my admittedly limited vantage point) that computation expressions are as a result much less composable than actual type machinery of the sort available in other languages (Haskell, Idris, etc.).

Now, the tradeoff there is that adding those adds a lot of complexity both to the compiler and to the libraries people are apt to write; there’s a reason Elm has totally eschewed that kind of type machinery to date. But Elm has also refused to just add syntax around ideas like this the way F♯ has here, and it makes for a much cleaner and frankly nicer language.

And that brings me to my third and final point: I’m really glad F♯ exists, and that it’s providing a pretty good experience of functional programming on the CLR. But—and I fully grant that a fair bit of this kind of thing is almost entirely subjective—it doesn’t feel good in the same way that Elm or Rust do. There is something very difficult to nail down here, but I get a vsiceral experience of joy when writing some languages and not others. Again: that will vary person to person, but I think there are things that make it more or less likely. Things that make it more likely, at least for me, include everything from self-consistency and predictability at the semantic level to the way the code lays out and flows at the visual/syntactical level.⁴ Sadly, F♯ just doesn’t hit the right notes⁵ for me. I’ll be much, much happier to write it than C♯ at work… but I really just want Elm and Rust and Idris to come save the day.

I am of course writing a lot of functional code in our JavaScript; JavaScript is a surprisingly good language for it.↩
It’s not that OOP is bad, exactly; it’s just that what passes for OOP in languages like C♯, Java, and yes, F♯, is relatively low utility to me—and I think OOP ideas are much more interesting and useful when applied at a systems level, e.g. in an Actor system, than at the level of individual “actors” within the system. Compare Erlang/Elixir: functional components, organized in what is arguably an incredibly object-oriented way.↩
The temptation extends beyond this book; O’Reilly’s Programming Rust (Jim Blandy and Jason Orendorff) reads as the same kind of defensive introduction to Rust for C++ developers.↩
And yes, nerds, syntax does matter. Try reading this sentence, nicely punctuated, and with spaces and capitalization. Now: tryreadingthissentencewithoutpunctuationorspacesorcapitalization. There may be a point after which it becomes less important, and a range of things which are equally good in an absolute sense, but it matters.↩
Pun not intended, but inevitable given the language names here.↩

Farewell, Dropbox

2017-07-06T21:00:00-04:00

Over the last few years, I’ve grown increasingly annoyed with Dropbox. There have been a number of fairly high-profile misbehaviors on their part—most notably, this one—and then this past week, they started sending me notifications advertising Dropbox for Business.

Notification ads are the worst.

So, as I was with Google a few years ago when they pushed me over the edge—also with notifications!—I’m out.

I don’t mind Dropbox’s wanting to have a sustainable business. To the contrary: as I often note, I’m quite willing to pay for software I use, and I currently use a number of paid services where free alternatives exist because I’d rather do that than pay for ads.¹ I do mind when a company—any company—decides that building their business means mistreating their users and customers. And harassing me with notifications about a variant of their product I don’t care about certainly crosses that line. Combine that with the misbehavior and the fact that Dropbox has a tendency to hammer my system for no apparent reason, and, well, I’m out.

Transition Plans

File syncing

For basic storage and access to files across my devices, the shift will be pretty easy: I already have paid iCloud storage for backing up Photos (it’s far easier and comparably priced to all the other options, so that’s what we use). So everything I’ve been doing with Dropbox I’ll be doing with iCloud Drive instead. And I have way more overhead there with a 250GB plan that I do in my current 9GB of Dropbox storage.

My writing setup

Probably the most vexing (or at least: vexing-seeming) change here will be to my writing workflow. For a long time, I’ve made an alias pointing from a folder in Dropbox on my main machine into the clone of the Git repository on that machine where I manage my website.² That has meant that I can edit the source version of a given post anywhere at any time, with any editor that has Dropbox integration. That was a winning combo for a long time, and it’s one thing I actually can’t do with iCloud Drive. (I tried, and it sort of works, for a little while; but iCloud Drive doesn’t seem to expect this scenario. In its defense, it’s a weird setup.) I realized in thinking it through this evening, though: it doesn’t actually matter to me with the ways my workflow has shifted—and, perhaps just as importantly, with the way that the iOS ecosystem has shifted.

For one thing, there are a lot of options for directly editing files from Git repositories on iOS now. I don’t need to have it in Dropbox to be able to open it in any one of several great iOS writing environments, whether to make a quick edit or to create a post from scratch. Both Working Copy and Git2Go work very well. But for another thing, I currently can’t generate the site without logging into my home machine anyway.³ So if I need to make a tweak, well… Blink.sh or Prompt will let me log in remotely and do what I need to. And a little bit of Vim or Emacs will let me make any quick edits that way if I really feel I must.

And one side effect of realizing that is that I can easily enough just copy a file from iCloud storage to my site’s working directory after writing it in a writing folder in iCloud if I so desire. Sure, that’s a little finicky, but for the most part I won’t really need to mess with it: I can just git push from my iPad, git pull on my iMac and be ready to do whatever I need.

Other apps

The last piece of the puzzle is the other “apps” that have made a home in my Dropbox. The reality, though, is that almost none of those actually matter to me. I don’t even look at the majority of that data, and other pieces of it —backups of GPS and heart-rate data from workouts, or copies of all my tweets from when I wanted to maintain a microblog on this site, for example—are really just needless at this point, as I have all of that data stored in several cloud platforms (in the case of workout data) and/or don’t care about being able to retrieve it (in the case of tweets). I can happily just shut those things down and call it a day.

In Conclusion

So that’s it: goodbye Dropbox; hello other tools. (This post written from an iPad, and stored in iCloud Drive before publishing.) It’s been a long, and mostly just-fine ride, but I’m getting off here.

Full disclosure here: I am not a Dropbox paying customer—though that is the fault of their perhaps overly aggressive early customer acquisition strategy. I have never needed to pay for Dropbox, even though I have many gigabytes stored in it, because I earned so much free storage for inviting other users early on.↩
You don’t want a Git repo sitting inside your Dropbox folder, but a symlink like this works just fine: you don’t end up with the conflicts that can happen with a full repo in Dropbox.↩
I’m hoping to change that a bit in two ways in the future, by having the generator live on a not-my-home-machine server and by making Lighting much easier to just drop in and use than my finicky Pelican setup currently is. But that depends on actually making Lightning, you know, work.↩

Set Up Mosh on macOS

2017-06-29T08:10:00-04:00

Last night I bumped back into Mosh (by way of this post), and decided to give it a whirl – I had seen it before, and in fact had even installed it, but had never gotten around to giving it a try.

If you’re not familiar with Mosh, it’s like SSH: a remote (terminal) connection to another machine. Unlike SSH, though, a single session can survive disconnects: it sets up a small server on the host machine and will reestablish the connection if it drops. It also responds immediately when you’re typing, even if there’s serious lag to the other server – it just gives you a nice visual signal (underlining) to let you know the other side hasn’t received what you’ve typed. This seems pretty nice, so I thought I’d set it up on my iMac so I could hit it from my iPad.

This isn’t complicated, but it also isn’t well-documented after the first step!

Steps

Install mosh.
- via the binary on their site
- by running brew install mosh
Find the install location for the server from your Terminal:
```
$ which mosh-server
```
Configure the firewall to allow the mosh server to install connections.
1. Open the Security and Privacy pane of the System Preferences app.
2. Choose the Firewall tab. Unlock it to make changes.
3. Click Firewall Options.
4. On the pane that opens, click the + button to add a new rule.
5. Navigate to the location you got in step 2 above. (One easy way to do this: hit ⌘ Cmd⇧ ShiftG, and paste in the output from the which command.) Click Add.
6. Find “mosh-server” in the list, and set it to Allow incoming connections.
7. Hit OK.
Persuade macOS to reload its firewall rules. (This may not be necessary, but it was for me.) You can do one of the following:
- restart your machine
- reload the normal rules manually:
```
$ sudo pfctl -f /etc/pf.conf
```
You may also need to open these ports on your router firewall. You should consider carefully whether you want a bunch of open ports sitting there or whether you want to just use a specific port and then always target that specific port by running mosh with the -p option:
```
$ mosh -p 60000 [email protected]
```
If you can connect locally but not remotely, this is probably what you need!

That should be all you need!

Write! app review

2017-06-26T21:15:00-04:00

As I’ve noted in the past, I’m always on the lookout for top-notch writing environments. I was recently contacted by the team behind Write! and asked if I would take a look at and review their app, and I was happy to obliged. I tested the app out fairly thoroughly by doing what I normally do with my writing apps: putting together a blog post or the like. I’ve written this review from start to finish in it, across my two Mac machines. I promised the authors an unbiased review, so here we go!

Overview

Write!¹ describes itself as a distraction-free text editor. It enters the market in an interesting way: the Mac offerings here are numerous, varied, and excellent. Offerings on Windows are fewer and further between, and in my experience of much lower quality. Distraction-free text editors outside the world of programming text editors barely exist at all on Linux, as far as I can tell.² Write! is cross-platform, targeting all three of these. And that, as we’ll see, is the story of this particular app—for good and for ill.

The Good

First, the good: the app seems to perform relatively well. Text entry, even on a fairly large document, is smooth and quick. (I imported the text of this ~7200-word paper to test it and it didn’t stutter a bit.) Especially given the time I’m going to spend on the not-so-good below, I want to take a moment to applaud the developers for getting that right. It’s one of the most important aspects of an app like this, and any number of apps I’ve used just fall down on large documents. Everything I’ve seen here makes it seem like Write! would handle much larger documents even than that paper with aplomb.

The app’s main writing area looks fairly nice, and the distraction-free/full-screen mode gets out of the way readily enough. The cloud sync that comes with the app is quick and seems reliable. I’ve worked on this document across the two Macs I use, with no sync issues whatsoever. The writing area also has a (toggleable) overview of the document on the right, a la Sublime Text. To the left is a toggleable outline view, which lets you drill down into the structure of your document if you have multiple heading levels. And within the writing area itself, you can expand and collapse sections demarcated by headings.

In general, the experience of writing in the app is nice. Not amazing, but genuinely nice.

The Just Okay

There’s a bit of a delay before any open tabs are hidden in that fullscreen mode, but it’s otherwise fairly typical of most “distraction-free” writing environments in that regard. The colors chosen for the light and dark writing themes are fine, but not great. Much the same is true of the typography: it’s relatively pleasant, if bland. There are a number of built-in themes, but no apparent way to customize them to be more to your liking.

The app also features built-in autocomplete—but I’m not really sure who the target audience is for auto-complete in this kind of environment. It’s not bad, per se, to have it, but it doesn’t add a lot of value for writing (as opposed to, say, programming), and I turned it off fairly quickly in the process of writing this review.

Publishing

The app includes some “publishing” tools. Currently it supports writing to either Write’s own site, or to Medium. Medium publishing is nice—it’s certainly the hip tool du jour—but you’re out of luck if you use WordPress, much less something like Ghost.³

Publishing to Write! itself seems to be mostly a way of letting people see in-progress drafts. The links aren’t particularly friendly, and while they’d be easy enough to share to Facebook or Twitter or the like, they have serious downsides over any of the free blogging options out there for anything other than getting some early feedback—there’s no organizational or navigational structure available, and for that matter nothing that even ties it to your name! At a minimum, Write! should clarify what this is for.

Business model

The business model here is a curious mix: they’re selling the app at $19.95 (USD), with a year included of their custom sync solution. That sync solution is one of the things they advertise most heavily, and while I can attest that it works well, adding another, bespoke sync solution to my life is not on my list of things I’d like to do. It’s particularly an issue from where I stand because it doesn’t actually get me any benefits over a syncing solution using Dropbox or iCloud, both of which I’ve used extensively with other writing apps in the last few years, with no issues.

Add onto that the fact that the sync and future updates become an annual purchase—

Starting one year after purchase, Cloud access and maintenance updates are $4.95/yr.

—and any of the myriad other editors look much better: they all just use a sync engine I already use and like, and they don’t have annual fees for a service I don’t care about.

That goes double when you consider that I’ll often do different phases of drafting a given post in different editors, depending on the kind of content and what I’m doing with it. For example, I often use Caret for drafting technical blog posts, but at times I’ll switch over to using Sublime Text, Atom, or VS Code for working on the details of a given code snippet. If I’m using Write’s custom sync solution, my documents don’t exist in a normal folder on my machine, so they aren’t available for that kind of easy switching and editing. Double that again because it also means I don’t have access to the content on my iPad—where I often use Ulysses, Editorial, 1Writer, or Byword to work on posts when I’m away from my Mac. There are no upsides for me, as far as I can tell, to using their sync system.

I put this in the “just okay” section, however, because I can imagine that it might be nice for someone who’s not already invested in an existing sync solution. Whether or not there are enough of those people out there to support the business model—I suspect not—is a separate question to whether it’s good or bad for users in a direct sense. Again: the custom sync system works well; I just don’t know whether it’s necessary (or worth the development time that had to be spent on it).

As for the business model on the whole: I’m not at all opposed to paying for good apps on an ongoing basis. To the contrary, I actually embrace it: as a software developer myself, I recognize that there are few (if any) other sustainable business models. However, the application needs to be pretty amazing to get me to buy it in the first place, still less to justify a recurring purchase.

The Bad

Sad to say, from my perspective—to be clear, as a long-time Mac user with very high standards for my writing tools—this isn’t an amazing app. In fact, on macOS, it’s actually a bad app in many ways.

Non-native UI

First, Write’s UI looks and behaves like a Windows app. It’s built on Qt, which does support native(-looking) widgets, but the developers chose not to use them – I assume in the interest of speed of development. If you’re on Windows, that’s fine. But this app will never look remotely native on macOS,⁴ and given the plethora of other really high quality writing apps on macOS—some of them with their own publication options!—there’s just no reason why you would pick this over one of those at that most basic level.

Two examples should illustrate how painfully non-native this app is visually. First, note the window action buttons in the upper right:

not native windows

These are Windows window action buttons; the normal Mac action buttons simply don’t exist! Similarly, there’s a slide-out menu that appears when you tap the hamburger in the top left:

slide out menu

This is a reasonably nice, though not totally native-feeling, way of tackling the menu problem… on Windows. On Mac, it’s just duplicating the functionality of the normal menubar. And when I say duplicating, I mean it exactly: those menus are the same as the ones the app puts in the real menubar; there’s no reason for them to appear within the body of the app, other than that the app isn’t designed to work without them.

Right-click behavior is strange: instead of the normal Mac (or even Windows!) menu, they’ve supplied their own, and it’s actually its own little modal window, not a menu at all:

right-click modal window

I definitely see the utility of the little modal, but most other apps I’ve seen with similar approaches do it on highlighting some text. That way they can leave the normal right-click menu in place, which helps keep the user comfortable in their normal workflows. That’s going to be particularly annoying if you happen to make heavy use of macOS’s services menu—I don’t use it often, but when I want it, I want it.

Keyboard shortcuts

Similarly, a number of standard keyboard shortcuts don’t work the same way, or don’t work at all, in Write! as they do in native Mac apps. Navigation controls aren’t quite right: ⌥→ jumps to the start of the next word instead of the end of the current word; ⌥← doesn’t skip over punctuation; both stop on e.g. apostrophes in Write! (they skip over them natively). Other common shortcuts are bound to the wrong things: Shift⌥-, for example, increases heading size instead of inserting an em dash. ⌘Delete doesn’t do anything; neither do ^⌘Space, (normally used for bringing up the special-character selector) or my beloved ^K (“kill to end of line”) or ^T ("transpose characters around cursor) combos.⁵ I imagine the list is longer; that’s just what I noticed in the course of writing this review!

Most egregiously, Write! steals the keyboard shortcut ⌘`, normally used to switch between windows on macOS, to focus itself. Failing to implement and indeed overriding text input commands is one (very bad) thing; this is another kind of failure entirely. Apps should never override core system behavior with their shortcuts! The fact that you can customize them doesn’t make this better; and the one time I tried to customize it (to turn off stealing the switch-window shortcut) it ended up overriding the A key’s behavior to create new documents instead of to, well, enter the letter “a”.

A lot of apps get some of those more obscure ones wrong, sadly, but proper use of Core Text is a must for a native app in my book—and missing those super common ones is a big no-no. I simply won’t use an app long term that doesn’t do that, because I find the mismatch between the rest of the OS (and my muscle memory!) and what the apps do too frustrating.

Markdown support

The app claims Markdown support, and it sort of has it. But the goal is clearly to have a rich-text editing experience which can translate Markdown into whatever the underlying format is on the fly, and then export it back out when desired—not to be a Markdown writing application. You can see direct evidence that this is their approach by writing in Markdown and e.g. creating italics with * characters. When you view the exported Markdown, it’ll be using _ characters instead. Other little things flag it up equally: Markdown items don’t get converted to their rich text implementations unless you add a space or some punctuation after typing them; if you go back and wrap words in link syntax, for example, or try to make it bold with a pair of *s, it won’t be converted at all. The export still works fine in that case,⁶ but it certainly doesn’t come off well for the writing experience in the app, inconsistent as it is.

It also doesn’t support Markdown itself fully or properly. Inline backtick characters (`) don’t generate inline code snippets. Instead, they generate standalone code blocks, as if using the usual four-space-indent or triple-backtick markers for code blocks in the actual Markdown spec and as supported in other apps. Nor can I find a way to insert hrules/divisions with triple-stars or triple-dashes.

Other nits

There are a few other small but significant problems as well. One is related to the business model: you actually have to sign in to start using the app. Granted all my positive comments about subscriptions above, it’s still the case that needing to sign in to a writing app (especially just to use the app for local documents!) is a non-starter for me. As with so many of the other negatives I noted, this is a compromise that I don’t need to make, because the other alternatives don’t force it on me.

There are also a bunch of basically rough edges. Pasting with ⌘V does indeed paste the text… and scrolls you to the top of the current document every time. A number of times, the selection of a given option failed: it simply wouldn’t stick. Other times, especially when selecting the default text theme, cursor selection seemed broken. I’m not sure whether those are problems with the Qt engine, the implementation, or some of both, but again: not a good look, especially in a crowded market. Right-clicking, beyond the problems mentioned above, also just wouldn’t work consistently. Sometimes I would right-click and the menu would close immediately so you couldn’t take any actions in it at all—probably a result of using a modal instead of a normal menu there. Regardless of the reason, it was frustrating.

Last but not least, the app is unsigned, which means that it literally won’t open by default on macOS as of a few versions back. Users can certainly get around that, but they shouldn’t have to: there’s no excuse for not signing a paid app for macOS (or Windows! But I’m not sure what its status is there) in 2017.

Conclusion

This is an interesting approach for an editor. Trying to build a truly cross-platform app, and especially one that isn’t using web technologies like Electron, is an admirable goal—in fact, it’s one that I may dare to tackle myself at some point. Cross-platform UI is also a very hard problem, and unfortunately this app makes clear just how difficult it is by falling down so often on really important details. In reality, the only way to do it well is to write all your core business logic in a way you can share and then supply actually-native user interfaces. Anything else will inevitably feel out of place at best.

As a result, Write! is deeply compromised as a Mac app, to the extent that I simply cannot recommend it for Mac users. If you’re on a Mac, you should look instead at Ulysses, Byword, and Caret. All of them feel much more native, and though they have different strengths and weaknesses, they’re all native (or mostly-very-effectively native-acting, in Caret’s case) apps. That doesn’t mean Write! is bad; it just means it’s not worth your time (a) if you’re on a Mac or (b) if you really care about standard Markdown behaviors.

As noted, though, the developers got some important parts of this very right: the app performs well, it looks decent on Windows, and their sync engine seems incredibly solid. Accordingly, if you’re on Windows, and don’t already have a particular commitment to Markdown proper, I might even cautiously recommend it—as a replacement for something like the old LiveWriter app, for example. The biggest hesitation I’d have there is the business model—and, as noted above, I’m not opposed in principle to subscription models for good apps; but I’m not really sure what the value proposition here is.

Yes, the app is named “Write!” – not “Write”. It’s not my favorite, not least because it means you have to type an exclamation point every time you write (!) it.↩
There are many reasons for that, including things to do with many Linux users’ antipathy toward paid or non-open software, which makes it very difficult for not only developers but especially designers to make a living. Never mind the incredibly small size of the audience by comparison.↩
And who knows if Medium will still be around in five years? But that’s for another post another time.↩
Or Linux, but then what exactly is native on Linux anyway? 😏 More seriously, this will look out of place on any Linux desktop environment.↩
These latter ones are sadly too often the case for cross-platform tech; I’ve filed issues on VS Code and Atom in the past that way.↩
I’ll actually give Write! one point over Ulysses here: Ulysses does some similar conversions under the hood to make the writing experience seem snazzier, and things which don’t get turned into their custom “text objects” can end up exported very strangely.↩

“Collection-Last Auto-Curried Functions”

2017-06-24T17:35:00-04:00

I’ve been using lodash for a while at work, and I love having it in our toolbox. But, as I increasingly embrace composition of smaller functions as a helpful approach to building up the final version of an overall transformation of some piece of data, I’ve increasingly wanted to be using lodash-fp instead—those “auto-curried… data-last methods” are nice.

I could belabor the difference with words, but a code sample will do better. Here’s how I would write the same basic transformation in both Lodash and lodash-fp.

// Lodash
const breakfasts = ['pancakes', 'waffles', 'french toast']

const uniqueLetters = _.flow([
  bs => _.map(bs, words),
  _.flatten,
  bs => _.map(bs, b => split(b, '')),
  _.flatten,
  _.uniq,
  ls => _.sortBy(ls, id),
])

console.log(uniqueLetters(breakfasts))

That gets the job done, but wouldn’t it be nice if we didn’t have to have all those anonymous functions (lambdas) throughout?

// lodash-fp
const uniqueLettersFp = _.flow([
  _.map(words),
  _.flatten,
  _.map(split('')),
  _.flatten,
  _.uniq,
  _.sortBy(id),
])

const breakfasts = ['pancakes', 'waffles', 'french toast']

console.log(uniqueLettersFp(breakfasts))

Suddenly the intent is much clearer with the noise introduced by the lambdas gone. You get this because the lodash-fp functions are:

auto-curried: that is, even though _.split takes the splitter and then a string, you can just write _.split('') and get back a function which takes a string as an argument.
data-last: because _.split takes the string to split last, it can be passed into an auto-curried function.

You need both to get that nice clean call to _.flow. But once you have both, it’s really, really hard ever to go back, because it’s so much nicer for building pipelines of functions.

…I need to see if I can help do the work to make lodash-fp available in Ember.js.

Typing Your Ember, Part 2

2017-05-07T22:00:00-04:00

(See the rest of the series. →)

In the first part of this series, I described how to set up a brand new Ember.js app to use TypeScript. In this part, I’m going to talk about starting to use TypeScript in the context of an existing Ember.js app.

This is, in many ways, even simpler than setting up an app for the first time, because you already have almost everything you need. The steps here are exactly what you’re used to if you’re used to using the Ember CLI ecosystem:

Install ember-cli-typescript:
```
ember install ember-cli-typescript
```
Start using TypeScript wherever you want in your app.

It really is that simple, for the most part. There are a couple qualifications, and a couple tips, though.

Let’s start with qualifications. There are open, unresolved issues about using ember-cli-typescript in your app in certain contexts. For example: using it with ember-browserify. While everything will build correctly in that case (even if the TypeScript compiler complains about being unable to resolve some things, the Ember CLI build pipeline will still work as expected), your editor integration won’t. There are a bunch of corners like this we’re still hammering out; those are the main things we need to get resolved before we can call this a “1.0.” We have the main stuff working, but, well… there’s more to do.

Along those same lines, you should take a close look at the Not yet supported section of the README. There are parts of Ember’s programming model which TypeScript certainly can support, but which we haven’t done the lifting to get the type declaration file to help with yet. (Looking for a place to pitch in and already comfortable doing some heavy lifting with some of TypeScript’s most advanced type features? We could use the help.)

One other thing to be aware of is that your tsconfig.json settings will affect what kind of resolution your editor gives you. If you have allowJs set to true in your tsconfig.json, your editors will resolve JS modules. Otherwise, they’ll only resolve TS modules. This can be incredibly annoying at times. However, this isn’t something we’ve nailed down in terms of what the default should be yet. (You can come tell us on GitHub if you have thoughts or insights there!) And the fact that Microsoft has left this configurable is suggestive: different projects may have different preferences here.

Now, for the tips. Note that these are just a couple quick pointers; I’ll come back and talk about structuring your project and more sophisticated uses of TypeScript in the future.

Don’t turn on --strict or the corresponding individual flags on day 1. Unless you have an extremely unusual and disciplined Ember.js codebase, you’ll have an incredible set of errors to deal with.
Don’t set the noEmitOnError flag to true in your tsconfig.json, for much the same reason. Since the state of type declaration files for Ember is best described as nascent at present, many of your files will have errors in them just from failed imports!
Don’t try to convert everything at once. Just pick the next feature or bug you’re working on, and start with the files you’re touching for that bug. Rename it to .ts, fix any major issues it flags up that you can—but stick as locally as possible. You’re apt to find a lot of small bugs as you start migrating, and some of them are things which are apt to affect your whole system because they touch central data types. It’s okay. You can come back to those later. For today, you can just be explicit about the weirdnesses.
As part of that: get comfortable—really, really comfortable—with union types. They’ll make it much easier to express the kind of code you’ve actually written.¹
Don’t worry about adding explicit types to everything. In fact, depending on how comfortable you are already with typed languages, you should probably take a pretty different tack with this:
- If you’re just stepping into the world of typed programming languages, you might start adding types where they’re the lowest risk: some place like your automated tests. That’ll help you start to see how to take advantage of them, while not impacting the way you write your app code until you have a better idea how best to employ the types.
- If you’re already really comfortable with typed programming languages, you might employ types where they’re most helpful: start with some types in the hairiest or trickiest spots of your app.

There is plenty more I could say, but I think that’s a good start for now. I’ll have lots more to add in later posts about the details of how specifically to get the most mileage out of types within an Ember.js app today.

Previous: Part 1 – Set your Ember.js project up to use TypeScript.

Also, I strongly encourage you to write types in terms of unions of types rather than in terms of optional properties on types. That might be surprising; I’ll explain it in more detail in a future post.↩

Typing Your Ember, Part 1

2017-05-05T00:10:00-04:00

(See the rest of the series. →)

In this first post in the series, we’re going to keep things simple and easy: we’re going to get an Ember.js app configured to use TypeScript. Later posts will cover some of the other details.

Because of the lovely Ember CLI ecosystem, this is a pretty straightforward process. I’m going to start from zero so that even if you’ve never written an Ember app before, you can get this up and running by following these instructions. These instructions have also been tested and confirmed to work across platforms—you can do this equally on Windows, macOS, or Linux.

Make sure you have Ember’s prerequisites installed. Get Node for your platform. Optionally (but highly recommended) install Yarn to manage your Node packages.¹

Install the Ember command lines tools globally:

yarn global add ember-cli

npm install --global ember-cli

Create an Ember app.
```
ember new my-ts-app --yarn
```
(Using the --yarn flag will make it so your app uses yarn and creates a yarn.lock file instead of using npm when it installs its dependencies.)
Now move to the root of the newly created app: this is where we’ll do everything else in the post.
```
cd my-ts-app
```
Add the ember-cli-typescript add-on.
```
ember install ember-cli-typescript
```
Generate your first UI component.
```
ember generate component some-input
```
Rename the files it generated from .js to .ts:
- app/components/some-input.js → app/components/some-input.ts
- tests/integration/components/some-input-test.js → tests/integration/components/some-input-test.ts
(Eventually, we’ll make it so that you get TypeScript for all newly generated components when using ember-cli-typescript.)

Add some content to the files:

{{!-- some-input.hbs --}}
{{input value=theValue change=(mut theValue)}}
{{theValue}}

// some-input.ts
import Ember from 'ember';

export default Ember.Component.extend({
  theValue: '',
});

Update your application.hbs file to remove the default {{welcome}} template and replace it with {{some-input}}
Spin up the Ember application with Ember CLI’s development server:
```
ember serve
```
You’ll likely note some warnings: the TypeScript compiler won’t be able to find some of the modules imported in your files. I’ll have more to say about this in a future post. For now, suffice it to say: don’t worry, Ember CLI is still resolving and compiling your modules just fine.²
Load the application by going to localhost:4200 in your browser. You should see a blank white screen with an input in it. Type in it, and see the input rendered to the page. Simple enough, but it’s using a TypeScript file compiled along the way!

And that’s it: we’re done setting up an Ember.js app to use TypeScript! In the next post, I’ll talk a bit about strategies for migrating an existing app to TypeScript—not just the mechanics of it, but also where and how to start actually integrating types into your code.

Next: Part 2 – Adding TypeScript to an existing Ember.js project.

I strongly prefer to use yarn over npm because yarn installs are predictable and repeatable, and if there’s one thing I don’t need to spend time on when developing our Ember.js app at Olo, it’s chasing problems with transitive dependencies that are different in the build server than in my local development environment. Yarn’s lockfiles mean what ends up built on the server is exactly what ended up built on my machine.↩
But if you’re curious, here’s a preview: we really need more type definitions for the Ember ecosystem. I’ll be covering how we build those in much more detail in a future installment.↩

Why Elm Instead of TypeScript?

2017-04-23T17:20:00-04:00

A few weeks ago, an acquaintance asked in a Slack channel we’re both in:

Can I ask a noob type Elm / JS question?

Why Elm instead of Typescript? The dev stack and functional programming?

I responded as follows, with only light tweaks to clarify a couple things (and I’ll be reusing some of this material as the basis of an internal tech talk I’m giving on the same subject at Olo in a few weeks):

A couple things Elm gives you:

It’s not tied to JS directly, which means it’s free to just do what is the best fit for the language rather than needing to be able to express all the quirks and oddities of JS. That’s the single biggest thing I find all the time with TS (which I use every day and do quite like): as good as it is, and as both powerful and expressive as its type system is, at the end of the day it’s… still a superset of JavaScript, and that can mean some really nice things, but it also means a lot of weird things.
Elm’s type system is sound; TypeScript’s is not. At a practical level, that means that if an Elm program type-checks (and thus compiles), you can be sure – not mostly sure, 100% sure – that it is free of things like undefined is not a function. TypeScript does not (and by design cannot) give you that guarantee. And when I say “by design,” I mean that its designers believed from the outset that soundness was in tension with developer productivity, so they intentionally left a number of “soundness holes” in the type system—there’s still a lot of opportunity for undefined is not a function, sad to say. You can make it less than in JS… but not none. (That’s even still true in the TypeScript 2.x series, though the various soundness flags they added in 2.0 and the --strict option coming in 2.3 do get you closer.) In Elm, you can make it truly none. It’s just a sort of known fact at this point that Elm codebases tend to have zero runtime errors.
Elm’s language design is a huge win.
1. Elm is a pure functional language. Because non-pure things are offloaded to the Elm runtime, every single function you write is pure. Same input means the same output.
2. Elm supports first-class currying and partial application. This makes it much, much easier to do the kind of functional-building-block approach that is natural in FP and which is attractive in (but a lot more work in) JS or TS. Example code to show what I mean—
  
  Javascript:
```
const add = (a, b) => a + b;
const add2 = (c) => add(2, c);
const five = add2(3);
```
  Elm:
```
add a b = a + b
add2 = add 2
five = add2 3
```
3. The combination of the above means that you can refactor and always be sure you get everything, which is truly magical. And the compiler errors are the best in the world (and that’s no exaggeration).

The way I’d summarize it is to say that Elm makes it easy to do the right thing and hard or impossible to do the wrong thing. TypeScript makes it possible to do the right thing, and gives you a couple switches you can flip to make it harder to do the wrong things, but will ultimately let you do anything.

Functions, Objects, and Destructuring in JavaScript

2017-03-27T18:00:00-04:00

A colleague just getting his feet wet with JavaScript, and coming from a background with much more C# than JavaScript, sent me a question on Slack the other day, and I realized the answer I’d written up was more generally helpful, so here you go!

I’m including the context of original question because I want to call out something really important: there are no dumb questions. When you’re just coming up to speed on any technology, stuff is going to be confusing. That goes double when making the jump as far as between something like C# and something like modern JS.

Hey this may be a really dumb question

but I’m a JavaScript n00b, and I have no idea what’s going on here

I’m not used to this syntax

I have this program:
function ab() {
   function fa() { console.log("A"); };
   function fb() { console.log("B"); };
   return {fa, fb};
};

let {fa, fb} = ab();

fa();
fb();
and it outputs
A
B
(as expected)

What I don’t understand is the syntax for the let part (or maybe even the return from ab())

What is ab() actually returning? An object with 2 function pointers?

What can’t I do a let {a, b} = ab() and then call a() and b()? I get syntax errors that a and b aren’t defined

edit to show code that doesn’t work (definition of ab() remains the same):
let {a, b} = ab();

a(); // will throw an error here
b();
I don’t understand why the names for fa and fb have to be the same across all scopes/closures (? am I using those terms correctly? JavaScript is an odd dance partner at times)

First, your (A) is basically correct, but the phrase “function pointers” is one you should banish from your mind entirely in this context. In JavaScript, functions are just items like any other. From the language’s perspective, there’s no difference between these things other than what you can do with them:

let foo = "a string";
function quux(blah) { console.log("blah is " + blah); }
let bar = quux;

Both foo and bar are just variables. (quux is a variable, too, but it behaves a little differently; I’ll cover that in a minute.) They have different types, and therefore different things you can do on them. foo has the length property and a bunch of string-specific methods attached. bar is callable. But both of them are just things in the same way, and at the same level in the program.

So in your original function ab() { ... }, what you’re doing is declaring two functions, fa and fb, and returning them attached to an object.

For various reasons which aren’t especially interesting, functions can have names…

function fa() { ... }

…and can be assigned to other variables…

let trulyISayToYou = function waffles() { console.log("are so tasty"); };

…and in fact you can define the functions themselves anonymously, that is, without any name attached to the function declaration itself: combine those:

let lookMa = function() { console.log("no function name!"); };

Doing function ab() { ... } simultaneously declares the function and hoists it, that is, it makes it available in that entire scope, regardless of where it is defined. So you can do this, even though it’s kind of insane most of the time and you shouldn’t:

quux();
function quux() { console.log('SRSLY?'); }

Now, about returning fa and fb from the function.

First, note that you normally define objects in a long form, like so:

let someObject = {
  a: true,
  b: 'some string'
};

console.log(someObject.a);  // prints true
console.log(someObject.b);  // prints "some string"

However, very, very often, you find yourself doing something like this:

// do some work to define what `a` and `b` should be, then...
let someObject = {
  a: a,
  b: b
};

Because this is such a common pattern, the 2015 version of JS introduced a “shorthand,” which lets you just write that last assignment like this:

let someObject = {
  a,
  b
};

And of course, for convenience we often write that on one line:

let someObject = { a, b };

Then you can combine that with the fact that you declared two items (functions, but again: that really doesn’t matter, they could be anything) with the names fa and fb, and what you’re doing is returning an object containing those two items in it: return {fa, fb} is equivalent to this:

let theFunctions = {
  fa: fa,
  fb: fb, 
};
return theFunctions;

What about the let assignment?

JS has three kinds of name bindings: var, let, and const. var bindings act like function: the names you use get “hoisted”. So:

console.log(neverDefined);  // throws an error
console.log(definedLater);  // prints undefined
var definedLater = "what";
console.log(definedLater);  // prints "what"

let and const behave much more like you’d expect: they’re only valid after they’re defined, and they’re scoped to the blocks they appear in. (var will escape things like if blocks, too. It’s crazy-pants.) The difference between let and const is that they create mutable or immutable bindings to a name.

So let a = true; is just creating a name, a, and binding the value true to it. Likewise, with const b = false; it’s creating a name, b, and binding the value false to it. And those won’t be hosted. Now, having done let a = true; we could on the next line write a = false; and that’s fine: let bindings are mutable; they can change. We’ll get an error if we try to do b = true; though, because const bindings are not mutable.

One thing to beware of with that: things like objects and arrays, being reference types, are not themselves created as immutable when you use const. Rather, the specific instance is immutably bound to the name. So:

const foo = { a: true };
foo.b = 'I can add properties!';  // okay
delete foo.a;  // okay
foo = { c: "assign a new object" };  // will error

You can change the internals of the item bound to the name, but not assign a new item to the name. For value types (numbers, booleans, etc.), that makes them behave like constants in other languages. You have to use something like Object.freeze to get actually constant object types.

That was a long digression to explain what you’re seeing in a general sense with let.

Finally, let’s come back around and talk about that assignment and why you need the names fa and fb.

As noted, ab() returns an object with two items attached, fa and fb. (And again: functions are just items in JS.) So you could also write that like this:

let theFunctions = ab();  // theFunctions is now the object returned
theFunctions.fa();  // and it has the `fa` item on it
theFunctions.fb();  // and the `fb` item, too

Of course, if your original ab() function had returned other properties, they’d be accessible there, too, in just the same way (though they wouldn’t be callable if they weren’t functions).

Again, this is a super common pattern: you want to immediately do something with the values returned on an object by some function, and you don’t necessarily want to type out the name of the object every time. So ES2015 introduced destructuring to help with this problem. I’ll do it without the function in the way to show how it works at the simplest level first.

let someObject = {
  foo: 'what is a foo anyway',
  bar: 'hey, a place to drink *or* a thing to hit people with',
  quux: 'is this like a duck'
};

console.log(someObject.foo);  // etc.

Now, if we wanted to get at foo, bar, and quux, we could always do that with someObject.quux and so on. But, especially if we have some large object floating around, we often just want a couple properties from it—say, in this case, foo and quux. We could do that like this:

let foo = someObject.foo;
let quux = someObject.quux;

And of course those new names don’t have to match:

let whatever = someObject.foo;
let weLike = someObject.quux;

However, because wanting to snag just a couple items off of objects like this is so common, the shorthand is available. In the case of the shorthand for destructuring, just like the case of the shorthand for object creation, the names have to match: otherwise, it wouldn’t know what to match them with.

let { foo, quux } = someObject;

So, going back to your original example: ab() returns an object which has the items fa and fb on it. You’re using the destructuring assignment there to get just fa and fb. There’s no reason they have to be those names in the outer scope, other than that you’re using the destructuring assignment. You could also do this:

let theFunctions = ab();
let oneOfThem = theFunctions.fa;
let theOtherOne = theFunctions.fb;
oneOfThem();  // does what fa() does
theOtherOne();  // does what fb() does

I think that covers everything your questions brought up; but please feel free to ask more!

The most important thing to take away is that even though yes, those are pointers to functions under the hood, in JS that’s absolutely no different than the fact that there are pointers to objects and arrays under the hood. Functions are just more items you can do things with. You can put them on objects, you can return them directly, you can take them as arguments, etc.

Hopefully that’s helpful!

Bonus content: in ES2015 and later, you can also define anonymous functions like this:

let someFunction = (someArg) => { console.log(someArg); };

This has some interesting side effects about the value of this in the body of the function you declare… but that’s for another time.

Pick the Right Tool for the Job

2017-03-17T22:00:00-04:00

Over the past few years, I’ve been experimenting with publishing microblog posts here on my website as the “canonical” source for them, inspired by some of Manton Reece’s early experiments that way. I have also spent a considerable amount of time trying to come up with a good way to share links, and have been rather stymied by the limitations of the static site generator I use (Pelican): it does not support customizing the link target of RSS items.

Both of these desires, combined with the breadth of my interests, have been motivating factors in my desire to build my own CMS/site generator.

But as of today, I think I am setting aside those two needs, at least for the present (though the underlying information architecture needs for my site are not thereby particularly diminished, so Lightning will still aim for roughly the same goals when i get back to it).

My reasoning here is two-fold.

The simple reality is that the vast majority of my microblogging has zero historical value. It is ephemeral; archiving it is essentially a useless gesture, a “because I can” or perhaps “the internet should be permanent” act of defiance. But in truth, if every one of my tweets vanished… it would not matter one whit. I have been considering this for some time, but it came home to me tonight while considering my second point:
I have been experimenting with Pinboard as a bookmark management service over the past few weeks, spurred on by yet once more needing to dig out if an email from three years ago a particular post. (You can see my public bookmarks here. It’s a work in progress as far as organization goes.) One lovely Pinboard feature (and there are many others, including having a simple, profitable business model! Yes, that is a feature as far as I am concerned) is the option of public RSS feeds for publicly-bookmarked items by author, by tag, etc. This is actually what led me to Pinboard in the first place (thanks, ayjay). My Pinboard RSS feed is http://feeds.pinboard.in/rss/u:chriskrycho/, and if you want to follow along and see what I think is worth reading with occasional comments… that’s where it will be.

In the course of writing this post, I also remembered that last night, App.net shut down. I downloaded my archive a few weeks ago… I think. Honestly, I don’t recall, and the truth is that I haven’t looked at old posts there in years, though I amassed some 15,000 in the years I was active there. ADN was beautiful and wonderful. But even the very good conversations I had there are past in much the same way a conversation when physically present with each other would be. We do not suppose we need audio/video recordings of conversations just in case we might want to search them later. I understand why someone might want to archive all of ADN. But I don’t feel that desire myself anymore.

Archival has value. But its value is not ultimate, and its value is not universal.

The somewhat-ephemeral things I care about archiving are, I find, links—not random thoughts or comments or even conversations, but articles and posts I want to be able to come back to later, or quickly find to share with someone.

So no more microblog posts here. If you want them, you can follow me on Twitter. (If I hear from enough people who would prefer to keep getting them via RSS, I will think about setting up some sort of automated RSS mirror of my stream.) But for my own part, I am content to let the ephemeral be ephemeral. And that is easier to countenance now that microblogging isn’t also a poor-man’s bookmarking tool for me.

This takes me around to the meta point I had in mind when I started the post: use tools for what they’re good at and don’t try to force them into roles they’re not well-suited for. Twitter is good for ephemera, bad for permanence, decent for finding content I wouldn’t encounter via RSS, horrible for conversation or substantive commentary. Pinboard is great for bookmarking things, for sharing links via RSS, and for seeing what bookmarks others are sharing; but it is not at all “social” in the modern sense, with no facilities for discussion or interaction other than reading others’ links and copying them to your own board. Twotter for ephemera and trivial conversations. Pinboard for links. Blog for longer content.

Differences of Opinion

2017-03-12T08:05:00-04:00

I could not possibly agree more with this view of teaching software/CS.

We are focused on introductory programming education at a high-school and collegiate level — what is often called “CS 1” and “CS 2” (roughly, the first year of college). Pyret is being actively used in everything from high-schools to upper-level collegiate courses, giving us a tight feedback loop.

Of course, even in that setting there are differences of opinion about what needs to be taught. Some believe inheritance is so important it should be taught early in the first semester. We utterly reject this belief (as someone once wisely said, “object-oriented programming does not scale down”: what is the point of teaching classes and inheritance when students have not yet done anything interesting enough to encapsulate or inherit from?). Some have gone so far as to start teaching with Turing Machines. Unsurprisingly, we reject this view as well.

What we do not take a dogmatic stance on is exactly how early state and types should be introduced. Pyret has the usual stateful operations. We discussed this at some length, but eventually decided an introduction to programming must teach state. Pyret also has optional annotations, so different instructors can, depending on their preference, introduce types at different times.

I’m delighted to see work on languages like Dr. Racket and Pyret, and the more so because the teams behind both have been willing to set aside many of the dogmas of how CS has been taught and actually do pedagogical research. Also: OOP is a useful tool, but I’m with them: treating inheritance as a first-semester concept is… nutty.

The whole “Why Pyret?” page is worth reading if you have any interest in programming languages or teaching software development and computer science.

Dear Tech CEOs: Yes, That Is Your Culture.

2017-02-21T17:00:00-05:00

It is common, when stories break about horrible company cultures in —as one did this week about Uber, and as one did last year about Amazon—for the CEOs to say things like:

What she describes is abhorrent and against everything Uber stands for and believes in. (Uber CEO Travis Kalanick’s statement quoted at Recode)

Or:

The article doesn’t describe the Amazon I know or the caring Amazonians I work with every day. (Amazon CEO Jeff Bezos’ statement quoted at Geekwire)

The problem with this is simple. You can say all day that these events don’t represent your culture. But they do. And if you’re not aware of it, that means you have two problems: the culture problem, and the fact that you’re so out of touch with what your company is actually like that you don’t know it has that culture problem.

And you have one other, even bigger problem. If that’s the culture of the company you’ve built, it’s your fault. You can’t foist it off on your underlings: you hired them. You can’t foist it off on the bureaucracy: you built it. You can’t foist it off on wrong priorities: you set those priorities. It’s on you.

Better Off Using Exceptions?

2017-02-20T12:00:00-05:00

I saw this post on error-handling in F^♯, “You’re better off using Exceptions” making the rounds on Twitter:

Exception handling is an error management paradigm that has often been met with criticism. Such criticisms typically revolve around scoping considerations, exceptions-as-control-flow abuse or even the assertion that exceptions are really just a type safe version of goto. To an extent, these seem like valid concerns but it is not within the scope of this article to address those per se.

Such concerns resonate particularly well within FP communities, often taken to the extreme: we should reject exceptions Show more…

And I get the argument, and in the specific context of F^♯—especially given how much C^♯-interoperating and therefore exception-throwing-code-interoperating there is there—it’s reasonable.

But it still makes me sad. (To be clear: exceptions were and are a big win over what you get in languages like C. I’ll take them any day over goto or segfault.)

You need to embrace exceptions in F^♯ because F^♯ has exceptions and because many of its libraries rely on exceptions. But my experience with Rust and other non-exception-using languages is that you don’t need exceptions in the general case.

The questions are: whether your language has good support for things like flat-mapping, and whether you’re willing to commit to letting the compiler help you with these problems.

To be sure: there’s more work involved up front to deal with that. But that’s a tradeoff I’m always willing to make. I’d rather have the compiler tell me if I’m failing to account for something than learn because I saw a runtime error report come up in Raygun, especially because that tends to mean an error that affects the user in some way.

Rust’s model gives you something like exceptions for truly unrecoverable errors, “panics.” A panic gives you all the context you’d get from an exception (one of the virtues of exceptions highlighted in that post), but you can only “catch” it at thread boundaries, and it otherwise just kills the program. Because it’s catastrophic, you only use it where you don’t have any way to recover in your immediate context. But where you can recover in your immediate context… using something like a highly descriptive enum (just as suggested at the end of that original post!) is a better option.

It’s well-understood in my circles that you shouldn’t use exceptions for things you can recover from; you should use them for things you can’t recover from. But in most languages which lean heavily on exceptions, you inevitably start using them for control flow. I say: if you can recover from an error… just recover from it! Account for recoverable errors as possible conditions in your program and carry on! If you can’t recover… don’t. Die and let some other part of your system kick things back off.

In summary: yes, if you’re in F^♯, use exceptions. It is the right thing to do in many cases (and you don’t have a choice in many others). But I’m hopeful for a future where we handle recoverable errors locally, and act like Erlang or Elixir otherwise.

On Blogging

2017-01-04T23:25:00-05:00

My wife is out of town and I had coffee at our church small group tonight, so I’m wide awake and up late, thinking about blogging. It’s been on my mind a lot lately. (And what follows is, appropriately, as you will see, blogging in the old style—which is to say: a bit rambly. I apologize. It’s the coffee.)

A friend at our small group meeting tonight mentioned his intent to start blogging this year. He had a lot of good reasons for jumping in, and I strongly encouraged it. Blogging is not for everyone—we’ll get to that—but blogging is good. This newish thing, writing-on-the-web-in-a-log, has been a part of my life for over a decade now. I wrote my first post on Xanga in the fall of 2005, and I have not gone more than a matter of weeks between posts since. It is not hyperbole to say I cannot imagine not blogging at this point. (The sheer number of words I published last year should serve to drive home that point: even in a year which was full to the brim, I somehow ended up publishing almost 70,000 words.)

And yet, blogging is not at all like it was in when I started in 2005. Both for good and for ill. The mid-2000s were in many ways the height of blogs’ power and reach. Individual sites still hosted all their own content; blogging networks were nascent; and the ability of small content providers to outdo the old guard was beginning to be felt at all levels. Bloggers made a difference in the 2004 election not so different than the way Twitter shaped all sorts of politics in 2016.

Twitter. That brings us to one of the things that has changed. Microblogging is new. And social media more generally has changed a great deal. Facebook was still the fledgling upstart nipping at MySpace’s heels when I published that first post on Xanga. Today, Facebook dominates the web, and Twitter—not even a product at all in 2005—has taken enormous chunks of the time and attention of the would-be writerly class (journalists especially).

I first took my own blogging seriously on the Blogger site which first ran in parallel with and then displaced that Xanga. And Blogger, too, evokes a different time: when individuals setting up blogs was trendy, and when the competition between WordPress and Blogger could be called a competition. (Much-neglected Blogger trucks on still, but WordPress powers perhaps a quarter of the sites on the web.)

But for all that, some things haven’t changed. Business plans still matter—and Ev Williams, founder of Blogger, Twitter, and Medium, still hasn’t figured out something truly sustainable. Attention-driven advertising of the same sort that powered Blogger then and now, and which powers Twitter and Facebook equally, continues to be a race toward the bottom. Sustainable publishing on the web is a mirage for all but a few, because content is plentiful and distinguishing features few. The Daring Fireballs of the world are notable, these days, not least for how few of them there are.

In some ways, there is something real to mourn in the passing of the web of those early days when I started blogging. People did own their own content (at least, to a far greater degree than now). Blogs linked to each other, using ping-backs to let sites know when they’d been linked. Comment sections flourished.

But that era also required a level of technical knowledge that was simply too high a bar for most people. To be sure, anyone could set up a blog with enough grit, and WordPress and Blogger lowered the bar. But subscribing to another blog meant wrangling RSS and learning the arcana of managing Google Reader (which soon swallowed all competitors before its own too-delayed demise). Twitter’s “follow” button seems a revelation by comparison; it is no wonder at all that first Tumblr and then Medium embraced the idea of blogs-as-social-media, “following” and all. Being able not only to respond, and only if the author so allowed, but also to initiate with anyone else on the service… the first time you @-mentioned someone well-known in your circles, and they responded—that was (and is) a heady thing.

Centralization is often a function of convenience. Facebook and Twitter make it simple for you to “connect with” or “follow” whomever you like. No digging for RSS feeds, wondering if they have a non-standard symbol for it or hoping desperately that it’s at the root of the site + feed.xml, or (if you really know the secrets of the web) that they set it up as a <link> tag with a rel='alternate' tag so it could just be discovered automatically by a smart-enough feed reader…

You see? If you aren’t technical yourself, your eyes just glazed over in that paragraph, and that’s the point. The technical details make sense if you understand them. But understanding them is hard; and more to the point, they don’t matter for what people actually want to accomplish.

This is the fundamental mistake of Manton Reece’s new micro.blog project (which I like in principle, and whose goals I clearly share). People as a whole don’t even know there might be a reason to prefer the open web, where everyone owned their own content and there was no central clearing-house of information. Facebook offers real value to people: it shows them things they’re interested, and keeps them coming back precisely by tailoring its algorithm to make sure they don’t see too many things they don’t want to see. (The polarization that helps foster may be dreadful, but it’s very good business.) The same goes for Twitter, regardless of the structure of its timeline: people self-select into their lists of whom to follow. Manton’s project is a good one in many ways—but the problem it solves is a Winning Slowly kind of problem, and one that takes a lot of selling when the problem Facebook solves is obvious: I want a new story and a picture of my cousin’s kid and a funny cat video. Decentralizing, whatever its benefits (and again: note well my bona fides here), makes those basic tasks harder. I’ve followed what is now the micro.blog project from a distance for years now—and I’ve always had this one, nagging but oh-so-important question. How does this solve a user problem?

The answer, if there is one, is a decades-long play. It’s a hedge against technological oligarchy. But how do you get people to care? What’s the pitch? The technical problems are easy compared to that—and the technical problems are not easy; they remain almost untouched in the last decade, and micro.blog has no intent to address some of the core issues. Real-time interaction is what makes Twitter Twitter; ping-backs aren’t even close. And that’s just Twitter; Facebook outstrips it by far.

And Medium? Medium doesn’t know what it wants to be when it grows up (and it never has; the same as Twitter). As a second pass at Blogger, it has better aesthetics and something like a mission. Ev writes:

So, we are shifting our resources and attention to defining a new model for writers and creators to be rewarded, based on the value they’re creating for people. And toward building a transformational product for curious humans who want to get smarter about the world every day.

That’s a lovely-sounding sentiment. It’s also—for today, at least—contentless business blather. “Building a transformational product for curious humans who want to get smarter about the world every day” sounds great, but it doesn’t mean anything. Medium is a beautiful product without a reason to exist. (How often do you see a founder basically admit: We have no idea what we’re doing here? But that’s roughly what Ev did.) That doesn’t mean it shouldn’t exist. It just means no one has thought of a good reason for it to just yet.

Medium as a centralized, social medium for longer-form writing is better than nothing. I’ll take Medium + Facebook + Google over just Facebook + Google any day. But is there something lost when every blog post looks the same, and when everyone is locked into one more centralized platform? Yes. Just as there is something gained by people having a place to look. The questions are: whether the costs are indeed higher than the benefits; and even if so whether people can be persuaded of those costs when they all take a decade to appear, and Medium is really pleasant to scroll through right now.

By whatever quirk of temperament, I’m old-school about blogs. I’d love for the open web to win out over all the centralizers. That’s not going to happen: centralization provides too much value to users. But we can hope the open web will flourish alongside centralized sources. And, far more importantly, we can work to that end.

We can show people why it matters, and teach them how to own for themselves even the things you publish on Facebook. We can make philosophies like POSSE easier to implement (because right now it’s just plain hard). We can let Twitter be secondary and our own blogs primary as a way of setting an example. We can do the hard technical work of figuring out something like real-time, decentralized, better-than-ping-back commenting-and-threading-and-responding for all sorts of content on the web.

But even solving those technical problems will require us to recognize that the bigger and more important problems are human problems. It’s going to require distinguishing, for example, between the web we have to save and mere ephemera. If there are goods of how-blogging-was-in-2008 that are worth keeping, what if anything do they have to do with whether the content of nearly any Twitter account is “owned” by the user who generated them? Will the user care, two decades from now? (Or two days?)

Put another way, we need to care about the open web not in some general or abstract sense, and certainly not just on its technical merits, but instead—and quite specifically—as one means of serving other people. If we cannot express it in those terms, we show that we do not understand the real problems at all. It wouldn’t be the first time a bunch of technically-oriented nerds missed the boat.

TypeScript keyof Follow-Up

2017-01-03T20:35:00-05:00

I recently wrote up some neat things you can do with keyof and mapped types in TypeScript 2.1. In playing further with those bits, I ran into some interesting variations on the approach I outlined there, so here we are.

In the previous post, I concluded with an example that looked like this:

type UnionKeyToValue<U extends string> = {
  [K in U]: K
};

type State
  = 'Pending'
  | 'Started'
  | 'Completed';

// Use `State` as the type parameter to `UnionKeyToValue`.
const STATE: UnionKeyToValue<State> = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
}

That UnionKeyToValue<State> type constraint requires us to fill out the STATE object as expected. The whole point of this exercise was to give us the benefit of code completion with that STATE type so we could use it and not be worried about the kinds of typos that too-often bite us with stringly-typed arguments in JavaScript.

It turns out we don’t need that to get completion, though. All editors which use the TypeScript language service will give us the same degree of completion if we start typing a string and then trigger completion:

string completion with TypeScript 2.1

Granted that you have to know this is a string (though the JetBrains IDEs will actually go a step further and suggest the right thing without needing the string key). But that’s roughly equivalent to knowing you need to import the object literal constant to get the completion that way. Six one, half dozen the other, I think.

This makes it something of a wash with the original approach, as long as you’re dealing in a pure-TypeScript environment. The big advantage that the original approach still has, of course, is that it also plays nicely with a mixed TypeScript and JavaScript environment. If you’re just progressively adding TypeScript to an existing JavaScript codebase, that’s possibly reason enough to stick with it.

Edit: an additional reason to prefer my original solution:

@chriskrycho @typescriptlang I think a benefit of your previous solution is that you can rename keys and all their usages.
— Timm (@timmpreetz) January 4, 2017

On ebooks (again)

2017-01-02T07:55:00-05:00

Yesterday, I tweet-complained about Lifeway’s ebook policy; posted here on this site:

For the record: @LifeWay has the single most user-hostile ebook policy I’ve encountered anywhere. Shame on whoever made these decisions. ∞

They invited feedback:

@chriskrycho We’re sorry you feel this way. Please DM us the specifics of your concern. We are always trying to improve. ∞

So they got it. But since the content of my response is not specific to Lifeway—I’ve seen the same kind of user/customer-hostile behavior in many places, albeit not to the same degree—I thought I’d repost it publicly:

Per your request, sending some info about ebooks. To be clear, I know whoever runs the social media account is far removed from this kind of decision, and I don’t hold anyone but the corporate decision-makers remotely responsible.

The current status on ebooks purchased at LifeWay is that you can read them only in the dedicated LifeWay app or in a web view (the latter of which is, frankly, terrible—both as a consumer of many such apps and as a web software developer).

Experientially, this makes for a massively more frustrating experience for the user. There is no reason why a customer should not be able to read the book on her Kindle, his Kobo, her Dell PC, his Mac, etc. except the idea that it will somehow “prevent piracy” to do so.

Empirically, however, these kinds of moves do not prevent piracy. Any and all DRM methods can and will be cracked (and indeed have been).

The net result, then, is a massive inconvenience to customers, with no actual increase in sales for the distributor.

For some corroborating evidence in this direction, please see here. Note especially this comment (emphasis mine):

Protecting our author’s intellectual copyright will always be of a key concern to us and we have very stringent anti-piracy controls in place. But DRM-protected titles are still subject to piracy, and we believe a great majority of readers are just as against piracy as publishers are, understanding that piracy impacts on an author’s ability to earn an income from their creative work. As it is, we’ve seen no discernible increase in piracy on any of our titles, despite them being DRM-free for nearly a year.

Please respect your customers. Let us put the ebooks in whatever reading app or device we choose.

And please also understand that until you do so, I will go out my way to warn people off of buying any ebook from Lifeway, because the current approach is so consumer-hostile.

Again, I understand entirely that the folks running social media have nothing to do with these policies; I hope you’re able to pass it along to those who can and do. Have a great 2017!

The Itch

2016-12-19T21:45:00-05:00

It took me until just a few weeks ago to put my finger on why typed functional programming, as a style and approach, has appealed to me so much as I started picking it up over the last year. For all its novelty, typed FP feels—over and over again—familiar. Strange to say, but it’s true.

This came home to me again when reading a short post on functors—i.e., mappable types. I’ve written a lot of JavaScript in the last few years, and it has been a source of constant frustration to me that Array implements the map method, but Object does not. Countless times, I have wanted to take an object shaped like { count: <number> } and transform that count. I’m not alone in that. There’s a reason that libraries like Underscore, Lodash, and Ramda all supply utilities to allow you to map over objects. There are also good reasons why it isn’t implemented for on Object.prototype: the reality is that coming up with a predictable and useful API for all Object instances is difficult at best: Objects are used for everything from dictionaries to records and strange combinations of the two. But still: there’s something there.

And reading this post on functors, it struck me what that “something” is: object types are, in principle, functors. Maybe it doesn’t make sense to have a single map implementation for every Object instance out there. But they’re perfectly mappable. I didn’t have a word for this before tonight, but now I do. Over and over again, this is my experience with functional programming.

There’s this familiar feeling of frustration I’m slowly coming to recognize—a mental sensation which is a little like the intellectual equivalent of an itch in a spot you can’t quite reach. You’re reaching for an abstraction to express an idea, but you don’t even know that there is an abstraction for it. You want to map over objects, and you don’t know why that seems so reasonable, but it does. And then someone explains functors to you. It scratches the itch.

Another example. Since I started programming eight and a half years ago, I’ve worked seriously with Fortran, C, C++ PHP, Python, and JavaScript. In each of those languages (and especially in the C-descended languages), I have found myself reaching for enums or things like them as a way of trying to represent types and states in my system in a more comprehensive way. I figured out that you should use enums not booleans a long time before I found the advice on the internet. I was encoding error types as enum values instead of just using ints almost as soon as I started, because it was obvious to me that ErrorCode someFunction() { ... } was far more meaningful than int someFunction() { ... } (even if the context of C meant that the latter often implied the former, and even if it was trivial to coerce one to the other).

Then I read Maybe Haskell, a book I’ve mentioned often on this blog because it was so revelatory for me. This is what I had been reaching for all those years—and then some. Handing data around with the constraints? Yes, please! I had played with unions, enums, structs with enums inside them, anything to try to get some type-level clarity and guarantees about what my code was doing. Haskell showed me the way; and since then Rust and Elm and F# have reinforced it many times over. Tagged unions are a joy. They let me say what I mean—finally.

I can still feel that itch. It’s shifted a little, but it’s still there: reaching for higher abstractions to let me tell the machine more clearly what I intend. Half a dozen times this year, I’ve realized: Here is where dependent types would be useful. They’re far beyond me, but close enough now I can see. I’m sure a year from now, I’ll have find some tools to scratch these itches, only to discover a few more.

keyof and Mapped Types In TypeScript 2.1

2016-12-17T23:25:00-05:00

In the last few months, I’ve been playing with both Flow and TypeScript as tools for increasing the quality and reliability of the JavaScript I write at Olo. Both of these are syntax that sits on top of normal JavaScript to add type analysis—basically, a form of gradual typing for JS.

Although TypeScript’s tooling has been better all along¹ I initially preferred Flow’s type system quite a bit: it has historically been much more focused on soundness, especially around the many problems caused by null and undefined, than TypeScript. And it had earlier support for tagged unions, a tool I’ve come to find invaluable since picking them up from my time with Rust.² But the 2.0 and 2.1 releases of TypeScript have changed the game substantially, and it’s now a very compelling language in its own right—not to mention a great tool for writing better JavaScript. So I thought I’d highlight how you can get a lot of the benefits you would get from the type systems of languages like Elm with some of those new TypeScript features: the keyof operator and mapped types.

Some readers may note that what I’m doing here is a lot of wrangling to cajole TypeScript into giving me the kinds of things you get for free in an ML-descended language. Yep. The point is that you can wrangle it into doing this.

Plain old JavaScript

Let’s say we want to write a little state machine in terms of a function to go from one state to the next, like this:

function nextState(state) {
  switch(state) {
    case 'Pending': return 'Started';
    case 'Started': return 'Completed';
    case 'Completed': return 'Completed';
    default: throw new Error(`Bad state: ${state}`);
  }
}

This will work, and it’ll even throw an error if you hand it the wrong thing. But you’ll find out at runtime if you accidentally typed nextState('Pednign') instead of nextState('Pending')—something I’ve done more than once in the past. You’d have a similar problem if you’d accidentally written case 'Strated' instead of case 'Started'.

There are many contexts like this one in JavaScript—perhaps the most obvious being Redux actions, but I get a lot of mileage out of the pattern in Ember, as well. In these contexts, I find it’s convenient to define types that are kind of like pseudo-enums or pseudo-simple-unions, like so:³

const STATE = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
};

Once you’ve defined an object this way, instead of using strings directly in functions that take it as an argument, like nextState('Started'), you can use the object property: nextState(STATE.Started). You can rewrite the function body to use the object definition instead as well:

function nextState(state) {
  switch(state) {
    case STATE.Pending: return STATE.Started;
    case STATE.Started: return STATE.Completed;
    case STATE.Completed: return STATE.Completed;
    default: throw new Error(`Bad state: ${state}`);
  }
}

Using the object and its keys instead gets you something like a namespaced constant. As a result, you can get more help with things like code completion from your editor, along with warnings or errors from your linter if you make a typo. You’ll also get slightly more meaningful error messages if you type the wrong thing. For example, if you type STATE.Strated instead of STATE.Started, any good editor will give you an error—especially if you’re using a linter. At Olo, we use ESLint, and we have it set up so that this kind of typo/linter error fails our test suite (and we never merge changes that don’t pass our test suite!).

This is about as good a setup as you can get in plain-old JavaScript. As long as you’re disciplined and always use the object, you get some real benefits from using this pattern. But you always have to be disciplined. If someone who is unfamiliar with this pattern types nextState('whifflebats') somewhere, well, we’re back to blowing up at runtime. Hopefully your test suite catches that.

TypeScript to the rescue

TypeScript gives us the ability to guarantee that the contract is met (that we’re not passing the wrong value in). As of the latest release, it also lets us guarantee the STATES object to be set up the way we expect. And last but not least, we get some actual productivity boosts when writing the code, not just when debugging it.

Let’s say we decided to constrain our nextState function so that it had to both take and return some kind of State, representing one of the states we defined above. We’ll leave a TODO here indicating that we need to figure out how to write the type of State, but the function definition would look like this:

// TODO: figure out how to define `State`
function nextState(state: State): State {
  // the same body...
}

TypeScript has had union types since the 1.4 release so they might seem like an obvious choice, and indeed we could write easily a type definition for the strings in STATES as a union:

type State = 'Pending' | 'Started' | 'Completed';

Unfortunately, you can’t write something like State.Pending somewhere; you have to write the plain string 'Pending' instead. You still get some of the linting benefits you got with the approach outlined above via TypeScript’s actual type-checking, but you don’t get any help with autocompletion. Can we get the benefits of both?

Yes! (This would be a weird blog post if I just got this far and said, “Nope, sucks to be us; go use Elm instead.”)

As of the 2.1 release, TypeScript lets you define types in terms of keys, so you can write a type like this:⁴

const STATE = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
};

type StateFromKeys = keyof typeof STATE;

Then you can use that type any place you need to constrain the type of a variable, or a return, or whatever:

const goodState: StateFromKeys = STATE.Pending;

// error: type '"Blah"' is not assignable to type 'State'
const badState: StateFromKeys = 'Blah';

interface StateMachine {
  (state: StateFromKeys): StateFromKeys;
}

const nextState: StateMachine = (state) => {
  // ...
}

The upside to this is that now you can guarantee that anywhere you’re supposed to be passing one of those strings, you are passing one of those strings. If you pass in 'Compelte', you’ll get an actual error—just like if we had used the union definition above. At a minimum, that will be helpful feedback in your editor. Maximally, depending on how you have your project configured, it may not even generate any JavaScript output.⁵ So that’s a significant step forward beyond what we had even with the best linting rules in pure JavaScript.

Going in circles

But wait, we can do more! TypeScript 2.1 also came with a neat ability to define “mapped types,” which map one object type to another. They have a few interesting examples which are worth reading. What’s interesting to us here is that you can write a type like this:

type StateAsMap = {
  [K in keyof typeof STATE]: K
}

And of course, you can simplify that using the type we defined above, since StateFromKeys was just keyof typeof STATE:

type StateAsMap = {
  [K in StateFromKeys]: K
}

We’ve now defined an object type whose key has to be one of the items in the State type.

Now, by itself, this isn’t all that useful. Above, we defined that as the keys on the STATE object, but if we tried to use that in conjunction with this new type definition, we’d just end up with a recursive type definition: StateFromKeys defined as the keys of STATE, StateAsMap defined in terms of the elements of StateFromKeys, and then STATE defined as a StateAsMap…

const STATE: StateAsMap = {
  Pending: 'Pending',
  Active: 'Active',
  Completed: 'Completed',
}

type StateFromKeys = keyof typeof STATE;

type StateAsMap = {
  [K in StateFromKeys]: K
}

You end up with multiple compiler errors here, because of the circular references. This approach won’t work. If we take a step back, though, we can work through this (and actually end up someplace better).

Join forces!

First, let’s start by defining the mapping generically. After all, the idea here was to be able to use this concept all over the place—e.g. for any Redux action, not just one specific one. We don’t need this particular State; we just need a constrained set of strings (or numbers) to be used as the key of an object:

type MapKeyAsValue<Key extends string> = {
  [K in Key]: K
};

In principle, if we didn’t have to worry about the circular references, we could use that to constrain our definition of the original STATE itself:

const STATE: MapKeyAsValue<State> = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
};

So how to get around the problem of circular type definitions? Well, it turns out that the K values in these StateObjectKeyToValue and StateUnionKeyToValue types are equivalent:

// Approach 1, using an object
const STATE = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
};

type StateFromKeys = keyof typeof STATE;
type StateObjectKeyToValue = {
  [K in StateFromKeys]: K  // <- K is just the keys!
};

// Approach 2, using unions
type StateUnion = 'Pending' | 'Started' | 'Completed';
type StateUnionKeyToValue = {
  [K in StateUnion]: K  // <- K is also just the keys!
};

Notice that, unlike the StateObjectKeyToValue version, StateUnionKeyToValue doesn’t make any reference to the STATE object. So we can use StateUnionKeyToValue to constrain STATE, and then just use StateUnion to constrain all the places we want to use one of those states. Once we put it all together, that would look like this:

type StateUnion = 'Pending' | 'Started' | 'Completed';

type StateUnionKeyToValue = {
  [K in StateUnion]: K
};

const STATE: StateUnionKeyToValue = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
};

By doing this, we get two benefits. First, STATE now has to supply the key and value for all the union’s variants. Second, we know that the key and value are the same, and that they map to the union’s variants. These two facts mean that we can be 100% sure that wherever we define something as requiring a State, we can supply one of the items on STATE and it will be guaranteed to be correct. If we change the State union definition, everything else will need to be updated, too.

Now we can make this generic, so it works for types besides just this one set of states—so that it’ll work for any union type with string keys, in fact. (That string-key constraint is important because objects in TypeScript can currently only use strings or numbers as keys; whereas union types can be all sorts of things.) Apart from that constraint on the union, though, we can basically just substitute a generic type parameter U, for “union,” where we had StateUnion before.

type UnionKeyToValue<U extends string> = {
  [K in U]: K
};

Then any object we say conforms to this type will take a union as its type parameter, and every key on the object must have exactly the same value as the key name:

type State = 'Pending' | 'Started' | 'Completed';

// Use `State` as the type parameter to `UnionKeyToValue`.
const STATE: UnionKeyToValue<State> = {
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
}

If any of those don’t have exactly the same value as the key name, you’ll get an error. So, each of the following value assignments would fail to compile, albeit for different reasons (top to bottom: capitalization, misspelling, and missing a letter).

const BAD_STATE: UnionKeyToValue<State> = {
  Pending: 'pending',  // look ma, no capitals
  Started: 'Strated',  // St-rated = whuh?
  Completed: 'Complete',  // so tense
};

You’ll see a compiler error that looks something like this:

[ts]
Type ‘{ Pending: “pending”; Started: “Strated”; Completed: “Complete” }’ is not assignable to type ‘UnionKeyToValue’.
Types of property ‘Pending’ are incompatible.
Type ‘“pending”’ is not assignable to type ‘“Pending”’.

Since the key and the name don’t match, the compiler tells us we didn’t keep the constraint we defined on what these types should look like. Similarly, if you forget an item from the union, you’ll get an error. If you add an item that isn’t in the original union, you’ll get an error. Among other things, this means that you can be confident that if you add a value to the union, the rest of your code won’t compile until you include cases for it. You get all the power and utility of using union types, and you get the utility of being able to use the object as a namespace of sorts.⁶

And the TypeScript language service—which you can use from a lot of editors, including VS Code, Atom, Sublime Text, and the JetBrains IDEs—will actually give you the correct completion when you start definition a type. So imagine we were defining some other union type elsewhere in our program to handle events. Now we can use the same UnionKeyToValue type to construct this type, with immediate, correct feedback from the TypeScript language service:

TypeScript live code completion of the mapped type

By inverting our original approach of using keyof (itself powerful and worth using in quite a few circumstances) and instead using the new mapped types, we get a ton of mileage in terms of productivity when using these types—errors prevented, and speed of writing the code in the first place increased as well.

Yes, it’s a little verbose and it does require duplicating the strings whenever you define one of these types.⁷ But, and this is what I find most important: there is only one source for those string keys, the union type, and it is definitive. If you change that central union type, everything else that references it, including the namespace-like object, will fail to compile until you make the same change there.

Updating a union

So it’s a lot more work than it would be in, say, Elm. But it’s also a lot more guarantees than I’d get in plain-old-JavaScript, or even TypeScript two months ago.

I’ll call that a win.

it’s no surprise that Microsoft’s developer tooling is stronger than Facebook’s↩
along with all the other ML-descended languages I’ve played with, including Haskell, F^♯, PureScript, and Elm.↩
Aside: to be extra safe and prevent any confusion or mucking around, you should probably call Object.freeze() on the object literal, too:
```
const STATE = Object.freeze({
  Pending: 'Pending',
  Started: 'Started',
  Completed: 'Completed',
})
```
Both convention and linters make it unlikely you’ll modify something like this directly—but impossible is better than unlikely.↩
Flow has supported this feature for some time; you can write $Keys<typeof STATE>—but the feature is entirely undocumented.↩
Set your "compilerOptions" key in your tsconfig.json to include "noEmitOnError": true,.↩
For namespacing in a more general sense, you should use… namespaces.↩
It would be great if we could get these benefits without the duplication—maybe someday we’ll have better support in JS or TS natively.↩

Rust development using VS Code on OS X, debugging included

2016-11-18T20:48:00-05:00

Super handy guide for getting a debugging/IDE environment set up for Rust.

Using Rust for ‘Scripting’

2016-11-14T22:00:00-05:00

Edit: fixed some typos, cleaned up implementation a bit based on feedback around the internet.

A lightly edited version of this post was syndicated in Hacker Bits, Issue 13.

I. Using Rust Instead of Python

A friend asked me today about writing a little script to do a simple conversion of the names of some files in a nested set of directories. Everything with one file extension needed to get another file extension. After asking if it was the kind of thing where he had time to and/or wanted to learn how to do it himself (always important when someone has expressed that interest more generally), I said, “Why don’t I do this in Rust?”

Now, given the description, you might think, Wouldn’t it make more sense to do that in Python or Perl or even just a shell script? And the answer would be: it depends—on what the target operating system is, for example, and what the person’s current setup is. I knew, for example, that my friend is running Windows, which means he doesn’t have Python or Perl installed. I’m not a huge fan of either batch scripts or PowerShell (and I don’t know either of them all that well, either).

I could have asked him to install Python. But, on reflection, I thought: Why would I do that? I can write this in Rust.

Writing it in Rust means I can compile it and hand it to him, and he can run it. And that’s it. As wonderful as they are, the fact that languages like Python, Perl, Ruby, JavaScript, etc. require having the runtime bundled up with them makes just shipping a tool a lot harder—especially on systems which aren’t a Unix derivative and don’t have them installed by default. (Yes, I know that mostly means Windows, but it doesn’t solely mean Windows. And, more importantly: the vast majority of the desktop-type computers in the world still run Windows. So that’s a big reason all by itself.)

So there’s the justification for shipping a compiled binary. Why Rust specifically? Well, because I’m a fanboy. (But I’m a fanboy because Rust often gives you roughly the feel of using a high-level language like Python, but lets you ship standalone binaries. The same is true of a variety of other languages, too, like Haskell; but Rust is the one I know and like right now.)

Edit the second: this is getting a lot of views from Hacker News, and it’s worth note: I’m not actually advocating that everyone stop using shell scripts for this kind of thing. I’m simply noting that it’s possible (and sometimes even nice) to be able to do this kind of thing in Rust, cross-compile it, and just ship it. And hey, types are nice when you’re trying to do more sophisticated things than I’m doing here! Also, for those worried about running untrusted binaries: I handed my friend the code, and would happily teach him how to build it.

II. Building a Simple “Script”

Building a “script”-style tool in Rust is pretty easy, gladly. I’ll walk through exactly what I did to create this “script”-like tool for my friend. My goal here was to rename every file in a directory from *.cha to *.txt.

Install Rust.
Create a new binary:
```
cargo new --bin rename-it
```

Add the dependencies to the Cargo.toml file. I used the glob crate for finding all the .cha files and the clap crate for argument parsing.

[package]
name = "rename-it"
version = "0.1.0"
authors = ["Chris Krycho <[email protected]>"]

[dependencies]
clap = "2.15.0"
glob = "0.2"

Add the actual implementation to the main.rs file (iterating till you get it the way you want, of course).

extern crate clap;
extern crate glob;

use glob::glob;
use std::fs;

use clap::{Arg, App, AppSettings};


fn main() {
  let path_arg_name = "path";
  let args = App::new("cha-to-txt")
    .about("Rename .cha to .txt")
    .setting(AppSettings::ArgRequiredElseHelp)
    .arg(Arg::with_name(path_arg_name)
      .help("path to the top directory with .cha files"))
    .get_matches();

  let path = args.value_of(path_arg_name)
    .expect("You didn't supply a path");
  let search = String::from(path) + "/**/*.cha";
  let paths = glob(&search)
    .expect("Could not find paths in glob")
    .map(|p| p.expect("Bad individual path in glob"));

  for path in paths {
    match fs::rename(&path, &path.with_extension("txt")) {
      Ok(_) => (),
      Err(reason) => panic!("{}", reason),
    };
  }
}

Compile it.
```
cargo build --release
```
Copy the executable to hand to a friend.

In my case, I actually added in the step of recompiling it on Windows after doing all the development on macOS. This is one of the real pieces of magic with Rust: you can easily write cross-platform code. The combination of Cargo and native-compiled-code makes it super easy to write this kind of thing—and, honestly, easier to do so in a cross-platform way than it would be with a traditional scripting language.¹

But what’s really delightful is that we can do better. I don’t even need to install Rust on Windows to compile a Rust binary for Windows.

III. Cross-Compiling to Windows from macOS

Once again, let’s do this step by step. Three notes: First, I got pretty much everything other than the first and last steps here from WindowsBunny on the #rust IRC channel. (If you’ve never hopped into #rust, you should: it’s amazing.) Second, you’ll need a Windows installation to make this work, as you’ll need some libraries. (That’s a pain, but it’s a one-time pain.) Third, this is the setup for doing in on macOS Sierra; steps may look a little different on an earlier version of macOS or on Linux.

Install the Windows compilation target with rustup.
```
rustup target add x86_64-pc-windows-msvc
```
Install the required linker (lld) by way of installing the LLVM toolchain.
```
brew install llvm
```
Create a symlink somewhere on your PATH to the newly installed linker, specifically with the name link.exe. I have ~/bin on my PATH for just this kind of thing, so I can do that like so:
```
ln -s /usr/local/opt/llvm/bin/lld-link ~/bin/link.exe
```
(We have to do this because the Rust compiler specifically goes looking for link.exe on non-Windows targets.)
Copy the target files for the Windows build to link against. Those are in these directories, where <something> will be a number like 10586.0 or similar (you should pick the highest one if there is more than one):
- C:\Program Files\Windows Kits\10\Lib\10.0.<something>\ucrt\x64
- C:\Program Files\Windows Kits\10\Lib\10.0.<something>\um\x64
- C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\lib\amd64
Note that if you don’t already have MSVC installed, you’ll need to install it. If you don’t have Visual Studio installed on a Windows machine at all, you can do that by using the links here. Otherwise, on Windows, go to Add/Remove Programs and opting to Modify the Visual Studio installation. There, you can choose to add the C++ tools to the installation.

Note also that if you’re building for 32-bit Windows you’ll want to grab those libraries instead of the 64-bit libraries.
Set the LIB environment variable to include those paths and build the program. Let’s say you put them in something like /Users/chris/lib/windows (which is where I put mine). Your Cargo invocation will look like this:
```
env LIB="/Users/chris/lib/windows/ucrt/x64/;/Users/chris/lib/windows/um/x64/;/Users/chris/lib/windows/VC_lib/amd64/" \
cargo build --release --target=x86_64-pc-windows-msvc
```
Note that the final / on each path and the enclosing quotation marks are all important!
Copy the binary to hand to a friend, without ever having had to leave your Mac.

To be sure, there was a little extra work involved in getting cross-compilation set up. (This is the kind of thing I’d love to see further automated with rustup in 2017!) But what we have at the end is pretty magical. Now we can just compile cross-platform code and hand it to our friends.

Given that, I expect not to be using Python for these kinds of tools much going forward.

Again: you can do similar with Haskell or OCaml or a number of other languages. And those are great options; they are in some ways easier than Rust—but Cargo is really magical for this.↩

What is Functional Programming?

2016-11-11T22:30:00-05:00

The following is a script I wrote for a tech talk I gave on functional programming. The recording isn’t (and won’t be) publicly available; but a script is often easier to reference anyway!

Edit: updated with corrected performance characterstics.

Hello, everyone. Today, we are going to talk about functional programming—asking what it is, and why we should care.

Clearing the Table: Functional Programming’s Reputation

Functional programming has something of a reputation: on the one hand, as incredible difficult, dense, full of mathematical jargon, applicable only to certain fields like machine learning or massive data analysis; on the other hand, as a kind of panacea that solves all of your problems. The reality, I think, is a little bit of both.

The world of functional programming does include a lot of jargon from the math world, and there are good reasons for that, but there is also a lot we could do to make it more approachable to people who don’t have a background in, say category. Category theory is useful, of course, and I think there are times when we might want to be able to draw on it. But gladly, functional programming doesn’t require you to know what an applicative functor is to be able to use it. (And, gladly, there’s a lot of increasingly-solid teaching material out there about functional programming which doesn’t lean on math concepts.)

On the other side, functional programming does give us some real and serious benefits, and that’s what I’m going to spend the first third or so of this talk looking at. But of course, it’s still just a tool, and even though it is a very helpful and very powerful tool, it can’t keep us from writing bugs. Still, every tool we can add to our belt for writing correct software is a win.

One more prefatory note before we get into the meat of this talk: unfamiliar terminology is not specific to functional programming. So, yes, when you see this list, it might seem a little out there:

Functor
Applicative
Monoid
Monad

And in truth, a number of those could have better names. But we have plenty of terminology we throw around in the world of imperative, object-oriented programming. To pick just one, obvious and easy example—what are the SOLID principles?

Single reponsibility
Open/closed
Liskov substitution
Interface segregation
Dependency inversion

You may not remember what it felt like the first time you encountered SOLID, but suffice it to say: “Liskov substitution principle” isn’t any more intuitive or obvious than “Monad”. You’re just familiar with one of them. The same is true of “applicative” and “Visitor pattern”. And so on. Granted, again: it would be nice for some of these things to have easier names, a big part of the pain here is just unfamiliarity.

So, with that out of the way, what is functional programming?

What is functional programming?

Functional programming is a style of programming that uses pure functions and immutable data for as many things as possible, and builds programs primarily out of functions rather than other abstractions. I’ll define all of those terms in a moment, but first…

Why do we care?

We care, frankly, because we’re not that smart. Let’s think about some of the kinds of things we’re doing with, say, restaurant software: clients, with locations, building baskets, composed of products with options and modifiers, which have a set of rules for what combinations are allowed both of products and of their elements as making up a basket, which turn into orders, which have associated payment schemes (sometimes a lot of them), which generate data to send to a point-of-sale as well as summaries for the customer who ordered it, and so on. There are a lot of moving pieces there. I’m sure a missed some non-trivial pieces, too. And if all of that is stateful, that’s a lot of state to hold in your head.

Let me be a bit provocative for a moment. Imagine you were reading a JavaScript module and it looked like this:

var foo = 12;
var bar = 'blah';
var quux = { waffles: 'always' };

export function doSomething() {
  foo = 42;
}

export function getSomething() {
  bar = quux;
  quux.waffles = 'never';
  return bar;
}

Everyone watching would presumably say, “No that’s bad, don’t do that!” Why? Because there is global state being changed by those functions, and there’s nothing about the functions which tells you what’s going on. Global variables are bad. Bad bad bad. We all know this. Why is it bad? Because you have no idea when you call doSomething() or getSomething() what kinds of side effects it might have. And if doSomething() and getSomething() affect the same data, then the order you call them in matters.

In a previous job, I spent literally months chasing a bunch of bugs in a C codebase where all of the state was global. We don’t do this anymore.

But really, what’s different about this?

class AThing {
  constructor() {
    this.foo = 12;
    this.bar = 'blah';
    this.quux = { waffles: 'always' };
  }

  doSomething() {
    this.foo = 42;
  }

  getSomething() {
    this.bar = this.quux;
    this.quux.waffles = 'never';
    return this.bar;
  }
}

We have some “internal” data, just like we had in the module up above. And we have some public methods which change that state. In terms of these internals, it’s the same. There are differences in terms of having instances and things like that, but in terms of understanding the behavior of the system—understanding the state involved—it’s the same. It’s global, mutable state. Now it’s not global like attaching something to the window object in JavaScript, and that’s good, but still: at the module or class level, it’s just global mutable state, with no guarantees about how anything works. And this is normal—endemic, even—in object-oriented code. We encapsulate our state, but we have tons of state, it’s all mutable, and as far as any given class method call is concerned, it’s all global to that class.

You have no idea, when you call a given object method, what it might do. The fact that you call it with an Int and get out a String tells you almost nothing. For all you know, it’s triggering a JSON-RPC call using the int as the ID for the endpoint, which in turn triggers an operation, responds with another ID, which you then use to query a database, and load a string from there, which you then set on some other member of the object instance, and then return. Should you write a method that does that? Probably not. But you can; nothing stops you.

When you call a method, you have no idea what it will do. JavaScript, TypeScript, C^♯, it doesn’t matter. You have literally no idea. And that makes things hard.

It often makes fixing bugs hard, because it means you have to figure out which particular state caused the issue, and find a way to reproduce that state. Which usually means calling methods in a particular order.
It makes testing hard. Again, it often entails calling methods in a particular order. It also means you often need mocks for all those outside-world things you’re trying to do.

Functional programming is an out. An escape hatch. An acknowledgement, a recognition, that holding all of this in our heads is too much for us. No one is that smart. And our software, even at its best, is hard to hold in our heads, hard to make sure that our changes don’t break something seemingly unrelated, hard to see how the pieces fit together—hard, in a phrase you’ll often hear from functional programming fans, hard to reason about.

So, how do we solve these problems? With functional programming!

What is functional programming?

Functional programming is basically combining four bigs ideas:

First class functions
Higher-order functions
Pure functions
Immutable data

The combination of these things leads us to a very different style of programming than traditional OOP. Let’s define them.

First class functions and higher-order functions

We’ll start by looking at the things that are probably most familiar to you if you’re a JavaScript developer (even if you haven’t necessarily heard the names): first-class functions and higher-order functions.

When we talk about first class functions, we mean that functions are just data—they’re first-class items in the language just like any other type. As such, a function is just another thing you can hand around as an argument to other functions. There’s no distinction between a function and a number or a string or some complex data structure. This is essential because, when you combine it with higher-order functions, it allows for incredible simplicity and incredible reusability.

Higher-order functions, in turn, are functions which take other functions as parameters or return them as their values. We’ll see this in detail in a worked example in a few, but for right now, let’s just use a really simple example that will be familiar to anyone who’s done much JavaScript: using map.

If we have a collection like an array and we want to transform every piece of data in it, we could of course do it with a for loop, and with iterable types we could use for ... of. But with map, we can just leave the implementation details of how the items in the array are iterated through, and instead worry about what we want to change. We can do that because map takes functions as arguments.

const initialValues = [1, 2, 3];
const doubledValues = initialValues.map(value => value * 2);

We did it there with a function explicitly, but we could just as easily extract the function like this:

const double = value => value * 2;
const initialValues = [1, 2, 3];
const doubledValues = initialValues.map(double);

This is possible because functions are just data—they’re first-class members of the language—and therefore functions can be arguments or return values—the language supports higher-order functions.

Pure functions

What about pure functions? Pure functions are functions with no effects. The input directly translates to the output, every time. The examples we looked at just a moment ago with map are all pure functions (and it’s a really weird antipattern to use effectful functions with map! Don’t do that! Use forEach if you must have an effect). Here are a few more super simple examples:

const add = (a, b) => a + b;
const toString = (number) => `The value is ${number}`;
const toLength = (list) => list.length;

Here are some examples of straightforward functions which are not pure:

const logDataFromEndpoint = (endpoint) => {
  fetch(endpoint).then(response => {
    console.log(response);
  });
};

let foo = 42;
const setFoo = (newValue) => {
  foo = newValue;
};

const getFoo = () => foo;

So a pure function is one whose output is solely determined by its input That means no talking to a database, no making API calls, no reading from or writing to disk.

And of course, you can’t do anything meaningful with just pure functions. We need user input, and we need to put the results of our computation somewhere. So the goal isn’t to write only pure functions. It’s to write mostly pure functions and to isolate all impure functions.

What this gets us is two things:

A much smaller list of things to worry about when we’re looking at a given function.
The ability to compose functions together more easily.

We have fewer things to keep in our heads when we look at any given pure function, because we don’t have to worry at all about whether something it touches has been changed by another function or not. We have inputs. We transform them into outputs. That’s it. Compare these two things in practice.

Here’s a traditional OOP approach:

class Order {
  constructor() {
    this.subTotal = 0.0;
    this.taxRate = 0.01;
  }

  getTotal() {
    return this.subTotal * (1 + this.taxRate);
  }
}

const order = new Order();
order.subTotal = 42.00;

const total = order.getTotal();

Note that the total is always dependent on what has happened in the object. If we write order.subTotal = 43, order.total will change. So if we want to test how total behaves, or if there’s a bug in it, we need to make sure we’ve made all the appropriate transformations to the object ahead of time. That’s no big deal here; the total getter is incredibly simple (and in fact, we’d normally just write it with a property getter). But still, we have to construct an order and make sure all the relevant properties are set to get the right value out of getTotal(). Things outside the method call itself affect what we get back. We have no way to test getTotal() by itself, and no way to debug it if there’s a bug without first doing some object setup.

Now, here’s a functional approach.

const order = {
  subTotal: 42.0,
  taxRate: 0.01
}

const getTotal = (subTotal, taxRate) => subTotal * (1 + taxRate);
const total = getTotal(order.subTotal, order.taxRate);

Note that the object is just data. It’s a record. And the function just takes a couple of arguments. If there needed to be a more complicated transformation internally, we could do that just as easily. Note that it also decouples the structure of the data from the actual computation (though we could pass in a record as well if we had a good reason to).

This makes it easily testable, for free. Want to make sure different tax rates get the correct output? Just… pass in a different tax rate. You don’t have to do any complicated work setting up an object instance first (which is especially important for more complex data types). It also makes it easier to chase down any bugs: the only thing you have to care about is that simple function body. There’s no other state to think about, because there’s no state at all here from the perspective of the function: just inputs and outputs.

This has one other really important consequence, which goes by the name referential transparency. All that means is that anywhere you see a pure function, you can always substitute the value it produces, or vice versa. This is quite unlike the Order::getTotal() method, where (a) it’s attached to an object instance and (b) it’s dependent on other things about that object. You can’t just substitute it in, or freely move it around, when you’re doing a refactor. Maybe you can, but you’d better hope that all the other state is shuffled around with it correctly. Whereas, with the standalone getTotal() function, all you need is its arguments, and you’ll always get the same thing back.

This is just like math: if you say, x = 5 when solving an algebraic equation, you can put 5 anywhere you see x; or, if it’s useful for factoring the equation or something, you can just as easily put x anywhere you see 5. And in math, that’s true for f(x) as well. When we use pure functions, it’s true for programming, too! That makes refactoring much easier.

As we’ll see in the example I walk through in a minute, it also lets us compose functions together far more easily. If all we have are inputs and outputs, then I can take the output from one function and use it as the input to the next.

Immutable data

Complementing the use of mostly pure functions is to use immutable data. Instead of having objects which we mutate, we create copies of the data as we transform it.

You’re probably wondering how in the world this can work (and also how you avoid it being incredibly computationally expensive). For the most part, we can rely on two things: smart compilers and runtimes, and the fact that we often don’t need to reuse the exact same data because we’re transforming it. However, as we’ll see below, in languages which don’t have native support for immutability, it can impose a performance penalty. Gladly, there are ways to work around this!

A Worked Example

Let’s get down to a real example of these ideas. This is a ‘code kata’ I do every so often. In this particular kata, you get a list of burger orders which looks like this:

[
  { condiments: ['ketchup', 'mustard', 'pickles'] },
  { condiments: ['tomatoes'] },
  { condiments: ['mustard', 'ketchup'] },
  // etc...
]

You’re supposed to take this list (of 10,000-some-odd burger variations!) and determine what the top ten most common orders (not just condiments, but orders) are. (The truth is, the list actually mostly looks like condiments: ['ketchup'] over and over again.) So as a preliminary, you can assume that the data is being loaded like this:

const getBurgers = () =>
  fetch('http://files.example.com/burgers.json')
    .then(request => request.json());

And we’ll print our results (which will always end up in the same format) like this:

const descAndCountToOutput = descAndCount => `${descAndCount[0]}: ${descAndCount[1]}`;

This is actually a perfect case to demonstrate how functional programming ideas can help us solve a problem.

Imperative

First, let’s look at what I think is a relatively reasonable imperative approach. Our basic strategy will be:

Convert condiments to descriptions.
1. Convert the objects to just their lists of condiments.
2. Sort those strings.
3. Turn them into descriptions by joining them with a comma.
Build up a mapping from description to count.
Sort that by count.
Get the top 10.
Print out the results.

getBurgers().then(burgers => {
  let totals = {};

  // 2. Build up a mapping from description to count.
  for (let burger of burgers) {
    // 1. Convert condiments to descriptions.
    // 1.1. Convert the objects to just their lists of condiments.
    const condiments = burger.condiments;
    // 1.2. Sort those strings.
    condiments.sort();
    // 1.3. Turn them into descriptions by joining them with a comma.
    const description = condiments.join(', ');

    // 2. Build up a mapping from description to count.
    const previousCount = totals[description];
    totals[description] = previousCount ? previousCount + 1 : 1;
  }

  // 3. Sort that by count.
  const sortableCondiments = Object.entries(totals);
  sortableCondiments.sort((a, b) => b[1] - a[1]);
  // 4. Get the top 10.
  const topTen = sortableCondiments.slice(0, 10);
  // 5. Print out the results.
  for (let descAndCount of topTen) {
    console.log(descAndCountToOutput(descAndCount));
  }
});

That’s pretty well-factored. But it’s pretty wrapped up on the specific details of this problem, and there’s basically nothing here I could reuse. It’s also relatively hard to test. There aren’t really a lot of pieces there we could break up into smaller functions if we wanted to figure out why something was broken. The way you’d end up fixing a bug here is probably by dropping debugger or console.log() statements in there to see what the values are at any given spot.

And this is where functional programming really does give us a better way.

Functional

Instead of thinking about the specific details of how to get from A to B, let’s think about what we start with and what we finish with, and see if we can build up a pipeline of transformations that will get us there.

We start with a list of objects containing arrays of strings. We want to end up with a list of the distinct combinations and their frequency. How can we do this? Well, the basic idea is the same as what we did above:

Convert condiments to descriptions.
1. Convert the objects to just their lists of condiments.
2. Sort those strings.
3. Turn them into descriptions by joining them with a comma.
Build up a mapping from description to count.
Sort that by count.
Get the top 10.
Print out the results.

To someone acquainted with functional programming, that looks like a bunch of maps, a reduce, and some sorts. And each of those using just simple, pure functions. Let’s see what that might look like. First, what are our transformations?

The step 1 transformations are all quite straightforward:

// 1. Convert condiments to descriptions.
// 1.1. Convert the objects to just their lists of condiments.
const toCondiments = burger => burger.condiments ? burger.condiments : [];
// 1.2. Sort those strings.
const toSortedCondiments = condiments => condiments.concat().sort();
// 1.3. Turn them into descriptions by joining them with a comma.
const toDescriptions = condiments => condiments.join(', ');

Step 2 is a little more involved: it involves building up a new data structure (totals) from an old one. This function is a reducer: it will build up totals by updating totals with each description from an array of them.

// 2. Build up a mapping from description to count.
const toTotals = (totals, description) => {
  const previousCount = totals[description];
  const count = previousCount ? previousCount + 1 : 1;
  totals[description] = count;
  return totals;
};

// 3. Sort that by count.
const byCount = (a, b) => b[1] - a[1];

We’ll see how to get just 10 in a moment; for now, let’s also wrap up the output:

// 5. Print it out
const output = value => { console.log(value); };

These are our base building blocks, and we’ll re-use them in each of the approaches I cover below. Note that we’ve now taken those same basic steps from our imperative approach and turned them into standalone, testable functions. They’re small and single-purpose, which always helps. But more importantly, (with two exceptions we’ll talk about in a minute) all of those transformations are pure functions, we know that we’ll get the same results every time we use them. If I want to make sure that burger condiments are converted correctly, I can test just that function.

describe('toCondiments', () => {
  it('returns an empty list when there is no `condiments`', () => {
    toCondiments({}).should.deepEqual([]);
  });

  it('returns the list of condiments when it is passed', () => {
    const condiments = ['ketchup', 'mustard'];
    toCondiments({ condiments }).should.deepEqual(condiments);
  });
})

This is a trivial example, of course, but it gets the point across: all we have to do to test this is pass in an object. It doesn’t depend on anything else. It doesn’t have any knowledge of how we’re going to use it. It doesn’t know that it’s going to be used with data coming from an array. All it knows is that if you give it an object with a condiments property, it’ll hand you back the array attached to that property.

The result is that, with all of these functions, we don’t have to deal with mocks or stubs or anything like that to be testable. Input produces output. Pure functions are great for this. Now, some of you may be thinking, “That’s great, but what about I/O, or databases, or any other time we actually interact with the world? What about talking to a point-of-sale?” I actually have some tentative thoughts about a future tech talk to look at how to do that in some detail, but for today, just remember that the goal is to write as many pure functions as possible, and to isolate the rest of your code from knowing about that. And of course, that’s best practice anyway! We’re just codifying it. We’ll see what that looks like in practice in just a minute.

Now, while we’re on the topic of pure functions, some of you with quick eyes may have noticed that two of these little functions we laid out are actually not pure: JavaScript’s Array.sort method operates in-place, for performance reasons, and so does our toTotals function. So a truly pure version of the sorting function looks like this:

const toSortedCondiments = condiments => condiments.concat().sort();

Similarly, we could define the toTotals to return a new object every time, like this:

const toTotals = (totals, description) => {
  const previousCount = totals[description];
  const count = previousCount ? previousCount + 1 : 1;
  const update = { [description]: count };
  return Object.assign({}, totals, update);
};

Unfortunately, given the amount of data we’re dealing with, that’s prohibitively expensive. We end up spending a lot of time allocating objects and garbage-collecting them. As a result, it’s tens of thousands of times slower. Running it on my 4GHz iMac, the in-place version takes less than 40ms. Doing it the strictly pure way—returning copies every time—takes ~53s. And if you profile it, almost all of that time is spent in assign (52.95s).

This takes us to an important point, though: it’s actually not a particularly big deal to have this particular data changed in place, because we’re not going to do anything else with it. And in fact, under the hood, this is exactly what pure functional languages do with these kinds of transformations—precisely because it’s perfectly safe to do so, because we’re the only ones who have access to this data. We’re generating a new data structure from the data that we were originally handed, and the next function will make its own new data structure (whether a copy or something else).

In other words, when we’re talking about a pure function, we don’t really care about internal mutability (though of course, that can bite us if we’re not careful). We’re really concerned about external mutability. As long as the same inputs get the same outputs every time, the rest of the world doesn’t have to care how we got that result.

Now let’s see how we use these functions.

Pure JavaScript

First, here’s a pure-JavaScript approach, but a more functional one instead of an imperative one:

getBurgers().then(burgers => {
  const descriptionToCount = burgers
    .map(toCondiments)
    .map(toSortedCondiments)
    .map(toDescriptions)
    .reduce(toTotals, {})

  const entries = Object.entries(descriptionToCount);

  [...entries]
    .sort(byCount)
    .slice(0, 10)  // 4. Get the top 10.
    .map(descAndCountToOutput)
    .forEach(output);
});

First, the good: our transformation is no longer all jumbled together. In fact, our code reads a lot like our original description did. Also, notice that we just have a bunch of functions operating on data: none of the functions used here have any knowledge about where the data comes from that they operate on.

But then we also have a couple things that are a little bit clunky. The main thing that sticks out is that sudden stop in the chain in the middle.

When we’re dealing with the Array type, everything is fine, but when we convert our data into a Map, we no longer have that option, so we have to jump through some hoops to do the transformation back into the data type we need. We’re stuck if the object type doesn’t have the method we need. We’re kind of trying to mash together the imperative and functional styles, and it’s leaving us in a little bit of a weird spot.

There’s another issue here, though, and it’s the way that using the method-style calling convention obscures something important. When we call most of those methods, we’re doing something quite different from what most methods do. A method normally is an operation on an object. These methods—most of them—are operations that return new objects. So it’s nice from a syntax perspective, but if we’re not already familiar with the behavior of a given method, it won’t be clear at all that we’re actually generating a bunch of totally new data by calling those methods.

And… two of these methods (sort and forEach) are not doing that, but are modifying an array in place instead.

Lodash

The first step away from this problem is to use a tool like Lodash.

// More functional, with _:
// We tweak how a few of these work slightly to play nicely.
const _toDescriptions = condiments => _.join(condiments, ', ');
const _byCount = _.property(1);

getBurgers().then(burgers => {
  const condiments = _.map(burgers, toCondiments);
  const sortedCondiments = _.map(condiments, toSortedCondiments);
  const descriptions = _.map(sortedCondiments, _toDescriptions);
  const totals = _.reduce(descriptions, toTotals, {});
  const totalPairs = _.toPairs(totals);
  const sortedPairs = _.sortBy(totalPairs, _byCount);
  const sortedPairsDescending = _.reverse(sortedPairs);
  const topTen = _.take(sortedPairsDescending, 10);
  const forOutput = _.map(topTen, descAndCountToOutput)
  _.forEach(forOutput, output);
});

But it seems like we lost something when we moved away from the object-oriented approach. Being able to chain things, so that each item worked with the previous item, was actually pretty nice. And needing all these intermediate variables is not so nice.

One way around this is to use Lodash’s _.chain method. That would have let us write it like this:

getBurgers().then(burgers => {
  const foo = _.chain(burgers)
    .map(toCondiments)
    .map(toSortedCondiments)
    .map(_toDescriptions)
    .reduce(toTotals, {})
    .toPairs()
    .sortBy(_byCount)
    .reverse()
    .take(10)
    .map(descAndCountToOutput)
    .value()
    .forEach(output);
});

And that is a win. But it only works because JavaScript is incredibly dynamic and lets us change the behavior of the underlying Array type. (You’d have a much harder time doing that in Java or C^♯!)

Perhaps just as importantly, it requires us to make sure that we do that _.chain() call on on anything we want to tackle this way. So, can we get the benefits of this some other way? Well, obviously the answer is yes because I wouldn’t be asking otherwise.

With Ramda.

But we can actually go a bit further, and end up in a spot where we don’t need to modify the object prototype at all. We can just do this with a series of standalone functions which don’t depend on being attached to any object. If we use the Ramda library,¹ we can tackle this with nothing but functions.

const getTop10Burgers = R.pipe(
  R.map(R.prop('condiments')),
  R.map(R.sortBy(R.toString)),
  R.map(R.join(', ')),
  R.reduce(toTotals, {}),
  R.toPairs,
  R.sortBy(R.prop(1)),  // will give us least to greatest
  R.reverse,
  R.take(10),
  R.map(descAndCountToOutput)
);

return getBurgers()
  .then(getTop10Burgers)
  .then(R.forEach(output));

Notice the difference between here and even where we started with Lodash: we’re no longer dependent on a specific piece of data being present. Instead, we’ve created a standalone function which can operate on that data, simply by “piping” together—that is, composing—a bunch of other, smaller functions. The output from each one is used as the input for the next.

One of the many small niceties that falls out of this is that we can refactor this just by pulling it apart into smaller acts of compositions.

Here’s an example of how we might use that. We defined those simple transformations for the condiments as a set of three functions, which converted them from objects with condiments elements, sorted them, and joined them into a string. Now, let’s build those into meaningful functions for each step:

// 1. Convert condiments to descriptions.
const burgerRecordsToDescriptions = R.pipe(
  R.map(R.prop('condiments')),
  R.map(R.sortBy(R.toString)),
  R.map(R.join(', ')),
);

// 2. Build up a mapping from description to count.
const descriptionsToUniqueCounts = R.pipe(
  R.reduce(toTotals, {}),
  R.toPairs,
);

// 3. Sort that by count.
const uniqueCountsToSortedPairs = R.pipe(
  R.sortBy(R.prop(1)),
  R.reverse,
);

// For (4), to get the top 10, we'll just use `R.take(10)`.
// We could also alias that, but it doesn't gain us much.

// 5. Print it out
const sortedPairsToConsole = R.pipe(
  R.map(descAndCountToOutput),
  R.forEach(output)
);

Then we can put those together into another, top-level function to do exactly our steps.

const getTop10Burgers = R.pipe(
  burgerRecordsToDescriptions,  // (1)
  descriptionsToUniqueCounts,   // (2)
  uniqueCountsToSortedPairs,    // (3)
  R.take(10)                    // (4)
);

getBurgers()
  .then(getTop10Burgers)
  .then(sortedPairsToConsole);  // (5)

Notice that, because each step is just composing together functions, “refactoring” is easy. And, to be sure, you have to be mindful about what comes in and out of each function. But that’s true in the imperative approach, too: you always have to keep track of the state of the object you’re building up, but there you’re doing it in the middle of a loop, so you’re keeping track of a lot more state at any given time. Functions with simple inputs and outputs give us a more explicit way of specifying the structure and state of the data at any given time. That’s true even in JavaScript, but it goes double if we’re in a typed language like F^♯, Elm, etc., where we can specify those types for the function as a way of designing the flow of the program. (That’s such a helpful way of solving problems, in fact, that I may also do a talk on type-driven design in the future!)

Note, as well, that we’ve now completely isolated our input and output from everything else. The middle there is a chain of pure functions, built out of other pure functions, which neither know nor care that the data came in from an API call, or that we’re going to print it to the console when we finish.

So this takes us back around to that first question: why do we care? At the end of the day, is this really a win over the imperative style? Is the final version, using Ramda, really better than the pure-JavaScript mostly-functional version we used at first?

Obviously, I think the answers there are yes. The Ramda version there at the end is way better than the imperative version, and substantially better than even the first “functional” JavaScript versions we wrote.

For me, at least, the big takeaway here is this: we just built a small but reasonable transformation of data out of a bunch of really small pieces. That has two big consequences—consequences we’ve talked about all along the way, but which you’ve now seen in practice:

Those pieces are easy to test. If something isn’t working, I can easily take those pieces apart and test them individually, or test the result of any combination of them. As a result, I can test any part of that pipe chain, and I can fix pieces independent of each other. No part depends on being in the middle of a looper where transformations are done to other parts.
Because they’re small and do one simple things, I can recombine those pieces any way I like. And you see that in the Ramda examples in particular: most of what we’re doing in those examples is not even something we wrote ourselves. They’re also really basic building blocks, available in basically every standard library.

One last thing: if you’re curious about performance… you should know that it does matter for data at scale. In my tests (which are admittedly extremely unscientific; unfortunately, I couldn’t get JSPerf running nicely with this particular set of variations), I found that the time it took to run these varied depending on the approach and the library. With a ~10k-record data set:

The imperative version, unsurprisingly, was the fastest, taking ~16–17ms.
After that, the chained lodash version and the pure-JS version were comparable, at ~32–36ms, or around twice as long to finish as the imperative version.
The plain lodash version was consistently a little slower yet, at ~38–43ms.
Ramda is slow: both variations consistently took over 90ms to finish.

Those differences added up on larger data sets: dealing with ~10,000,000 records, the times ranged from ~12s for the imperative version, to ~15s for the lodash and pure-JS variants, to ~50s for the Ramda version.

They were all pretty darn quick. Compilers, including JavaScript JITs, are incredibly smart. Mostly you can just trust them; come back and profile before you even think about optimizing things. But you should know the performance characteristics of different libraries and consider the implications of what the language does well and what it doesn’t. Ramda is likely slower because of the way it curries every function—something that works well in languages with native support for it, e.g. F^♯ or Elm or Haskell, but imposes a penalty in languages which don’t… like JavaScript. That said, if you’re not in the habit of processing tens of thousands of records, you’re probably okay using any of them.

or lodash-fp, but Ramda is a bit better documented and I just like it a little better↩

Why Everything is Broken

2016-11-01T20:45:00-04:00

It’s something of a joke among many of the software developers I know to wonder aloud how anything works. We’re all very painfully aware of how hard it is to write correct code, of how hard it is to account for all the corner cases that will arise, and of how hard it is to write user interfaces (of any sort) that make good sense to the user.

And our assumptions are broken in weird ways, but we don’t even realize it. Our paradigms for computing are built on decisions made 40–50 years ago, and in many cases there is no good reason to continue doing things that way in a vacuum. But we’re not in a vacuum, and we have incredible resources built on top of those existing paradigms, and rewriting everything form scratch in a saner way with the lessons we’ve learned in the intervening years seems impossible.

All of this came home to me again this evening in one of those startlingly painful moments of realization at how ridiculous this stack of cards we’ve built really is.

I was helping a colleague, a designer who’s been learning HTML and CSS, figure out why his page wasn’t displaying properly on GitHub Pages. The site was loading, and the image assets were loading, but the style sheets weren’t. In fairly short order, I pulled up the site, noticed that the paths were to /css/style.css, glanced at the source and noted that the actual content was at /CSS/style.css, and said: “Oh, you just need to make the case of these match!” I explained: the URL proper (everything up through .com or whatever domain ending) is case-insensitive, but everything after that is case-sensitive.

There are reasons for that, some historical and some having to do with the fact that you can just serve a web page directly from a server, so the paths on your file system map to the paths on the web. And if your file system is case-sensitive, then the URL has to respect that.

That is, in a word, dumb. Don’t get me wrong: again, I see the perfectly defensible technical reasons why that is so. But it’s a leaky abstraction. And when you look closely at them, nearly all of our abstractions leak, and badly.

The pain of that moment was realizing, that like so many other things in tech, this particular thing is still broken for two reasons:

It’s too hard or too painful to change it. (That’s a big one here; the web has a pretty firm commitment to absolute backwards compatibility forever, modulo a few things like killing Flash.)
We get used to it, and just come to accept the ten thousand papercuts as normal, and eventually even forget about them until something comes up again and forces us to see them. Usually in the form of someone learning for the first time.

We can’t necessarily do a lot about (1). We don’t have infinite time or money, and reinventing everything really is impossible. We can do wacky experiments, and iterate toward better solutions that can gradually replace what was there originally.

But (2) is the bigger one. We need to stop accepting the papercuts as just part of how things are—and especially, stop seeing our acclimation to them as a badge of honor to be earned—and start treating them as rough edges that ought to be sanded off over time wherever possible. Notice the things that trip up new learners, and if you can, get rid of them. If you can’t get rid of them, make note so that you are sure to cover it when you’re helping someone in the future. And explain the whys for those little edge cases: even worse than not knowing them, in some ways, is knowing them but not understanding them—having a bag of little tricks you can use but never being able to progress because you can’t see how they fit together.

Making our tech better starts, in many ways, with recognizing the problems we have. It requires us not to accept (much less embrace or revel in) the status quo, and always to push ourselves to do better. So iterate like made to get away from (1) and to fix (2).

Rust vs. React Native—What?

2016-10-07T08:20:00-04:00

I was recently discussing some thoughts I’ve had on building a top-notch application experience in a Slack team I belong to, and noted that I believe that a Rust core with native UIs is a massively winning strategy. A friend in the group responded that he thinks “React + JS is eating the world right now” and that “Rust as awesome for if you want to write a JS vm, or something like that… or a compiler… anything involving lots of speed and stability.” What follows is my response, lightly edited to remove details specific to that friend and to add a few further thoughts.

Here’s the thing: I don’t care what’s eating the world today, for three reasons:

I just want to build the best stuff I can build, and native UIs are still massively better than React and even React Native¹ in innumerable ways. There are clear advantages to React Native + JavaScript, and times when you absolutely should take that approach. But there are also a lot of times and reasons why you shouldn’t. Heck, even if you just want killer performance in browsers, our future includes things like Rust-to-WebAssembly, and that’s a good thing.

What was eating the world five years ago? Ten? Is it still eating the world today? I don’t feel obliged to follow those trends (not least because, not being a consultancy, following those trends doesn’t buy me anything for the things I want to do; your tradeoffs and mine look way different).

I’m actually getting really tired of just treating as acceptable or normative the performance characteristics of browsers. Browsers are awesome. But we can (and should) do a lot better in terms of user experience, and I don’t see browsers catching up to what you can do with e.g. Cocoa (Touch). Sure, that doesn’t matter that much for building yet-another-storefront. (Again, there are different tradeoffs for every single app!) But why in the world are we in a spot now where one of the most popular text editors in the world is slower than any text editor of five years ago? That’s not a necessary decision, and you can (and should) go after the same degree of ease-of-extensibility that Atom has had—perhaps even using things like HTML and CSS for skinning!—while not tying yourself to the browser and its upsides and downsides for everything. We have incredibly powerful machines, and the user experience is often getting slower. I’m looking for ways to change that.

Again, JS+React² may be exactly the right tradeoff for a lot of apps, and given what consultancies (like my friends’s!) are doing, I think doing that with ReactNative for apps is a very good move. It makes good sense business-wise, and it makes good sense in terms of the apps you’re likely to be delivering. Don’t hear me for a second saying Rust is the best for everything. I think it, or something like it, is a very good choice for many things, though, and it shouldn’t be dismissed simply because it’s a very different world from doing Ruby or Elixir or JavaScript.

So much for my initial response. On reflection, I wanted to expand it a bit. So here’s another few hundred words!

Beyond this, I think there’s a bit of a false dichotomy here: the idea that “lots of speed and stability” aren’t values we should be seeking more aggressively for all our apps. Fully granted that not every app needs the same degree of each of those, and moreover that there are a lot of ways to get to those goals. Still: speed and stability are core user experience values. I don’t really care how you get at those goals, whether it’s with Rust, or Elixir or Clojure, or, yes, React with TypeScript or Flow. I do think that Rust is, for the moment at least, uniquely positioned to add real value in this space because it gives screaming performance but with so many niceties we’re used to when writing languages like Python or Ruby and so much of the power you get in languages like OCaml or F♯.³ But at the end of the day, I think all apps should focus much more on speed and stability than they do today. We have supercomputers in our pockets, and we’re often shipping apps that are slower and more finicky.

But I have this dream of a world where apps aren’t needlessly power-hungry or memory-intensive, where every swipe and or click or scroll results in buttery-smooth responses. We won’t get there by saying, “You know, Facebook is doing x so that’s good enough for me.”

Of course every developer, and any given product shop or consultancy, is going to have to make decisions about which stacks it invests in. If you’re primarily shipping web applications, investing in Elixir and React with React Native for your apps is a very sensible move. Most of your clients’ native apps may not need the degree of polished performance you might get from writing their iOS app in Swift and their Android app in Kotlin and the core in Rust (or even C++). That tradeoff is a tradeoff.

But let’s remember that there is real value there, and that some apps do deserve that investment. We should evaluate the tradeoffs at every turn, and our core considerations should enduringly include speed and stability. Don’t dismiss Rust (or Swift, or F♯) out of hand.

Equally importantly, we need to stop assuming that just because something is eating the world today means it’s also the future. Betting big on Flash in the mid-2000s wasn’t a bad move by a long shot. But its massive popularity then wasn’t a good predictor for its future. That goes double, frankly, for projects coming out of Facebook or Google or similar: big companies like that have the resources to drop everything and use a new language, or a new tool, as it suits them. If you don’t believe me, look at the actual open-source records of both of those companies! What’s hot today is far more relevant to a consultancy than to a product shop. And in both cases, choosing tech suitable for the job at hand is more important yet.

My friend gets that, for what it’s worth. He’s making the right moves for his business as the owner of a consultancy. I just want him—and lots of other people—to see where languages like Rust and Swift and F♯ might be worth considering. And speed and stability matter in a lot of places besides just compilers and VMs.

I’m aware that React-Native ultimately binds down to native widgets. It’s still not quite the same.↩
or, frankly, Ember or whatever else; React is great, but it is also overhyped.↩
Swift too, and honestly for a lot of things Swift is an easier experience for not that much less performance than Rust. But as of today you can’t ship core functionality in Swift for Android or Windows.↩

32 Theses (and several more words) on Podcasting

2016-08-09T10:30:00-04:00

A month ago, Alan Jacobs asked about quality conservative Christian podcasts. Here’s a big part of why there are so few (at Mere Orthodoxy):

As a Christian in the world of podcasting—I have both a “two dudes talking” show (Winning Slowly) and also a “one dude talking with maybe a brief musical intro and outro” show (New Rustacean)—I found much to agree with, but also much to clarify and a few things to disagree with…

…

First, a set of theses on podcasting as a medium. Some of these are obvious; none are intended to be tendentious. Some of them warrant further explanation—for which, see below….

After which, 32 theses (and another ~3,000 words) on the constraints and challenges of podcasting as a medium.

Aside: the format of this particular piece is heavily inspired by Jacobs’ own “79 Theses on Technology. For Disputation.”

Rust and Swift (xviii)

2016-07-24T15:10:00-04:00

I am reading through the Swift book, and comparing it to Rust, which I have also been learning over the past few months. As with the other posts in this series, these are off-the-cuff impressions, which may be inaccurate in various ways. I’d be happy to hear feedback! Note, too, that my preferences are just that: preferences. Your tastes may differ from mine. (See all parts in the series.)

Part I: Ownership Semantics vs. Reference Counting

Perhaps unsurprisingly, the Swift book follows on from its discussion of initialization with a discussion of deinitialization, and here the differences between Rust and Swift are substantial, but (as has so often been the case) so are the analogies.

In Rust, memory is, by default, stack-allocated and -deallocated, but with a very impressive system for tracking the lifetime of that data and allowing its to be moved from one function to another. The Rust compiler tracks the ownership of every given item in the program as it is passed from one function to another, allowing other parts of the program to “borrow” the data safely, until a given piece of data goes out of scope entirely. At that point, Rust runs its destructors automatically. As part of its system for managing memory safely, Rust also tracks where and when a program attempts to access any given piece of data (whether directly or via reference), and will refuse to compile if you try to reference data in a place where it has already gone out of scope and been cleaned up (“dropped,” in Rust-speak).

If this was a bit fuzzy, don’t worry: there’s a lot to say here. It’s arguably the most distinctive feature of the language, and it’s also the main thing that tends to trip up newcomers to the language. If you’re interested in further material on the topic, my own most succinct treatment of it is in an early episode of New Rustacean, my Rust developer podcast, and the official documentation is very good. For now, suffice it to say: Rust does extremely rigorous compile-time checks to let you do C or C++-style memory management, but with absolute guarantees that you won’t have e.g. use-after-free bugs, with a default to handling everything on the stack.

It is of course impossible to handle everything on the stack, so there are heap-allocated types (e.g. vectors, a dynamically sized array-like type), which are fundamentally reference (or pointer) types. But those follow the same basic rules: Rust tracks the pointers throughout their uses, and when they go out of scope, Rust automatically tears down not only the pointer but also the data behind it. There are times, though, when you can’t comply with Rust’s normal rules for handling multiple-access to the same data. For those situations, it also supplies some “smart pointer” container types, Rc and Arc, the reference-counted (non-thread-safe) and atomically reference-counted (thread-safe) types. Both types just wrap up a type that you intend to put on the heap with reference-counters, which increment and decrement as various pieces of a program get access to them. Note that, unlike the compiler-level, compile-time checks mentioned earlier, these are run-time counts and they therefore incur a small but real runtime performance penalty. (The distinctions between the two types have to do with how they guarantee their memory safety and what kinds of a guarantees are required for cross-thread safety, and they’re important for writing Rust but not so important for this comparison, so I’ll leave them aside.¹)

In Swift, all class instances (which are pass-by-reference types) are tracked with automatic reference counting and cleaned up automatically when there are no more references to them. Don’t confuse Rust’s “atomically reference-counted” type with Swift’s “automatically reference-counted” type. Unlike Rust’s behavior in having everything checked at compile-time, reference counting is a run-time check in Swift, just as it is with the Rc and Arc types in Rust.² But it happens for all reference types all the time in Swift, not just when specified manually as in Rust. (Value types seem to be always passed by value, though the compiler has some smarts about that so it doesn’t get insanely expensive.) It’s automatic in that the compiler and runtime handle it “behind the scenes” from the developer’s perspective.

Swift’s approach here isn’t quite the same as having a full-on garbage-collected runtime like you’d see in Java, C^#, Python, Ruby, JavaScript, etc. (and so doesn’t have the performance issues those often can). But it also isn’t like Rust’s default of having no runtime cost. It’s somewhere in the middle, with a goal of very good performance but good developer ergonomics. I think it achieves that latter goal: for the most part, it means that you don’t have to think about memory allocation and deallocation explicitly. Certainly there are times when you have to think about how your program handles those issues, but neither is it right up in your face like it is in Rust,³ nor does it come with the costs of a heavier runtime (from startup, to GC pauses, to non-deterministic performance characteristics).⁴

To make it concrete, the following snippets do basically the same thing—but note that the reference counting is explicit in Rust. We’ll start with Rust, doing it the normal way:

struct WouldBeJedi {
    name: String,
    rank: u8,
    description: String,
}

impl WouldBeJedi {
    fn new(name: &str, rank: u8, description: &str) -> WouldBeJedi {
        WouldBeJedi {
            name: name.to_string(),
            rank: rank,
            description: description.to_string()
        }
    }
}

fn main() {
    let trainee = WouldBeJedi::new(
        "Zayne Carrick", 1, "not very competent, but still a great hero");

    // When calling the function, we pass it a reference, and it
    // "borrows" access to the data. But the validity of that access
    // is checked at compile time. `main()` keeps the "ownership"
    // of the data.
    describe(&trainee);

    // When `main` ends, nothing owns the data anymore, so
    // Rust cleans it up. If something were still borrowing the
    // data (say, if we'd passed a reference into another thread),
    // this would actually be a compile error, because references
    // have to be guaranteed to live as long as the thing they
    // point back to. Rust has tools for managing that, as well,
    // its "lifetimes", but we can leave them aside for this example.
}

fn describe(trainee: &WouldBeJedi) {
    // Rust checks at compile time to make sure there are no
    // mutable "borrows" of the data, and therefore knows
    // that it is safe to reference the data here, because it can
    // be *sure* nothing will change it at the same time.

    // Under the covers, this macro will actually call a
    // function with the data we pass it, so Rust actually checks
    // the ownership and borrowing state here, too. Again, all
    // at compile time, and therefore with no runtime penalty.
    println!("{} (rank {}) is {}.",
             trainee.name,
             trainee.rank,
             trainee.description);

    // When we exit the function, Rust notes that it is no
    // longer "borrowing" the data.
}

And here’s the Swift code—note as well that we use a class not a struct here:

class WouldBeJedi {
    let name: String
    let rank: UInt8
    let description: String

    init(name: String, rank: UInt8, description: String) {
        self.name = name
        self.rank = rank
        self.description = description
    }
}

func main() {
    let aTrainee = WouldBeJedi(
        name: "Zayne Carrick",
        rank: 1,
        description: "not very competent, but a great hero")

    // When calling the function, the reference count goes up
    // here, too, but it's implicit, rather than explicit.
    describe(aTrainee)

    // The implicit reference count Swift maintains for `aTrainee`
    // will go from 1 to 0 here, and Swift will do its cleanup of the
    // object data.
}

func describe(_ trainee: WouldBeJedi) {
    // When we enter this function, Swift bumps the reference
    // count, from 1 to 2. Both `main` and `describe` now have a
    // reference to the data.

    // No need for the unwrapping or any of that; Swift handles it
    // all automatically... thus the name of the technology!
    print("\(trainee.name) (rank \(trainee.rank)) is \(trainee.description).")

    // When we exit the function, Swift bumps the reference count
    // back down to 1 automatically.
}

Finally, here is the (much longer, because all the reference counting is done explicitly) reference-counted Rust version:

use std::rc::Rc;

pub struct WouldBeJedi {
    name: String,
    rank: u8,
    description: String,
}

fn main() {
    let trainee = WouldBeJedi {
        name: "Zayne Carrick".to_string(),
        rank: 1,
        description: "not very competent, but a great hero".to_string()
    };
    let wrapped_trainee = Rc::new(trainee);

    // Start by calling `clone()` to get a *reference* to the
    // trainee. This increases the reference count by one.
    let ref_trainee = wrapped_trainee.clone();
    // Then pass the reference to the `describe()` function.
    // Note that we *move* the reference to the function, so
    // once the function returns, the reference will go out
    // of scope, and the reference count will decrement.
    describe(ref_trainee);

    // When `main` ends, several things will happen in order:
    // 1. The reference count on the `wrapped_trainee` will
    //    go to zero. As a result, the `wrapped_trainee`
    //    pointer---the `Rc` type we created---will get
    //    cleaned up.
    // 2. Once `wrapped_trainee` has been cleaned up, Rust
    //    will notice that there are no more references
    //    anywhere to `trainee` and clean it up as well.
    //    (More on this below.)
}

fn describe(trainee: Rc<WouldBeJedi>) {
    // We now have a *reference* to the underlying data, and
    // therefore can freely access the underlying data.
    println!("{} (rank {}) is {}.",
             trainee.name,
             trainee.rank,
             trainee.description);


    // When we exit the function, Rust destroys this *owned*
    // clone of the reference, and that bumps the reference
    // count back down to 1 automatically.
}

Note that if we strip out all the explanatory comments and details, the normal versions of the Rust and Swift code are pretty similar.

Rust—

struct WouldBeJedi {
    name: String,
    rank: u8,
    description: String,
}

impl WouldBeJedi {
    fn new(name: &str, rank: u8, description: &str) -> WouldBeJedi {
        WouldBeJedi {
            name: name.to_string(),
            rank: rank,
            description: description.to_string()
        }
    }
}

fn main() {
    let trainee = WouldBeJedi::new(
        "Zayne Carrick",
        1,
        "not very competent, but still a great hero");

    describe(&trainee);
}

fn describe(trainee: &WouldBeJedi) {
    println!("{} (rank {}) is {}.",
             trainee.name,
             trainee.rank,
             trainee.description);
}

Swift (as usual, is slightly briefer than Rust)—

class WouldBeJedi {
    let name: String
    let rank: UInt8
    let description: String

    init(name: String, rank: UInt8, description: String) {
        self.name = name
        self.rank = rank
        self.description = description
    }
}

func main() {
    let aTrainee = WouldBeJedi(
        name: "Zayne Carrick",
        rank: 1,
        description: "not very competent, but a great hero")

    describe(aTrainee)
}

func describe(_ trainee: WouldBeJedi) {
    print("\(trainee.name) (rank \(trainee.rank)) is \(trainee.description).")
}

Note that in both of these implementations, all the actual cleanup of the memory is handled behind the scenes—this feels much more like writing Python than writing C, especially for complex data types. Not least because this same kind of nice cleanup can happen for complex, heap-allocated types like dynamically-sized vectors/arrays, etc. Both languages just manage it automatically. (The same is true of modern C++, for the most part, but it has a more complicated story there because of its relationship with C, where malloc and free and friends run rampant and are quite necessary for writing a lot of kinds of code.) Most of the time, when you’re done using data, you just stop using it, and both Rust and Swift will clean it up for you. The feel of using either language is fairly similar, though the underlying semantics are quite different.

Part 2: Deconstruction/Deinitialization

Both Rust and Swift recognize that, the ordinary case notwithstanding, there are many times when you do need to run some cleanup as part of tearing down an object. For example, if you had an open database connection attached to an object, you should return it to the collection pool before finishing tear-down of the object.

In Rust, this is accomplished by implementing the Drop trait and supplying the requisite drop method. Imagine we had defined a Jedi type, with a bunch of details about the Jedi’s lightsaber (including whether the Jedi even has a lightsaber. We know from the Star Wars movies that lightsabers turn off automatically when the Jedi dies, or even just drops it for that matter. We can implement all of this in Rust using just the Drop trait. Here’s a pretty full example.⁵ (Note that both of these implementations draw heavily on material I covered in previous posts.)

#[derive(Debug)]
enum Color {
    Red,
    Blue,
    Green,
    Purple,
    Yellow
}

enum SaberState {
    On,
    Off,
}

struct Lightsaber {
    color: Color,
    blades: u8,
    state: SaberState
}

impl Lightsaber {
    pub fn new(color: Color, blades: u8) -> Lightsaber {
        if blades > 2 {
            panic!("That's just silly. Looking at you, Kylo.");
        }

        Lightsaber { color: color, blades: blades, state: SaberState::Off }
    }

    pub fn on(&mut self) {
        self.state = SaberState::On;
    }

    pub fn off(&mut self) {
        self.state = SaberState::Off;
    }
}

struct WouldBeJedi {
    name: String,
    lightsaber: Option<Lightsaber>,
}

impl WouldBeJedi {
    pub fn new(name: &str, lightsaber: Option<Lightsaber>) -> WouldBeJedi {
        WouldBeJedi { name: name.to_string(), lightsaber: lightsaber }
    }

    pub fn describe(&self) {
        let lightsaber = match self.lightsaber {
            Some(ref saber) =>
                format!("a {:?} lightsaber with {:} blades.", saber.color, saber.blades),
            None => "no lightsaber.".to_string()
        };

        println!("{} has {}", self.name, lightsaber)
    }
}

// Here's the actually important bit.
impl Drop for WouldBeJedi {
    fn drop(&mut self) {
        if let Some(ref mut lightsaber) = self.lightsaber {
            lightsaber.off();
        }
    }
}


fn main() {
    let saber = Lightsaber::new(Color::Yellow, 1);
    let a_jedi = WouldBeJedi::new("Zayne Carrick", Some(saber));
    a_jedi.describe();
}

We can do much the same in Swift, using its deinitializers, which are fairly analogous to (but much simpler than) its initializers, and fulfill the same role as Rust’s Drop trait and drop() method.

enum Color {
    case red, blue, green, purple, yellow
}

enum SaberState {
    case on, off
}

struct Lightsaber {
    let color: Color
    let blades: UInt8
    var state: SaberState = .off

    init?(color: Color, blades: UInt8) {
        if blades > 2 {
            print("That's just silly. Looking at you, Kylo.")
            return nil
        }

        self.color = color
        self.blades = blades
    }

    mutating func on() {
        state = .on
    }

    mutating func off() {
        state = .off
    }
}

class WouldBeJedi {
    let name: String
    var lightsaber: Lightsaber?

    init(name: String, lightsaber: Lightsaber?) {
        self.name = name
        self.lightsaber = lightsaber
    }

    deinit {
        self.lightsaber?.off()
    }

    func describe() {
        let saberDescription: String
        if let saber = self.lightsaber {
            saberDescription = "a \(saber.color) lightsaber with \(saber.blades) blades."
        } else {
            saberDescription = "no lightsaber."
        }

        print("\(name) has \(saberDescription)")
    }
}

func main() {
    let saber = Lightsaber(color: .yellow, blades: 1)
    let aJedi = WouldBeJedi(name: "Zayne Carrick", lightsaber: saber)
    aJedi.describe();
}

This is a bit briefer, but that’s mostly down to Swift’s shorthand for optionals (the ? operator), which we’ll get to in a future post.

Curiously, struct and enum types cannot have deinitializers in Swift. I expect this has something to do with their being value types rather than reference types, but the book offers no comment. (If a reader knows the answer, I’d welcome clarification.)

Much as in the discussion of of initializers, the usual patterns with Rust and Swift’s approach come into play. Rust opts to build the pattern on the same basic language machinery (traits). Swift uses a bit of syntactical sugar dedicated to the purpose. It’s undeniable that the Swift is a bit briefer.

However, there are a couple upsides to Rust’s approach. First, it is applicable on all types, where Swift’s applies only to classes. Second, there is no additional syntax to remember. Drop is just a trait like any other, and drop a method like any other. Third, then, this means that you can run it explicitly elsewhere if you need to, and as a result you can define whatever kind of custom deconstruction behavior you might need. If we’d created a_jedi above in Rust, we could simply write a_jedi.drop() anywhere:

fn prove_incompetent(a_jedi: WouldBeJedi) {
    // make some series of grievous mistakes which mean
    // you're no longer able to be a Jedi and as such,
    // among other things, lose your lightsaber...
    a_jedi.drop();
    // other stuff
}

Or (going a bit more abstract) we could define a daring_derring_do() method which called drop() itself:

impl WouldBeJedi {
    pub fn daring_derring_do(self) {
        // do some other operation, like freeing slaves from
        // a secret colony of slavers. But if it fails...
        self.drop();
    }
}

Or, really, define any behavior which culminated in a drop() call. That’s extremely powerful, and it’s the upside that comes with its just being a trait whose behavior we have to define ourselves.

That takes us back to one of the fundamental differences in design between the two languages. Rust goes out of its way to leave power in the hands of the user, at the cost of requiring the user to be a bit more explicit. Swift prioritizes brevity and productivity, but it gets there by taking some of the power out of the hands of the developer. Neither of these is wrong, per se. They’re just aiming for (and in this case, I think, fairly successfully landing in) somewhat different spots on a spectrum of tradeoffs.

Previous: More on initializers!

I did, however, cover them quite recently on my podcast. Yes, this is another shameless plug.↩
Mostly, anyway. I believe the Swift compiler also does some degree of static analysis similar to that done by Rust—though to a much lesser extent and, speaking purely descriptively, much less rigorously (it just has different goals). Swift then uses that analysis to handle things at compile time rather than via reference counts if it’s able to determine that it can do so.↩
We could, if we so desired, get this same basic behavior in Rust. We can easily imagine a world in which every type was automatically wrapped in Rc or Arc, and in fact, I’d be very interested to see just such a language—something which was only a thin layer over Rust, keeping all its semantics but wrapping some or all non-stack-allocated types in Rc or Arc as appropriate. (Something like this, but done behind the scenes rather than manually opted into.) You’d incur some performance coasts, but with the benefit that you’d have an extremely ergonomic, practical, ML-descended language quite appropriate for slightly higher-level tasks, and without the radical shift required by switching to a lazily-evaluated, purely functional language like Haskell.↩
Notably, those tradeoffs are often entirely worth it, and high-performance VMs have astoundingly good characteristics in many ways. The JVM, the CLR, and all the JavaScript VMs have astonishingly excellent performance at this point.↩
I might have gotten slightly carried away in the details here. I’m just a little bit of a nerd.↩

Consistency in User Interfaces

2016-07-15T10:37:00-04:00

People sometimes ask what I mean when I say Git’s UI is maddeningly inconsistent. Here’s a concrete example: what are the commands to list tags, branches, and stashes?

git tag --list
git branch --list
git stash list

Follow that up by noticing the difference in meaning for the -v flag between the commands:

git branch -v: verbose mode: list the hash with an abbreviated commit summary
git tag -v: verify a tag against its GPG signature
git stash list -v: no-op, completely ignored

This is disastrously bad user interface design, and there is literally no reason for it except that the developers of Git, led by Linus Torvalds, don’t care about designing for end users. They hack in whatever commands seem to make the most sense right here and right now, and call it good—and then imply or directly state that anyone who has a problem with it is stupid or lazy.

But users are neither stupid nor lazy, and it is not stupid or lazy to want a system to behave in a a consistent way. Imagine if the buttons on you car’s media dashboard (a plastic one where the labels stay the same) did different things depending on whether you were in Drive or Reverse. Or if the light switches in your house behaved differently if you were using your toaster than if you were vacuuming, “on” and “off” labels notwithstanding.

Good user interface design is no less applicable to a command-line utility than to a pretty iOS app. Don’t let Linus Torvalds or anyone else tell you otherwise.

Rust and Swift (xvii)

2016-06-30T23:00:00-04:00

In the last part, I talked through the first chunk of the Swift book’s material on initializers. But it’s a long section, and I definitely didn’t cover everything. (I also got a few bits incorrect, and thankfully got great feedback to tighten it up from Twitter, so if you read it right after I posted it, you might skim back through and find the places where I added “Edit: …”)

Picking up from where we left on, then. Swift has a number of further initializer types, some of which map rather directly to the way initializers work in Rust, and some of which have no direct analog at all.

In the first category are the memberwise initializers Swift supplies by default for all types. The most basic init method just uses the names of the members of any given struct or class type in Swift (as in the previous section, I’m going to use the types the Swift book uses for simplicity):

struct Size {
    var height = 0.0, width = 0.0
}

someSize = Size(height: 1.0, width: 2.0)

This actually looks almost exactly like the normal way we construct types in Rust, where the same basic pattern would look like this:

struct Size {
    height: f64,
    width: f64,
}

some_size = Size { height: 1.0, width: 2.0 }

There are two big differences between the languages here. The first, and most immediately apparent, is syntactical: in this case, Rust doesn’t have a function-call syntax for creating instances, and Swift does. Swift’s syntax is similar to one of the several C++ constructor patterns, or especially to Python’s initializer calls (if we made a point to be explicit about the keyword arguments):

class Size:
    height = 0.0
    width = 0.0
    def __init__(height, width):
        self.height = height
        self.width = width

someSize = Size(height=1.0, width=2.0)  # unnecessarily explicit

The second, and more significant, is that the default, memberwise initializer in in Swift is only available if you have not defined any other initializers. This is very, very different from Rust, where there’s not really any such thing as a dedicated initializer—just methods. If we defined Size::new or Size::default or Size::any_other_funky_initializer, it wouldn’t make a whit of difference in our ability to define the type this way.¹ However, and this is important: because Rust has field-level public vs. private considerations, we cannot always do memberwise initialization of any given struct type there, either; it is just that the reasons are different. So:²

mod Shapes {
    struct Rectangle {
        pub height: f64,
        pub width: f64,
        area: f64,
    }
}

fn main() {
    // This won't work: we haven't constructed `Size::area`, and as we noted
    // last time, you cannot partially initialize a struct.
    let some_size = Shapes::Size { height: 1.0, width: 2.0 };

    // But neither will this, because `area` isn't public:
    let some_other_size = Shapes::Size { height: 1.0, width: 2.0, area: 2.0 };
}

Swift lets you refer to other initializers on the same type (reinforcing that init() is basically a kind of method, albeit one with some special rules and some special sugar). You do that by calling self.init(), and—very importantly—you can only call it from within another initializer. No funky reinitializations or anything like that. The net result is that if you have a couple different variations on ways you might initialize a type, you still get the benefit of reusability; you don’t have to reimplement the same initialization function over and over again. Do whatever additional setup is required in any given instance, and then call a common base initializer.

With Rust, again, we just have methods, so you could of course call them wherever you like. However, those methods are distinguished as being type-level or instance-level methods by their signatures, rather than by keyword. If the first argument is (some variant on) self, it’s an instance method, otherwise, a type-level method. This eliminates any potential confusion around the initializers:

struct Foo {
    pub a: i32
}

impl Foo {
    pub fn new(a: i32) -> Foo {
        Foo { a: a }
    }

    pub fn bar(&self) {
        // yes:
        let another_foo = Foo::new();
        // no (won't even compile):
        // let self_foo = self.new();
    }
}

You can (of course!) build up a type through multiple layers of methods which are useful to compose an instance together. This is what the builder pattern is all about. There are definitely times when you want to be able to tweak how your initialization plays out, and being able to do that without just passing in some hairy set of options in a special data type is nice.

One other important qualification on the Swift initializers: those default, memberwise constructors you get for free? You only get them for free if you don’t define your own initializers. (The closest analogy to this in Rust is that you’ll have issues if you try to both #[derive(Default)] and impl Default for Foo, since both will give you an implementation of Foo::default().) You can get around this in Swift by using an extension. We’ll come back to that in a future post.³ You can also get around it by supplying a parameter-less, body-less initializer in addition to any other initializers you supply, so: init() {}. (This, frankly, seems like a hack to me. It’s a useful hack, given the other constraints, but these kinds of things pile up.) Similarly, you can just reimplement member-wise initializers yourself if you have a reason to (say, if you’ve implemented any others and therefore the defaults no longer exist).

Now things take a turn into Swift-only territory again as we look at initialization in the context of inheritance. (As mentioned last time: Rust will eventually get inheritance-like behavior, but it’s coming much later, and is not going to be exactly like classical inheritance. Rust strongly favors composition over inheritance, where Swift lightly does but still supports the latter.)

Swift has two kinds of initializers for class initializers. One, a designated initializer, is required; a designated initializer must fully initialize every property on a class, and call the superclass initializer (assuming there is one). These can be inherited, but again: they are required.

There are also convenience initializers, which provide variant APIs for setting up any given class. These (by definition, given what we said a moment ago) must call a designated initializer along the way. These could be useful in a lot of different scenarios: setting up variants on the class (as in our temperature examples from before), doing alternate setup depending on initial conditions, etc.

The only difference between the two syntactically is that convenience initializers get the convenience keyword in front of the init declaration, so:

class Foo {
    var bar : Int
    let quux: String
    // designated
    init(_ bar: Int, _ quux: String) {
        self.bar = bar
        self.quux = quux
    }

    // A convenience method which only takes the string.
    convenience init(_ quux: String) {
        self.init(0, quux)
    }
}

The Swift book gives a set of rules about how these delegated and convenience initializers must behave. The short version is that convenience initializers (eventually) have to call a delegated initializer from their own class, and designated initializers have to call a designated initializer from the superclass. This is an implementation detail, though: from the perspective of a user of the class, it doesn’t matter which initializer is called.

The other important bit about Swift class initialization is that it is a two-phase process, which you might think of as “primary initialization” and “customization.” The primary initialization sets up the properties on a class as defined by the class which introduced them. The following sample should illustrate how it plays out:

class Foo {
    let plainTruth = "Doug Adams was good at what he did."
    let answer = 0

    init() {
        baz = answer / 2
    }
}

// Bar inherits from Foo
class Bar: Foo {
    let question = "What is the meaning of life, the universe, and everything?"
    let answer = 42

    init() {
        super.init()  // calls Foo.init()
    }

    convenience init(newQuestion question: String, newAnswer answer: Int) {
        self.question = question
        self.answer = answer
        self.init()  // calls own `init()`
    }
}

When building a Bar via either the designated or convenience initializer, plainTruth and answer will be set up from Foo, then question will be set and answer will be reassigned in Bar. If the convenience initializer is used, then it will also override those new defaults with the arguments passed by the caller, before running the designated initializer, which will in turn call the superclass designated initializer. The machinery all makes good sense; I appreciate that there are no weird edge cases in the initialization rules here. (There are a bunch of special rules about which initializers get inherited; I’m just going to leave those aside at this point as they’re entirely irrelevant for a comparison between the languages. We’re already pretty far off into the weeds here.)

Obviously, none of this remotely applies to Rust at all. Not having inheritance does keep these things simpler (though of course it also means there’s a tool missing from your toolbox which you might miss). And of course, the rules around method resolution are not totally trivial there, especially now that impl specialization is making its way into the language. But those don’t strictly speaking, affect initialization.

To account for the case that initialization can fail, Swift lets you definite failable initializers, written like init?(). Calling such an initializer produces an optional. You trigger the nil valued optional state by writing return nil at some point in the body of the initializer. Quoting from the Swift book, though, “Strictly speaking, initializers do not return a value…. Although you write return nil tro trigger an initialization failure, you do not use the return keyword to indicate initialization success.” These failable initializers get the same overall behavior and treatment as normal initializers in terms of delegating to other initializers within the same class, and inheriting them from superclasses.

class Foo {
    let bar: Int
    init?(succeed: Bool) {
        if !succeed {
            return nil
        }

        bar = 42
    }
}

let foo = Foo(true)
print("\(foo?.bar)")  // 42
let quux = Foo(false)
Print("\(foo?.bar)")  // nil

This is another of the places where Swift’s choice to treat initialization as a special case, not just another kind of method, ends up having some weird side effects. If init calls were methods, they would always just be returning the type. This is exactly what we see in Rust, of course. To be clear, there are reasons why the Swift team made that choice, and many of them we’ve already touched on incidentally; the long and short of it is that inheritance adds some wrinkles. These aren’t constructors, they’re initializers. The point, per the Swift book, is “to ensure that self is fully and correctly initializer by the time that initialization ends.” If you’re familiar with Python, you can think of Swift initializers as being quite analogous to __init__(self) methods, which similarly are responsible for initialization but not construction. When we build a type in Rust, by contrast, we’re doing something much more like calling Python __new__(cls) methods, which do construct the type.

Edit: interestingly, I’m informed via Twitter that Swift initializers can also throw errors. (Thanks, Austin!) The Swift book doesn’t mention this because it hasn’t gotten to error-handling yet (and so, neither have we).⁴

You can of course write failable constructors in Rust, too:

struct Foo {
    bar: i64,
};

impl Foo {
    pub fn optional_new(succeed: bool) -> Option<Foo> {
        if succeed { Some(Foo { bar: 0 }) }
        else { None }
    }
}

let foo = Foo::optional_new(true);
match foo {
    Some(f) => println!("{}", f.bar),
    None => println!("None"),
};

There are conditions in both languages where you’d want to do this: places where an initialization can fail, e.g. trying to open a file, or open a websocket, or anything where the type represents something that is not guaranteed to return a valid result. It makes sense then that in both cases, returning an optional value is the outcome. Of course, Rust can equally well have an initializer return a Result<T, E>:

struct Waffles {
    syrup: bool,
    butter: bool,
}

impl Waffles {
    fn properly(all_supplies: bool) -> Result<Waffles, String> {
        if all_supplies {
            Ok(Waffles { syrup: true, butter: true } )
        }
        else {
            let msg = "Who makes waffles this way???";
            Err(msg.to_string())
        }
    }
}

let waffles = Waffles::properly(true);
match waffles {
    Ok(_) => println!("Got some waffles, yeah!"),
    Err(s) => println!("{:}", s),
};

This is simply not the kind of thing you can do in Swift, as far as I can tell. The upside to Swift’s approach is that there is one, standard path. The downside is that if you have a scenario where it makes sense to return an error—i.e., to indicate why a class failed to initialize and not merely that it failed—you’re going to have to jump through many more hoops.⁵ Edit: See above; Swift can do this. Moreover, the underlying semantics aren’t especially different from Rust’s. However, it does introduce yet more syntax, rather than just being a normal return. But we’ll talk about that in more detail when we get to error-handling.⁴ The downside for Rust is that there’s no shorthand; everything is explicit. The upside is the flexibility to do as makes the most sense in a given context, including defining whatever types you need and returning them as you see fit. If you need a type like PartialSuccessPossible<C, P, E> where C is a complete type, P a partial type, and E an error, you can do that. (I’m not saying that’s a good idea, for the record.) That in turn flows out of building even higher level language features on lower-level features and not introducing new syntax for the most part. Trade-offs!

And with that, we’re done talking about initializers. This was a huge topic—but it makes sense. If you don’t nail this down carefully, you’ll be in for a world of hurt later, and that goes whether you’re designing a language or just using it to build things.

Previous: Initialization: another area where Swift has a lot more going on than Rust.
[**Next: Deinitialization: ownership semantics and automatic reference counting][18]

Also recall that in Rust, we would set the default values either by using the #[derive(Default)] annotation or by implementing the Default trait ourselves.↩
I’m including a module because of a quirk around the public/private rules: within the same module, area isn’t hidden and you can actually go ahead and initialize the object.↩
Depending on how you think about extensions, either Rust doesn’t have anything quite like them… or every type implementation is just an extension, because impl allows you to extend any data type in basically arbitrary ways (a few caveats of course). More on all of this when we get there.↩

Here’s a preview of what that would look like, though (fair warning, there’s a lot going on here we haven’t talked about!):

enum Setup {
    case succeed
    case error
    case fail
}

enum BarSetupError: ErrorProtocol {
    case argh
}

class Bar {
    let blah: Int
    init?(setup: Setup) throws {
        switch setup {
        case .succeed:
            blah = 42
        case .error:
            throw BarSetupError.argh
        case .fail:
            return nil
        }
    }
}

do {
    let bar = try Bar(setup: .succeed)
    print("\(bar!.blah)")

    let baz = try Bar(setup: .fail)
    print("\(baz?.blah)")

    let quux = try Bar(setup: .error)
    print("\(quux?.blah)")
} catch BarSetupError.argh {
    print("Oh teh noes!")
}

The output from this would be 42, nil, and Oh teh noes!.↩

~~It’s conceivable this is actually possible, but nothing in The Swift Programming Language even hints at it, if so.~~ See above!↩

Y Combinators, how do they even work?

2016-06-19T09:20:00-04:00

I was reading a post by Matt Might, a computer science professor at the University of Utah, about Y Combinators, and I was having a hard time tracking with some of it just by reading. The way I normally solve this problem is to write it out—and, optimally, to write it out in something roughly like Literate Haskell or Literate CoffeeScript. That’s exactly what you’ll find below; this is basically commentary on Might’s original post.

A few other prefatory notes:

Since this is commentary, I’m not focusing on explaining combinators in general. For a very helpful explanation, though, both of what combinators are and why you’d ever want to use them, read this.
The Y Combinator itself isn’t all that useful for ordinary programming. It is really useful as a way of thinking about how programming works, and that’s why I was reading about it and trying to figure out what was going on in Might’s original post.
This didn’t actually all make sense to me until I also read Might’s post, “Equational derivations of the Y combinator and Church encodings in Python”. Which is a crazy post. But kind of fun.

Note for background (this was new to me today): λv.e is the function which maps v to e. In ECMAScript 2015 or later (hereafter just JS):

const λv_e = v => e

The Y Combinator is a higher-order functional: it is a function which takes a functional/higher-order function. Quoting from Might:

The Y combinator takes a functional as input, and it returns the (unique) fixed point of that functional as its output. A functional is a function that takes a function for its input. Therefore, the fixed point of a functional is going to be a function.

And a “fixed point” is an input to a function equal to the output of the function. (Not all functions have such.) A fixed point is where f(x) = x. He uses the example x = x² − 1, which has two solutions, two fixed points.

He starts out with the total recursion form—also known as the “crash all the things!” form—of the Y-combinator. (I’m using letters to denote the version of the combinator; this is Y-naive.)

const Yn = (F) => F(Yn(F))  // all the recursing!

“Crash all the things”… because of one pesky little detail: it calls itself immediately, and so recurses infinitely. Which is actually kind of a problem.

Might then asks: What if we transformed this a bit? He notes that we can transform with lambda calculus to expand what we’re doing, so:

Y(F) = F(λx.(Y(F))(x))

(I haven’t done this kind of thing since undergraduate math work I did for physics, but as I was thinking about it, it made sense. I’m used to trying to remove extraneous variables when dealing with software, but in this case we’re using it as a tool for transforming the equation into a form that is equivalent but expressed differently.)

And λx.(Y(F))(x) is equivalent to the fixed point. It’s the function which takes x as an argument and results in Y(F)(x); but Y(F) is just another argument, so this looks just like our original f(x) = x, but with Y(F) substituted for f. Can we write this in JS?

Here’s my implementation, using modern JS; note that it still recurses. (I’m calling this updated Y-transformed, so Yt.)

const Yt = (F) => F((x) => Yt(F)(x))

His version:

function Y(F) { return F(function(x) { return Y(F)(x); }); }

Mine and his are equivalent; here’s his version transformed to modern JS:

const Y = (F) => F((x) => Y(F)(x))

Might then says:

Using another construct called the U combinator, we can eliminate the recursive call inside the Y combinator, which, with a couple more transformations gets us to:

I hated it when profs (or books!) did this when I was in college, and it frustrates me here, too. I want to see the transformation. I really wish Might didn’t skip how the U combinator works or what transformations he applies, because then he jumps to this form:

Y = (λh.λF.F(λx.((h(h))(F))(x)))(λh.λF.F(λx.((h(h))(F))(x)))

Writing this out in JS is going to be a real bear. More to the point, I don’t know how he got to it; now I need to go look up the U Combinator it seems.

…which I’ve now done. So:

In the theory of programming languages, the U combinator, U, is the mathematical function that applies its argument to its argument; that is U(f) = f(f), or equivalently, U = λf.f(f).

That is, the U Combinator is the case where you apply a function to itself: U(f) = f(f)—you can see that in the result there, where the first expression is the same as the argument handed to it (and both are functions). It’s also there in the h(h) calls.
The transformations are just transforming from a function-argument for to a lambda form, I think. The kind of thing where you go from function a(b) { return c } to var a = function(b) { return c } in JS. (Better, in modern JS, to const a = (b) => c.)

I’ll return to that in a moment. First, writing up the JS. The innermost term is (repeated) λx.((h(h))(F))(x), so we’ll start by writing this out.

const λ_inner = (x) => (h(h)(F))(x)

We need the definition of h next; this comes from further out, the transformation λh.λF.F(λ_inner) (where we’re substituting the λ_inner we just wrote to make this a bit easier to get our heads around).

Remembering that each “.” in the equation represents a mapping, i.e. a JS function call, we have this (writing it with function definitions starting new lines to clarify):

Here’s what I came up with as a fairly direct translation into JS:

const Y = (
  (h) =>
    (F) => F((x) => (h(h)(F))(x))  // substituting λ_inner from above
) (
  (h) =>
    (F) => F((x) => (h(h)(F))(x))  // substituting λ_inner from above
)

His (note that things are aligned as they are so that it’s clear which functions match up):

var Y = function (F) {
 return (function (x) {
  return F(function (y) { return (x(x))(y);});
  })
        (function (x) {
  return F(function (y) { return (x(x))(y);});
  }) ;
} ;

His transformed to modern JS:

const Y = (F) => (
  (x) => F((y) => x(x)(y))
) (
  (x) => F((y) => x(x)(y))
)

His and mine are not quite the same (though I know they’re equivalent because they both work). I really wish he’d explained how he got this substitution as well! More importantly, I wish he’d been consistent in his notation; changing variable names is… frustrating when you’re trying to follow someone’s work.

When I get stuck on something like this, the way I figure it out is by writing out how the substitutions would work at each step. See below.

In any case, now that we have the Y combinator, we can use it with FactGen, a functional which, if you pass it the factorial function, passes back the factorial function. FactGen itself isn’t recursive. But with the Y Combinator, it builds a function which is not recursive; it doesn’t reference itself anywhere. It just needs the right kind of “factory”: a function which returns another funtion which itself is recursive. Here’s a standard recursive factorial implementation (identical to the one Might supplies, though modernized):

const FactGen =
  (fact) =>
    (n) => n === 0 ? 1 : n * fact(n - 1)

You call that like this:

Y(FactGen)(5)  // 120

The Y(FactGen) call gets back a function which then runs on whatever input you hand it (a fairly standard pattern with curried arguments), so you could also write it like this:

const factorial = Y(FactGen)
factorial(5)  // 120

But I’m still not sure how his and mine are equivalent.

A note: wrapping things in (...) in JS defines that wrapped content as a distinct expression. As long as the type of a given expression is a function, it can be called with an argument. So (function() {})() or (() => {})() takes a no-operation function and immediately executes it.

So in his Y combinator, the substitution goes like this:

const Y = (F) => (  // F is FactGen
  // x is the identical function passed as argument below
  (x) =>
    // Run FactGen by taking the function below as its `fact`
    // argument.
    F(
      // `y` is the argument passed to the result of Y, e.g.
      // `fact(5)`. Recall that `x` is the function below; we
      // call it with itself. Calling x(x) will get the actual
      // factorial function returned by `FactGen`.
      (y) => x(x)(y)
    )
// We close the *expression* which defines the outer function,
// and call it with this next expression as an argument.
) (
  // and x here is the same function, passed as argument
  (x) =>
    // Again, run `FactGen` with this function as its argument.
    F(
      // `y`, again, will be the integer. `x(x)` again will be
      // the actual factorial function.
      (y) => x(x)(y)
    )
)

This is pretty funky! But it works; the two anonymous functions call each other rather than recursing directly.

In mine, it goes like this, instead:

const Ymine = (
  // Where in Might's example, the `x` function was where the
  // U Combinator was applied, here (because I followed the
  // original notation he gave) it's `h`. So it's `h` which is
  // the same function handed back and forth as argument
  // to itself.
  (h) =>
    // `h` takes a functional, which takes `FactGen` as its
    // parameter. This is similar to the outermost function in
    // Might's version.
    (F) =>
      // As in Might's version, we call `FactGen` here.
      F(
        // The form is *similar* but not identical to his,
        // because of the extra call structure. `h(h)(F)` is the
        // factorial function.
        //
        // Note that then he has `y` where I have `x`; my `x`
        // and his `y` are just the result of the computation
        // (in this case, the integer factorial).
        (x) => (h(h)(F))(x))
) (
  // This is identical to the above; it's using the U Combinator.
  (h) => (F) => F((x) => (h(h)(F))(x))
)

This is how his simplification worked: instead of generating the factorial function each time, it generated it just the once and then used it.

I still couldn’t do the simplification he did myself. It’ll take more practice using and thinking about combinators and combinatorial logic before I get there, but that’s okay. That’s how learning works.

And that’s enough playing with combinatorials for now. (Except that I’m kind of tempted to see if I can go implement the U or Y combinators—or both—in Rust.)

If you’re curious how I worked this out… I expanded the JS representations of the final forms (here’s the code) and then stepped through the result in my JavaScript dev tools, watching how the function calls worked and what the values of each intermediate value were. It’s fascinating, and well worth your time.

Vectors and Iterator Access in Rust

2016-06-16T20:59:00-04:00

In the midst of doing my reading and research for New Rustacean episode 15 (which will be out fairly soon after I post this), I bumped into this little tidbit. It doesn’t fit in the episode, so I thought I’d share it here.

When you’re dealing with vectors in Rust, a common misstep when working with them via iterators is to move them when you only to borrow them. If you write for i in x where x is an iterator, you’ll move the iterator into the looping construct. Instead, you should nearly always write for i in &x to borrow a reference to the iterator, or for i in &mut x if you need to get a mutable reference to it.

Testing Ember.js Mixins (and Helpers) With a Container

2016-06-09T20:35:00-04:00

Updated to note that the same concerns apply to helpers. You can always see the full revision history of this item here.

Today I was working on an Ember.js mixin for the new mobile web application we’re shipping at Olo, and I ran into an interesting problem when trying to test it.

When you’re testing mixins (or helpers), you’re generally not working with the normal Ember container.¹ In fact, the default test setup for mixins doesn’t have any container in play. It just looks like this (assuming you ran ember generate mixin bar in an app named foo):

import Ember from 'ember';
import BarMixin from 'foo/mixins/bar';
import { module, test } from 'qunit';

module('Unit | Mixin | bar');

// Replace this with your real tests.
test('it works', function(assert) {
  let BarObject = Ember.Object.extend(BarMixin);
  let subject = BarObject.create();
  assert.ok(subject);
});

Note two things:

It uses the basic Qunit module setup, not the ember-qunit moduleFor setup.
It assumes you’re generating a new object instance for every single test.

Both of those assumptions are fine, if you don’t need to interact with the container. In many cases, that’s perfectly reasonable—I’d go so far as to say that most mixins and helpers probably shouldn’t have any dependency on the container.

In the specific case I was working on, however, the point of the mixin was to abstract some common behavior which included all the interactions with a service. This meant making sure the dependency injection worked in the unit test. This in turn meant dealing with the container. So let’s see what was involved in that. (You can generalize this approach to any place in the Ember ecosystem where you need to test something which doesn’t normally have the container set up.)

We start by switching from the basic qunit helpers to using the ember-qunit helpers.

// Replace this...
import { module, test } from 'qunit';
module('Unit | Mixin | bar');

// with this:
import { moduleFor, test } from 'ember-qunit';
moduleFor('mixin:bar', 'Unit | Mixin | Bar');

The moduleFor() helper has two things going for it—one of which we need, and one of which isn’t strictly necessary, but has some nice functionality. In any case, this will help when registering a container. Those two features:

It does support the use of the container. In fact, it’s declaring how this mixin relates to the container in the first argument to the helper function: 'mixin:foo' is the definition of the mixin for injection into the container.
Any functions we define on the options argument we can pass to the moduleFor() helper are available on the this of the test.

Now, in the first version of this, I had set up a common Ember.Object which had mixed in the BarMixin, so:

const BarObject = Ember.Object.extend(BarMixin);

Then, in each test, I created instances of this to use:

test('test some feature or another', function(assert) {
  const subject = BarObject.create();
  // ...do stuff and test it with `assert.ok()`, etc.
});

The problem was that any of those tests which required a container injection always failed. Assume we have a service named quux, and that it’s injected into the mixin like this in foo/app/mixins/bar.js:

import Ember from 'ember';

export default Ember.Mixin.create({
  quux: Ember.inject.service()
});

Any test which actually tried to use quux would simply fail because of the missing container (even if you specified in the test setup that you needed the service):

test('it uses quux somehow', function(assert) {
  const subject = BarObject.create();
  const quux = subject.get('quux');  // throws Error
});

Specifically, you will see Attempting to lookup an injected property on an object without a container if you look in your console.

Taking advantage of the two ember-qunit features, though, we can handle all of this.

import Ember from 'ember';
import { moduleFor, test } from 'ember-qunit';

const { getOwner } = Ember;

moduleFor('mixin:bar', 'Unit | Mixin | bar', {
  // The `needs` property in the options argument tells the test
  // framework that it needs to go find and instantiate the `quux`
  // service. (Note that if `quux` depends on other injected
  // services, you have to specify that here as well.)
  needs: ['service:quux'],

  // Again: any object we create in this options object will be
  // available on the `this` of every `test` function below. Here,
  // we want to get a "test subject" which is attached to the
  // Ember container, so that the container is available to the
  // test subject itself for retrieving the dependencies injected
  // into it (and defined above in `needs`).
  subject() {
    BarObject = Ember.Object.extend(BarMixin);

    // This whole thing works because, since we're in a
    // `moduleFor()`, `this` has the relevant method we need to
    // attach items to the container: `register()`.
    this.register('test-container:bar-object', BarObject);

    // `Ember.getOwner` is the public API for getting the
    // container to do this kind of lookup. You can use it in lots
    // of places, including but not limited to tests. Note that
    // that because of how the dependency injection works, what we
    // get back from the lookup is not `BarObject`, but an
    // instance of `BarObject`. That means that we don't need to
    // do `BarObject.create()` when we use this below; Ember
    // already did that for us.
    return getOwner(this).lookup('test-container:bar-object');
  }
});

test('the mixin+service does what it should', function(assert) {
  // We start by running the subject function defined above. We
  // now have an instance of an `Ember.Object` which has
  // `BarMixin` applied.
  const subject = this.subject();

  // Now, because we used a test helper that made the container
  // available, declared the dependencies of the mixin in `needs`,
  // and registered the object we're dealing with here, we don't
  // get an error anymore.
  const quux = subject.get('quux');
});

So, in summary:

Use the ember-qunit helpers if you need the container.
Define whatever dependencies you have in needs, just as you would in any other test.
Register the mixin-derived object (whether Ember.Object, Ember.Route, Ember.Component, or whatever else) in a method on the options argument for moduleFor(). Use that to get an instance of the object and you’re off to the races!

One final consideration: while in this case it made good sense to use this approach and make the service injection available for the test, there’s a reason that the tests generated by Ember CLI don’t use moduleFor() by default. It’s a quiet but clear signal that you should reevaluate whether this is in fact the correct approach.

In general, mixins are best used for self-contained units of functionality. If you need dependency injection for them, it may mean that you should think about structuring things in a different way. Can all the functionality live on the service itself? Can all of it live in the mixin instead of requiring a service? Can the service calls be delegated to whatever type is using the mixin?

But if not, and you do need a mixin which injects a service, now you know how to do it!

Side note: The documentation around testing mixins is relatively weak, and in general the testing docs are the weak bits in the Ember guides right now.² After a conversation with @rwjblue on the Ember Community Slack, though, I was able to get a handle on the issue, and here we are. Since it stumped me, I’m guessing I’m not the only one.

When this happens, write it up. I’ve been guilty of this too often in the past few months: learning something new that I couldn’t find anywhere online, and then leaving it stored in my own head. It doesn’t take a particularly long time to write a blog post like this, and if you’re stuck, chances are very good someone else is too.

If you’re not familiar with the “container”, this is where all the various dependencies are registered, and where Ember looks them up to inject them when you use methods like Ember.inject.service().↩
Something I intend to help address in the next week or two via a pull request, so if you’re my Ember.js documentation team friend and you’re reading this… it’s coming. 😉↩

Rust and Swift (xvi)

2016-06-07T23:30:00-04:00

Thanks to ubsan, aatch, and niconii on the #rust-lang IRC for a fascinating discussion of the current status of Rust’s initialization analysis, as well as some very interesting comments on what might be possible to do in the future. Everything actually interesting about Rust in this post comes from the conversation I had with them on the evening of March 13.

The rules various languages have around construction and destruction of objects are extremely important for programmer safety and ergonomics. I think it’s fair to say that both Swift and rust are actively trying to avoid some of the mistakes made in e.g. C++ which poorly affect both its safety and its ease of use for developers, albeit it in some superficially different ways. Both languages also support defining how types are destroyed, which we’ll come back to in a future discussion.

The basic aim both Rust and Swift have in this area seems to be the same: avoid partially initialized objects. (You don’t want partially initialized objects. Ask Objective C developers.)

Swift does this via its rules around initializers. Rust does it by requiring that all the values of a type be initialized at its creation. So, for example, the following looks like it should work, but it doesn’t. You can initialize the variable piecemeal, but you cannot use it:

#[derive(Debug)]  // to make it printable.
struct Foo {
    pub a: i32,
    pub b: f64,
}

fn main() {
    // This will compile, but `foo` will be useless.
    let mut foo: Foo;
    foo.a = 14;
    foo.b = 42.0;

    // This would actually fail to compile. Surprising? A bit!
    // println!("{:?}", foo);

    // This will work, though, because it fully constructs the type.
    let foo2 = Foo { a: 14, b: 42.0 };
    println!("{:?}", foo);
}

(The reasons why this is so are fairly complicated. See the addendum at the end for a brief discussion.)

In any case, this means that especially with more complex data types, providing standard constructor-style methods like new or default is conventional and helpful. (If the type has non-public members, it’s also strictly necessary.)

Swift has a number of options for initializers, which correspond to things you in most cases can do in Rust, but in a very different way.

First, Swift allows you to overload the init method on a type, so that you can have different constructors for different starting conditions. (This is, to my recollection, the first time any kind of overloading has come up so far in the Swift book—but that could just be my memory failing me. Certainly I haven’t referenced it in any previous discussion, though.)

The example offered by the Swift book is illuminating for the different approaches the languages take, so we’ll run with it. Here’s a class defining a Celsius type in Swift:

struct Celsius {
    let temp: Double

    init(fromFahrenheit f: Double) {
       temp = 1.8 * (f - 32.0)
    }

    init(fromKelvin k: Double) {
        temp = k - 273.15
    }
}

// Create an instance each way
let freezing = Celsius(temp: 0)
let balmy = Celsius(fromFahrenheit: 75.0)
let absoluteZero = Celsius(fromKelvin: 0.0)

Note the internal and external parameter names. This is a common idiom Swift keeps (albeit with some non-trivial modification, and with more to come). More on this below; first, the same basic functionality in Rust:

struct Celsius {
    temp: f64
}

impl Celsius {
    fn from_fahrenheit(f: f64) -> Celsius {
        Celsius { temp: 1.8 * (f - 32.0) }
    }

    fn from_kelvin(k: f64) -> Celsius {
        Celsius { temp: k - 273.15 }
    }
}

// Create an instance each way
let freezing = Celsius { temp: 0 };
let balmy = Celsius::from_fahrenheit(75.0);
let absoluteZero = Celsius::from_kelvin(0.0);

(Note that there might be other considerations in implementing such types, like using a Temperature base trait or protocol, or employing type aliases, but those are for later entries!)

You can see a point I made about Swift’s initializer syntax back in part x: the way Rust reuses normal struct methods while Swift has the special initializers. Neither is clearly the “winner” here. Rust gets to use existing language machinery, simplifying our mental model a bit by not adding more syntax. On the other hand, the addition of initializer syntax lets Swift use a fairly familiar type construction syntax even for special initializer cases, and a leaves us with a bit less noise in the constructor method. Note, though, that initializers in Swift are special syntax; they’re not just a special kind of method (as the absence of the func keyword emphasizes)—unlike Rust, where initializers really are just normal struct or instance methods.

The Swift book notes this distinction:

In its simplest form, an initializer is like an instance method with no parameters, written using the init keyword.

The new keyword is the thing I could do without. Perhaps it’s just years of writing Python, but I really prefer it when constructors for types are just sugar and you can therefore reimplement them yourself, provide custom variations, etc. as it suits you. Introducing syntax instead of just picking a standard function to call at object instantiation means you lose that. At the same time, and in Swift’s defense, I’ve only rarely wanted or needed to use those facilities in work in Python. It’s a pragmatic decision—and it makes sense as such; it’s just not where my preference lies. The cost is a bit higher than I’d prefer relative to the gain in convenience.

Back to the initializers and the issue of overloading: the external parameter names (the first parameter) is one of the main ways Swift tells apart the initializers. This is necessitated, of course, by the choice of a keyword for the initializer; Rust doesn’t have any need for this, and since Rust doesn’t have overloading, it also can’t do this. In Rust, different constructors/initializers will have different names, because they will simply be different methods.

[Edit: I’m leaving this here for posterity, but it’s incomplete. See below.] One other important thing falls out of this: the external parameter names are required when initializing a type in Swift. Because those parameter names are used to tell apart the constructor, this is not just necessary for the compiler. It’s also an essential element of making the item readable for humans. Imagine if this were not the case—look again at the Celsius example:

struct Celsius {
    let temp: Double

    init(fromFahrenheit f: Double) {
       temp = 1.8 * (f - 32.0)
    }

    init(fromKelvin k: Double) {
        temp = k - 273.15
    }
}

// Create an instance each way
let freezing = Celsius(0)
let balmy = Celsius(75.0)  // our old fromFahrenheit example
let absoluteZero = Celsius(0.0)  // our old "fromKelvin example

We as humans would have no idea what the constructors are supposed to do, and really at this point there would necessarily just be one constructor unless the later options took elements of another type. That would be fairly similar to how overloading works in C++, Java, or C^♯, and while method overloading in those langauges is very powerful, it can also make it incredibly difficult to figure out exactly what method is being called. That includes when the constructor is being called. Take a look at the long list of C^♯ DateTime constructors, for example: you have to either have this memorized, have the documentation open, or be able simply to infer from context what is going on.

Given the choice of a keyword to mark initializers, then, Swift’s rule about external parameter name usage wherever there is more than one initializer is quite sensible.

[Edit: several readers, most notably including Joe Groff, who works on Swift for Apple, pointed out that Swift does support overloading, including in init() calls, and uses types to distinguish them. Moreover, you can leave off the label for the parameter. My initial summary was simply incorrect. I think this is a function of my not having finished the chapter yet.]

Second, both languages support supplying default values for a constructed type. Swift does this via default values defined at the site of the property definition itself, or simply set directly from within an initializer:

struct Kelvin {
    var temp: Double = 0.0  // zero kinetic energy!!!
    init () {
        temp = 305.0  // Change of plans: maybe just freezing is better
    }
}

In Rust, you can not supply default values directly on a property, but you can define any number of custom constructors:

struct Kelvin {
    temp: f64,
}

impl Kelvin {
    fn abs_zero() -> Kelvin {
        Kelvin { temp: 0.0 }
    }

    fn freezing() -> Kelvin {
        Kelvin { temp: 305.0 }
    }
}

We could of course shorten each of those two one line, so:

fn abs_zero() -> Kelvin { Kelvin { temp: 0.0 } }

The Rust is definitely a little noisier, and that is the downside of this tack. The upside is that these are just functions like any other. This is, in short, exactly the usual trade off we see in the languages.

Rust also has the Default trait and the #[derive(default)] attribute for getting some basic defaults for a given value. You can either define a Default implementation yourself, or let Rust automatically do so if the underlying types have Default implemented:

struct Kelvin {
    temp: f64,
}

// Do it ourselves
impl Default for Kelvin {
    fn default() -> Kelvin {
        Kelvin { temp: 305.0 }
    }
}

// Let Rust do it for us: calling `Celsius::default()` will get us a default
// temp of 0.0, since that's what `f64::default()` returns.
#[derive(default)]
struct Celsius {
    temp: f64,
}

This doesn’t get you quite the same thing as Swift’s initializer values. It requires you to be slightly more explicit, but the tradeoff is that you also get a bit more control and flexibility.

There’s actually a lot more to say about initializers—there are many more pages in the Swift book about them—but this is already about 1,700 words long, and I’ve been slowly chipping away at it since March (!), so I’m going to split this chapter of the Swift book into multiple posts. More to come shortly!

Addendum: No Late Initialization in Rust

Returning to the first Rust example—

#[derive(Debug)]  // to make it printable.
struct Foo {
    pub a: i32,
    pub b: f64,
}

fn main() {
    // This will compmile, but `foo` will be useless.
    let mut foo: Foo;
    foo.a = 14;
    foo.b = 42.0;

    // This would actually fail to compile. Surprising? A bit!
    // println!("{:?}", foo);
}

You can’t do anything with that data for a few reasons (most of this discussion coming from ubsan, aatch, and niconii on the #rust-lang IRC back in March):

Rust lets you “move” data out of a struct on a per-field basis. (Rust’s concept of “ownership” and “borrowing” is something we haven’t discussed a lot so far in this series; my podcast episode about it is probably a good starting point.) The main takeaway here is that you could return foo.a distinctly from returning foo, and doing so would hand that data over while running the foo destructor mechanism. Likewise, you could pass foo.b to the function created by the println! macro
Rust allows you to re-initialize moved variables. I haven’t dug enough to have an idea of what that would look like in practice.
Rust treats uninitialized variables the same as moved-from variables. This seems to be closely related to reason #2. The same “I’m not sure how to elaborate” qualification applies here.

I’ll see if I can add some further comments on (2) and (3) as I hit the later points in the Swift initialization chapter.

Rust and C++ function definitions

2016-06-03T18:01:00-04:00

I just put my finger on one of the (many) reasons Rust reads better than C++: the visual consistency of its function definitions. Compare—

Rust has:

fn foo() -> i32 { /* implementation */ }
fn bar() -> f32 { /* implementation */ }

C++ has:

int foo() { /* implementation */ }
double bar() { /* implementation */ }

That consistency adds up over many lines of code. There are many other such choices; the net effect is that Rust is much more pleasant to read than C++.

Note: I’m aware that C++11 added the auto foo() -> <type> syntax. But this actually worsens the problem. A totally new codebase which uses that form exclusively (which may not always be possible, because the semantics aren’t the same) would have roughly the same visual consistency as Rust in that particular category. (Plenty of others would still be a mess.) But the vast majority of C++ codebases are not totally new. Adding the form means your codebase is more likely to look this this:

int foo() { /* implementation */ }
auto quux() -> uint32_t { /* implementation */ }
double bar() { /* implementation */ }

That is, for the record, more visual inconsistency—not less!

Free Dynamic DNS for Remote Login via SSH

2016-05-31T20:10:00-04:00

I recently set up a hostname and mapped it to a dynamic IP address for my home machine so that I can log into it via SSH¹ from anywhere without needing to know what the IP address is. This is handy because I need to do just that on a semi-regularly basis: I’ll be out with my work laptop at a coffee shop, and need something that’s on my personal machine at home, for example.²

A friend asked me to describe it, so here I am. (Hi, Todd!) This was pretty straightforward for me, and it should be for you, too.

Pick one of the many free dynamic DNS providers. I picked No-IP after a very short bit of digging. In the future I may switch to a more full-featured solution, not least because I’m planning to separate out my DNS management from my hosting and my domain registrar later this year.³ For now, though, No-IP is good enough.
- Register.
- Pick a domain name.
- Add your current IP address. (If you need to find out what it is, you can literally just ask the internet: whatsmyip.org will tell you.)
Set up a local service to talk to the dynamic DNS provider, so that when your external IP address changes (and from time to time it will, if you’re not paying your ISP for a dedicated IP address). You can do this one of two ways:
- By installing a service on your main machine. No-IP and other large providers all have downloads where you can just install an app on your machine that goes out and talks to the service and keeps the IP address up to date.⁴
- By configuring your router. This is the route I took, because the router I have⁵ fully supports dynamic DNS services right out of the box.⁶ Look for something like Dynamic DNS and follow the configuration instructions there to get it talking to your dynamic DNS service provider. Mine has a built-in list which included No-IP; I just added my username and password and the domain name I specified back in Step 1, checked an Enable DDNS box, and connected.

That’s it. Even if you’re not a huge networking geek (which, for all my nerdiness, I really am not), you can set it up. From that point forward, if you have other things configured locally on your machine for network access (e.g. enabling SSH by toggling Remote Login to On in the Sharing preferences pane on OS X), you can just use the new domain you configured instead of the IP address. If that domain was e.g. <chriskrycho.example.com>, you could just ssh [email protected] and be off to the races.

Have fun!⁷

or mosh, which I’m hoping to check out this week↩
Or, when I was traveling and my Windows VM crashed while I was in the airport, and I was able to work from the VM on my home machine instead via SSH magic I’ll cover in a future blog post.↩
Having each of those in a separate place is nice: it means that if the others change, you only have to deal with that set of concerns. For example, if you move hosting providers, you don’t also have to migrate all your DNS settings—just tweak the couple that are relevant to the move.↩
Here’s the download page for No-IP, for example.↩
this one, as recommended by The Wirecutter↩
So will most open-source router firmwares, especially OpenWRT or DD-WRT, if they run on your router. I’ve done that in the past.↩
This tiny post has a hilarious number of footnotes. I noticed this early on, and instead of reworking it… I just ran with it.↩

SpaceX: "First-stage landing | Onboard camera"

2016-05-28T20:58:00-04:00

OH WOW: SpaceX first-stage landing footage… from the onboard camera. This is blow-your-mind incredible.

Ben Thompson: "Peter Thiel, Comic Book Hero"

2016-05-28T20:50:00-04:00

I always appreciate Ben Thompson’s takes, but this—on the Thiel/Gawker imbroglio—is one of his best posts ever.

Ember.js: "Introducing Subteams"

2016-05-24T19:10:00-04:00

In which one tech I really like (Ember.js) steals a great idea from another tech I really like (Rust).

2016-05-12 13:01

2016-05-12T13:01:00-04:00

This bit from the fish design document perfectly captures what git does wrong (emphasis mine):

When designing a program, one should first think about how to make a intuitive and powerful program. Implementation issues should only be considered once a user interface has been designed.

Rationale:

This design rule is different than the others, since it describes how one should go about designing new features, not what the features should be. The problem with focusing on what can be done, and what is easy to do, is that too much of the implementation is exposed. This means that the user must know a great deal about the underlying system to be able to guess how the shell works, it also means that the language will often be rather low-level.

Ulysses, Byword, and “Just Right”

2016-03-26T08:00:00-04:00

I’m trying out Ulysses again, as it’s been updated substantially since I last used it. I think the main thing to say about it is that it’s gorgeous and a really great editor, and that there is nonetheless something about it which makes it feel not quite as fluid as Byword always has.

Neither of them quite nails it for my purposes, though:

Neither is quite there for text that includes a lot of code samples. (Basically: neither supports the GitHub variations on Markdown, which are incredibly important for a lot of my writing
Neither has the ability to do things like autocompletion of citations from something like BibLatex. (No standalone app does, to my knowledge.)
Ulysses’ most powerful features only work in its iCloud bucket. And they’re not standard: rather than embracing CriticMarkup for comments, they have their own. The same is true of e.g. their code blocks.
Ulysses converts any other Markdown documents to its own custom variant when you open them. Had those documents formatted a way you liked (e.g. with specific kinds of link or footnote formatting)? Don’t expect them to still be that way.
Byword really does one thing well: opening and writing single documents. It does this extremely well, but it also has none of the library management that is useful for larger projects.

Both of these apps are really wonderful in many ways, and I think it’s fair to say that they’re perfect for many writers. My wife, for example, does nearly all her fiction writing in Ulysses; it works wonderfully for her. But for the kinds of writing I do—usually technical in one way or another—it is limited in its utility. That’s not really a critique of the apps. It’s more the recognition that I have some pretty unusual requirements of my writing apps.

That said, I don’t think I’m the only person out there who has these particular needs. I am, for example, hardly the only person working with citations and academic text, or writing Markup with lots of code samples in it. And as much as you can bend general-purpose text editors like Atom to your will,¹ it’s not the same as a dedicated writing app that focuses—in the ways that Ulysses and Byword both do—on just being a great tool for writing. Writing and writing code are not the same, after all. A tool that’s really well-optimized for the latter isn’t necessarily well-optimized for the former.

Keep your ears open. You might just be hearing more about this in the future.

Trust me, I have: I have Zen mode installed, a custom Byword-like theme I use when I just want to write, and even a citation autocompletion package integrated with it. It’s not bad. But I still don’t love it as a first-choice writing tool.↩

Rust and Swift (xv)

2016-03-12T14:45:00-05:00

The next chapter in the Swift book focuses on inheritance, a concept which does not yet exist in Rust.

Swift embraces classical inheritance for class data types. As noted previously, Rust’s struct covers much of the ground covered by Swift’s struct and class types together (value and reference types, etc.). However, what Swift’s class types bring to the table is inheritance-based (and not just composition-based) extension of types.

This is a bit of an interesting point: it is an area where, as of today, Swift can do something that is flat impossible in Rust—a rarity.

However, the status quo will be changing sometime in the next year or so, as there is a Rust RFC which has been accepted and is in the process of being implemented which paves the way for inheritance. (Discussions are ongoing as to the best way to implement it for Rust. Classical inheritance with vtables as in Swift is probably not going to be the approach.)

The reason Rust’s core team chose to proceed without inheritance for the 1.0 release of the language last May is simple: at a philosophical level, they prefer (as in general most developers increasingly acknowledge that we should all prefer) composition over inheritance. Prefer, not universally choose, because there are situations in which inheritance is the correct choice. But there is a reason that programming with interfaces rather than via sub-classing is a “best practice” for many scenarios in languages like Java or C#.

Rust’s trait system gives you composition in some remarkably powerful ways, allowing you to do things that in C++, for example, have to be accomplished via a combination of inheritance and overloading. Swift, likewise, supplies a protocol system and allows extensions to define further behavior on top of existing data structures. From what I’ve gathered, those approaches are preferred over inheritance in Swift for the same reason Rust shipped 1.0 without it!

But Swift does have inheritance, so it’s worth seeing how it works.

First, any class which doesn’t declare a parent from which to inherit is a base class. This is an important difference from, say, Python, where all classes inherit from Object (leaving aside custom metaclasses).

The syntax choices Swift has made around sub-class declarations are sensible and readable: class SubClass: ParentClass is eminently readable and doesn’t have any obvious points of overlap with other elements in the language.

Indeed, many of the choices made around classes are quite sensible. Overrides, for example, are made explicit via the override keyword. While I’ve sometimes poked fun at Swift’s tendency to add keywords everywhere, this seems like a reasonable place to have one, and it’s nice that overrides are explicit rather than implicit. The same is true of the use of super to refer to the superclass. I’m not sure of the implementation details, but super appears to act as just a special/reserved name for an object: all the syntax around it is normal object instance syntax, which is as it should be.

The limitations around overriding properties all make sense. You can override a read- or write-only parent property as both readable and writable, but you can’t override a readable or writable property not to be readable or writable respectively. Presumably this is because the method lookup for properties always checks up the inheritance chain for getters or setters, so if one is present, you can’t just get rid of it. (You could of course override with a no-op function that spews a warning or some such, but that would pretty clearly be an abuse of the parent API. There might be times you would do that with a third-party library parent class, but in your own code it should be avoided: it indicates a problem in your API design that you need to address instead.)

Finally, we have Swift’s final keyword—and yes, pun intended. It marks whatever block-level item it is attached to—whether class, method, or property—as non-overridable. Attempts to override an item marked final are compile-time failures. (The same kind of thing exists in Java and C#.) In and of itself, this isn’t especially interesting. It is interesting to ponder whether you should make classes subclass-able or not in your API design. There has been an active debate, in fact, whether classes in Swift should become final by default in Swift 3.0, rather than open by default. The debate centers on the danger of unintended consequences of overriding, which ultimately takes us back around to the preference for composition, of course.

All of this, among other things, raises the very interesting question of what this will look like in Rust when, eventually, we get inheritance there. After all, we know it will be quite different in some ways:

It presumably won’t involve a distinct data type constructor, a la Swift’s distinction between struct and class: there may be syntactic sugar involved, and there will definitely be new functionality present, but it will certainly be built on the existing language features as well. There’s a good chance it will basically look like just a special case of impl SomeTrait for SomeStruct, which would fit very well with the ways Rust solves so many other problems.
Rust doesn’t have many of the things which Swift takes care to special-case for overriding with final, but it will need to address that case for inherited methods and data in some way. (The proposal linked above uses a distinction between default and blanket implementations for trait specialization to pull this off; if those words don’t mean anything to you, don’t worry: I’ve read that post and RFC half a dozen times before I got a really solid handle on all the pieces involved.)
It will be a relative latecomer to the language, rather than baked in from the start, and therefore will likely seem a secondary way of solving problems, especially at first. (This is, I think, both intentional and good.)

Rust and Swift (xiv)

2016-03-10T21:25:00-05:00

Rust and Swift both support defining subscript access to a given data type, like SomeType[accessedByIndex]. Unsurprisingly, given everything we’ve seen so far, Rust does this with traits, and Swift with a keyword.

In Rust, you can define subscript-style access to a type by implementing the Index and/or IndexMut traits, which allow indexing into a given location in a kind of type. The implementation simply requires one function, which is called when you use the [] operator. That function, index or index_mut, implements how to do the lookup for the specific type. The impl block indicates not only that Index or IndexMut is being implemented, but also the type of the key used: impl Index<Bar> for Foo { ... }, where access would look like a_foo[some_bar].

The two kinds of traits and corresponding methods define the behavior for immutable and mutable data type, as their name suggest.

Since the trait is defined generically, you can implement whatever kinds of accessors you like to the same underlying data structure, including generics accessors with trait bounds.

It is perhaps telling that in Rust you just find these traits in the general std::ops module, where all the core language operations and associated operators are defined. Rust doesn’t do “operator overloading” so much as it simply provides operators as one more class of trait potentially applicable to your type. (The family resemblance to Haskell’s type classes and similar in other languages is obvious.)

In Swift, you define indexing behavior with the subscript keyword. Subscripts act very similarly to Swift’s computed properties. They can be made read- or write-only by including or excluding get and set function definitions, just like computed properties.

The behavior is in fact so closely aligned with the computed property syntax and behavior that I initially wondered if it wasn’t just a special case. It is not (though I’m sure much of the parsing machinery can be shared). As the designation of subscript as a keyword strongly implies, and unlike in Rust, this is a separate language construct, not building on existing language machinery.

Swift, like Rust, allows you to define arbitrary accessors. However, since the behavior relies on the subscript construct rather than generics and protocols (Swift’s equivalent to Rust’s traits), you define different kinds of accessors via multiple subscript blocks. (Presumably these could take generic arguments, but I haven’t tested that to be sure.)

Both languages proceed to use these as ways of accessing types as makes sense—e.g. for not only arrays or vectors, but also dictionaries in Swift and HashMap types in Rust.

Since you can define the behavior yourself, you can also use complex types as keys. The languages approach this a bit differently, though. In Rust, if you wanted a compound key, you would need to define either a simple container struct or use a tuple as the argument. In Swift, because it uses the same basic syntax as computed properties, you can just define as many method arguments, of whatever type, as you want.

Takeaway: Rust uses traits; Swift uses a keyword. We probably could have guessed that when we started, at this point!

The Future of JavaScript

2016-03-02T12:30:00-05:00

I gave a short tech talk at my new employer Olo today, covering a number of the changes current and forthcoming in ECMAScript 2015 and later. Alas, I ran out of time in preparation and didn’t get to cover everything I wanted—I would have liked very much to cover modules, and to cover fat-arrow-functions in more depth than I did. I’ll look forward to hopefully giving further tech talks at Olo in the future, and perhaps giving this one, expanded and finished out a bit, elsewhere. (If you’d like me to give a talk, including this one, just let me know!) In the meantime, you can take a look at the slides, which I think will be helpful and interesting!

And yes, there were a lot of really delightful Doctor Who references in this talk. Because of course there were!

Static Site Generators and Podcasting

2016-02-28T12:50:00-05:00

Presently, I publish both Winning Slowly and New Rustacean ¹ using what is admittedly a bit of a quirky approach. It works well for me, and I think it’s worth documenting for other nerdy types out there, but if you’re just getting going with podcasting and you’re looking for the easy way to do it, let me warn you: this isn’t it. Something like SoundCloud and a blog for show notes, or WordPress with Blubrry PowerPress is what you want instead. This approach works extremely well for statically-generated sites, however, and I imagine a few people out there might find it useful.

The short version

Generate the feeds with Feeder.
Generate the site statically with something else (and it really doesn’t matter what).
Copy the feed into the generated site.

The long version

I generate the sites themselves with Pelican and cargo doc, respectively. I was already comfortable with Pelican because it’s what I use to generate this site (with a few tweaks to the standard configuration, especially using Pandoc rather than the Python Markdown implementation), so I ran with it for building the Winning Slowly site, and it has worked quite well for building the site itself. It just gets built locally and deployed via GitHub Pages.

However, it does not have built-in support for generating podcast feeds, even just the general case with enclosures. iTunes podcast support would have taken a lot of work to add.² Instead, I chose to build the RSS feed semi-manually. Semi-manually, because doing it totally manually is a recipe for making mistakes. XML is many things, but “easy to write correctly by hand” is not one of them. I use Feeder to manage the feeds, and it makes sure that the enclosure and iTunes elements are set correctly.

The biggest upside to this is that I can use Pelican without modification to how it generates feeds (apart from optionally turning them off entirely). It just copies the feed I generate to the output file during its normal build process. As suggested above, I also don’t generate the other feeds which Pelican supports, as we have no need for them; we only care about the podcast feed.

This process works equally well, with very little modification, for New Rustacean. In that case, I’m generating the content by running Rust’s documentation tool, cargo doc³ to render the “API docs” which serve as show notes. Notice the family resemblance between my “show notes” and, say, the Diesel docs, which are both generated the same way. This is not a normal way of building a podcast website; you can hear me explain why I did it this way in New Rustacean e001: Document all the things! In any case, I just take the show note-relevant parts of the documentation and put it in Feeder, generate the feed, and copy that as part of the build process.

That’s it!

And, incidentally, Sap.py and my sermons feed.↩
If I stick with Pelican long-term, I might look into adding it anyway, but honestly, I don’t love Pelican. The reasons have little to do with Pelican for itself, and a lot more to do with my particular and somewhat peculiar needs. That’s a post for another day. In any case, I’m likelier to use another generator—even one I write myself!—than to do the work to make Pelican do what I want.↩
Technically, Rust’s documentation tool is rustdoc, which cargo doc wraps around. I never actually use rustdoc directly, though.↩

Rust and Swift (xiii)

2016-02-28T11:15:00-05:00

Rust and Swift both have methods which are attached to given data types. However, whereas Rust takes its notion of separation of data and functions rather strictly, Swift implements them on the relevant data structures (classes, structs, or enums) directly. In other words, the implementation of a given type’s methods is within the body of the type definition itself in swift, whereas in Rust it is in an impl block, usually but not always immediately adjacent in the code.

This goes to one of the philosophical differences between the two languages. As we’ve discussed often in the series, Rust reuses a smaller set of concepts—language-level primitives—to build up its functionality. So methods on a type and methods for a trait on a type are basically the same thing in Rust; they’re defined in almost exactly the same way (the latter includes for SomeTrait in the impl expression). In Swift, a method is defined differently from a protocol definition, which we’ll get to in the future. The point is simply this: the two take distinct approaches to the relationship between a given type definition and the implementations of any functions which may be attached to it.

Another important difference: access to other members of a given data type from within a method is explicit in Rust and implicit in Swift. In Rust, the first parameter to an instance method is always self or &self (or a mutable version of either of course), much as in Python. This explicitness distinction is by now exactly what we expect from the two languages.

Both use dot notation, in line with most other languages with a C-like syntax, for method calls, e.g. instance.method() in Swift and instance.method() in Rust. The latter is just syntactical sugar for T::method(&instance) or T::method(instance) where T is the type of the instance (depending on whether the item is being borrowed or moved). Given its implicit knowledge of/access to instance-local data, and the distinctive behavior of Swift methods (see below), I don’t think the same is, or even could be, true of Swift.

All of Swift’s other behaviors around functions—internal and external names, and all the distinctions that go with those—are equally applicable to methods. Similarly, with the sole change that the first parameter is always the instance being acted on, a Rust methods follow all the same rules as ordinary Rust functions (which is why you can call the struct or enum method with an instance parameter as in the example above).

Swift does have a self—it is, of course, implicit. It’s useful at times for disambiguation—basically, when a parameter name shadows an instance name. This will look familiar to people coming from Ruby.

The strong distinction Swift makes between reference and value types comes into play on methods, as you might expect, as does its approach to mutability. Methods which change the values in value types (struct or enum instances) have to be declared mutating func. This kind of explicit-ness is good. As we discussed in Part 10, Rust approaches this entire problem differently: types are not value or reference types; they are either mutable and passed mutably (including as mut self or &mut self), or they are not. If an instance is mutable and passed mutably, a method is free to act on instance data. And in fact both languages require that the instance in question not be immutable. In fact, everything we said in Part 10 about both languages applies here, just with the addendum that private properties are available to methods.

The distinction, you’ll note, is in where the indication that there’s a mutation happens. Swift has a special keyword combination (mutating func) for this. With Rust, it’s the same as every other function which mutates an argument. This makes Rust slightly more verbose, but it also means that in cases like this, the existing language tooling is perfectly capable of handling what has to be a special syntactical case in Swift.

Both Swift and Rust let you out-and-out change the instance by assigning to self, albeit in fairly different ways. In Swift, you’d write a mutating method which updates the instance proper like this:

struct Point {
    var x = 0.0, y = 0.0
    mutating func changeSelf(x: Double, y: Double) {
        self = Point(x: x, y: y)
    }
}

In Rust, you’d need to explicitly pass a mutable reference and dereference it. (If you tried to pass mut self instead of &mut self, it would fail unless you returned the newly created object and assigned it outside.) Note that while the full implementation here is a couple lines longer, because of the data-vs.-method separation discussed earlier, the implementation of the method itself is roughly the same length.

pub struct Point {
    pub x: f64,
    pub y: f64,
}

impl Point {
    pub fn change_self(&mut self, x: i32, y: i32) {
        *self = Point { x: x, y: y };
    }
}

Note that though you can do this, I’m not sure it’s particularly Rustic. My own instinct would be to get a new Point rather than mutate an existing one, in either language, and let the other be cleaned up “behind the scenes” as it were (with automatic memory management in Swift or the compiler’s automatic destruction of the type in Rust)—purer functions being my preference these days.

You can do this with enum types as well, which the Swift book illustrates with a three-state switch which updates the value type passed to a new value when calling its next() method. You can do the same in Rust, with the same reference/dereference approach as above.

Here’s a three-state switch in Swift:

enum ThreeState {
    case First, Second, Third
    mutating func next() {
        switch self {
        case First:
            self = Second
        case Second:
            self = Third
        case Third
            self = First
        }
    }
}

And the same in Rust:

enum ThreeState { First, Second, Third }
impl ThreeState {
    pub fn next(&mut self) {
        match *self {
            ThreeState::First => *self = ThreeState::Second,
            ThreeState::Second => *self = ThreeState::Third,
            ThreeState::Third => *self = ThreeState::First,
        }
    }
}

Both languages also have what Swift calls “type methods”, and which you might think of as “static class methods” coming from a language like Java or C♯. In Swift, you define them by adding the static or class keywords to the func definition. The class func keyword combo is only applicable in class bodies, and indicates that sub-classes may override the method definition.

struct Bar {
    static func quux() { print("Seriously, what's a `quux`?") }
}

func main() {
    Bar.quux()
}

In Rust, you simply drop self as a first parameter and call it with :: syntax instead of . syntax:

struct Bar;
impl Bar {
    pub fn quux() { println!("Seriously, what's a `quux`?"); }
}

fn main() {
    Bar::quux();
}

As usual, Rust chooses to use existing language machinery; Swift uses new (combinations of) keywords.

Rust and Swift (xii)

2016-02-27T22:30:00-05:00

A note on publication: I had this drafted in early January and simply forgot to publish it. Whoops!

As noted in my discussion of the product types in Rust and Swift, Swift distinguishes between classes and structs, with the former being reference types and the latter being value types. All structs are value types in Rust. (That you can wrap them in a pointer for heap-allocation with one of the smart pointer types, e.g. Box or Rc or Arc, doesn’t change this fundamental reality.) This underlying difference gives rise to one the big difference between Swift classes and Rust structs: a constant class instance in Swift can still have its fields mutated; not so with a Rust struct instance. But also not so with a Swift struct instance, as it turns out! There isn’t a straightforward way to do this with Box<T> in Rust; you could do it with something like an Rc<T> or Arc<T>, though.
Swift’s lazy keyword, and associated delayed initialization of properties has, as far as I know, no equivalent whatsoever in Rust. And while I can see the utility in principle, I’m hard-pressed to think of any time in my working experience where the behavior would actually be useful. Rather than having lazy properties, I would be far more inclined to separate the behavior which should be initialized at a later time into its own data structure, and supplying it via inversion of control if it is necessary for an actions taken by other data structures. (This seems—at first blush at least—to be a way of supporting the un- or partially-initialized data types possible in Objective C?)
Swift has computed properties, a concept familiar to Python developers (and relatively recently introduced in JavaScript). These can be quite handy, as they let you define a property to be accessed like any other (someInstance.theProperty) while being defined with functions which compute the value dynamically. A common, trivial example: if you defined a Person with firstName and lastName members, you could define a computed property, fullName, which was built using the existing values.
Rust doesn’t have computed properties at all. This is because of its design decision to deeply separate data from behavior, essentially stealing a page from more pure-functional languages (Haskell etc.). This is (one reason) why you don’t define the implementation of a struct method in the same block as the members of the struct. See an excellent explanation here.
It’s also closely related the way Rust favors composition over inheritance (by making the latter impossible, at least for now!). By separating impl from struct and enum, Rust makes it not only straightforward but normal to define new behavior for a given item separately from the data description. This, combined with the use of traits (like Swift’s protocols) as the primary way of sharing behavior between objects, means that you don’t have to worry about conforming to some interface when you define a given type; it can always¹ be defined later, even by entirely other modules or even other crates (packages).
In any case, the result is that it’s not at all Rustic² to have something like getters or setters or computed properties. It makes sense to have them in Swift, though, which has a more traditionally object-oriented type system (though with some neat additions in the form of its protocol type classes, which are analogous to Rust’s traits—but we’ll come to those in a future post). This is a wash: it’s just a function of the slightly different approaches taken in object design in the two systems. If you have a Swift-style type system, you should have computed properties. If you have a Rust-like type system, you shouldn’t.
I’m shocked—utterly shocked!—to find that Swift provides a default newValue argument for setters for computed properties, and shorthand for defining read-only properties. By which I mean: I find this kind of thing entirely unsurprising at this point in Swift, but I don’t like it any better. Making so much implicit just rubs me the wrong way. Once you know the language, it’s fine of course: you’ll recognize all the patterns. It just seems, in an interesting way, to add cognitive load rather than reducing it. That may just be me, though!
Interestingly, Swift also allows you to set watchers on given properties—functions called with the new or the removed value whenever the value of the computed property is updated or touched for any reason. It has two of these built in: willSet and didSet. You can override these to get custom behavior when a normal property is about to change. (You can of course just implement the desired behavior yourself in the set method for a computed property.)
Since Rust doesn’t have properties, it doesn’t have anything analogous. I can’t think of a particularly straightforward way to implement it, either, though you might be able do some chicanery with a trait. Of course you can always define a setter method which takes a value and optional callbacks for actions to take before and after setting the value; the thing that’s nice in Swift is that it gives you these as built-in capabilities within the language itself. (Now I’m wondering if or how you could implement an Observable trait, though! Might have to play with that idea more later.) It’s worth remembering , in any case, that Rust doesn’t have these because it doesn’t have properties.
Curiously, Swift provides the same functionality for “global” and “local” variables in a given context. In both cases, this is suggestive of the underlying object model for both modules and functions in Swift.
Now I’m curious what the representation of a module is in Swift; is it part of the general object system in some way?
This likewise gets me asking: what is a module in Rust? It’s a block item, clearly, and accordingly defines a scope (as do functions, if and match expressions, and so on). It’s not a compilation unit (as it is in C or C++). What other machinery is attached to it?
Both of these questions can be answered by reading the source code for the languages (Rust, Swift), of course. Putting that on my to-do list.
Swift also has type properties: values common to all instances of a given type. These are directly analogous to class properties (or class attributes) in Python or prototype properties in JavaScript.
Rust doesn’t have anything like this to my knowledge. You could accomplish something similar using a module-level variable with a 'static lifetime,³ much as you could in C—but that wouldn’t be an item on the type itself, of course.
The static declaration of item in Swift suggests what a possible implementation might look like in Rust: defining a member like a_static_long: 'static i64. There might be some interesting challenges around that, though; I don’t know enough to comment meaningfully. At the least, it seems like it would be an odd fit with the rest of the memory management approach Rust takes, and it would make it a bit harder to reason correctly about the behavior of data in a given type. (There are certainly issues there around mutability guarantees and lifetime checking!)
Because of the differences in underlying approach to data types and implementation, this is one of the areas where the superficially (and sometimes actually) similar languages diverge a lot.

leaving aside details about trait specialization still being hashed out in Rust↩
This is now my preferred term for “idiomatic Rust”—directly analogous to “Pythonic,” but with the upside of being an actual word, and one with pleasantly evocative connotations to boot.↩
There’s nothing analogous to Rust’s concept of explicit lifetimes in Swift, as far as I can tell. The static keyword in Swift, like that in C, Objective-C, and C++, is sort of like Rust’s 'static lifetime specifically, for variables at least—but Rust’s lifetime is substantially more sophisticated and complex than that analogy might suggest.↩

“I Don't Know When I'd Use That”

2016-01-17T10:00:00-05:00

I was reading an interesting Stack Overflow discussion of the value of higher-kinded types (hereafter HKTs), and noted someone repeatedly commenting, “But when would you use this in a real app?” To put it the way another blog post about HKTs (in Rust), they are “a feature people do not really know what to do with.”

Don’t get me wrong: I’m sympathetic to that desire for concrete examples. I’m interested in these kinds of things not primarily for their intellectual value but for their pragmatic value (though I don’t think those two are as distinct as many people do). I’d also love to see some more real-world examples in those discussions. All too often, the discussions of types in Haskell end up being quite abstract and academic—no surprise, given the language’s origin. But I’m also aware that quite often it’s difficult to see how a given kind of abstraction is useful without jumping into a language which has that abstraction available and using it.

People often get turned off by Haskell (and other similarly high-abstraction languages like Scala) because of challenging terms like monad, applicative, functor, and so on. And again: I get that. To grok Haskell, you need to wrap your head around a lot of math ideas—mainly various properties of sets.

But I remember feeling the same way six years ago when I started playing with JavaScript and jQuery and every tutorial out there simply assumed existing familiarity and comfort with functions as arguments or return values. Coming from the world of Fortran and C, my head ached for weeks as I tried to make sense of what I was seeing. Even when I finally got it, I didn’t like it. Over the last several years, though, I’ve become increasingly comfortable and even reliant on closures, composition of functions to transform data, and so on as I worked regularly in Python and JavaScript.

That experience has taught me that my current inability to see the utility of a given abstraction means little about the abstraction. It’s primarily an indicator of my own inexperience.

To the question of the utility HKTs in general—in Haskell, Rust, or somewhere else—I don’t have the knowledge myself (yet) to supply a good answer. Heck, I can’t even explain them very well. (Other people can, though!) But I can say that reading Maybe Haskell showed me clearly that such things can be very useful. Even if I am not yet comfortable using that tool, I see how learning to use it would be profitable in the long-term. And like any good tool, even if you don’t need it every day… when you want it, you really want it.

Women in Rust

2016-01-10T15:25:00-05:00

I posted these bullet points last night as a series of tweets on my main account.

∞ January 9, 2016 21:11

A thing I’d really, really like to see change—this is from the New Rustacean Twitter data. Unsurprising, but awful:

@newrustacean Twitter follower gender data
∞ January 9, 2016 21:12

Takeaway: the @rustlang community has many strengths, but like every tech community, we need to improve here—a lot.
∞ January 9, 2016 21:14

Standing offer: if you’re a female @rustlang dev, I’d love to feature your experience learning Rust on the show.
∞ January 9, 2016, 21:15

I’ll be doing some interview @newrustacean episodes soon-ish—I want as many female voices in the mix as possible.

Rust and Swift (xi)

2016-01-10T10:00:00-05:00

I’ve still been (slowly) working through the Swift book and comparing Swift and Rust; I have another draft started which I’ll hopefully finish this week. And I still find the comparison deeply profitable. The two languages continue to evolve in interesting ways, and the comparison is all the more interesting now that Swift is open-source and its future open for community input (just as Rust is).

Something I’ve been thinking about for several months, and which the brief discussion of Swift, Go, and Rust at the end of the latest Accidental Tech Podcast brought back to my mind, is the question of what the next generation of systems-level programming language should be. And my answer is: there shouldn’t be just one. The best possible thing for the space, in many ways, is for there to be a healthy diversity of options and lots of competition in the space. We don’t want to have ten different systems programming languages to deal with, I think—but three or four or five would be much preferable to having one or two (closely related) as we have in the decades of C and C++ dominance.

Don’t get me wrong: both languages (and perhaps especially C) do many things exceptionally well. For all that they are (justly) maligned for some of their problems, the longevity of both C and C++ has a great deal to do with how well they fit the problem domain, and how much they’ve empowered developers to accomplish within that space (which is very, very large).

The problem, though, at least as I see it, is that the existence of only two really serious systems programming languages for the last several decades has led a lot of developers to think that C and C++‘s ways of solving problems are the only way to solve problems. The languages we use shape the way we think about possible solutions, and when a given language doesn’t recognize entire classes of different approaches, that deeply limits developers’ ability to tackle certain issues. (See also the interesting CppCast interview with D’s Andrei Alexandrescu in which he makes similar points.)

The most obvious thing missing from both is the ability to do truly functional-style programming. C of course is also lacking classes and thus is much more difficult to use for any sort of object-oriented programming.¹ Neither has anything remotely like Rust’s traits or Swift’s extensions; C++ has only gotten lambdas recently.

All of this comes out to mean that the set of tools available to systems programmer has necessarily been missing any number of things available in languages outside that context. In some cases, this may be a necessary consequence of the kinds of programming being done: when you need totally deterministic memory and compiler behavior, dynamic typing and a non-trivial runtime are simply not options. But in many cases, they are simply a function of the history of the languages’ development and history. Being an ALGOL descendant, and especially a C descendant, means there are some fundamental choices about the language which will differ from those made in a language descended from ML.

All of which is to say: C and C++ have been really useful tools in many ways, but having only C and C++ available for serious systems programming work over the last decades has left many developers blind to or simply unaware of the real advantages other paradigms might offer them.

So going forward, I don’t want there to be a winner in the systems programming space. I’d rather see D, Rust, Swift, Go, and maybe even a few other contenders all stay strong—finding their own niches and continually pushing each other and learning from each other. That will give us a space in which different languages are free to try out different approaches to the same problems, without being tied to the specific constraints faced by other languages. Built-in greenthreading? Go! Hindley-Milner types, memory safety, and zero runtime? Rust! Something in beween, highly expressive and with different type systems and tradeoffs around memory management, etc.? Swift, or D!

Having a robust, thriving set of competitors in the market will be good for the languages themselves. But it will also be good for developers. It will take off some of the blinders that come from a single language (or a pair of very closely related languages) dominating the ecosystem. It will make it likelier that people will be more familiar with different programming paradigms. And that can only be a good thing, as far as I’m concerned.

It is of course entirely possible to do non-classical OOP; the point is that C entirely lacks language-level facilities for OOP, inheritance, etc.↩

Rust and Swift (x)

2015-12-06T11:25:00-05:00

Swift and Rust both have “product types” as well as the enum “sum types.” In Rust, these are struct types; Swift splits them into classes and structs.
“Product types” will be much more familiar to programmers coming from a C-like background, or indeed most object-oriented programming languages: these are the same basic kind of thing as classes, structs, and objects in other languages. These include all the value types which compose them, unlike sum types—enum—which have only one of the value types which compose them.
Right off the bat, I note the Swift book’s somewhat amusing reticence to call out C and C-descended languages:

Unlike other programming languages, Swift does not require you to create separate interface and implementation files for custom classes and structures.

Because there’s such a long list of languages not directly descended from C which do that, right? 😉
Rust differs not only from Swift but from every other modern language I have used in not having a constructor syntax for its instantiations. Whereas C++ has new NameOfType() and Python and Swift both have NameOfType(), “constructors” for Rust structs are just functions which return an instance constructed using literal syntax, by convention NameOfType::new().
Let’s make a struct defining a location in a plane, you might do this in Swift (leaving aside initializer values; I’ll come back to those later). These definitions look very similar. Swift:
```
struct Point {
    var x: Double var y: Double
}
```
Rust:
```
struct Point {
    x: f64,
    y: f64,
}
```
Creating the types looks a little different, though. Here’s a constructor in Swift:
```
let point = Point(x: 0, y: 0)
```
And the two ways we could construct the type in Rust, a literal constructor (fairly similar to constructing dict literals in Python or object literals in JavaScript):
```
let point = Point { x: 0.0, y: 0.0 };
```
Or a constructor method, new:
```
// "Constructor"
impl Point {
    fn new(x: f64, y: f64) -> Point {
        Point { x: x, y: y }
    }
}

let another_point = Point::new(0, 0);
```
Observe: these two things in Rust are the same under the covers (though if Points had non-public internals, they would be non-trivially different: you couldn’t construct it with its private members externally). As usual, Rust opts to keep the language relatively small in these core areas. Given the plethora of ways you can construct something in e.g. C++, I count that a big win.
Another difference: Swift has syntax for default values; Rust uses a trait instead. In Swift, you simply supply the default value in the definition of the struct or class:
```
struct Point {
    var x = 0.0 var y = 0.0
}

let point = Point()
```
In Rust, you use std::default::Default, which provides a standard value for a given type, and for simple types can be supplied by the compiler even for custom types. Here is the equivalent Rust code:
```
use std::default::Default;

#[derive(Default)]
struct Point {
    x: f64,
    y: f64,
}

let point = Point::default();
```
This is reasonable enough, but we can also supply our own custom implementation if we so desire:
```
use std::default::Default;

struct Point {
    x: f64,
    y: f64,
}

impl Default for Point {
    fn default() -> Point {
        Point { x: 0.0, y: 0.0 }
    }
}

let point = Point::default();
```
Of course, this is trivial for this type, but you can see how it could be useful for more complex types.
The tradeoffs here are our usual suspects: Rust’s re-use of an existing concept/tool within the language (trait) vs. Swift’s use of syntax. Rust is slightly more explicit, making it obvious that a default value is being created—but Swift is perfectly readable and the syntax is consistent with many other languages, and it is shorter.

Both languages use . syntax for member access. Swift:

println("The point is: \(point.x), \(point.y)")

Rust:

println!("The point is {:}, {:}", point.x, point.y);

Swift lets you define items within a struct as mutable or constant. So you can create a variable struct instance, with some of its items immutable:
```
struct PointOnZAxis {
    var x: Double var y: Double let z = 0.0
}

var point = PointOnZAxis(x: 4.0, 5.0)
point.x = 5.0 point.y = 6.0
// This wouldn't compile, though:
// point.z = 1.0
```
This is pretty handy for a lot of object-oriented programming approaches.
And Rust doesn’t have it. There are ways to accomplish the same thing; this isn’t the end of the world. Still, it’s an interesting omission, and it’s very much by design. Rust used to have this feature, and dropped it—and for good reason. Say you had a mutable field in a mutable struct, and then an immutable reference to it; should the mutable field be mutable, or immutable, with that reference?

The Rusty way to do this is to differentiate between public and private data. The above examples don’t make the public/private distinction particularly clear, because they assume everything is within the same module. However, many times, this will not be the case.

mod geometry {
    pub struct Point {
        x: f64,
        pub y: f64,
    }

    impl Point {
        pub fn new() -> Point {
            Point { x: 0.0, y: 0.0 }
        }

        pub fn set_x(&mut self, x: f64) {
            self.x = x;
        }
    }
}

fn main() {
    // Won't compile: the `x` field is private.
    // let mut p = geometry::Point { x: 0.0, y: 0.0 };

    // Will compile: the `new` method is public.
    let mut p = geometry::Point::new();

    // Won't compile: `x` isn't public.
    // p.x = 4.0;
    // You can use the setter, though:
    p.set_x(4.0);

    // You *can* set `y` directly, though, because it's public.
    p.y = 14.0;

    // You can't set fields either way if the instance is immutable.
    let q = geometry::Point::new();

    // This fails because `set_x` requires a mutable reference, but `q` is
    // immutable.
    // q.set_x(4.0);

    // This fails because `q` is immutable, and so all its fields are, too.
    // q.y = 14.0;
}

This is an interesting way of handling this issue. Rust takes the fairly standard use of information hiding (one of the basic principles of most object-oriented programming techniques) and combines it with the language’s normal mutability rules to make it so that the mutability of any given instance data is quite clear: all public members are just as mutable as the struct. If a member isn’t potentially publicly mutable, it isn’t publicly accessible. I really like this, though it took some mental readjustment.
There’s one other difference here, and it’s actually one of the areas Swift and Rust diverge substantially. Rust has struct for all product types; Swift splits them into struct types and class types.
Swift classes have inheritance; there is presently no inheritance in Rust.
Additionally, whereas Rust determines whether to use pass-by-reference or-value depending on details of the type (whether it implements the Copy trait) and expected arguments to a function, Swift makes that distinction between class (pass-by-reference) and struct (pass-by-value) types. Quirky.
Not bad, per se. But quirky.

Edit: I recently bumped into some discussion of data types in C♯ along with C, C++, and Java (here) and discovered that Swift is stealing this idea from C♯, which makes the same copy/reference distinction between struct and class.
One consequence of this: in Rust, you’re always rather explicit about whether you’re accessing things by value vs. by reference. Not so in Swift; you have to remember whether the item you’re touching is a struct type or a class type, so that you can remember whether a given assignment or function call results in a reference or a copy. This is necessary because Swift doesn’t let you make that explicit (trying to hide the memory management from you). And it’s not alone in that, of course; many other high-level languages obscure that for convenience but still require you to think about it in certain circumstances. I’ve been bitten in the past by the value/reference distinction when thinking through the behavior of Python objects, for example, so that’s not a critique of Swift. Moreover, having the distinction between struct and class types does let you be more explicit than you might in e.g. Python about how given data will be handled.
I won’t lie, though: I like Rust’s approach better. (Shocking, I know.)
All that nice initializer syntax for Swift struct types is absent for its class types, which seems strange to me.
Swift supplies some syntax for object identity, since it’s useful to know not only whether two class instances have the same data, but are in fact the same instance. You can use === and !==. Handy enough. To get at this kind of equivalence in Rust, you have to use raw pointers (which are often but not always unsafe; you can do this specific comparison without being unsafe, for example) to check whether the memory addresses are the same.

Rust and Swift (ix)

2015-11-09T22:20:00-05:00

Right off the bat when looking at the definitions for Swift’s and Rust’s enum types, a difference pops out: the use of the keyword case to introduce an enum member in Swift. In one sense, this overloads that keyword, but in another sense it’s fine: pattern matching and enums go hand in hand, so the use in both cases is fairly natural. Rust doesn’t have any special syntax to designate the elements of an enum; they’re just separated by commas.
I am not at all shocked to find that Swift has a variant syntax for its unit type case declarations, where a single case keyword precedes a list of comma-separated cases defined on a single line. (At this point, I would be more surprised not to find a variant syntax for something in Swift!)
Something truly wonderful about both Rust and Swift: enumerated types aren’t just wrappers around integer values. They’re real types of their own. This is powerful.
Rust and Swift also share in having enumerated types that can hold values. The most prominent of these so far in the Swift book are optionals, the Optional enum type, corresponding very closely to Rust’s Option type. Having had these for a bit in playing with Rust, and having gotten familiar with the utility of types like these while reading Maybe Haskell—a delightful book which introduces Haskell and functional programming using Haskell’s Maybe type—I now miss them profoundly in languages which don’t have them. (Which is to say: every language I use on a regular basis professionally: C, C++, Python, JavaScript, etc.).
Swift’s enum types don’t have integer values by default—but they can have them if you define a type and assign a value to each enum case at the definition. These “raw values” are distinct from the “associated values” noted just above. I expect these exist primarily for ease of interoperation with Objective-C.
Rust doesn’t have anything like this, at least that I can think of. The main place it would be useful would be for foreign function interfaces (as in Swift), and this is one of several such gaps in Rust, along with the lack of a straightforward way to map to C’s union types. ~~There are trade offs in terms of adding the functionality to the language, though, as it substantially increases the complexity of what an enum value can be, I think.~~

Edit: This was incorrect. From the Rust Reference section on Enumerations:
Enums have a discriminant. You can assign them explicitly:
```
enum Foo {
    Bar = 123,
}
```
If a discriminant isn’t assigned, they start at zero, and add one for each variant, in order.

You can cast an enum to get this value:
```
let x = Foo::Bar as u32; // x is now 123u32
```
This only works as long as none of the variants have data attached. If it were Bar(i32), this is disallowed.
Initialization of Swift’s raw-valued enum type is quite similar, and pleasantly so, to Python’s initialization of enums.
In a surprising change from the usual, Swift’s syntax for binding variable names when pattern matching against an enum is more verbose than Rust’s, requiring the use of either a leading let on the case statement if all the elements are of the same type, or a let in front of each element otherwise:
```
var matchedValue: String
let matchee = 3.14159
switch matchee {
case 3.14159:
    matchedValue = "pi"
case _:
    matchedValue = "not pi"
}
```
In Rust, a matched pattern can simply bind its value directly:
```
let matchee = 3.14159;
let matchedValue = match matchee {
    3.14159 => "pi".to_string(),
    _ => "not pi".to_string()
};
```
Swift has the ability to do recursive enumerations with its indirect type. This is conceptually interesting, but off the top of my head I can’t think of a time when this would have been useful at any point since I started programming seven and a half years ago. The book’s example of a recursive function aliasing arithmetic expressions is fine, but not particularly illuminating to me. I suspect, though, that it might make more sense if I were more familiar with pure functional programming paradigms.

Edit: a friend points out:

Indirect enums are useful for recursive types in general. There are a lot of these: Lists, trees, and streams are the big ones that come to mind.
All those same lines: Rust does not have the ability to have recursive enumerations at present (or recursive struct types, for that matter), at least without heap-allocating with Box along the way. You can construct such a type, in other words, but you have to be explicit about how you’re handling the memory, and it can’t be stack-allocated.
- For an example of a recursive enumeration type (as well as an interesting/hilarious example of how you can easily confuse the compiler if you do this wrong), see this Rust forum post.
- For some discussion on stack- and heap-allocated memory in Rust, I’ll shamelessly promote my Rust podcast, New Rustacean: take a listen to e005: Allocate it where?

CSS Fallback for OpenType Small Caps

2015-10-19T20:00:00-04:00

I wrote this up as a question on Stack Overflow a bit over a year ago. It has continued to get a fair bit of traffic, so I’ve republished it here and cleaned it up a bit.

The Problem

Over the last year, I’ve worked on a site where small caps are important: setting the text of the Bible. In the Old Testament the name of God is transliterated as Lord but in small caps—not “LORD” but Lord (RSS readers will want to click through and see this on my site). However, the state of OpenType small caps support at the moment is… less than optimal. Safari (even up through Safari 9 on El Capitan, from which I am typing this) still doesn’t support the -webkit-font-feature-settings: 'smcp' option, and a lot of the hits for this website will be coming from mobile.

Unfortunately, “graceful degradation” is problematic here: if you specify both font-variant: small-caps and font-feature-settings: 'smcp' in a browser that supports the latter (e.g. Chrome), the font-variant declaration overrides it, so the horribly ugly old-style version still comes into play. (Note: this is as it should be per the spec: the font-variant declaration has a higher priority than the font-feature-settings declaration). Given the current implementations of font-variant: small-caps, though—shrunken capitals rather than actual small capitals—the result is that using font-variant: small-caps realists in not-so-gracefully degrading everyone’s reading experience.

In the past, I have exported the small caps as a distinct webfont and specified them directly; see this post for a simple example: the first line of each paragraph is specified that way.

While I can do the same thing here (and at least in theory could deliver a pretty small typeface, since I really only need three characters: o, r, and d), I’d prefer simply to enable sane fallbacks. As noted above, however, that’s not possible. I am open to but would very much prefer to avoid server-side solutions (browser detection, etc.) as a point of complexity that is better to minimize, especially given how rapidly browsers change. How else might one solve this problem, and especially are there existing solutions for it?

In the future, font-variant: small-caps will handle this nicely, as per the spec it should display a small-capitals-variant of the typeface if the typeface supplies it. However, at present, no browser supports this (at least, none that I can find!). This means that instead, they all render fake small capitals simply by scaling down actual capitals. The result is typographically unpleasant, and unacceptable on this project.

The Solution(s)

I spent a considerable amount of time researching this and wrestling with it. After digging around as best I could, the top solutions for now are:

`@supports`

Take advantage of the @supports rule in browsers. This is what I initially opted to do on this project.¹ You use the rule this way:

.some-class {
    font-variant: small-caps;
}

@supports(font-feature-settings: 'smcp') {
    .some-class {
        font-variant: normal;
        font-feature-settings: 'smcp';
    }
}

(I’ve simplified by leaving out the prefixed versions; you’ll need to add the -webkit- and -moz- prefixes to get this actually working.) This has the advantage that support for real small caps and support for the @supports rule are very similar:

@supports: Can I Use Feature Queries?: Chrome 31+, Firefox 29+, Opera 23+, Android 4.4+, Safari 9+, Edge 12+, Chrome for Android
font-feature-settings: Using Small Caps & Text Figures on the Web: Chrome, Firefox, IE10+

This isn’t perfect: since IE10/11 don’t implement @supports, you miss one browser—sort of. At this point, IE is a legacy browser, and Edge has had @supports available from the start. Thus, this gets you most of the way there, and it should be future-facing: this should progressively enhance the site nicely. The normal (bad, but functional) small caps are displayed in the meantime, and when browsers eventually get around to using OpenType small caps by default for font-variant: small-caps, this will continue to work just fine. It’s “progressive enhancement” and it’ll work nicely for most purposes.²

Typeface subsetting

As mentioned above, one can create a subset of the typeface that includes only small capitals. This is what I have done for the small caps on this site; see the example in the first paragraph.

To pull this off, you’ll need to start by subsetting the typeface. You can do this manually with a font tool, or (the simpler way) you can use FontSquirrel’s custom subsetting tool in their webfont generator. (Note: You must check the license and confirm that the typeface in question allows this kind of modification. See below.) In the web font generator, first upload the file you wish to modify. Then choose the Expert radio button. Most of the settings you can leave as they are; they’re good sane defaults. Midway down the page you’ll see OpenType Flattening options. Here, select only “Small Caps”. Run the generator. The result will be a complete replacement of the normal lowercase letters with the small caps set.³

In that case, you can simply apply a style to the elements you want to have small capitals, e.g.:

.divine-name {
    font-family: 'my_typeface_smcp', 'my_typeface', serif;
}

The major advantage to this approach is consistency: that typeface is going to display on every browser out there, back to IE5.5, as long as you deliver it correctly using the various hooks required by @font-face.

There are a few disadvantages to this approach, though:

It means delivering another font file. In my case, this would be an acceeptably low size (since I actually only need four characters), but it’s still something to consider in general. It is in any case another HTTP request, which is going to further slow the page load time or at least give you some flash of unstyled text when it reloads.
It may violate the licenses of the typefaces in question. For at least one of the fonts I used on this project, it does: the license explicitly forbids rebuilding the font using tools like FontSquirrel. (FontSquirrel was the tool I used for this approach before, and it works quite well.) This is a make-or-break issue for using a subset of a typeface to accomplish the goal. That being said, if you have a good reason to do it, you may be able to get support from the vendor (especially if they’re a small shop). For the project that prompted this question, I was able to do just that with a nice email—the designer is a great guy.

The other major reason not to do it this way is that it has a significantly higher maintenance cost. If at any point you need to change or update the typeface, you have to go through the subsetting process all over again. By contrast, the first option will simply work, though admittedly not as pleasantly as one might hope, and will not only continue to work but will actually improve over time as browsers increase their implementation of the CSS3 standard.

Conclusion

I opted for the second solution on HolyBible.com—typography was one of the driving differentiators for the site, so I prioritized it and did the necessary legwork for it. In general, though, the first option should work well for most sites. In any case, both ways work, though the first one is a better example of progressive enhancement. And we can all look forward to the day when true small-caps support is available on every browser, right?

For various reasons (especially see note 2 below), I actually opted for the second approach outlined here, which is the same approach I was trying to avoid. Alas.↩
Issues remain: even in the latest Chrome (46 as of the time of this post), using the font-feature-settings: 'smcp' approach has some issues. For example, if you turn on letter-spacing (a fairly common recommendation for small caps), the small caps will revert to normal lowercase letters.↩
From the FontSquirrel blog post that introduced the feature:

If you have a font with OpenType features, you can now flatten some of them into your webfont. For instance, some fonts have small caps built in, but they are completely inaccessible in a web browser. By selecting the “Small Cap” option, the Generator will replace all the lowercase glyphs with the small cap variants, giving you a small cap font. Please note that not all OpenType features are supported and if the font lacks OpenType features, using these options won’t create them.

↩

Rust and Swift (viii)

2015-10-18T11:50:00-04:00

Rust and Swift handle function definition fairly similarly, at least for basic function definitions. In fact, for most basic functions, the only difference between the two is the keyword used to indicate that you’re declaring a function: fn in Rust and func in Swift.
Likewise, both return an empty tuple, (), called the unit type in Rust or Void in Swift. Note, however, that this unit/Void type is not like C(++)’s void or Java’s null: you cannot coerce other types to it; it really is an empty tuple.
Type declarations on functions are basically identical for simple cases, though they vary into the details as you get into generics and constraints in generics.
I have no idea why the Swift team chooses to represent function names like this: function_name(_:second_param:third_param:<etc.>). Perhaps it’s a convention from other languages I’m simply unfamiliar with, but it seems both odd and unhelpful: eliding the first parameter name obscures important information. Also, why use colons for the delimiter?

Edit: I’m informed via Twitter and App.net that this reflects how function names work in Objective C, and derives ultimately from Smalltalk.
Being able to name the items in a returned type in Swift is quite handy; it’s something I have often wanted and had to work around with dictionaries or other similar types in Python.
We’ll see how I feel once I’ve been writing both for a while, but initially I strongly prefer Rust’s more-obvious (if also somewhat longer) -> Option<i32> to return an optional integer to Swift’s -> Int?. I am quite confident that I’ll miss that trailing ? somewhere along the way.
I’m sure there’s a reason for Swift’s internal and external parameter names and the rules about using _ to elide the need to use keyword arguments (but automatically eliding the first one) and so on… but I really can’t see the utility, overall. It seems like it would be better just to have Python-like args and keyword args.
That’s doubly so given that Swift’s rules for default-valued parameters map exactly to Python’s: they need to go at the end, after any parameters which don’t have default values.
Swift’s variadic parameters are nice—though of course limited, since if you have more than one, the compiler may not know how to resolve which destination parameter a given argument belongs with. (I imagine the compiler could be extended to be able to handle multiple variadic parameters as long as they were all of different types, but that’s probably not worth the work or the potential confusion it would introduce.) In any case, it’s a small nicety that I do wish Rust had.
Swift’s variable parameters are… interesting. I can see the utility, sort of, but (probably from years of habit with C and Python and pass-by-reference types), it’s just not a pattern that makes a lot of sense to me right now. No doubt I’ll get used to them in idiomatic Swift, but while Rust doesn’t have a similar feature, I suspect I won’t miss it.
In/out parameters—that is, mutable pass-by-reference types—are available in both languages. The syntax is very different here, as are the semantics.

Swift has the inout keyword, supplied before a parameter definition:
```
func adds4ToInput(inout num: Int) {
    num += 4;
}
```
Rust has instead a variation on every other type definition, declaring the type in this case to be a mutable reference:
```
fn adds_4_to_input(num: &mut i32) {
    num += 4;
}
```
As usual, in other words, Swift opts to use new syntax (in this case, a dedicated keyword) while Rust opts to use the same syntax used everywhere else to denote a mutable reference. In fairness to Swift, though, this is something of a necessity there. From what I’ve seen so far, Swift generally doesn’t (and perhaps can’t?) do pointers or references explicitly (though of course it’s handling lots of things that way under the covers); arguments to functions are a special case, presumably present primarily for interoperability with Objective-C.
Swift’s function type definitions, as used in e.g. function parameter definitions, are quite nice, and reminiscent of Haskell in the best way. Rust’s are pretty similar, and add in its trait usage—because functions types are traits. Once again, I really appreciate how Rust builds more complicated pieces of functionality on lower-level constructs in the language. (Swift may be doing similar under the covers, but the Swift book doesn’t say.)
Again, though, the downside to Rust’s sophistication is that it sometimes bundles in some complexity. Returning a function in Swift is incredibly straightforward:
```
func getDoubler() -> (Int) -> Int {
    func doubler(number: Int) -> Int {
        return number * 2
    }

    return doubler
}

func main() {
    let doubler = getDoubler()
    println("\(doubler(14))")  // -> 28
}
```
Doing the same in Rust is a bit harder, because—as of the 1.3 stable/1.5 nightly timeframe—it requires you to explicitly heap-allocate the function. Swift just takes care of this for you.
```
fn get_doubler() -> Box<Fn(i32) -> i32> {
    fn doubler(number: i32) -> i32 {
        number * 2
    }

    Box::new(doubler)
}

fn main() {
    let doubler = get_doubler();
    println!("{:}", doubler(14));  // -> 28
}
```
If you understand what’s going on under the covers here, this makes sense: Rust normally stack-allocates a function in a scope, and therefore the doubler function goes out of scope entirely when the get_doubler function returns if you don’t heap-allocate it with Box::new.
In both languages, closures and “ordinary” functions are variations on the same underlying functionality (as it should be). In Rust’s case, functions and closures both implement the Fn trait. In Swift’s case, named functions are a special case of closures.
The Swift syntax for a closure is, well, a bit odd to my eye. The basic form is like this (with the same “doubler” functionality as above):
```
{ (n: Int) -> Int in return n * 2 }
```
For brevity, this can collapse down to the shorter form with types inferred from context, parentheses dropped and the return keyword inferred from the fact that the closure has only a single expression (note that this wouldn’t be valid unless in a context where the type of n could be inferred):
```
{ n in n * 2 }
```
The simplicity here is nice, reminiscent in a good way of closures/lambdas in other languages.¹ The fact that it’s a special case is less to my taste.
Rust’s closure syntax is fairly similar to Swift’s brief syntax. More importantly, there’s no special handling for closures’ final expressions. Remember: the final expression of any block is always returned in Rust.
```
|n| n * 2
```
If we wanted to fully annotate the types, as in the first Swift example, it would be like so:
```
|n: i32| -> i32 { n * 2 }
```
There are even more differences between the two, because of Rust’s ownership notion and the associated need to think about whether a given closure is being borrowed or moved (if the latter, explicitly using the move keyword).
Swift has the notion of shorthand argument names for use with closures.² The arguments to a closure get the default names $0, $1, etc. This gets you even more brevity, and is quite convenient in cases where closures get used a lot (map, sort, fold, reduce, etc.).
```
{ $0 * 2 }
```
If that weren’t enough, Swift will go so far as to simply reuse operators (which are special syntax for functions) as closures. So a closure call could simply be + for a function expecting a closure operating on two numbers; Swift will infer that it needs to map back to the relevant method definition on the appropriate type.
The upside to this is that the code can be incredibly brief, and—once you’re used to it, at least—still fairly clear. The downside to this is yet more syntax for Swift, and the ever-growing list of things to remember and ways to write the same thing I expect will lead to quite a bit of instability as the community sorts out some expectations for what is idiomatic in any given instance.
And if that weren’t enough, there is more than one way to supply the body of a closure to a Swift function that expects it: you can supply a block ({ /* closure body */ }) after the function which expects it. Yes, this can end up looking nearly identical to the form for declaring a function:
```
someFunctionExpectingAnIntegerClosure() { n * 2 }
```
But you can also drop the parentheses if that’s the only argument.
```
someFunctionExpectingAnIntegerClosure { n * 2 }
```
In terms of the mechanics of closures, and not just the syntax, the one significant difference between Rust and Swift is the same one we’ve seen in general between the two languages: Swift handles the memory issues automatically; Rust makes you be explicit about ownership. That is, as noted above about the closures themselves, in Rust you may have to move ownership to get the expected behavior. Both behave basically like closures in any other language, though; nothing surprising here. Both also automatically copy values, rather than using references, whever it makes sense to do so.
Swift autoclosures allow for lazy evaluation, which is neat, but: yet more syntax! Seriously. But I think all its other closure syntaxes also allow for lazy evaluation. The only reason I can see to have the special attribute (@autoclosure) here is because they added this syntax. And this syntax exists so that you can call functions which take closures as if they don’t take closures, but rather the argument the closure itself takes. But of course, this leads the Swift book to include the following warning:

Note: Overusing autoclosures can make your code hard to understand. The context and function name should make it clear that the evaluation is being deferred.

Yes, care needed indeed. (Or, perhaps, you could just avoid adding more special syntax that leads to unexpected behaviors?)
Good grief. I’m tired now. That’s a half-dozen variants on closure syntax in Swift.
Remember: there’s still just one way to write and use a closure in Rust.
This takes me back to something I noticed early on in my analysis of the two languages. In Swift, there’s nearly always more than one way to do things. In Rust, there’s usually one way to do things. Swift prefers brevity. Rust prefers to be explicit. In other words, Swift borrows more of its philosophy from Perl; Rust more from Python.
I’m a Python guy, through and through. Perl drives me crazy every time I try to learn it. You could guess (even if you hadn’t already seen) where this lands me between Rust and Swift.

This post is incredibly long, but I blame that on the (frankly incredible) number of variants Swift has on the same concept.

Compare the closure syntaxes especially in Ruby and ES6+.↩
For a similar example in another up-and-coming language, see Elixir, which does almost exactly the same but with & in place of $.↩

Jeb Bush on net neutrality

2015-09-24T07:15:00-04:00

Dear Republicans: your opposition to net neutrality might be justifiable as something other than kowtowing to megacorporations if you ever got around to proposing something else. As is, all you’re doing is propping up some of the nastiest, most anti-consumer companies in the country and sustaining monopolies and duopolies, supposedly in the name of “free markets”.

Stop it.

N.b. This isn’t intrinsically a partisan issue. It’s become one, but mostly because Republicans have felt compelled to do the bidding of the telecom industry for… reasons.

The only thing worse than a government monopoly is a private monopoly.

If Republicans wanted to push for local loop unbundling in place of net neutrality, almost everyone would be for it. (The exception: telecom companies.)

Rust and Swift (vii)

2015-09-19T15:00:00-04:00

I am reading through the Swift book, and comparing it to Rust, which I have also been learning over the past month. As with the other posts in this series, these are off-the-cuff impressions, which may be inaccurate in various ways. I’d be happy to hear feedback! Note, too, that my preferences are just that: preferences. Your tastes may differ from mine. (See all parts in the series.)

Both Rust and Swift have pattern-matching, and with what appears to be fairly similar basic behavior. (I touched on this briefly in my first post in the series.) In Rust this goes under the match construct, with matches specified like <pattern> => <expression|statement>, optionally with guards specified with if expressions. In Swift, patterns are matched using the switch construct, with matches specified like case <pattern>: <expression|statement>, optionally with guards specified with where expressions. (where is also used in Rust, but for generic constraints, not pattern match guards.)
Both languages allow you to bind names to a matched pattern: Swift with case let <name> and Rust simply by using the name in a normal destructuring expression as part of the match definition.

Edit: that’s not quite right. In Rust, you use the @ operator with the variable name you want to bind in the match.

Edit the second: I was mixed up, because Rust actually has both of those options. You can either match directly, e.g. when getting the value of an Option type: Some(value) as the pattern will bind value. But if you need to bind a specific part of more complicated data structure, the @ operator is present to let you do it in a fairly straightforward way.
Both languages allow for the use of _ as a “wildcard” in match definitions. Since match definitions in Rust use the patterns directly, the equivalent of Swift’s C-like default is simply a wildcard match pattern (_ => <-expression|statement>).

One significant difference: like its if blocks, Rust’s match blocks are expressions, so they can be assigned. I.e., you can write this:

let test = 5u32;
let description = match test {
    0..10 => "less than ten",
        _ => "greater than ten",
}
println!("{?:}");  // "less than ten"

Swift doesn’t let you do this; the same thing there would be written like this:

let test: UInt32 = 5
var description: String
switch test {
    case 0..<10:
        description = "less than ten"
    default:
        description = "greater than ten"
}
println("\(description)")

Both languages have break statements, but in Rust they’re only used in loop constructs, while Swift (like C) uses them to escape cases as well. The Swift book gives an example of one place they’re necessary in a switch: to match a case and do nothing there (e.g. default: break). In Rust, you would simply supply an empty block for that scenario (e.g. _ => {}).
Correctly, both languages force you to match exhaustively on relevant patterns. If you’re matching an enumerated type, for example, you must handle every enumerated value. You can of course do this with wildcard patterns or with Swift’s default, but the good thing is that both languages will refuse even to compile if a given pattern isn’t matched.
Swift’s default behavior around its switch statements is sane: it does not automatically fall through into the next statement. It does let you do this, without checking the condition on the next statement (as in C), using the fallthrough keyword. Rust, by contrast, simply doesn’t allow this at all.
Both languages supply named control statements (loops, etc.), with slightly different syntax for naming them. Rust’s, curiously, shares its syntax with lifetime definitions—more on those in a future post.
I don’t believe Rust has anything quite like Swift’s guards, which allow you to leave normal or expected control flow in the main body of a block, with a secondary block for cases where the guard isn’t matched. This isn’t a huge deal, but it does fit as a nice convenience into the typical if let pattern in Swift. Basically, it just lets you elide an empty if block and supply only the else block.

Edit: a friend points out that Swift guards also require you to exit the current scope, so it’s unambiguous what you’re doing if you use them.

Rust and Swift (vi)

2015-09-19T09:00:00-04:00

It kind of feels like this summarizes a lot of things about the overall design of Swift:

Although the two forms are functionally identical, the shorthand form is preferred and is used throughout this guide when referring to the type of an array. —The Swift Programming Language (Swift 2 Prerelease)

The documentation for the various types in Rust’s std::collections module is hilarious and great. Highly recommended.

One thing that jumped out at me reading this chapter of the Swift book (though I don’t think it’s been explicitly discussed yet): Rust doesn’t have named parameters; Swift does. There are good reasons for that in both cases, but I suspect this is one of the small details I’ll miss the most in Rust. I’ve been spoiled by Python.

Swift’s Array type is analogous to Rust’s Vec type (usually created with the vec! macro), not its Array type. Rust Vecs and Swift Arrays are dynamically sized and created on the heap, whereas Rust’s Arrays are statically sized and created on the stack. Syntax for creating Arrays in both languages is quite similar (though the results are different):

Swift:
- Fixed size: let an_array: [Int] = [1, 2, 3]
- Variable size: var an_array = [1, 2, 3]
Rust:
- Array: let an_array: [i32, 3] = [1, 2, 3];
- Vector: let a_vector: Vec<i32> = vec![1, 2, 3];

That’s the long form, of course; both languages have type inference, so you’d rarely write it like that. The usual form would be with the type in all of those cases:

Swift:
- Fixed size: let an_array = [1, 2, 3]
- Variable size: var an_array = [1, 2, 3]
Rust:
- Array: let an_array = [1, 2, 3];
- Vector: let a_vector = vec![1, 2, 3];

Rust also adds the concept of “slices,” which provide access to segments of arrays, and are heap-allocated as pointers to a given item in the array and a length (number of elements) included.

Array operations in Swift are all pretty reasonable, and surprisingly descriptive. They remind me in a good way of Python’s list methods.

There are a lot of ways to interact with Vecs in Rust. (That’s not a bad thing.) A bit surprising to me was the absence of an enumerate method, on Vec itself, but then I discovered that it exists in the IntoIter struct in the same module, which fully implements the Iterator trait. As a result, it has an enumerate function returning an Enumerate struct instance. (Under the covers, I suspect Swift Arrays just implement an Iterable protocol, which is similar to this approach in some ways.)

This makes a point I’m coming back to fairly often: Rust doesn’t necessarily put everything on a single object definition, but rather into a set of related struct or enum types and traits. This is really powerful, but it’s going to take some mental adjustment. In this way, Swift’s structure and semantics are much more like the languages I’m used to than Rust’s are (but even there, the use of protocols gives it considerable new flexibility).

Note that I said semantics, not syntax. Swift and Rust are a great example of how very similar syntax can mask differences in semantics. (For another such example, compare JavaScript’s syntax and semantics to Java’s: they’re superficially similar syntactically, and light years apart semantically.)

Swift’s Set type and Rust’s roughly analogous HashSet both have a contains method which behaves much like Python’s in keyword. Indeed, and perhaps unsurprisingly, the two types implement many of the same methods in general. This is perhaps to be expected given that the language around sets (as a mathematical concept being mapped down into a representation in a program) is quite standardized.

Because of their stricter typing systems, both Rust and Swift require you to specify the types used in their mapping constructs (Rust has HashMap and Swift has Dictionary), though of course both can infer this as well in certain cases. At the most basic level, you can’t use arbitrary (hashable) types as keys in mixed fashion like you can in e.g. Python’s dict type, but in practice this shouldn’t matter, for two reasons:

It’s generally inadvisable to use different types for keys in the same dictionary anyway. To me, at least, that usually indicates the need to step back and think more carefully about the types and data structures I’m using.
For the occasional case where it is appropriate, I wonder if you could declare the type as generic in either Rust or Swift. I’m putting this down as a TODO item for myself to find out!

I really wish that Swift used the Python-style curly-brace delimited syntax ({'key': 'value'}) for its dictionary literal initializers. I can see, from a syntax reason, why it doesn’t: that would overload the block syntax (which Python can avoid because it’s white-space delimited). But it’s really convenient.

Along similar lines, I can see why the Swift designers chose to make all iterables have literal initializers using braces ([...]); it makes parsing fairly straightforward. However, the result is that it’s pretty difficult to see at first glance what you’re dealing with. It could quite easily be an Array, a Set, or a Dictionary.

This highlights a too-little-appreciated aspect of programming language design: readability. However much we programmers enjoy writing code, the reality is that we will all spend a great deal of our time—probably even a majority of it—reading it instead. Thus, while we should care about conveniences for writing code, and being overly verbose can be a pain, we should also concern ourselves with the ease of comprehending code when it is read, and the syntax and conventions a language embraces are a big part of this.

The Dictionary type in Swift is a pretty close analog to Python’s dict, right down to several of the method names. the same is true of Rust’s HashMap. That’s not a bad thing by any stretch of the imagination.

Rust and Swift (v)

2015-09-12T13:45:00-04:00

I’ve been working on learning Swift over the past couple weeks, and had spent the month prior to that doing a deep first dive on Rust. This kind of approach, learning two languages basically at the same time, is entirely new to me, and for good reason. Programming languages are not trivial to learn, and to learn them meaningfully one must practice with them a great deal.

I’m doing this largely of necessity. I’m hoping to build an application with a very capable, performant cross-platform core language (Rust), but planning to ship a native OS X app (first) when all is said and done. My desire to make the core libraries portable rules out Swift immediately. To be frank, so does the fact that it’s an Apple language: I am happy to use Apple’s tools on its platform, but I don’t want to shackle myself to their choices in the long run. Too, having good Rust experience is likely to be valuable in many other contexts.

So I need to learn both.

And, while I wouldn’t ordinarily recommend this course of action—indeed, unless you already have a fair bit of programming experience and already know several languages, I’d actively recommend against it—I’m finding it enormously profitable. The languages have been designed in roughly the same time frame, cite many of the same influences, and overlap substantially in terms of audience and goals. Yet they are, as this series has already highlighted, quite different languages in many ways.

Learning them in parallel is helping me see the trade-offs each one has made, and force me to think about why they differ in the ways they do. In particular, I think I have a much better idea what’s going on “under the covers” in each language and therefore know what to expect of them better. This, in turn, has dramatically deepened my grasp of the languages relative to the amount I’ve been looking at them, compared to previous language-learning efforts. (It also helps that I’ve already learned a number of languages, of course, and that I’ve been pushing my brain into the learning-programming-languages space via reading about Haskell, functional patterns in JavaScript, and so on this year.)

I have a long way to go in both languages, of course. Reading on nights and weekends, and the little bit of playing I’ve been able to do with each of them, is no replacement for just sinking my teeth into a project and finding the pain points. Nonetheless, I’m really glad to be learning these two languages together. If you’re up for a challenge, try it sometime! You’ll be surprised how much you learn.

If-expressions in Rust

2015-09-12T11:05:00-04:00

I love the fact that all if statements in Rust are expressions. It gives you a great deal of expressitivity in the language.

Let’s contrast with Python (which I love, for the record). In Python, you can do something like this:

some_condition = True
if some_condition:
    a_value = "Yeah!"
else:
    a_value = "Oh, sads."

Those are statements in the body of the if/else block; you can’t assign the block itself to a_value. However, like C, C++, Java, etc., Python does provide an expression-type conditional, a ternary expression.

So you can also do this:

some_condition = True
a_value = "Yeah" if some_condition else "Oh, sads."

This expression form of the if block is what all Rust if blocks are. So in Rust, the normal long form is:

let some_condition = true;
let a_value = if some_condition {
    "Yeah!"
}
else {
    "Oh, sads."
}

(You could also write this with a let mut a_value and then set its value inside the conditional blocks, but that’s not at all good form in Rust.)

And of course, you can shorten that rather nicely where the expressions are brief enough:

let some_condition = true;
let a_value = if some_condition { "Yeah!" } else { "Oh, sads." }

But this gets really nice when you have more complicated work to do in a Rust conditional. It doesn’t matter how many things going on inside an if expression; it’s still an expression. As such, you can also write this:¹

let some_condition = true;
let a_value = if some_condition {
    let the_answer = 42;
    let theme = "Take my love, take my land...";
    "Yeah!"  // An expression!
}
else {
    let the_question = "What do you get when you multiply six by nine?";
    let song = "You can't take the sky from me!";
    "Oh, sads."  // An expression!
}

Obviously this is totally contrived and silly; the point is that no matter what the internals are, if blocks are expressions, and their final expressions can be assigned like any other.

As a note: I got here because I was originally thinking you couldn’t do a one-liner like you can in Python. As shown above, that’s totally false, and in fact the Rust version is much more capable than Python’s, because you don’t need a dedicated ternary when all if blocks are expressions. Rust used to have a C-style ternary (<condition> ? <value if true> : <value if false>) but it was removed during the lead-up to the 1.0 release—a decision I wholeheartedly affirm.

Note that under normal conditions the compiler won’t actually accept this because of the unused names.↩

Rust and Swift (iv)

2015-09-10T21:05:00-04:00

Both Swift and Rust directly address the issue of having to worry about memory allocation and safety. They do it in different ways, though: Swift by automatic reference counting, Rust by its concept of ownership. For a lot of day-to-day development, I can see the Swift approach being a win for the same reason a language like Python or Ruby is: having that all handled for you is nice. Having the power Rust gives you comes at the price of increased cognitive load from having to reason about ownership.

To put it another way: all programming languages have to make trade-offs. Although I like Rust’s better than Swift’s so far, I’ve no doubt I will find any number of things to appreciate about Swift over Rust. You can’t have everything.

This caught my attention in part because dealing with things like strings (or other pass-by-value types) in Swift is rather more straightforward than in Rust. The outcomes are much the same, but since all Strings in Swift are passed by value (never by reference), you simply don’t have to think about modification—even safe modification!

Rust of course had the Copy trait which lets you do this, but the point is that the “ergonomics” are slightly nicer in Swift.

Also, the string interpolation Swift does is nice. That’s one thing I really wish Rust had. It’s Python-style string formatting macro is great, but being able to interpolate values ("strings with \(variables)" or even "embedded expressions like \(2 + 4)") is very nice.

Swift’s approach to strings in general seems well-thought-through and gives appropriate levels of attention to the details which make handling complex or non-Western languages much more manageable. As a typography geek, I appreciate this a great deal.

That said, since Swift’s strings do handle all those edge cases for Unicode, you lose some standard string access patterns and lose much (maybe all?) insight into the internal structure of the string. That may be good, and may be bad, depending on the circumstance. Like I said: trade-offs.

Actually, on reading further, the way Swift handles Unicode strings is pretty nice. It does give you insight into those, via specific methods for different representations. I particularly appreciate that it’s you deal with them as the standalone String type as well as giving you direct access to the code points—and not just one Unicode code point set, but any of UTF8, UTF16, or UTF32 (Unicode scalars). Trust Apple to pay close attention to text.

Rust’s strings are good, but not quite as sophisticated (presumably for simplicity around the memory mapping). All Rust String or str instances are composed of UTF32 Unicode scalars, encoded as UTF8 sequences. It doesn’t have some of the convenience methods Swift does for getting any of the other representations. That said, I expect this should show up rarely if at all in my ordinary usage. Importantly, the fundamental storage is the same: both use scalars.

This was the first section where it didn’t feel like Rust was just a clear overall “winner” over Swift. Some of the trade-offs between the language designs are more apparent here, and I do appreciate the “ergonomics” of Swift in a number of these things.

Rust and Swift (iii)

2015-09-07T11:55:00-04:00

I just hit operators in the Swift book. First question: are operators special syntax, or are they sugar for protocols? (Every modern language I use or even have played with handles them as sugar for another language construct—Python, Ruby, Io, Elixir, and Rust, to name just a few ranging over a substantial variety of ages and styles.)

Oh. I did the requisite digging, and operators are functions (which is okay) defined in the ~~global namespace (:sigh:)~~ Swift module.¹ I say “okay” rather than good because the justification offered is that this is the only way to make the operators work as binary operators between existing instances of types. But that elides the fact that, if that’s the case, it is so because of other language design decisions. This seems like a perfect place to use a protocol, but perhaps (unlike Rust’s trait) they’re not sufficiently capable to handle this? That’s an open question; I have no idea about the answer.

Interestingly, Rust has several fewer operators than Swift, even apart from those mentioned in my previous post. It drops the pre- and post-increment operators entirely (as does Python), since their results can always be accomplished in other ways with less potential for confusion. Swift keeps them, no doubt in part because most (Objective) C programs are deeply familiar with them and with idioms associated with them.

I learned a few new things about Rust’s operators as well: the Boolean || and && operators and its bitwise | and & operators differ not only in that the former are short-circuit operators and the latter are not. Obviously you can also do things like bit-wise flag operations with the latter, but the reference emphasizes the short-circuiting behavior. This makes perfect sense, but it wasn’t something I’d ever considered explicitly before.

There is no ternary operator in Rust, because of how it handles the relationship between expressions and statements. Swift keeps it. That’s an interesting reflection of differences in design: Rust dropped it because if blocks are expressions, so it’s redundant, and they have had a goal of removing unnecessary features. (See the discussion on dropping the ternary operator—with an interesting aside from Brendan Eich on JavaScript—here). Note that this is not a criticism of Swift, just an observation, though I do really like Rust’s expression-driven approach.

The ?? “nil coalescing operator”, on the other hand, I actively dislike. This seems like shorthand for the sake of shorthand, partly necessitated by the existing drive toward shorthand with optional types in Swift. Sometimes brevity can lead to decreased clarity. Eliding too much, or subsuming it into shorthand, makes the language harder to hold in your head and requires you to slow down more for parsing each line.

Nothing surprising (or different) between the standard boolean operators in the two languages.

I wonder how many times the word “concise” (or synonyms of it) appear in the Swift book? It’s increasingly clear to me reading that brevity is one of the primary design goals. Maybe it’s just me, but that actually seems a little weird. Brevity is good so far as it goes, but legibility is much better.

See edit in discussion of functions and global namespace in part ii.↩

Rust and Swift (ii)

2015-09-06T10:20:00-04:00

At first blush, I find the extra syntax around optionals in Swift more confusing than helpful. I think this comes down to my preference for a more Python-like approach: “Explicit is better than implicit” and “There should be one– and preferably only one –obvious way to do it” both militate against the multiple different ways you can handle optional values in Swift. Optional types are created in one of two ways:

with the ? operator on a type definition, creating an explicitly wrapped type which must be checked in some way.
with the ! operator on a type definition, creating an “implicitly unwrapped optional” by forcibly unwrapping it (and creating a runtime error if the optional is empty)

After creating an optional, you can get at its contents by:

using the if let or while let constructs to bind the optional value’s non-nil value for a block
using the ! operator on a variable name, explicitly unwrapping it (and creating a runtime error if the optional is empty)

By contrast, in Rust you always have to explicitly unwrap the item, using the unwrap method or pattern matching. There are no implicitly unwrapped types. Moreover, there is no special syntax around creating optional types in Rust: you just declare them with an Option type or another type that impls the Option behavior. The “shortcut” behavior around error handling, try!, isn’t special syntax, but application of another standard language construct (in this case, a macro).

The discussion of assert in the Swift book re-raises the question about the global namespace:

“You write an assertion by calling the global assert(_:_:) function.”

This continues to suggest strongly that Swift does in fact have a true global namespace, not an automatically-imported prelude. That can make a big difference for applications in certain spaces (e.g. systems programming), when you might have good reason to want to replace the standard library’s approach with a different one. (See Rust’s #[no_std] docs and the related RFC.)

Edit: “strongly suggests” or no, I have now been reliably informed that I was mistaken—and am happy to have been wrong here. As in Haskell, these functions are implicitly imported and belong to the Swift module.

In Rust, assert! is a macro, not a function, which is an interesting but perhaps not especially important distinction in this particular case. (It might be, though; I’d have to see the implementation of each to see how they play out differently.)

In any case, this also highlights another large difference between the two: testing is front and center in Rust, and barely receives a mention so far in the Swift book (and isn’t in the table of contents). Having language-level support for testing is a big deal.

Language tour and first chapter of the language guide down, my sense is that Swift is a substantially better language than C or C++ (and presumably than Objective C, but since I don’t know that language I can’t speak to it) for app design, but that Rust is a better language yet. Both far more modern than their predecessors, but they approach the same problems in surprisingly different ways, relatively similar syntax notwithstanding. So far, I like the Rust approach better.

In particular, more syntax is not my preferred way to tackle these things. Providing good language constructs and primitives on which to build seems better in many ways:

It substantially reduces the cognitive load for the developer, by keeping the number of constructs small and simply varying how they are applied.
It increases the quality of those primitives, because it forces the language deadness to make sure they actually address the full problem space.
It lets developers approach the same problem in ways the language design team may not have anticipated, and over time the community may find shared conventions that improve on the std approach, and nothing has to change in the language spec (or the compiler!) to adopt those changes.
In general, then, it makes change much easier to manage, and change can be community-driven rather than requiring the language design team to manage it.¹

This may of course be intentional on Apple’s part with Swift. Maintaining tight control over its tooling is very typical of modern Apple.↩

Rust and Swift (i)

2015-09-04T22:59:00-04:00

I started writing these responses in a Slack channel of developers I participate in as I worked through the Swift book. I realized after a bit that it would make a better blog post than chat room content, so here we are. This is all entirely off-the-cuff: me just thinking out loud as I read; this is by no means expert opinion.

I later turned this into the first part of a whole series comparing Rust and Swift!

..< – seriously?

That has to be one of the most annoying operators I’ve ever seen. It ends up with cognitive noise because <name initially processes as “starting a generic” and you have to re-parse it visually and mentally.
After the first chapter of the Swift book, my impression is “a poor man’s Rust”; my gut feel based on that first pass and everything I’ve seen and read about Swift over the past two years is that it’s roughly what you would get if you took Rust’s syntax and replaced Rust’s hard safety goals with the aim of mapping to ObjC semantics. (To be fair to Apple, that interoperability was probably necessary.)
An example that jumps out at me as immediately illustrative of the difference in approach the languages take is the way you pass structures by reference vs. copy. In Swift, that’s done via two completely distinct language constructs, structs and classes respectively.

In Rust, there is just the struct type to handle both of those. They’re immutable unless you declare them with mut, and you can pass them via copy simply by implementing the Copy trait (which seems roughly analogous to Swift’s protocol, but I’ve not yet dug deeply enough to see how they differ). Those things aren’t baked into the language, but use simpler language building blocks to define behavior into the library.
I saw someone do a write up a while back arguing that Go isn’t a bad language, it just isn’t a good language. My first impression of Swift, after having spent the last month with Rust, is very much along those lines.
Huh. Here’s something that I appreciate about Rust, Haskell, and others now that I didn’t before: there’s a difference between implicitly/automatically importing a prelude or a given set of standard library functions, and having actually global functions. Does Swift actually have functions like print in a global namespace, as the book seems to imply, or they being imported automatically a la Rust/Haskell/etc.?

Edit: it appears Swift does likewise, but that you can’t access the relevant module directly. Which is halfway there.
Hmm. Why have Double and Float—just for ObjC interop, I guess?

Edit: follow-up from a conversation with a friend: it’s because you have 32- and 64-bit architectures out there; sometimes you don’t want 64 bits of floating point precision for that reason. Note that Rust also has this distinction; you can declare things as f32 or f64.
Extending the above note on classes and structs and protocols vs. Rust’s approach: the same thing is true about extension, which is a distinct concept from implementing a protocol; again, in Rust these are both just handled with a single language construct, impl. That’s not because impl is overloaded, but rather because the underlying language machinery is the same for the two things. (edited)
(I’ve a feeling learning Swift is going to turn me into even more of a Rust fanboy.)
Reading the two books in close sequence like this is proving really productive mentally for thinking about how the two handle the same issues. I’ve never done anything quite like this before, and it’s fascinating.
I have an increased appreciation for Rust’s use of semi-colons to turn expressions into statements, and thereby to distinguish clearly between the two (among other things, allowing for implicit return of anything that’s an expression).
Another interesting comparison: Rust’s match and Swift’s switch and case fill the same role of pattern matching. I’m curious to see how they differ. Does Swift do matching on arbitrary expressions?

Also, I see where the syntax choices came from in both, and while I slightly prefer Rust’s, I think both make reasonably good sense; Swift’s will understandably be more familiar to C and ObjC programmers, and that’s a perfectly defensible approach. Seen that way, it is expanding on the C-style construct (even if it’s actually doing something substantially more sophisticated than that under the hood by being a form of actual pattern matching).

Next: Basic types and the syntax around them.

On Editing Podcasts

2015-08-24T20:16:00-04:00

Last week, Alan Jacobs posted a few thoughts on the overall quality of podcasts. While he’s since acknowledged that part of his challenge with podcasts is that his bar is extremely high, I think his original piece bears quoting and responding to briefly, including a few thoughts about how Stephen and I handle Winning Slowly.

From his piece:

Podcasts, overall, are

People struggling to articulate for you stuff you could find out by looking it up on Wikipedia (e.g. In Our Time);

People using old-timey radio tricks to fool you into thinking that a boring and inconsequential story is fascinating (e.g. Serial);

People leveraging their celebrity in a given field as permission to ramble incoherently about whatever happens to come to their minds (e.g. The Talk Show); or

People using pointless audio-production tricks to make a pedestrian story seem cutting-edge (e.g. Radiolab).

I actually happen to basically agree with those critiques. However, one category he left out is: people podcasting the way people blog. And this is where many of the most interesting podcasts I listen to come in. It’s also basically where Winning Slowly fits: you can think of our show like an audio version of a blog post. It’s not as carefully considered or edited as a long-form magazine piece (or, in its respective medium, a professionally produced radio show). But like blog posts, the fact that it’s a bit more off the cuff and that it’s not the incredibly tight work that you find in a magazine can actually be attractive at times. Many of my favorite podcasts are very conversational and not heavily produced.

But—and here I think Jacobs is absolutely correct—all of the shows I really enjoy make a point to edit their shows. They clean up the audio from artifacts, they cut segments that were off topic, they make sure the levels are good between the different members of the podcast, and so on. And while you don’t have to do those things to have a podcast, any more than you need to edit the things you write to have a blog, you do need to do them if you want to have a good show. Sadly, this is where a number of shows I otherwise might enjoy show themselves to the door.

There is a reason Stephen and I spent a whole “beta” season of Winning Slowly ¹ working not only on what we wanted the show to be about, but finding its voice and tone, the structure of the episodes, and the quality of our audio. We wrestled with the audio output from mediocre microphones and adopted seemingly silly practices like putting blankets over our heads and microphones and laptops while recording so that we can get better sound spaces. We have taken the time to learn about compression and limiting and other audio editing techniques, and work hard to get the mix between our intro and outro music and our own voices correct. And we cut things mercilessly.

For example, here is the blooper reel from 3.05, which consists of only the funny parts of what I cut from the show (there was probably as much again that I just removed and didn’t include):

That doesn’t begin to touch all the “umms” and long pauses and overly heavy breathing and do-overs we cut out (though, because this was a particularly rough episode, it does give you an idea). The result, as I think most of our listeners would agree, is a show that’s pretty tight as far as the audio goes.

In terms of content, different shows will have a different feel, of course. Some will require more planning. New Rustacean, a new show on learning Rust I’m hoping to launch later this week or early next week, requires a lot of planning. Sap.py, the fun little show my wife and I are about to launch, about her adventures in learning Python, requires basically no planning. Winning Slowly doesn’t require a lot of formal planning, but it does require Stephen and me to keep a good eye on ongoing stories in our fields of technology, religion, ethics, and art, and to discuss big-picture ideas regularly and actively. Some episodes, we outline carefully (like the one we recorded today, which will come out next Tuesday). For others, we can basically just wing it (like the one we recorded a week ago and which comes out tomorrow). But if our podcast is good, and I really do think it is, it is because we take the time to work at making it good. Just like you have to do on a blog, or really anything else in life.²

13 published episodes, and one we dropped entirely!↩
One big difference between a podcast and a blog is that it actually takes a lot more work to make a good podcast than a good blog post. Audio editing is much more involved than editing writing, and speaking intelligently for any length of time—whether off the cuff, with a detailed outline, or as an interviewer—is much harder to get right than writing, where you can polish to your heart’s content.↩

High- and Low-Level Programming Languages

2015-08-07T20:00:00-04:00

It occurred to me while listening to Edwin Brady talk about Idris on the Type Theory Podcast,¹ having just spent a few weeks starting to learn Rust: “low-level” has at least two meanings in software. One is whether something has manual memory management or is garbage collected, reference counted, or otherwise manages memory itself. This is what people often mean when they talk about C, C++, etc. as being “low-level” and languages like Python or Ruby or C♯ being high-level.

But then you toss in a language like Rust, and things start to get a little more complicated. Rust can do the same kind of direct memory management that makes C or C++ a good language for things like writing operating system kernels. [1,2,3] But it is also memory-safe, at least in ordinary usage. Like C♯, you have to be explicit about any unsafe code, with the unsafe keyword on any blocks that do memory management that isn’t safe. And the vast majority of Rust code is safe.

More than that, though, Rust feels like a high-level language. It gives you higher-kinded functions, generics, traits-based composition of types, hygienic macros, and the implementation of many important parts the essentials of the language in the library. If you need to patch something, or extend something, you can do that in a straightforward way. In short, it gives you lots of good abstractions like you would expect in a high-level language.

Rust is low-level in that you can write (and people are writing) systems-level programs in it. It is high-level in that it lets you express things in ways normally associated with languages like Haskell or OCaml or Python or Ruby. To put it simply: it’s low-level in its ability to address the computer, and high-level in the abstractions it hands to a programmer. That’s a powerful combination, and I hope more languages embrace it in the years to come.

Yes, I know that’s insanely nerdy. What did you expect?↩

Reeder 3 for Mac Beta

2015-07-30T09:26:00-04:00

Ooh, look! A beta for Reeder 3! Shiny!

SMuFL and MusicXML to W3C

2015-07-28T12:29:00-04:00

Another one in the music industry—but in this case, companies taking the long view and advancing the good of the whole community, rather than just their own bottom line. (Spreadbury, the guy behind SMuFL, was one of the team laid off in the aforementioned layoff from the Sibelius team, and now heads the product development for a new notation software tool from Steinberg.)

Sibelius 8

2015-07-28T12:25:00-04:00

Avid: charging Sibelius users more money than ever for less value than ever, after laying off their dev team a couple years ago just to maximize profits.

This is not Winning Slowly material here, folks. They lost me (and many other) customers along the way, and they’re headed further down that road here.

Subscription models for software can be valuable and reasonable—but the providers have to justify them with product to match. Avid isn’t, and hasn’t been. I’ve no doubt they’re continuing to profit in the short term, but this will no doubt erode their market position and waste an amazing product in the long term. Greed destroys good things.

Academic Markdown and Citations

2015-07-26T13:50:00-04:00

Much of my past few weeks were taken up with study for and writing and editing a paper for one of my classes at Southeastern. I’ve been writing all of my papers in Markdown ever since I got here, and haven’t regretted any part of that… except that managing references and footnotes has been painful at times.

Footnotes in Markdown look like this:

Here is some text.[^fn]

[^fn]: And the footnote!

This poses no problems at all for normal footnotes. Academic writing introduces a few wrinkles, though, which means that this has always been the main pain point of my use of Markdown for writing papers.

Many academic citation styles (including the Chicago Manual of Style, on which our seminary’s style guide is based) tend to have a long version of the footnote appear first, followed by short versions later. Nearly all academic citations styles make free use of the “ibid.” abbreviation for repeated references to save space, time, and energy. Here is how that might look in manually-written footnotes, citing the very paper in which I sorted this all out:

Some text in which I cite an author.[^fn1]

More text. Another citation.[^fn2]

What is this? Yet _another_ citation?[^fn3]

[^fn1]: So Chris Krycho, "Not Exactly a Millennium," chriskrycho.com, July 22,

2015, http://v4.chriskrycho.com/2015/not-exactly-a-millennium.html
(accessed July 25, 2015), ¶6.
[^fn2]: Contra Krycho, ¶15, who has everything _quite_ wrong.
[^fn3]: ibid.

This seems straightforward enough, though it is a bit of work to get the format right for each different kind of citation (articles, books, ebooks, electronic references to articles…). Things really get complicated in the editing process, though. For example, what if I needed to flip the order of some of these notes because it became clear that the paragraphs needed to move around? This happens frequently during the editorial process. It becomes particularly painful when dealing with the “ibid.”-type references, because if I insert a new reference between two existing references, I have to go back in and manually add all that the reference content again myself.¹

Enter Pandoc and BibTEX.

Managing Citations

The idea of plain-text solutions to academic writing is not especially new; only the application of Markdown to it is—and that, only relatively. People have been doing this, and documenting their approaches, for quite a while. Moreover, tools for managing references and citations have existed for quite some time as well; the entire LATEX toolchain is largely driven by the concerns of academic publishing, and as such there are tools in the LATEX ecosystem which address many of these problems.²

One such is BibTEX, and the later (more capable) BibLATEX: tools for managing bibliographies in LATEX documents. The BibTEX/BibLATEX approach to managing citations in a document is the use of the \cite command, with the use of “keys” which map to specific documents: \cite{krycho:2015aa}, for example.

This is not Markdown, of course. But other folks who have an interest in Markdown and academic writing have put their minds to the problem already. Folks such as Jon MacFarlane, the originator and lead developer of Pandoc, perhaps the single most capable text-conversion tool in existence. As it turns out, Pandoc Markdown supports a citation extension to the basic markup. It’s just a variant on the BibTEX citation style that feels more at home in Markdown: a pair of braces and an @, plus the citation key, like [@krycho]. Moreover, Pandoc knows how to use BibTEX libraries, as well as many others, and Citation Style Languages (CSLs) to generate markup in exactly the format needed for any given citation style.³

Instead of writing out all those citations details by hand, then, I can just format my footnotes like this (assuming the citekey I had set up for the article was krycho:revelation:2015):

Some text in which I cite an author.[^fn1]

More text. Another citation.[^fn2]

What is this? Yet _another_ citation?[^fn3]

[^fn1]: [@krycho:revelation:2015], ¶6.
[^fn2]: Contra [@krycho:revelation:2015], ¶15, who has everything _quite_ wrong.
[^fn3]: [@krycho:revelation:2015].

This is much simpler and, importantly, has the exact same form for each citation. Pandoc will take care of making sure that the first reference is in the long form, later references are in the short form, and repeated references are in the “ibid.” form as appropriate. It even renders a properly sorted and structured Works Cited section.⁴

The slightly complex command I used to generate a Word document from a Markdown file with citations (using my own BibTEX library and the Chicago Manual of Style CSL) on the command line is:⁵

$ pandoc revelation.md --smart --standalone \
--bibliography /Users/chris/Documents/writing/library.bib \
--csl=/Users/chris/Documents/writing/chicago.csl -o revelation.docx

To see an extended sample of this kind of usage in practice, take a look at the Markdown source for the paper I wrote last week, using exactly this approach. Every footnote that references a specific source simply has a cite key of this variety. The header metadata includes a path to the bibliography file and a CSL. (These could be configured globally, as well, but I chose to specify them on a per-file basis so that if I want or need to use different styles or a separate library for another file at a later time, I can do so with a minimum of fuss. More on this below.)

Here is the rendered result. You can see that it automatically generated everything right down to the “ibid.”-style footnotes. I made a few, fairly minimal tweaks (replacing the search URL with an ATLA database catalog reference and inserting a section break before the Works Cited list), and turned the paper in—confident, for the first time since I started seminary, that all of the references were in the right order and the right format. With carefully formatted reference documents (with their own style sets),⁶ I was able to generate an actually nice PDF version of the paper from another Word document, as well.⁷

And, better yet, you don’t even have to put citations in footnotes. As @anjdunning pointed out in a tweet response to the original version of this post:

@chriskrycho Don’t put citekeys in a footnote: write everything as inline citations and it will also generate notes when asked by CSL def. ∞ July 26, 2015 17:19

In my standard example from above, then, you could simply do this:

Some text in which I cite an author.[@krycho:revelation:2015, ¶6]

More text. Another citation.[Contra @krycho:revelation:2015, ¶15, who has
everything *quite* wrong.]

What is this? Yet _another_ citation?[@krycho:revelation:2015]

This will generate the same markup for my purposes here; and as @anjdunning noted, it goes one step further and does what’s appropriate for the CSL. This might be handy if, for example, you wanted to use the Chicago notes-bibliography style in one format, but switch to a simpler parenthetical citation style for a different medium—or even if you had a paper to submit to different journals with different standards. Having the citations inline thus has many advantages.

Now, there are still times when you might want to split those out into distinct footnotes, of course. That second one is a good candidate, at least for the way I tend to structure my plain-text source. I find it useful in the case of actual footnote content—i.e. text that I’m intentionally leaving aside from the main text, even with reference to other authors—to split it out from the main flow of the paragraph, so that someone reading the plain text source gets a similar effect to someone reading the web or Word or PDF versions, with the text removed from the flow of thought. In any case, it’s quite nice that Pandoc has the power and flexibility such that you don’t have to.

Finally, you don’t actually need the brackets around the citekey, depending on how you’re using the reference. If you wanted to cite the relevant author inline, you can—and it will properly display both the inline name and a reference (footnote, parenthetical, etc.) in line with the CSL you’ve chosen. If I were going to quote myself in a paper, I would do something like this:

As @krycho:revelation:2015 comments:

> This was a hard paper to write.

This is extremely powerful, and while I didn’t take advantage of it in my first paper using these tools, you can bet I will be in every future paper I write.

All those references

Of course, as is probably apparent, managing a BibTEX library by hand is no joke. Entries tend to look like this:

@book{beale:revelation:2015,
        Date-Added = {2015-07-20 21:16:02 +0000},
        Date-Modified = {2015-07-20 21:21:05 +0000},
        Editor = {G. K. Beale and David H. Campbell},
        Publisher = {William B. Eerdmans Publishing Company},
        Title = {Revelation: A Shorter Commentary},
        Year = {2015}}

While there is a lot of utility in having that data available in text, on disk, no one wants to edit that by hand.⁸ Gladly, editing it by hand is not necessary. For this project, I used the freely available BibDesk tool, which is a workable (albeit not very pretty and not very capable) manager for BibTEX:

BibDesk – open to the library for my Revelation paper

Once I filled in the details for each item and set a citekey for it, I was ready to go: BibDesk just stores the files in a standard .bib file on the disk, which I specified per the Pandoc command above.

BibDesk gets the job done alright, but only alright. Using a citation and reference management tool was a big win, though, and I fully intend to use one for every remaining project while in seminary—and, quite possibly, for other projects as well. Whether that tool is BibDesk or something else is a different matter entirely. (More on this below.)

To the web!

I wanted something more out of this process, if I could get it. One of the reasons I use plain text as a source is because from it, I can generate Word documents, PDFs, and this website with equal ease. However, Python Markdown knows nothing of BibTEX or citekeys, to my knowledge—and since I render everything for school with Pandoc, I have long wanted to configure Pelican to use Pandoc as its Markdown engine instead of Python Markdown anyway.

As it happens, I actually set this up about a month ago. The process was pretty simple:⁹

I installed the pandoc-reader Pelican extension.
I set the plugin path in my Pelican configuration file.
I specified the arguments to Pelican I wanted to use.

The only additional tweaks necessary to get citation support were calling it with the '--filter pandoc-citeproc' arguments, which lets it process any bibliography data supplied in the header metadata for the files. Calling Pandoc with --bibliography <path to bibliography> (as in my example above) is a shortcut for calling it with --metadata <path to bibliography> and the --filter pandoc-citeproc arguments. I could just supply the bibliography directly in the call from Pelican, but this would limit me to using a single bibliography file for all of my posts—something I’d rather not limit myself to, since it might make sense to build up bibliographies around specific subjects, or even to have smaller bibliographies associated with each project (exported from the main bibliography), which could then be freely available along with the contents of the paper itself.¹⁰ (On this idea, see a bit more below under The Future.)

One word of warning: Pandoc is much slower to generate HTML with --filter pandoc-citeproc than without the filter, and the larger your site, the more you will feel this. (The time to generate the site from scratch jumped from about 10s to about 30s for me, with 270 articles, 17 drafts, 2 pages, and 1 hidden page, according to Pelican.) Pandoc has to process every article to check for citations, and that’s no small task. However, if you have Pelican’s content caching turned on, this is a one-time event. After that, it will only be processing any new content with it; total generation time is back down where it was before for me: the effort is all in generating the large indexes I use to display the content for the landing pages and for category and tag archives.

And the result: that same paper, rendered to HTML on my website, with citations and works cited, generated automatically and beautifully.

Other site generators

I don’t know the situation around using Pandoc itself in other generators, including Jekyll—I simply haven’t looked. I do know, however, that there is some tooling for Jekyll specifically to allow a similar workflow. If you’re using Jekyll, it looks like your best bet is to check out jekyll-scholar and the citeproc-ruby project, which (like pandoc-citeproc) enables you to embed citations and filter them through CSLs to generate references automatically. As a note: you should definitely be able to get those working on your own deployment sites, but I have no idea whether it’s possible to do them with the GitHub Pages variant of Jekyll. (If anyone who reads this knows the answer to that, let me know on Twitter or App.net, and I’ll update the post accordingly.)

The future

In addition to continuing to use BibTEX with BibDesk as a way of managing my citations in the short term, I’m thinking about other ways to improve this workflow. One possibility is integrating with Scholdoc as it matures, instead of pandoc, and maybe (hopefully, albeit unlikely) even contributing to it somewhat. I’m also open to using other citation library tools, though my early explorations with Mendeley and Zotero did not particularly impress me.

There are substantial advantages for the applications (and thus for most users) to maintaining the data in an application-specific format (e.g. an SQLite database) rather than on the file system—but the latter has the advantage of making it much easier to integrate with other tools. However, Zotero and Mendeley both natively export to BibTEX format, and Mendeley natively supports sync to a BibTEX library (Zotero can do the same, but via third-party plugins), so those remain viable options, which I may use for future projects.

I also want to look at making my library of resources available publicly, perhaps (a) as a standalone library associated with each project, so that anyone who wants to can download it along with the Markdown source to play with as an example and (b) as a general library covering my various reading and research interests, which will certainly be irrelevant to most people but might nonetheless provide some value to someone along the way. I’m a big fan of making this kind of data open wherever possible, because people come up with neat things to do with it that the original creators never expect. Not everything should be open—but lots of things should, and this might be among them.

Summary

I’m pretty happy with the current state of affairs, the aforementioned interest in other reference managers notwithstanding:

I can set up the citations once, in a tool designed to manage references, instead of multiple times in multiple places.
I can use Pandoc and a CSL to get the citations formatted correctly throughout a paper, including generating the bibliography automatically.
I can use the same tooling, integrated into my static site generator, to build a web version of the content—with no extra effort, once I configured it properly the first time.

Perhaps most importantly, this helps me meet one of my major goals for all my writing: to have a single canonical source for the content, which I will be able to access in the future regardless of what operating system I am using or what publishing systems come and go. Simple plain text files—Markdown—get me there. Now I’ve put good tools around that process, and I love it even more.

Coming up with names for footnotes in Markdown can be painful in general for long documents. If you try to name them manually, like I do for posts on my website, you will very quickly end up wasting time on the names. If you try to number them, they will end up out of order in a hurry. My own previous solution to this problem quickly became unwieldy for larger papers, and required a lot of hand-editing. Gladly, I no longer deal with that manually. Instead, I do all my drafting in Ulysses, where you just type (fn) and it creates a footnote automatically, and will move that footnote object around transparently as you edit, handling all the number-setting, etc. on its own.↩
The irony of site for software which boasts that it is “a high-quality typesetting system” and looks like this is not lost on me…↩
If you used the installers on Pandoc’s website, pandoc-citeproc comes with it. If you installed it via a package manager (e.g. by running brew install pandoc), it may not have, so you’ll need to install it manually yourself (e.g. brew install pandoc-citeproc).↩
All of the content, including the rendered footnotes and the bibliography, has sensible content types set on it: headers are headers, body text is body text, etc. You can then customize to match the specifications of your style guide. I have a Chicago/Turabian style set set up with the formatting rules to match.↩
Actually, it was even hairier than this, because I also had a --reference-docx path/to/template.docx specified. If you think it’s perhaps a bit too complex, well, I agree. I plan to turn that into a command line alias in pretty short order, because remembering it every time is just not going to happen.↩
Using the --reference-docx argument to Pandoc, you can hand it a document that already uses your desired style set, so you don’t have to go in and apply it manually.↩
I could have done that with Pandoc’s LATEX PDF tools, as well, but didn’t really feel like taking the time to tweak the LATEX template for it.↩
Probably someone does, but not me, and not most people!↩
If you’re using Pelican, you can take a look at my Pelican configuration file here to see the full configuration for using Pandoc this way.↩
Optimally, I’d really just prefer to be able to set all of these arguments at a per-file level—i.e., not use --filter pandoc cite-proc unless the file actually specifies a bibliography. And I could hack Pelican to do that; I’ve actually already messed around with other, semi-related bits regarding Pelican and Pandoc’s shared handling of YAML metadata. But I’d prefer to keep my installation as “vanilla” as possible to minimize the cost of setting things up again on a new machine or after a crash, etc.↩

HTML5 Location, , and SVG

2015-06-20T10:30:00-04:00

For quite some time, I have been frustrated by a bug in HolyBible.com: Firefox would not render SVGs using the <use xlink:xhref="#some-SVG-ID"></use> pattern. Today, I set aside my ongoing work on new user-facing functionality and dedicated what working time I had to hunting down the cause of this and fixing it at last.

I was surprised to find the culprit: the <base> tag. If you don’t know what the <base> tag is, you’re not alone. It is not used all that much in general, and I had never actually seen it on a site before starting on this project last year.

So what went wrong? How do these two things play together?

I am using (and reusing) SVG items throughout the HolyBible.com interface, taking advantage of the ability to define symbols and reference them with the <use> tag, like so:

<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" style="display: none">
  <symbol id="logo-shape" viewBox="0 0 256 256">
    <title>Logo</title>
    <desc>The HolyBible.com logo: sunrise breaking over an open book (the Bible).</desc>
    <path id="logo-light" d="M172.1 116.3l5.1-4.1-12.5-.5 32-26.3-41.4 18.4 11-20.1L148 96l12.2-37.5L138.8 91l.1-36.2-10.3 34.4L114 36.1l4.3 54.9-22.2-34.9 13 39.9-18.3-12.4 11 20.1-42.5-19.2 32.8 26.9-10.4.8 4.4 3.9c13.1-1.6 27.4-2.7 42.4-2.7 15.4 0 30.1 1.2 43.6 2.9z"/>
    <path id="logo-book" d="M199.9 219.9c-47.4-9.8-96.4-9.8-143.8 0-6-28.9-12-57.7-17.9-86.6 59.3-12.3 120.4-12.3 179.7 0-6 28.9-12 57.8-18 86.6z"/>
  </symbol>
</svg>

<!-- somewhere else on the page -->
<svg>
  <use xlink:href="#logo-shape"></use>
</svg>

Throughout all my early prototyping, this worked perfectly across all modern browsers. (For more, see CSS Tricks.) Now, when I started moving from the prototype phase into actually building the application in Angular last fall, I learned that you have to set the base URL for the application using the <base> tag to use the HTML5 Location API with Angular 1.x. If you want URL-based, rather than #-based navigation in an Angular app, you need this. Following the recommendation of whatever documentation and tutorials I found, I set it so:

<base href="/">

Again, this was the recommendation I saw in every bit of documentation and every tutorial, so I assumed it would have no problems. As it turns it, that’s not the case. (This is a recurring theme in my experience with Angular.) In Chrome, Safari, and IE9+, this works exactly as expected. In Firefox, however, it does not. The use of the <base> tag changes the behavior of #-based URLs on a page. Specifically, it makes it so that if you’re at a URL that isn’t the base route, anchor links don’t behave as expected. In order to make the <use> tag as expected, we would have to use the same URL as the base tag. Among other things, this would require making sure that any place we used the <use> tag, we would have to set that—not exactly a good idea, given that it would entail an awful lot of changes if the base URL were ever changed.

What if, instead, we did this?

<script>document.write('<base href="' + document.location.origin + '" />');</script>

This way, when the page renders, it writes the document location based on the current location. The URL history still behaves as expected with Angular, but the relative URLs for IDs behave as expected in Firefox again, while not breaking the behavior in any other browsers.

But… then you’ll navigate to another page, and Firefox will be back to not working.

The solution, it turns out, only came into being after I’d done the initial implementation, and I have no idea how much later it found its way into the Angular docs. However, even though it now exists in the docs, it’s by no means obvious why you should do it this way, and certainly no mention of SVG! This might not seem odd to you… but it should, given that the only reason that Angular introduced this API change was to account for exactly this issue.¹

As the Angular docs note, leaving out the <base> tag means all your URLs have to be absolute if you want to use HTML5 location and the $locationProvider. If you want to use SVGs with <use> and Firefox, though, that’s what you have to do (and therefore that’s what I’m doing).

Fun times, right?

The closest it gets is this reference:

Links that only contain a hash fragment (e.g. <a href="#target">) will only change $location.hash() and not modify the url otherwise. This is useful for scrolling to anchors on the same page without needing to know on which page the user currently is.

Even this, however, only hints at the root of the SVG issue.↩

How to Build a Single-Page App API Right

2015-06-09T22:16:00-04:00

When I was first working on HolyBible.com, I struggled for quite a while to wrap my head around the right way to structure its API—and in truth, I actually didn’t come up with what I would call the right solution. I came up with a working solution, and the site performs all right, most of the time. However, our goal as developers shouldn’t be “all right, most of the time.” It should be “really well, all the time.” A big part of what I did wrong came from the bad advice I found in reading up on the issue along the way. This is my shot at helping you, dear reader, avoid making the same mistake.

The challenge

When building a client-side application, we need to get the data for each view so that we can render it. In the case of HolyBible.com, that means everything from actual Bible text to study Bible notes, about pages, etc. The question is how to do this: we need to be able to load an actual page from our server, and we need a way to request data (rather than whole pages) from the server.

(More experienced developers already know where this is going: that last sentence there has the key to this whole thing. I know. But the internet doesn’t. I learned this the hard way.)

The mistake

Here’s the mistake I made: I built the Bible data API as (essentially) a single endpoint. When I went looking for advice on how to build this in Angular and Node/Express, every single tutorial or blog post I found outlined the same basic solution: routes for your data endpoints, and catch-all route that returns the basic frame page for everything else. So, for HolyBible.com, that would come out with route matchers for e.g. /data/gen.1.1, and for any other specific routes needed (for other views, static resources, etc.), with a default behavior of just dropping a static, basically empty template at the catchall * route. Then, once the application has loaded, it can inspect the URL and load the relevant data.

This works. It’s exactly what I did on HolyBible.com, in fact. But it’s slow.

Don’t get me wrong: the time until the initial page load is actually relatively quick (though I plan to improve it substantially over the next couple months). The real problem is that the initial page load doesn’t include any content.

I hate this. That’s why people are on the site: not to see my neat skills with JavaScript, just to read the Bible. And they have to wait, because once the page does load, Angular has to spin up the full application, see what content should have been loaded, and request it.

The solution

Don’t write one API. Write two. They should be structured nearly identically, but one of them will be a page API endpoint, and one will be a data API endpoint. In the context of HolyBible.com, here’s how that would play out.¹ One endpoint would be based purely on the standard URL, something like holybible.com/jhn.3.16. The other would be to retrieve a set of data associated with a given address, like holybible.com/data/jhn.3.16. This is only a little different from the approach suggested above, but that small difference matters—in fact, it matters a lot.

Instead of having the /jhn.3.16 route get handled by a catchall * route on the back end, it gets its own API endpoint, which looks for URLS of this shape and hands back a full page. That API endpoint is responsible to actually render the content of the page appropriately—in this case, with something like the whole chapter of John 3.² That gets handed back to the browser, so the very first thing the user sees is not a blank page while the JavaScript framework spins up and requests data, but rather the Bible text they asked for in the first place.

Meanwhile, the JavaScript framework can spin up, and load any required session data, etc. and start managing the UI like normal. Once we get to this point, the framework can go ahead and request a data payload from the /data/<reference> endpoint. So, for example, if there is a navigation control on the page (as on HolyBible.com and indeed most sites), clicking to navigate to Job 14 could, instead of requesting /job.14.4, fetch the data from the other endpoint by running an AJAX request to /data/job.14.4.

The backend thus supplies both a /<resource> and a /data/<resource> route. This might seem redundant, but we’ve just seen why it isn’t, Moreover, if you have any logic that needs to be in place—in our example here, a Bible reference parser, for example, to decide what content should be supplied—you can easily reuse it between the two routes. The differences is simply in the form of the data returned: is it a fully-rendered template, or just the data?

So what?

This approach has two big advantages over the catch-all approach that was frequently recommended in e.g. Angular SPA tutorials I read.

It’s progressive enhancement. If the JavaScript fails, or the user has it disabled, or it fails to load because it’s loaded asynchronously, the user still gets the page they asked for. Moreover, as long as the page content is build carefully (links built appropriately for other content, and so on), the entire application could continue to work even if the JavaScript never becomes available.
It’s performant. Loading the content this way will be much faster than the standard approach recommended for single-page apps. As noted above, it gets the content to the user immediately, then lets the JavaScript UI bits come into play. Since future page loads can take advantage of both caching and smaller data payloads, the whole thing can actually be faster than either a pure client-side or a pure server-side approach. That is, once the client-side application is running, it can just update its views with data delivered via AJAX, rather than reloading the whole page. But before that, the user doesn’t have to wait to see something useful until the JavaScript framework spins up.

It’s not often an approach gives you progressive enhancement and actually increases the performance of an application, but this one does. Better yet, you can apply this in just about any framework: it’s equally applicable to AngularJS with ExpressJS, Backbone with Rails, Ember with Django, Aurelia with Phoenix, or any other combination you come up with.

Note: this is not the actual API structure of HolyBible.com, or even particularly close to it. Remember, I learned everything I’m writing here by doing it wrong.↩
Or possibly a section which constitutes a semantic block of data. I have some thoughts on chunking Bible data semantically rather than by chapter and verse for this kind of thing. That’s another post for another day, though.↩

Corporate and Government Surveillance

2015-06-02T22:43:00-04:00

Brian Auten shared this speech by Sen. Sheldon Whitehouse on Facebook, and I wrote up what follows in response.

I broadly agree with the critique of the libertarian/TP angle on government as essentially an appendage to business. I am by no means hostile to the government in general or in principle, nor even to spying, nor even to warranted (double entendre intended) use of data for law enforcement. The idea that all government is bad is woefully incorrect; it is better to speak of abuses, either of government or of business or indeed of any sphere exceeding its right domain or acting inappropriately within its domain.
There is a profound and important difference between corporate data collection and federal government data collection: one of them, people accede to directly (though see below); the other they accede to (at best!) indirectly through elected representatives, with whom they may profoundly disagree and against whom they have no recourse (unlike the case of, say, Google or Facebook—one can simply stop dealing with them). Whatever information I have granted to a corporation, I have chosen to grant them, and I can stop doing so with future information at any time. I cannot do so with the NSA, FBI, etc.
That distinction may be relatively meaningless for most people in practice, given that the terms, means, and consequences of the data collection carried about by corporations are often obscure to the point of incomprehensibility.
As such, a serious reformation ought to occur in the realm of business and the way that people’s information is handled. Treating information about customers as the primary point of transactional value has significantly deleterious costs on any number of things.
For this reason, I consistently advocate for and (where possible) choose to use services which are supported by direct payment, rather than by advertising, and so on. This is not always possible, but where it is, we should consider taking that path.
Nonetheless, because of the government’s power of coercion—a power not held by corporations, though to be sure they can exercise significant force of a certain sort through legal machinery/chicanery—the collection of metadata by the government does pose a more potent and long-term threat to liberty than that by corporations.
As such, people are absolutely right to be more tolerant of corporate data collection than of federal data collection. That they ought to be less tolerant of corporate data collection by no means suggests that their hostility to unwarranted governmental data collection should be diminished: quite the contrary.
Therefore, while some of the criticism of the government’s data collection may well be driven by the sorts of corporate interests he suggests, and while much of the opposition from companies like Facebook and Google is indeed hypocritical, the criticism is still warranted. The NSA has clearly and repeatedly overstepped even the extremely wide bounds granted it by the Patriot Act, and the Patriot Act itself licensed behavior that should be horrifying to people concerned with the long-term effects of mass surveillance on governance.

Python Enums, ctypes.Structures, and DLL exports

2015-05-28T18:00:00-04:00

For one of my contracts right now, I’m writing a ctypes Python interface to existing C code. I got stuck and confused for quite a while on getting the interface to a given function to build correctly, and along the way had to try to understand the from_param class method. The official docs are… fine… but the examples provided don’t cover the most common/basic use case: defining a simple, non-ctypes data type as an argument to a DLL-exported function.

Let’s say you have a C function exported from a DLL; for convenience we’ll make it something rather silly but easy to understand:

/** my_exported.h */
#include "exports.h"

typedef enum {
    ZERO,
    ONE,
    TWO
} MyEnum;

MY_API int getAnEnumValue(MyEnum anEnum);

The implementation just gives back the integer value of the function:

int getAnEnumValue(MyEnum anEnum) {
    return (int)anEnum;
}

As I said, a very silly example. Note that you don’t technically need the (int) cast there; I’ve just put it in to be explicit about what we’re doing.

How would we use this from Python? Assuming we have a DLL named my_dll which exports the getAnEnumValue function, we’d load it up roughly like this:¹

import ctypes as c

my_dll = c.cdll.LoadLibrary('my_dll')

Then, we bind to the function like this:

get_an_enum_value = my_dll.getAnEnumValue

Now, when you do this, you usually also supply the argtypes and restype values for these functions. If you’re like me, you’d think, “Oh, an enum—a perfect opportunity to use the Enum type in Python 3.4+!” and then you’d do something like this:

import ctypes as c
from enum import IntEnum

class MyEnum(IntEnum):
    ZERO = 0
    ONE = 1
    TWO = 2

my_dll = c.cdll.LoadLibrary('my_dll')
get_an_enum_value = my_dll.getAnEnumValue
get_an_enum_value.argtypes = [MyEnum]
get_an_enum_value.restype = c.c_int

That seems sensible enough, but as it is, it won’t work: you’ll get an error:

TypeError: item 1 in _argtypes_ has no from_param method

This is because argtypes values have to be either existing ctypes types² or supply either:

a from_param classmethod, or
an _as_parameter_ attribute

You can use ctypes.Structure subclasses natively that way, because the Structure class supplies its from_param classmethod. The same is not true of our custom enum class, though. As the docs put it:

If you have defined your own classes which you pass to function calls, you have to implement a from_param() class method for them to be able to use them in the argtypes sequence. The from_param() class method receives the Python object passed to the function call, it should do a typecheck or whatever is needed to make sure this object is acceptable, and then return the object itself, its _as_parameter_ attribute, or whatever you want to pass as the C function argument in this case. Again, the result should be an integer, string, bytes, a ctypes instance, or an object with an _as_parameter_ attribute.

So, to make the enum type work, we need to add a from_param class method or an _as_parameter_ attribute to it. Thus, either of these options will work:

class MyEnum(IntEnum):
    ZERO = 0
    ONE = 1
    TWO = 2

    # Option 1: set the _as_parameter value at construction.
    def __init__(self, value):
        self._as_parameter = int(value)

    # Option 2: define the class method `from_param`.
    @classmethod
    def from_param(cls, obj):
        return int(obj)

In the constructor-based option, the value argument to the constructor is the value of the Enum instance. Since the value of anan IntEnum is always the same as the integer to whcih it is bound, we can just return int(value).

The from_param approach works a little differently, but with the same results. The obj argument to the from_param method is the object instance, in this case the enumerated value itself. Any Enum with an integer value can be directly cast to int (though it is possible for Enum instances to have other values, so be careful), and since we have an IntEnum here, we can again just return int(obj) directly.

Now, let’s say we want to apply this pattern to more than a single IntEnum class, because our C code defines more than one enumeration. Extracting it to be common functionality is simple enough: just create a class that implements the class method, and inherit from it.

class CtypesEnum(IntEnum):
    """A ctypes-compatible IntEnum superclass."""
    @classmethod
    def from_param(cls, obj):
        return int(obj)


class MyEnum(CtypesEnum):
    ZERO = 0
    ONE = 1
    TWO = 2

Our final (working!) Python code, then, would be:

# Import the standard library dependencies
import ctypes as c
from enum import IntEnum


# Define the types we need.
class CtypesEnum(IntEnum):
    """A ctypes-compatible IntEnum superclass."""
    @classmethod
    def from_param(cls, obj):
        return int(obj)


class MyEnum(CtypesEnum):
    ZERO = 0
    ONE = 1
    TWO = 2


# Load the DLL and configure the function call.
my_dll = c.cdll.LoadLibrary('my_dll')
get_an_enum_value = my_dll.getAnEnumValue
get_an_enum_value.argtypes = [MyEnum]
get_an_enum_value.restype = c.c_int

# Demonstrate that it works.
print(get_an_enum_value(MyEnum.TWO))

The output will be 2, just as you’d expect!

An important note: The type definition we’ve provided here will work for argtypes or restype assignments, but not as one of the members of a custom ctypes.Structure type’s _fields_ value. (Discussing how you’d go about doing that is beyond the scope of this post; the most direct approach is just to use a ctypes.c_int and note that it is intended to be used with a given IntEnum/CtypesEnum type.)

Thanks to @oluseyi for being my rubber ducky while I was working this out earlier this week!

I’m leaving out the part where we build the DLL, and also the part where we locate the DLL, and only using the Windows convention. If you’re on a *nix system, you should use 'my_dll.so' instead, and in any case you need to make sure the DLL is available in the search path.↩
I love the redundancy of “ctypes types,” don’t you?↩

Open Source is Neat

2015-05-17T22:52:00-04:00

I confess: my first response to seeing this page was a flash of anger: Hey, he didn’t just learn from my site configuration, he actually stole my site design_!_ And then I remembered: I open-sourced the design precisely so people could do that. This was just the first time I’ve ever actually had someone reuse something I did and shared like this. It was a strange (but ultimately wonderful) feeling. I hope to have it again many more times.

In any case, I rather like the tweaks Andrew Comenga made to my design to make it his own; go take a look!

A Modern Python Development Toolchain

2015-05-16T22:40:00-04:00

Most of my development time these days—and especially the majority of my happiest time!—is spent working in Python. As such, I’ve experimented off and on over the last few years with the best workflow, and have settled down with a set of tools that is very effective and efficient for me. I’m sure I’m not the only one who’s had to wrestle with some of the issues particular to this toolchain, and I know that information like this can be valuable especially for people just starting off, so I thought I would document it all in one place.¹

Note: when talking about a given program, I will italicize it, like brew or git or python. When talking about things to type, I will make them a code block like git clone <a repository>. For any extended samples, I will make them full-on code blocks:

import re

def a_neat_function():
    my_string = "Isn't it cool?"
    if re.match(r"i\w+", my_string, flags=re.I):
        print(my_string)

The main tools I use are: a good text editor (I like all of Sublime Text, Atom, TextMate, and Chocolat; each has its own strengths and weaknesses) or sometimes a full IDE, version control software (I appreciate and use both Git and Mercurial), and three dedicated tools to which the rest of this post is devoted: pyenv, pip, and virtual environments.

Everyone is going to have their own preferences for version control tools and an editor; but the recommendations I make regarding Python installations, package management, and workspaces/virtual environments should be fairly standard for anyone doing Python development on a Unix-like system in 2015.

Python Proper

First up: Python itself. OS X ships with a built-in copy of Python 2; in the latest version of Yosemite, it’s running Python 2.7.6. The latest version of Python 2 is 2.7.9, so that isn’t terribly far behind—but it is still behind. Moreover, OS X does not ship with Python 3, and since I do all of my development in Python 3² I need to install it.

Homebrew

For a long time, I managed all my Python installations with homebrew. If you’re not familiar with it, homebrew is a package manager that lets you installed tools on the command line, similar to what you get from aptitude or yum on Ubuntu or Fedora respectively.³ If you’re not using homebrew yet, I highly recommend it for installing command-line tools. (If you’re not using command-line tools yet, then the rest of this post will either bore you to death, or prove extremely enlightening!) If you haven’t started yet, now’s a good time: go install it!.

While homebrew is great for installing and managing packages in general, I can’t say this loud enough: don’t manage Python with homebrew. It’s finicky, and really isn’t meant for all the things you have to do to manage more than one version of Python at a time.⁴ (There’s a reason there’s a whole troubleshooting section devoted to it.) If you think it’s crazy that I might want more than one copy of Python installed a time, well… let’s just say I suspect you’ll change your mind after doing a bit more development. (At the most basic, most people will end up wanting both Python 2 and 3 installed, and will want to upgrade them as bug fixes and the like come out.)

pyenv

Instead of installing via homebrew, use it to install pyenv, and use that to manage your installations. pyenv is a dedicated tool for managing your “Python environment,” and it excels at that. If you were on a Mac with homebrew installed, your setup process to add the latest version of Python might look something like this:

$ brew install pyenv
$ echo 'eval "$(pyenv init -)"' >> ~.profile
$ source ~/.profile
$ pyenv install 3.4.3

Line by line, that (a) installs pyenv, (b) adds a hook to your shell profile,⁵ (c) updates your current session using the updated profile, and (d) installs the latest version of Python (as of the time I’m writing this). Now you have a full version of Python 3.4.3 alongside the system install of Python 2.7.6. If you wanted to install 2.7.9, or 2.2.3, or the development version of PyPy3, you could easily do that as well.

In addition, pyenv lets you specify which version to use globally (pyenv global <name>) and which version to use in a given directory structure (pyenv local <name>). So if you prefer to use Python 3 in general, but need to use Python 2 on one project, you can just navigate to the root of that project and set it:

$ pyenv global 3.4.3
$ cd path/to/my/project
$ pyenv local 2.7.9

This will create a simple plain text file, .python-version, whose contents will be just 2.7.9—but for everything under path/to/my/project, typing python will launch Python 2.7.9, while typing it outside that folder will launch Python 3.4.3. (If you want, you can just create the .python-version file yourself manually and give it the name of a version. There’s nothing special about it all; it’s just the place pyenv looks to know which Python version to use.)

Managing Python Packages

There are four basic approaches to managing Python packages:

installing them manually
using a system-level package manager like homebrew, yum, or aptitude
using easy_install
using pip

The vast majority of the time, the right choice is using pip. Over the last few years, pip has become the default install tool for Python packages and it now ships natively with it on every platform. Suffice it to say: if you need to install a package, do not install it not with homebrew (or aptitude or yum). Install it with pip. It integrates with Python better, it always has access both to the latest versions of Python packages (including those only available in e.g. development repositories on GitHub or Bitbucket or wherever else) and to all previously released versions, and it’s the community’s main tool for the job.

That said, occasionally it makes sense to install packages manually by downloading them and running python setup.py install or to use a system-level package manager. On the other hand, given pip’s ability to do everything easy_install does, and its ability to do quite a few more things as well, there really isn’t a time to use easy_install. Using the language-supplied tools keeps everything playing nicely together. Perhaps just as importantly, it is the only way to make sure everything behaves the way it should when you start using…

Virtual Environments

When working with a variety of different clients, or simply on different projects, it is common not only to end up with different versions of Python but also with different sets of packages or—tricker still!—different versions of the same package required for different projects. Virtual environments provide a solution: they reuse the main Python executable (by creating links on the file system to it), but create isolated “workspaces” for the various packages you might install.

That way, in one workspace, you might have version 1.2 of a package installed, and in another you might have version 3.3 installed—because those are the required dependencies for something else you’re doing. This isn’t a hypothetical situation. For quite a while with one of my clients, we had pinned a particular version of the Python documentation package we use because it broke our use case after an update—but I still wanted to have the latest version of that tool in my other projects. Setting up virtual environments neatly solves that problem.

venv and virtualenv

If you have Python 3.3 or later, you have a built-in tool for this called pyvenv; if you have Python 3.4 or later, it supports pip right out of the gate so you don’t have to install it yourself. If you’re on older versions, you can install virtualenv (pip install virtualenv) and get the same basic tooling: pyvenv was inspired by virtualenv. Then you can create virtual environments with the pyvenv or virtualenv commands, and use those to isolate different setups from each other. If you haven’t started using virtual environments yet, start now!

pyenv with virtualenv

I know, the similarity of names for pyenv and pyvenv is unfortunate. If it helps, you can call the latter as venv rather than pyvenv. But, more importantly, one of the areas pyenv is much better than homebrew is its support for managing virtual environments. Install pyenv-virtualenv:

$ brew install pyenv-virtualenv
$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.profile

Now you’re off to the races: you’ll never have to type pyvenv <path to a virtual environment>, because instead you can just type pyenv virtualenv <version> <name> and pyenv will take care of setting it up for you. Even better: all the nice tricks I listed above about setting directory-specific and global preferences for which Python version to use work equally well with virtual environments managed via pyenv. In other words, you can do something like this:

$ pyenv install 2.7.9
$ pyenv install 3.4.3
$ pyenv global 3.4.3
$ pyenv virtualenv 2.7.9 my-virtual-environment
$ cd path/to/my/project
$ pyenv local my-virtual-environment

The .python-version file will contain my-virtual-environment. The Python version will be 2.7.9. The environment will be isolated, just as if you had run pyvenv to set up a virtual environment. Everything works together beautifully! Moreover, you can easily reuse virtual environments this way, because you can set the local value in more than one place. For example, I use the same virtual environment for this site and Winning Slowly, because they have slightly different site configurations but all the same Python dependencies. Creating it was simple:

$ pyenv install 3.4.3
$ pyenv virtualenv 3.4.3 pelican
$ cd ~/Sites/chriskrycho.com
$ pyenv local pelican
$ cd ~/Sites/winningslowly.org
$ pyenv local pelican

I named the virtual environment after the tool I use to generate the sites, and reused it in both sites. Both now have a .python-version file that reads pelican. Now, anytime I’m working anywhere under ~/Sites/chriskrycho.com or ~/Sites/winningslowly.org, I have the same tooling in place.

Summary

The combination of pip, pyenv and virtual environments makes for a very simple, straightforward process to manage Python environments these days:

Install Python versions with pyenv.
Install Python packages with pip.
Set up virtual environments with pyenv-virtualenv.

If you stick to those basic rules, Python itself shouldn’t give you any trouble at all.

All the usual caveats apply, of course: this may or may not work well for you; it’s just what works for me, and I make no claim or warranty on the tools below—they’re working well for me, but I don’t maintain them, so if they break, please tell the people who maintain them! Also, because I do nearly all my development on a Mac (I test on Windows, but that’s it), the following is necessarily fairly specific to OS X. You can readily adapt most of it to Linux, though, or even to a Cygwin install on Windows—I do just that when I have cause. But my main tool is a Mac, so that’s what I’ve specialized for.↩
Lucky me, I know!↩
Yes, I know that those are wrappers around Debian and Arch, and I know about apt-get and rpm. No, that information isn’t especially relevant for the rest of this post.↩
For example, if you upgrade your Python installation using homebrew and then cleanup the old version (e.g., by running the typical brew update && brew upgrade && brew cleanup sequence)—say, from 3.4.2 to 3.4.3—and you have virtual environments which depended on 3.4.2… well, you’re in a bad spot now. A very bad spot. Have fun getting back to a working state!↩
You can of course drop it directly in .zshrc or .bash_profile or wherever else. My setup puts all common handling in .profile and runs source .profile as the first action in any other shell configurations.↩

Tolle Lege!

2015-05-01T10:30:00-04:00

I was delighted to be able to give a talk at BibleTech this year. I spoke for almost exactly 40 minutes on the subject of digital typography, with a focus on some of the nitty-gritty details that make texts readable… or not. Here is the screen capture and audio from the talk!

You can also access the slides whenever you like (though note that they were designed to be complements to the talk, not the content of the talk, and as such they elide a great deal of the content).

Lessons Learned

2015-04-12T13:49:00-04:00

Since mid July 2014, I have been working on a complete redesign and re-build of HolyBible.com. The good folks at Puritan Reformed Theological Seminary who own the site wanted to replace its previous content with a Bible reading tool. While there’s still a lot to wrap up, the project is nearing its conclusion, and I thought I’d note a few things I’ve learned (in some cases, learned again) along the way. I want to say up front, lest these be taken the wrong way: I’m extremely proud of the work I’ve done, and the application I’ve delivered does work to the specifications I was hired to meet. More than that, it does it well. But, of course, it could do it better. The following thoughts are therefore not, “How I failed” but rather “How I will do this even better next time around.”

Single page apps are great, but not always the right choice. I made the decision, based on my expectations and understandings of what I would need, to develop the site as a single-page web application. This was a mistake. Not the worst mistake ever: it has its upsides, including performance once the app spins up, but for the kind of content I have here, I would take a different tack today. Better in this case to deliver static content and update it dynamically as appropriate than to try to load all the content dynamically every time.

At a technical level, that would probably mean supplementing standard HTML with Backbone instead of developing it as a single-page app in Angular. For the backend, while I did it in Node.js and that would work fine, I’d probably do a straight Django app (especially with a few of the goals I learned about after the project was well along in development).
Progressive enhancement or graceful degradation are hard in web applications, but they still matter. In the past, I’ve always taken a hard line on making sure things either degrade gracefully or are simply enhanced by JavaScript content. In the architecture decisions I made for this app, I failed to take that into account (largely because I thought it would just need to work as a web app, but see above). I regret that enormously at this point; it would be much better in this particular case to have content available even if the additional functionality doesn’t work. Even if you are doing something where you are building an app, finding ways to make it work on poor connections, older browsers, etc. matters. I’m still thinking a lot about the best way to do this in the future.
More popular doesn’t mean better. Angular has a ton of traction and uptake, and that was deceptive early on. I won’t so easily be fooled in the future. Angular is so very popular in part because Google can put serious money behind its development—and its marketing. But it’s not the best for many applications; if you’re not in the business of developing your own custom framework, it’s not even close to the best. Use Ember or Knockout or any number of other full-stack frameworks rather than a meta-framework.

How to avoid making that mistake? Well, for my part since then, I’ve learned to look not just as the quantity of material in a given community, but its quality. For example, Ember has incredible documentation (far better than Angular’s), and they also have a much clearer vision and a more dependable approach to development (strict semantic versioning, etc.). Had I taken the time to read both sets of docs more carefully and think through the consequences of their designs more thoroughly, I could have recognized this before starting. Next time, I will do just that.

I will also look at the way the community behaves. The Ember community is far friendlier for newcomers from what I’ve seen than the Angular community—no slam meant on the Angular crowd, but the Ember folks are just doing that really well. That matters, too. (I can’t speak for other communities, of course; these are just the groups I’ve watched the most.)

All in all, Ember would have been the better fit between these two (even though, as noted above, it also wouldn’t have been the best fit).
Unit tests really are the best. I did a vast majority of this project with unit tests—the first time I’ve ever been able to do that for a whole project. In other projects, I’ve been able to do it for parts, but never this much. It saved my bacon a lot. Where I got in a hurry and felt like I didn’t have time to write the tests, I (inevitably and predictably!) ended up spending a lot of time chasing down hard-to-isolate bugs—time I could have avoided by writing well-tested (and therefore better-factored) code in the first place. Lesson learned very thoroughly. Server- and client-side unit tests are really good. They’re also sometimes hard; getting mocks set up correctly for dealing with databases, etc. can take a while. That difficulty pays for itself, though.
Unit tests really don’t replace API documentation. I have seen people advocate test-driven-development as a way of obviating the need to do major documentation of an API. This is, in a word, ridiculous. Having to read unit tests if you want to remember how you structured an API call is a pain in the neck. Don’t believe it. Design your API and document it, then do test-driven development against that contract.
Sometimes ‘good enough’ is enough. There is always more to be done, and inevitably you can see a thousand things that could be improved. But ‘good’ shipping code is far more valuable than ‘perfect’ code that never ships. You should never ship bad code, but sometimes you do have to recognize ‘good enough’ and push it out the door.
Full-stack development is fun, but it’s also really hard. I wrote every scrap of code in HolyBible.com proper (though of course it relies on a lot of third-party code). It was very, very difficult to manage that all by myself; it’s a lot to hold in one’s head. (One of the reasons I chose Node was because keeping my implementation and testing all in one language helped reduce that load somewhat.) Would I do it again? Sure. But very much chastened about the difficulties involved. It has been enormously rewarding, and I like being a full-stack developer. But it’s a lot of work, and now I know more clearly just how much.

I could say a great deal more about the technical side of things especially, but my biggest takeaway here is that a lot of the hardest and most important work in developing software has nothing to do with the code itself. Architecture and approach shape far more than the implementation details (even if those details still matter an awful lot). And popularity is not at all the same as either quality or (especially) suitability for a given task. In the future, I will be better equipped for the necessary kinds of evaluation, and will hopefully make still better decisions accordingly.

The NSA wants tech companies to give it 'front door' access to encrypted data

2015-04-12T13:16:00-04:00

The Verge:

“I don’t want a back door,” Rogers said. “I want a front door. And I want the front door to have multiple locks. Big locks….”

Rogers suggests the adoption of “front door” access will allow for essential security measures while keeping data safe from hackers or an outside attack. But opponents of the idea note that even broken into pieces, a master digital key creates security flaws. “There’s no way to do this where you don’t have unintentional vulnerabilities,” Donna Dodson, chief cybersecurity adviser at the Commerce Department’s National Institute of Standards and Technologies, told the Post.

That last bit is absolutely true. The government basically wants to make sure it can spy on anyone, any time it wants. That’s a bad, bad plan.

Unsurprisingly, In Flux

2015-04-08T16:05:00-04:00

This started as a series of posts on App.net. I resolved a while ago that if I was tempted to do that, I should just write a blog post instead. I failed at that resolution, but at a friend’s suggestion, am adapting it into a blog post anyway. You can see the posts that prompted it here and here.

The state of JavaScript frameworks today is a scale, really, from not-at-all-monolithic to totally-monolithic, in roughly this order: Backbone – React & Angular – Ember – Meteor.
Backbone and related library Underscore are really collections of common JS tools and patterns you can use to write apps, but they’re not frameworks, per se. You’ll write all your own boilerplate there.
React and Angular supply much more of the functionality, but Angular is a “meta-framework” that aims to do some boilerplate but let you construct your own custom app framework.
Angular is very powerful, but it’s kind of like Git: wires are exposed; you have to understand a lot about the internals to get it to do what you want. Its routing functionality is pretty limited out of the box, too—so much so that there’s a near-standard third-party router.
React, as I understand it, supplies a paradigm and associated tools oriented primarily at view state management, though with capabilities via extensions for routing, etc. These tools are extremely powerful for performance in particular. It’s not a full framework, and the docs expressly note that you can just use React for the view layer with other tools if you want.
In any case, Angular and React do different things from each other, but both do substantially more than Backbone.
Ember is a full framework, strongly emphasizing shared conventions (with a lot of common developers from Rails). It’s perhaps less adaptable than React or Angular, but is much more full-featured; you have very little boilerplate to do.
Meteor is like Ember, but does server-side Node as well as client-side stuff, with the goal being to minimize code duplication, sharing assets as much as possible.
Of all of those, Ember has easily (easily!) the best-explained roadmap, most articulate leadership, and best development path. They are also aggressively adopting the best features of other frameworks wherever it makes sense.
Angular is currently in flux, as Google has announced Angular 2.0 will be basically a completely different framework; there will be no direct migration path for Angular 1.x apps to Angular 2.0+. Total rewrite required.
Ember uses a steady 6-week release schedule with very careful regression testing and semantic versioning, with clear deprecation notices and upgrade paths, and is therefore both rapidly iterating and relatively stable for use.
If you just need a set of tools for enhance functionality on otherwise relatively static pages, Backbone+Underscore is a great combo. If you already have a bunch of things in place but want a dedicated view layer, React is good.¹
If you’re writing a new, full-on web application (SPA, or organized in whatever other way), I think Ember is the very clear winner at this point. I have good confidence in their leadership and they’re firing on all cylinders.

Regarding Angular, @mikehoss posted:

For the record they are doing that to make it more mobile-friendly. The Ang1 has abysmal performance on mobile. Besides a time machine, this maybe the best option. And Miško is a bit of a jerk.

I can’t speak to his comment about Miško (Miško Hevery, one of the leads on AngularJS), but I agree about Angular itself: the rewrite needs to happen. Angular 1.x is a mess—as are its docs. It’s just not a good time to be using 1.x for any new projects.

I’ll add to these points that I’ve used Angular for the last 9 months on HolyBible.com development. As I noted: the documentation is pretty rough, and in a lot of cases you really do have to understand what the framework is doing and how before you can get it to do the things you want. This is, in one sense, exactly the opposite of what I’m looking for in a framework—but it makes sense given Angular’s goal of being a meta-framework.

Rather like Git, though, which was originally going to be infrastructure for version control systems which would have their own interface, but eventually just had a “good enough” interface that we’re all now stuck with, Angular is being used as a framework, not just as a meta-framework, and it’s unsurprisingly not great for that.

Take this for what it’s worth: not the final word (by a long stretch) on JavaScript frameworks, but rather the perspective of one guy who notably hasn’t used all of the frameworks, but has spent some time looking at them. Moreover, I haven’t particularly edited this; it’s more a summary in the kind of short-form posts that I originally created than a detailed analysis. The only things I’ve done are expand some of the notes on Angular and React, and add the footnote on React.

I really don’t know a ton about React, but I do think a lot of what I do know about it is cool from a programming perspective. From a designer perspective, however, it’s a bit of a pain: React’s “JSX” domain-specific language is much less friendly to developers than standard HTML, and therefore than either Ember or Angular, both of which implement their templating via HTML templating languages. There’s a substantil tradeoff there: React’s model is interesting not only academically but in practice because of the performance results it produces. It’s worth note, though, that others have recognized this and are adopting it to varying degrees; notably, Ember is incorporating the idea of minimizing changes to the DOM by keeping track of state and updating only differences, rather than refreshing the whole tree, in the new rendering engine (HTMLBars) they’re rolling out over the past several and future several releases.↩

The New Macbook

2015-03-13T08:00:00-04:00

I have seen and heard lots of discussion of the new Macbook this week, and have been thinking about its appeal and Apple’s strategy a bit along the way. At first I was extremely skeptical of the only-one-port approach, but the more I’ve thought about it, the more sense it makes to me. Why? Market segmentation.

This is a MacBook, not a MacBook Pro. I need more ports than this. But Jaimie? I don’t remember the last time I saw her plug anything into the machine besides its power cord. This is a MacBook for ordinary users, not a machine for power-users. Now, I still think that the loss of MagSafe is a bit sad; it has saved us more than once (especially with young children in the house). But in terms of the needs of ordinary users, a single port that can double as video out or USB input really is perfect.

In the meantime, it lets Apple cleanly differentiate between its MacBook and MacBook Pro lines. If you need the ports for expandability—because you’re a power user—you get a Pro. If you don’t, you get the MacBook. The tradeoffs with CPU make sense here, too: a computer that performs about like a 2012 MacBook Air would not be my favorite for development work. But for the writing work that Jaimie does? Again, the performance levels there are perfectly reasonable. It’ll do everything she needs, and do it well. Throw in the retina screen, and it’ll be really nice for her purposes.

In fact, I fully expect that we’ll end up getting her a 2nd or 3rd generation machine when we need to replace her current (a 2010 white MacBook) sometime in 2016–17.

So: better done than I initially thought, Apple.

The Tablet “Productivity” Problem

2015-02-25T21:35:00-05:00

I’m thinking this one through out loud. I rather hope that I can take these nascent thoughts and turn them into a more fully-fledged essay over the course of this year, so if you have thoughts, I’d love to hear them. Hit me up on Twitter, ADN, or via [email protected]">email. In the meantime… consider this a rough draft of a larger idea I’m working out.

I saw a post by internet acquaintance Jason Irwin (@matigo on ADN) yesterday about how he doesn’t find tablets especially compelling. There were quite a few things he said in the piece that did not resonate with me (and even a few suspicions I think are out and out incorrect), but generally on technology things like this I simply say: to each his own. So what follows is not so much a response to Jason’s post as some thoughts inspired by it.

Jason hit on a meme that’s been extremely common about tablets in general and iPads in particular: that you cannot do real work on them, only “consumption”. What is meant, nearly always, in such discussions, is that it is harder to write, develop software, and other keyboard-intensive activities using an iPad than a traditional laptop or desktop form factor. This is certainly true of those activities. Even of a few other activities Jason mentions, iPads do very well.¹ But there is another, more important issue here.

We (quite readonably) tend to define productivity poorly in terms of output. In that sense, there are many categories for which the iPad is not as capable as a laptop. It is true, for example, that I do not do a lot of writing or software development on my iPad (a retina Mini)—I’ll start drafts of blog posts (part of this was dictated on my iPhone!), and occasionally log into and do administrative work on a server via SSH using an iPad client. That doesn’t mean it isn’t a valuable device for me, though. It simply means that “valuable” and “productive” are not synonyms.

Less helpfully, however, we also tends to define “value” in terms of “productivity”. People say that iPads are not valuable to them because they do not specifically allow them to be productive in the sense outlined above… but then, there are a great many valuable things that are not producing content. I use my iPad daily for a wide array of things, and find it enormously preferable to a laptop for nearly all of them. True, many of them are “consumptive”—but since when did that become a bad thing?

I recognize that the answer may seem obvious against the backdrop of a consumerist culture against which many an anti-consumerism critique has rightly been levied. But think about what we mean by “consumption” in this case. Nearly every day I use my iPad both for reading and for displaying (and for learning) music. To be sure, I also watch the occasional YouTube video, interact on Twitter and App.net, and so on. But the vast majority of what I do with an iPad is best summed up as learning.² Whether it is reading through a few carefully selected RSS feeds in Unread, reading the news in Circa,³ working through reading for school in iBooks,⁴ perusing the EmberJS docs in preparation for a major project I’ll be starting with the tool in a few months, or reading the Bible every morning, I do a lot of reading on my iPad. Add in the fact that I use it for music as I practice piano, and I get an awful lot of mileage out of it every day.

Now, none of this negates Jason’s post in particular. If he doesn’t get that kind of traction out of an iPad, that’s no skin off my back. But I do think that the criticism of devices which are primarily “consumptive”—perhaps implied in Jason’s post; certainly stated outright in many other responses to the iPad—is misplaced. Whether simply for entertainment (joy in the arts is good!) or in reading (joy in the arts or in self-betterment is good!) or in the myriad other ways that people put the iPad⁵ to use that are not making something new, there is value in the kinds of consumption done with it.

Are there valid critiques to be offered of tablets, including that certain kinds of consumptive habits are problematic? Of course. But reducing things to their productive utility is ethically flawed, and reducing human pursuits to their productive output even more so. It is just fine if @matigo isn’t the sort of guy who loves an iPad. It is not fine if tech pundits want to slam the iPad and other tablets because they have a misanthropic view of human flourishing—and make no mistake, the utilitarian calculus so often levied against tablets is just that. People are more than what they make; their time is valuable even (and sometimes especially) when not producing anything tangible at all.

Notably, his point about keyboards that differ for different applications has been addressed quite thoroughly in that market! Most music apps ship with music-oriented interfaces, not traditional QUERTY-style keyboards.↩
Yes, in theory I could do that on another, less expensive device—but I had a Nexus 7 and nothing I have seen about Android tablets since then convinces me the Android tablet ecosystem has meaningfully improved in the last couple years. The experience factor in using things really does matter to me, and iOS gives me an enormously better experience in every category, even with its foibles and flaws, and nowhere more so than in the massively better app ecosystem.↩
An app my friend Stephen Carradini and I like so much that we did a whole episode of Winning Slowly on it!↩
I like ePUB way better than Kindle’s proprietary, and haven’t gotten around to finding a replacement for Readmill yet.↩
And yes, with plenty of other tablets, too! If you’re a Microsoft Surface person, that’s splendid as well.↩

Facebook's "Security" Requirements

2015-02-21T12:35:00-05:00

I went to set up 2-step login (AKA 2-factor authentication, or what Facebook calls “Login Approvals”) on Facebook yesterday morning, and was greeted with this lovely message when I clicked “enable”:

Your current Firefox settings might make it hard to use Login Approvals. It’s probably because:

You sometimes clear your cookies.

Your browser is set to automatically clear cookies whenever it closes.

You use your browser’s “private browsing” or “incognito” mode.

You’re using a new browser.

It may take a few days after fixing these issues before you will be able to enable Login Approvals. You also may need to log out and then log in again after fixing these settings for the changes to take effect.

Visit the Help Center for step-by-step directions on how to fix these settings.

I use Firefox for the social media access I do online—and because I don’t like being tracked, I tell Firefox not to remember history and to delete cookies as soon as I close the browser, and I run μBlock ¹ and Disconnect.²

When you attempt to enable 2-step login, Facebook checks your security policy… and will not let you turn it on if your settings are like mine. They supply the message above, with no option to proceed anyway. Of course, there is no technical issue with using 2-step login with a browser configured this way. I use it for GitHub, Google, my domain registrar, and every other service with 2-step login.

Facebook probably has two motives here. The better one is user experience: it would be frustrating if you are a non-tech-savvy user who doesn’t understand the consequences of setting this given the browser settings I have. But of course, if they were primarily just concerned with that, they could give the warning and then let users say, “Go ahead; I know what I’m getting into.” The second, less obvious but almost certainly more important motive from Facebook’s point of view, is to discourage people from using a browser the way I do. They want to be able to monetize my Facebook use better, and this means not just my time on Facebook, but my time all over the web. Facebook wants to know what I’m looking at any time I’m surfing anywhere so that they can tailor their ads to me.

I’m not interested in being tracked that way.

Apparently, Facebook isn’t interested in letting people have actual, modern security unless they’re willing to be tracked that way.

We have a problem here.

As it turns out, of course, people like me aren’t particularly valuable customers to Facebook anyway, so they probably don’t mind the fact that they’re losing more and more of our time. But losing that time they are. My use of Facebook is diminishing at an ever-increasing rate, for countless little reasons like this, where Facebook’s ad-driven motivations push them to treat me poorly. Too bad for them.

If anyone tells you that blocking ads is “stealing”, they’re talking up nonsense. The Internet is built in such a way that if nothing else you can always just request the plain text version of a website, and that’s extremely important for many reasons, including accessibility. I choose to leave ads on for any number of sites I want to support, but at the end of the day it’s every publisher’s choice how theyw ant to make money. If a newspaper supports itself with ads and coupons, I have every right to throw them in the trash without a glance; the same is true online.↩
Yes, I know this isn’t foolproof and I’m still being tracked. It’s impossible not to be tracked to some degree or another. What I am doing here is decreasing the degree to which companies can track me.↩

Growing Up Together

2014-11-15T00:30:00-05:00

A few years ago, you might have caught me in a grumpy moment grousing about JavaScript. I distinctly did not like writing it. Every time I sat down to deal with it, I found myself in a tangled mess of plain JavaScript, jQuery, and DOM manipulations that inevitably left me tearing my hair out.¹ I found it difficult to write in the first place, and even harder to maintain in the long run. I could not come up with good ways to organize it, especially because so much of what I was doing was so thoroughly ad hoc in nature. Cobble this together over here; scrounge together those things over there; hope nothing collides in the middle.

In the last four months, I have written several thousand lines of JavaScript, and I have loved it.

For my latest major project, relaunching HolyBible.com, I wrote the front end in AngularJS and the back end as an Express app (the most popular NodeJS web framework). I’ve written gobs of tests in Jasmine (using jasmine-node for server-side tests) and drawn on tons of other open-source packages.

And I have loved it.

A small example: a moment ago, looking up the link for Jasmine, I noted that the latest version released today. My response was, “Ooh—cool!”²

What changed? Well, mostly I changed, but also JavaScript changed a bit. We both grew up over the last four years. On the JavaScript side of things, a lot of good design patterns and tools have come into play in that span. I’m sure there were plenty of good, disciplined web developers writing clear, careful, well-organized client-side JavaScript four years go. But in the interval, that kind of JavaScript got a lot more prominent, in part because it has had help from the rapid rise of server-side JavaScript in the form of Node.js and its flourishing ecosystem of components and tools. Build tools like Browserify and development tools like LiveReload and Codekit have combined with best practices learned from those long years of jQuery/DOM-manipulation hell so that these days, good JavaScript is a lot like good programming in any other language: highly modular, carefully designed, and well-organized.

In the same period of time, I have matured enormously as a developer (just enough to see how far I still have to go, of course). At the point where I most hated JavaScript, I also really struggled to see the utility of callbacks. Frankly, it took me the better part of a month just to get my head around it—most of the tutorials out there just assumed you understood them already, and, well: I didn’t. Functions as first-class members of a language was new to me at that point. Fast-forward through several years of full-time Python development, lots of time spent reading about software development and some harder computer science concepts, and my perspective on JavaScript has shifted more than a little. Closures are beautiful, wonderful things now. Functions as arguments to other functions are delightful and extremely expressive. Prototypal inheritance—trip me up though it sometimes still does—is a fascinating variation on the idea of inheritance and one that I think I like rather better than classical inheritance.³

There are still things I don’t love about JavaScript. Its syntax owes far too much to the C family of languages to make me happy; I quite like the way that CoffeeScript borrows from Python (white-space-delimited blocks, use of equality words like is and boolean rules like and rather than === and && respectively, etc.). And I am looking forward to a number of features coming in the next version of JavaScript—especially generators and the const and let keywords, which will allow for much saner patterns.

But all of that is simply to say that I am now starting to know JavaScript enough to know that its real issues aren’t the surface-level differences from the other languages with which I’m familiar. They’re not even the warts I noted here. They’re things like the mix of classical and prototypal inheritance in the way the language keywords and object instantiation work. But I don’t mind those. Every language has tradeoffs. Python’s support for lambdas is pretty minimal, despite the utility of anonymous functions, for example. But I like the tradeoffs JavaScript makes.⁴

In other words, I discovered the same thing so many other people have over the last few years: JavaScript isn’t just a good choice for utilitarian reasons. Beneath that messy exterior is a gem of a language. I’m having a lot of fun with it.

Thus the early balding starting by my temples.↩
My wife’s bemused response: “Is that another language?” Take that as you will.↩
The couple weeks I got to spend playing with Io certainly helped! Io’s prototypal inheritance is semantically “purer” than JavaScript’s, which is quite an improvement in my view. JavaScript’s new keyword and the pseudo-classical object pattern it brings along can go rot in a bog.↩
Truth be told, I like them even better from the perspective of CoffeeScript, which hides a lot of the rough edges of JavaScript and, as noted above, brings in quite a few things I like from Python. For my part, I intend to write as much CoffeeScript as possible going forward.↩

A Ridiculous Situation

2014-11-07T21:00:00-05:00

One of the pieces of code I’m maintaining has an absurd situation in its build structure—honestly, I’m not sure how it ever compiled. For simplicity’s sake, let us assume the four following files:

main.c
secondary.c
writer.h
calculator.h

The project has many more files than this, of course, but these are the important ones for demonstrating this particular piece of insanity (which shows up many places in the codebase).

I’m reproducing here some dummy code representing an actual set of relationships in the codebase. The functions and module nameshave been changed; the relationships between the pieces of code have not.¹ When I started trying to build the program that included what I am representing as main.c below, this is the basic structure I found:

`main.cpp`

This is the main module of the program. In the actual code in which I found this particular morass, it was actually code generated by the UI builder in Visual Studio 6² and then turned into an unholy mess by a developer whose idea of good programming involved coupling the various parts of the code as tightly as possible.³

#include "calculator.h"
#include "secondary.h"

int a=0, int b=0;

int addNumbers(a, b) {
    return a+b;
}

void doBadThingsWithGlobals(int * someNumber) {
    a = 6;
    *someOtherNumber = 5;
}

#include "writer.h"

void main() {
    a = 3;
    doBadThingsWithGlobals(&b);
    addNumbers(a, b);
    doStuffWithNumbers(a,b);
    subtractNumbers(b, a);
}

// More insanity follows...

Yes, the main function and the doBadThingsWithGlobals function are both modifying global state, and yes, there is an include statement midway down through the module. (Just wait till you see what it does.)

“secondary”

Here is a secondary module which has been somewhat cleaned up. It has normal relationships between header and source files, and includes all its dependency headers at the top of the file. It has a header which defines the public API for the module, and that even has inclusion guards on it.

`secondary.h`

#ifndef SECONDARY_H
#define SECONDARY_H

int doStuffWithNumbers();

#endif SECONDARY_H

`secondary.c`

The doStuffWithNumbers function here calls addNumbers:

#include "secondary.h"
#include "calculator.h"

int doStuffWithNumbers(int x, int y) {
    addNumbers(x, y);
}

But wait! you say, That function isn’t defined here! Ah, and you would be right, except that it doesn’t refer to the addNumbers function in main.c. It refers to a function implementation in calculator.h.

`calculator.h`

int addNumbers(int p, int q) {
    return p + q;
}

int subtractNumbers(int r, int s) {
    return r - s;
}

Strangely, this addNumbers function is identical to the one in main.c. Even more strangely, it is defined—not merely declared, actually defined—in the header file! Nor is this the only such function. Look at the details of writer.h, which was mysteriously included above in the middle of the main module.

`writer.h`

void writeStuff() {
    fprintf(stdout, "a: %d, b: %d", a, b);
}

Once again, we have a full-fledged implementation in the header file. Why, you ask? Presumably because the developer responsible for writing this code never quite got his head around how C’s build system works. The entirety of one of the central components of this software—an element that in any normal build would be a common library—was a single, approximately 2,000-line header file. (Say hello to calculator.h up there; that’s what I’m abstracting away for this example.)⁴

Worse: it is printing the values of a and b, and no, I am not skipping some part of writer.h. It is getting those from main.c, because it was included after they were defined, and the build process essentially drops this header inline into main.c before it compilation.⁵ So here we have a header file with the implementation of a given piece of code, included in a specific location and defined in such a way that if you change where it is included, it will no longer function properly (since the variables will not have been defined!)

Worse, there are conflicting definitions for one of the functions used in main.c, and because of its dependency on other functions in calculator.h (e.g. subtractNumbers in this mock-up), it cannot be removed! Moreover, given the many places calculator.h is referenced throughout the code base, it is non-trivial to refactor it.⁶

If this sounds insane… that’s because it is.

If you’re curious how I dealt with it, well… I renamed the addNumbers() function in main.c to _addNumbers() and put a loud, angry TODO on it for the current release, because the only way to fix it is to refactor this whole giant mess.

The takeaway of the story, if there is one, is that people will do crazier, weirder, worse things than you can possibly imagine when they don’t understand the tools they are using and just hack at them till they can make them work. The moral of the story? I’m not sure. Run away from crazy code like this? Be prepared to spend your life refactoring?

How about: try desperately not to leave this kind of thing for the person following you.

That’s actually not wholly true, because these pieces of code are also duplicated in numerous places throughout the codebase. We’ve eliminated as many as possible at present… but not all of them, courtesy of the crazy dependency chains that exist. Toss in a dependency on Visual Studio 6 for some of those components, and, well… suffice it to say that we’re just happy there are only two versions floating around instead of the seven that were present when I started working with this codebase two and a half years ago.↩
Yes, that Visual Studio 6. The one from 1998. Yes, that’s insane. No, we haven’t managed to get rid of it yet, though we’re close. So close.↩
I am not joking. Multi-thousand line functions constituting the entirety of a program are not just normal, they are pretty much the only way that programmer ever wrote. When you see the code samples below, you will see why: someone was lacking an understanding of C’s build system.↩
Also, that’s the piece of code of which I found seven different versions in various places when I started. Seven!↩
I once ran into some code working on a different project for an entirely different client where there had been a strict 1,000-line limit to C source files, as part of an attempt to enforce some discipline in modularizing the code. Instead of embracing modularity, the developers just got in the habit of splitting the source file and adding #include statements at the end of each file so that they could just keep writing their non-modular code.↩
I have tried. Twice. I’m hoping that the third time will be the charm.↩

Nailed It

2014-10-22T22:15:00-04:00

Yesterday, something rather remarkable happened. Someone leaked a copy of the trailer for Avengers: Age of Ultron… and Marvel, rather than throwing a hissy fit, just tweeted:

Dammit, Hydra. (October 22, 7:50 PM EST)

Pitch perfect. It’s self-aware, self-referential in a funny way without being too clever-seeming or coming off like it’s trying too hard, and just a generally good response. The team could have fought it (though ultimately that would have just made things worse), but instead Marvel played its hand perfectly. The response was was funny and demonstrated that the folks who work there actually understand how the internet works.

That alone would have been good enough to put Marvel in a league of its own when it comes to managing things not going the way hoped for. But (after what I’m sure was considerable back-room wrangling), they followed it up an hour and a half later with another, equally fantastic tweet:

Here it is! Watch the @Avengers: #AgeofUltron Teaser Trailer right NOW: http://youtu.be/tmeOjFno6Do #Avengers (October 22, 9:18 PM EST)

Your average old-media company these days would have thrown a fit and made a stink about the release of their media. They would have done everything in their power to get the video taken down. Many companies have done just that under similar circumstances, aiming to get the trailer, snippets of the movie, music, or the like removed from the internet. But that simply isn’t how the internet works: it famously “treats censorship like damage and routes around it” (John Gilmore). Once a video is online, it’s online. Someone, somewhere, still has a copy of it and can put it back up. So rather than fight it… Marvel just rolled with it and made the best of the situation. They cracked a joke, went ahead and put the trailer out themselves, and earned general approval from the internet. Again.

Despite being a decades-old company, Marvel is clearly a new media company through and through at this point. They managed to dodge the Streisand effect quite nicely, turning what could have been an opportunity for hostility all around into a PR coup and a win that they couldn’t have scored on their own.

Other old (and new!) media companies, take note. This is the way you play the game. You recognize when the cat is out of the bag and you run with it. Own it. Make it your own somehow. Don’t let it own you. The internet is a big, crazy, chaotic place, and you can never hope to control it—nor even the narrative about you and your stuff, whatever that may be—like you might have been able to do twenty-five years ago. But that’s okay. If you can roll with the punches, you can still come out ahead, and you’ll look a little more human doing it. I call that winning.

(Go Marvel.)

The Next Generation of Version Control

2014-10-16T21:45:00-04:00

The current state of affairs in version control systems is a mess. To be sure, software development is far better with any of the distributed version control systems in play—the three big ones being Git, Mercurial (hg), and Bazaar (bzr), with a few other names like Fossil floating around the periphery—than it ever was in a centralized version control system. There are definitely a few downsides for people converting over from some standard centralized version control systems, notably the increased number of steps in play to accomplish the same tasks.¹ But on the whole, the advantages of being able to commit locally, have multiple complete copies of the repository, and share work without touching a centralized server far outweigh any downsides compared to the old centralized system.

That being so, my opening statement remains true, I think: The current state of affairs in version control is a mess. Here is what I mean: of those three major players (Git, Hg, and Bazaar), each has significant downsides relative to the others. Git is famously complex (even arcane), with a user interface design philosphy closely matching the UI sensibilities of Linus Torvalds—which is to say, all the wires are exposed, and it is about as user-hostile as it could be.² It often outperforms Hg or Bazaar, but it has quirks, to say the very least. Hg and Bazaar both have much better designed user interfaces. They also have saner defaults (especially before the arrival of Git 2.0), and they have better branching models and approaches to history.³ They have substantially better documentation—perhaps especially so with Bazaar, but with either one a user can understand how to use the tool without having to understand the mechanics of the tool. This is simply not the case with Git, and while I enjoy knowing the mechanics of Git because I find them interesting, having to understand the mechanics of a tool to be able to use it is a problem.

But the other systems have their downsides relative, to Git, too. (I will focus on Hg because I have never used Bazaar beyond playing with it, though I have read a good bit of the documentation.) Mutable history in Git is valuable and useful at times; I have rewritten whole sequences of commits when I realized I committed the wrong things but hadn’t yet pushed.⁴ Being able to commit chunks instead of having to commit whole files at a go is good; I feel the lack of this every time I use Hg.⁵ (Needing to understand the file system that Git invented to make sure you do not inadvertently destroy your repository is… not so good.) A staging area is nice,⁶ (even if having to stage everything manually can be in the pain in the neck⁷).

In short, then, there was no clear winner for this generation. Each of the tools has significant upsides and downsides relative to the others. Git has become the de facto standard, but not because of its own superiority over the alternatives. Rather, it won because of other forces in the community. Mostly I mean GitHub, which is a fantastic piece of software and easily the most significant driving factor in the wider adoption of Git as a tool. The competition (Bitbucket and Launchpad) are nowhere near the same level of sophistication or elegance, and they certainly have not managed to foster the sorts of community that GitHub has. The result has been wide adoption of Git, and a degree of Stockholm Syndrome among developers who have adopted it and concluded that the way Git works is the way a distributed version control system should work.

It is not. Git is complicated to use and in need of tools for managing its complexity; the same is true of Hg and Bazaar, though perhaps to a slightly lesser extent because of their saner branching models. This is what has given rise to the plethora of different formal workflows representing various attempts to manage that complexity (which have been applied to other systems as well). Managing branching, linking that workflow to issues, and supplying associated documentation for projects have also cropped up as closely associated tasks— thus the popularity of GitHub issues and Bitbucket wikis, not to mention Fossil’s integration of both into the DVCS tool itself. None of the tools handle differences between file systems very elegantly (and indeed, it took years for Git even to be useable on Windows). All of them especially struggle to manage symlinks and executable flags.

So there is an enormous opportunity for the next generation of tools. Git, Hg, and so on are huge steps forward for developers from CVS, Visual SourceSafe, or SVN. But they still have major weaknesses, and there are many things that not only can but should be better. In brief, I would love for the next-generation version control system to be:

distributed (this is now a non-negotiable);
fast;
well-documented—at least as well as Hg is, and preferably as well as Bazaar is;
well-designed, which is to say having a user interface that is actually a user-interface (like Hg’s) and not an extremely leaky abstraction around the mechanics;⁸
fast;
file-system oriented, not diff-oriented: this is one of Git’s great strengths and the reason for a lot of its performance advantages;
extensible, with a good public API so that it is straightforward to add functionality like wikis, documentation, social interaction, and issue tracking in a way that actually integrates the tool;⁹
and last but not least, truly cross-platform.

That is a non-trivial task, but the first DVCS that manages to hit even a sizeable majority of these desires will gain a lot of traction in a hurry. The second generation of distributed version control has been good for us. The third could be magical.

A point that was highlighted for me in a conversation a few months ago with my father, a programmer who has been using SVN for a long time and found the transition to Git distinctly less than wonderful.↩
Anyone who feels like arguing with me on this point should go spend five minutes laughing at the fake man pages instead.↩
Few things are as hotly debated as the relative merits of the different systems’ branching models and approaches to history. At the least, I can say that Hg and Bazaar’s branching models are more to my taste.↩
Yes, there are extensions that let you do this with Hg, but they are fragile at best in my experience, and substantially less capable than Git’s.↩
Yes, I know about Hg’s record extension. No, it is not quite the same, especially because given the way it is implemented major GUI tools cannot support it without major chicanery.↩
Yes, I know about Hg’s queue extension, too. There is a reason it is not turned on by default, and using it is substantially more arcane than Git’s staging are. Think about that for a minute.↩
Yes, there is the -a flag. No, I do not want to have to remember it for every commit.↩
Let’s be honest: if Git’s abstraction were a boat, it would sink. It’s just that leaky.↩
GitHub does all of this quite well… but they have had to write heaps and gobs of software around Git to make it work.↩

Pushing Into C's Corner Cases

2014-08-12T09:00:00-04:00

I’m working on a project that is all in C because of its long history and legacy. We’re slowly modernizing the codebase and writing all our new code in Python (using NumPy, C extensions, and so on for performance where necessary). Occasionally, I just want to bang my head against the wall because there are things we can do so simply in any modern language that you just can’t do in any straightforward way in C. For example, I have file writers that all work exactly the same way, with the single exception that the format string and the data that you put into it vary for each file.

In Python, this would be straightforward to handle with the class machinery: you could simply specify the format string in each inheriting class and define the data points to be supplied at the top of an overriding function, call the parent function with super() and be done.

To do something similar in pure C is nearly impossible. You can supply a format string with each function (or module, or however you separate out the code), and if you feel especially clever you could convert all your data types to strings and pass them as a list to be printed by the standard function. The net result would be longer and less maintainable than simply having a set of essentially-duplicate functions, though.

Don't Be Rude

2014-07-12T15:30:00-04:00

Note

I have left the original post here as I wrote it, but there is an important addendum at the bottom of the post that you should make sure to read (and note in particular the further addendum).

This post came off as pretty critical of MarketCircle, and that really wasn’t my point. I wanted to use a bad experience I had with MarketCircle to illustrate a general principle, not to poke at any particular company. I did that poorly in this particular piece; for some follow-up on that see this post which I wrote later that day, analyzing how and why this piece so spectacularly failed to accomplish my desired goals.

In any case, I do not want this piece to turn people off of using MarketCircle’s software. I leave the unedited version below because I believe in having the intellectual integrity to own one’s mistakes. This was one of mine.

The quickest way to make me bid your company or product farewell is to patronize me. Don’t talk down to me. Never treat me like anything but an adult. The moment you do, I am gone.

Given which: farewell MarketCircle, and adieu Billings.

A story: When I started working as a freelance software developer on the side a few years ago,¹ I looked at my options for tracking time and invoicing clients. I eventually settled on Billings, by MarketCircle. It’s solid software: it is reliable, works well, and does everything I need it to, including tracking different clients and projects easily and sending them estimates or invoices. Best of all, from my perspective, it was a local app. You could sync with a server out in the cloud somewhere via Billings Pro, but you did not have to, and you could use the Mac-native application, not some web app out there. Last but not least, it had a great menubar app. I was sold, and I gladly dropped $40 for a single-user license.

Fast forward to June 2013. MarketCircle, like a lot of software development companies, came to the conclusion that it is really hard to develop software as a series of discrete releases, for which you get people to pay over and over again. Perfectly sensibly, they discontinued development on and support for their standalone software and provided a (discounted!) migration path for users to upgrade to the Pro (syncing, etc.) version of the software. Note that they did not do anything to disable functionality in existing Billings installations—just provided an upgrade path and stopped developing it. That is the right way to handle it. So far so good.

I am a software developer, and I have seen the pressures that exist in this industry. This move made good business sense to me, and I liked Billings as a product. I was quite willing to look at their Pro plan, and possibly even to invest in it, despite the fact that I did not need it, because I believe in supporting the developers of the software I use.

I emailed them a couple follow-up questions. One of them, and among the most important to me because of how I work for one particular client:

I note that in Billings Pro, unlike in Billings, I can’t track multiple slips simultaneously. This is problematic for me, as I often do this to keep track of hours worked against a “Personal projects” bit so I can see my hourly variations. That’s a make-or-break kind of thing for me—any chance you guys will change that behavior?

To elaborate: I like to track my total hours worked every week in a simple way, so I have a “Personal” timer going alongside the project timer for whatever I am doing. The fact that Billings let me do this was one of the selling points for me. Even so, I did not necessarily expect them to support the functionality going forward. The response I got started out reasonably enough:

We allow you to have multiple active timers, but you can only time one task at a time in both applications. In Billings, there was a bug with this, however, this was corrected in Billings Pro.

Thus far, fair enough: they saw this as a bug. I disagreed, but I understand.

Then this, though:

While we all multi-task we cannot work on two billable items at once.

Whoops. You just talked down to me.

You also clearly didn’t read the original email, because you followed up by asking this:

Can you explain a little more about what you track and how and I can see if there’s a different way to do this in Billings Pro that will give you the same result?

Hmm. Let me get this straight: I told you what I track and how I use your software, and you thought the appropriate response was to instruct me on what I can and cannot do with it? Clearly not having even read the original question carefully?

Nope.

Let me explain: you don’t tell your customers that they can’t use your software in ways peculiar to them. You particularly do not do so as though explaining to a child that we simply cannot do certain things. If a user has a quirky way of using your software, you can of course say you don’t intend to support that quirky behavior—but you do not get to tell them that their unanticipated usage is wrong, and especially not in a condescending tone

I cancelled my Billings Pro trial within five minutes of receiving that email. The original software I kept: I was at a busy time in the year, switching time- tracking software is non-trivial, and it wasn’t hurting me a bit to keep using the original software anyway. As I am evaluating time tracking software again, not least because I do not know through how many OS X upgrades Billings will continue to perform properly, MarketCircle isn’t on the list. It only took one bad experience to leave a bad taste in my mouth and convince me to move on.

At this point, it looks like I’m headed to Harvest. It turns out they don’t support multiple timers, either. But they haven’t talked down to me, and that makes all the difference in the world.

There is a takeaway here for anyone paying attention. Namely: respect your customers. Do not talk down to them. Do not assume their uses for your software are wrong, or stupid, even if they are not what you intended. (If anything, that means your users have thought of use cases you didn’t.)

It is going to far to say that the customer is always right. Sometimes, the customer is wrong. Sometimes, I am wrong as a customer. But the customer is always someone to respect. The moment you stop treating your customer with respect is the moment you cross the line into being a company with which I want to do business to one I will avoid.

Edit and Addendum

When I posted this on App.net, a few thoughtful acquaintances pushed back, noting that the customer service interactions did not read as condescending to them. It is possible that I misread the original customer service rep’s tone in interacting in me. This is a constant danger in dealing with text-only communication. I take some responsibility for that—but I also note that the frustration had already built up in the course of a conversation that had already included a number of failures to respond to address or respond to my questions and concerns.

I should also note that I didn’t mean this as a critique of MarketCircle in particular, though re-reading the post in light of the response, it is clear it comes off more that way than I intended. My interactions with MarketCircle were meant simply to illustrate the broader point: customer service matters, and even one bad customer experience can turn off a customer.

But the takeaway from this addendum is a bit different: I can sympathize with the difficulties facing the customer service rep. I failed at precisely the same task of communicating my intent in writing effectively. Now, whether that rep meant it the way I took it or the way others took it in reading the post, he certainly did not accomplish what he meant to with the exchange. My sympathies are with him. I am perfectly willing (though not perhaps happy; humility is rarely particularly pleasant) to say that I got it wrong here.

All that being said… I still have a bad taste in my mouth, and I am still leery of doing further business with MarketCircle. And that does make the original point in a way, because the emotional response from a bad experience, even one you did not intend, doesn’t fade quickly or at all, even in the face of reasonable articulations of alternative explanations for the bad experience. You have to work at a good customer experience continually.

Further Addendum

MarketCircle actually saw this piece—presumably via my link on Twitter—and got back to me, looking to fix this issue, which I really appreciated. In some sense, then, they are doing exactly what I advocated in this piece.

Early 2010, if you’re curious.↩

Goodbye, Notifications

2014-07-11T18:50:00-04:00

In this week’s episode of Accidental Tech Podcast, hosts Casey Liss and John Siracusa mentioned that they have the sound aspect of notifications disabled on their iDevices (Liss’ iPhone, Siracusa’s iPod Touch). Strange though it might seem, the thought hadn’t occurred to me. I like getting the notice of things having happened on my social media accounts, but I’d concluded recently that I actively disliked having the interruption even of a buzz in my pocket: it forces a mental context shift which inevitably degrades my concentration on whatever task I am about.

I spent ten minutes this evening and went through my iPhone’s notification settings. The only things which have audible or vibrating notifications now are phone calls (including FaceTime) and text messages. Everything else I disabled. Now, I still have notifications on a number of other items: they can show up in Notification Center, and they can put markers on the home screen apps. After all: if I already have my phone out, it is almost certainly no problem to see a notification come in, and I definitely want to be able to glance at the app on my home screen and see that someone has interacted with me in some way. But when I don’t have my phone out? It is unhelpful. It is distracting.

I actually turned on app badges for a number of apps for which I had previously disabled them, because they had been extraneous when I was getting noises or buzzes for the apps and services in question. I also tweaked a number of other apps: some can show app badges but not appear in notification center. Most cannot show anything on the lock screen at all. If I want to check on notifications, I can look explicitly.

We will see how the experiment goes. Even just a few hours in, though, I can already say I like it. I did not get any buzzing in my pocket when a few people interacted with me on App.net, or Instagram, or anywhere else. And, social media being what it is, none of those interactions are temporally important (however much it might feel otherwise). They will still be there waiting when I get back.

Now, this does not automatically make me more productive. I still need self control to be most effective in using my time. It does take away a few of the most obvious distractions and interruptions that make it hard to focus, though, and that is definitely a win.

Economies of Scale

2014-07-11T10:35:00-04:00

I was reading through an interesting Ars Technica article on the new Long Range Strike Bomber (LRS-B) proposal the Air Force is soliciting. It’s generally interesting to me in part because I’ve worked on a related project in the past, and we talked fairly often about how the LRS-B program might impact it. The article is worth your time. This quote from Robert Gates in the middle of the article, which touches on the program the LRS-B would replace, caught my attention, though:

What we must not do is repeat what happened with our last manned bomber. By the time the research, development, and requirements processes ran their course, the aircraft, despite its great capability, turned out to be so expensive—$2 billion each in the case of the B-2—that less than one-sixth of the planned fleet of 132 was ever built.

Looking ahead, it makes little sense to pursue a future bomber—a prospective B-3, if you will—in a way that repeats this history. We must avoid a situation in which the loss of even one aircraft—by accident, or in combat—results in a loss of a significant portion of the fleet, a national disaster akin to the sinking of a capital ship. This scenario raises our costs of action and shrinks our strategic options, when we should be looking to the kind of weapons systems that limit the costs of action and expand our options.

Now, in one sense, Gates was absolutely right. On the other hand, he seems to have committed a classic blunder in dealing with these kinds of costs: economies of scale matter. Part of the reason the per-unit price of the B-2 was so high was precisely that we only bought 20 of them. While the units were individually expensive to manufacture and maintain, because of unique materials used in their construction and so on, they were much more expensive to manufacture in small numbers than they would have been in large numbers. There are basically two reasons for this:

The manufacturing process couldn’t do what it does best (turn out large numbers of standardized parts and thereby reduce costs).
The costs of development—research, software development, etc.—were all distributed over a much smaller pool than they would have been had the government purchased more aircraft.

This second point is incredibly important to understand. It is certainly true that the absolute cost of buying 132 B-2s would have been high, possibly astronomically and unaffordably high. What it would not have been is $264 billion. Even assuming that manufacture costs were fully half of the cost-per- plane (almost certainly not the case), it would have been barely over half that. Assume that the B-2 cost $1B per plane to build, and that the other $10B was research. Well, that’s still an expensive plan… but the total cost is something like $144B, not $264B. Those economies of scale matter.

This same reality is a point made later in the article by another commentator, but I couldn’t let it go. Things like this drive me nuts, because they’re such a common failing in our political discourse. Ignorance of basic economics from the people making decisions with this kind of economic impact is profoundly unhelpful.

Bundling!

2014-05-13T14:35:00-04:00

“Why do we have to choose between print and digital?” asked Richard Curtis at Digital Book World last week, before tackling the topic of bundling—getting ebooks at reduced cost or even free when buying a physical copy of the book. Drawing an analogy from music purchases that have moved in the same direction, he suggests that publishers ought to be bundling, and then poses the query: When you purchase a print book you should be able to get the e-book for…

the full combined retail prices of print and e-book editions
an additional 50% of the retail price of the print edition
an additional 25% of the retail price of the print edition
$1.00 more than the retail price of the print edition
free

He suggests that this proves to be something of a conundrum for decision-makers in the publishing industry. With respect, and while recognizing that it probably feels like a conundrum to the publishers, I think the answer is really quite simple. Publishers can dramatically increase their profits, and do it in a way that readers will love. (This is the part where you call me crazy. Up next is the part where I show you why I’m not.)

All or nothing

First, we should note that while readers would always choose (e) and publishers would love it if they could get away with (a), the reality is that both of these leave one party out in the cold. Publishers need readers, and readers need publishers. Publishers need readers or they die. Readers need publishers as providers of quality content—not only as the gatekeepers but also as polishers who take good books and make them great. Any system that will pan out well must therefore respect both sides of that equation. Both (a) and (e) fail that test immediately.

In the case of (a), the consumer can rightly point out that the cost of distribution of a book is minimal, trivial even, in the grand scheme of book production. That goes double for ebooks: the cost of running a server is a pittance compared to the cost of writing, editing, and proofing a book. “So,” any smart reader says, “I’ve already paid for the book. Why should I have to pay just as much again for the ebook?”

In the case of (e), the consumer is getting something of real value—the ebook, with its associated portability, the ability to create [communal interactions][craigmod] around the content through shared marginalia, and so forth—but without recognizing any infrastructure costs this poses to the publisher. As always, there is no free lunch, and that is as it should be.¹ The worker deserves his wages, and that includes the editor who turns a manuscript into ebook form—especially for good ebooks, which entail a great deal of work beyond simply running the print manuscript through a conversion script. That involves real people’s time, and therefore costs real money.

Neither of these options, then, is ultimately good for the market. The readers will rightly reject paying the full price again for a book in a different form; they’ve been conditioned by too many interactions on the internet not to recognize that digital transmission of files the size of a book is, while not costless, not costly either. On the other hand, publishers still need to make money, and they do sink real time and money into the ebook—not at the distribution point, but in the infrastructure involved in the preparation of the manuscript and readying it for digital and physical publication.

Again: publishers need readers and readers need publishers.

Percentage games

Percentage-based cuts—like Curtis’ options (b) and (c)—are much more sensible and reasonable from the perspective of both the consumer and the publisher. In each of these cases, the publisher is granting that the customer has already made a purchase—perhaps a significant one, in the case of a hardcover fiction book. Indeed, when we move out into the realm of reference books or textbooks, the consumer has already given the publisher quite a lot of money. Thus, options (b) and (c) are much friendlier to the consumer than choice (a), while still affording the publisher some profits, unlike (e). This is clearly a step in the right direction.

The percentage option quickly runs into issues when we start thinking about how such a scheme would work in practice, though. Is it 25% of the hardcover but 50% of the paperback, so that the publisher can recoup more of the costs? In this scheme, it is difficult to match the actual cost of the ebook sale to its relative value compared to the physical copy. Moreover, it’s difficult to standardize. When purchasing a textbook at $150, should someone have to pay another $37.50 or $75 to have a digital copy? It seems unlikely that preparing an ebook of a textbook is really 5-6 times more costly than the preparation of a fiction ebook, which on a percentage basis would come out around $6.50 or $13 for the hardcover at those rates, or $2 or $4 for paperbacks.

Equally important: will people pay that much for a digital copy? Publishers may want to study the question in depth by testing the market, but this is a waste of time. The answer is obvious to anyone under the age of 30: no. The market simply won’t support those kinds of costs on the upper end of the spectrum.

Again, customers may recognize that they are subsidizing more than simply the cost of distribution, but the preparation and distribution of the ebook don’t justify an additional percentage on these scales beyond some point. I suspect that most customers are willing to pay extra to get the ebook in addition to the physical copy—just not, in most cases, that much extra.

Aside: on reasonability and trained markets

We must recognize that markets can be trained. People have come to see $0.99 as a reasonable price for individual songs. There was nothing inevitable about that outcome; it is a direct result of the success of the iTunes store. Had prices been set at $1.49 or $0.33, it’s likely we would have settled on that number as a reasonable price. Similarly, TV show episodes sell at $1.99,² and people seem to treat that as a reasonable price: the perceived value matches the cost well. They could have been $1 or $2.50, and consumers probably would have settled in with those numbers equally well.

Of course, had the price been too high, we would have rejected it entirely: markets can be trained, but they’re not capable of stretching into just any shape at all.

Admittedly, the music market remains volatile, but consumers on the whole don’t seem to balk at spending a dollar on a song. While the piracy rate remains high, iTunes and similar markets provide an outlet for those who are interested in purchasing their music legitimately.³

This outcome results from the combination of a trained market and a sensible cost/value relationship that allowed the training to occur in the first place. Book publishers should aim for the same outcome: profitability on the basis of perceived reasonability of their prices. This will require training the market, but that is possible so long as their expectations are reasonable.

A reasonable target

Price points

Curtis’ final suggested price point is close to the mark, but I think some revision is in order. Remember: the aim is to buoy both customer satisfaction and publisher profitability. Here’s my proposed pricing scheme for fiction (which could be adapted to other parts of the market fairly straightforwardly):

Standalone ebook: $4.99
Paperback:
- Book: $7.99
- Bundle: $9.99
Trade paperback:⁴
- Book: $14.99
- Bundle: $15.99
Hardcover:
- Book: $26.99
- Bundle: $26.99

I’m basing these on the current pricing schemes in the market—these are the normal suggested retail prices for paperbacks, trade paperbacks, and hard covers—and on the assumption that the publisher’s goal is to maximize revenue, while the consumer’s goal is to get the most content at a price he feels is reasonable. I’m also taking into account the existing profit curves for publishers: paperbooks are relatively low margin, while hardcovers are the major profit points, at least when they’re successful.

Rationale

First, and most importantly, I believe the market will support these price points. The standalone ebook is less expensive than the paperback, as it should be, since its distribution costs are much lower than the costs of printing and shipping paperbacks. At the same time, ebooks sales will still generate revenue for the publisher; $5 is not a meaningless amount of money.

For each tier upwards, the cost of the bundled ebook drops. The publisher thus acknowledges the increasing profitability of each tier as well as the increasing cost to the reader. At the same time, the lowered bundling cost incentivizes the user toward the higher profit items. In each case, the bundling cost is sufficiently low as to be in the “impulse purchase” range for most users.⁵

Readers will be far more likely to front the cost of a hardcover if an ebook comes bundled with it, because the value proposition is so much better. At the same time, this is unlikely to decrease the profits of the publisher, because the margins are much higher for hardcovers.

In fact, bundling at these rates will likely increase publisher profits from ebooks, as most readers currently choose between ebook and physical books. The price of a hardcover is simply too high to allow for the purchase of both. (Even when this is not actually true, it seems true to consumers, which is equally important in determining their behavior.) With a sufficiently lower barrier to getting the additional content, the likelihood that the reader purchases both goes up substantially.⁶

This potential for increased profitability is compounded by the availability of the bundle at initial purchase time. A consumer who has already committed to spending $8 on a book is unlikely to balk at $10, and even less to balk at the transition from $15 to $16. In many cases, the publisher will earn more money from the book purchase than before, but the reader is still getting a good deal on the ebook. This is exactly the combination that makes for a flourishing market.

Finally, the simplicity of these numbers is extremely helpful. Standardizing these prices will decrease the friction inherent in making the purchase decision, which increases the likelihood that a purchase will be made. I’m not suggesting a cartel—price standardization is natural in this sort of market—and I believe the price points I’ve suggested are where the market will settle in the long run. The publishers who get there first will earn enormous goodwill from their readers in the short term, as well as demonstrating their leadership in the industry in ways that set them up for long term success.

Bundle up

A smart approach to bundling could be enormously beneficial to the publishing industry. In addition to the pure numerical profitability of the approach outlined above—no small detail in an industry struggling to adapt to the realities of the new economy—it establishes that the publishers are responsive to customers in a way that other large media have not seemed to be. Nothing is so helpful to a company’s long-term sustainability as for consumers to like it. Reasonable bundling prices would go a long way toward helping readers see publishers as friends, rather than enemies.

Obviously these numbers work best in the context of fiction. The value propositions are entirely different in other contexts; a cookbook is an entirely different thing than a copy of The Hobbit. Across the board, though, publishers should keep the same goals in mind: profitability by means of reasonability and approachability. Be allies of the readers, not their enemies. Make it easy and affordable for them to pay you for your work, and they will.

My thanks to Stephen Carradini for invaluable contributions to this piece in two forms: many long conversations about this very topic, and a helpful edit of the actual content.

[craigmod]: http://craigmod.com/journal/post_artifact/#section_4 “Post Artifact Books and Publishing, Section 4: The post-artifact system” from Craig Mod"

Additionally, there is a signaling problem here: “free” suggests “low value” in a way that publishers rightly want to avoid. See “iA Writer: On Prices and Features”, Section 2: Cost, by Oliver Richtenstein for a lengthy and sensible exploration of this issue. The issue of signaling value should be taken into account in my suggestions later, as well. But more on that below.↩
When they sell at all, of course. I’ve written about this problem [before][piracy]: piracy explodes when there is demand without supply. It also tends to grow at a higher rate when the cost is perceived as unreasonable. TV shows priced at $5/episode wouldn’t do well; they seem to sell quite briskly at $1.99. Publishers run the risk of fomenting piracy by setting their prices too high.↩
I have never seen someone complain that a song is too expensive at a dollar who was willing to pay anything. A penny would be too pricey from the pirates’ point of view.↩
Trade paperbacks (TPBs) are similar in size to hardcovers, but have soft covers similar to those in a paperback. Fiction TPBs typically go for around $15. Over the last few years, publishers have started shifting away from the low-margin paperback market into these trade paperbacks, which provide a bit higher profit for them. Personally, I don’t mind, because these books tend to be higher quality paper and bindings. If I’m sitting down with a monster like one of the books in The Wheel of Time, this is far and away the best format for a physical copy.↩
On the signaling issue: the price of the ebook is sufficiently high as to continue to signal real value here, I think. However, in the case of other kinds of books, this scheme should be revisited. A complex EPUB3 with embedded videos or interactive content should signal that it offers a higher value proposition than other ebooks with a higher price point; in some cases, if that content is sufficiently central to the value proposition of the book, it might be more expensive than the physical copies.

Similarly, a textbook might sell for $150, its ebook at $50, and the bundle at $165—because the cost of preparing a textbook ebook may be substantially higher than that of preparing a fiction ebook. Signaling matters, but overpricing is as much a risk here as underpricing.↩
This has the added benefit of making the purchase of new books over used books more attractive to the consumer: if the coupon for ebook at reduced rate is only available at new book purchase, a $3 used book suddenly has a much lower value proposition relative to the original when the reader is interested in having an ebook copy as well, since the cost of having both is still $8.

Of course, this leads us to the question of ebook resale, which is currently a legally murky area at best, and requires considerable legal and intellectual development.↩

Why the Smart Reading Device of the Future May Be … Paper

2014-05-03T10:45:00-04:00

One thing I didn’t talk about in comparing reading experiences on a Kindle and on an iPad the other day is the elephant in the room: old-fashioned books. I enjoy Kindle and iPad, but I still love books best. Turns out I’m not alone… and there might just be reason for it.

Brandon Keim at Wired:

Paper books were supposed to be dead by now. For years, information theorists, marketers, and early adopters have told us their demise was imminent. Ikea even redesigned a bookshelf to hold something other than books. Yet in a world of screen ubiquity, many people still prefer to do their serious reading on paper.

Count me among them. When I need to read deeply—when I want to lose myself in a story or an intellectual journey, when focus and comprehension are paramount—I still turn to paper. Something just feels fundamentally richer about reading on it. And researchers are starting to think there’s something to this feeling.

iPad vs. Kindle

2014-04-30T21:20:00-04:00

I’ve been a happy owner of both a Kindle and an iPad Mini for the last several months, and it occurred to me tonight that I use them very similarly in some ways. Both are primarily reading devices for me. What is different is the kinds of material I read on each.¹

My Kindle is a first generation Paperwhite, in fairly good condition. (It has one significant quirk in that it sometimes turns on without the power button being pushed. Alas.) I use it nearly every day right now. I have most of my school books on it, and several of my favorite novels. I’m rereading Patrick Rothfuss’ The Wise Man’s Fear right now, and so I spend a good half a hour a day on the Kindle for that alone. I also get a lot of my seminary reading done on the device.

On the iPad, on the other hand, I read a lot of web pages, nearly all via Instapaper. I had sometimes had Instapaper items delivered to my Kindle, and that worked fairly well, but I much prefer the experience of using the app on the iPad. I opt to do pretty much any technical reading on the device: its screen just works much better for dealing with things like code samples embedded in a blog post—not least because I can scroll easily if I need to! I also do basically all my Bible reading on the iPad. It is far easier to navigate to different parts of the text, switch translations (or original languages!) while keeping my place there on any of the top-tier iPad apps than on the Kindle. And I sometimes read comics on the iPad—something I would not try in a million years on the current Kindle screen!

A friend asked a few months ago if I thought one would obviate the other. Given the qualification that neither is in any sense truly a necessity—we could quite easily get along without either—my answer after several months with both is no. Though the devices are similar in a number of ways, they fit into very different niches. The things I actively enjoy on each are very different. The Kindle is good for much longer-form reading, and its lack of distractions is nice (though I often take advantage of the Do Not Disturb mode on the iPad when I actually want to accomplish things besides talking on social media). The iPad is better for anything with color, for technical documents, and for anything where navigation more complex than one-page-after-another is important. I would not particularly want to read a novel on it, though!

I will be curious to see if the devices converge at some point in the future.² At present, no technology gives both the responsiveness and gorgeous color of the iPad and the low-contrast, pleasant long-form reading experience offered by the Kindle’s e-ink. If at some point we get a technology that does both, it will be pretty amazing. In the meantime… we still have pretty amazing pieces of technology, and I enjoy them both a lot.

I also use the iPad for a number of other things: App.net and Twitter and so on, Paper, starting some ideas for blog posts, etc. But mainly I read on it!↩
No, Amazon’s Kindle Fire series of tablets are nothing like that convergence: they are functionally just poor-man’s-iPads hooked into Amazon’s ecosystem. Note that I’m not making a comment about the quality or lack thereof on the devices—only that they’re much reduced in capabilities compared to an iPad or Android (e.g. Nexus 7).↩

A Little Crazy

2014-04-29T19:30:00-04:00

I’m going to do something a little crazy, I’ve decided. I’m going to go ahead and do like I wrote a bit back, and make Step Stool actually a thing over the course of the rest of the year. Not so crazy. What is a bit nuts is the way I’ve decided to go about that process. In short: as close to the hardest way possible as I can conceive.

Over the last couple weeks, I’ve been spending a fair bit of time toying with Io. It’s a neat little language, very different in its approach to a lot of things than the languages I’ve used previously. My programming language history is very focused on the “normal” languages. The vast majority of real- world code I’ve written has been in one of C, PHP, or Python. I’ve done a good bit of Javascript along the way, more Fortran than anyone my age has any business having done, and a little each of Java and Ruby. Like I said: the normal ones. With the exception of Javascript, all of those are either standard imperative, object-oriented, or mixed imperative and object-oriented languages. Python and Ruby both let you mix in a fair bit of functional-style programming, and Javascript does a lot of that and tosses in prototypal inheritance to boot.

But still: they’re all pretty mainstream, “normal” languages. Io isn’t like that at all. For one thing, it’s hardly popular in any sense at all. Well-known among the hackers¹ I know, perhaps, but not popular by any measure. It’s small. And it’s very alien in some ways. It’s prototypal inheritance, not normal inheritance. Courtesy of Javascript, I have a little familiarity with that, but it’s definitely still not my default way of thinking about inheritance. Python’s inheritance model (the one I use most frequently) is essentially the same as that in C++, Java, PHP, and so on—it’s normal class-driven inheritance. Io goes off and does full-blown prototypal inheritance; even just the little I’ve played with it has been fun.

Io also does a bunch of other things a lot different from the other languages I’ve used. First, there are no keywords or—formally speaking—even operators in the language. Every action (including ones like + or for) is simply a message. Every value is an object (so 1.0 is just as fully an object as an arbitrarily-defined Person). The combination means that writing 1 + 2 is actually just interpreted as the object 1 receiving the + message carrying as its “argument” the 2 object (really just the message contents). This is completely different at a deep paradigm level from the normal object-oriented approach with object methods, even in a language like Python where all elements are objects (including functions). The net result isn’t necessarily particularly different from calling methods on objects, but it is a little different, with have some interesting consequences. Notably (though trivially—or at least, so it seems to me at this point), you can pass a message to the null object without it being an error. More importantly, the paradigm shift is illuminating.

Io also has far more capabilities in terms of concurrency than any of the other languagues with which I’m familiar, because it actively implements the Actor Model, which means its implementation of messaging instead of object method calls can behave in concurrent ways. (I’d say more if I understood it better. I don’t yet, which is one of the reasons I want to study the language. Concurrency is very powerful, but it’s also fairly foreign to me.) It’s also like Lisp in that its code can be inspected and modified at runtime. I’ve wanted to learn a Lisp for several years for this kind of mental challenge, but the syntax has always just annoyed me too much ever to get there. Io will give me a lot of its benefits with a much more pleasant syntax. It has coroutines, which are new to me, and also helpful for concurrency.²

The long and short of it is that the language has a ton of features not present in the languages I have used, and—more importantly—is paradigmatically different from them. Just getting familiar with it by writing a goodly amount of code in it would be a good way to learn in practice a bunch of computer science concepts I never had a chance to learn formally.³

By now, as long as I’ve rambled about Io, you’ve probably figured out where I was going in that first paragraph. I’ve decided to stretch my brain a bit and write Step Stool in Io. There are bunches of static site generators out there in Python already, many of them quite mature. (This site is running on one of them as of the time I write this post—it’s quite solid, even its quirks and limitations occasionally annoy me.) The point of Step Stool has always been twofold, though. First, I’ve wanted to get to a spot where I was really running my own software to manage my site, letting me do whatever I want with it and guaranteeing I always understand it well enough to make those kinds of changes. Second, I’ve just wanted to learn a whole bunch along the way. Third, it’s right there in the website link: step-stool.io! How could I pass up such an opportunity?

It is that second goal that has pushed me to do this crazy project this crazy way. It’s crazier than just teaching myself a language in order to do the static site generator itself, too, because there are a few other pieces missing that I’ll need to write to make this work… like a Markdown implementation and an HTML templating language. I’ve never written anything remotely like either before, so I’m going to take the chance to learn a lot of new things. For the Markdown implementation, rather than relying on regular expression parsing (like most Markdowns do), I’m going to use a Parsing Expression Grammar. That will certainly be more efficient and reliable, but—more importantly—it is also outside my experience. I have yet to start thinking through how to tackle the HTML templating language implementation (though I know I am going to make it an Io implementation of Slim, which I quite like).

In any case, I’m going to be taking a good bit longer to get Step Stool finished. That is all right: I am going to learn a ton along the way, and I am quite sure I will have a blast doing it. And that is exactly what these kinds of projects are for.

I’ll post updates as I go, with the things I’m learning along the way. Hopefully they’ll be interesting (or at least entertaining).

Hackers in the original sense of the world. Not “crackers”, but people who like hacking on code, figuring things out the hard way.↩
Python 3.5 is actually adding coroutines, and I’m excited about that. I’ll feel much more comfortable with them there having used them in Io, I’m sure!↩
I got here backwards, as it were—by way of an undergraduate degree in physics. I don’t regret that for a second: I got a much broader education than I could have managed while getting an engineering degree, and most importantly learned how to learn: easily the most important skill anyone gains from any engineering degree.↩

Learning QML, Part 1

2014-04-11T15:30:00-04:00

For part of my work with Quest Consultants, I’ve been picking up Qt’s QML toolkit to use in building out the UI. The declarative syntax and ability to define one’s own model in non-C++- or Python-specific ways is quite nice. That said, the learning process has had more than a few bumps along the way. I decided to go ahead and write those up as I go, both for my own reference and in the hope that it may prove useful to others as I go.

QML is a Javascript-like language for declarative programming of a user interface. So it’s a Javascript-based language that sort of behaves like HTML. In fact, it behaves like Javascript in terms of how you define, access, and update properties, and you can embed full-featured (mostly) Javascript functions and objects in it.

But when you have nested QML Types, you end up with them behaving more like HTML.

The weirdest bit, and the thing that I’m having the hardest time adjusting to, is that you can only edit properties of root Types when you’re working with an instance of that Type. And those Types are defined by documents.

So, to give the simplest possible example, let’s say I defined a new type called Monkey, in the Monkey.qml file, like this:

// Monkey.qml
import QtQuick 1.1

Item {
    id: monkey_root
    property int monkey_id: -1
    property string monkey_name: "I don't have a name!"

    Item {
        id: monkey_foot
        property string monkey_foot_desc: "The monkey has a foot!"
    }
}

I can use that in another file. If they’re in the same directory, it’s automatically imported, so I can just do something like this:

//main.qml
import QtQuick 1.1

// Rectangle is exactly what it sounds like. Here we can display things.
Rectangle {
    id: the_basic_shape
    height: 400
    width: 400
    color: green

    Monkey {
        id: monkey_instance
        monkey_id = 42
        monkey_name = "George"  // he's kind of a curious little guy
    }

    Text {
        text: monkey_instance.monkey_name
        color: "red"
    }
}

That creates a (really ugly) rectangle that prints the Monkey’s name in red text on a green background. It’s impossible to access directly the monkey_foot element, though, which means that composing more complex objects in reusable ways is difficult. In fact, I haven’t come up with a particularly good way to do it yet. At least, I should say that I haven’t come up with a good way to create high-level reusable components yet. I can see pretty easily how to create low-level reusable components, but once you start putting them together in any specific way, you can’t recompose them in other ways.

From what I’ve gotten my head around so far, this ends up being less flexible than either HTML templating languages (which are, or at least can be, completely declarative) or normal Javascript (which is obviously not declarative). Mind you, it’s all sorts of interesting, and I have a pretty decent idea what I’m going to do to implement our UI with it, but it’s taken me most of the day to get a good handle on that, and my head still feels a bit funny whenever I’m trying to see how best to create composable components.

Note, too, that this is the only way to create a new basic type of object in QML: it has to be the root level object in a QML document. I would really like to be able to access internal declarations—to have named internal types/objects. Unfortunately, QML doesn’t let you do this. I suspect this has to do with how the QML type system works: it actually binds these types to C++ objects behind the scenes. This is a non-trivially helpful decision in terms of the performance of the application, but it certainly makes my brain a little bit twitchy.

There are two basic consequences of this structure. First, any types you need to be able to use in other QML objects have to be defined in their own QML documents. Second, it is (as near as I can see so far, at least) difficult to create good generic QML types of more complex structures that you can then use to implement specific variations. For example: if you want to create accordions, you can create a fair number of the low-level elements in generic ways that you can reuse, but once you get to the relationships between the actual model, delegate, and view elements, you will need to create them in custom forms for each distinct approach.

This is more like creating HTML documents than Javascript, which makes sense, if you remember that QML is Javascript-based but declarative. You just have to remember that while you can define some reusable components, the full-fledged elements are like full HTML pages with a templating system: you can include elements, but not override their internal contents. In QML, you can override some of their contents, which is nice—but that is not the primary way to go about it.

Feels Right

2014-04-04T21:30:00-04:00

I had spent most of the last week and a half working on getting FirebirdSQL configured and ready to use for a project I’m working on with Quest Consultants. It was slow going. The tool is decent, but the documentation is spotty and it felt like everything was just a bit of a slog—to get it working correctly, to get it playing nicely with other pieces of the development puzzle, to get it working across platforms.¹ Then, because I had done something a little bit silly in my eagerness to get up and going last week and written code without a testable configuration, I hit a wall today. The queries weren’t working. I had made a bug.

I spent a substantial part of the day chasing down that bug, and then a conversation with user agronholm on the SQLAlchemy IRC channel (freenode/#sqlalchemy) got me thinking. The Firebird team describes one of their options as an “embedded” server, but agronholm pointed out that what they really mean is portable. It’s running a standalone server and client, but it’s not part of the same thread/process (like SQLite is). Then agronholm very helpfully asked—my having mentioned my preference for PostgreSQL earlier—“Does Postgres not have a portable version?” Two minutes later, we had both found PostgreSQL Portable, and I rejoiced.

It took me less than half an hour to get it downloaded and set up and to confirm that it would work the way we need for this particular piece of software. (Firebird had taken me a good three hours, what with digging through badly organized and not terribly clear documentation.) It took me less than half an hour more to get PostgreSQL to the same point that I’d finally gotten Firebird to after multiple hours working with it. And I was so very happy. What had been an especially frustrating work day now had me quietly smiling to myself constantly for the last two and a half hours as I finished tracking down the bug that had set me on this path in the first place.

Several years ago, when I first started doing web development, I got my feet wet in database work with MySQL—probably the single most common starting point for anyone going that route, courtesy of the ubiquity of the standard Linux-Apache- MySQL-PHP stack.² A year after that, I picked up some work that was already using PostgreSQL and fell in love almost immediately.³ Something just felt better about running psql than running mysql on the command line. Postgres’ implementation of the SQL standard felt more natural. Even the tiniest little details like the way tables display when you query them in psql was nicer. In less than a week, I was sold and haven’t looked back. While I’ve used MySQL out of convenience on shared hosting from time to time, PostgreSQL is unquestionably my preferred database target.

Today’s experience brought that all home again. That grin on my face all afternoon felt a bit silly, but it highlights the difference that really good software design makes. I am not just talking about how it looks here—though, to be sure, PostgreSQL is prettier than FirebirdSQL—but how it works. PostgreSQL feels responsive, its command set makes a lot of sense and is easy to use, and it is extremely well documented. In fact, I would go so far as to say that it is the best documented open source software I have ever used, as well as among the very most robust. (The only other open source software I find to be as incredibly rock-solid and reliable as PostgreSQL is the Linux kernel. I am by no means an expert on either, or on open source software in general, but the Linux kernel is an unarguably amazing piece of work. So is PostgreSQL.) All those tiny little details add up.

It’s a good reminder for me as I write software that yes, the things I care about—the small matters that would be so easy to overlook when customers express no interest in them—really do matter. People may not know that things like typography make a difference in their experience, but those subtle, often imperceptible things matter. They may not consciously notice the differences in your interface design (even a command line interface), but it will change their experience of the software. Do it poorly, or even in a just-good-enough-to-get- by fashion, and you’ll annoy or simply bore them. Do it well, and you might just delight them—even if they can’t tell you why.

Examples

To make my point a little more visible, I thought it might be useful to post samples of SQL to accomplish the same task in the two different database dialects.

FirebirdSQL:⁴

CREATE TABLE projects (
  id INT NOT NULL PRIMARY KEY,
  title VARCHAR(32) NOT NULL,
  file_name VARCHAR(32) NOT NULL,
  file_location VARCHAR(256) NOT NULL,
  CONSTRAINT unique_file UNIQUE (file_name, file_location)
);
CREATE SEQUENCE project_id_sequence;
SET TERM + ;
CREATE TRIGGER project_id_sequence_update
  ACTIVE BEFORE INSERT OR UPDATE POSITION 0
  ON projects
AS
BEGIN
  IF ((new.id IS NULL) OR (new.id = 0))
    THEN new.id = NEXT VALUE FOR project_id_sequence;
END+
SET TERM ; +

PostgreSQL

CREATE TABLE projects (
  id SERIAL NOT NULL PRIMARY KEY,
  title VARCHAR(32) NOT NULL,
  file_name VARCHAR(32) NOT NULL,
  file_location VARCHAR(256) NOT NULL,
  CONSTRAINT unique_file UNIQUE (file_name, file_location)
);

It is not just that the PostgreSQL example is shorter and clearer—it is that it is shorter and clearer because its designers and developers have taken the time to make sure that the shorter, cleaner way works well, and have documented it so you can know how to use that shorter cleaner way without too much difficulty.

I do most of my development on a Mac, but do all the testing on the target platform (Windows) in a VM.↩
At this point, I would only use one of those by default if I were building a web app: Linux. I’d use nginx instead of Apache, PostgreSQL instead of MySQL, and Python (though Ruby, Javascript via node.js, C# and the .NET stack, or just about anything but PHP would do fine).↩
Almost immediately because at that point configuration on OS X was a bit of a pain. That is no longer the case.↩
To be perfectly fair to Firebird, it is improving. The upcoming 3.0 series release will make these two a lot more similar than they are at present, and clean up a number of other issues. What it won’t do is get the feel of using Firebird more like that of using Postgres, or make the installation procedure smoother or easier, or make the documentation more complete.↩

FirebirdSQL and IntelliJ IDEA (etc.)

2014-03-28T09:00:00-04:00

Setting up IntelliJ IDEA’s built-in database tools to work with FirebirdSQL requires a particular setup configuration, which I’m documenting here for public consumption.

These setup tools should be applicable to any of JetBrains’ other Java-based IDEs which include database support (e.g. PyCharm, RubyMine, WebStorm, etc.). Note: the following apply to IntelliJ IDEA 12 and the associated platforms, but not to the IDEA 13 platform, which made substantial changes to how databases are configured. The underlying details are consistent, but the interface has changed. I have tested on PyCharm 3.1 to confirm that.

This was all done on OS X 10.9, so I also make no guarantees that this works on other platforms, though the likelihood that it behaves the same on Linux is fairly good. I will update the post if and when I have confirmed that it does.

Steps to configuring a database correctly for use with IDEA/etc. Note that steps 1–3 are fairly obvious; the real point of interest is in steps 4 and 5, which took me the longest time to figure out.

Download the latest version of the Firebird Java drivers for your operating system and your Java version. (You can check your Java version by running java -version at the command line.) Extract the downloaded zip file. The extracted folder should include a file named jaybird-full-<version>.jar (<version> is currently 2.2.4).
In IDEA, in the database view, add a new data source: in the Database view (accessible via a menu button on the right side of the screen), right click and choose New -> Data Source.
Under JDBC driver files, browse to the location where you extracted the Jaybird driver files and select jaybird-full-<version>.jar.
Under JDBC driver class, choose org.firebirdsql.jdbc.FBDriver.
Under Database URL, specify jdbc:firebirdsql://localhost:3050/ followed by either the full path to the database in question or a corresponding alias.¹ A full path might look like this on Windows:
```
jdbc:firebirdsql://localhost:3050/C:/my_project/the_database.db
```
With an alias, you would instead have:
```
jdbc:firebirdsql://localhost:3050/the_alias
```
Then specify valid values for the User and Password fields from your existing configuration of the database.
Click the Test Connection button and make sure the configuration works.

That should do it. Note that the driver choice and path configuration both matter. On OS X, I found that only the FBDriver with this (and one other, older-style and therefore not recommended) path setup worked successfully.

Observations, corrections, additional information, and miscellaneous comments welcomed on App.net or Twitter.

I strongly recommend configuring an alias in the aliases.conf file in the Firebird home directory (usually set as $FIREBIRD_HOME during installation on *nix systems). This lets you move the database around at will, update just the configuration file, and not have to update any references to the database file whatsoever.↩

The End of Surfing

2014-03-26T20:00:00-04:00

Sometime in the last few months it occurred to me that I no longer “surf” the internet. I read, to be sure, and every once in a long while I even go on a spree where I follow links from one site to another (or just in a long trail on Wikipedia). In general, however, I no longer surf. I suspect I am not alone in this: if we took a straw poll I would venture that most of my friends offline and acquaintances online alike spend rather less time in “browsing” mode than they do reading Facebook or Twitter or Instagram. Motion from link to link has been replaced by individual hops out onto Buzzfeed or a viral cat picture website.¹

The obvious explanation for all of this is already there in what I’ve written: Facebook and Twitter and all the rest of the social media web. To be sure, the advent of social media and the increasing degree to which social media have captured user attention on the web are a significant factor in the end of the old surfing/browsing behavior. This is a dream come true for those social media giants which have found ways to deliver ads to their many millions of users and thereby turn enormous profits.

At the same time, I think there is an oft-overlooked factor in the shifting nature of the web over the last decade: the browser. In fact, if there is any single cause behind the death of old-fashioned surfing, I would point to Firefox 1.0: the browser which popularized tabbed browsing to increasingly large sections of the internet-using public.² The open-source browser steadily ate away at Internet Explorer’s then absurd levels of dominance, until Internet Explorer 8 included of tabs itself. By the time that Chrome came on the scene, tabbed browsing had long since become a given.

So why do I think that tabbed browsing of all things contributed to the end of “browsing” and “surfing” as our dominant mode of reading the internet? Simply put: it broke linearity. Previously,³ one’s experience of the web was single- stranded, leaping from one point to another in a line that however contorted was always connected by the forward and backward buttons on the browser. The moment tabbed browsing came on the scene, that line was broken. Following a link might mean it opened in a new tab instead of moving the whole view forward to it.

Surfing as I remember it in the late ’90s and early ’00s was inherently the experience of getting lost along that timeline, finding myself dozens of links along the chain and wondering how I had ended up there, and then being able to trace my way back. With tabs, that traceability was gone. With it went the inherent tension that we faced with every link: to follow, or not? To get sucked down into this vortex or that? Because in all likelihood, we knew, we were not going to be coming back to this page. With tabs, though, I could open both of those pages without ever leaving this one. I could start new journeys without ending the old. But there was a hidden cost: that newly opened tab had no history. It was a clean slate; before that newly opened link there was only a blank page. If I closed the original from which I had opened it, there was no going back.⁴ If I closed this new tabs, there was no going forward to them. The line was broken.

From there it was only a short step to the idea of a single site being the center from which one ventured out to other points on the web before returning: the Facebooks and Twitters of the world. In some sense, Facebook’s entire model is predicated on the idea that it is natural to open a new tab with that juicy Buzzfeed content while keeping Facebook itself open in a background tab. Would it work in that old linear model? Sort of. Would it feel natural? Never.

All of this because of tabs. Invention’s most significant results are rarely those the minds behind it expect. When we are designing things—whether a piece of furniture or a piece of the web—we have to remember that design decisions all have repercussions that we may not see. Technology is never neutral. Particular innovations may or may not be morally significant, but they always produce changes in people’s behavior. Design has consequences.

For the record, lots of that hopping from link to link was on Buzzfeed- like and viral-cat-picture-like sites, too. I am not concerned with the kind of content being read here, so much as the way it is being read.↩
Note that I am not crediting Firefox 1.0 with creating the tabbed browser—only with popularizing it. That distinction matters.↩
Excepting having multiple browser windows open, which I am sure people did—but to a much lesser extent.↩
Yes, yes, browser history and re-open closed tab commands. But the experience of those is different, and that’s what we’re talking about here.↩

Doing It Myself

2014-03-21T22:14:00-04:00

Last summer, I started work on a project I named Step Stool—aiming to make a static site generator that would tick of all the little boxes marking my desires for a website generator. In due time, the project got put on hold, as I started up classes again and needed to focus more on my family than on fun side projects.

Come the beginning of 2014, I was ready to bit WordPress farewell once and for all, though. While Ghost looks interesting, since I do all my writing in Markdown files, there is something tempting about the canonical version of the documents being the version on my computer (and thus also on my iPad and iPhone and anywhere I have Dropbox and/or Git access). I did not have time at the beginning of the year to finish writing Step Stool, and I knew as much,¹ so instead I moved to Pelican as a stop-gap. There were lots of good reasons to pick Pelican: it has an active development community, fairly thorough documentation,² and it’s in Python and uses Jinja2 templates—the same basic approach I had taken with Step Stool, and the same toolset.

Unfortunately, while I have been glad to be away from WordPress, my experience with Pelican so far has only reinforced my desire to get Step Stool done. There are lots of little things that it does in ways that just annoy me. Many of them have to do with configuration and documentation. On the latter, while the documentation is fairly complete, there are quite a few holes and gaps. (Yes, yes, open source software and anyone can add to the docs. That’s great—it really is—but if I’m going to use someone else’s solution, it had better just work. Otherwise, I’d rather spend my time getting my own going.)

For example, if you want to see how the pagination actually works, good luck figuring it out from the documentation. You’ll need to go looking at the way the sample themes (yes, both of them) are implemented to start getting a feel for it. Along the same lines, many of the objects that get handed to the templates are not fully documented, so it is difficult to know what one can or cannot do. I do not particularly want to spend my time adding debug print statements to my templates just to figure out what options I have available.

The same kinds of things hold true for configuration options. Moreover, the configuration is done through a Python module. While that makes the module easier to integrate on the code side of things, it makes its actual content much less transparent than one might hope. Python is not really well optimized for writing configuration files—nor is any normal programming language. Configuration is inherently declarative, rather than imperative.

This is not to say that Pelican is bad software. It is not. It is, however, a fairly typical example of open source software implemented by committee. It has holes (some of them serious), bumps, and quirks. Here is the reality: so will Step Stool, though they will be the quirks that come from an individual developer’s approach rather than a group’s. But the one thing I can guarantee, and the reason I am increasingly motivated to get back to working on Step Stool. And yes, I do have a couple other projects on my plate as well—contributions to the Smartypants and Typogrify modules, my own Spacewell typography project, and quite possibly a Markdown Poetry extension. But I would like very much to just get back to doing this myself. There is freedom in rolling my own solution to things. I will not always have time to do these kinds of things; I figure I should do them when I can.

So here’s to Step Stool, and—more importantly—to writing your own software just to scratch that itch.

I spent quite a bit of time tweaking my friend Vernon King’s Jekyll-powered site, I got Winning Slowly off the ground, including designing the site from scratch and implementing it (also in Pelican), and I did some substantial redesign work on this site. That was more than enough for my three week break—as evidenced by the fact that I didn’t get to the sort of 1.0 version of this site until just a week or so ago.↩
Emphasis on “fairly.” More on that in a moment as well.↩

Why Is American Internet So Slow?

2014-03-07T19:55:00-05:00

Pretty damning of the current (lack of a) regulatory regime, if you ask me:

According to a recent study by Ookla Speedtest, the U.S. ranks a shocking 31st in the world in terms of average download speeds. The leaders in the world are Hong Kong at 72.49 Mbps and Singapore on 58.84 Mbps. And America? Averaging speeds of 20.77 Mbps, it falls behind countries like Estonia, Hungary, Slovakia, and Uruguay.

Goodbye, Chrome

2014-02-24T15:20:00-05:00

Last week, Chrome crossed the line for me. I deleted it from my system to clean up its many hooks into my system—I searched out every trace of it I could find—and will put it back on my system only for testing websites. Why? Because it’s just too creepy now.

Here’s the story: two weekends ago, I was sitting at a coffee shop working on a friend’s website, when up popped a series of Google Now OS X desktop notifications from Chrome, informing me of the weather, a package having recently shipped, and so on.

There were just two problems with this:

I never gave Chrome permission to do anything of the sort.
I was not signed into Chrome or any Google service at the time.

Number 1 is bothersome. Number 2 is so far beyond bothersome that I took the nuclear option. Let’s walk through them.

Google apparently decided to start opting people into Google Now on the Chrome 33 Beta. Opting people into anything new is nearly always a bad idea in my view; opting someone into something that actively integrates with email, calendar, etc. without asking them is just creepy. Now, full disclosure: I had previously granted Google access to some of this data for Google Now on my Android phone (though I have since moved to an iPhone). However, as is usual for Google these days, the company took that permission in one context and treated it as global permission in all contexts.

No doubt the box I checked when I gave them access to that data in the first place legally allowed them to continue touching it. That did not particularly bother me. Rather, it was the assumption that I wanted the same kind of interactions from the service in a different context. This is typical of Google —typically un-human-friendly, that is. People do different things with their phones than with their browsers, and have different expectations of what each will do. More importantly, though, even if we might want our browsers to start supplying us the kinds of sometimes-valuable information that we get from Google Now (or Apple or Microsoft’s similar tools), we generally want the opportunity to make that decision. Increasingly, Google is making that decision for its users, leaving them to opt out and turn it off if they so desire. That is not a policy I particularly like. So: strike one. Or rather: strike several dozen, of the sort that had me moving away from Google’s services for quite some time— but it probably still wouldn’t have pushed me across the line to this kind of hard kill-it-with-fire mentality.

What did? That would be the part where Chrome started sending me desktop Google Now notifications. Without asking me. In a browser to which I was not logged in, nor logged into any Google services.

I will say that again to be clear: I was not signed into Chrome. I was not signed into any Google services in the browser. I had not allowed the browser to create desktop notifications. And it started sending me Google Now notifications for my main Google account. Worse: nothing I could do with the browser itself changed that behavior. (Unsurprising: there was no way Chrome should have been able to do that in the first place, logged out of all Google services as I was.)

Goodbye, Chrome. You’re just too creepy.

This was not the first time I have seen Chrome engage in behavior that does not respect its users. I have repeatedly run into cases where clearing the cache and deleting browsers… doesn’t. Cookies sometimes still stick around. Private browsing sessions inherit cookies from the main window (and sometimes vice versa). Closing a private session and launching a new one would sometimes still include cookies and cache from a previous session (bank accounts still logged in, etc.). Chrome had thus long been untrustworthy to me. But this was a bridge too far. This was not just slightly unnerving. It was creepy.

Call it a bug if you like. It is likely that it was, in fact, a bug. So, most likely, were the other cases I saw above. But these are the kinds of bugs that make a browser fundamentally untrustworthy, and they are the kinds of bugs that are that much creepier coming from a company whose profit comes almost entirely from selling advertising—that is, from selling user information to advertising companies. The deal was that we trusted Google not to abuse that information. Unfortunately, that deal just keeps getting worse all the time. (Pray they do not alter it further.)

I will have a copy of the browser on my system for testing purposes, but for nothing else. Goodbye, Chrome. And for that matter: goodbye, Google services. Over the course of the rest of this year, I will be moving myself completely off of all Google services (mail, calendar, etc.), with the sole exception of (non- logged-in) search. You’re just too creepy now.