Frontend Masters Boost RSS Feed https://frontendmasters.com/blog Helping Your Journey to Senior Developer Fri, 20 Dec 2024 17:02:04 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 225069128 Scroll-Driven & Fixed https://frontendmasters.com/blog/scroll-driven-fixed/ https://frontendmasters.com/blog/scroll-driven-fixed/#respond <![CDATA[Chris Coyier]]> Fri, 20 Dec 2024 17:00:57 +0000 <![CDATA[Blog Post]]> <![CDATA[CSS]]> <![CDATA[Scroll-Driven Animations]]> https://frontendmasters.com/blog/?p=4812 <![CDATA[It's quite fun to have an element react to another element scrolling in an unexpected way! ]]> <![CDATA[

Scroll-driven animations is a good name. They are… animations… that are… scroll-driven. As you scroll you can make something happen. The most basic kind, where a @keyframe is ran 0% to 100% as the element is scrolled 0% to 100% is particularly easy to wrap your mind around.

I also think it’s fun to mess with the expectations of scrolling.

In very light, fun-only, non-crucial ways. Not in ways that would hurt the access of content.

Like what if we made an element that definitely scrolled:

.scrolling-element {
  height: 100dvh;
  /* ... make something inside it taller than it is ... */
}

But then all the content within it was position: fixed; so it didn’t move normally when the element was scrolled.

.scrolling-element {
  height: 100dvh;
  > * {
    position: fixed;
  }
}

Instead, we could have the elements react the scroll position however we wanted.

.scrolling-element {
  scroll-timeline-name: --my-scroller;
  scroll-timeline-axis: block;
  
  > * {
    position: fixed;
    animation: doSomethingCool linear;
    animation-timeline: --my-scroller;
  }
}

@keyframes doSomethingCool {
  100% {
    rotate: 2turn;
  }
}  

Here’s that basic setup:

I bet you could imagine that this is the same exact trick for a “scroll position indicator” bit of UI. Position that <div> as like a 2px tall bar and have the scaleX transform go from 0 to 100% and donezo.

I’ll use the same spirit here to have a whole grid of cells use that “scale to zero” animation to reveal a hidden picture.

I think that hidden picture thing is fun! I’m imagining a game where you have to guess the picture by scrolling down as little as possible. Like “name that tune” only “name that movie still” or whatever.

In this next one I took the idea a bit further and create randomized positions for each of the grid cells to “fly off” to (in SCSS).

I find that extraordinary that that kind of interaction can be done in HTML and CSS these days.

]]>
https://frontendmasters.com/blog/scroll-driven-fixed/feed/ 0 4812
1 dataset. 100 visualizations. https://frontendmasters.com/blog/1-dataset-100-visualizations/ https://frontendmasters.com/blog/1-dataset-100-visualizations/#respond <![CDATA[Chris Coyier]]> Thu, 19 Dec 2024 20:15:41 +0000 <![CDATA[The Beat]]> <![CDATA[Data]]> <![CDATA[Data Viz]]> https://frontendmasters.com/blog/?p=4860 <![CDATA[Imagine this simple data set: Norway Denmark Sweden 2004 5 4 13 2002 8 10 15 Pretty simple, but there is interesting stuff going on. Someone might be trying to reference an individual bit of information, but they also might be looking to compare data, or look at the rate of change of the data […]]]> <![CDATA[

Imagine this simple data set:

NorwayDenmarkSweden
20045413
200281015

Pretty simple, but there is interesting stuff going on. Someone might be trying to reference an individual bit of information, but they also might be looking to compare data, or look at the rate of change of the data over time, all of which are things this data has.

The agency Ferdio created 100 visualizations of that data. Why so many? They can be used to emphasize different parts of the story being told with the data (and some are just… bad.)

]]>
https://frontendmasters.com/blog/1-dataset-100-visualizations/feed/ 0 4860
Introducing TanStack Start https://frontendmasters.com/blog/introducing-tanstack-start/ https://frontendmasters.com/blog/introducing-tanstack-start/#respond <![CDATA[Adam Rackis]]> Wed, 18 Dec 2024 17:43:51 +0000 <![CDATA[Blog Post]]> <![CDATA[Data]]> <![CDATA[JavaScript]]> <![CDATA[Server Side Rendering]]> <![CDATA[TanStack]]> https://frontendmasters.com/blog/?p=4810 <![CDATA[TanStack Start enhances the TanStack Router by adding a server layer that improves performance through server-side rendering (SSR) and isomorphic loaders.]]> <![CDATA[

The best way to think about TanStack Start is that it’s a thin server layer atop the TanStack Router we already know and love; that means we don’t lose a single thing from TanStack Router. Not only that, but the nature of this server layer allows it to side-step the pain points other web meta-frameworks suffer from.

This is a post I’ve been looking forward to writing for a long time; it’s also a difficult one to write.

The goal (and challenge) will be to show why a server layer on top of a JavaScript router is valuable, and why TanStack Start’s implementation is unique compared to the alternatives (in a good way). From there, showing how TanStack Start actually works will be relatively straightforward. Let’s go!

Please keep in mind that, while this post discusses a lot of generic web performance issues, TanStack Start is still a React-specific meta-framework. It’s not a framework-agnostic tool like Astro

Why Server Rendering?

Client-rendered web applications, often called “Single Page Applications” or “SPAs” have been popular for a long time. With this type of app, the server sends down a mostly empty HTML page, possibly with some sort of splash image, loading spinner, or maybe some navigation components. It also includes, very importantly, script tags that load your framework of choice (React, Vue, Svelte, etc) and a bundle of your application code.

These apps were always fun to build, and in spite of the hate they often get, they (usually) worked just fine (any kind of software can be bad). Admittedly, they suffer a big disadvantage: initial render performance. Remember, the initial render of the page was just an empty shell of your app. This displayed while your script files loaded and executed, and once those scripts were run, your application code would most likely need to request data before your actual app could display. Under the covers, your app is doing something like this

The initial render of the page, from the web server, renders only an empty shell of your application. Then some scripts are requested, and then parsed and executed. When those application scripts run, you (likely) send some other requests for data. Once that is done, your page displays.

To put it more succinctly, with client-rendered web apps, when the user first loads your app, they’ll just get a loading spinner. Maybe your company’s logo above it, if they’re lucky.

This is perhaps an overstatement. Users may not even notice the delay caused by these scripts loading (which are likely cached), or hydration, which is probably fast. Depending on the speed of their network, and the type of application, this stuff might not matter much.

Maybe.

But if our tools now make it easy to do better, why not do better?

Server Side Rendering

With SSR, the picture looks more like this

The server sends down the complete, finished page that the user can see immediately. We do still need to load our scripts and hydrate, so our page can be interactive. But that’s usually fast, and the user will still have content to see while that happens.

Our hypothetical user now looks like this, since the server is responding with a full page the user can see.

Streaming

We made one implicit assumption above: that our data was fast. If our data was slow to load, our server would be slow to respond. It’s bad for the user to be stuck looking at a loading spinner, but it’s even worse for the user to be stuck looking at a blank screen while the server churns.

As a solution for this, we can use something called “streaming,” or “out of order streaming” to be more precise. The user still requests all the data, as before, but we tell our server “don’t wait for this/these data, which are slow: render everything else, now, and send that slow data to the browser when it’s ready.”

All modern meta-frameworks support this, and our picture now looks like this

To put a finer point on it, the server does still initiate the request for our slow data immediately, on the server during our initial navigation. It just doesn’t block the initial render, and instead pushes down the data when ready. We’ll look at streaming with Start later in this post.

Why did we ever do client-rendering?

I’m not here to tear down client-rendered apps. They were, and frankly still are an incredible way to ship deeply interactive user experiences with JavaScript frameworks like React and Vue. The fact of the matter is, server rendering a web app built with React was tricky to get right. You not only needed to server render and send down the HTML for the page the user requested, but also send down the data for that page, and hydrate everything just right on the client.

It’s hard to get right. But here’s the thing: getting this right is the one of the primary purposes of this new generation of meta-frameworks. Next, Nuxt, Remix, SvelteKit, and SolidStart are some of the more famous examples of these meta-frameworks. And now TanStack Start.

Why is TanStack Start different?

Why do we need a new meta-framework? There’s many possible answers to that question, but I’ll give mine. Existing meta-frameworks suffer from some variation on the same issue. They’ll provide some mechanism to load data on the server. This mechanism is often called a “loader,” or in the case of Next, it’s just RSCs (React Server Components). In Next’s (older) pages directory, it’s the getServerSideProps function. The specifics don’t matter. What matters is, for each route, whether the initial load of the page, or client-side navigation via links, some server-side code will run, send down the data, and then render the new page.

Need to bone up on React in general? Brian Holt’s Complete Intro to React and Intermediate React will get you there.

An Impedance Mismatch is Born

Notice the two worlds that exist: the server, where data loading code will always run, and the client. It’s the difference and separation between these worlds that can cause issues.

For example, frameworks always provide some mechanism to mutate data, and then re-fetch to show updated state. Imagine your loader for a page loads some tasks, user settings, and announcements. When the user edits a task, and revalidates, these frameworks will almost always re-run the entire loader, and superfluously re-load the user’s announcements and user settings, in addition to tasks, even though tasks are the only thing that changed.

Are there fixes? Of course. Many frameworks will allow you to create extra loaders to spread the data loading across, and revalidate only some of them. Other frameworks encourage you to cache your data. These solutions all work, but come with their own tradeoffs. And remember, they’re solutions to a problem that meta-frameworks created, by having server-side loading code for every path in your app.

Or what about a loader that loads 5 different pieces of data? After the page loads, the user starts browsing around, occasionally coming back to that first page. These frameworks will usually cache that previously-displayed page, for a time. Or not. But it’s all or none. When the loader re-runs, all 5 pieces of data will re-fire, even if 4 of them can be cached safely.

You might think using a component-level data loading solution like react-query can help. react-query is great, but it doesn’t eliminate these problems. If you have two different pages that each have 5 data sources, of which 4 are shared in common, browsing from the first page to the second will cause the second page to re-request all 5 pieces of data, even though 4 of them are already present in client-side state from the first page. The server is unaware of what happens to exist on the client. The server is not keeping track of what state you have in your browser; in fact the “server” might just be a Lambda function that spins up, satisfies your request, and then dies off.

In the picture, we can see a loader from the server sending down data for queryB, which we already have in our TanStack cache.

Where to, from here?

The root problem is that these meta-frameworks inevitably have server-only code running on each path, integrating with long-running client-side state. This leads to conflicts and inefficiencies which need to be managed. There’s ways of handling these things, which I touched on above. But it’s not a completely clean fit.

How much does it matter?

Let’s be clear right away: if this situation is killing performance of your site, you have bigger problems. If these extra calls are putting undue strain on your services, you have bigger problems.

That said, one of the first rules of distributed systems is to never trust your network. The more of these calls we’re firing off, the better the chances that some of them might randomly be slow for some reason beyond our control. Or fail.

We typically tolerate requesting more than we need in these scenarios because it’s hard to avoid with our current tooling. But I’m here to show you some new, better tooling that side-steps these issues altogether.

Isomorphic Loaders

In TanStack, we do have loaders. These are defined by TanStack Router. I wrote a three-part series on Router here. If you haven’t read that, and aren’t familiar with Router, give it a quick look.

Start takes what we already have with Router, and adds server handling to it. On the initial load, your loader will run on the server, load your data, and send it down. On all subsequent client-side navigations, your loader will run on the client, like it already does. That means all subsequent invocations of your loader will run on the client, and have access to any client-side state, cache, etc. If you like react-query, you’ll be happy to know that’s integrated too. Your react-query client can run on the server, to load, and send data down on the initial page load. On subsequent navigations, these loaders will run on the client, which means your react-query queryClient will have full access to the usual client-side cache react-query always uses. That means it will know what does, and does not need to be loaded.

It’s honestly such a refreshing, simple, and most importantly, effective pattern that it’s hard not being annoyed none of the other frameworks thought of it first. Admittedly, SvelteKit does have universal loaders which are isomorphic in the same way, but without a component-level query library like react-query integrated with the server.

TanStack Start

Enough setup, let’s look at some code. TanStack Start is still in beta, so some of the setup is still a bit manual, for now.

The repo for this post is here.

If you’d like to set something up yourself, check out the getting started guide. If you’d like to use react-query, be sure to add the library for that. You can see an example here. Depending on when you read this, there might be a CLI to do all of this for you.

This post will continue to use the same code I used in my prior posts on TanStack Router. I set up a new Start project, copied over all the route code, and tweaked a few import paths since the default Start project has a slightly different folder structure. I also removed all of the artificial delays, unless otherwise noted. I want our data to be fast by default, and slow in a few places where we’ll use streaming to manage the slowness.

We’re not building anything new, here. We’re taking existing code, and moving the data loading up to the server in order to get it requested sooner, and improve our page load times. This means everything we already know and love about TanStack Router is still 100% valid.

Start does not replace Router; Start improves Router.

Loading Data

All of the routes and loaders we set up with Router are still valid. Start sits on top of Router and adds server processing. Our loaders will execute on the server for the first load of the page, and then on the client as the user browses. But there’s a small problem. While the server environment these loaders will execute in does indeed have a fetch function, there are differences between client-side fetch, and server-side fetch—for example, cookies, and fetching to relative paths.

To solve this, Start lets you define a server function. Server functions can be called from the client, or from the server; but the server function itself always executes on the server. You can define a server function in the same file as your route, or in a separate file; if you do the former, TanStack will do the work of ensuring that server-only code does not ever exist in your client bundle.

Let’s define a server function to load our tasks, and then call it from the tasks loader.

import { getCookie } from "vinxi/http";
import { createServerFn } from "@tanstack/start";
import { Task } from "../../types";

export const getTasksList = createServerFn({ method: "GET" }).handler(async () => {
  const result = getCookie("user");

  return fetch(`http://localhost:3000/api/tasks`, { method: "GET", headers: { Cookie: "user=" + result } })
    .then(resp => resp.json())
    .then(res => res as Task[]);
});

We have access to a getCookie utility from the vinxi library on which Start is built. Server functions actually provide a lot more functionality than this simple example shows. Be sure to check out the docs to learn more.

If you’re curious about this fetch call:

fetch(`http://localhost:3000/api/tasks`, { method: "GET", headers: { Cookie: "user=" + result } });

That’s how I’m loading data for this project, on the server. I have a separate project running a set of Express endpoints querying a simple SQLite database. You can fetch your data however you need from within these server functions, be it via an ORM like Drizzle, an external service endpoint like I have here, or you could connect right to a database and query what you need. But that latter option should probably be discouraged for production applications.

Now we can call our server function from our loader.

loader: async ({ context }) => {
    const now = +new Date();
    console.log(`/tasks/index path loader. Loading tasks at + ${now - context.timestarted}ms since start`);
    const tasks = await getTasksList();
    return { tasks };
  },

That’s all there is to it. It’s almost anti-climactic. The page loads, as it did in the last post. Except now it server renders. You can shut JavaScript off, and the page will still load and display (and hyperlinks will still work).

Streaming

Let’s make the individual task loading purposefully slow (we’ll just keep the delay that was already in there), so we can see how to stream it in. Here’s our server function to load a single task.

export const getTask = createServerFn({ method: "GET" })
  .validator((id: string) => id)
  .handler(async ({ data }) => {
    return fetch(`http://localhost:3000/api/tasks/${data}`, { method: "GET" })
      .then(resp => resp.json())
      .then(res => res as Task);
  });

Note the validator function, which is how we strongly type our server function (and validate the inputs). But otherwise it’s more of the same.

Now let’s call it in our loader, and see about enabling streaming

Here’s our loader:

loader: async ({ params, context }) => {
    const { taskId } = params;

    const now = +new Date();
    console.log(`/tasks/${taskId} path loader. Loading at + ${now - context.timestarted}ms since start`);
    const task = getTask({ data: taskId });

    return { task };
  },

Did you catch it? We called getTask without awaiting it. That means task is a promise, which Start and Router allow us to return from our loader (you could name it taskPromise if you like that specificity in naming).

But how do we consume this promise, show loading state, and await the real value? There are two ways. TanStack Router defines an Await component for this. But if you’re using React 19, you can use the new use psuedo-hook.

import { use } from "react";

function TaskView() {
  const { task: taskPromise } = Route.useLoaderData();
  const { isFetching } = Route.useMatch();

  const task = use(taskPromise);

  return (
    <div>
      <Link to="/app/tasks">Back to tasks list</Link>
      <div className="flex flex-col gap-2">
        <div>
          Task {task.id} {isFetching ? "Loading ..." : null}
        </div>
        <h1>{task.title}</h1>
        <Link 
          params={{ taskId: task.id }}
          to="/app/tasks/$taskId/edit"
        >
          Edit
        </Link>
        <div />
      </div>
    </div>
  );
}

The use hook will cause the component to suspend, and show the nearest Suspense boundary in the tree. Fortunately, the pendingComponent you set up in Router also doubles as a Suspense boundary. TanStack is impressively well integrated with modern React features.

Now when we load an individual task’s page, we’ll first see the overview data which loaded quickly, and server rendered, above the Suspense boundary for the task data we’re streaming

When the task comes in, the promise will resolve, the server will push the data down, and our use call will provide data for our component.

React Query

As before, let’s integrate react-query. And, as before, there’s not much to do. Since we added the @tanstack/react-router-with-query package when we got started, our queryClient will be available on the server, and will sync up with the queryClient on the client, and put data (or in-flight streamed promises) into cache.

Let’s start with our main epics page. Our loader looked like this before:

async loader({ context, deps }) {
    const queryClient = context.queryClient;

    queryClient.ensureQueryData(
      epicsQueryOptions(context.timestarted, deps.page)
    );
    queryClient.ensureQueryData(
      epicsCountQueryOptions(context.timestarted)
    );
  }

That would kick off the requests on the server, but let the page render, and then suspend in the component that called useSuspenseQuery—what we’ve been calling streaming.

Let’s change it to actually load our data in our loader, and server render the page instead. The change couldn’t be simpler.

async loader({ context, deps }) {
  const queryClient = context.queryClient;

  await Promise.allSettled([
    queryClient.ensureQueryData(
      epicsQueryOptions(context.timestarted, deps.page)
    ),
    queryClient.ensureQueryData(
      epicsCountQueryOptions(context.timestarted)
    ),
  ]);
},

Note we’re awaiting a Promise.allSettled call here so the queries can run together. Make sure you don’t sequentially await each individual call, as that would create a waterfall, or use Promise.all, as that will quit immediately if any of the promises error out.

Streaming with react-query

As I implied above, to stream data with react-query, do the exact same thing, but don’t await the promise. Let’s do that on the page for viewing an individual epic.

loader: ({ context, params }) => {
  const { queryClient, timestarted } = context;

  queryClient.ensureQueryData(
    epicQueryOptions(timestarted, params.epicId)
  );
},

Now if this page is loaded initially, the query for this data will start on the server and stream to the client. If the data are pending, our suspense boundary will show, triggered automatically by react-query’s useSuspenseBoundary hook.

If the user browses to this page from a different page, the loader will instead run on the client, but still fetch those same data from the same server function, and trigger the same suspense boundary.

Parting Thoughts

I hope this post was useful to you. It wasn’t a deep dive into TanStack Start — the docs are a better venue for that. Instead, I hope I was able to show why server rendering can offer almost any web app a performance boost, and why TanStack Start is a superb tool for doing so. Not only does it simplify a great deal of things by running loaders isomorphically, but it even integrates wonderfully with react-query.

The react-query integration is especially exciting to me. It delivers component-level data fetching while still allowing for server fetching, and streaming—all without sacrificing one bit of convenience.

]]>
https://frontendmasters.com/blog/introducing-tanstack-start/feed/ 0 4810
Calibre Website Speed Test https://frontendmasters.com/blog/calibre-website-speed-test/ https://frontendmasters.com/blog/calibre-website-speed-test/#respond <![CDATA[Chris Coyier]]> Tue, 17 Dec 2024 20:39:31 +0000 <![CDATA[The Beat]]> <![CDATA[Performance]]> https://frontendmasters.com/blog/?p=4832 <![CDATA[Calibre, the website performance testing app, launched a one-off Website Speed Test page anyone can use for free. This is nice it requires no special knowledge to use (anyone can type in a URL), the test can be run from significant geolocations (that probably aren’t where you live), and the test result is saved and […]]]> <![CDATA[

Calibre, the website performance testing app, launched a one-off Website Speed Test page anyone can use for free. This is nice it requires no special knowledge to use (anyone can type in a URL), the test can be run from significant geolocations (that probably aren’t where you live), and the test result is saved and can be shared.

webpagetest.org is the classic in this space. Both are trying to sell you more advanced services. Just a matter of which you find more useful.

]]>
https://frontendmasters.com/blog/calibre-website-speed-test/feed/ 0 4832
React 19 and Web Component Examples https://frontendmasters.com/blog/react-19-and-web-component-examples/ https://frontendmasters.com/blog/react-19-and-web-component-examples/#respond <![CDATA[Chris Coyier]]> Mon, 16 Dec 2024 16:37:38 +0000 <![CDATA[Blog Post]]> <![CDATA[JavaScript]]> <![CDATA[React]]> <![CDATA[Web Components]]> https://frontendmasters.com/blog/?p=4800 <![CDATA["... props that match a property on the Custom Element instance will be assigned as properties, otherwise they will be assigned as attributes."]]> <![CDATA[

There is lots of news of React 19 and going stable and now supporting Web Components. Or… “custom elements” I should say, as that refers to the HTML expression of them as <dashed-elements>, which is where the trouble laid. Apparently it was hard for React to know, in JSX, if props should be treated as a property or an attribute. So they’ve just decided this is how it will work:

  • Server Side Rendering: props passed to a custom element will render as attributes if their type is a primitive value like stringnumber, or the value is true. Props with non-primitive types like objectsymbolfunction, or value false will be omitted.
  • Client Side Rendering: props that match a property on the Custom Element instance will be assigned as properties, otherwise they will be assigned as attributes.

That’s enough to pass all the tests at Custom Elements Everywhere, which tracks such things. (And apparently every single framework is now 100% fine. Cool.)

This got me thinking about what this actually means and how I might make use of it. I use both React and Web Components sometimes, but only rarely do I combine them, and the last time I did I had more issues with the Shadow DOM than I did with React doing anything funky.

So here I’ve made a LitElement and rendered it within a React component:

What I was thinking there was… what if I make a <designsystem-button> and need a click handler on it? Turns out that’s not really a problem. You can just slap a React onClick right on it and it’s fine.

<MyCard>
  <p>blah blah</p>
  <!-- this is fine -->
  <designsystem-button onClick={() => {}}></designsystem-button>
</MyCard>

That wasn’t a problem anyway, apparently.

What is a problem is if I want to send in a function from React-land for the Web Component to call. You’d think we could send the function in how LitElement generally wants you to do that like:

<!-- nope -->
<designsystem-button .mySpecialEvent=${specialEvent}>

But nope, JSX really doesn’t like that “dot syntax” and won’t compile.

So you gotta send it in more like this:

<designsystem-button onSpecialEvent={() => mySpecialEvent()}

Then in order to “call” that event, you “dispatch” and event named properly. Like:

this.dispatchEvent(new CustomEvent("SpecialEvent", { bubbles: true }));

Here’s that with a “raw” Web Component:

I took that idea from Jared White’s article Oh Happy Day! React Finally Speaks Web Components. Where he’s got some other examples. Another is passing in a “complex object” which is one of those things that would have been impossible in React 18 and under apparently, and now we can do:

]]>
https://frontendmasters.com/blog/react-19-and-web-component-examples/feed/ 0 4800
Anchoreum https://frontendmasters.com/blog/anchoreum/ https://frontendmasters.com/blog/anchoreum/#respond <![CDATA[Chris Coyier]]> Fri, 13 Dec 2024 17:57:31 +0000 <![CDATA[The Beat]]> <![CDATA[Anchor]]> <![CDATA[CSS]]> <![CDATA[Game]]> https://frontendmasters.com/blog/?p=4802 <![CDATA[A “game” where you enter the right CSS to move to the next level, teaching you CSS anchor positioning along the way. In the vein of Flexbox Froggy, which I know clicked with a ton of people. Made by Thomas Park, who’s company Codepip actually makes a ton of these experiences.]]> <![CDATA[

A “game” where you enter the right CSS to move to the next level, teaching you CSS anchor positioning along the way.

In the vein of Flexbox Froggy, which I know clicked with a ton of people. Made by Thomas Park, who’s company Codepip actually makes a ton of these experiences.

]]>
https://frontendmasters.com/blog/anchoreum/feed/ 0 4802
Introducing Fly.io https://frontendmasters.com/blog/introducing-fly-io/ https://frontendmasters.com/blog/introducing-fly-io/#comments <![CDATA[Adam Rackis]]> Thu, 12 Dec 2024 15:18:54 +0000 <![CDATA[Blog Post]]> <![CDATA[Fly.io]]> <![CDATA[Hosting]]> https://frontendmasters.com/blog/?p=4742 <![CDATA[If it can go in a Docker, Fly can host it, and they'll help you with that. Adam Rackis takes a look at the platform and shows off all the things he likes about it.]]> <![CDATA[

Fly.io is an increasingly popular infrastructure platform. Fly is a place to deploy your applications, similar to Vercel or Netlify, but with some different tradeoffs.

This post will introduce the platform, show how to deploy web apps, stand up databases, and some other fun things. If you leave here wanting to learn more, the docs are here and are outstanding.

What is Fly?

Where platforms like Vercel and Netlify run your app on serverless functions which spin up and die off as needed (typically running on AWS Lambda), Fly runs your machines on actual VM’s, running in their infrastructure. These VMs can be configured to scale up as your app’s traffic grows, just like with serverless functions. But as the continuously run, there is no cold start issues. That said, if you’re on a budget, or your app isn’t that important (or both) you can also configure Fly to scale your app down to zero machines when traffic dies. You’ll be billed essentially nothing during those periods of inactivity, though your users will see a cold start time if they’re the first to hit your app during an inactive period.

To be perfectly frank, the cold start problem has been historically exaggerated, so please don’t pick a platform just to avoid cold starts.

Why VMs?

You might be wondering why, if cold starts aren’t a big deal in practice, one should care about Fly using VMs instead of cloud functions. For me there’s two reasons: the ability to execute long-running processes, and the ability to run anything that will run in a Docker image. Let’s dive into both.

The ability to handle long-running processes greatly expands the range of apps Fly can run. They have turn-key solutions for Phoenix LiveView, Laravel, Django, Postgres, and lots more. Anything you ship on Fly will be via a Dockerfile (don’t worry, they’ll help you generate them). That means anything you can put into a Dockerfile, can be run by Fly. If there’s a niche database you’ve been wanting to try (Neo4J, CouchDB, etc), just stand one up via a Dockerfile (and both of those DBs have official images), and you’re good to go. New databases, new languages, new anything: if there’s something you’ve been wanting to try, you can run it on Fly if you can containerize it; and anything can be containerized.

But… I don’t know Docker

Don’t worry, Fly will, as you’re about to see, help you scaffold a Dockerfile from any common app framework. We’ll take a quick look at what’s generated, and explain the high points.

That said, Docker is one of the most valuable tools for a new engineer to get familiar with, so if Fly motivates you to learn more, so much the better!

If you’d like to go deeper on Docker, our course Complete Intro to Containers from Brian Holt is fantastic.

Let’s launch an app!

Let’s ship something. We’ll create a brand new Next.js app, using the standard scaffolding here.

We’ll create an app, run npm i and then npm run dev and verify that it works.

screenshot of a running Next.js app

Now let’s deploy it to Fly. If you haven’t already, install the Fly CLI, and sign up for an account. Instructions can be found in the first few steps of the quick start guide.

To deploy an app on Fly, you need to containerize your app. We could manually piece together a valid Dockerfile that would run our Next app, and then run fly deploy. But that’s a tedious process. Thankfully Fly has made life easier for us. Instead, we can just run fly launch from our app’s root directory.

Fly easily detected Next.js, and then made some best guesses as to deployment settings. It opted for the third cheapest deployment option. Here’s Fly’s full pricing information. Fly let’s us accept these defaults, or tweak them. Let’s hit yes to tweak. We should be taken to the fly.io site, where our app is in the process of being set up.

For fun, let’s switch to the cheapest option, and change the region to Virginia (what AWS would call us-east-1).

Hit confirm, and return to your command line. It should finish setting everything up, which should look like this, in part.

If we head over to our Fly dashboard, we should see something like this:

We can then click that app and see the app’s details

And lastly, we can go to the URL listed, and see the app actually running!

Looking closer

There’s a number of files that Fly created for us. The two most important are the Dockerfile, and fly.toml. Let’s take a look at each. We’ll start with the Dockerfile.

# syntax = docker/dockerfile:1

# Adjust NODE_VERSION as desired
ARG NODE_VERSION=20.18.1
FROM node:${NODE_VERSION}-slim as base

LABEL fly_launch_runtime="Next.js"

# Next.js app lives here
WORKDIR /app

# Set production environment
ENV NODE_ENV="production"

# Throw-away build stage to reduce size of final image
FROM base as build

# Install packages needed to build node modules
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential node-gyp pkg-config python-is-python3

# Install node modules
COPY package-lock.json package.json ./
RUN npm ci --include=dev

# Copy application code
COPY . .

# Build application
RUN npm run build

# Remove development dependencies
RUN npm prune --omit=dev


# Final stage for app image
FROM base

# Copy built application
COPY --from=build /app /app

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD [ "npm", "run", "start" ]

A Quick Detour to Understand Docker

Docker is a book unto its own, but as an extremely quick intro: Docker allows us to package our app into an “image.” Containers allow you to start with an entire operating system (almost always a minimal Linux distro), and allow you to do whatever you want with it. Docker then packages whatever you create, and allows it to be run. The Docker image is completely self-contained. You choose the whatever goes into it, from the base operating system, down to whatever you install into the image. Again, they’re self-contained.

Now let’s take a quick tour of the important pieces of our Dockerfile.

After some comments and labels, we find what will always be present at the top of a Dockerfile: the FROM command.

FROM node:${NODE_VERSION}-slim as base

This tells us the base of the image. We could start with any random Linux distro, and then install Node and npm, but unsurprisingly there’s already an officially maintained Node image: there will almost always be officially maintained Docker images for almost any technology. In fact, there’s many different Node images to choose from, many with different underlying base Linux distro’s.

There’s a LABEL that’s added, likely for use with Fly. Then we set the working directory in our image.

WORKDIR /app

We copy the package.json and lockfiles.

# Install node modules
COPY package-lock.json package.json ./

Then run npm i (but in our Docker image):

RUN npm ci --include=dev

Then we copy the rest of the application code:

# Copy application code
COPY . .

Hopefully you get the point. We won’t go over every line, here. But hopefully the general idea is clear enough, and hopefully you’d feel comfortable tweaking this if you wanted to. Two last points though. See this part:

# Install packages needed to build node modules
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential node-gyp pkg-config python-is-python3

That tells the Linux package manager to install some things Fly thinks Next might need, but in actuality probably doesn’t. Don’t be surprised if these lines are absent when you read this, and try for yourself.

Lastly, if you were wondering why the package.json and lockfiles were copied, followed by npm install and then followed by copying the rest of the application code, the reason is (Docker) performance. Briefly, each line in the Dockerfile creates a “layer.” These layers can be cached and re-used if nothing has changed. If anything has changed, that invalidates the cache for that layer, and also all layers after it. So you’ll want to push your likely-to-change work as low as possible. Your application code will almost always change between deployments; the dependencies in your package.json will change much less frequently. So we do that install first, by itself, so it will be more likely to be cached, and speed up our builds.

I tried my best to provide the absolute minimal amount of a Docker intro to make this post make sense, without being overhwelming. I hope I’ve succeeded. If you’d like to learn more, there’s tons of books and YouTube videos, and even an entire course here on Frontend Masters.

Fly.toml

Now let’s take a peek at the fly.toml file.

# fly.toml app configuration file generated for next-fly-test on 2024-11-28T19:04:19-06:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = 'next-fly-test'
primary_region = 'iad'

[build]

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[vm]]
  size = 'shared-cpu-1x'

This is basically the config file for the Fly app. The options for this file are almost endless, and are documented here. The three most important lines are the next three.

auto_stop_machines = 'stop'

This tells Fly to automatically kill machines when they’re not needed, when traffic is low on our app.

auto_start_machines = true

The line above tells Fly to automatically spin up new machines when it detects it needs to do so, given your traffic. Lastly, this line

min_machines_running = 0

That line allows us to tell Fly to always keep a minimum number of machines running, no matter how minimal your current traffic is. Setting it to zero allows for no machines to be running, which means your next visitor will see a slow response as the first machine spins up.

You may have noticed above that Fly spun up two machines initially, even though there was no traffic at all. It does this by default to give your app a higher availability, that is, in case anything happens to the one machine, the other will (hopefully) still be up and running. If you don’t want or need this, you can prevent it by passing --ha=false when you run fly launch or fly deploy (or you can just kill one of the machines in the dashboard – Fly will not re-create it on subsequent deploys).

Machines won’t bill you if they’re not running

When a machine is not running, you’ll be billed essentially zero for it. You’ll just pay $0.15 per GB, per month, per machine (machines will usually have only one GB).

Adding a database

You can launch a Fly app anytime with just a Dockerfile. You could absolutely find an official Postgres Docker image and deploy from that. But it turns out Fly has this built in. Let’s run fly postgres create in a terminal, and see what happens

It’ll ask you for a name and a region, and then how serious of a Postgres setup you want. Once it’s done, it’ll show you something like this.

Fly postgres create

The connection string listed at the bottom can be used to connect to your db from within another Fly app (which you own). But to run database creation and migration scripts, and for local development you’ll need to connect to this db on your local machine. To do that, you can run this:

fly proxy 5432 -a <your app name>

Now you can connect via the same connection string on your local machine, but on localhost:5432 instead of flycast:5432.

Making your database publicly available

It’s not ideal, but if you want to make your Fly pg box publicly available, you can. You basically have to add a dedicated ipv4 address to it (at a cost of $2 per month), and then tweak your config.

Consider using a dedicated host for serious applications.

Fly’s built-in Postgres support is superb, but there’s some things you’ll have to manage yourself. If that’s not for you, Supabase is a fully managed pg host, and it’s also superb. Fly even has a service for creating Supabase db’s on Fly infra, for extra low latency. It’s currently only in public alpha, but it might be worth keeping an eye on.

Interlude

If you just want a nice place to deploy your apps, what we’ve covered will suffice for the vast majority of use cases. I could stop this post here, but I’d be remiss if I didn’t show some of the cooler things you can do with Fly. Please don’t let what follows be indicative of the complexity you’ll normally deal with. We’ll be putting together a cron job for running Postgres backups. In practice, you’ll just use a mature DB provider like Supabase or PlanetScale, which will handle things like this for you.

But sometimes it’s fun to tinker, especially for side projects. So let’s kick the tires a bit and see what we can come up with.

Having Fun

One of Fly’s greatest strengths is its flexibility. You give it a Dockerfile, and it’ll run it. To drive that point home, let’s conclude this post with a fun example.

As much as I love Fly, it makes me a little uneasy that my database is running isolated in some VM under my account. Accidents happen, and I’d want automatic backups. Why don’t we build a Docker image to do just that?

I’ll want to run a script, written in TypeScript, preferably without hating my life: Bun is ideal for this. I’ll also need to run the actual pg_dump command. So what should I build my Dockerfile from: the bun image, which would lack to pg utilities, or the pg base, which wouldn’t have bun installed. I could do either, and use the Linux package manager to install what I need. But really, there’s a simpler way: use a multi-stage Docker build. Let’s see the whole Dockerfile

FROM oven/bun:latest AS BUILDER

WORKDIR /app

COPY . .

RUN ["bun", "install"]
RUN ["bun", "build", "index.ts", "--compile", "--outfile", "run-pg_dump"]

FROM postgres:16.4

WORKDIR /app
COPY --from=BUILDER /app/run-pg_dump .
COPY --from=BUILDER /app/run-backup.sh .

RUN chmod +x ./run-backup.sh

CMD ["./run-backup.sh"]

We start with a Bun image. We run a bun install to tell Bun to install what we need: aws sdk’s and such. Then we tell Bun to compile our script into a standalone executable: yes, Bun can do that, and yes: it’s that easy.

FROM postgres:16.4

Tells Docker to start a new stage, from a new (Postgres) base.

WORKDIR /app
COPY --from=BUILDER /app/run-pg_dump .
COPY --from=BUILDER /app/run-backup.sh .

RUN chmod +x ./run-backup.sh

CMD ["./run-backup.sh"]

This drops into the /app folder from the prior step, and copies over the run-pg_dump file, which Bun compiled for us, and also copies over run-backup.sh. This is a shell script I wrote. It runs pg_dump a few times, to generate the files the Bun script (run-pg_dump) is expecting, and then calls it. Here’s what that file looks like:

<strong>#!/bin/sh</strong>

PG_URI_CLEANED=$(echo ${PG_URI} | sed -e 's/^"//' -e 's/"$//')

pg_dump ${PG_URI_CLEANED} -Fc > ./backup.dump

pg_dump ${PG_URI_CLEANED} -f ./backup.sql

./run-pg_dump

This unhinged line:

PG_URI_CLEANED=$(echo ${PG_URI} | sed -e 's/^"//' -e 's/"$//')

is something ChatGPT helped me write, to strip the double quotes from my connection string environment variable.

Lastly, if you’re curious about the index.ts file Bun compiled into a standalone executable, this is it:

import fs from "fs";
import path from "path";

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const numToDisplay = (num: number) => num.toString().padStart(2, "0");

const today = new Date();
const date = `${today.getFullYear()}/${numToDisplay(today.getMonth() + 1)}/${numToDisplay(today.getDate())}`;
const time = `${today.getHours()}-${numToDisplay(today.getMinutes())}-${numToDisplay(today.getSeconds())}`;
const filename = `${date}/${time}`;

const REGION = "us-east-1";
const dumpParams = {
  Bucket: "my-library-backups",
  Key: `${filename}.dump`,
  Body: fs.readFileSync(path.resolve(__dirname, "backup.dump")),
};
const sqlParams = {
  Bucket: "my-library-backups",
  Key: `${filename}.sql`,
  Body: fs.readFileSync(path.resolve(__dirname, "backup.sql")),
};

const s3 = new S3Client({
  region: REGION,
  credentials: {
    accessKeyId: process.env.AWS_ID!,
    secretAccessKey: process.env.AWS_SECRET!,
  },
});

s3.send(new PutObjectCommand(sqlParams))
  .then(() => {
    console.log("SQL Backup Uploaded!");
  })
  .catch(err => {
    console.log("Error: ", err);
  });

s3.send(new PutObjectCommand(dumpParams))
  .then(() => {
    console.log("Dump Backup Uploaded!");
  })
  .catch(err => {
    console.log("Error: ", err);
  });

I’m sure someone who’s actually good with Docker could come up with something better, but this works well enough.

To see this whole thing all together, in one place, you can see it in my GitHub.

Scheduling a custom job

We have a working, valid Docker image. How do we tell Fly to run it on an interval? Fly has a command just for that: fly machine run. In fact, it can take a schedule argument, to have Fly run it on an interval. Unfortunately, the options are horribly limited: only hourly, daily, and monthly. But, as a workaround you can run this command at different times: this will set up executions at whatever interval you selected, scheduled off of when you ran the command.

fly machine run . --schedule=daily

If you ran that command at noon, that will schedule a daily task that runs at noon every day. If you run that command again at 5pm, it will schedule a second task to run daily, at 5pm (without interfering with the first). Each job will have a dedicated machine, but will be idle when not running, which means it will cost you almost nothing; you’ll pay the normal $0.15 per month, per GB on the machine.

I hate this limitation in scheduling machines. In theory there’s a true cron job template here, but it’s not the simplest thing to look through.

Odds and ends

That was a lot. Let’s lighten things up a bit with some happy odds and ends, before we wrap up.

Custom domains

Fly makes it easy to add a custom domain to your app. You’ll just need to add the right records. Full instructions are here.

Secrets

You’ll probably have some secrets you want run in your app, in production. If you’re thinking you could just bundle a .env.prod file into your Docker image, yes, you could. But that’s considered a bad idea. Instead, leverage Fly’s secret management.

Learning More

This post started brushing up against some full-stack topics. If this sparked your interest, be sure to check out the entire course on full-stack engineering here on Frontend Masters.

Wrapping Up

The truth is we’ve truly, barely scratched the surface of Fly. For simple side projects what we’ve covered here is probably more than you’d need. But Fly also has power tools available for advanced use cases. The sky’s the limit!

Fly.io is a wonderful platform. It’s fun to work with, will scale to your application’s changing load, and is incredibly flexible. I urge you to give it a try for your next project.

]]>
https://frontendmasters.com/blog/introducing-fly-io/feed/ 1 4742
Responsive Tables & Readable Paragraphs https://frontendmasters.com/blog/responsive-tables-readable-paragraphs/ https://frontendmasters.com/blog/responsive-tables-readable-paragraphs/#comments <![CDATA[Chris Coyier]]> Wed, 11 Dec 2024 19:01:17 +0000 <![CDATA[Blog Post]]> <![CDATA[CSS]]> <![CDATA[Responsive Design]]> <![CDATA[Table]]> https://frontendmasters.com/blog/?p=4725 <![CDATA[You can make a table responsive by letting it horizontally scroll. But if you do that, make sure any paragraph style text isn't any wider than the screen.]]> <![CDATA[

I have this habit where when I’m watching a TV show where I like to read about it on my phone while I’m watching it. Reviews, summaries, fan theories, whatever. Seems like that would be distracting — but I think it’s extra engaging sometimes. I’d often end up on Wikipedia where they do episode informational summaries in a particular layout where the small screen layout had a horribly obnoxious side effect.

Here (was) the issue:

To their credit, they made the data table responsive in that it’s not zoomed out or cut off, you can horizontally scroll it. But horizontal scrolling is super terrible when you’re trying to read a paragraph of text.

Also to their credit, they’ve also (recently?) made this a bit better.

They put a wrapper element over the show description and added max-width: 90vw; to the styles. It was kinda funny though, as I happen to notice that and I was looking at a page where that was still a smidge too wide and it was cutting off like 50px of text so there was still a bit of horizontal scrolling needed.

The problem is that 90vw is a “magic number”. It’s essentially saying “pretty close to the width of the screen, but like, not all of it, because there is some padding and stuff the account for.” It’s just a guess. I get it, sometimes you gotta just be close and move on, but here I’ve literally seen it fail, which is the whole downside of magic numbers.

If they were trying to be perfect about it, that max-width would be set to account for all the other spacing. Something like:

.shortSummaryText {
  /* Viewport width minus 1rem of body padding on each side 
     and 0.5rem of padding inside the table cell, 
     minus the borders. */
  max-width: calc(100dvw - 3rem - 2px); 
}

Maybe those things are custom properties that could be grabbed, which would be even nicer. This kind of thing can be hard to maintain so I get it.

Notice in the screenshot above they also added position: sticky; and stuck it to the left side, which is a super classy touch! That way when the table is scrolled to see the other bits of information in a row, the readable paragraphs stay readable, rather than scroll over into just blank white nothingness.

I did a fork of the classic Under-Engineered Responsive Tables to include this feature.

]]>
https://frontendmasters.com/blog/responsive-tables-readable-paragraphs/feed/ 1 4725
Automatically Contrasted Colors https://frontendmasters.com/blog/automatically-contrasted-colors/ https://frontendmasters.com/blog/automatically-contrasted-colors/#respond <![CDATA[Chris Coyier]]> Tue, 10 Dec 2024 18:29:18 +0000 <![CDATA[The Beat]]> <![CDATA[Accessibility]]> <![CDATA[Color]]> <![CDATA[CSS]]> https://frontendmasters.com/blog/?p=4781 <![CDATA[What is a good contrast text color on a black background? White. What about on a white background? Black. What about on #f06d06? Less clear. Devon Govett posted a good trick to having CSS pick for you, which works across browsers today. Lea Verou has a much deeper dive. There is supposed to be a […]]]> <![CDATA[

What is a good contrast text color on a black background? White. What about on a white background? Black. What about on #f06d06? Less clear. Devon Govett posted a good trick to having CSS pick for you, which works across browsers today. Lea Verou has a much deeper dive. There is supposed to be a CSS color-contrast() function, but it’s not usable across browsers yet.

]]>
https://frontendmasters.com/blog/automatically-contrasted-colors/feed/ 0 4781
Drizzle Database Migrations https://frontendmasters.com/blog/drizzle-database-migrations/ https://frontendmasters.com/blog/drizzle-database-migrations/#respond <![CDATA[Adam Rackis]]> Mon, 09 Dec 2024 15:23:12 +0000 <![CDATA[Blog Post]]> <![CDATA[Data]]> <![CDATA[Drizzle]]> <![CDATA[Migrations]]> https://frontendmasters.com/blog/?p=4692 <![CDATA[Drizzle ORM is a powerful object-relational mapper that combines SQL capabilities with a strongly typed API, enabling complex queries. Here we'll look at using it's ability to help with migrations, both code-first and database-first.]]> <![CDATA[

Drizzle ORM is an incredibly impressive object-relational mapper (ORM). Like traditional ORMs, it offers a domain-specific language (DSL) for querying entire object graphs. Imagine grabbing some “tasks”, along with “comments” on those tasks from your database. But unlike traditional ORMs, it also exposes SQL itself via a thin, strongly typed API. This allows you to write complex queries using things like MERGEUNION, CTEs, and so on, but in a strongly typed API that looks incredibly similar to the SQL you already know (and hopefully love).

I wrote about Drizzle previously. That post focused exclusively on the typed SQL API. This post will look at another drizzle feature: database migrations. Not only will Drizzle allow you to query your database via a strongly typed API, but it will also keep your object model and database in sync. Let’s get started!

Our Database

Drizzle supports Postgres, MySQL, and SQLite. For this post we’ll be using Postgres, but the idea is the same for all of them. If you’d like to follow along at home, I urge you to use Docker to spin up a Postgres database (or MySQL, if that’s your preference). If you’re completely new to Docker, it’s not terribly hard to get it installed. Once you have it installed, run this:

docker container run -e POSTGRES_USER=docker -e POSTGRES_PASSWORD=docker -p 5432:5432 postgres:17.2-alpine3.20

That should get you a Postgres instance up and running that you can connect to on localhost, with a username and password of docker / docker. When you stop that process, your database will vanish into the ether. Restarting that same process will create a brand new Postgres instance with a completely clean slate, making this especially convenient for the type of testing we’re about to do: database migrations.

Incidentally, if you’d like to run a database that actually persists its data on your machine, you can mount a volume.

docker container run -e POSTGRES_USER=docker -e POSTGRES_PASSWORD=docker -p 5432:5432 -v /Users/arackis/Documents/pg-data:/var/lib/postgresql/data postgres:17.2-alpine3.20

That does the same thing, while telling Docker to alias the directory in its image of /var/lib/postgresql/data (where Postgres stores its data) onto the directory on your laptop at /Users/arackis/Documents/pg-data. Adjust the latter path as desired. (The other path isn’t up for debate, as that’s what Postgres uses.)

Setting Up

We’ll get an empty app up (npm init is all we need), and then install a few things.

npm i drizzle-orm drizzle-kit pg

The drizzle-orm package is the main ORM that handles querying your database. The drizzle-kit package is what handles database migrations, which will be particularly relevant for this post. Lastly, the pg package is the Node Postgres drivers.

Configuring Drizzle

Let’s start by adding a drizzle.config.ts to the root of our project.

import { defineConfig } from "drizzle-kit";

export default defineConfig({
  dialect: "postgresql",
  out: "./drizzle-schema",
  dbCredentials: {
    database: "jira",
    host: "localhost",
    port: 5432,
    user: "docker",
    password: "docker",
    ssl: false,
  },
});

We tell Drizzle what kind of database we’re using (Postgres), where to put the generated schema code (the drizzle-schema folder), and then the database connection info.

Wanna see more real-world use cases of Drizzle in action? Check out Scott Moss’ course Intermediate Next.js which uses it and gets into it when the project gets into data fetching needs.

The Database First Approach

Say we already have a database and want to generate a Drizzle schema from it. (If you want to go in the opposite direction, stay tuned.)

To create our initial database, I’ve put together a script, which I’ll put here in its entirety.

CREATE DATABASE jira;

\c jira

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50),
    name VARCHAR(250),
    avatar VARCHAR(500)
);

CREATE TABLE tasks (
    id SERIAL PRIMARY KEY,
    name VARCHAR(250),
    epic_id INT,
    user_id INT
);

CREATE TABLE epics (
    id SERIAL PRIMARY KEY,
    name VARCHAR(250),
    description TEXT,
    due DATE
);

CREATE TABLE tags (
    id SERIAL PRIMARY KEY,
    name VARCHAR(250)
);

CREATE TABLE tasks_tags (
    id SERIAL PRIMARY KEY,
    task INT,
    tag INT
);

ALTER TABLE tasks
    ADD CONSTRAINT fk_task_user
    FOREIGN KEY (user_id)
    REFERENCES users (id);

ALTER TABLE tasks
    ADD CONSTRAINT fk_task_epic
    FOREIGN KEY (epic_id)
    REFERENCES epics (id);

ALTER TABLE tasks_tags
    ADD CONSTRAINT fk_tasks_tags_tag
    FOREIGN KEY (tag)
    REFERENCES tags (id);

ALTER TABLE tasks_tags
    ADD CONSTRAINT fk_tasks_tags_task
    FOREIGN KEY (task)
    REFERENCES tasks (id);

This will construct a basic database for an hypothetical Jira clone. We have tables for users, epics, tasks and tags, along with various foreign keys connecting them. Assuming you have psql installed (can be installed via libpq), you can execute that script from the command line like this:

PGPASSWORD=docker psql -h localhost -p 5432 -U docker -f database-creation-script.sql

Now run this command:

npx drizzle-kit pull

This tells Drizzle to look at our database and generate a schema from it.

Drizzle pull

Files generated

Inside the drizzle-schema folder there’s now a schema.ts file with our Drizzle schema. Here’s a small sample of it.

import { pgTable, serial, varchar, foreignKey, integer } from "drizzle-orm/pg-core";
import { sql } from "drizzle-orm";

export const users = pgTable("users", {
  id: serial().primaryKey().notNull(),
  username: varchar({ length: 50 }),
  name: varchar({ length: 250 }),
  avatar: varchar({ length: 500 }),
});

export const tasks = pgTable(
  "tasks",
  {
    id: serial().primaryKey().notNull(),
    name: varchar({ length: 250 }),
    epicId: integer("epic_id"),
    userId: integer("user_id"),
  },
  table => {
    return {
      fkTaskUser: foreignKey({
        columns: [table.userId],
        foreignColumns: [users.id],
        name: "fk_task_user",
      }),
      fkTaskEpic: foreignKey({
        columns: [table.epicId],
        foreignColumns: [epics.id],
        name: "fk_task_epic",
      }),
    };
  }
);

The users entity is a table with some columns. The tasks entity is a bit more interesting. It’s also a table with some columns, but we can also see some foreign keys being defined.

In Postgres, foreign keys merely create a constraint that’s checked on inserts and updates to verify that a valid value is set, corresponding to a row in the target table. But it has no effect on application code, so you might wonder why Drizzle saw fit to bother creating it. Essentially, Drizzle will allow us to subsequently modify our schema in code, and generate an SQL file that will make equivalent changes in the database. For this to work, Drizzle needs to be aware of things like foreign keys, indexes, etc, so the schema in code, and the database are always truly in sync, and Drizzle knows what’s missing, and needs to be created.

Relations

The other file Drizzle created is relations.ts. Here’s a bit of it:

import { relations } from "drizzle-orm/relations";

export const tasksRelations = relations(tasks, ({ one, many }) => ({
  user: one(users, {
    fields: [tasks.userId],
    references: [users.id],
  }),
  epic: one(epics, {
    fields: [tasks.epicId],
    references: [epics.id],
  }),
  tasksTags: many(tasksTags),
}));

export const usersRelations = relations(users, ({ many }) => ({
  tasks: many(tasks),
}));

This defines the relationships between tables (and is closely related to foreign keys). If you choose to use the Drizzle query API (the one that’s not SQL with types), Drizzle is capable of understanding that some tables have foreign keys into other tables, and allows you to pull down objects, with related objects in one fell swoop. For example, the tasks table has a user_id column in it, representing the user it’s assigned to. With the relationship set up, we can write queries like this:

const tasks = await db.query.tasks.findMany({
  with: {
    user: true,
  },
});

This will pull down all tasks, along with the user each is assigned to.

Making Changes (Migrations)

With the code generation above, we’d now be capable of using Drizzle. But that’s not what this post is about. See my last post on Drizzle, or even just the Drizzle docs for guides on using it. This post is all about database migrations. So far, we took an existing database, and scaffolded a valid Drizzle schema. Now let’s run a script to add some things to the database, and see about updating our Drizzle schema.

We’ll add a new column to tasks called importance, and we’ll also add an index on the tasks table, on the epic_id column. This is unrelated to the foreign key we already have on this column. This is a traditional database index that would assist us in querying the tasks table on the epic_id column.

Here’s the SQL script we’ll run:

CREATE INDEX idx_tasks_epic ON tasks (epic_id);

ALTER TABLE tasks
    ADD COLUMN importance INT;

After running that script on our database, we’ll now run:

npx drizzle-kit pull

Our terminal should look like this:

Drizzle pull again

We can now see our schema updates in the git diffs:

Drizzle pull changes

Note the new columns being added, and the new index being created. Again, the index will not affect our application code; it will make our Drizzle schema a faithful representation of our database, so we can make changes on either side, and generate updates to the other. To that end, let’s see about updating our code, and generating SQL to match those changes.

The Code First Approach

Let’s go the other way. Let’s start with a Drizzle schema, and generate an SQL script from it. In order to get a Drizzle schema, let’s just cheat and grab the schema.ts and relations.ts files Drizzle created above. We’ll paste them into the drizzle-schema folder, and remove anything else Drizzle created: any snapshots, and anything in the meta folder Drizzle uses to track our history.

Next, since we want Drizzle to read our schema files, rather than just generate them, we need to tell Drizzle where they are. We’ll go back into our drizzle.config.ts file, and add this line:

schema: ["./drizzle-schema/schema.ts", "./drizzle-schema/relations.ts"],

Now run:

npx drizzle-kit generate

Voila! We have database assets being created.

Drizzle pull changes

The resulting sql file is huge. Mine is named 0000_quick_wild_pack.sql (Drizzle will add these silly names to make the files stand out) and looks like this, in part.

CREATE TABLE IF NOT EXISTS "epics" (
	"id" serial PRIMARY KEY NOT NULL,
	"name" varchar(250),
	"description" text,
	"due" date
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "tags" (
	"id" serial PRIMARY KEY NOT NULL,
	"name" varchar(250)
);
--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "tasks" (
	"id" serial PRIMARY KEY NOT NULL,
	"name" varchar(250),
	"epic_id" integer,
	"user_id" integer
);

Making a schema change

Now let’s make some changes to our schema. Let’s add that same importance column to our tasks table, add that same index on epicId, and then, for fun, let’s tell Drizzle that our foreign key on userId should have an ON DELETE CASCADE rule, meaning that if we delete a user, the database will automatically delete all tasks assigned to that user. This would probably be an awful rule to add to a real issue tracking software, but it’ll help us see Drizzle in action.

Here are the changes:

And now we’ll run npx drizzle-kit generate and you should see:

As before, Drizzle generated a new sql file, this time called 0001_curved_warhawk.sql which looks like this:

ALTER TABLE "tasks" DROP CONSTRAINT "fk_task_user";
--> statement-breakpoint
ALTER TABLE "users" ADD COLUMN "importance" integer;--> statement-breakpoint
DO $$ BEGIN
 ALTER TABLE "tasks" ADD CONSTRAINT "fk_task_user" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE cascade ON UPDATE no action;
EXCEPTION
 WHEN duplicate_object THEN null;
END $$;
--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "idx_tasks_epicId" ON "tasks" USING btree ("epic_id");

It added a column, overwrote the foreign key constraint we already had to add our CASCADE rule, and created our index on epic_id.

Mixing & Matching Approaches

Make no mistake, you do not have to go all in on code-first, or database-first. You can mix and match approaches. You can scaffold a Drizzle schema from a pre-existing database using drizzle-kit pull, and then make changes to the code, and generate sql files to patch your database with the changes using drizzle-kit generate. Try it and see!

Going Further

Believe it or not, we’re only scratching the surface of what drizzle-kit can do. If you like what you’ve seen so far, be sure to check out the docs.

Concluding Thoughts

Drizzle is an incredibly exciting ORM. Not only does it manage to add an impressive layer of static typing on top of SQL, allowing you to enjoy the power and flexibility of SQL with the type safety you already expect from TypeScript. But it also provides an impressive suite of commands for syncing your changing database with your ORM schema.

]]>
https://frontendmasters.com/blog/drizzle-database-migrations/feed/ 0 4692