New HTTP Val Runtime in Preview

Max McDonnell on

We built a new runtime for HTTP vals that is much faster at scale. We’ve seen up to a 5x speedup for some vals. This is a preview feature, so expect rough edges. We’d love your feedback on it.

Usage

We created a new val type, HTTP (Preview), to let you opt-into the new runtime. Here’s how:

  1. Create a normal HTTP val
  2. Click on the val’s type HTTP and select HTTP (Preview) in the dropdown menu.

That’s it! Run your val 5 times in quick succession to see the difference.

Architecture

Up until now, every time you run a val it starts a new Deno process and installs all your dependencies from scratch. This takes 100 milliseconds or more, depending on how many dependencies you have.

With the new HTTP (Preview) val, we keep your Deno process around and forward multiple requests to it. This means that the first request will still have the 100ms+ cold start time, but every request after that will be much faster. For example, the simplest possible val that just returns “hello world!” will take 100ms to boot up the first time, but then only 10ms for subsequent requests. To be clear, that’s not including network latency to the val (which from NYC is 90ms), just the time it takes for the val to respond.

A user from NYC hitting a “hello world!” val can expect to see a 180ms response time for the first request and 90ms for every request after that using HTTP (Preview). Whereas with the normal HTTP val type, they would see 190ms for every request. The difference is even more pronounced for vals with more dependencies and large dependencies like WASM modules. We migrated the Date Me Directory and saw total request time go from 800ms to 200ms!

We want to be conservative with this rollout, so we’re only keeping HTTP (Preview) vals alive for 10 seconds after the end of the last request they receive. As we get more comfortable keeping vals alive for longer, we’ll increase this timeout window, particularly for Pro users. We’re considering ways to keep vals alive indefinitely, but that will require some changes to our pricing to make it sustainable.

Limitations

There are currently two limitations to HTTP (Preview) compared to the normal HTTP type:

  1. console.log and other logging output aren’t captured
  2. You can’t cancel a val that is currently running

We expect to have fixes for these limitations in the next couple weeks. In the meantime, we recommend using the normal HTTP val type in development and then switching to HTTP (Preview) for production.

Breaking changes

There is one breaking change with this new feature: it only re-runs your val handler, not any of the other code in your val. This means that if you have any code outside of your handler that you want to run on every request, you’ll need to move it into your handler.

For example, this val will return a new random number for every request in our existing system, but after you switch to the next version, it’ll return the same number - because the call to Math.random() is only run when the val is started up, not when a new request is handled.

const number = Math.random();


export default function(req: Request) {
  return Response.json(number);
}

The upgrade path is to move anything you want to happen on every request into the request handler:

export default function(req: Request) {
  const number = Math.random();
  return Response.json(number);
}

Now each call to this val will return a new random number. This is how most serverless platforms work, and we think it’s a good model to follow. You’ll only need to migrate your code if you upgrade to the new runtime.

Rollout

This new runtime has the potential to be much faster and more scalable than our current approach, so we want it to be the default soon. But it’s a big change, so we’re being careful. Today we’re rolling it out as an opt-in val type, HTTP (Preview).

After we’ve ironed out the kinks and achieved feature parity between the two runtimes, we’ll make this the default type for new HTTP vals, drop the (Preview), and encourage everyone to upgrade to the new system for that big performance boost! Depending on how it works out, we might be also be able to auto-upgrade some vals to the new system.

Feedback

We’d love to hear your feedback on this new feature. You can send a message in our #general or #bugs channels in our Discord or email us at [email protected].

Edit this page