Configure service routes

Media CDN provides advanced HTTP routing capabilities that allow you to map traffic to specific edge configurations and origins at a fine-grained level.

Configure a route rule

Configure a route rule for a Media CDN service.

Match requests

A Media CDN configuration contains a set of routes defined in the Routing section for an EdgeCacheService resource. These routes match requests based on (at least) a host. For more details about how traffic is directed to an origin, see HostRule and PathMatcher. Each route is able to define its own CDN configuration, rewrites, redirects, CORS policies, custom HTTP headers, and origin mapping. Routes can share origins.

For example, you can route requests for manifests to a specific origin and define a short-lived cache TTL and a negative caching policy. Requests for segments can be split to another origin by using headers and query parameters to break out specific manifest types or users.

The following example shows how to route requests that match a specific header, query parameter, and path prefix for the host media.example.com:

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 10
      origin: staging-live-origin
      matchRules:
      - prefixMatch: /vod/
        headerMatches:
        - headerName: "x-staging-client"
          presentMatch: true
        queryParameterMatches:
        - name: "live"
          exactMatch: "yes"
      routeAction:
        cdnPolicy:
          defaultTtl: 5s

Path matching

Media CDN supports full (exact), prefix, and wildcard path matching. Path matching can be combined with host, header, and query parameter-based matching to construct fine-grained request routing rules.

Following are three ways to match on a URL path.

For more details, see the API specification for MatchRule.

For example, to match all requests starting with /stream/, create a route rule similar to the following:

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    - *.vod.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      matchRules:
      - prefixMatch: /stream/

This example explicitly includes the trailing slash in the match rule:

• A request to media.example.com/stream/id/1234/hls/manifest.m3u8 matches this route.
• A request to media.example.com/stream-eu/id/4567/hls/manifest.m3u8 does not match this route.

In the second case, Media CDN returns an HTTP 404 error, unless there was another route or a catch-all route configured.

For guidance about how predecence works for routes with similar prefixes, see the route priority and ordering section.

Pattern (wildcard) matching

Pattern matching lets you match multiple parts of a URL, including partial URLs and suffixes (file extensions), by using wildcard syntax.

You can also associate one or more path segments with named variables in a pathTemplateMatch field, and then refer to those variables when rewriting the URL in a pathTemplateRewrite field. This lets you reorder and remove URL segments before the request is sent to your origin.

The following example shows how you can match on two different URL suffixes:

# EdgeCacheService.routing.pathMatchers[]
    routeRules:
    - priority: 1
      description: "Match video segments"
      matchRules:
      - pathTemplateMatch: "/**.ts"
      - pathTemplateMatch: "/**.m4s"
      origin: prod-video-storage

The supported syntax includes the following.

Notes:

• If you aren't rewriting a URL, use the simpler * and ** operators.
• When using variables to capture path segments, any parts of the URL that aren't captured by a variable can't be referenced in a subsequent pathTemplateRewrite. For an example, see the capturing path variables section.
• You can't refer to variables in a subsequent pathTemplateRewrite that don't exist in the pathTemplateMatch on the same route.
• Variables are case-sensitive, with{FORMAT}, {forMAT}, and {format} representing different variables and values.
• You can specify up to 10 operators (wildcards or variables) in a match. The pathTemplateMatch and pathTemplateRewrite fields must not exceed 255 characters.

Example: Match on a file extension

The following example shows a common use-case for wildcard operators: matching all path segments up to a suffix.

In this case, you do the following:

• Fetch video manifests (playlists) ending in .m3u8 and .mpd from the manifest origin, applying a short (5-second) TTL to these responses because they change regularly.
• Fetch video segments ending in .ts and .m4s from the segment origin, and apply a longer (1-day) TTL to these responses.

This approach is often employed when using SSAI (Server-Side Ad Injection) or DAI (Dynamic Ad Insertion) services, and for live video where the manifest is updated every few seconds.

The following configuration demonstrates how to configure your Media CDN routing to support this:

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    # the first route only matches video manifests
    - priority: 1
      matchRules:
      - pathTemplateMatch: "/**.m3u8" # "**" matches all path segments
      - pathTemplateMatch: "/**.mpd"
      origin: manifest-origin
      routeAction:
        cdnPolicy:
          cacheMode: FORCE_CACHE_ALL
          defaultTtl: 5s
    # the second route matches video segments, fetches them
    # from a separate origin server, caching them for a longer
    # duration (1 day).
    - priority: 2
      matchRules:
      - pathTemplateMatch: "/**.ts"
      - pathTemplateMatch: "/**.m4s"
      origin: segment-origin
      routeAction:
        cdnPolicy:
          cacheMode: FORCE_CACHE_ALL
          defaultTtl: 86400s

Example: Capture path variables

The following example shows how to use named variables to describe one or more path segments.

These variables can be used in a pathTemplateRewrite to rewrite the path prior to the request being sent to the origin, or to make a complex pathTemplateMatch self-describing.

routing:
  hostRules:
  - hosts:
    - media.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      matchRules:
      # Matches a request of "/us/en/hls/123139139/segments/00001.ts"
      - pathTemplateMatch: "/{country}/{lang}/{format}/{id}/{file=**}"
      origin: my-origin
      routeAction:
        urlRewrite:
          # Rewrites to "/123139139/hls/segments/00001.ts"
          pathTemplateRewrite: "/{id}/{format}/{file}"

Specifically:

• Each {name} variable captures a single path segment. A path segment is all characters between a pair of / ("slashes") in a URL path.
• A variable of {name=**} captures all remaining path segments; in this case, it matches both segments/00001.ts and master.m3u8.
• In the pathTemplateRewrite on the same route, you refer back to some of the variables that you captured in the pathTemplateMatch. You explicitly omit the {country} and {lang} variables because they don't match the directory structure on the origin.

With this example, the following occurs:

• An incoming request URL of /us/en/hls/123139139/segment_00001.ts matches the pathTemplateMatch and is rewritten to /123139139/hls/segment_00001.ts prior to being sent to the origin.
• An incoming request URL of /us/123139139/master.m3u8 does not match the pathTemplateMatch, and receives an HTTP 404 (Not Found) response.
• An incoming request URL of /br/es/dash/c966cbbe6ae3/subtitle_00001.vtt also matches the pathTemplateMatch and is rewritten to /c966cbbe6ae3/dash/subtitle_00001.vtt prior to being sent to the origin.

To learn more about how wildcard matching interoperates with URL rewriting, see the rewrites section.

Host matching

Each service can match on multiple hostnames, with each set of hostnames containing their own group of routes (known as path matchers). In the most common case, all hostnames for a service map to a single set of shared routes with a single list of hosts and a single path matcher.

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    - *.vod.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    # list of routes for the configured hosts
    - priority: 999
      matchRules:
      - prefixMatch: /
      origin: DEFAULT_ORIGIN

Hosts that don't match are served a default HTTP 404 page. To accept any host, you can include the wildcard character * as a hostRules[].hosts[] entry.

You can also define groups of routes (for example, grouping by country or live versus on-demand). Because each service has a single security policy, we generally recommend having one service for each market (geography) or workload that you have.

Notes:

• Host (or HTTP/2 :authority) headers that contain a port are implicitly matched against a configured host. You don't need to explicitly specify the ports.
• If the request is over HTTP, a hostRules[].hosts[] entry of *.vod.example.com matches us.vod.example.com and us.vod.example.com:80.
• If the request is over HTTPS (TLS), a hostRules[].hosts[] entry of *.vod.example.com matches us.vod.example.com:443.

For more details, see the API specification for HostRule.

Match on headers and query parameters

You can configure routes to match on specific header and query parameter names, as well as the presence of a header value (prefix, suffix, or exact match).

Query parameter and header matching are logical "AND"—the request must match all of the query parameters and header keys (and values, if specified) to match the given route.

For example, if you want to route requests with a specific header field name and header value to an origin named alternate-origin, configure your match conditions within routeRules[].matchRules[].headerMatches[]:

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      origin: alternate-origin
      matchRules:
      - prefixMatch: "/videos/"
        headerMatches:
        - headerName: "x-device-name"
          exactMatch: "roku"

In this example, requests with /videos/ at the beginning of the URL and the x-device-name: roku header match this route. Requests missing this header name or with a different value don't match this route.

For more details, see the API specification for HeaderMatch.

Similarly, to match against query parameters, specify one or more queryParameterMatches as follows:

name: prod-service
routing:
  hostRules:
  - hosts:
    - media.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      origin: eu-live-origin-prod
      matchRules:
      - prefixMatch: "/videos/"
        queryParameterMatches:
        - name: "playback_type"
          exactMatch: "live"
        - name: "geo"
          exactMatch: "eu"

In this example, a client request of https://cdn.example.com/videos/1234/abcd/xyz.m3u8?playback_type=live&geo=eu matches this route.

For more details, see the API specification for QueryParameterMatcher.

Define a catch-all (default) route

By default, Media CDN returns an HTTP 404 (Not Found) error if a request doesn't match any configured routes.

To configure a catch-all route, for a given pathMatcher (collection of routes), do the following:

• Create a routeRule with the lowest priority (highest number)—for example, 999, which is the lowest possible route priority.
• Configure a matchRule with a prefix match of / (matches all request paths).
• Configure (one of) an origin or urlRedirect on the route.

For example, to configure a catch-all route that directs all unmatched requests to a default origin named my-origin, create a new route with priority: 999 and a matchRules[].prefixMatch of / as follows:

name: prod-service
routing:
  hostRules:
  - hosts:
    - cdn.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 999
      origin: my-origin
      matchRules:
      - prefixMatch: /

You can optionally rewrite the URL prior to the origin fetch, or redirect to a default page (such as your landing page) instead of sending the request "as is" to the origin.

Route priority and ordering

Each route in an array of routeRules[] must have a priority associated with it.

More specific routes should be set to a higher priority (smaller number). A route matching a prefix of /stream/ with a priority of 1 otherwise prevents a more specific route of /stream/live/eu/ with a priority of 5 from matching any requests.

• The highest priority route is "1", and the lowest is "999".
• You can't configure two or more routeRules with the same priority. Priority for each rule must be set to a number between 1 and 999 inclusive.
• Defining a catch-all route lets you send all non-matching requests to a default origin or redirect them to a landing page or endpoint.

In the following example, you can see that the /live/us/ route would never be matched because the /live/ route is at a higher priority:

routeRules:
- priority: 1
  description: "Live routes"
  matchRules:
  - prefixMatch: /live/
  routeAction:
    cdnPolicy:
      defaultTtl: 5s
- priority: 2
  description: "U.S based live streams"
  matchRules:
  # This would never be matched, as the /live/ prefixMatch at priority 1
  # would always take precedence.
  - prefixMatch: /live/us/
  routeAction:
    cdnPolicy:
      defaultTtl: 5s
- priority: 999
  description: "Catch-all route"
  matchRules:
  - prefixMatch: /

To address this, you put the more specific (longer) route at a higher priority:

routeRules:
- priority: 1
  description: "U.S based live streams"
  matchRules:
  # The more specific (longer) match is at a higher priority, and now
  # matches requests as expected.
  - prefixMatch: /live/us/
  routeAction:
    cdnPolicy:
      defaultTtl: 5s
- priority: 2
  description: "Live routes"
  matchRules:
  - prefixMatch: /live/
  routeAction:
    cdnPolicy:
      defaultTtl: 5s
- priority: 999
  description: "Catch-all route"
  matchRules:
  - prefixMatch: /

This allows the more specific route to match requests correctly. A request prefixed with /live/eu/ would still fall through to the /live/ route at priority 2.

Method filtering

By default, Media CDN proxies only GET, HEAD, and OPTIONS methods to your origin and filters out the methods that can modify your origin.

You can override this default behavior for a specific route rule by specifying the methods that you would like proxied to your origin. Besides GET, HEAD, and OPTIONS, Media CDN supports PUT, POST, DELETE, and PATCH.

To configure support for a set of methods for a route rule, specify a routeMethods section that has an allowed_methods value for each method.

routeRules:
- priority: 5
  description: "Video uploads"
  routeMethods:
    allowedMethods: ["PUT", "POST", "OPTIONS"]
  matchRules:
  - pathTemplateMatch: "/uploads/**.ts"
  origin: prod-video-storage
- priority: 10
  description: "Video serving"
  routeMethods:
    allowedMethods: ["GET", "HEAD"]
  matchRules:
  - pathTemplateMatch: "/videos/**.ts"
  origin: prod-video-storage

Path normalization

Path normalization describes how Media CDN combines multiple representations of a URL into a single, canonical representation under specific scenarios.

Path normalization can directly improve cache hit rates by reducing the number of request URLs that represent the same content, and by mitigating origin errors for origins that expect normalized paths.

Incoming requests are normalized as per the following:

• Multiple consecutive slashes are merged into a single slash. For example, a URL path of /videos///12345/manifest.mpd is normalized to /videos/12345/manifest.mpd.
• Path segments are normalized according to RFC 3986 Section 6.2.2.3. For example, the path /a/b/c/./../../g is normalized to /a/g based on the "remove dot segments" algorithm defined in RFC 3986. This normalization happens before checking the cache or forwarding the request to the origin.
• Requests are not percent-encoding normalized. For example, a URL with a percent-encoded slash character (%2F) is not decoded into the un-encoded form.

URLs remain case-sensitive and are not case-normalized. Many URLs contain case-sensitive base64 encodings, including URLs with signed request tokens.

Rewrites and redirects

The following sections provide examples on how to rewrite requests and configure redirects.

Rewrite request URLs

Media CDN supports host and path rewrites. Rewrites change the URL sent to the origin and let you modify hosts and paths as needed. Host and path rewrites are at the route level, letting you define what specific requests are rewritten based on any matcher, including path, query parameter, and request header.

For more details, see the API specification for RouteAction.UrlRewrite.

Following are three ways to rewrite a request:

Notes:

• Rewritten URLs don't change the cache key. The cache key is based on the request URL sent by the client.
• Rewriting occurs after route matching and signed request validation. Routes aren't re-matched against another match rule.

Example: Remove a path prefix

For example, to rewrite a client request URL from /vod/videos/hls/1234/abc.ts to /videos/hls/1234/abc.ts (removing /vod from the path), you can use the pathPrefixRewrite feature:

name: prod-service
routing:
  hostRules:
  - hosts:
    - cdn.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      origin: my-origin
      matchRules:
      - prefixMatch: "/vod/videos/"
      routeAction:
        urlRewrite:
          pathPrefixRewrite: "/videos/"

A pathPrefixRewrite works by replacing the entire path prefix matched in the matchRules[].prefixMatch with the value of pathPrefixRewrite.

To rewrite a hostname (for example, rewriting cdn.example.com to my-bucket.s3.us-west-2.amazonaws.com), you can configure the following:

name: prod-service
routing:
  hostRules:
  - hosts:
    - cdn.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 1
      origin: my-origin
      matchRules:
      - prefixMatch: "/videos/"
      routeAction:
        urlRewrite:
          hostRewrite: "my-bucket.s3.us-west-2.amazonaws.com"

In this case, the origin request URL would change from cdn.example.com/videos/* to my-bucket.s3.us-west-2.amazonaws.com/videos/*. You can also combine both host and path rewrites within a single route.

Example: Use variables to rewrite URLs

To use pathTemplateMatch and pathTemplateRewrite to rewrite parts of an incoming request URL, see the capturing variables section.

Redirect requests

Media CDN supports three types of redirects:

• Host redirects, which redirect only the host (domain), keeping the path and query parameters unchanged.
• Path redirects, which replace the path in full.
• Path prefix redirects, which only replace the matching prefix.

Redirects default to HTTP 301 (Moved Permanently), but can be configured to return different redirect status codes on a per-route basis.

The following configuration is an example of a prefix-based redirect, where you redirect users visiting https://cdn.example.com/on-demand/* to https://cdn.example.com/streaming/*.

name: prod-service
routing:
  hostRules:
  - hosts:
    - cdn.example.com
    pathMatcher: example_routes
  pathMatchers:
  - name: example_routes
    routeRules:
    - priority: 10
      matchRules:
      - prefixMatch: "/on-demand/"
      urlRedirect:
        # The prefix matched in matchRules.prefixMatch is replaced
        # by this value
        prefixRedirect: "/streaming/"
        redirectResponseCode: TEMPORARY_REDIRECT # corresponds to a HTTP 307

This example also changes the redirect to a temporary redirect, which prevents clients (such as browsers) from caching it. This can be useful if you expect to change it in the near future.

The supported redirectResponseCode values are shown in the following table.

Notes:

• A route can either direct traffic to an origin or return a redirect to the client. You can't set both the origin and urlRedirect fields at the same time.
• Routes that redirect to HTTPS require that at least one SSL certificate is attached to the service.

For more details, see the API specification for RouteRule.UrlRedirect.

Redirect all requests to HTTPS

To redirect all requests to HTTPS (instead of HTTP), you can configure each of your services to redirect all client requests to HTTPS automatically. Clients connecting over HTTP are sent an HTTP 301 (Permanent Redirect) response with the Location header set to the same URL by using "https://" instead of "http://".

gcloud edge-cache services update MY_SERVICE \
    --require-tls

Request issued for: [MY_SERVICE]
Updated service [MY_SERVICE].

  name: MY_SERVICE
  requireTls: true

You can also choose to set the Strict-Transport-Security header as a response header to direct clients to always connect directly over HTTPS.

Use third-party storage backends

Media CDN supports connecting to publicly reachable HTTP endpoints outside of Google Cloud, including AWS S3 storage buckets, Azure Blob Storage, and other storage providers. This can be useful if you have a multicloud architecture, or are yet to migrate data to Cloud Storage using the Storage Transfer Service.

A minimal origin configuration that configures a virtual-hosted bucket in AWS S3:

name: MY_S3_ORIGIN
originAddress: BUCKET-NAME.s3.REGION.amazonaws.com

If you aren't using a bucket name that matches the hostnames configured for your EdgeCacheService resources, you must also configure a host rewrite for routes associated with this origin (or origins). Otherwise, the Host header set by the client request is used when fetching from the origin.

For example, to map all requests with a path prefix of /legacy/ to your external bucket, you can configure both a hostRewrite and a pathPrefixRewrite to strip this prefix from the origin request:

routeRules:
  - description: legacy backend
    matchRules:
    - prefixMatch: "/legacy/"
    routeAction:
      urlRewrite:
        hostRewrite: BUCKET-NAME.s3.REGION.amazonaws.com
        pathPrefixRewrite: /
      cdnPolicy:
        cacheMode: CACHE_ALL_STATIC
        defaultTtl: 3600s

For more information about how the host header is set on origin requests, see the origin request headers documentation.

Cross-Origin Resource Sharing (CORS)

Cross-Origin Resource Sharing (CORS) is a browser-centric approach to securely making cross-origin requests. CORS policies allow you to automatically set CORS headers, such as Access-Control-Allow-Origins, based on a per-route policy.

Configure CORS

Media CDN lets you define a CORS policy on a route for an EdgeCacheService.

A CORS policy defines these rules with a common set of HTTP headers. You can set common CORS headers in responses, such as Access-Control-Allow-Origin, Access-Control-Max-Age, and Access-Control-Allow-Headers. These headers allow you to make cross-origin calls to your Media CDN services that might be hosted on a different domain (origin) from your website's frontend and might prevent cross-origin requests that you don't explicitly permit.

For example, player.example.com and api.example.com are different origins (in the browser sense), and you might want to have your frontend application make requests to api.example.com to fetch the next playlist or refresh a listing of related content. Similarly, player.example.com might need to reach out to cdn.example.com to fetch video playlists and video segments: cdn.example.com needs to indicate that this is OK, and that player.example.com is an allowed origin, as well as any other rules (allowed headers, whether cookies can be included).

To take another example, if you want to allow stream.example.com as an origin and a X-Client-ID header in CORS requests, you can configure a corsPolicy on a route, as follows:

corsPolicy: maxAge: 600 allowOrigins: ["stream.example.com"] allowHeaders:
["X-Client-ID"]

A corsPolicy is configured at routing.pathMatchers[].routeRules[].routeAction.corsPolicy within an EdgeCacheService. Each routeRule can define a different corsPolicy as needed, or none at all.

If you define a corsPolicy value and also set a custom response header by using the responseHeadersToAdd fields on a route with the same name, the custom response header takes precedence over, and is used instead of, the corsPolicy value.

If the origin response sets HTTP headers, and you have a corsPolicy value set, the corsPolicy values are used instead. The values are not collapsed or combined to avoid sending invalid header values to a client, or unintentionally setting a more permissive policy than intended.

The special {origin_request_header} is populated with the Origin HTTP header in the client request. This can be set as a custom response header value on a route, for the Access-Control-Allow-Origin header.

For more details, see the API specification for RouteAction.CORSPolicy.

CORS policy fields

The following table describes the fields that a CORS policy contains.

Example corsPolicy

The following configuration example shows a basic corsPolicy configuration:

routeRules:
- priority: 1
  matchRules:
  - prefixMatch: /stream/
  routeAction:
    cdnPolicy:
      defaultTtl: 3600s
    corsPolicy:
      allowOrigins:
      - "https://stream.example.com"
      - "https://stream-staging.example.com"
      maxAge: 86400s # some browsers might only honor up to 7200s or less
      allowMethods:
      - "GET"
      - "HEAD"
      - "OPTIONS"
      allowHeaders:
      - "Content-Type"
      - "If-Modified-Since"
      - "Range"
      - "User-Agent"
      exposeHeaders:
      - "Content-Type"
      - "Content-Length"
      - "Date"

Troubleshoot routing

If some requests don't retrieve matching results or return errors, check the following:

• A route must have a matchRule with exactly one of prefixMatch, fullPathMatch, or pathTemplateMatch defined. The API returns an error if you don't include one of those fields.
• Ensure that the priority of each route is set correctly: more specific (longer) routes should be given a higher priority over shorter, broader route matches.
• By default, only GET, HEAD, and OPTIONS requests are supported. To configure support for other methods, see Route methods. The methods that are not enabled for a particular route are rejected with an HTTP 405 (Method Not Allowed) error.
• HTTP GET requests with a body, or any request with trailers, are rejected with an HTTP 400 error, because request bodies are not allowed in GET requests.
• Query parameter and header matching are logical "AND"—the request must match all of the query parameter or header keys (and values, if specified) to match the given route.

What's next

• Review the documentation about configuring caching.
• Understand how to connect to different origins.
• Set up TLS (SSL) certificates for your service.
• Configure signed requests to authenticate access to content.