What is a service mesh?

A service mesh is a dedicated infrastructure layer within a software application that handles communication between services. A service mesh can handle traffic routing, security, observability, and resiliency functions, while abstracting these complexities away from individual services.

Modern applications rely on services that communicate with each other. Think of your last visit to an online store. You might have used the site’s search bar to browse products. That search represents a service. Maybe you also saw recommendations for related products or added an item to your online shopping cart. Those are both services, too. The service that communicates with the inventory database needs to communicate with the product webpage, which needs to communicate with the user’s online shopping cart. The retailer might also have a service that gives users in-app product recommendations. This service will communicate with a database of product tags to make recommendations, and it also needs to communicate with the same inventory database that the product page needed.

Modern applications are often broken down in this way, as a network of services that each perform a specific function. To execute its function, one service might need to request data from several other services. But what if some services get overloaded with requests, like the retailer’s inventory database? This is where a service mesh comes in—it manages communication among services, optimizing how all the moving parts work together.

A service mesh can be considered a microservices architecture pattern. Microservices are a style of application architecture where a collection of independent services communicate through lightweight application programming interfaces (APIs). A microservices architecture is a cloud-native approach to building software in a way that allows each core function within an application to exist independently. Unlike app development in other architectures, individual microservices can be built by small teams with the flexibility to choose their own tools and coding languages. Microservices are built independently, communicate with each other, and can individually fail without escalating into an application-wide outage.

Service-to-service communication is what makes microservices possible. As the microservices architecture grows, it becomes more complex to manage. If an app contains dozens or hundreds of services interacting with each other, challenges arise around network failures, monitoring and tracing, balancing traffic loads, and securing communication among different microservices. Addressing these issues entirely through custom code would be inefficient. Service mesh provides a consistent solution to handle these challenges without having to change the code of individual services.

A service mesh manages the communication between services using a data plane and a control plane. Service mesh doesn’t introduce new functionality to an application’s runtime environment—apps in any architecture have always needed rules to specify how requests get from point A to point B. What is different about a service mesh is that it takes the logic governing service-to-service communication out of individual services and abstracts it to a layer of infrastructure.

To do this, a service mesh is built into an app as an array of network proxies. Proxies are a familiar concept—if you’re accessing this webpage from a work computer, there’s a good chance you just used one. Here’s how it works:

1. As your request for this page went out, it was received by your company’s web proxy.
2. After passing the proxy’s security measure, it was sent to the server that hosts this page.
3. Next, this page was returned to the proxy and checked against its security measures again.
4. And then it was sent from the proxy to you.

In a service mesh, each service instance is paired with a “sidecar” proxy that runs alongside each service and intercepts all inbound and outbound network traffic. Each sidecar proxy sits alongside a microservice and routes requests to and from other proxies. The proxy handles tasks like traffic routing, load balancing, enforcing security policies, and collecting telemetry data. Instead of communicating directly with one another, services send requests through their sidecar. The sidecars handle inter-service communication. All of this comprises the data plane.

The control plane manages the configuration and policy distribution across the data plane. The control plane also distributes traffic routing rules, manages security certificates between services, configures components to enforce policies, and collects telemetry.

Without a service mesh, each microservice needs to be coded with logic to govern service-to-service communication. This makes communication failures harder to diagnose because the logic that governs interservice communication is hidden within each service.

Learn more about microservices

Istio is an open source service mesh platform that controls how microservices share data with one another. It controls the flow of traffic, enforces policies, and monitors communications in a microservices environment. It includes APIs that let Istio integrate into any logging platform, telemetry, or policy system. Istio can run in a variety of on-premise, cloud, containerized, and virtualized environments.

Istio’s architecture is divided into the data plane and the control plane. Istio uses Envoy proxies, which are high-performance proxies that are deployed as sidecars and mediate traffic for all services within the service mesh. In the data plane, developers can add Istio support to a service by deploying a sidecar proxy within the environment.

Istio’s service mesh includes a new ambient mode that will remove the need for sidecar proxies in service mesh, replacing them with node-level proxies and intermediate gateways called waypoints. Waypoint proxies run outside of application pods and are managed independently of applications.

Learn more about Istio at Red Hat Developer

Every new service added to an app, or new instance of an existing service running in a container, complicates the communication environment and introduces new points of possible failure. In a complex microservices architecture, it can be nearly impossible to locate problems without a service mesh.

There are several advantages to using a service mesh, including:

• Improved security. A service mesh uses mutual Transport Layer Security (mTLS) to ensure communication between services is encrypted and secure, and sensitive user data is protected. It adds an extra layer of security without requiring additional encryption be added to each service manually. A service mesh can improve role-based access control (RBAC) and policies for securing APIs, and it can automate certificate management and key rotation.
• Policy enforcement. Service meshes include centralized configuration for service policies like quotas, rate-limiting, and authentication and authorization. It provides control over service interactions through access policies. Policies are enforced at the proxy level, which helps create consistency across services.
• Traffic management. Service mesh can help your apps manage traffic to individual services based on load conditions, versions, and user-specific rules. For example, if you’re rolling out a new version of your inventory service, you can use a canary deployment to send only 5% of the traffic to the new service. (A canary is a smaller test deployment.) If that works, you can increase the traffic.
• Health checks and observability. It can be difficult to view how microservices are interacting in real time, but with a service mesh you can implement built-in observability tools like distributed tracing and metrics collection. Sidecars in a service mesh collect metrics (request counts, latency, error rates) and send them to the control plane or monitoring tools.
• Fault tolerance and increased resilience. When microservices encounter failures, a service mesh can help by automating retries and fallbacks. If a service fails or becomes unresponsive, the service mesh will retry based on predefined rules and can reroute traffic to alternative services. This means the app can handle failure gracefully when a service becomes unavailable, ensuring users still have a good experience. The service mesh also collects data on how long it took before a retry succeeded. This data can inform future rules on optimal wait time, ensuring that the system does not become overburdened by unnecessary retries.

With a service mesh, development and operations teams are better equipped to handle migration from monolithic applications to cloud-native apps―collections of small, independent, and loosely coupled microservice applications. 

Organizations can experience challenges when implementing a service mesh, including:

• Complexity and integration with existing systems. A service mesh can be difficult to set up, manage, and integrate with existing systems. Organizations can encounter challenges if they are working in a large, distributed environment across multicloud and on-premise systems, or have not previously used a service mesh in their environment.
• Resource requirements and operational overhead. Service meshes can increase the operational overhead of managing applications because each service instance now has a sidecar proxy, which increases CPU and memory usage. Managing and troubleshooting, particularly in large-scale deployments, can be complex, and maintaining performance and scale can be more difficult as a result.
• Skills gaps. Teams need training to understand service mesh features, configuration, and best practices. Debugging failures can be severe, especially when issues arise due to complex routing rules or mTLS misconfigurations. Many organizations find that their existing teams lack expertise in service mesh technology, which can present challenges with getting started and using service meshes effectively.

An API gateway is an API management tool that sits between a client and a collection of backend services. In this case, a client is the application on a user’s device and the backend services are those on an enterprise server. An API gateway is a way to decouple the client interface from your backend implementation. When a client makes a request, the API gateway breaks it into multiple requests, routes them to the right places, produces a response, and keeps track of everything.

A service mesh secures internal service communication while allowing external traffic through the API gateway. A service mesh used together with an API gateway can ensure policies are applied uniformly across internal services. 

Learn more about Red Hat 3scale API management

Red Hat offers integrated API management, service mesh, and infrastructure platform products to help you build a comprehensive service management architecture. Red Hat® OpenShift® Service Mesh provides a uniform way to connect, manage, and observe microservices-based applications. It offers behavioral insight into—and control of—the networked microservices in your service mesh.

Built on the open source Istio project, OpenShift Service Mesh provides additional functionality with the inclusion of other open source projects like Kiali (Istio console) and Jaeger (distributed tracing), which supports collaboration with leading members of the Istio community. OpenShift Service Mesh helps developers increase productivity by integrating communications policies without changing application code or integrating language-specific libraries.

Red Hat OpenShift Service Mesh is preinstalled, tested, and optimized for Red Hat OpenShift. It provides compatibility with OpenShift-specific features like operators and continuous integration and continuous delivery (CI/CD) pipelines. It comes with Red Hat’s enterprise support and is regularly updated and patched for security and stability. OpenShift Service Mesh works across multiple Red Hat OpenShift clusters, creating consistency across hybrid cloud or multicloud environments.