Microservices: Design Patterns

In essence, a design pattern is about describing a reusable solution to a problem when given a specific context. Using a tried-and-tested solution from a design pattern can save a lot of time and increase the quality of the implementation compared to spending time inventing the solution ourselves.

Some examples of design patterns in microservices

1. Service Discovery
2. Edge Server
3. Reactive Microservices
4. Central Configuration
5. Centralized Log Analysis
6. Distributed Tracing
7. Circuit Breaker
8. Control Loop
9. Centralized Monitoring and Alarms

Each is described lightly below: the problem, a solution, and the requirements for the solution.

Service Discovery

Problem: How can clients find microservices and their instances? Microservices instances are typically assigned dynamically allocated IP addresses when they start up (e.g., when running in containers), making it hard for a client to make a request to a microservice that exposes, say, a REST API over HTTP.

Solution: Add a new component — a service discovery service — to the system landscape, which keeps track of currently available microservices and the IP addresses of their instances.

Solution requirements:

• Automatically register/unregister microservices and their instances as they come and go.
• The client must be able to make a request to a logical endpoint for the microservice; the request is routed to one of the available instances.
• Requests to a microservice must be load-balanced over the available instances.
• We must be able to detect instances that are currently unhealthy, so requests aren’t routed to them.

Two implementation strategies: client-side routing (the client uses a library that communicates with the service discovery service to find the proper instances) and server-side routing (the service discovery infrastructure also exposes a reverse proxy that all requests go through, which forwards them to a proper microservice instance on behalf of the client).

Edge Server

Problem: In a system landscape of microservices, it’s often desirable to expose some microservices to the outside while hiding the rest from external access. The exposed microservices must be protected against requests from malicious clients.

Solution: Add a new component, an edge server, that all incoming requests go through. An edge server typically behaves like a reverse proxy and can be integrated with a discovery service for dynamic load-balancing.

Solution requirements:

• Hide internal services that shouldn’t be exposed outside their context — only route requests to microservices configured to allow external requests.
• Expose external services and protect them from malicious requests — use standard protocols and best practices such as OAuth, OIDC, JWT tokens, and API keys to ensure clients are trustworthy.

Reactive Microservices

Problem: Traditionally, Java developers implement synchronous communication using blocking I/O (e.g., a RESTful JSON API over HTTP). Blocking I/O means a thread is allocated from the OS for the length of the request. If concurrent requests go up, a server might run out of available threads, causing longer response times or crashes. A microservice architecture typically makes this worse, since a chain of cooperating microservices is often used to serve a request — the more microservices involved, the faster available threads drain.

Solution: Use non-blocking I/O so no thread is allocated while waiting for processing in another service (a database or another microservice).

Solution requirements:

• Whenever feasible, use an asynchronous programming model — send messages without waiting for the receiver to process them.
• If a synchronous model is preferred, use reactive frameworks that execute synchronous requests using non-blocking I/O, without allocating a thread while waiting for a response.
• Microservices must be resilient (capable of producing a response even if a dependency fails) and self-healing (able to resume using a dependency once it’s operational again).

Central Configuration

Problem: A large number of deployed microservice instances raises questions: how do you get a complete picture of the configuration in place across all running instances, and how do you update configuration while ensuring all affected instances are updated correctly?

Solution: Add a central configuration service that stores the configuration of all the microservices; each microservice queries it for its configuration on startup.

Solution requirements: Store configuration for a group of microservices in one place, with different settings per environment (development, test, production).

Centralized Log Analysis

Problem: Traditionally, an application writes log events to local log files. With a large number of microservice instances spread across many servers: how do you get an overview of what’s going on across the landscape? How do you detect instances writing error messages? If end users report problems, how do you find related log messages and identify the root-cause instance?

Solution: Add a component for centralized logging, capable of:

• Detecting new microservice instances and collecting log events from them.
• Interpreting and storing log events in a structured, searchable way in a central database.
• Providing APIs and graphical tools for querying and analyzing log events.

Solution requirements:

• Microservices stream log events to standard output (stdout), rather than microservice-specific log files — this makes it easier for a log collector to find the events.
• Microservices tag log events with a correlation ID (see Distributed Tracing below).
• A canonical log format is defined, so log collectors can transform events from different microservices into that format before storing them — required for querying and analysis across the collected data.

Distributed Tracing

Problem: It must be possible to track requests and messages flowing between microservices while processing an external request. For example: if end users file support cases about a specific failure, how do you identify the root-cause microservice? If a support case mentions a specific order number, how do you find all log messages related to processing that order across every microservice involved? If users report unacceptably long response times, how do you identify which microservice in the call chain is causing the delay?

Solution: Ensure all related requests and messages are marked with a common correlation ID that’s part of every log event. Using the correlation ID, the centralized logging service can find all related events — and if a log event includes a business identifier (customer, product, order ID), you can find all related events for that identifier too. To analyze delays in a call chain, collect timestamps for when requests, responses, and messages enter and exit each microservice.

Solution requirements:

• Assign unique correlation IDs to all incoming or new requests/events in a well-known place, like a standardized header.
• When a microservice makes an outgoing request or sends a message, it must add the correlation ID to it.
• All log events must include the correlation ID in a predefined format, so the centralized logging service can extract and index it.
• Trace records must be created for when requests, responses, and messages enter or exit a microservice instance.

Circuit Breaker

Problem: A system landscape using synchronous intercommunication can be exposed to a chain of failures. If one microservice stops responding, its clients can get into problems too, and the failure can propagate recursively, taking out major parts of the system. This is especially common with blocking I/O — combined with a large number of concurrent requests and a slowly-responding service, thread pools can drain quickly, causing callers to hang or crash, and the failure can spread to the caller’s caller, and so on.

Solution: Add a circuit breaker that prevents new outgoing requests from a caller if it detects a problem with the service it calls.

Solution requirements:

• Open the circuit and fail fast (without waiting for a timeout) if problems are detected.
• Probe for failure correction (a half-open circuit) — allow a single request through periodically to check if the service is healthy again.
• Close the circuit if the probe detects normal operation. This makes the system resilient and self-healing.

Control Loop

Problem: In a system landscape with many microservice instances spread across many servers, it’s very difficult to manually detect and correct problems like crashed or hung instances.

Solution: Add a control loop component. It constantly observes the actual state of the system landscape, comparing it with a desired state specified by the operators. If the two states differ, it takes action to make the actual state match the desired one.

In the world of containers, a container orchestrator such as Kubernetes can be used to implement a control loop.

Centralized Monitoring and Alarms

Problem: If observed response times or hardware resource usage become unacceptably high, it can be very hard to discover the root cause — for example, you need to analyze hardware resource consumption per microservice.

Solution: Add a monitor service capable of collecting metrics about hardware resource usage at each microservice instance level.

Solution requirements:

• Collect metrics from all servers used by the system landscape, including autoscaling servers.
• Detect new microservice instances as they launch and start collecting metrics from them.
• Provide APIs and graphical tools for querying and analyzing collected metrics.
• Support defining alerts triggered when a specified metric exceeds a threshold.