Circuit breaker pattern

Long Vu
2 min readNov 17, 2024

--

The Circuit Breaker pattern works like an electrical circuit breaker by stopping interactions between services when it detects a high rate of failures, such as timeouts or server overloads. This interruption allows the failing service to recover while maintaining overall system stability.

The pattern has three stages: Closed, Open, and half-open.

How it work

In which locations is a circuit breaker implemented?

  • Microservices: Be implemented in the API gateway layer to manage downstream services and all circuit breaker policies.
    Example: Kong Gateway, Netflix Zuul + Hystrix
[Client] → [API Gateway + Circuit Breaker] → [Microservices]
  • Service to service: Be implemented in each service to protect external service calls.
    Libraries: resilient4j, Hystrix, Opossum, Polly, etc.
[Service A] → [Circuit Breaker] → [Service B]
  • Database Layer: Be implemented in the database layer to protect against database overload and handle connect pooling issues.
[Service] → [Circuit Breaker] → [Database]
  • Infrastructure Layer: Be implemented in a service mesh(example: Istio) to manage traffic and resilient patterns.
[Services] → [Service Mesh + Circuit Breaker] → [External Services]

Key best practices for circuit breaker.

Configuration:

One circuit breaker per dependency
Set proper timeout values
Configure failure thresholds wisely
Set appropriate recovery time


Fallback:

Always have a fallback plan
Use cached data when possible
Return sensible default values
Handle partial failures


Monitoring:

Track circuit states
Monitor failure rates
Set up alerts
Log important events


Testing:

Test all states (Open/Closed/Half-Open)
Test failure scenarios
Verify fallback behavior
Include in CI/CD


Implementation:

Keep it simple
Use proven libraries
Separate from business logic
Document configurations

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Long Vu
Long Vu

Written by Long Vu

Product builder, Engineering Manager, AI enthusiastic

No responses yet