The Circuit Breaker pattern works like an electrical circuit breaker by stopping interactions between services when it detects a high rate of failures, such as timeouts or server overloads. This interruption allows the failing service to recover while maintaining overall system stability.
The pattern has three stages: Closed, Open, and half-open.
How it work
In which locations is a circuit breaker implemented?
- Microservices: Be implemented in the API gateway layer to manage downstream services and all circuit breaker policies.
Example: Kong Gateway, Netflix Zuul + Hystrix
[Client] → [API Gateway + Circuit Breaker] → [Microservices]- Service to service: Be implemented in each service to protect external service calls.
Libraries: resilient4j, Hystrix, Opossum, Polly, etc.
[Service A] → [Circuit Breaker] → [Service B]- Database Layer: Be implemented in the database layer to protect against database overload and handle connect pooling issues.
[Service] → [Circuit Breaker] → [Database]- Infrastructure Layer: Be implemented in a service mesh(example: Istio) to manage traffic and resilient patterns.
[Services] → [Service Mesh + Circuit Breaker] → [External Services]Key best practices for circuit breaker.
Configuration:
One circuit breaker per dependency
Set proper timeout values
Configure failure thresholds wisely
Set appropriate recovery time
Fallback:
Always have a fallback plan
Use cached data when possible
Return sensible default values
Handle partial failures
Monitoring:
Track circuit states
Monitor failure rates
Set up alerts
Log important events
Testing:
Test all states (Open/Closed/Half-Open)
Test failure scenarios
Verify fallback behavior
Include in CI/CD
Implementation:
Keep it simple
Use proven libraries
Separate from business logic
Document configurations
