This site is in English. Use your browser's built-in translate feature to read it in your language.
DIM Data Integration Mastery™
Home › Patterns › Resilience Patterns › Circuit Breaker Pattern

Circuit Breaker Pattern

Overview

The Circuit Breaker pattern is a critical stability pattern in distributed systems architecture, designed to provide automatic protection against cascading failures and resource exhaustion. Named after electrical circuit breakers that protect electrical circuits from damage caused by overcurrent, the software Circuit Breaker pattern serves as an intelligent intermediary that monitors the health and availability of external service dependencies.

Theoretical Foundation

The Circuit Breaker pattern is rooted in fault tolerance theory and systems reliability engineering. It addresses the fundamental challenge of partial failure handling in distributed systems, where individual components can fail independently without bringing down the entire system. The pattern embodies the principle of "failing fast" - immediately returning an error when a dependency is known to be unavailable, rather than waiting for inevitable timeouts.

Core Principles

1. Isolation of Failure

The Circuit Breaker acts as a bulkhead, isolating failures in one service from affecting other parts of the system. This prevents the "domino effect" where one failing component triggers a cascade of failures throughout the distributed architecture.

2. Resource Protection

By preventing futile calls to failing services, the pattern protects valuable system resources: - Thread pools from being exhausted by blocked calls - Connection pools from being depleted by hanging connections - Memory from accumulating timeout exceptions - Network bandwidth from repeated failed requests

3. Temporal Failure Handling

The pattern recognizes that many failures in distributed systems are temporal - services may recover on their own given sufficient time. The Circuit Breaker provides a mechanism for automatic recovery detection without requiring manual intervention.

4. Graceful Degradation

Rather than complete system failure, the Circuit Breaker enables graceful degradation where the system continues to operate with reduced functionality when dependencies are unavailable.

Why Circuit Breakers are Essential in Integration Architecture

1. Network Unreliability

In integration scenarios, services communicate across unreliable networks where: - Latency can vary unpredictably - Packet loss may occur intermittently
- Network partitions can temporarily isolate services - DNS resolution failures can prevent service discovery

2. Third-Party Service Dependencies

Modern integration architectures often depend on external services that are: - Outside organizational control - SaaS providers, payment gateways, authentication services - Subject to rate limiting - API quotas and throttling mechanisms - Experiencing variable load - Performance degradation during peak usage - Under maintenance - Scheduled and unscheduled downtime

3. Distributed System Complexity

In distributed architectures, the "distributed monolith" anti-pattern can emerge where: - Deep service chains create multiple failure points - Synchronous communication amplifies latency and failure impact - Service interdependencies create complex failure scenarios - Load balancing may not account for service health

4. Integration Pattern Challenges

Common integration patterns face specific challenges that Circuit Breakers address:

Benefits in Integration Contexts

1. Immediate Failure Detection

2. Resource Optimization

3. System Resilience

4. Operational Excellence

5. Business Continuity

Integration Architecture Applications

1. API Gateway Pattern

Circuit Breakers in API gateways protect against: - Backend service failures affecting multiple API consumers - Rate limit violations from overwhelmed downstream services - Authentication service outages blocking all API access

2. Service Mesh Integration

In service mesh architectures, Circuit Breakers provide: - Sidecar-level protection for all service-to-service communication - Policy-driven configuration for different service tiers and criticality levels - Distributed circuit breaker state shared across service instances

3. Event-Driven Architectures

For asynchronous integration patterns: - Event processor protection when downstream event handlers fail - Dead letter queue integration for permanently failed event processing - Backpressure management when event consumers cannot keep up

4. Legacy System Integration

When integrating with legacy systems: - Protection against legacy system instability and unpredictable performance - Gradual migration support by providing fallbacks during system transitions - Resource consumption control for resource-constrained legacy environments

Relationship to Other Resilience Patterns

The Circuit Breaker pattern works synergistically with other resilience patterns:

This layered approach creates defense in depth for distributed system reliability.

How Circuit Breaker Works

The Circuit Breaker pattern operates like an electrical circuit breaker in three distinct states:

States

1. CLOSED State (Normal Operation)

2. OPEN State (Failing Fast)

3. HALF-OPEN State (Testing Recovery)

State Transition Diagram

    success < threshold
    ┌─────────────────┐
    │     CLOSED      │
    │ (Normal Ops)    │
    └─────────────────┘
           │ │
  failures │ │ reset success
  > thresh │ │
           ▼ │
    ┌─────────────────┐
    │      OPEN       │◄──── test request fails
    │  (Fail Fast)    │
    └─────────────────┘
           │
           │ timeout elapsed
           ▼
    ┌─────────────────┐
    │   HALF-OPEN     │
    │ (Testing)       │
    └─────────────────┘
           │
           └──── test request succeeds

Key Components

1. Failure Counter

Tracks consecutive failures and success rates:

public class FailureCounter {
    private int consecutiveFailures = 0;
    private int totalRequests = 0;
    private int failures = 0;

    public void recordSuccess() {
        consecutiveFailures = 0;
        totalRequests++;
    }

    public void recordFailure() {
        consecutiveFailures++;
        failures++;
        totalRequests++;
    }

    public double getFailureRate() {
        return totalRequests > 0 ? (double) failures / totalRequests : 0.0;
    }
}

2. State Manager

Controls circuit state transitions:

public enum CircuitState {
    CLOSED,   // Normal operation
    OPEN,     // Failing fast
    HALF_OPEN // Testing recovery
}

public class CircuitBreakerStateManager {
    private CircuitState currentState = CircuitState.CLOSED;
    private long lastFailureTime;
    private int testRequestCount = 0;

    public boolean shouldAllowRequest(CircuitBreakerConfig config) {
        switch (currentState) {
            case CLOSED:
                return true;
            case OPEN:
                if (hasRecoveryTimeoutElapsed(config)) {
                    transitionToHalfOpen();
                    return true;
                }
                return false;
            case HALF_OPEN:
                return testRequestCount < config.getMaxTestRequests();
        }
        return false;
    }
}

3. Recovery Timer

Manages timeout periods and recovery testing:

public class RecoveryTimer {
    private final long recoveryTimeoutMs;
    private long lastFailureTimestamp;

    public boolean isRecoveryTimeoutElapsed() {
        return (System.currentTimeMillis() - lastFailureTimestamp) >= recoveryTimeoutMs;
    }

    public void recordFailure() {
        this.lastFailureTimestamp = System.currentTimeMillis();
    }
}

Configuration Parameters

Essential Settings

Parameter Description Typical Values
Failure Threshold Number of consecutive failures to trigger OPEN state 3-10
Recovery Timeout Time to wait before testing recovery (HALF-OPEN) 30s-300s
Request Volume Threshold Minimum requests before calculating failure rate 10-50
Success Threshold Successful requests needed to close circuit 1-5

Example Configuration

# Circuit breaker for Service A
circuit-breaker.service-a.failure-threshold=5
circuit-breaker.service-a.recovery-timeout=60s
circuit-breaker.service-a.request-volume-threshold=20
circuit-breaker.service-a.success-threshold=3

# Circuit breaker for Service B
circuit-breaker.service-b.failure-threshold=3
circuit-breaker.service-b.recovery-timeout=30s
circuit-breaker.service-b.request-volume-threshold=10
circuit-breaker.service-b.success-threshold=2

Implementation Examples

1. Basic Circuit Breaker Implementation

@Component
public class BasicCircuitBreaker {
    private final CircuitBreakerConfig config;
    private final FailureCounter failureCounter;
    private final CircuitBreakerStateManager stateManager;
    private final RecoveryTimer recoveryTimer;

    public <T> T execute(Supplier<T> operation, Supplier<T> fallback) {
        if (!stateManager.shouldAllowRequest(config)) {
            // Circuit is OPEN - fail fast
            return fallback.get();
        }

        try {
            T result = operation.get();
            onSuccess();
            return result;
        } catch (Exception e) {
            onFailure(e);
            return fallback.get();
        }
    }

    private void onSuccess() {
        failureCounter.recordSuccess();
        if (stateManager.getCurrentState() == CircuitState.HALF_OPEN) {
            // Enough successes to close circuit
            if (failureCounter.getConsecutiveSuccesses() >= config.getSuccessThreshold()) {
                stateManager.transitionToClosed();
            }
        }
    }

    private void onFailure(Exception e) {
        failureCounter.recordFailure();
        recoveryTimer.recordFailure();

        if (failureCounter.getConsecutiveFailures() >= config.getFailureThreshold()) {
            stateManager.transitionToOpen();
        }
    }
}

2. Integration with HTTP Client

@Service
public class ExternalServiceClient {
    private final RestTemplate restTemplate;
    private final BasicCircuitBreaker circuitBreaker;

    public ContactResponse updateContact(ContactRequest request) {
        return circuitBreaker.execute(
            // Primary operation
            () -> restTemplate.postForObject("/contact/update", request, ContactResponse.class),
            // Fallback operation
            () -> createErrorResponse("External service unavailable - circuit breaker OPEN")
        );
    }

    private ContactResponse createErrorResponse(String message) {
        return ContactResponse.builder()
            .status("ERROR")
            .errorCode("CIRCUIT_BREAKER_OPEN")
            .errorMessage(message)
            .retryable(false)
            .build();
    }
}

3. Quarkus Implementation with MicroProfile

@ApplicationScoped
public class QuarkusCircuitBreakerService {

    @CircuitBreaker(
        failureRatio = 0.5,           // 50% failure rate threshold
        requestVolumeThreshold = 10,   // Minimum 10 requests
        delay = 60000,                 // 60 second recovery timeout
        successThreshold = 3           // 3 successes to close circuit
    )
    @Retry(
        maxRetries = 3,
        delay = 1000,
        delayUnit = ChronoUnit.MILLIS
    )
    @Timeout(value = 30, unit = ChronoUnit.SECONDS)
    public String callExternalService(String data) {
        // External service call
        return externalServiceClient.processData(data);
    }

    @Fallback
    public String fallbackResponse(String data) {
        return "Service temporarily unavailable. Please try again later.";
    }
}

Integration Patterns

1. Circuit Breaker with Retry

Combine Circuit Breaker with Retry for comprehensive resilience:

@Service
public class ResilientServiceClient {

    public ResponseEntity<String> callWithRetryAndCircuitBreaker(RequestData data) {
        return circuitBreaker.execute(
            () -> retryTemplate.execute(context -> 
                restTemplate.postForEntity("/api/process", data, String.class)
            ),
            () -> ResponseEntity.status(503)
                .body("Service unavailable - please try again later")
        );
    }
}

2. Per-Endpoint Circuit Breakers

Separate circuit breakers for different service endpoints:

@Component
public class MultiEndpointCircuitBreaker {
    private final Map<String, BasicCircuitBreaker> circuitBreakers = new ConcurrentHashMap<>();

    public <T> T executeForEndpoint(String endpoint, Supplier<T> operation, Supplier<T> fallback) {
        BasicCircuitBreaker cb = circuitBreakers.computeIfAbsent(
            endpoint, 
            key -> new BasicCircuitBreaker(getConfigForEndpoint(key))
        );
        return cb.execute(operation, fallback);
    }
}

3. Monitoring and Metrics

@Component
public class CircuitBreakerMetrics {
    private final MeterRegistry meterRegistry;

    public void recordCircuitBreakerState(String serviceName, CircuitState state) {
        Gauge.builder("circuit_breaker_state")
            .tag("service", serviceName)
            .tag("state", state.name())
            .register(meterRegistry, state.ordinal());
    }

    public void incrementCircuitBreakerTransition(String serviceName, String fromState, String toState) {
        Counter.builder("circuit_breaker_transitions")
            .tag("service", serviceName)
            .tag("from", fromState)
            .tag("to", toState)
            .register(meterRegistry)
            .increment();
    }
}

Best Practices

1. Configuration Guidelines

2. Fallback Strategies

3. Monitoring and Alerting

4. Testing Strategies

Common Pitfalls

1. Inappropriate Timeouts

2. Shared Circuit Breakers

3. Lack of Monitoring

4. Inadequate Fallbacks

Integration in Distributed Systems

In distributed integration scenarios, Circuit Breaker protects against:

External System Failures

// Service A Integration
@CircuitBreaker(name = "service-a")
public ContactResponse updateServiceAContact(ContactRequest request) {
    return serviceAClient.updateContact(request);
}

// Service B Integration  
@CircuitBreaker(name = "service-b")
public ContactResponse updateServiceBContact(ContactRequest request) {
    return serviceBClient.updateContact(request);
}

Compensation Transaction Protection

// Rollback operations also need protection
@CircuitBreaker(name = "service-a-rollback")
public void rollbackServiceAContact(String contactId, ContactData originalData) {
    serviceAClient.revertContact(contactId, originalData);
}

Conclusion

The Circuit Breaker pattern is essential for building resilient distributed systems. It provides:

When properly implemented and configured, Circuit Breaker significantly improves system reliability and user experience during service disruptions.

References

← Back to All Patterns