Problem
You are building an API Gateway that serves as the single entry point for a microservices architecture. The gateway must route incoming HTTP requests to the correct backend service based on the request path, method, and headers.
Requirements
- Path-based routing: Route requests to different services based on URL path prefixes (e.g.,
/api/users/ goes to the Users service, /api/orders/ goes to the Orders service).
- Header-based routing: Support routing based on custom headers (e.g.,
X-API-Version: 2 routes to v2 of a service).
- Rate limiting: Apply per-client rate limits based on API key or IP address.
- Authentication: Validate JWT tokens before forwarding requests to downstream services.
- Health checks: Monitor backend service health and remove unhealthy instances from the routing pool.
Constraints
- The gateway handles 10,000 requests per second across all services.
- Routing decisions must add less than 5ms of latency.
- The gateway should support at least 10 backend services.
- Configuration changes (adding/removing routes) should not require a gateway restart.
- Failed backend services should be detected within 30 seconds.
What to Design
- The routing table data structure and matching algorithm
- How configuration is stored and hot-reloaded
- The middleware pipeline (order of auth, rate limiting, routing)
- Health check mechanism and circuit breaking
- How you would handle request/response transformation