API Gateway Explained for System Design Interviews
How to explain and defend an API gateway in a system design interview — what it actually does as the single entry point to a microservices backend (routing, authentication, rate limiting, TLS termination, aggregation, observability), how it differs from a load balancer and a reverse proxy, the failure modes interviewers probe (the gateway as a single point of failure, latency, the fat-gateway anti-pattern), and the BFF pattern — with the follow-ups you answer out loud.
November 21, 202615 min read

API gateways for system design interviews, explained
Sketch a microservices design on the whiteboard and you will draw a single box where every client request lands before it reaches any service — and the moment you label it "API gateway," the interviewer wants to know what that box actually does, and why it is not just a load balancer with a fancier name. An API gateway is the single entry point in front of a backend of many services: it takes every incoming request, decides which service should handle it, and along the way does the work that would otherwise be copy-pasted into every service — authentication, rate limiting, TLS termination, logging. It is a favourite system design interview concept precisely because "put a gateway in front" is easy to say and the real understanding lives in what it offloads, how it differs from the other boxes that look like it, and what breaks when all your traffic flows through one component.
The reason it shows up in almost every microservices round is that it sits on the boundary between clients and a fleet of independently-evolving services, and that boundary needs governance. Without a gateway, every client must know the address of every service, and every service must re-implement auth, throttling and logging; with one, clients hit a single stable endpoint and the cross-cutting concerns live in one place. But a single front door is also a single point of failure and a latency hop, and an interviewer will push on exactly those. This guide walks the things a strong candidate names out loud: what an API gateway actually does, how it differs from a load balancer and a reverse proxy, the failure modes the follow-ups target, and the BFF pattern that extends it.
Why API gateways come up in system design interviews
An API gateway rarely gets its own whiteboard prompt. It appears inside a larger design — a food-delivery app, a payment system, any microservices backend — at the moment you split a monolith into services and realise the client can no longer talk to each one directly. You draw a box labelled "API gateway," and the interviewer leans in: not whether to add one, but what it handles, what it does not, and what happens when it goes down. The gateway is a favourite probe because "add a gateway" is the easy half of the sentence and everything true and hard — the offloaded concerns, the single-point-of-failure story, the thin-vs-fat line — lives in the second half.
The interviewer rarely lets "requests go through the API gateway" stand on its own. "You've put every client request through one gateway. What exactly does it do to a request before it reaches a service — and what happens to your entire API the moment that gateway falls over?" That question — the responsibilities you offload and the risk you concentrate — is the one a memorised "it routes requests" answer can never survive, and the one this topic is really testing.
The rounds on Zynter are modelled on how the loop runs, not scraped from any candidate's recall. What matters for prep is the shape: you name the gateway, and the next sentence out of the interviewer's mouth asks what it offloads or how it stays available.
What the interviewer is really testing
An API-gateway answer is read on the same four dimensions as any design decision. Problem solving — did you use the gateway to centralise cross-cutting concerns, or just redraw a load balancer? Communication — can you say in one sentence how a gateway differs from a reverse proxy and a load balancer? Technical execution — do you handle its availability (run several behind a load balancer) and keep it thin rather than stuffing business logic into it? Time management — did you spend your words on routing, auth, aggregation and the SPOF story, not on listing every feature a vendor ships? Reading this guide teaches the concept; it does not teach you to defend it while someone asks what happens when the gateway dies.
What an API gateway actually does
Start with the job, because "single entry point" is only the headline. A gateway takes every client request and, before (or instead of) forwarding it to a service, performs the cross-cutting work that every service would otherwise duplicate:
- Routing — map the incoming request (by path, host, or version) to the right backend service, so the client addresses one endpoint and never needs to know the service topology.
- Authentication & authorization — verify the caller once (validate a JWT or session, check scopes) at the edge, so each service trusts an already-authenticated request instead of re-implementing login.
- Rate limiting & throttling — enforce per-client quotas centrally to protect the backend from abuse and overload.
- TLS termination — decrypt HTTPS at the gateway so backend services can speak plain HTTP internally, and certificates live in one place.
- Aggregation / composition — fan one client request out to several services and combine the responses, so a mobile screen makes one call instead of six.
- Observability & policy — centralised logging, metrics, tracing headers, request validation, and response transformation.
The sentence that scores: the gateway is where the concerns that are identical across every service — auth, rate limiting, TLS, logging — live once, so the services stay focused on business logic and the client sees one stable, governed entrance. Naming aggregation specifically is a nice signal, because it is the one responsibility a plain proxy does not have.
API gateway vs load balancer vs reverse proxy
This is the comparison the interview turns on, because all three sit in front of your servers and look alike on a diagram. The distinction is what layer they work at and what decision they make:
- Load balancer — distributes incoming traffic across a pool of identical backend servers for scalability and availability, using an algorithm (round-robin, least-connections) and health checks to avoid dead servers. It cares about which replica, not what the request is.
- Reverse proxy — sits in front of servers and manages and protects the connection: TLS termination, caching, compression, request rewriting, security filtering. It fronts backends generically; a load balancer is a kind of reverse proxy, but a reverse proxy is not necessarily balancing.
- API gateway — an application-aware reverse proxy specialised for an API over many services: it routes by API/path/version to different services and adds the API-management concerns (auth, rate limiting, versioning, aggregation, per-consumer policy). It cares about what the request is and who is calling.
Say the distinction crisply: a load balancer spreads traffic across copies of one service; a reverse proxy fronts and protects servers; an API gateway is an API-aware front door that routes to many different services and enforces cross-cutting policy. They are not alternatives — a real system often has all three, with a gateway sitting behind a load balancer. And because the gateway is itself a service, you run several instances of it and put a load balancer in front of them, which is exactly how the three compose.
Failure modes: the gateway as a single point of failure
A single front door is a single point of failure, and this is where most of the follow-ups land, because routing all traffic through one component concentrates both risk and latency.
Availability — it must not be the SPOF. Every request flows through the gateway, so if it dies, your whole API dies. The fix is the obvious one, said plainly: run multiple gateway instances behind a load balancer (or DNS/anycast), across availability zones, so one failing instance is removed and traffic shifts — the gateway tier is made redundant exactly like any other critical service. Health checks and graceful draining apply here too.
Latency — it adds a hop. The gateway sits on the critical path of every request, so its own processing (auth checks, transforms) is pure added latency. You keep it thin and fast, cache what you can (e.g. token validation results), and avoid heavy synchronous work in the request path.
The fat-gateway anti-pattern. The tempting mistake is to pour business logic into the gateway until it becomes a distributed monolith that every team must coordinate to change. Keep it to cross-cutting concerns — routing, auth, rate limiting, aggregation — and leave domain logic in the services. Add circuit breakers and timeouts so a slow or failing downstream service does not pile up connections at the gateway and drag everything down.
The point to state plainly: the gateway concentrates traffic, so you pay for it with redundancy (to survive its failure), discipline (to keep it thin), and resilience (circuit breakers for bad downstreams). Naming all three — not just "we'll run two of them" — is what reads as experience.
The BFF pattern and where the gateway fits
"Add a gateway" is ambiguous until you say whether you mean one gateway for everyone or one tuned per client — and a strong answer knows the variations:
- Backend-for-frontend (BFF) — instead of a single gateway serving web, mobile and third-party clients identically, you run a gateway per client type. A mobile BFF aggregates and trims responses for a small screen on a flaky network; a web BFF shapes data for a browser. Each client gets a tailored front door without bloating one shared gateway with every client's special-casing.
- Gateway offloading — the general principle: move a cross-cutting concern (auth, TLS, rate limiting, caching) out of every service and into the shared edge, so it is implemented and operated once.
- When you actually need all three — a small app can combine load balancing, reverse-proxy duties and gateway logic in one component (an NGINX, say). As services and security requirements grow, teams split them out, because each solves a distinct problem; saying "you don't need all three on day one" is a maturity signal.
The useful framing: the gateway is where you centralise the concerns that are the same for every service, and the BFF is what you reach for when "every client" stops being true. Naming the BFF variation, rather than assuming one monolithic gateway, reads as someone who has shipped more than a diagram.
How to talk about API gateways in an interview
The probes from every section rhyme. Master these five and you can defend a gateway in any microservices design, not just this checklist.
- "What does the API gateway actually do to a request?" → Routes it to the right service and applies the cross-cutting concerns in one place — authentication, rate limiting, TLS termination, logging, and sometimes aggregation of several services into one response. (problem solving)
- "How is it different from a load balancer?" → A load balancer spreads traffic across identical copies of one service; an API gateway is application-aware and routes by path/version to many different services while enforcing API policy. They often coexist, gateway behind the load balancer. (communication)
- "The gateway goes down — what happens to your API?" → Everything behind it is unreachable, so you never run one: multiple gateway instances behind a load balancer across zones, with health checks, so a failed instance is dropped and traffic shifts. (technical execution)
- "Isn't the gateway just added latency?" → It is a hop, so you keep it thin and fast, cache token validation, and avoid heavy synchronous work — and you add circuit breakers so a slow downstream doesn't back up at the gateway. (technical execution)
- "Web and mobile need different responses — one gateway?" → Reach for a backend-for-frontend: a gateway per client type that aggregates and shapes responses for that client, instead of bloating one shared gateway. (problem solving)
Each maps to a debrief dimension. Reading them teaches the answers; it does not teach you to deliver them while someone interrupts with "and when the gateway dies?" — that only comes from saying them out loud.
How to prepare: a short plan
You do not need a new topic for every design. You need the gateway vocabulary automatic, so when a design splits into services you reach for the right front door without thinking.
- Day 1: Learn what the gateway offloads — routing, auth, rate limiting, TLS termination, aggregation, observability — until you can list them and say why each belongs at the edge rather than in every service.
- Day 2: Drill the comparison (gateway vs load balancer vs reverse proxy) and the failure story (SPOF → multiple instances behind a load balancer, thin gateway, circuit breakers). These are where the follow-ups live.
- Day 3: Place a gateway inside a real microservices design (a food-delivery app, a payment flow): route the client to services, offload auth and rate limiting, reach for a BFF where clients differ, and make it redundant. Then do it out loud, timed, with someone asking what happens when it falls over.
This explainer feeds the broader Top 25 system design interview questions — a gateway sits at the front of almost every microservices design, so getting it automatic pays off across the whole list.
Frequently asked questions
What does an API gateway do in system design?
An API gateway is the single entry point in front of a microservices backend. It receives every client request and routes it to the correct service, and along the way it handles the cross-cutting concerns that would otherwise be duplicated in every service: authentication and authorization (validating a token once at the edge), rate limiting and throttling, TLS termination, request validation and transformation, observability (logging, metrics, tracing), and often aggregation — fanning one client request out to several services and combining the responses. The result is that clients talk to one stable, governed endpoint while the backend services stay focused on business logic and can evolve independently.
What is the difference between an API gateway and a load balancer?
They solve different problems and frequently coexist. A load balancer distributes incoming traffic across a pool of identical backend servers to improve scalability and availability, choosing a replica with an algorithm like round-robin or least-connections and using health checks to avoid dead servers — it cares about which copy handles the request, not what the request is. An API gateway is application-aware: it routes a request by its path, host or version to one of many different services and enforces API-level policy such as authentication, rate limiting and aggregation. In a real system you often have both, with the API gateway running as several instances behind a load balancer.
Is an API gateway the same as a reverse proxy?
An API gateway behaves like a reverse proxy at the network level — both sit in front of backend servers and forward requests — but the gateway is a specialised, application-aware reverse proxy focused on API management. A plain reverse proxy fronts and protects servers with concerns like TLS termination, caching, compression and request rewriting, treating the backend generically. An API gateway adds the API-consumer concerns on top: routing to different services by API, authentication and authorization, rate limiting, versioning, and aggregating multiple services into one response. So every API gateway is a reverse proxy, but a reverse proxy is not necessarily an API gateway.
Why is an API gateway a single point of failure, and how do you fix it?
Because every client request flows through it, so if the single gateway instance fails, the entire API becomes unreachable even when the backend services are perfectly healthy. The fix is to make the gateway tier redundant just like any other critical component: run multiple gateway instances behind a load balancer (and across availability zones), with health checks so a failed instance is removed from rotation and traffic shifts to the healthy ones. You also keep the gateway thin and fast to limit the latency it adds on the critical path, and add circuit breakers and timeouts so a slow or failing downstream service cannot pile up connections at the gateway and bring it down.
How do I practise a system design interview out loud?
Reading about API gateways is not the same as defending one while an interviewer asks what happens when it dies. On Zynter, the HLD track runs live voice rounds modelled on how real system design loops run — you talk through a design on a shared canvas, and a voice interviewer follows up one question at a time ("what does the gateway offload?", "how is it different from a load balancer?", "the gateway falls over — now what?"), then gives you a debrief on problem solving, communication, technical execution and time management. The score is a practice signal, not a hiring decision.
Practise it on Zynter
You can define an API gateway. The round is won on defending what it offloads and how it survives failure.
Pick an HLD system design round, put a gateway in front of your services on the canvas, and defend it out loud to a voice interviewer that keeps asking "what does it do that a load balancer doesn't?" and "the gateway just died — what happens to your API?" — then read the debrief. Zynter.ai is on-demand mock interviews modelled on real company loops, with 150+ rounds across 20 companies including the HLD system design track (Sep 2026).
Practise an HLD system design round → Browse all interview tracksAlmost nobody freezes on the code. They freeze on the follow-up.
Answer a real round to a voice interviewer that keeps asking why. No scheduling, no subscription, first session ready in under a minute.
Keep reading
All posts
ACID Properties in DBMS Explained for Interviews
How to explain ACID properties in a system design interview — what atomicity, consistency, isolation and durability each guarantee for a database transaction, the isolation levels and the read anomalies they prevent (dirty, non-repeatable and phantom reads), ACID vs BASE and when you relax the guarantees for scale, and how databases implement them with write-ahead logs and locking or MVCC — with the follow-ups you answer out loud.
Gowtham PotureddiNov 21, 202615 min read

Design a Live Video Comments System: System Design Interview
How to design a live video comments system in a system design interview, the way the round runs — requirements, the real-time fan-out architecture, and the deep dives that decide it: why you push with SSE instead of polling and separate read from write traffic, how pub/sub with viewer co-location fans a comment to the right servers, and how mega-streams switch to sampling and CDN snapshots when millions watch at once. Built around the follow-ups an interviewer pushes.
Gowtham PotureddiNov 21, 202621 min read

Design Spotify: Music Streaming System Design Interview
ow to design Spotify in a system design interview, the way the round runs — requirements, the control-plane and data-plane split, and the deep dives that decide it: how audio is chunked, served from a CDN via short-lived signed URLs with next-track prefetch for gapless playback, how the huge read-heavy catalog and collaborative playlists are stored and cached, and how Discover Weekly is batch-computed from listening signals. Built around the follow-ups an interviewer pushes.
Gowtham PotureddiNov 21, 202619 min read