8. Reverse Proxies and nginx
“The app never meets the street. It only meets the receptionist.”
The story: ten thousand people and one polite server
Section titled “The story: ten thousand people and one polite server”It is 9:40 on a Sunday night. Your Next.js app runs beautifully on localhost. You have pushed it to the Oracle VPS, opened port 3000 in the firewall, and typed http://203.0.113.10:3000 into your phone. It loads. You send the link to the team. Forty minutes later someone says the page is blank, someone else says their browser shows a scary warning, and a third person says it loaded for them but took nine seconds. You have just met most of this chapter, arriving at 2 a.m. in a group chat.
Around 1999, a programmer named Dan Kegel wrote a page that became famous under a name that sounds like a bad software release: the C10K problem. The question was simple to state. How do you get one server to handle ten thousand connections at the same time? It sounds quaint now, but it was a real wall. The dominant model of the day was one thread or one process per connection. Each of those carried its own stack and bookkeeping, and most of them sat idle, waiting on a client that was slow, far away, or asleep. Ten thousand idle connections meant gigabytes of memory spent on people doing nothing at all.
Apache, which ran a large share of the web through most of that decade, grew up in the process-per-connection world. Its classic prefork design kept a pool of processes and handed each connection to one of them. It was sturdy and easy to reason about, and the bill came in memory. Apache later gained threaded and event-driven options, so this is a story about defaults and history more than destiny. But defaults shape a generation of instincts, and a lot of sysadmins still picture a web server as a crowd of busy workers, one per visitor.
The alternative was event-driven. A small number of worker processes each run a loop that asks the kernel, in effect, “which of my sockets have something to say right now?” and then handles only those. Operating systems grew good at answering that question efficiently over the following years. On Linux the mechanism is epoll, on the BSD family it is kqueue, and the older select and poll calls do the same job less well at scale.
Igor Sysoev, a Russian engineer working on the infrastructure of Rambler, a large internet portal, began writing nginx around 2002 and released it publicly around 2004, if memory serves. He wanted a server that could hold thousands of connections on modest hardware without falling over. It did that, and it spread. Reverse proxying, serving static files, and load balancing were strengths from early on, and its small, fast, event-driven design became the default answer to a question every growing site eventually asks: what should sit in front of my application?
The story has a second thread too, and it matters for the rest of this chapter. nginx was built to be small and to do one thing at a time very well. It reads a text file, it does what the file says, and it stays out of your way. That plain-text, reload-without-drama character is why, a couple of decades on, it is still the thing people reach for when they want control. Keep that in mind. Most of the pain you will feel with nginx comes from the file, not from the server.
What a reverse proxy is
Section titled “What a reverse proxy is”Every chapter from here on will carry the mantra, and this is the chapter where the receptionist finally gets a name tag:
Name → Address → Route → Door → Guard → Receptionist → App
The name is your domain. The address is the IP. The route is the path the packets take, the door is the port, and the guard is the firewall. Chapter 1 gave you the first two, chapter 3 gave you doors, and chapter 7 put your app in its office at the back, listening on 127.0.0.1:3000. The receptionist is the reverse proxy, and it is the piece that turns a crowd at the front desk into a single, well-behaved request at the back.
A forward proxy stands in front of clients. Your office internet gateway, a school’s web filter, or a corporate proxy that logs what staff browse are all forward proxies. The client knows it is using one, and the website on the other end usually sees the proxy’s address, not the person’s. The proxy acts for the people inside.
A reverse proxy stands in front of servers. The visitor talks to it, believes it is the site, and never sees the office at the back. Nothing about the visitor’s browser has to change. The receptionist holds the door, decides which office each request belongs to, and hands the reply back the way it came.
The same software can play either role. nginx, Caddy, HAProxy and Apache can all act as forward or reverse proxies, depending on which side of the conversation they stand on. The program does not change. Only the direction does.
Why put a receptionist in front of Node
Section titled “Why put a receptionist in front of Node”Your Next.js or Express app already listens for HTTP. So why not let the browser talk to it directly? You can, and for ten minutes on a laptop it is fine. On a public VPS, a reverse proxy earns its place for a short list of reasons, and you will meet each of them in the lab.
- TLS termination. The certificate and the handshake live at the door. Your app speaks plain HTTP on a private address, and nobody needs to teach Node about certificate renewal. Chapter 9 is about that work.
- Many apps, one address. The box-office site, the research SaaS, the dance-academy app and the team-status tool can all share one public IP, such as
203.0.113.10. The receptionist reads theHostheader and sends each request to the right office. - Static files. A JPEG or a built stylesheet does not need your Node process at all. nginx serves files from disk faster, with less memory, and without waking your application code.
- Compression and caching headers. One line in the proxy compresses responses for every app behind it, including the ones you forgot about.
- Buffering slow clients. This is the reason the story at the top of this chapter matters. A phone on a crowded train may take several seconds to receive a response. If Node talks to that phone directly, Node holds the connection open for as long as the phone takes. nginx reads the whole reply from your app quickly, then feeds it to the phone at the phone’s pace. Your app gets back to work almost at once.
- Rate limiting and blocking. Scanners rattle every doorknob on the internet, every day. A receptionist can slow down a client that knocks a hundred times a second, and can turn away paths that your app never serves.
- Hiding internals. The office extension, the Node version banner and the stack trace stay inside the building. The street sees one polite voice.
The world without a receptionist
Section titled “The world without a receptionist”Say you are Priya, and your ₹12,000 phone is on a Pune tower with one bar. Your team deploys the status tool with next start -p 3000, opens port 3000 in the firewall, and shares http://203.0.113.10:3000 in the group. It works on your phone, so you go to bed.
Here is what that setup quietly costs you. The URL has a port number in it, which looks odd and breaks when someone pastes it into a chat that rewrites links. There is no HTTPS, so the browser shows a warning and the lock is missing, which is a problem your users will read as “this site is dangerous”. The firewall is open on a port that was never meant to face strangers. Your Node process is the public face, so a memory leak in one page takes down the whole site. And when the second app arrives, you have no clean way to put it on the same address, because port 3000 is already taken by the first.
The 2 a.m. call usually comes in the form of a blank page. In one composite version of this story, the app restarts during a deploy, the browser gets a connection reset, a teammate in another timezone sees a stale copy, and the on-call person spends forty minutes checking DNS before anyone thinks to look at the process list. It is a composite, but variations of it happen to most people who run a public app. The receptionist does not stop bugs. It stops the street from seeing the back office while the back office is being rearranged.
nginx’s mental model
Section titled “nginx’s mental model”nginx is a small program with a simple shape. A master process starts as root, reads the configuration file, opens the privileged ports such as 80 and 443, and then starts a few worker processes. The workers do the actual work. The master almost never handles a request. It watches the workers, restarts any that die, and reloads the configuration when you ask it to.
Two things follow from that shape, and both will save you an afternoon. First, the config is read at start-up and on reload, not on every request. Editing the file does nothing until you tell nginx to re-read it, and a broken file stops a reload from taking effect, which is exactly what you want. Second, a reload swaps in new workers and lets the old ones finish their current requests. That is why you reload instead of restart. A restart drops everyone who is mid-download. A reload usually does not.
The config file itself is a tree of blocks. The top level is the main context. Inside it sit an events block that tunes connection handling, and an http block that holds everything about web traffic. Inside http you place server blocks, one per site or per door. Inside a server you place location blocks, which decide what happens to each path. The shape of the whole thing is this:
http { server { listen 80; server_name app.pipeslab.in;
location / { proxy_pass http://127.0.0.1:3000; } }}Read it from the outside in. The http block says “web traffic lives here”. The server block says “I answer on port 80 for this name”. The location block says “for every path, pass it to the app on loopback”. That is the whole idea, and the rest of this chapter is about the rules for picking which block, at each level, handles a given request.
Server blocks: which door, which name
Section titled “Server blocks: which door, which name”nginx reads each request in two stages. First it asks which door the connection came through: the listen port, and the address if you gave one. Then it asks which name the visitor wanted, which is the Host header the browser sent. Those two keys pick a server block. Only inside that block does nginx look at the path, and it is the path that picks a location.
The matching rules are short. nginx gathers every server block that listens on that port. It compares the Host header with each server_name. An exact name beats a wildcard, and a wildcard beats a regular expression. If nothing matches, the request goes to the block marked default_server for that port. If you marked none, it goes to the first server block in the configuration that listens on that port.
That last rule causes more trouble than almost anything else in nginx. You add a new site file, nothing in it is wrong, and suddenly every domain on the box serves the new site. The old file did not break. It just stopped being the first one read.
Some teams make the fallback explicit by answering unknown names with nothing at all:
server { listen 80 default_server; return 444;}The 444 code tells nginx to close the connection without sending a response. It is the polite nothing. Scanners that hit your IP address by number, asking for names that do not exist, get silence back. They are doorknob-rattlers. Their rattling is weather, not a personal grudge.
There is a catch with the Host header, and it matters for the rest of the book. The browser sends whatever the URL said, and a script can send anything at all. That makes Host excellent for routing and useless for security. Anyone can point a request at your IP and type your domain into the header. Never decide who is allowed in based on the name alone. Chapter 12 uses this same routing idea to put many subdomains on one box, and you can read about the names themselves in Chapter 12.
Location matching: the classic confusion
Section titled “Location matching: the classic confusion”Here is the part that trips up almost everyone, including people who have run nginx for years. You have a server block that looks right. You have a location /api/v2/ block. A request for /api/v2/logo.png lands somewhere you did not expect, and you start to suspect the config is haunted.
It is not haunted. nginx does not read your location blocks from top to bottom and take the first match. It runs a fixed procedure, and that procedure matters far more than the order of lines in your file:
- If an exact match (
location = /health) fits the URL, use it and stop. Nothing else is checked. - Otherwise, look at every prefix location (
location /api/) that fits, and remember the longest one. Keep going. Do not stop yet. - If the remembered prefix was written with
^~, use it and stop. The regexes are skipped. - Otherwise, try the regex locations in the order they appear in the file.
~is case-sensitive,~*is case-insensitive. The first regex that matches wins. - If no regex matches, use the prefix you remembered in step 2.
Step 4 is the trap. A regex beats a prefix, even a much longer one, unless that prefix is marked ^~.
The same logic explains a common pattern. location ^~ /static/ keeps everything under that path with the static block, whatever its file extension. The caret is there so that no image regex can steal a file from it. And location = /health is cheap for the same reason: an exact match short-circuits the whole search, which is exactly what you want for a health check that your load balancer hits every few seconds.
Try it: the matching playground
Section titled “Try it: the matching playground”The demo below uses a fixed sample config, shaped like a real one. It has a pipeslab.in server, an app.pipeslab.in server with six locations, and a default_server that catches anything else. Pick a preset or type a URL, and watch nginx’s two-stage decision happen one step at a time. The first stage picks a server by the Host header. The second picks a location by the rules above. The winner is highlighted, and every loser gets a reason.
Try it: which block wins?
Static fallback: nginx first picks a server by the Host header, and falls back to the default_server when nothing matches. Then it picks a location. An exact match wins at once. Otherwise it remembers the longest matching prefix, a ^~ prefix stops the search there, regexes are tried in file order (the first match wins), and the remembered prefix is the fallback. With a trailing slash on proxy_pass, nginx swaps the matched prefix for the URI, so /api/users/42 reaches the app as /users/42.
Try these four, in this order, and notice what changes each time:
app.pipeslab.in/api/v2/logo.png: the regex wins over the longer prefix. This is the gotcha.app.pipeslab.in/static/logo.png: the^~prefix stops the search before any regex is asked.app.pipeslab.in/health: the exact match ends the search immediately.unknown.example/: no server name matches, so thedefault_servertakes the request and drops it.
Then switch on the trailing-slash toggle and try app.pipeslab.in/api/users/42. That is the bonus question, and the next section explains what you are looking at.
The trailing slash trap
Section titled “The trailing slash trap”This is the bonus question from the playground, and it is the one that bites people who copy configs from a tutorial. Two blocks look almost the same. The only difference is one character at the end of proxy_pass:
location /api/ { proxy_pass http://127.0.0.1:3000;}
location /api/ { proxy_pass http://127.0.0.1:3000/;}When proxy_pass has no URI part after the host and port, nginx passes the request URI through untouched. A visitor who asks for /api/users/42 reaches your app as /api/users/42. Your app had better have routes under /api/.
When proxy_pass has a URI part, even just a slash, nginx does a swap. It removes the part of the request URI that the location matched and puts the proxy_pass URI in its place. The matched /api/ goes, and the app receives /users/42. That is what many small apps want. It is also what breaks the apps that were written to expect the prefix.
The rule to remember is that the slashes must agree. A location ending in a slash, paired with a proxy_pass ending in a slash, is the clean pair. The mismatched pairs are where people lose an afternoon, because nginx does not warn you. It just sends a different path, and your app answers 404 for a route that works fine when you curl port 3000 directly.
My advice is to keep the prefix in the app, and leave the slash off proxy_pass. Let the app own its own paths. Add the slash only when you want to strip a prefix on purpose, and put a comment on that line saying so.
Headers through the proxy
Section titled “Headers through the proxy”Here is the bit that bites everyone once. The visitor connects to nginx from a real address. nginx then opens a second connection to your app on 127.0.0.1:3000, and from the app’s point of view the caller is nginx, which lives on the same box. So req.ip says 127.0.0.1 for every visitor, and your rate limiter treats the whole planet as one very busy person.
The fix is headers. The receptionist writes the visitor’s details on a sticky note and hands it to the app.
Three headers carry most of the weight:
Hostis the name the visitor asked for. By default nginx sends the upstream’s own address instead, so your app builds links and redirects to127.0.0.1:3000. Pass$hostthrough.X-Forwarded-Foris the chain of addresses the request has passed through. The standard$proxy_add_x_forwarded_forvalue appends the visitor’s address to whatever arrived.X-Forwarded-Protosays whether the visitor usedhttps. Without it your app believes it is on plain HTTP, buildshttp://links, and can send the visitor into a redirect loop with nginx.
Headers are only text, and anyone can type them. A visitor can send X-Forwarded-For: 1.2.3.4 on their own request. The append-style value keeps the forged entry and adds the real one after it, so the rightmost entry is the one your proxy actually saw. Tell the app to trust the proxy hop and nothing else. Most frameworks have a “trust proxy” setting for exactly this. If your app believes the header from anyone, anyone can be anyone.
WebSockets start life as an ordinary HTTP request that asks to upgrade. Proxies drop the Upgrade and Connection headers by default, so the handshake fails quietly and your live status feed never connects. The fix is a few lines on that one location:
location /ws/ { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade";}Two more limits explain most mysteries. client_max_body_size defaults to 1 MB. A 2 MB upload gets a 413 Request Entity Too Large from nginx before your app sees a single byte, so the app’s own logs stay silent, which is why it feels haunted. The timeouts, proxy_connect_timeout, proxy_send_timeout and proxy_read_timeout, decide how long nginx waits on the app. The read timeout defaults to 60 seconds. Hold that number in mind for the next section, because it is where the 504s come from.
Reading the proxy’s errors
Section titled “Reading the proxy’s errors”When something breaks behind nginx, the visitor often sees a plain page that ends with the word nginx. That is the receptionist talking, not your app. Three codes do most of the talking, and each one points at a different layer.
A 502 means nginx tried the app and got nothing usable. The usual reason is boring. The app crashed, it was never started after a reboot, or it listens on a different port from the one in proxy_pass. Connection refused is the classic sign: nobody is at that door. The fix lives in the app layer, so you start the app and check its address.
A 503 means the upstream answered, and the answer was “I cannot take this right now.” Sometimes that is a maintenance page you wrote yourself. Sometimes every server in an upstream group is marked down. Sometimes the app has a queue and the queue is full. The fix lives in the pool or in the app’s own capacity.
A 504 means nginx connected, sent the request, and waited. The app did not answer within proxy_read_timeout, which is 60 seconds by default. Here the slow route is usually the real problem: a query with no index, or a call out to a payment API that is having a bad afternoon. Raising the timeout will make the 504 go away, and the page will still be slow. Raise it only when you know the route is legitimately long, such as a report or an upload that takes its time. Otherwise fix the route.
Test, then reload, never restart blind
Section titled “Test, then reload, never restart blind”Two habits save more weekends than any clever configuration. The first is to test the config before you apply it. nginx has a built-in check that reads the whole configuration and tells you whether it parses, and it does this without touching the server that is running. A typo in a location line is caught there, not at 2 a.m. in front of a live site.
The second habit is to reload, not restart. A reload asks the master process to read the new config, start fresh workers with it, and let the old workers finish the requests they already hold before they exit. Visitors notice nothing. A restart drops every open connection and leaves a gap while the new process starts. Reload is the default, and restart is for the times you have a very good reason.
Logs are the first place to look, before you guess. The access log is a diary of every request: when it arrived, the path, the status code, and how many bytes went back. The error log is where nginx explains itself. A 502 almost always leaves a line there, and the sentence you are hoping for is usually something like “connect() failed, connection refused” with the upstream address beside it. That one sentence can save an hour of guessing about firewalls.
Alternatives, and which one I would pick
Section titled “Alternatives, and which one I would pick”nginx is not the only receptionist in town, and for a small app it is not the one I would hand you first. Here are the three you will meet most, and where each one earns its place.
Caddy: my pick for small apps
Section titled “Caddy: my pick for small apps”Caddy is a web server and reverse proxy written in Go. Its headline feature is that it obtains and renews TLS certificates on its own. You name the domain, point DNS at the box, and Caddy does the certificate work and the redirect from plain HTTP to HTTPS. The config stays short. This Caddyfile does the same job as the nginx block above, with compression added:
app.pipeslab.in { encode gzip reverse_proxy 127.0.0.1:3000}That is the whole file. Caddy sets the forwarded headers for you, passes WebSocket upgrades through without a special block, and reads the way it is written. The nginx version of the same routing, without any certificate lines, looks like this:
server { listen 80; server_name app.pipeslab.in;
location / { proxy_pass http://127.0.0.1:3000; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; }}Neither file is harder than the other to write. The difference is which lines you have to remember. With nginx you carry the header list in your head, and you add the certificate setup on top of it in Chapter 9. With Caddy the defaults are already the right ones, and you spend your attention on the app.
Traefik: when Docker labels are the point
Section titled “Traefik: when Docker labels are the point”Traefik watches a provider, usually Docker, and builds its routes from labels on your containers. You start a container with a label saying which hostname it answers to, and the route appears. Nothing is reloaded and no file is edited. That is lovely when apps come and go all day. It is a trap when you forget which label made a route exist, because the routing lives in many small places instead of one file.
If your apps already live in Docker Compose and you like the idea of the container describing its own route, Traefik is a fine choice. If you want to read the whole routing table in one screen, pick Caddy.
HAProxy: when load balancing is the job
Section titled “HAProxy: when load balancing is the job”HAProxy is a load balancer first. It is excellent at health checks, connection limits, queueing per backend, and detailed live statistics. You reach for it when you have several copies of an app behind one address and the question you care about is how traffic is spread across them. It is not built to serve files or manage certificates for a single small site, and it is more machinery than a one-app VPS needs.
My recommendation
Section titled “My recommendation”For your projects, a Next.js app on a VPS, a dance-academy app on Compose, or an internal team tool, pick Caddy. It gets HTTPS right by default, the config fits on one screen, and the things you would otherwise hand-write in nginx are already there.
Pick nginx when you need its specific controls, when you already run it across many sites, or when your team knows it and would rather not learn a second tool. Pick Traefik when the Docker labels are the point of the setup. Pick HAProxy when you have a pool of backends and balancing them is the real problem.
Do not choose by reputation. Choose by who will read the config at 2 a.m., and by how many tools you want to know. Moving between these later is an afternoon’s work, because the receptionist’s job is small. The app behind it does not change at all.
The lab: put a receptionist in front of hello-pipes
Section titled “The lab: put a receptionist in front of hello-pipes”Goal. On the lab VPS, make the public door answer through a reverse proxy, with hello-pipes hidden behind it on a private port. Then break things on purpose, three ways, and read what the proxy tells you each time.
The steps, in order.
- Make the office private.
hello-pipesshould listen on 127.0.0.1 only, so the street cannot reach its port directly. Confirm that from your laptop, not from the VPS. - Put the receptionist in front. Run Caddy on the public ports, and point a name at it. Your lab can reach it by IP with a Host header, or by a test name you control. Make one request and read the page.
- Notice the identity problem.
hello-pipesnow reports every visitor as 127.0.0.1, because the receptionist is the one knocking. Pass the real visitor address along, then tell the app which hop to believe. Check that the app now shows your laptop’s address. - Stop
hello-pipesand send the request again. The receptionist is at the desk, but the office behind is closed. - Start the app again. Add a slow route that waits longer than the proxy is willing to wait, and shorten that patience. Send a request to the slow route.
- Open the access log and the error log. Find the line for each of the three requests, and match each status code to the layer that produced it.
How you’ll know it worked.
- You can say, without looking, which layer produced each of the three status codes, and which file to open first.
- The app shows the real visitor address only after you passed the forwarded header and told the app to trust the proxy hop.
- The access log has one line per request with the status code. The error log says the reason in words.
- Each config change was tested and then reloaded, not restarted blind.
What you now know
Section titled “What you now know”- A reverse proxy works for the server. A forward proxy works for the client.
- nginx has one master that owns the ports and a few workers that do the work. The server block is picked by the door and the Host header.
- Location matching runs in a fixed order: exact, then longest prefix (with
^~stopping early), then regex in file order, and the remembered prefix if no regex matches. - The trailing slash on
proxy_passdecides what path the app receives. - Forwarded headers are how the app learns the truth. Trust only the hop in front of it.
- 502, 503 and 504 each point at a different layer. Read the logs before you touch anything.
- For a small app, Caddy gives you HTTPS and a short config. nginx is the right tool when you need its control or its ecosystem.
Here is the whole path, once more, in the order a request travels: Name → Address → Route → Door → Guard → Receptionist → App. You now know the last three links in that chain well enough to debug them. Well done, and go and break something on your lab box.
Self-quiz
Section titled “Self-quiz”1. What is the difference between a forward proxy and a reverse proxy?
A forward proxy acts for the client and hides the client from the server. A reverse proxy acts for the server and hides the server’s internals from the client.
2. A request arrives on port 443 for a name no server block claims. Which block answers?
The default_server for that address and port, if one is marked. Without a marked default, nginx falls back to the first block listening on that port. Either way, it never guesses by path.
3. The config has location = /health, location /, and location ^~ /static/. Which block wins for /health, and which for /static/logo.png?
/health wins immediately because an exact match ends the search. /static/logo.png goes to the ^~ block, which stops the search before any regex is checked.
4. After you add a proxy, the app logs every visitor as 127.0.0.1. What is missing, and why can’t you just trust whatever header you find?
The proxy is not passing X-Forwarded-For, or the app is not reading it. Any visitor can send that header themselves, so the app must trust it only when the request comes from the proxy’s own address.
5. A visitor gets a 502. Which layer do you open first?
The app behind the proxy. A 502 means the proxy could not get an answer from the upstream, usually because the app is down or not listening on the port the proxy expects.
6. Why reload the proxy instead of restarting it?
A reload makes the master start new workers with the new config, while the old workers finish their in-flight requests. A restart drops those connections and leaves a gap in service.
Exercises
Section titled “Exercises”Three warm-ups to check the vocabulary, four lab tasks on the lab VPS, and two staff questions for the day you own the box. Try each one before you open the answer.
Warm-up 1. Two proxies, two directions. A forward proxy speaks for the client. A reverse proxy speaks for the server. In a sentence each, say whose identity the other side sees, and name one job each one does for you.
Answer
A forward proxy represents the visitor, so the website sees the proxy and not the person behind it. A reverse proxy represents the server, so the visitor sees the front door and never the office. Your reverse proxy’s jobs are TLS termination, one IP for many apps, buffering slow clients, and hiding the port your Node app listens on.
Warm-up 2. Where does the path go? Given location /api/ { proxy_pass http://127.0.0.1:3000; }, what path does the app see for a request to /api/users/42? Now add a slash so the line reads proxy_pass http://127.0.0.1:3000/;. What does the app see now?
Answer
Without a URI part, nginx passes the original path, so the app sees /api/users/42. With the trailing slash, the matched /api/ prefix is replaced by /, so the app sees /users/42. Your Next.js routes have to match whichever one you chose. A mismatch here is one of the most common reasons a proxied app returns 404 for pages that work fine on localhost.
Warm-up 3. Name the layer. Your proxy returns three different codes over a week: 502, 503 and 504. For each one, say which layer you open first.
Answer
502 means the proxy could not get an answer from the upstream at all. The app crashed, is not listening, or the proxy points at the wrong port, so open the app’s process and its listening port first. 503 means the request was turned away, often because a rate limit such as limit_req tripped or the app is shedding load, so open the limits and the load. 504 means the app accepted the request but was too slow for the proxy’s patience, so open the slow route and the timeout values such as proxy_read_timeout.
Lab 4. Only the proxy gets in. On the lab VPS, hello-pipes should answer only on its own loopback address, and nginx should be the one door on port 80. Prove that the app is unreachable from the public IP on its own port, and that the front door still works.
Hint
Look at which address and port the Node process is bound to, then try the app’s port from your laptop against the public address. Your laptop can use the lab’s public IP directly, so there is no need for a domain yet.
Model solution
The app listens on 127.0.0.1:3000, so a connection to 203.0.113.10:3000 is refused or times out. A request to port 80 on the same IP reaches nginx, which forwards it to the app. If the app were bound to 0.0.0.0, the first test would succeed, and that is the exposure this exercise is about.
Lab 5. The app sees the wrong visitor. Open hello-pipes through the proxy and read the client IP it prints. Then add the forwarding headers to your server block, reload, and read it again. Explain why the first answer was 127.0.0.1 and why you should trust the header only from your own proxy.
Hint
The proxy’s location block needs Host, X-Forwarded-For and X-Forwarded-Proto set. Test the config before you reload, and send the request from your laptop, so the client address you expect is the laptop’s public address.
Model solution
Before the headers, the app sees 127.0.0.1 because the TCP connection really does come from nginx on the same machine. After the headers, the app reads the forwarded address, which is your home address, and the scheme https if you terminate TLS at the proxy. The app should accept the header only when the direct connection comes from 127.0.0.1, because any outside client can send a fake X-Forwarded-For.
Lab 6. Stop the app and read the 502. Stop the Node process, load the front door, and find the line in the error log that explains the failure. Start the app again and confirm the error goes away.
Hint
The error log is the first place to look. Search it for the word upstream after you make the request, and note the port nginx tried to reach.
Model solution
The log says the connection to the upstream was refused, which nginx turns into a 502 for the visitor. The fix is in the app layer, not in nginx. Restarting the app makes the same request succeed, and nothing in the proxy config changed.
Lab 7. A slow route becomes a 504. Add a route to hello-pipes that waits about ten seconds, then request it through the proxy with a read timeout of three seconds. Read the status code and the error log line, then restore the timeout to its normal value.
Hint
Find the timeout directive that controls how long nginx waits for the upstream to send a response header. Set it low for the test, run the request with the status line visible, and keep the old value in a comment so you can put it back.
Model solution
The visitor gets 504 Gateway Time-out. The error log says the upstream timed out while reading the response header. The app did accept the request, so this is a slow route, and the fix is in the code or in a deliberate timeout choice, not in restarting nginx.
Staff 8. Pick a proxy for the next year. Your team will run six small apps on one VPS for the next twelve months, with two people who know nginx and one who knows only Docker. Choose one of Caddy, nginx, Traefik or HAProxy, and say what would make you change your mind.
Answer
Caddy, for a small set of apps, because automatic HTTPS and a short config remove the certificate chores that cause most outages on small boxes. Keep nginx if the team already runs it and needs fine control, such as detailed caching rules or a module you cannot replace. Choose Traefik if every app is a container and labels are the source of truth. HAProxy is for heavy load balancing across many backends with health checks, and six small apps are not that. Change your mind if the apps grow into many backends with traffic that needs serious balancing.
Staff 9. Triage a webhook that looks broken. A payment provider’s webhook returns 413 on large payloads, the app logs every request from 127.0.0.1, and some signature checks fail only in production. List what you check, in order, and say which layer owns each fix.
Answer
Start with the 413, which is a size limit at the proxy, so check client_max_body_size in the server or location block, and raise it only as far as the largest legitimate payload. Next, the 127.0.0.1 client address means the forwarding headers are missing or the app does not trust them, which is an app and proxy pairing fix. The signature failures are often a body problem: if the proxy or a middleware re-encodes the body, the signature no longer matches, so verify against the raw bytes exactly as received. Finish by reading the access and error logs to confirm which request failed and with what status.
Book 2 of a series. Book 1: The Glue.