10. DNS for Operators
“Nothing propagates. Caches just run out of patience.”
The story: the 2 a.m. call
Section titled “The story: the 2 a.m. call”This is a composite, stitched together from a few migrations over the years, with the names changed. A dance academy in Pune moves its app from an old VPS to a new one on a Friday evening. The new server is up. The deploy is green. The health check passes from the laptop that ran the migration, which is the DNS cousin of “works on localhost,” and everyone goes home.
At 2 a.m. on Saturday a message lands in the group chat. Half the students cannot reach the site. A hard refresh does nothing. One student says it loads fine over mobile data but not on the academy’s Wi-Fi. Another says it loads on her phone, but shows last month’s timetable.
Nobody changed the app. Somebody did change a number, though, on Friday around nine in the evening. The number was the IP address that app. points to, and the name was the one students type. The old number was still being handed out, with full confidence, by the resolvers that sat between each student and the truth.
The fix took four minutes, which is also how long it would have taken to change the record back. What took the rest of the night was working out why two phones on two networks could disagree about one name, and why the disagreement would end by itself at a moment nobody could read off a dashboard. The answer was a timer. Every DNS answer carries one, and most people who run servers have never looked at it.
This chapter is about that timer and the machinery around it. By the end you should be able to plan a DNS change so the 2 a.m. call never happens, read a dig answer without guessing, and explain to a product manager why “it will propagate in a few hours” is the wrong sentence to put in a ticket.
How a name becomes an address
Section titled “How a name becomes an address”Think of DNS as the city directory. You know the name of the shop, you want its street address, and the directory tells you where to go. The directory is not one big book in one room. It is a set of books, each holding one section, and each book points you to the next one.
Here is the walk for app.pipeslab.in, the first time anybody on your network asks for it in a while.
Your laptop does not work the answer out. It has a small helper, called a stub resolver, whose only job is to hand the question to a recursive resolver. The recursive resolver does the legwork. On most home connections it belongs to your ISP. Plenty of people point their devices at 1.1.1.1 or 8.8.8.8 instead, which are public recursive resolvers run by Cloudflare and Google. They are the same kind of thing, run by different people, and both sit between you and the hierarchy.
The recursive resolver then walks the hierarchy, asking one party at a time.
The numbers on the arrows are the order of questions, and they are worth reading slowly, because the whole chapter hangs off them.
- Your laptop asks the resolver: what is the address of
app.pipeslab.in? - The resolver asks a root server. The root does not know your name, and it does not try to. It knows which servers look after
.in, so it refers the resolver there. - The resolver asks the
.inservers. They do not hold the details ofpipeslab.ineither, but they know which servers do, so they refer the resolver onward. - The resolver asks the authoritative server for
pipeslab.in. This is the party that owns the record, and it answers with the address.
Notice what the resolver does with each reply. It does not stop at the first useful thing it hears. A referral is a pointer, and the resolver follows it. An answer is the end of the walk. That distinction matters later, when we get to caching, because each step in the walk leaves a copy behind.
Why it is built this way
Section titled “Why it is built this way”DNS arrived in the early 1980s. A widely told story has it that the early internet kept its names in a single text file, copied around by hand, and that the file grew faster than anyone could maintain it. Whatever the exact history, the problem was clear enough: a single list of every name does not scale, and it does not survive when one person forgets to copy the latest version.
So DNS split the job. Each zone is run by the party responsible for it. The root delegates to the top-level domains, and each top-level domain delegates to the registries and operators below it. Nobody has to know everything. Everybody only has to know whom to ask next. That is the entire trick, and it is why DNS keeps working when huge parts of the system are down.
The world without it
Section titled “The world without it”Without that split, you would need either a single master server that every laptop on earth queried for every name, or a file on every machine that you had to update by hand. Both failed in exactly the way the old text file did. The hierarchy is slower for the first lookup, because it makes several trips, but it is the reason the first lookup can be answered at all.
The caches that make it fast
Section titled “The caches that make it fast”If every visitor walked the whole hierarchy every time, the root servers would melt, and the first lookup in any city would take a visible second. So every party keeps copies. Root answers are cached for a long time, because the root’s answer about .in hardly ever changes. The .in referrals are cached for hours or days. The authoritative answer is cached for whatever lifetime the owner chose, which we will come to in a minute.
The recursive resolver is the busiest cache of all. Thousands of people using the same resolver share its memory, so a popular name is looked up once and then answered from memory for everybody who asks in the next few minutes. Your browser keeps its own copy, and so does your operating system. By the time a visitor reaches your server, there may be four or five caches between their screen and your records. Keep that number in mind. It will come back.
The best way, for a small production setup
Section titled “The best way, for a small production setup”For a single VPS and a handful of names, our opinion is firm. Do not run your own authoritative DNS server. Use the nameservers your registrar or DNS host gives you, use the set they provide (usually two or more, on different networks), and let them answer for your zone. Running your own is a fine hobby. It is a poor way to earn a weekend off. The question of how the registrar hands your zone to those nameservers is the subject of the next chapter, on domains and delegation, and it is worth reading before you change anything at the registrar.
If you want the basics of how a resolver talks to the network underneath, the Glue Book covers them in its networking chapter. We will not repeat that here. We will assume you know that a resolver sends a UDP question and gets a UDP answer, and we will concentrate on the parts that bite operators.
TTL and the “propagation” myth
Section titled “TTL and the “propagation” myth”Every answer DNS gives you comes with a timer attached. It is called the TTL, short for time to live, and it is a number of seconds. The owner of the record picks it. A resolver that receives the answer keeps it along with the countdown, and for that many seconds it hands out the same answer without asking anyone again. When the timer hits zero, the copy is thrown away, and the next question goes back up the hierarchy.
That is the whole mechanism, and the part missing from it is the important part. Nothing is pushed. The authoritative server does not phone every resolver on earth to say the address has changed. It does not keep a list of who asked. It changes its own answer and waits. Every other copy keeps saying the old thing until its timer runs out, and then it asks again and hears the new thing.
We call this propagation, and the word is the trouble. Propagation suggests a wave that starts at a source and reaches each place once, in order. DNS does not behave like that. A change does not spread. It becomes visible one resolver at a time, as each cached copy expires at its own moment. Some expire in a minute, some in an hour, and nobody controls the order.
Read the figure as three resolvers that were all asked the same question, each holding a copy. The office resolver cached the old address at 09:58 with a five minute TTL, so it serves the old address until 10:03. The ISP resolver cached at 09:45 with a one hour TTL, so it serves the old address until 10:45, forty five minutes after the change. The public resolver cached a few seconds before the change, with a one minute TTL, and switched almost at once. Each one is correct by its own lights. Each is doing exactly what it was told.
So when someone asks when the change will propagate, the honest answer is “when the longest live timer runs out.” That is the largest TTL among the copies, counted from the moment each resolver last fetched the old answer. You cannot read that number from a dashboard, and you cannot force it from the server. You can only plan around it, and planning starts before the change.
The drill: lower the timer first
Section titled “The drill: lower the timer first”Think of the change as three steps. The first one happens days before anybody touches an address.
Step one, lower the TTL on the record you are about to change. If it is sitting at 86400, a day, bring it down to 300 or 60. Lowering it is itself a change, and it reaches resolvers in the same slow way. Resolvers still holding the old long timer keep it until that timer runs out, so wait at least as long as the old TTL before you move on. If the record was at a day, the lowering needs a day of runway.
Step two, once the short TTL has been in place for at least the old TTL, change the address. From here, every resolver that asks holds the new answer for at most five minutes, and the old address has nowhere left to hide for long.
Step three, once the change has settled and you have watched the traffic move, raise the TTL back to something sensible, a few hours, so that you are not paying for constant lookups forever.
The drill is the whole idea in one line: you can only shorten a timer that has not started yet. Once a long timer is running in someone else’s cache, lowering the number at the authoritative does nothing for that copy.
Try the timer yourself. The demo below plays the walk for one name. You can set the TTL, change the record at the authoritative, and advance the clock. Watch the resolver keep serving the old address until its countdown reaches zero, and notice that the authoritative server was the only party that ever needed to be told.
Try it: resolve a name, change the record, then watch the cache count down
clock: 0 s
Recursive resolver cache
| Name | Answer | TTL left |
|---|
Authoritative for pipeslab.in
Static fallback: the resolver walks root, then the .in servers, then the authoritative one, and caches each answer for its TTL. A changed record stays invisible to a resolver until its cached copy runs out. A typo gets NXDOMAIN, and that is cached for the SOA negative TTL of 300 s, so the typo can stick even after the record exists.
A word about the propagation checkers, the websites that ask a long list of public resolvers at the same moment and show you what each one says. They are useful, because they give you a snapshot of who holds what right now. They cannot show you the timer. When the snapshot says “old address” for some resolvers, that is not a bug in your change. It is the timer telling you how long you have left.
Record types, and what each one is for
Section titled “Record types, and what each one is for”A zone is a set of records, and each record has a type that says what kind of answer it carries. You will meet ten of them often enough to need a working picture of each. Here they are, with the job each one does for an operator.
| Type | What it holds | What you use it for |
|---|---|---|
| A | An IPv4 address | Pointing a name at a server you own, like app.pipeslab.in at the lab VPS |
| AAAA | An IPv6 address | The same job for IPv6. Forget it and some visitors quietly take a slower path |
| CNAME | Another name | Making one name an alias for another, usually a provider’s hostname |
| MX | A mail server name, with a preference number | Telling the world where mail for the domain should go |
| TXT | Free text | Verification tokens, SPF and DKIM policy, anything a service asks you to publish |
| NS | A nameserver name | Saying which servers are authoritative for the zone. This is the delegation |
| SOA | Zone metadata, with timers | Serial number, refresh timers, and the negative caching TTL we meet later |
| CAA | Which certificate authorities may issue certificates | Stopping a rogue authority from issuing for your name |
| SRV | Host and port for a named service | Telling a client where a service lives, with priority and weight |
| PTR | A name for an address | Reverse lookup: the name that belongs to an IP, used by mail servers and logs |
Most of the time you will live in A, AAAA, CNAME, MX, and TXT. NS and SOA belong to the zone itself and are usually set once by your DNS host. CAA is a quiet security record that more operators should use. SRV is common in the world of game servers and some chat protocols, and PTR is the one you set at the place that controls your IP address, not in your zone. We come back to MX and TXT in the email chapter, where they carry most of the weight.
The CNAME rule
Section titled “The CNAME rule”A CNAME says, in effect, “this name is another name, go and look there.” Because it is an alias, a resolver follows it and then asks again. The rule that catches people is that a name holding a CNAME can hold nothing else. No A next to it, no MX, no TXT. The standard requires it, and resolvers and zone checkers enforce it.
Figure 10.3 shows the two cases side by side.
That bottom row is the trap, and it has a name. The apex is the bare domain itself, pipeslab.in with nothing in front of it. The apex must carry NS and SOA records, and in practice it also carries your MX and TXT records. So it cannot be a CNAME. That is the apex CNAME problem, and it bites the moment you want the bare domain to point at a load balancer or a platform whose address you do not control. You want pipeslab.in to follow a hostname the provider can move, and the rule says no.
The workaround is a family of record types that your DNS host invents on top of the standard. Some call it ALIAS, some ANAME, and the usual description is CNAME flattening. The host looks up the target itself, and then serves ordinary A records for your apex, built from the target’s current addresses. To a visitor it looks like a plain A record. To you it is a provider feature, not a standard record type, and the timer on those flattened addresses is the provider’s to set. We come back to flattening, and to proxied records that do something similar, in the Cloudflare chapter. Until then, the working rule is simple enough: a name that is an alias stays an alias, and the apex stays out of the alias business unless your host offers flattening and you understand what it is doing.
The dig: reading the output like a receipt
Section titled “The dig: reading the output like a receipt”Most people run dig the way they once ran ping. Type the name, see whether it works, move on. That wastes the best part of the tool. The output is a small receipt. It tells you who answered, how old the answer is, and how long the trip took. Here is an answer from your ISP’s resolver, lightly trimmed:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41822;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; QUESTION SECTION:;app.pipeslab.in. IN A
;; ANSWER SECTION:app.pipeslab.in. 213 IN A 203.0.113.10
;; Query time: 18 msec;; SERVER: 192.0.2.53#53(192.0.2.53) (UDP)Read it the way you would read a bill, top to bottom.
- status is the verdict. NOERROR means the name exists, even when the answer section is empty. NXDOMAIN means the name does not exist. SERVFAIL means the server tried and gave up, very often because something higher up the chain is broken.
- flags describe the conversation.
qrsays this is a response.rdsays you asked for recursion, andrasays the server will do it. The flag to watch isaa, authoritative answer. When it is present, the server answering owns the zone and the data is the source. When it is missing, you are reading someone’s copy. - the sections are question, answer, authority and additional. The authority section matters most for negative answers, where it carries the SOA record. We come back to that.
- the TTL column is the number people miss. At the owner it shows the configured value, say 300. At a cache it shows the seconds left. Ask the same question twice and the number falls. If it does not fall, you are not talking to a cache.
- query time is how long this server took. Eighteen milliseconds means a cache hit or a nearby server. A few hundred means the recursion had to walk the chain.
- SERVER is whose answer you have. Most people skip this line, and it is the one that tells you which resolver you were actually talking to.
Ask the owner, then ask the crowd
Section titled “Ask the owner, then ask the crowd”Two questions cover most DNS mysteries. What does the owner say? And what do the big public resolvers say? Ask the owner by naming its nameserver as the server you query. You get aa and the full TTL. Ask 1.1.1.1 or 8.8.8.8 and you get a cache, with a TTL counting down.
When the two disagree, you do not have a DNS bug. You have a timing problem, and the TTL column tells you how long it will last. That comparison is the whole migration story in one picture, and it is the skill this chapter is trying to give you.
The third tool is +trace, a dig option rather than a separate program. It ignores your resolver and walks the chain itself. The root answers with a referral to the .in servers. The .in servers answer with a referral to the pipeslab.in nameservers. The authoritative server answers the question. Each hop prints what it said, with its TTLs. When a name resolves on one network and not another, trace shows the exact link that breaks. In practice the usual culprit is a delegation that points at nameservers which no longer answer. That is a registrar problem, and Chapter 11 opens that box.
Figure 10.4 puts the two kinds of answer side by side.
Notice that the IP is identical in both panels. Caching does not change what the answer says. It changes how old the answer is allowed to be.
Negative caching: why a typo can stick
Section titled “Negative caching: why a typo can stick”Not every answer is a yes. Ask for aap.pipeslab.in and the authoritative server replies NXDOMAIN, which means no such name. A resolver caches that reply too, and this is the part people miss. The rule is written down in RFC 2308, and it exists for a good reason: if a thousand phones keep asking for a name that does not exist, you do not want every one of those questions landing on the authoritative server.
How long is the “no” kept? The zone’s SOA record carries a minimum field, and RFC 2308 says the negative TTL is the lesser of the SOA record’s own TTL and that minimum. Our lab zone uses 300 s, and the demo uses the same number. So a typo can stick for up to five minutes.
That sounds harmless until a human is involved. You mistype a name, get NXDOMAIN, realise the record is missing, create it, and retry ten seconds later. The resolver still says NXDOMAIN, and by its own rules it is right to. Your laptop and browser may hold their own copies of the “no” as well. Nothing is broken. The cache is doing what caches do.
Past the cache: encrypted lookups and split views
Section titled “Past the cache: encrypted lookups and split views”Encrypted lookups: DoT and DoH
Section titled “Encrypted lookups: DoT and DoH”Classic DNS travels in plain text on port 53. Anyone on the path, whether that is the coffee shop’s Wi-Fi or your ISP, can read every name you ask and can sometimes change the answer. Two fixes exist, and both encrypt the question between you and the resolver. DNS over TLS (DoT) wraps the same DNS messages in TLS on port 853. DNS over HTTPS (DoH) carries them inside ordinary HTTPS on port 443, so they look like web traffic. They are specified in RFC 7858 and RFC 8484.
Be clear about what this buys you. The resolver still sees every name you ask. You have moved trust from the network to the resolver you chose, which is usually a good trade for a laptop on hostile Wi-Fi. It does not prove that the answer is true. That is a separate job, done by DNSSEC, which signs the records themselves and is a topic of its own.
The best way, for a small production setup: on a VPS, the resolver your provider gives you is fine, and your attention belongs on the records, not the transport. Turn on DoH or DoT in the browser and on the laptop, where it costs nothing and the trade is worth it.
Split-horizon: two answers for one name
Section titled “Split-horizon: two answers for one name”A split-horizon setup gives different answers to different askers. From the internet, app.pipeslab.in is 203.0.113.10, the public address. From inside the lab network, the same name is 10.0.0.5, the private address, so the traffic stays on the private side and never crosses the public internet. Servers do this with views, which BIND calls them, or with a split-DNS service that answers internal clients differently.
The war story is the same every time. A developer on the office network sees the private address, fixes something against it, and the outside world never sees the change. Then everyone says it works on their machine, and they are right, because their machine is inside the other view. Tailscale’s MagicDNS is a cousin of the same idea, and we come back to it in Chapter 16.
War stories: when the directory is the outage
Section titled “War stories: when the directory is the outage”Most DNS trouble is small: a typo, a stale record, a cache whose timer has not run out yet. Two outages sit at the far end of the scale. Both are worth knowing by their shape, because you will meet smaller versions of them.
Dyn, 2016: a botnet knocks on the door
Section titled “Dyn, 2016: a botnet knocks on the door”On 21 October 2016, if memory serves, a botnet built from insecure internet-connected devices, mostly cameras and recorders, flooded Dyn with queries. Dyn was a large managed DNS provider, and many well-known sites used it as their authoritative nameservers. When Dyn could not answer, those sites were hard to reach for many people for hours, even though their own servers were fine. The botnet was Mirai, and the story is still told as the moment many people learned that a pile of cheap devices could take down infrastructure they never knew they depended on.
The lesson is about where the single point sits. Your site’s name is a pointer, and every resolver that wants your site must ask your nameservers for it. If all of those nameservers sit behind one provider, that provider is a single door, and a flood at the door is an outage for every customer behind it, however healthy each customer’s own server is.
Two things follow, and they pull in opposite directions. The first is to avoid keeping every nameserver in one place. Give your zone nameservers from two providers, so one door closing leaves the other open. The second is that TTL is a shield as well as an agility dial. A cache that already holds your answer keeps serving it while your nameservers are down. A long TTL on stable records therefore helps you during an attack, and a short TTL is worth its cost only on the records you expect to move. Chapter 11 takes up who really controls your zone, the registrar or the DNS provider, and why that question matters here.
AWS us-east-1, October 2025: the record that went empty
Section titled “AWS us-east-1, October 2025: the record that went empty”In October 2025, a fault in AWS’s us-east-1 region took down or degraded a long list of services for hours. As the public account described it, the trigger was DNS. The regional DynamoDB endpoint’s records were maintained by automation, and a race condition in that automated DNS management left the records empty. Clients could no longer find the endpoint at all, so everything that depended on DynamoDB failed in turn, and the failures cascaded from there. Later write-ups refined the details, so treat this as the shape of the story, not a precise timeline.
The lesson is uncomfortable. DNS automation is code with the power to delete the answer. A missing record gives you an outage with no bad IP to blame. It deserves the same review, guardrails and rollback you give application code. Recovery is also slower than the fix, because answers cached during the outage keep failing until their timers run out, and you cannot reach into a resolver on the other side of the world and clear them.
“It’s always DNS,” used properly
Section titled ““It’s always DNS,” used properly”Every operator has a joke about DNS, and this one earns its place as a heuristic. When something that worked yesterday fails in a way that smells like a name, check DNS early. The symptoms are recognizable. It works by IP but not by name. It resolves to the wrong place for some people. It works from one network and not from another. Checking is cheap: one dig against a public resolver and one against the authoritative server tells you whether the name is right before you spend an hour inside the application.
It is not always DNS. It is often enough, and it sits underneath everything else, so it is the first place to look and the last place to assume. Chapter 9 shows how certificate issuance depends on the same names being right, which is a good reason to check the names before you check the certificates.
The lab: watch the timers run out
Section titled “The lab: watch the timers run out”This lab uses the lab world from the start of the book: the throwaway VPS at 203.0.113.10 and the placeholder domain pipeslab.in. Nothing here touches production. Create the records in whatever DNS panel your lab domain uses, and keep a plain text log as you go, because the timings are the lesson.
The goal. By the end, you should be able to predict, before you press anything, when each resolver will stop giving the old answer. You will have made an A record for app.pipeslab.in, asked the same question of three different askers, walked the lookup from the root down, moved the record with a planned TTL drop, and watched a typo stick.
The conceptual steps.
-
Create the records. Make an A record for
pipeslab.inand another forapp.pipeslab.in, both pointing at203.0.113.10. Write down the TTL your provider sets by default. It is a number of seconds, not a promise. -
Ask three askers the same question. Ask your ISP’s resolver, then a public resolver such as 1.1.1.1, then the nameserver that is authoritative for the zone itself. Write down the answer, the TTL and how long each one took. Ask the caching resolver a second time, a minute later, and see what the TTL did.
-
Walk the lookup from the top. Ask for the full path from the root down to the answer. Name each hop in order: the root, the
.inservers, the nameservers forpipeslab.in, and the final A record. For each hop, write down whether that server gave you a referral or the answer. -
Run a migration drill. Find the current TTL on
app.pipeslab.inand log it. Lower the TTL to 60 seconds, then wait out the old TTL, so every cache has had a chance to pick up the short value. Only then change the A record to203.0.113.20, a second throwaway address. Watch the resolvers flip, and note the moment each one first gives you the new answer. -
Let a typo stick. Ask for
aap.pipeslab.in, a name that does not exist, and note the answer. Then create that record, pointing it at the same address. Ask the same caching resolver again straight away, and then ask the authoritative server. Compare the two.
How you’ll know it worked. The lab is done when your log answers four questions without guessing.
- Which timer controlled each answer you saw, and how old was that copy?
- When did the migration finish, and was it close to the old TTL rather than the moment you changed the record?
- Why did the typo keep failing after the record existed, and which server would have given you the new answer straight away?
- Before you ran step four, could you predict roughly when the flip would happen?
If you cannot answer one of them, go back to the numbers in the log. The answer is almost always there, and nobody needs to be blamed for propagation.
What you now know
Section titled “What you now know”You now know that DNS is a tree of directories. Each one points you to the next stop, there is a cache at every stop, and every cache runs on a timer. A recursive resolver does the walking for you. The root and the TLD servers only point onward, and your zone’s nameservers hold the answer. Your ISP’s resolver, 1.1.1.1 and 8.8.8.8 are caching resolvers, and each can give a different answer to the same question, depending on what it has already stored.
- Propagation is not an event. Caches expire on their own timers. To plan a change, lower the TTL a full old TTL in advance, then change the record.
- Record types have jobs. A and AAAA point at addresses. CNAME is an alias, and it cannot share a name with anything else. The apex is where that rule bites, which is why flattening, ALIAS and ANAME exist. MX and TXT carry mail and verification. NS and SOA describe the zone. CAA limits who may issue certificates. SRV and PTR cover services and reverse lookups.
- dig is a thinking tool. Read the status, the answer and authority sections, and the TTL column. Query a specific server to separate what a cache says from what the owner says. Use trace when you want the whole walk.
- Negative answers are cached too. A typo can stick for as long as the zone’s negative TTL allows.
- Encryption and split views change the path or the answer, not the rules. DoT and DoH protect the question on its way to the resolver. Split-horizon gives different answers to different askers, on purpose.
- A directory can fail as a single point. The Dyn botnet and the 2025 AWS story both show that the lookup layer is shared infrastructure, and that automation can empty it.
You have a working map of the whole directory now. That is a real milestone, and it is the part most people never get to. Chapter 9 shows why the same names matter for certificates, and Chapter 11 covers registrars, delegation and who really controls your zone. Chapter 12 uses these records for subdomains, Chapter 13 for mail, and Chapter 17 for flattening and proxied records. Appendix B is the reference table for every record type. If you want the basics again first, the Glue Book’s network chapter is the friendly start: https://the-glue-book.pages.dev/part-1/ch05-network/
Self-quiz
Section titled “Self-quiz”1. Why does a cold lookup walk root, then TLD, then authoritative, while a warm one often goes straight to the authoritative server?
Because the resolver caches the referrals. The NS records for in and for pipeslab.in, along with their glue, carry their own TTLs. While they are fresh, the resolver skips the upper levels and asks the zone’s authoritative servers directly.
2. A resolver shows app.pipeslab.in with 1794 seconds left on a record published with TTL 3600. How old is the cached copy, and what does that mean for a change?
The copy is 1806 seconds old (3600 minus 1794). Any resolver holding that copy keeps serving the old answer for up to the 1794 seconds left, then refreshes. Nothing you do at the authoritative server can shorten that wait for it.
3. Why can’t you put a CNAME on pipeslab.in itself, the zone apex?
A CNAME says “this name is only an alias for another name”, and it cannot share a name with any other record. The apex must hold SOA and NS records, so a CNAME there is illegal. That gap is the apex problem, and it is why providers offer ALIAS, ANAME, or CNAME flattening as workarounds.
4. Where does a resolver learn how long to remember an NXDOMAIN answer?
From the SOA record in the authority section of the negative response. Its minimum (negative) TTL field sets how long the “no such name” answer may be cached. So a typo can stick for a while even after you create the record, because the negative answer is still in the cache.
5. What does DNS over HTTPS or TLS protect, and what does it leave exposed?
It protects the lookup between your device and the resolver from anyone watching the network path, such as the coffee-shop Wi-Fi or your ISP. It does not hide the name from the resolver operator, which still sees every query. Authoritative servers still see the name the resolver asks for.
6. What is split-horizon DNS, and how does it usually go wrong?
It means the same name gets different answers depending on who asks. An internal view might return 10.0.0.5 for app.pipeslab.in, while the public view returns 203.0.113.10. It goes wrong when the views drift apart. You change one, forget the other, and people on the wrong side of the split see a site that no longer matches what you think you deployed.
Exercises
Section titled “Exercises”Three warm-ups come first. The lab and staff exercises follow, and the commands live only inside the lab hints.
Warm-up 1. Walk a cold lookup. Your laptop asks its ISP resolver for api.pipeslab.in for the first time. List, in order, each server the resolver talks to and what each one tells it. Then say what changes on a second lookup a minute later.
Answer
The resolver asks a root server about in, and the root returns a referral to the .in servers. It asks a .in server about pipeslab.in, and gets a referral to the pipeslab.in nameservers. It asks one of those authoritative servers for api.pipeslab.in, and gets the A record with its TTL. A minute later the resolver already holds the answer, so it may reply without contacting anyone. If the answer has expired, it asks only the authoritative server, because the delegation is still cached.
Warm-up 2. Read the timer. Your resolver shows app.pipeslab.in with 1794 seconds left, on a record published with TTL 3600. How old is the cached copy? If you change the record at the authoritative server right now, when can this resolver first serve the new IP?
Answer
The copy is 1806 seconds old: 3600 minus 1794. This resolver keeps serving the old IP until its countdown reaches zero, which is 1794 seconds from now. So the new IP reaches it at the earliest when that cache entry expires, roughly half an hour away. The wait is never longer than the original TTL.
Warm-up 3. Place the CNAME. Which of these is legal? (a) www.pipeslab.in as a CNAME to pipeslab.in. (b) pipeslab.in itself as a CNAME to pipeslab-host.example.net. For (b), say what you would do instead.
Answer
(a) is legal, as long as www has no other records at that name. (b) is illegal, because the apex holds SOA and NS records, and a CNAME cannot coexist with them. For the apex, use an A or AAAA record pointing at the server, or use your provider’s ALIAS, ANAME, or flattened CNAME if it offers one. Ch 17 covers what flattening does to the answer you get.
Lab 1. Ask three resolvers the same question. Look up the A record for app.pipeslab.in three ways: through your ISP’s resolver, through 1.1.1.1, and directly from the authoritative nameserver for pipeslab.in. Note the answer, the TTL, the query time, and the SERVER line for each. Then say which answer you would trust right after you change a record, and why.
Hint
Find the authoritative nameservers first, with a query for the NS records of pipeslab.in. Leave out the @ for your ISP’s resolver, since that is your system default. For the others, put the server after an @.
dig NS pipeslab.in +shortdig app.pipeslab.in A +noall +answer +statsdig @1.1.1.1 app.pipeslab.in A +noall +answer +statsdig @<one-of-those-ns-names> app.pipeslab.in A +noall +answer +statsModel solution
The ISP and 1.1.1.1 should agree when the record is stable, but their TTL column counts down, because it shows how many seconds are left in their cache. The authoritative answer shows the full TTL you set, and its header carries the aa flag. Trust the authoritative answer right after a change, because a cache may still hold the old IP for up to one full TTL.
Lab 2. Walk the tree with trace. Run a trace from the root down to app.pipeslab.in. Label each hop: root, .in, the pipeslab.in authoritative servers, and the final answer. Find the exact line where the answer changes from a referral to a result. Then say what you would check first if the answer were wrong, and the parent’s delegation pointed at old nameservers.
Hint
Trace does not use your resolver’s cache. It walks from the root every time, so it shows the delegation chain itself.
dig +trace app.pipeslab.in AModel solution
The root refers you to the .in servers. .in refers you to the pipeslab.in nameservers. The authoritative server returns the record, with the aa flag set. If the .in referral lists old nameservers, every correct record at the new provider is invisible. That is a delegation problem, which Ch 11 covers, not a record problem.
Lab 3. Run the migration drill. Move app.pipeslab.in from 203.0.113.10 to 203.0.113.20 with no visible downtime. Write the plan as timed steps, first. Then watch the ISP resolver and 1.1.1.1 flip, and note the moment each one switches. Keep the old server answering during the overlap.
Hint
Lower the TTL at the authoritative first, then wait out the old TTL before you change the IP. For a practice run, start from a TTL of 300, so the wait is five minutes. Watch the flip with a loop.
while true; do date; dig @1.1.1.1 app.pipeslab.in A +noall +answer; sleep 10; doneModel solution
At T0, lower the record’s TTL to 60. Wait at least the old TTL, which is 3600 seconds in the real zone, so every cache has refetched the 60-second answer. Then, at T1, change the A record to 203.0.113.20. Each resolver flips within 60 seconds of T1, depending on when it last refreshed. Once every resolver shows the new IP, raise the TTL back to 3600. Skip the wait and the flip can take an hour, which is the “propagation” myth in action.
Lab 4. Make a typo stick. Look up aap.pipeslab.in, which does not exist. Note the status, the SOA record in the authority section, and its last field. Ask the ISP resolver again. Then create aap.pipeslab.in at the authoritative with TTL 60, and ask the resolver straight away. Say how long the NXDOMAIN lasts, and how you would shorten that wait ahead of time.
Hint
The authority section appears with +authority. The last field of the SOA is the negative TTL. Creating the record does not clear the resolver’s negative cache.
dig aap.pipeslab.in Adig aap.pipeslab.in A +noall +authorityModel solution
The status is NXDOMAIN. The SOA in the authority section carries the negative TTL, which is 300 seconds in the figure above. The resolver keeps saying no for that long, even after you create the record. To shorten it, lower the SOA minimum before a planned change, the same way you lower an A record TTL. You cannot usually flush an ISP resolver, so plan for the wait.
Staff 1. Design two-provider DNS. You want a second DNS provider for pipeslab.in, after a single-provider outage has hurt you. Describe the NS setup, how you keep the zones identical, and the TTLs you pick for app records, NS, and SOA. Name the trade-off between a short TTL and a long one.
Model answer
List both providers’ nameservers at the registrar, so the parent delegation names both. Keep the zone identical at each provider, through zone transfer or the provider’s sync tool, and check that both answer the same records. App records can sit at 300 seconds, dropped to 60 before a change. NS and SOA stay long, around 86400, because they are slow to change and cached widely. A short TTL gives fast cutover but more queries, and if a provider fails, a long cache still keeps users working through a short outage. Delegation changes are slow, so do them before you need them.
Staff 2. Guard against an empty record. An automation tool removes records it thinks are stale. Two workers race: one writes an empty set for an endpoint while another reads the old set. Propose three guardrails.
Model answer
First, refuse any change that leaves a name with no address records, unless a human approves it. That guards the invariant that matters most. Second, serialise writers per zone, with a version check, so a stale plan cannot overwrite a newer one. Third, roll out in stages: apply to a test zone, probe the name from several outside resolvers, and revert automatically to a saved snapshot when resolution fails.
Book 2 of a series. Book 1: The Glue.