DNSSEC SERVFAIL: Find the Broken DS, DNSKEY, or RRSIG

SERVFAIL is frustrating because it names a result, not an owner. The fastest useful split is a paired query against the same validating resolver: if the normal query fails but the same query succeeds with +cd, DNS data exists and DNSSEC validation is the differentiator. That does not yet prove whether the broken link is parent-side DS, child-side DNSKEY, an RRSIG, authenticated denial of existence, or DNS transport.

Treat the incident as a chain-of-trust trace. Preserve the failing response, read any Extended DNS Error, compare the parent and child directly, and repair only the system that owns the first contradiction. Do not make “disable DNSSEC” the opening move. Removing the wrong record in the wrong order can extend the outage for every validating resolver that still has earlier data cached.

Start by proving validation is the difference

Run both requests from the same machine, against the same resolver, for the same name and record type. Replace example.com and the resolver address with the failing zone and a validator that reproduces the incident.

dig @1.1.1.1 example.com A +dnssec +nocmd +noall +comments +answer
dig @1.1.1.1 example.com A +dnssec +cd +nocmd +noall +comments +answer

The first query asks the recursive resolver to validate normally. +cd sets Checking Disabled for that request, so the resolver may return data without rejecting it as bogus. Cloudflare’s DNSSEC troubleshooting guide and the ISC BIND DNSSEC guide both use this differential.

Interpret the pair conservatively:

• Normal query is SERVFAIL, while +cd returns the expected answer: validation is the leading branch.
• Both queries fail: investigate authoritative reachability, delegation, rate limiting or another DNS failure before blaming DNSSEC.
• Both queries succeed: reproduce against the resolver and record type that users report, because cache state or authoritative inconsistency may be hiding the fault.
• One public resolver fails while several others succeed: collect evidence before declaring recovery; validators can hold different cached DS, DNSKEY and signature material.

Repeat the normal query against at least two additional validators. Google recommends this comparison because a failure reproduced across independent public resolvers points back toward the domain or its authoritative servers. Keep the complete header and OPT section rather than relying only on +short.

for resolver in 8.8.8.8 1.1.1.1 9.9.9.9; do
  echo "=== $resolver ==="
  dig @"$resolver" example.com A +dnssec +nocmd +nostats
done

This loop is observational. It changes no DNS state and gives the incident record three independent resolver receipts.

Read the resolver’s failure receipt

Modern validators may attach an Extended DNS Error in the OPT pseudo-section. EDE adds diagnostic detail to SERVFAIL; RFC 8914 does not make it a mandatory or stable application signal. Absence of EDE therefore proves nothing, while a present code can shorten the next step.

Four EDE clues narrow ownership

Google Public DNS documents these codes and warns that SERVFAIL can also carry No Reachable Authority or network errors. EDE 22 or 23 is not automatically a DNSSEC record mismatch. It may mean every listed authority failed to answer, refused the query, or could not complete transport.

Ask direct authorities before editing the zone

Recursive results mix cache, validation and upstream behavior. Query every authoritative server directly to see what each one currently publishes.

dig @1.1.1.1 example.com NS +cd +short
dig @ns1.example.net example.com DNSKEY +dnssec +nocmd +noall +answer
dig @ns2.example.net example.com DNSKEY +dnssec +nocmd +noall +answer

Replace the example names with the actual NS set. A provider migration can leave one old server, one unsigned server or one stale signer in the delegation. Agreement from one authority is not enough when resolvers can reach several.

Walk the chain from parent to signed answer

DNSSEC validation is a dependency path. The parent publishes a DS; the child apex publishes one or more DNSKEY records; those keys validate signatures over zone data; NSEC or NSEC3 proves negative answers. Network transport must deliver all of it.

Parent DS is registrar or registry state

Ask the parent path for the delegation signer record. +trace helps show which parent authority supplied it.

dig example.com DS +trace
dig example.com DS +dnssec +nocmd +noall +answer

Cloudflare’s current troubleshooting guide explains that DS contains a digest derived from the child’s key plus identifying metadata. If a domain moved to new authoritative nameservers but the previous DS remained at the registrar, validators are instructed to expect a matching key that the new provider may not serve.

Record the full DS tuple: key tag, algorithm, digest type and digest. Matching only the algorithm number is not a match. During a legitimate rollover, more than one DS may coexist; the safe question is whether at least one published parent DS authenticates a currently served child key through the intended transition.

Child DNSKEY must be present at every authority

Query the zone apex directly against each nameserver, then retain the answer and its RRSIG(DNSKEY).

for ns in ns1.example.net ns2.example.net; do
  dig @"$ns" example.com DNSKEY +dnssec +nocmd +noall +answer
done

The DNSKEY flags often identify key roles: 257 commonly marks a key-signing key and 256 a zone-signing key, although combined-signing-key designs also exist. Role labels help explain the zone; they do not replace cryptographic comparison. Use the DNS provider’s published DS value or a DNSSEC-aware tool to verify the actual parent/child link.

RRSIG time and coverage belong to the signer

A matching DS and DNSKEY can coexist with expired, not-yet-valid or missing signatures. Ask for the failing record type with DNSSEC data from every authoritative server.

dig @ns1.example.net example.com A +dnssec +nocmd +noall +answer
dig @ns2.example.net example.com A +dnssec +nocmd +noall +answer
date -u

Read the RRSIG inception and expiration timestamps in UTC. Compare them with a trusted current clock, then check whether every authority serves equivalent signed data. Time correction is not a substitute for repairing a signer that stopped refreshing signatures.

Negative answers need valid proof too

Some outages appear only for missing names, wildcard paths or particular record types. DNSSEC authenticates nonexistence with NSEC or NSEC3. Test a deliberately nonexistent label that cannot collide with production data.

dig @1.1.1.1 does-not-exist-20260803.example.com A +dnssec +nocmd +nostats

A valid result is normally NXDOMAIN or NODATA with signed denial proof, not SERVFAIL. Google documents EDE 12 for incomplete NSEC/NSEC3 proof. Repair belongs in the authoritative signing pipeline, not in an application server that merely depends on DNS.

Large signed answers must survive TCP fallback

DNSSEC adds keys and signatures, so DNSKEY responses can exceed a resolver’s advertised UDP size. Truncation is normal only when the client can retry over TCP. RFC 7766 requires DNS implementations to support both transports.

dig @ns1.example.net example.com DNSKEY +dnssec +bufsize=1232
dig @ns1.example.net example.com DNSKEY +dnssec +tcp

If UDP reports truncation and TCP succeeds, the protocol is working. If TCP times out or port 53 is blocked, repair the authoritative firewall, load balancer or network path. Do not rotate keys to “fix” a transport failure unless response-size engineering is independently required.

Repair the owner, then let caches become evidence

At this point the first contradiction should have an owner. Save screenshots or raw command output, current TTLs and a UTC timestamp before changing anything. Teams can use Voxfor’s security field notes for adjacent operational hardening, but the live repair should remain limited to this zone and this chain.

Stale DS after a DNS provider move

When the parent still advertises the old provider’s DS and the new child does not serve its matching key, choose one coherent target state with the registrar and DNS provider. Either publish the new provider’s supported DS while its key is already served, or remove the stale parent DS as part of a deliberate DNSSEC disablement process.

Never remove child DNSKEY material first while a parent DS still declares the zone secure. Cloudflare advises removing DS at the registrar, waiting at least 1.5 times its TTL in its platform-specific cleanup flow, and only then deleting remaining child DNSSEC material. Other providers may automate the transition differently, so follow their current sequence while preserving the same parent-before-child safety principle.

If the move also changed failover or geographic routing, separate traffic steering from DNSSEC trust before diagnosing application health. Both systems use DNS, but only the signing chain explains validation failure.

Broken KSK or DS rollover

A rollover is a timed overlap, not a button press. RFC 7583 describes why cached DNSKEY and DS data plus parent-zone interaction determine when a state transition is safe. Restore a state in which at least one parent DS authenticates a currently served child key, then allow the relevant cached data to age out before retiring the old path.

Do not invent timing from a dashboard’s “propagated” badge. Capture parent DS TTL, child DNSKEY TTL, signer state and resolver results. Rollback remains possible only while the former key and signatures are still valid and publishable. Once private key material or the former provider is unavailable, recovery becomes a coordinated parent/child repair rather than a simple reversal.

Missing or expired signatures

When DNSKEY matches DS but the failing RRset lacks a current RRSIG, restore the authoritative signer or its publication pipeline. Check key access, scheduled signing, zone serial progression and whether every nameserver received the newly signed zone. Avoid editing ordinary A or MX records merely to stimulate a resign unless the DNS product’s official recovery method calls for it.

Verification must cover the apex and the record type users actually request. One valid A signature does not prove MX, TXT, wildcard or negative answers are complete.

Transport or authority failure wearing a DNSSEC symptom

If direct queries reveal inconsistent authorities, TCP failure or timeout, repair reachability first. Google recommends testing each authority with DNSSEC-sized queries and both UDP and TCP. Keep firewall changes narrow to authoritative DNS service on port 53 from legitimate clients; never expose an administrative interface as part of DNS recovery.

Ownership spread across registrar, DNS provider and server network often causes delay. Teams that want a single accountable operator can assign DNS operations to Voxfor while retaining registrar approval and change records.

Close only after validators agree

Recovery is complete when evidence agrees at every layer, not when one browser reloads. Re-run the original record type with normal validation against at least three public resolvers, query every authority directly, test a negative answer, and repeat the DNSKEY request over UDP and TCP.

Record four items for the next operator: final parent DS tuple, active child key tags, earliest signature expiration, and UTC time when the last observed stale TTL should be gone. Add the exact registrar or DNS-provider change reference and name the person or team authorized to retire any old key.

Only then close the incident. A verified chain means the parent vouches for a key the child actually serves, signatures cover the answers users request, negative responses are authenticated, and transport delivers the proof consistently. That is stronger evidence than “DNS started working again,” and it makes the next migration or rollover safer.