Fix Linux VPS Clock Drift Without Breaking Running Services

Clock drift rarely introduces itself as “clock drift.” It arrives as a certificate that is “not yet valid,” an expired login token that was just issued, a cron job recorded in the wrong minute, database rows that appear out of order, or logs whose timeline cannot explain an incident.

Fixing those symptoms starts with a boundary: do not change the clock until you know whether the error is display, synchronization, source selection or application state. A running Chrony service proves only that a process is running. It does not prove that a usable source is selected, that the kernel clock has absorbed the remaining correction, or that running software can tolerate an immediate wall-clock jump.

This field procedure uses Chrony because it is common on current Ubuntu and Red Hat-family systems and is designed for virtual machines and unstable clocks. Operators should still identify the installed time stack before changing it. Accurate time is a Linux operations dependency, not a reason to install a second synchronizer beside the first.

Time failures arrive wearing another application’s error

Linux applications normally read the kernel’s system clock. Chrony maintains its own NTP estimate and adjusts that system clock toward selected sources. Those are related clocks, but the distinction matters: chronyc tracking can report a remaining system correction while Chrony is still slewing the kernel clock gradually.

Timezone is a separate layer. Europe/Berlin, America/New_York and UTC change how one instant is displayed; they do not change whether the instant itself is correct. If date -u agrees with a trustworthy outside observer but local time looks wrong, fix the timezone rather than touching NTP.

Real UTC error can affect several systems at once:

• TLS certificates and signed tokens have validity windows.
• Kerberos and similar authentication protocols enforce clock-skew limits.
• Cron and other wall-clock schedulers can run late, early, twice or not at all around a discontinuity.
• Database timestamps and event ordering may move backward even when the database stays online.
• Distributed logs lose causal order, which weakens incident evidence.

Because the blast radius belongs to applications, a clock correction is an application change. Treat the offset measurement, correction method and workload proof as one change record.

Capture four clocks before touching any service

Start with observers that answer different questions. Record their output and the collection time in the same incident directory or ticket.

Observer 1: UTC and local display

date --iso-8601=ns -u
date --iso-8601=ns
timedatectl status

Compare the two date lines. A correct UTC line plus an unexpected local line points toward timezone configuration. timedatectl status also shows the configured zone, RTC presentation and a high-level synchronization flag, but that flag is not a substitute for Chrony evidence.

Observer 2: an outside HTTP date

curl -fsSI https://www.cloudflare.com/ | sed -n 's/^date: //Ip'

HTTP Date is a coarse sanity check, not a precision time source. Network delay, intermediary behavior and whole-second formatting limit it. A difference of minutes is meaningful; a difference of milliseconds is not measurable this way. Use it to catch a badly wrong guest clock without treating a web server as NTP.

Observer 3: Chrony’s selected estimate

chronyc tracking
chronyc sources -v
chronyc sourcestats

These commands separate current correction, source selection and measurement history. The current Chrony command manual documents the exact fields and source-state symbols; use the installed man chronyc when a distribution ships an older version.

Observer 4: the workload’s own timeline

Select one timestamped artifact created before the incident: an application log line, database row, issued token, backup manifest or queue event. Compare its wall time with a monotonic duration or an independent remote timestamp where possible. That artifact answers the operational question that NTP output cannot: did wrong time already enter business state?

Do not restart Chrony, clear logs or run makestep during this capture. The first snapshot is the only evidence of the original state.

Read tracking and sources as separate evidence

chronyc tracking explains the system’s current estimate and correction. chronyc sources -v explains which candidates are reachable and selectable. Reading only one can produce a false green result.

Three passes through chronyc tracking

First, identify the source. Reference ID and Ref time show what Chrony selected and when the latest measurement was processed. A stale reference time deserves investigation even if the daemon is active.

Next, read correction. System time is the current difference between Chrony’s NTP clock and the system clock. Last offset describes the most recent update, while RMS offset summarizes a longer history. A small last offset does not erase a large remaining system correction that is still being slewed.

Finally, judge confidence. Frequency estimates how quickly the local clock would gain or lose time without correction. Skew is uncertainty in that estimate. Root delay and root dispersion contribute to the accuracy bound back to the reference clock. Leap status: Normal is useful, but it does not by itself prove a small offset.

Decode the source-state column

In chronyc sources -v, ^* marks the selected NTP server and ^+ marks another acceptable source being combined. ^- is selectable but not currently used. More concerning states include ^x for a source Chrony considers inconsistent, ^~ for excessive variability and ^? for a source that is not selectable.

Reach is an octal history of recent responses. 377 means all eight recent polls received a valid response, but reachable samples can still fail quality tests. Conversely, a newly started source may need several measurements before selection. Reachability and trust are different gates.

When every source is ^?, inspect DNS, routing, outbound firewall policy, provider-supplied source names and journal messages before editing offsets manually:

chronyc activity
journalctl -u chrony -u chronyd --since '30 minutes ago' --no-pager

One machine should have one time owner

Package migrations and copied images can leave more than one time client installed. Avoid fixing that ambiguity by enabling every service you recognize.

Inventory active synchronizers

systemctl is-active chrony chronyd systemd-timesyncd ntp ntpd 2>/dev/null
systemctl is-enabled chrony chronyd systemd-timesyncd ntp ntpd 2>/dev/null

Service names vary by distribution: Ubuntu commonly uses chrony.service, while Red Hat-family packages use chronyd.service. A missing unit is not an error in itself. The goal is to identify which installed unit intentionally owns the system clock.

Ubuntu’s current server documentation says Chrony became the default starting with Ubuntu 25.10 and explains that timesyncd steps back when Chrony is installed. Older supported Ubuntu releases may still begin with systemd-timesyncd. Red Hat Enterprise Linux uses Chrony in its current time-synchronization guidance. Diagnose the image you actually run instead of assuming one universal default.

timedatectl timesync-status specifically monitors systemd-timesyncd according to the systemd manual. On a Chrony-owned host, use chronyc for detailed source and correction evidence.

Include virtualization in the ownership record

VPS guests also depend on a virtual clocksource and hypervisor behavior. Suspend, delayed virtual CPU scheduling and live migration can affect timekeeping. Linux kernel documentation describes KVM timekeeping and migration boundaries, but guest-level symptoms alone cannot prove a host defect.

Capture the guest clocksource and recent boot messages before escalating:

cat /sys/devices/system/clocksource/clocksource0/current_clocksource
dmesg --ctime | grep -Ei 'clocksource|timekeeping|tsc' | tail -40

If the offset repeatedly jumps after migrations or suspends while several healthy NTP sources agree, include exact timestamps, tracking, sources -v, clocksource output and provider event history in the ticket. Teams evaluating environments for time-sensitive workloads can compare cloud VPS workload options without assuming that a plan change alone repairs guest configuration.

FAQ

Is a wrong timezone the same as clock drift?

No. A timezone changes how Linux displays an instant, while clock drift changes the underlying system time relative to UTC. If date -u is correct, adjust the timezone; do not step the clock.

What does ^? mean in chronyc sources -v?

^? means Chrony does not currently consider that NTP server selectable. Common reasons include an unreachable or unsynchronised source, unresolved address, insufficient measurements or a source rejected by selection limits. Check chronyc activity, reach, journal messages and source configuration before replacing the clock manually.

Is chronyc makestep safe on a production VPS?

Not automatically. chronyc makestep can jump wall time immediately, and Chrony’s own manual warns that some software can be seriously affected. Use it only when the measured offset justifies a step, the workload impact is understood, sensitive writers or schedulers are controlled and post-change acceptance tests are ready.

Why can TLS or login fail when the network is reachable?

TLS certificates, signed sessions and authentication protocols evaluate validity windows against wall time. A reachable peer can still be rejected when the VPS believes the certificate or token is not yet valid or already expired. Correct time, then repeat the exact authentication path instead of treating connectivity as proof.

Should Chrony and systemd-timesyncd run together?

One intended synchronizer should own the Linux system clock. Distribution packaging may automatically make systemd-timesyncd step back when Chrony is installed, but copied images and manual changes can create ambiguity. Inventory active and enabled units before changing either service.

How should Chrony clock health be monitored?

Monitor whether Chrony has a selected source, whether leap status becomes Not synchronised, whether reference time goes stale, and whether remaining correction or skew exceeds a workload-specific limit. Pair guest metrics with an outside signal; teams can monitor NTP reachability from independent infrastructure while retaining Chrony metrics for actual guest correction.

Can a VPS migration cause clock trouble?

Suspend and migration can complicate guest timekeeping, but correlation is not proof. Preserve provider event timing, guest clocksource, Chrony source states and offset history. Escalate when repeated jumps align with infrastructure events despite healthy guest configuration and sources.

Decide whether the clock may slew or must step

Chrony normally slews: it speeds up or slows down the system clock until the correction is absorbed. Slewing preserves wall-clock continuity, which is safer for many running applications, but a large error can take too long to disappear.

A step changes wall time immediately. That may be necessary when authentication is already unusable or the remaining correction exceeds the time the service can wait. It also creates the highest risk of backward timestamps, duplicate scheduler windows and inconsistent distributed state.

Write the decision in workload terms

Choose slew when the application can tolerate gradual correction and current error is within its validity window. Choose a controlled step only when all of these are true:

• the offset is measured from more than one observer;
• source selection is understood and trustworthy;
• gradual correction would exceed the business recovery window;
• time-sensitive writers, queues and wall-clock schedulers have a stop, pause or reconciliation plan;
• external acceptance tests are prepared before the change.

There is no universal “safe offset.” A stateless cache, Kerberos domain member, payment workflow and distributed database have different tolerances. Application semantics set the threshold.

Bound the step inside a change window

Chrony documents makestep for forcing the remaining correction and warns about application impact. If a step is approved, record tracking immediately before it, quiesce only the sensitive workload components identified in the plan, then run:

sudo chronyc makestep
chronyc waitsync 60 0.01
chronyc tracking

waitsync 60 0.01 waits up to about ten minutes at the default ten-second interval for synchronization and less than 10 milliseconds of remaining correction. Adjust that acceptance bound to the workload rather than copying 10 milliseconds as a universal service-level objective.

If a backward step already caused duplicate jobs or conflicting timestamps, returning the clock to correct UTC does not reconcile business data. Keep the incident open until application owners resolve those consequences.

Repair the source path, not only the symptom

A one-time step can hide a broken source path for a few minutes. Repair why Chrony lost trustworthy measurements.

Inspect effective source configuration

grep -RHE '^[[:space:]]*(server|pool|peer|makestep|rtcsync|minsources|require)[[:space:]]' /etc/chrony.conf /etc/chrony/chrony.conf /etc/chrony/conf.d 2>/dev/null

Confirm hostnames resolve, intended pools are present, and image-specific fragments have not overridden provider or organization sources. iburst accelerates initial measurement after startup; it does not make an unreachable source healthy. The current Chrony configuration manual explains source options, NTS and selection controls in detail.

Do not open inbound UDP/123 for an ordinary NTP client merely because synchronization failed. Client replies return to the client flow; serving time to other hosts is a separate intent and exposure decision. Use firewall logs and chronyc sources -v to diagnose actual path behavior.

Recheck after DNS, routing or config repair

TIME_SERVICE=chrony  # use chronyd on Red Hat-family systems
sudo systemctl restart "$TIME_SERVICE"
chronyc waitsync 60 0.01
chronyc tracking
chronyc sources -v

Restart only the installed owner. If both restart commands would succeed because two clients are active, resolve ownership first. A clean result includes a recent reference time, selected source, normal leap state and remaining correction inside the documented workload bound.

Image builders should promote time health into first-boot acceptance so an instance cannot enter rotation merely because cloud-init and the time daemon both report completion.

Test tokens, databases, jobs and logs after correction

Time synchronization is not the acceptance test; it is a prerequisite for acceptance.

Repeat the exact path that failed:

1. Request and validate a fresh TLS connection or signed token.
2. Execute a read and controlled write against the affected database, then inspect timestamps and ordering.
3. Confirm the next scheduled job time without manually triggering business work twice.
4. Compare application, proxy and remote-monitor logs across the correction boundary.
5. Resume paused writers or schedulers one at a time when their evidence is clean.

For a backward correction, search the incident interval for duplicate identifiers, non-monotonic timestamps and delayed queue items. For a forward correction, check whether timers or jobs suddenly became overdue. Monotonic timers usually continue to measure durations safely, but applications that compare wall-clock timestamps can still behave differently.

Keep the before/after Chrony snapshots with application evidence. A final Leap status: Normal screenshot cannot prove that tokens, jobs and data survived.

Make drift a monitored property

Monitoring only systemctl is-active chronyd detects a dead process, not a bad estimate. Build alerts from fields tied to workload risk:

• no selected source or Leap status: Not synchronised;
• reference time older than the expected polling/recovery window;
• remaining system correction above the application threshold;
• skew or root distance growing beyond the expected network baseline;
• repeated steps, clocksource changes or jumps aligned with migration events.

Collect those values as time series where possible. One sample explains the present; a trend distinguishes persistent frequency error, intermittent source loss and infrastructure-correlated jumps.

Avoid a universal millisecond threshold across every VPS. Public Internet sources, LAN sources and specialized PTP environments have different accuracy envelopes. Set warning and critical boundaries from the application’s actual token, database, scheduler and incident-correlation needs.

End with a clock evidence packet

Close the change with a compact record another operator can verify:

• UTC and local time before and after;
• active time owner and installed version;
• selected source, reference time and reach;
• system correction, frequency, skew and root distance;
• timezone and clocksource;
• cause, repair, slew/step decision and workload threshold;
• authentication, database, job and log acceptance results;
• monitor query, alert boundary and owner.

Clock recovery is complete only when the guest estimate is bounded and the application timeline is trustworthy again. Continue through Linux operations field guides when the same evidence-first discipline is needed for services, storage, networking or capacity incidents.