Every lean team has one. A crontab on a box that predates the current on-call rotation, running a job that touches production data, owned by a user account nobody remembers creating. It works, so it stays. Then it fires at 03:00 on a Sunday, the person it pages has never seen the script, and the first fifteen minutes of the incident are spent answering a more basic question: what is this job, and who was supposed to know about it?

This is not a story about cron being fragile. Cron is remarkably durable. The problem is that cron encodes almost no ownership metadata, and the metadata it does encode is easy to misread. An audit of scheduled jobs is really an exercise in reconstructing intent from a format that was never designed to carry it.

What the crontab format actually tells you

The crontab(5) format is five time-and-date fields, an optional username in system crontabs, and a command. That is the whole schema. There is no owner field, no description field, no ticket reference, no expiry. The manual page is explicit that commands in a given crontab execute under the user who owns that crontab, and that LOGNAME and HOME are set from the owner’s /etc/passwd entry while SHELL defaults to /bin/sh unless overridden. Ownership, in the operational sense, is inferred from the account, not declared.

That inference breaks in predictable ways. A job in /etc/cron.d/ carries a username field because those are system jobs used for more than one user. A job in a personal crontab does not, because the account is the identity. When the account is a shared service account like deploy or app, the crontab tells you which Unix user runs the job and nothing about which human is accountable for it. Those are different questions, and conflating them is how a job ends up firing at the wrong person.

Two format details matter more than they look. First, comments are not allowed on the same line as a cron command; the manual page notes they are treated as part of the command. So the tempting habit of appending # owner: alice to a job line does not document anything — it changes the command. Second, if both day-of-month and day-of-week are restricted, the job runs when either matches. A line like 30 4 1,15 * 5 runs at 04:30 on the 1st, the 15th, and every Friday. Teams that read it as “the 1st and 15th, if it’s a Friday” have a job running roughly four times more often than they believe.

Time handling is the other quiet source of surprise. The manual page states that non-existent times during a daylight-saving transition never match, so jobs scheduled in the skipped hour do not run, and times that occur twice cause matching jobs to run twice. The cron(8) page describes the daemon’s own handling: for local time changes under three hours, jobs that would have run in a skipped interval are run immediately, and a backward adjustment avoids running the same job twice. These two descriptions are not contradictory — they describe different layers — but the practical upshot for an audit is that “it runs at 02:30 daily” is not a stable statement across DST boundaries. If a job’s correctness depends on running exactly once per day, that assumption deserves a test, not a shrug.

Where the job’s output goes, and why that determines who gets paged

Failure attribution for cron jobs is mostly a question of where output and errors land. The cron(8) page is direct: any output is mailed to the owner of the crontab, or to the address in MAILTO if set. The crontab(5) page adds the detail that MAILTO="" suppresses mail entirely, and that MAILFROM controls the envelope sender. It also notes that -s directs job output to syslog, which is useful when sendmail is absent or mail is disabled.

This is the mechanism behind a common failure mode. A job’s MAILTO points at a distribution list that was retired, or at an individual who left, or at nothing because someone set MAILTO="" during a noisy week and never reverted it. The job fails silently for months. The first signal is a downstream symptom — stale data, a missing report, a backup that was never verified — and the person who notices is whoever depends on the output, not whoever owns the job.

An audit should therefore record, for each job: the effective MAILTO, whether mail is suppressed, whether -s is in use, and where the job’s own logging goes if it writes any. A job that logs to a file nobody reads and mails to an address nobody monitors is, for incident purposes, unowned regardless of what the crontab says.

System cron versus systemd timers: the logging and dependency difference

Lean teams often inherit a mix. System cron and systemd timers differ in ways that affect an audit’s scope.

Cron’s model is stateless and time-based. The daemon examines stored crontabs each minute and runs what matches. It reloads changed crontabs without a restart, either by checking modtimes every minute or via inotify. It has no concept of a job depending on another job, no built-in retry, and no structured exit-status record beyond whatever the command itself emits. The cron(8) page documents clustering support via a shared /var/spool/cron and a .cron.hostname file, which is worth knowing if you have ever wondered why a job runs on exactly one host in a group — and worth flagging, because that file is itself an undocumented dependency.

Systemd timers, by contrast, are units. They carry a description, they can declare dependencies and ordering, they record activation results in the journal, and they can be inspected with systemctl list-timers. For an audit, the practical difference is that a timer’s failure is queryable after the fact through journald, while a cron job’s failure may exist only as a mail message that was never delivered or a syslog line that rotated away.

This is a tradeoff, not a verdict. Cron is simpler to reason about and present on nearly everything. Timers give you structured state and dependency handling at the cost of more configuration surface and a unit-file review that most teams never do. The audit question is not which is better; it is whether you can answer “did this job run, and did it succeed?” for every scheduled job you have. If the answer depends on reading mail, the answer is no.

Backup jobs deserve their own audit pass

Scheduled jobs that touch backups are the ones where an undocumented crontab entry has the highest cost, because a backup job that has been failing quietly is indistinguishable from a backup job that has been succeeding quietly until the day you need to restore.

PostgreSQL’s continuous archiving documentation is explicit about the dependency chain. To recover using continuous archiving, you need a continuous sequence of archived WAL files extending back at least as far as the start time of your base backup. The archive command must return zero exit status if and only if it succeeds; a nonzero status tells PostgreSQL the file was not archived and it will retry periodically. The documentation also warns that when the archive command is terminated by a signal or fails with an exit status greater than 125, the archiver process aborts and restarts, and that failure is not reported in pg_stat_archiver. That last point is the audit-relevant one: a monitoring check built only on pg_stat_archiver can miss a class of archiving failures entirely.

The documentation further notes that if archiving falls significantly behind, the amount of data that would be lost in a disaster increases, and pg_wal/ accumulates unarchived segments that can eventually exhaust disk. It recommends monitoring the archiving process. For a lean team, that recommendation translates into a specific audit item: for every scheduled job that archives WAL, record what monitors it, what the alert threshold is, and whether the monitor distinguishes “archiving is behind” from “archiving is broken.”

pgBackRest’s user guide is useful here for a different reason: it documents the retention and dependency semantics that determine whether a restore will actually work. A differential backup requires the prior full backup to be valid; an incremental backup requires all prior incrementals back to the prior differential, the differential itself, and the full backup. The guide states that a restore requires the backup files and one or more WAL segments. The audit implication is that a retention policy which expires a full backup while keeping incrementals that depend on it produces a repository that looks populated and cannot be restored. If your scheduled expire job runs on a crontab nobody owns, that is the failure you are auditing for.

Restic’s documentation covers the same territory from the repository side. It documents restic check for integrity and consistency verification, and it documents exit status codes and JSON output for scripting. The relevant audit point is that a scheduled restic check that runs but whose output goes to a mailbox nobody reads provides no assurance. The check has to be wired to something that fails loudly.

None of this substitutes for an actual restore. A scheduled backup job that has never been restored from is a hypothesis. The recovery checklist approach — writing down the steps before you need them — is the natural companion to this audit, because the audit tells you which jobs exist and the checklist tells you what to do when one of them turns out to have been lying.

A workable audit procedure

The goal is a single inventory that answers, for every scheduled job: what runs, as whom, on which hosts, writing where, alerting whom, and who is accountable. The procedure below is deliberately mechanical.

1. Enumerate. Collect every source of scheduled execution: user crontabs in /var/spool/cron, system crontabs in /etc/crontab and /etc/cron.d/, /etc/anacrontab, systemd timers via systemctl list-timers --all, and any scheduler that lives inside an application (Celery beat, Sidekiq cron, Kubernetes CronJobs). The last category is the one most often missed, because it is not on the box you are auditing.

2. Resolve the effective environment. For each cron entry, record the effective SHELL, HOME, PATH, MAILTO, and CRON_TZ. The manual page notes that PATH is set by cron unless -P is used, and that HOME and SHELL can be overridden in the crontab while LOGNAME cannot. A job that works interactively and fails under cron is usually a PATH or HOME difference, and recording the effective values turns that from a debugging session into a lookup.

3. Trace the command. Follow the command to its script, and record what the script does at a level a new on-call engineer could act on. If the script calls other scripts, follow those too. The audit output is not a copy of the code; it is a sentence or two of intent plus a pointer to the source.

4. Identify the accountable human. This is the step that requires a conversation, not a command. For each job, name a person, not a team. If nobody will claim it, that is the finding — not a gap in the spreadsheet. A job with no accountable owner is a candidate for deletion, and deletion is a legitimate audit outcome.

5. Record the failure path. For each job, answer: if this fails, who finds out, how, and how long would it take? If the answer is “nobody” or “when someone notices the data is wrong,” write that down. It is the most important column in the inventory.

6. Test the schedule assumptions. For any job whose correctness depends on running exactly once per interval, verify the DST behavior and the day-of-month/day-of-week semantics against the manual page rather than against intuition. For jobs using step values, remember that steps are evaluated within their field: */35 in the minute field runs at minute 0 and minute 35, not every 35 minutes.

7. Validate syntax before changes. The crontab(5) page notes that crontab syntax can be tested before installation using the -T option. Use it. An audit that introduces a broken crontab is a self-inflicted incident.

What to do with the inventory

The inventory is not the deliverable. The deliverable is a set of decisions: jobs to delete, jobs to move to a timer with structured logging, jobs to add monitoring for, jobs to document in a runbook, and jobs to hand to a named owner.

Two of those decisions deserve emphasis for lean teams.

Deletion is underrated. A scheduled job that has run without anyone noticing its output for a year is, in most cases, not load-bearing. The cost of keeping it is not zero: it is another thing that can page someone at 03:00, another thing to audit next year, another line in the inventory that dilutes attention. The audit is the moment to ask whether the job still earns its place.

Monitoring is the other. A scheduled job without a failure signal is a job you will learn about from its consequences. The minimum viable signal is a heartbeat: something that alerts when the job doesn’t run, not just when it exits nonzero. Cron’s mail-based failure reporting covers the second case and not the first, which is why a job that never starts is the quietest failure of all.

Postmortems for scheduled-job failures

When a scheduled job does cause an incident, the postmortem is where the audit’s findings become durable. Google’s SRE book chapter on postmortem culture is a useful reference for the practice, and its framing applies directly here. The chapter defines a postmortem as a written record of an incident, its impact, the actions taken to mitigate or resolve it, the root cause or causes, and the follow-up actions to prevent recurrence. It lists common triggers including user-visible downtime beyond a threshold, data loss of any kind, on-call engineer intervention, resolution time above a threshold, and monitoring failure — the last of which is exactly what an unmonitored cron job produces.

The chapter’s argument for blamelessness is the part that matters for this topic. It states that a blameless postmortem focuses on identifying contributing causes without indicting any individual or team, and that it assumes everyone involved acted with good intentions given the information they had. The reasoning is practical: if blame is the norm, people stop surfacing issues, and the issues that go unsurfaced are the ones that cause the next incident. For a scheduled job that fired at the wrong person, the blameless framing points at the real question — why did the system allow a job to exist without a named owner and a failure signal? — rather than at whoever wrote the crontab entry three years ago.

The chapter also recommends defining postmortem criteria before an incident occurs, so that everyone knows when one is required. For lean teams, a workable criterion is: any scheduled job whose failure was discovered by a human noticing a downstream symptom gets a postmortem. That is a narrow, testable trigger, and it captures the class of failure this audit is designed to reduce.

Keeping the inventory alive

An audit that produces a document and no process is a snapshot that decays. The inventory needs a home and a review cadence. Two practices keep it current without much overhead.

First, tie the inventory to onboarding and offboarding. When someone joins, the jobs they inherit are part of their onboarding. When someone leaves, the jobs they owned are part of their offboarding checklist. This is the same access-hygiene discipline applied to scheduled work, and it is the only reliable way to prevent the “owned by a departed engineer” failure mode.

Second, make the inventory a required input to any change that adds a scheduled job. A new cron entry or timer should not merge without an owner, a failure signal, and a line in the inventory. This is a small process cost paid at the moment of creation, when the context is fresh, instead of a large reconstruction cost paid during an incident.

Questions that come up during these audits

How do I find crontabs I don’t know about? Start with the documented locations: /var/spool/cron for user crontabs, /etc/crontab and /etc/cron.d/ for system jobs, and /etc/anacrontab. Then check systemd timers and any application-level scheduler. The cron(8) page lists the files and directories the daemon checks, which is the authoritative starting point.

Why did a job run twice? Check the DST behavior first. The crontab(5) page notes that times occurring more than once during a transition cause matching jobs to run twice. Also check whether the job exists in more than one crontab, or on more than one host in a group where clustering is not configured.

Why didn’t a job run at all? Check whether the scheduled time fell in a DST gap, which the manual page says will never match. Check whether the crontab file is missing a trailing newline, which the manual page warns causes cron to consider the crontab at least partially broken and write a warning to syslog. Check whether the file permissions satisfy cron’s requirement that crontab files be regular files or symlinks to regular files, not executable or writable by anyone but the owner.

Where did the job’s output go? Check MAILTO in the crontab. If it is set and non-empty, mail goes there. If it is set but empty, no mail is sent. If it is unset, mail goes to the crontab owner. If -s is in use, output goes to syslog instead.

How do I know a backup job actually produced a restorable backup? You don’t, from the job’s exit status alone. PostgreSQL’s documentation requires a continuous WAL sequence back to the base backup’s start time, and pgBackRest’s documentation describes the dependency chain between full, differential, and incremental backups. The only reliable verification is a restore, which is why the recovery checklist and the audit belong together.

Should we migrate everything to systemd timers? Not necessarily. Timers give you structured logging and dependency handling; cron is simpler and more portable. The decision should follow from whether you can currently answer “did this job run and succeed?” for each job. If you can, the migration is optional. If you can’t, the migration is one way to fix that, and adding a heartbeat monitor is another.

The point of the exercise

An unowned crontab entry is not a crisis. It is a small, accumulating liability that stays invisible until the moment it becomes expensive. The audit is cheap: a few hours of enumeration and a conversation per job. The alternative is discovering the inventory during an incident, one job at a time, with the clock running and the person who wrote the job unavailable.

The output that matters is not the spreadsheet. It is the set of jobs you deleted, the jobs you gave an owner, and the jobs you wired to a signal that fires before a human notices the consequences. Everything else is documentation, and documentation that nobody reads is just another unowned artifact.