How to Maintain a Service You Inherited from a Team That No Longer Exists

How to Maintain a Service You Inherited from a Team That No Longer Exists

Inheriting a production service from a team that no longer exists is a strange kind of ownership. You now hold the runtime, the deployment path, the access model, the alerting, and the recovery assumptions—without any of the original context. This is not a handoff. It is an archaeology project with a live site attached. For lean technical teams of two to fifteen engineers running cloud-native infrastructure on AWS, GCP, or bare metal, the danger is not that the service is complicated. The danger is that the service is silent about its own failure modes. The work is to make the inherited system legible, recoverable, and boring again.

This article walks through a repeatable sequence for taking over an orphaned service: inventory what actually runs, map the failure and recovery path, reduce access and configuration drift, rebuild monitoring around the metrics you can verify, and run a recovery drill before the first real incident. The goal is not to rewrite the service. The goal is to make the next engineer—possibly you at 02:00—able to operate it without tribal knowledge.

Two engineers reviewing a whiteboard diagram of a service architecture
Start by drawing the system as it actually runs, not as the old documentation claims it runs.

Start with a Runtime Inventory, Not the README

The first artifact to build is a runtime inventory: what is deployed, where it runs, what it depends on, and who can touch it. Do not start with the repository README. READMEs describe intent. The runtime describes behavior. In inherited systems, the two often diverge by months or years.

For AWS, begin with aws resourcegroupstaggingapi get-resources filtered by the service tag, then cross-check with aws ec2 describe-instances, aws rds describe-db-instances, and aws lambda list-functions. For GCP, use gcloud asset search-all-resources with a project or label filter. For bare metal, start with ss -tulpn on each host and compare the listening ports to the process list. The inventory should answer four questions per component: what is it, who deploys it, what does it call, and what calls it.

Record the inventory in a plain text file or a table in the repository. Do not build a new tool for this. A docs/ directory with one file per component is enough. The inventory is not documentation for its own sake. It is the input for the recovery checklist and the access review that follow.

Map the Failure and Recovery Path Before Touching Anything

Once the inventory exists, map the failure path for the two or three most likely incidents: database unavailable, dependency timeout, disk full, credential expiry, and deploy rollback. For each, write the exact commands or console steps required to detect the failure, stop the bleeding, and restore service. This is the same discipline as writing the recovery checklist before you need it, but applied to a system you did not build.

The recovery path should include the rollback command for the current deployment mechanism. If the service deploys via kubectl apply, the rollback is kubectl rollout undo deployment/<name>. If it deploys via AWS CodeDeploy, the rollback is an aws deploy rollback-deployment call or a console action. If it deploys via a shell script on a bare-metal host, the rollback is a copy of the previous binary or configuration. If you cannot identify the rollback path in under ten minutes, that is a finding, not a footnote. Write it down and schedule a fix.

Do not attempt to improve the architecture during this phase. The inherited service is in production. The first job is to make the current state recoverable. Improvements come after the recovery drill proves the baseline works.

Reduce Access and Configuration Drift in the First Week

Orphaned services accumulate access. Former team members retain IAM roles, SSH keys, API tokens, and database credentials. The first week of ownership should include an access review that removes every identity that cannot be tied to a current on-call or deployment need.

On AWS, use aws iam get-account-authorization-details and look for users and roles with unused access keys. The PasswordLastUsed and AccessKeyLastUsed fields are the evidence. On GCP, use gcloud projects get-iam-policy and compare members against the current team roster. On bare metal, review /etc/sudoers, ~/.ssh/authorized_keys, and any service account files. Remove or disable anything stale. If a credential is shared, rotate it and store the new value in the team’s existing secret manager—AWS Secrets Manager, GCP Secret Manager, or a self-hosted vault.

Configuration drift is the second target. Compare the running configuration to the repository. For Kubernetes, kubectl get deploy -o yaml against the committed manifests. For AWS, aws cloudformation describe-stacks or aws ssm get-parameter against the stored values. For bare metal, diff /etc/ files against the configuration management repo. Every difference is either a manual fix that was never committed or a configuration change that was never applied. Both are risks. Record them and close the gap.

A laptop showing a terminal with configuration files and a diff view
Configuration drift is easiest to see when you diff the running system against the repository.

Rebuild Monitoring Around Metrics You Can Verify

Inherited services often have alerting that is either too noisy or too quiet. The old team tuned it for their own mental model. You need to rebuild it around the four signals that matter for a service without dedicated SRE coverage: latency, errors, saturation, and a business-level health check.

Start with the metrics you can verify directly. For a web service, that means request duration at the load balancer or application, error rate from the application logs or the load balancer metrics, and a synthetic check that exercises a real user path. On AWS, CloudWatch metrics for ALB request count, target response time, and HTTP 5xx count are the baseline. On GCP, Cloud Monitoring metrics for the load balancer and the application. On bare metal, Prometheus with the node_exporter and an application exporter if one exists.

Do not import the old team’s dashboards wholesale. Rebuild them from the current runtime inventory. Each dashboard panel should map to a component in the inventory and a step in the recovery checklist. If a panel does not help you decide whether to page someone, remove it. Alert on symptoms, not on individual host metrics. A high CPU alert is only useful if it predicts a user-facing failure. A high error rate alert is useful immediately.

Run a Recovery Drill Before the First Real Incident

The recovery checklist is a hypothesis until you run it. Schedule a one-hour drill in the first two weeks of ownership. Pick the most likely failure from the failure path map—usually a database restart or a dependency timeout—and execute the checklist exactly as written. Do not skip steps. Do not improvise. The drill is a test of the checklist, not a test of your memory.

During the drill, record every place where the checklist was wrong, incomplete, or ambiguous. Did the rollback command require a flag that was not documented? Did the health check pass before the service was actually ready? Did the alert fire at all? Each gap is a work item. Fix the checklist immediately after the drill, then run the drill again the following week. Two successful drills in a row is the minimum bar for calling the service recoverable.

This is the same pattern used in Google’s SRE approach to emergency response, where the goal is to make the response path mechanical before the incident adds pressure. For a lean team, the drill is the only reliable way to build that muscle without a dedicated SRE.

Document the Service as a Set of Runbooks, Not a Wiki

The final step is to write the runbooks. A runbook is a short, command-first document for a specific operational task: deploy, rollback, restart, scale, credential rotation, and incident response. Each runbook should fit on one screen. If it does not, split it.

Store the runbooks in the same repository as the service, in a runbooks/ directory. This keeps the operational knowledge next to the code it operates. Use plain Markdown or plain text. The runbook should include the exact command, the expected output, and the rollback or verification step. Do not write paragraphs of explanation. Write the command, the check, and the undo.

For the incident response runbook, include the escalation path, the communication channel, and the link to the recovery checklist. The runbook is not a substitute for the checklist. It is the entry point that tells the on-call engineer where to start.

Tradeoffs and What to Skip

There are several tempting projects to skip during the first month of ownership. Do not rewrite the service. Do not migrate it to a new platform. Do not introduce a new observability stack. Do not refactor the deployment pipeline. Each of these projects consumes the time you need for the inventory, the access review, the monitoring rebuild, and the recovery drill. The inherited service is already in production. The risk is not that it is old. The risk is that it is unowned.

There is also a tradeoff in how much to automate. For a lean team, automation is valuable only when the underlying process is stable. Automating a broken recovery path just makes the failure faster. Run the manual checklist first. Automate only the steps that survive two successful drills.

What This Creates for the Site

This article is part of a recurring column on taking over orphaned infrastructure. The next article in the series will cover how to run a service handoff interview when the previous team is still partially available, including the ten questions that produce the most useful operational answers. If you have inherited a service and found a gap in this sequence, send a note through the contact page. Reader questions become the next runbook.

A small team gathered around a monitor during an incident review
The goal is to make the next incident boring: detected, diagnosed, and recovered from a checklist.

Frequently Asked Questions

What is the first thing to do when you inherit a service from a team that no longer exists?

Build a runtime inventory. List every running component, its dependencies, and its access model. Use cloud provider APIs or host-level commands to capture what actually runs, not what the old documentation claims. The inventory is the input for the recovery checklist, the access review, and the monitoring rebuild.

How do you know if an inherited service is recoverable?

Run a recovery drill. Pick the most likely failure, execute the recovery checklist exactly as written, and record every gap. Fix the checklist and run the drill again. Two successful drills in a row is the minimum bar for calling the service recoverable. A checklist that has never been run is a guess.

Should you rewrite an inherited service to make it easier to maintain?

Not in the first month. The service is already in production. The immediate risk is that it is unowned, not that it is old. Focus on inventory, access review, monitoring, and recovery drills. A rewrite or platform migration can be evaluated after the service is operationally legible and recoverable.

What is the difference between a runbook and a recovery checklist?

A runbook is a command-first document for a specific operational task, such as deploy, rollback, or credential rotation. A recovery checklist is the ordered sequence of steps for responding to a specific failure. The runbook is the entry point. The checklist is the response path. Both should be stored in the service repository.