Systems Design 8 min read

Script, Configure, or Orchestrate

There are three ways to make a machine do what you want. You can script it, writing the steps out in order. You can configure it, describing the end state you want and letting a tool work out how to get there. Or you can orchestrate it, coordinating many such units across many machines over time. None of these is more advanced than the others. Each is the correct answer to a different shape of problem, and the recurring mistake, at every level of experience, is to reach for the mode you know best and stretch it well past the point where a different one was right. The skill that matters is not mastery of any single mode. It is recognizing, before you hit the wall, which one the problem in front of you is actually asking for.

The three modes, briefly

Imperative work is scripting. You specify the steps: do this, then that, check the result, hand off. A shell script is imperative, and for sequencing and gluing tools together it is exactly the right instrument.

Declarative work is configuration management. You specify the end state rather than the steps, and a tool computes the difference between how things are now and how you said they should be, then closes that gap. These packages installed, these users present, this file with these contents, this service running. You do not write the how. The tool owns the how, and owns making it safe to run over and over. This is the category Ansible belongs to.

Orchestration is coordination. You are no longer describing one host's state but keeping a system of services coherent across many machines over time, with scheduling, health checking, dependency ordering, and scaling. The unit of concern has moved from a machine to a fleet. This is the tier where Kubernetes and its neighbors live.

The tell that you have outgrown a script

The first boundary is the one people cross without noticing they crossed it. A script begins as a clean sequence. Then it grows a check: create the user if it is not already there. Then another: install the package if it is missing, and the correct version if the version is wrong. Then repair the file if its contents have drifted. When the dominant logic of your script has become look at the current state, compute how it differs from what I want, and apply only the difference, you are no longer writing a script. You are writing a convergence engine by hand, and worse than the tools built for nothing else, because they get idempotency right by construction while your pile of accumulated if-statements does not.

The boundary is about the moment state management quietly became the whole job, and line count has nothing to do with it. If you want a mechanical test rather than a feeling, count the branches that exist only to answer "is it already like this?" When those outnumber the branches that do actual work, the script has inverted: the steps have become a minority of the code and the state inspection has become the majority. That ratio is the tell, and it is visible in a diff long before it is visible in an outage.

There is a second signal worth watching for, and it arrives later and hurts more. Ask whether anyone can predict what your script does on a host it has already run against successfully. A convergence tool answers that question by design: nothing, because the state already matches. A hand-built equivalent answers it only by inspection, one branch at a time, and the answer changes every time someone adds a case. When the second run of a script is harder to reason about than the first, you have taken on the hardest obligation in the declarative category without any of the machinery that makes it tractable.

The tell that you have outgrown configuration management

The second boundary is quieter, because fewer problems reach it. Configuration management holds a host in a desired state. When the question stops being what state is this host in and becomes which services run where, in what order they depend on each other, what happens when one of them dies, and how the whole thing scales under load, you have crossed into orchestration. The tell is that you are hand-managing placement and restarts and cross-service dependencies, and steadily losing. That is a coordination problem, and coordinating a changing fleet is precisely what an orchestrator exists to own.

The sharper version of that tell is about time. Configuration management runs and finishes; it is a transaction against a host, and between runs nothing is watching. Orchestration is a control loop that never exits, continuously comparing desired against actual and acting on the difference. So the real question at this boundary is whether your problem needs something awake between runs. If the answer is that a failure at three in the morning has to be noticed and acted on before the next scheduled run, no amount of configuration management fills that gap, because the thing you need is a process that was watching when it happened.

Crossing at the wrong time cuts both ways

Cross too late and you get the artifact everyone in operations has met: the thousand-line script that provisions a fleet, the fragile loop over a list of hosts, the convergence engine held together with retries, the thing nobody will touch because only its author knows which of its checks are load-bearing.

Cross too early and you pay a different tax. A three-line scheduled job does not need an orchestrator, and wrapping it in one buys a platform's worth of complexity to solve what a single cron entry already solved. Reaching for the heavy declarative tool because it is the modern-sounding answer is the same error as the overgrown script, simply run in the opposite direction. The ideal and the actual are both real, and the skill is knowing which terrain you are standing on.

The asymmetry between those two errors is worth naming, because it should change how you bet when you are unsure. Crossing too late produces a mess you can migrate out of incrementally, one responsibility at a time, with the old script still running while you do it. Crossing too early produces a dependency that everything else gets built against, and unwinding it means unwinding whatever grew on top. Both are mistakes. Only one of them compounds.

What I can tell you, and what I cannot

I can tell you where these lines sit because I have spent two decades watching imperative scripts fail at exactly the things declarative tools exist to solve, and watching heavy tooling dragged in to babysit problems a script had already handled cleanly. Recognizing the boundary is one skill. Operating the tool on the far side of it fluently is a different skill, and I am going to be straight about which of the two this piece is.

It is the first one. I know when a problem belongs in configuration management and why, and I will argue that boundary with anyone. My hands-on depth in the specific declarative and orchestration tools that live past the line is something I am still building, deliberately and in the open, and I would rather say that plainly than perform a fluency I have not earned yet. The judgment is the part that transfers between tools and outlives any of them. The tool fluency is the part you should verify separately, in me or in anyone who claims it. A portfolio that blurs those two is hoping you will not check. This one is pointing at the line on purpose.

Why the judgment is the durable part

The tools on the declarative and orchestration side move fast. Frameworks rise and fade, and the one that is correct this year may not be the one that is correct next year. What does not move is the shape of the decision: is this a sequencing problem, a state problem, or a coordination problem. Answer that correctly and the current right tool becomes a lookup. Answer it wrong and no amount of tool skill rescues you, because you are aiming a good tool at the wrong shape of problem. This is the working half of modernizing a system you cannot redeploy: you do not have to have adopted every tool to reason correctly about which one a given problem is asking for.

So before the next script grows its fourth "if it does not exist, create it," stop and name the problem. If you are sequencing, script it. If you are enforcing state, describe the state and hand the how to something built to own it. If you are coordinating a fleet of services over time, that is orchestration, and a script pretending otherwise does not end well.

The naming costs one minute at the start of the work, and it is the only point at which the answer is cheap to act on. Every branch you add afterward raises the price of moving, which is why the question belongs at the beginning of a script's life rather than at the point where someone finally admits it has become something else.

Systems Design DevOps