CD and Deployment Strategies

Deployment is the moment a registered model version starts answering real traffic. CD for machine learning links two releases — the service code and the model weights — and chooses a strategy that can be reversed in seconds, plus an automated signal that decides whether it should be.

The idea

Two releases, deliberately decoupled:

• Code — the serving image, built once per commit and deployed like any web service.
• Weights — a version chosen from the registry, often without rebuilding the image, via a config value, environment variable, or alias.

This is why serving code should resolve models:/churn@champion instead of hardcoding a checkpoint path. Moving an alias promotes or rolls back a model with no image build and no code review.

The standard strategies:

• Recreate — stop, then start. Simple, with downtime.
• Rolling — replace replicas gradually. Requires the service to tolerate mixed versions for a few minutes.
• Blue-green — run two full environments and switch traffic. Instant rollback at double the cost.
• Canary — send a small slice of real traffic to the new version and watch error rate, latency, and the business metric before widening.
• Shadow — mirror real traffic to the new version without using its answers. Zero user risk; the right tool when the model is hard to A/B test.

Guardrails that make the strategy real: an automated rollback on SLO breach, a warmup so the first requests do not pay model-load cost, and a written runbook for the rare manual promotion.

Worked example

Promoting model version 9 to replace version 8:

1. Point one canary replica at challenger (version 9) while the rest keep champion (version 8).
2. Watch error rate, p95 latency, and the business metric over a fixed window.
3. If the gate passes, move champion to version 9 — one registry call. Every replica picks it up on its next refresh.
4. If the gate fails, production never changed. Rollback is the alias move back to 8.

Note that the image was deployed once. Only the alias changed, so the risky part of the release (new weights) is independently reversible from the boring part (a code deploy).

In code

- name: point champion at the candidate version
  run: python -m mlflow_alias set --name churn --alias champion --version ${{ inputs.version }}
- name: wait for rollout
  run: kubectl rollout status deploy/model-api --timeout=120s
- name: roll back on failure
  if: failure()
  run: kubectl rollout undo deploy/model-api

The rollback step must be able to run without a human, or it will not run at all when it matters.

Check yourself

1. Why keep model promotion separate from image deployment?
2. When is a shadow deployment better than a canary?
3. What has to be automated for rollback to be trustworthy at 3 a.m.?

Key takeaways

• Deploy service code and promote models on separate, independently reversible rails.
• Use canary to limit user impact and shadow for zero-risk comparison.
• An alias move plus an automatic rollback on SLO breach is the whole safety net.