docs/guides/deploying to production

Deploying to Production

Best practices for safely rolling out AI models to live robots — including simulation testing, staged rollouts, and rollback strategies.

Pre-Deployment Checklist

Before pushing any model to live hardware, verify:

Model accuracy ≥ your defined acceptance threshold
Validation loss has converged (no large gap vs. training loss)
Inference latency p99 is within your real-time budget
Model has passed simulation testing (see Step 1)
A rollback target is identified (previous active version)
Monitoring alerts are configured for this robot

1 Run Simulation Tests

Artemis integrates with Gazebo and Webots for hardware-in-the-loop testing. Run your model against a physics simulation before touching live hardware:

bash

kairo simulate \
  --model mdl_abc123 \
  --env gazebo \
  --scenario pick-place-standard \
  --runs 100

# Running 100 simulation episodes...
# Success rate:      96/100 (96%)
# Avg cycle time:    4.2 s
# Collision events:  2
# Simulation report: sim_rpt_xyz.pdf

⚠

A simulation success rate below 90% is a strong signal the model needs more training data before production deployment.

2 Staged Rollout

For fleets with multiple robots, always use a staged rollout. Deploy to a small percentage first, monitor for 30 minutes, then complete the rollout:

bash

# Deploy to 20% of the fleet first
kairo deploy push \
  --model mdl_abc123 \
  --fleet flt_xyz123 \
  --rollout-percent 20

# Check status after 30 minutes
kairo deploy status --fleet flt_xyz123

# If healthy, complete the rollout
kairo deploy complete --fleet flt_xyz123

Or deploy to a specific robot for initial validation:

bash

kairo deploy push \
  --model mdl_abc123 \
  --robot rob_a1b2c3d4   # canary robot

3 Full Production Push

Once validation passes, deploy to all robots:

bash

kairo deploy push \
  --model mdl_abc123 \
  --fleet flt_xyz123

# Deploying to 24 robots...
# [████████████████████] 24/24 complete
# All robots running mdl_abc123 v2

ℹ

Deployments are zero-downtime. Artemis loads the new model weights alongside the running model, switches inference traffic once the new node is healthy, then unloads the old model.

4 Rolling Back

If a deployment causes issues, rollback to the previous version instantly:

bash

# Rollback a single robot
kairo deploy rollback --robot rob_a1b2c3d4

# Rollback the entire fleet
kairo deploy rollback --fleet flt_xyz123

# Rollback to a specific version
kairo deploy rollback \
  --robot rob_a1b2c3d4 \
  --version v1

Artemis caches the two most recent deployed versions on each robot, so rollbacks complete in under 5 seconds without re-downloading model weights.

Automatic Rollback

Configure automatic rollback triggers in your deployment config. If a metric breaches a threshold after a new deployment, Artemis rolls back automatically:

yaml

# kairo-deploy.yaml
auto_rollback:
  enabled: true
  window: 10m          # evaluate metrics for 10 min after deploy
  triggers:
    - metric: error_rate
      threshold: 0.05  # rollback if error rate exceeds 5%
    - metric: inference_latency_p99
      threshold: 100ms # rollback if p99 latency exceeds 100 ms

←

Training Your First Model

Fleet Monitoring

→

Was this page helpful?