Example: testing and CI¶

This example covers running rodeo-cli in automated and testing contexts — from fast unit tests on any machine to full integration tests on a KVM host.

Two tiers of testing¶

Tier	What it covers	Where it runs	Time
Unit / lint	Config parsing, CLI surface, preflight logic, labseed, phase event model	Any machine (no KVM)	~1 min
Integration	Full deploy pipeline: kvm_host → vms → pxe_server → cluster → rancher	Self-hosted KVM host	1–3 h per profile

You need both. Unit tests catch regressions in Python code. Integration tests catch the real failures: disk sizing, network, iPXE chain, etcd bootstrap.

Unit tests¶

Run on any OS — no KVM, no libvirt, no sudo needed:

pip install -e ".[dev]"
ruff check rodeo tests
pytest tests/ -v
# Ansible roles (optional locally; required in CI):
pip install "ansible-lint>=25" "ansible-core>=2.16,<2.19"
ansible-galaxy collection install -r rodeo/data/ansible/requirements.yml
ansible-lint rodeo/data/ansible

To keep your host clean (or when the host has a live libvirt that interferes with the stop/start tests), run the same checks in throwaway containers — this mirrors the CI matrix exactly:

scripts/test-in-container.sh                 # Python 3.10 + 3.12, ruff + pytest
scripts/test-in-container.sh 3.12            # one version
scripts/test-in-container.sh 3.12 -- -k labinabox   # pass pytest args after --

What the unit tests cover:

Test file	Area
`test_config.py`	Plan loading, Jinja2 rendering, `-P` overrides, `??` secret resolvers, validation
`test_runner.py`	Pipeline events, Instruqt guard, vars file generation
`test_ansible_consistency.py`	Profile ↔ Ansible defaults ↔ plan template alignment
`test_ansible_vars_contract.py`	Vars file keys match what Ansible roles consume
`test_pxe_integration.py`	Playbook order, pxe_server role, boot order, JOIN VIP guard
`test_cluster.py`	ClusterPhase poll loops, timeouts, node parsing
`test_rancher.py`	RancherPhase API call sequence, timeout handling
`test_plan_cmd.py`	`rodeo plan` diff output
`test_labseed.py`	Profile scaffolding, `rodeo new`, secrets generation
`test_preflight.py`	`detect_host`, `recommend_profile`, RAM/disk checks

Test the CLI surface without KVM¶

rodeo up --no-deploy seeds a lab and runs preflight but stops before the deploy pipeline starts. Useful for validating the on-ramp flow in CI without a KVM host:

rodeo up --no-deploy --yes --profile test --dir /tmp/test-lab
ls /tmp/test-lab   # rodeo-plan.yaml + definition.yaml should be present
cat ~/.rodeo/secrets.yaml   # auto-generated if it did not exist

Test config loading and overrides¶

# Dry-run: load and validate a plan, print what would deploy
cd /tmp/test-lab
rodeo plan

# Override a value and see it reflected
rodeo plan -P resources.harvester.memory_mib=20480

Integration tests — manual¶

On a host that meets the bare metal requirements, run one profile at a time:

# Clean any previous state first
rodeo clean --all --yes --secrets --force-network

# Deploy and verify (--no-tmux skips the session wrap for scripted runs)
rodeo up --profile test --yes --no-tmux
rodeo status

# Clean up for the next run
rodeo clean --all --yes --secrets --force-network

Profile test order (smallest to largest)¶

Run in this order to fail fast on smaller profiles before investing time in bigger ones:

rancher (~10 min) — fastest; validates the boot + Rancher path with no PXE
test (~60 min) — 2-node Harvester; validates the full iPXE chain
harvester-ha (~90 min) — 3-node Harvester; validates 3-member etcd HA
harvester (~120 min) — full lab; validates Rancher import on top of Harvester

AWS live smoke (`i7i.8xlarge` + NVMe)¶

Manual checklist for infra-adaptive host context (provision or BYO). Needs AWS creds and nothing else: the default AMI (SLES 16 PAYG) needs no Marketplace subscription, and i7i.8xlarge supports nested virt.

pip install 'rodeo-cli[aws]'
# Provision path (recommended tier → i7i.8xlarge for harvester; region capacity checked first):
rodeo up --yes --profile harvester --target aws --instance-tier recommended
# Or BYO: SSH to an existing i7i.8xlarge, set deployment_target: aws, then:
#   rodeo up --yes --profile harvester

Testing your own changes: the instance installs rodeo-cli from GitHub, so a local working tree is invisible to it — and by default an already-bootstrapped host keeps the code it was first installed with. Push, then pass the ref:

rodeo up --yes --no-tmux --target aws --ref feat/my-fix

--no-tmux matters for scripted or agent-driven runs: without it up re-execs into tmux and the caller never sees the output. The remote log to read is /root/.rodeo/logs/aws-up.log on the host — far more informative than what surfaces locally.

Verify on the KVM host after kvm_host:

resources.harvester.disk_gb is 300 and resources.rancher.disk_gb is 60 — flat per-node floors, not scaled by node count (plan / vars)
storage.backend: nvme and image_dir is mounted on instance-store NVMe (findmnt, lsblk)
Guest disks live under that image_dir (not root EBS alone)
Harvester VIP / rodeo status healthy; tear down with rodeo destroy --cloud --yes if provisioned

Unit coverage: tests/test_host_context.py, tests/test_kvm_host_nvme_storage.py.

Longer agent/ops checklist (single-host + fleet): aws-fleet-claude-test-plan.md.

Integration tests — GitHub Actions (self-hosted runner)¶

The integration workflow is at .github/workflows/integration.yml. It runs on a self-hosted runner with the labels self-hosted, kvm, sles16.

Trigger manually¶

From the GitHub UI: Actions → Integration tests → Run workflow → select a profile or leave blank for all.

From the CLI:

# All profiles (nightly-style)
gh workflow run integration.yml

# One profile only
gh workflow run integration.yml -f profile=test

What each job does¶

Pre-run cleanup — wipes whatever the previous run left (rodeo clean --all --yes --secrets --force-network), ignores failures so a broken previous run does not block the next one
Deploy — rodeo up --profile <name> --yes
Verify — rodeo status (asserts VMs are running and VIP is reachable)
Post-run cleanup — always runs, even if deploy failed

max-parallel: 1 ensures only one profile runs at a time on the shared KVM host. fail-fast: false means all four profiles run even if one fails, giving you the full picture.

Setting up a self-hosted runner¶

See CONTRIBUTING.md or the GitHub Actions runner docs for setup steps. The runner must have:

rodeo install-deps run once (KVM, libvirt, ansible, kubectl)
rodeo binary on PATH (via install-deps --link)
Passwordless sudo for the runner user

Nightly schedule¶

The workflow also runs nightly at 02:00 UTC via a cron trigger. Failed runs create a failed check on the main branch.

Testing a specific Ansible role change¶

When you change an Ansible role (e.g. roles/kvm_host/tasks/firewall.yml), the unit tests do not catch regressions — they do not run Ansible. You must run a live integration test before merging.

Minimum regression for a role change:

rodeo deploy --from kvm_host --force   # re-run the changed role
rodeo status                           # verify the cluster still works

For the PXE chain (pxe_server role), the minimum is a full test profile deploy from scratch — the boot chain is only validated end-to-end when Harvester installs successfully.

Testing definition.yaml changes¶

After editing a definition.yaml, validate it before deploying:

rodeo plan --profile mylab   # must succeed with no errors

Then test with the smallest matching profile:

rodeo clean --all --yes --secrets
rodeo up --profile mylab --yes
rodeo status

If the definition changes node count or IPs, the entire deploy must run from scratch — do not use --from cluster because the VM disk and DHCP lease changes come from the vms phase.

Testing secret resolution¶

# Verify all ?? placeholders resolve
rodeo deploy --check

# Test a specific resolver
RODEO_PASSWORD=testpassword rodeo plan -P credentials.harvester_os_password=??env:RODEO_PASSWORD

deploy --check runs the preflight suite (including secret validation) without starting any phase.