cli.meta.ci package¶

Submodules¶

cli.meta.ci.chunks module¶

Split the ordered discovery rows into the serial chunks of one sweep.

A sweep is one orchestrator run. It deploys its rows in serial chunk blocks, each small enough that its last runner wave still starts inside the queue window (cli.meta.ci.slots sizes them). Two rules shape the split:

Priority first, never mixed. Priority rows sort to the head of the list and the split forces a chunk boundary at the priority/regular seam, so a chunk is either all priority or all regular. The seam chunk stays short rather than being topped up with regular rows – that is what guarantees every priority row is done before the first regular one starts. Priority is never subject to the sweep offset: it leads every sweep.
The head leads unless an operator says otherwise. Chunk 0 starts at the top of the ranking. The list is ordered coverage-first, so its head is what a run most needs proven, and a sweep that cannot afford the whole list leaves the tail behind rather than sliding a window over it.

offset moves that start deliberately: it is how an operator reaches the tail without waiting for anything. It defaults to 0, so every run that does not ask for it deploys the same head and can be compared against the last one. This used to be derived from the sweep number instead, which bought tail coverage at the price of the head – chunk 0 began wherever the arithmetic pointed, the rows that matter most ran only every few sweeps, and no two runs were comparable.

cli.meta.ci.chunks.plan(priority: Sequence[T], regular: Sequence[T], *, size: int, blocks: int, budget: int, offset: int = 0) → list[list[T]]¶

The chunks one sweep deploys, priority blocks first.

Parameters:

priority – rows of the priority line, in query order.
regular – rows of the regular line, in query order.
size – rows one chunk may hold (slots.chunk_size).
blocks – chunk blocks the workflow declares (slots.chunk_blocks).
budget – rows the run job cap allows in total (slots.available).
offset – how many regular rows to skip before filling the chunks. 0 (the default) starts at the head of the ranking. A value past the end of the list leaves the regular chunks empty rather than wrapping: an operator who asks for a window beyond the tail gets an empty run, not a silent restart at the top.

Returns:

at most blocks chunks, none larger than size, together holding at most budget rows. Priority chunks come first and are never topped up with regular rows.

cli.meta.ci.chunks.slice_chunks(rows: Sequence[T], size: int) → list[list[T]]¶: Cut rows into consecutive blocks of at most size.

cli.meta.ci.dev_runtime module¶

Answer the development-runtime matrix questions from the distro SPOT.

Exists so workflow steps can ask for the matrix, and for the distro behind a matrix entry, without embedding a python one-liner per question.

python3 -m cli.meta.ci.dev_runtime images python3 -m cli.meta.ci.dev_runtime distro –image fedora:latest python3 -m cli.meta.ci.dev_runtime matrix –mode rotate –run 42

cli.meta.ci.dev_runtime.image_label(image: str) → str¶

Shorten a dev-runtime image to its last two registry segments.

quay.io/centos/centos:latest becomes centos:latest; a two-segment image such as debian:13 is returned unchanged.

Parameters:: image – full image reference.
Returns:: The last two segments, joined by the separator that preceded the last.

cli.meta.ci.dev_runtime.main(argv: list[str] | None = None) → int¶

cli.meta.ci.dev_runtime.workspace_matrix(images: list[str], mode: str, run: int) → list[dict[str, str]]¶

Assign exactly one workspace track to every dev-runtime image.

Parameters:

images – dev-runtime images, in SPOT order.
mode – compose or swarm to pin every image to that track, rotate to split the images across both.
run – run number; its parity flips a rotated split, so two consecutive runs cover both tracks on every image.

Returns:

One {"image", "track", "icon", "label"} entry per image.

cli.meta.ci.image_ref module¶

Render a distro image reference from the meta/distros.yml SPOT.

Exists so shell and workflow steps can compose the same reference the env layer composes, instead of spelling ghcr.io/... out a second time.

python3 -m cli.meta.ci.image_ref –kind environment
–distro debian –owner acme –repository infinito-nexus –tag ci-abc

cli.meta.ci.image_ref.main(argv: list[str] | None = None) → int¶

cli.meta.ci.matrix module¶

Build the CI deploy matrix of one sweep, chunk by chunk.

Usage:

python -m cli.meta.ci.matrix –index N [–sweep S] [–modes auto]: [–whitelist “…”] [–priority “…”] [–lifecycles “…”] [–tor auto] [–distros “…”] [–filesystem “…”]

This is the pipeline the deploy jobs discover through, and the single place the run’s shape is decided:

Two discovery queries, keeping today’s filter semantics: the priority line is queried on its own whitelist so priority roles run even when the diff-derived whitelist would not have selected them, and the regular line is queried on the effective whitelist with the priority rows withdrawn. Concatenated, they are the sweep’s ordered candidate list. Both lists are selection tokens (utils.github.variant.selection): what a token pins narrows the row, what it leaves open the line decides as it always did.
Every row is assigned its deploy mode, tor state, distro and filesystem by its position in that list (utils.github.variant.axes).
The list is cut into serial chunks with a hard boundary at the priority/regular seam (cli.meta.ci.chunks), sized by the run’s job and queue budget (cli.meta.ci.slots).

--index then prints one chunk as the matrix JSON. Every chunk block runs the same computation and takes its own slice, so the blocks agree without passing state between them.

cli.meta.ci.matrix.DROPPED = ('priority', 'id', 'covered', 'clone')¶: Entry keys the plan table reads but the matrix JSON withholds, so a deploy job cannot reference one: they describe the row’s rank, not its deployment.

cli.meta.ci.matrix.build_sweep(*, modes: tuple[str, ...], whitelist: str, priority: str, lifecycles: str, sweep: int, tor_mode: str, vpn_mode: str, distros: tuple[str, ...], filesystems: tuple[str, ...], offset: int = 0) → list[list[dict[str, str]]]¶: Every chunk of the sweep, priority blocks first.

cli.meta.ci.matrix.candidates(*, modes: tuple[str, ...], whitelist: str, priority: str, lifecycles: str) → list[dict]¶: The sweep’s ordered candidate rows: priority line first, then the regular line, each annotated with the modes it offers and whether it is priority.

cli.meta.ci.matrix.chunks_of(entries: list[dict[str, str]], offset: int | str | None = 0) → list[list[dict[str, str]]]¶

Cut the sweep’s entries into its chunks, priority blocks first.

Chunk 0 holds the head of the ranking unless offset moves it; the order inside a chunk is then sorted (utils.github.variant.axes.sort_key()), so the job list reads by role rather than by rank.

cli.meta.ci.matrix.entries_of(*, modes: tuple[str, ...], whitelist: str, priority: str, lifecycles: str, sweep: int, tor_mode: str, vpn_mode: str, distros: tuple[str, ...], filesystems: tuple[str, ...]) → list[dict[str, str]]¶: Every candidate row of the sweep, axes assigned, in global order.

cli.meta.ci.matrix.main(argv: list[str] | None = None) → int¶

cli.meta.ci.matrix.offset_index(raw: int | str | None, regular: list[dict[str, str]]) → int¶

Where the regular chunks start reading.

A number is that many rows; anything else is a selection token (utils.github.variant.selection) and the start is the first regular row it names. Resuming at a role beats counting rows by hand: the ranking shifts whenever a variant is added, so the index that meant ‘carry on behind nextcloud’ yesterday means something else today, while the token still says it.

Parameters:

raw – the offset input, a count or a token.
regular – the regular line in ranking order.

Raises:

SystemExit – a token that names no regular row. Starting at 0 instead would silently redeploy the head an operator meant to skip.

cli.meta.ci.matrix.resolve_offset(raw: int | str | None = None) → str¶: The offset a run was given, as written. None reads INFINITO_CI_OFFSET; offset_index() decides what it means.

cli.meta.ci.plan module¶

Render the plan of one CI sweep: every candidate row, and where it lands.

Usage:

python -m cli.meta.ci.plan [–distros “debian”] [–filesystem “zfs”]: [–whitelist “…”] [–priority “…”] [–modes auto] [–lifecycles “…”] [–sweep N] [–cli]

The plan runs the same pipeline the deploy jobs discover through (cli.meta.ci.matrix), so what the table shows and what CI deploys cannot diverge. One row per role#variant candidate in global query order, with the chunk it falls into and the axes it was assigned.

Status per row: ⭐ a priority row, ✅ deployed by this sweep, ❌ beyond the sweep’s budget (raise --offset to reach it), ✂️ cut as redundant coverage — a clone or a row an earlier one already embeds, which no sweep deploys. --cli renders fixed-width terminal tables instead of Markdown.

📖 marks the row that replays the role’s README instructions after its deploy, and names the mode the replay runs in – which is the guide’s own, not the row’s (utils.roles.guide).

🆔 numbers the rows of this table, 1..n, and 🔰 names the 🆔 of the earlier row that already embeds this one – empty when nothing does. A row with a 🔰 is redundant coverage: whatever it would prove, the row it points at proves first. 🐑 marks the other reason a row is cut: it shares its service set with an earlier role, which stands in for it.

cli.meta.ci.plan.cells(entries: list[dict[str, str]], plan: list[list[dict[str, str]]]) → list[tuple[str, ...]]¶

cli.meta.ci.plan.main(argv: list[str] | None = None) → int¶

cli.meta.ci.plan.render_cli(title: str, rows: list[tuple[str, ...]]) → str¶

cli.meta.ci.plan.render_markdown(title: str, rows: list[tuple[str, ...]]) → str¶

cli.meta.ci.query module¶

Single source of truth for the CI app-discovery query.

Usage:

python -m cli.meta.ci.query [–modes “swarm compose host”] [–matrix]: [–format json]

One query per run, not one per deploy mode. Every row is a role#variant selection, so the row basis, the ranking and the budget cut all speak the same unit and the cut can select a subset of a role’s variants. Which deploy mode a row actually runs in is decided afterwards by the rotation in utils.github.variant.bundles; the query only decides which rows exist.

The mode selection is a filter, not an axis: --modes swarm keeps the rows whose role can be deployed as a swarm stack, --modes "compose swarm" keeps the rows at least one of the two can take. The shared filter is mode-clause + INFINITO_WHITELIST + INFINITO_BLACKLIST, the order is INFINITO_DISCOVERY_SORT, whose first key sorts clones last (one representative per dna cluster stays ahead of the budget cut, so redundant same-service-set variants fall behind it first), and the lifecycle envelope is INFINITO_LIFECYCLES.

--matrix renders the full ordered candidate list, so the matrix order IS the selection priority. Every human-facing view of that list goes through here rather than re-deriving the sort and the filter: hand-rolling --sort "$INFINITO_DISCOVERY_SORT" --filter "compose == true" reads the wrong column (compose is what a role can do, test_compose is what CI tests), which silently shows an order no run will ever take.

cli.meta.ci.query.build_filter(modes: tuple[str, ...], whitelist: str = '', blacklist: str = '') → str¶: The complexity --filter expression for one run.

cli.meta.ci.query.discover(modes: tuple[str, ...] = ('compose', 'swarm', 'host'), *, whitelist: str = '', blacklist: str = '', lifecycles: str = '') → list[str]¶: The ordered role#variant tokens the query yields.

cli.meta.ci.query.discover_rows(modes: tuple[str, ...] = ('compose', 'swarm', 'host'), *, whitelist: str = '', blacklist: str = '', lifecycles: str = '') → list[dict]¶: The ordered candidate rows, each the complexity payload of one role#variant. Uncapped: the chunker decides what a sweep spends.

cli.meta.ci.query.main(argv: list[str] | None = None) → int¶

cli.meta.ci.query.resolve_modes(raw: str) → tuple[str, ...]¶

The deploy modes a run may draw from. auto (or empty) means every mode; anything else is the listed subset, in MODES order.

Raises:: SystemExit – a token that names no known mode – a typo must not silently narrow the run to nothing.

cli.meta.ci.query.row_modes(row: dict, modes: tuple[str, ...] = ('compose', 'swarm', 'host')) → tuple[str, ...]¶: The selected deploy modes this row can actually run in. Never empty for a row the query returned – the filter kept it precisely because at least one selected mode claims it.

cli.meta.ci.query.sort_spec() → str¶: The discovery sort as declared in default.env.

cli.meta.ci.query.token(row: dict) → str¶: The role#variant selection token of one query row.

cli.meta.ci.report_failures module¶

Open (or comment on) a ‘CI failure: <role>’ issue per role whose deploy failed, linking the run artifacts and inlining the decisive rescue excerpt.

class cli.meta.ci.report_failures.Failure(mode: str, variant: str, tor: bool, distro: str, filesystem: str, vpn: bool = False)¶

Bases: NamedTuple

The combination one red deploy job died on – everything the artifact name is built from, so the reporter downloads the run it is reporting.

distro: str¶: Alias for field number 3

filesystem: str¶: Alias for field number 4

mode: str¶: Alias for field number 0

tor: bool¶: Alias for field number 2

variant: str¶: Alias for field number 1

vpn: bool¶: Alias for field number 5

cli.meta.ci.report_failures.artifact_name(role: str, failure: Failure) → str¶

The rescue-diagnostics artifact one deploy job uploads.

The slug comes from axes.artifact_slug(), the same call the matrix entry carries into the workflow, so the name this looks for and the name CI uploads cannot drift apart.

cli.meta.ci.report_failures.decisive_excerpt(rescue_dir: Path, *, max_lines: int = 40) → str¶: First error-context / meta / containers file in rescue_dir, truncated.

cli.meta.ci.report_failures.failed_roles(jobs: list[dict]) → dict[str, list[Failure]]¶

Map role -> [Failure] for every failed deploy job.

A deploy job is titled <mode glyph><tor glyph><distro glyph><filesystem glyph><display name> <variant> with an optional trailing ⭐ for a priority row. The middle is resolved through the display-name codec rather than matched as a raw role id: job names carry display names, so a regex over web-app-… silently matched nothing and every failure went unreported.

cli.meta.ci.report_failures.issue_body(role: str, failures: list[Failure], *, run_url: str, excerpt: str) → str¶

cli.meta.ci.report_failures.list_jobs(run_id: str, repo: str) → list[dict]¶

cli.meta.ci.report_failures.main(argv: list[str] | None = None) → int¶

cli.meta.ci.report_failures.report(run_id: str, repo: str) → int¶

cli.meta.ci.slots module¶

Derive the deploy-chunk budget of one CI workflow run.

Usage:: python -m cli.meta.ci.slots [–matrix] [–format json] [–field <name>]

A chunk is one serial block of deploy jobs in the orchestrator. Two independent ceilings bound it, and the smaller one wins:

Queue clock. GitHub cancels a job that sat queued for INFINITO_CI_QUEUE_HOURS. With INFINITO_CI_CONCURRENCY runners free and every job assumed to burn its full timeout-minutes, a chunk drains in waves, and the last wave that still starts inside the queue window is floor(queue_hours / job_timeout). A chunk therefore holds at most concurrency * waves jobs. Assuming every job runs to its timeout is deliberately pessimistic – no job can outlast it – so the drain estimate can only overshoot, never undershoot.
Run job cap. GitHub caps a workflow run at INFINITO_CI_JOB_LIMIT matrix jobs, counted across the whole run. Every non-deploy job the orchestrator chain spawns eats into that budget, so this module statically and CONSERVATIVELY counts them:
- every orchestrator job reserves its full job count – reusable workflows are opened and summed, static matrices multiplied out, dynamic (fromJson) matrices estimated at _DYNAMIC_MATRIX_ESTIMATE;
- the chunk blocks themselves reserve only their static jobs (the discover step); their dynamic per-row matrices are exactly the slots being budgeted.
The worst entry point (entry-*.yml) adds its own jobs to the same run before calling the orchestrator; that overhead is subtracted too.

What is left is available(), the rows one sweep can deploy. It is spent by chunk_count() blocks of chunk_size() rows. Because the orchestrator cannot generate a variable number of jobs, the block count is also capped by INFINITO_CI_MAX_CHUNKS, the number of chunk blocks the workflow YAML actually declares (chunk_blocks()); a sweep that would need more rows than those blocks hold leaves the rest to the next sweep’s offset.

cli.meta.ci.slots.available(repo_root: Path | None = None) → int¶: Deploy rows one run may spend across all its chunks.

cli.meta.ci.slots.budget(repo_root: Path | None = None) → dict[str, int]¶

cli.meta.ci.slots.chunk_blocks() → int¶: Chunk blocks call-orchestrator.yml declares, the most a sweep may plan.

cli.meta.ci.slots.chunk_count(repo_root: Path | None = None) → int¶: Chunk blocks a sweep fills, capped by the blocks the YAML declares.

cli.meta.ci.slots.chunk_size(repo_root: Path | None = None) → int¶

Rows one chunk may hold before its tail risks the queue cancel.

INFINITO_CI_CHUNK_SIZE takes precedence when set; 0 derives the value from the runner slots a chunk may assume and the waves the queue window affords.

cli.meta.ci.slots.entry_overhead(repo_root: Path | None = None) → int¶: Worst-case jobs an entry-*.yml adds to the run around its orchestrator call (the orchestrator’s own jobs are already counted).

cli.meta.ci.slots.job_timeout_minutes(repo_root: Path | None = None) → int¶: The deploy job’s own timeout-minutes, read straight out of the workflow so the wave arithmetic cannot drift from the value CI enforces.

cli.meta.ci.slots.main(argv: list[str] | None = None) → int¶

cli.meta.ci.slots.render_matrix() → str¶: The budget as a CLI table: per-job reservations, then the totals and the chunk arithmetic they leave room for.

cli.meta.ci.slots.reserved_breakdown(repo_root: Path | None = None) → list[tuple[str, int]]¶: Per orchestrator job: the job count it reserves in the run budget. Chunk blocks contribute only their static jobs (the discover step); their dynamic per-row matrices are the budgeted slots themselves.

cli.meta.ci.slots.reserved_slots(repo_root: Path | None = None) → int¶

cli.meta.ci.slots.rows_per_sweep(repo_root: Path | None = None) → int¶: Rows one sweep covers: the chunk blocks it fills, bounded by the run job cap. Rows beyond this roll into the next sweep via its offset.

cli.meta.ci.slots.setting(key: str) → int¶: One integer CI setting, read through the shared env SPOT.

cli.meta.ci.slots.waves(repo_root: Path | None = None) → int¶: Runner waves that still start inside the queue window. Floored at 1: a job timeout at or above the window means one wave, never zero.

cli.meta.ci.subset_roles module¶

Resolve the explicit role subset declared in a pull-request body.

Activated by the 🧩 Subset label: the entry-pull-request workflow only runs this module when the label is present. It gives fork contributors the selective-CI capability that entry-manual-steer.yml (workflow_dispatch) grants maintainers, which forks cannot trigger.

It reads the PR body from $PR_BODY, extracts the first fenced code block whose YAML defines a roles: list, validates that every listed id is an existing roles/<id> directory, and writes whitelist (space-separated) to $GITHUB_OUTPUT – the same output the diff-derived resolver produces.

Unlike the fail-safe diff resolver (which widens to __ALL__ on any problem), the subset path is strict: invalid YAML, a missing/empty roles: list, or an unknown role id each abort with a clear message and a non-zero exit, so a mistyped subset never silently deploys the wrong set. Role ids must match ^[a-z0-9-]+$, which also blocks path traversal before any filesystem lookup.

Invoked from scripts/github/resolve/pr_affected_roles.sh via python -m cli.meta.ci.subset_roles.

cli.meta.ci.subset_roles.extract_roles(body: str)¶

cli.meta.ci.subset_roles.fail(message: str) → NoReturn¶

cli.meta.ci.subset_roles.main() → None¶

cli.meta.ci.validate module¶

Check a run’s selection inputs against the branch before anything deploys.

Usage:

python -m cli.meta.ci.validate [–whitelist “…”] [–priority “…”]: [–modes auto] [–tor auto] [–distros “…”] [–filesystem “…”] [–lifecycles “…”]

A selection token names a row that has to exist on this branch (utils.github.variant.selection). Nothing else in the chain can tell the operator that early: the matrix builder raises on the first bad token, one per chunk job, so a stale list costs one full CI round per entry and reports the second problem only after the first is fixed. Every problem is collected here instead, and the exit code gates the run.

What makes a token bad, in the order this checks it:

it does not parse, or names a mode or onion state that does not exist;
it pins a variant the role does not declare (any more): the usual case is a list carried over from an older run whose variants have since been renumbered;
it pins a mode the row cannot take, an onion state the row, the mode or the run’s own tor axis rules out, or a distro or filesystem the run’s own pool does not hold.

A token naming a role the discovery query does not return at all is reported too, but as a warning, whether or not it pins axes: the diff-derived whitelist legitimately names roles the envelope filters out, and it pins the variants a change reaches. Role names themselves are checked against roles/ before the run starts (scripts/github/resolve/effective_whitelist.sh), so a typo is caught there rather than swallowed here.

cli.meta.ci.validate.main(argv: list[str] | None = None) → int¶

cli.meta.ci.validate.problems(tokens: str, *, modes: tuple[str, ...], tor_mode: str, distros: tuple[str, ...], filesystems: tuple[str, ...], lifecycles: str, label: str) → tuple[list[str], list[str]]¶

Every reason the tokens of one input cannot deploy on this branch.

A token that narrows nothing beyond the role name is answered by existence: it asks for the role in whatever axes the rotation picks, so any discovered row satisfies it. Rows are keyed by (name, variant) and a variant is an int, so looking such a token up variant-first can only miss, which reported every collapsed priority entry as matching nothing and left a role that really had no row indistinguishable from one with a full set.

Parameters:

tokens – the raw whitelist/priority value.
modes – the run’s selected deploy modes.
tor_mode – the run’s tor axis.
distros – the distro pool the run draws from.
filesystems – the filesystem pool the run draws from.
lifecycles – the run’s lifecycle envelope.
label – the input’s name, for the messages.

Returns:

(errors, warnings).

cli.meta.ci package¶

Subpackages¶

Submodules¶

cli.meta.ci.chunks module¶

cli.meta.ci.dev_runtime module¶

cli.meta.ci.image_ref module¶

cli.meta.ci.matrix module¶

cli.meta.ci.plan module¶

cli.meta.ci.query module¶

cli.meta.ci.report_failures module¶

cli.meta.ci.slots module¶

cli.meta.ci.subset_roles module¶

cli.meta.ci.validate module¶

Module contents¶