utils.roles package

Subpackages

Submodules

utils.roles.categories module

Category-tree SPOT.

meta/categories.yml at the repository root declares the nested category tree every role name is derived from: the dash-joined path of a node is the role’s category prefix (web -> app yields web-app-*), and nodes carry the invokable, stage, modes and run_after policy the dispatcher, the stage lookup and the inventory generator read.

This module owns the file’s location and the tree-walking primitives. Consumers MUST resolve the path through categories_file() rather than spelling it themselves, so a future move touches one line.

FILE_META_CATEGORIES stays repository-root relative and is not pre-joined onto PROJECT_ROOT at import time on purpose: the root is not fixed. utils.roles.validation.invokable is exercised against a patched PROJECT_ROOT and test fixtures build the file inside a temporary tree. A module level absolute constant would freeze the root at import time and silently ignore both.

The file is not optional. It lives beside roles/, which the packaging config excludes from the wheel just as it excludes meta/, so any context that can resolve a role name also carries this file. A missing file is a broken checkout and raises rather than degrading to an empty category tree, which would silently strip every category prefix off every role name.

Keep this module free of Ansible imports and cheap to import: it is pulled in by plugins/filter/invokable_paths.py, which runs in contexts where Ansible is not installed (see tests/unit/python/utils/cache/test_data.py).

utils.roles.categories.categories_file(root: Path | None = None) → Path

Absolute path of the category file below root.

Parameters:

root – tree to resolve against; the repository root when omitted.

utils.roles.categories.flatten_categories(tree, prefix='')

Flattens nested category tree to all possible category paths.

utils.roles.categories.load_categories_tree(categories_file)

Return the roles mapping of the category file.

Parameters:

categories_file – path-like of a meta/categories.yml.

utils.roles.credits module

Attribution recorded in markdown ## Credits sections.

The link that belongs to a credited person lives nowhere but the Credits lines themselves: galaxy_info.author carries the name without a URL, and galaxy_info.company names the project owner rather than the role’s author, so neither can supply it. This module derives the name -> URL map from the linked mentions already present in the repository, which keeps the README generator and the credits-attribution lint reading the same source instead of a hand-maintained list.

Adding a contributor therefore needs no code change: one linked credit anywhere makes the generator emit that link for every role they author.

Patterns exported here:

  • LINKED_RE [Name](https://url) plus the bold markers a Credits

    line wraps it in.

  • BARE_BOLD_RE **Name** that is not itself a link’s text.

  • PERSON_RE a multi-word proper name whose parts carry no dot. The

    dot is what separates people from the products and projects that also appear in Credits lines (“Infinito.Nexus Project”). A person written with an initial (“J. Smith”) therefore falls out of enforcement rather than into a false positive, which is the safe direction for a lint.

utils.roles.credits.author_urls() → dict[str, str]

Name -> URL for every person credited with exactly one URL repo-wide.

Names linked to conflicting URLs are omitted, so an ambiguous mapping never silently picks a winner.

utils.roles.credits.collect_author_urls(documents: list[list[str]]) → dict[str, set[str]]

Map every credited person to the set of URLs they are linked with.

A name with more than one URL is a conflict the caller decides about; this function reports rather than resolves it.

Parameters:

documents – one entry per file, each the file’s lines.

utils.roles.credits.credits_lines(lines: list[str]) → list[tuple[int, str]]

(1-based line number, text) for every line inside a ## Credits section, ending at the next heading of any level.

Parameters:

lines – the file’s lines, without trailing newlines.

utils.roles.credits.is_person(name: str) → bool
utils.roles.credits.iter_markdown_documents() → Iterator[tuple[str, list[str]]]

Yield (path, lines) for every tracked markdown file.

Files the process cannot read are skipped rather than raised: this walk backs the README build, and an unreadable agent scratch file such as .claude/loop.md must not abort generating documentation.

utils.roles.dependency_resolver module

class utils.roles.dependency_resolver.RoleDependencyResolver(roles_dir: str)

Bases: object

get_role_dependencies(role_name: str, *, resolve_include_role: bool = True, resolve_import_role: bool = True, resolve_dependencies: bool = True, resolve_run_after: bool = False) → set[str]
resolve_transitively(start_roles: Iterable[str], *, resolve_include_role: bool = True, resolve_import_role: bool = True, resolve_dependencies: bool = True, resolve_run_after: bool = False, max_depth: int | None = None) → set[str]

utils.roles.deploy module

Deploy-shape helpers shared by the complexity matrix and the Cosmos builder.

Both need the same facts about a role: whether it ships its own container stack, and which deploy modes it supports. Keeping the logic here means the two callers never drift.

utils.roles.deploy.role_deploy_modes(role_dir: Path, role_name: str | None = None) → dict[str, bool]

Return the deploy modes a role supports, keyed by mode with its modes.<mode>.enabled SPOT value.

A stack role (ships a compose template) is offered compose and swarm; a host-only role is offered host. The two are mutually exclusive: swarm needs a stack, host configures the machine instead.

utils.roles.deploy.role_has_stack(role_dir: Path) → bool

True iff the role renders its own container stack, i.e. ships a templates/*compose*.yml.j2. Host-only roles (backup cron, wireguard, swapfile) and pure service-injectors carry no compose template and return False. Presence (not an image key) is the signal so build-from-source stacks that pull no registry image still count as a stack.

utils.roles.display module

Codec between a role id and the display name CI names carry.

A role id is the machine handle (web-app-nextcloud); the display name is what a human reads off a GitHub Actions job or run title (网络应用·Nextcloud). The category path renders as the hanzi labels meta/categories.yml declares, run together into one block, then a ·, then the role’s own README.md heading or the part of the id the categories do not already carry.

The name never holds a space: the whitelist and priority workflow inputs are space-separated lists, so a display name with a space in it would make a two-role list unparseable.

RoleDisplayName.decode() accepts a raw role id unchanged, so every consumer can decode unconditionally and a human dispatching a run by hand never has to type hanzi.

class utils.roles.display.RoleDisplayName(root: Path | None = None)

Bases: object

Encode role ids to display names and back, for one repository tree.

category_labels(app_id: str) → list[str]

The hanzi label of every category the role id walks through.

Parameters:

app_id – role id, e.g. 'web-app-nextcloud'.

decode(text: str) → str | None

The role id text names, or None when it names no role.

Accepts a display name with or without its variant suffix, and a raw role id unchanged – job names from runs that predate the display names still parse.

Parameters:

text – a display name or a role id.

decode_list(names: str) → str

Space-separated display names rendered as space-separated role ids.

The axis suffix a CI selection token may carry (#0,2@swarm+tor) is split off before the lookup and re-attached after it, so translating a list never drops what the operator pinned on it.

Parameters:

names – space-separated display names or role ids; anything that names no role passes through untouched.

encode(app_id: str, variant: str = '') → str

The display name for app_id, with the variant shard appended.

The last fragment is the shorter of the heading and the role id’s own section: a heading that outgrows the id it is meant to read better than has lost its point.

A token with no directory under roles/ encodes to itself: it is a sentinel like __ALL__, or a role the tree does not hold, and neither has a heading or a category to read.

Parameters:

  • app_id – role id, e.g. 'web-app-nextcloud'.

  • variant – variant CSV the job covers, e.g. '0,1' ('': none).

encode_list(app_ids: str) → str

Space-separated role ids rendered as space-separated display names.

Parameters:

app_ids – space-separated role ids; sentinels like __ALL__ and anything that names no role pass through untouched.

id_fragment(app_id: str) → str

The part of the role id the category labels do not already carry.

A role named after its own category (update) leaves no remainder; the whole id stands in, so the fragment is never empty.

Parameters:

app_id – role id, e.g. 'web-app-nextcloud'.

property roles_dir: Path
title(app_id: str) → str

The role’s README.md heading as one separator-joined fragment.

Whitespace inside the heading becomes a · so a display name never holds a space and stays one token in the space-separated whitelist and priority inputs. Variation selectors left behind by an emoji in the heading are dropped.

Parameters:

app_id – role id, e.g. 'web-app-nextcloud'.

utils.roles.display.display_names() → RoleDisplayName

The codec for the repository the process runs in.

utils.roles.display.main(argv: Sequence[str] | None = None) → int

Translate one space-separated list between role ids and display names.

Parameters:

argv – ['<display names>'] to decode, ['--encode', '<role ids>'] to encode.

utils.roles.display.split_axes(token: str) → tuple[str, str]

Split a CI selection token into its name and its axis suffix.

#, @, +, % and / introduce an axis and appear in neither a role id nor a display name, so the first of them is the seam.

Parameters:

token – e.g. 'web-app-nextcloud#0,2@swarm+tor%debian/zfs'.

utils.roles.guide module

Which deploy row replays the role’s README instructions, and in what mode.

The instructions test used to be a job of its own that picked one random role per run. Riding along on a deploy row instead costs no extra runner and covers every role the sweep touches. The row it rides on is the role’s smallest variant – the one enabling the fewest services – because that is the shape the README’s Production block describes: the bare role, with the optional providers switched off.

The replay is a second deploy on the same runner, so the row it picks must be one the guide can actually deploy: the README has to carry a Production block, and the guide’s own mode (compose for a role that ships a stack, host for one installed onto the machine) has to be a mode the role runs and tests.

utils.roles.guide.PRODUCTION_HEADING = '### Production'

The README section scripts/github/guide/compose_deploy.sh replays. A role without it has no instructions to test.

utils.roles.guide.guide_deployable(app: str) → str

The mode the guide would replay app in, or '' when it cannot.

Independent of the variant, and cached: the answer is a property of the role’s files, and utils.github.variant.axes.assign() asks it once per row of every chunk.

utils.roles.guide.guide_variant(app: str, variants_per_app: Mapping[str, Sequence[Mapping[str, Any]]] | None, allowed: Collection[str]) → tuple[str, str]

The variant of app that carries the replay, and the mode it runs in.

('', '') when the role has no instructions to replay or this sweep deploys no variant that could carry them.

Parameters:

  • app – the role.

  • variants_per_app – rendered variant configs per app; without them no variant can be told to be the smallest one.

  • allowed – the variants this sweep deploys, as the matrix writes them.

utils.roles.guide.smallest_variant(variants: Sequence[Mapping[str, Any]], allowed: Collection[str] | None = None) → str

The variant that enables the fewest services, as the matrix writes it; the lowest index wins a tie, and '' means none qualifies.

Parameters:

  • variants – the role’s rendered variant configs, in index order.

  • allowed – the variants this sweep actually deploys. A variant the sweep cuts as redundant coverage never reaches a runner, so hanging the replay on it would silently drop the role from the instructions test; the next-smallest deployed variant carries it instead.

utils.roles.lifecycle module

The CI-tested lifecycle envelope, read from default.env (the SPOT).

utils.roles.lifecycle.tested_lifecycles() → frozenset[str]

Lifecycle stages the CI test-deploy discovery exercises.

Reads the INFINITO_LIFECYCLES value from default.env so the envelope has a single source of truth instead of being duplicated as a hardcoded set at every consumer.

utils.roles.mapping module

Role mapping SPOT.

This module is the single source of truth for role-level dependencies: the paths inside a role directory, the role types each path may legitimately appear under, the mandatory shape of those paths, and the marker entries that drive role-type detection itself. Both lint tests and the role-type predicate live downstream of this file:

  • lint tests import ROLE_FILES to forbid type-scoped artefacts on the wrong role type, to assert that mandatory files are present, and to assert that mandatory dotted-path entries inside those files exist;

  • the role-type predicate lives in utils.roles.type (see utils.roles.type.get_role_type()) and walks the marker flag in this file’s entries to derive a role’s type from the values declared in its own files.

Path constants

Every ROLE_FILE_* constant carries the file’s path relative to the role directory itself (roles/<role-name>/). Callers compose the absolute path by joining with the resolved role directory:

from utils.roles.mapping import ROLE_FILE_META_SERVICES

services_path = (
    PROJECT_ROOT / "roles" / "web-app-matomo" / ROLE_FILE_META_SERVICES
)

Type vocabulary and shape

ROLE_TYPE_* constants name the role categories the project distinguishes. Each ROLE_FILES[<path>]['types'] value is a list of type-scoping entries with the shape:

{
    "type":      <ROLE_TYPE_*>,
    "mandatory": <bool>,                # MUST the file/entry be set
    "allowed":   <bool>,                # MAY the file/entry be set
    "entries": [                        # dotted-path facts about the file
        {
            "path":      "application_id",
            "mandatory": True,
            "allowed":   True,
            "marker":    True,          # the role IS this type when set
        },
        {
            "path":      "ports.local.http",
            "mandatory": False,
            "allowed":   True,
        },
    ],
}

Two complementary flags control the per-type policy:

  • mandatory (default False): the file (or dotted-path entry) MUST be present and non-empty for a role of this type. Implies allowed: True; setting mandatory: True together with allowed: False is a schema error.

  • allowed (default True): the file (or dotted-path entry) MAY be present for a role of this type. Set allowed: False to forbid the file/entry on roles of this type so the lint layer can surface the offence with a precise reason.

Roles whose type set does not match any explicit entry inherit the type’s policy from the wildcard fallback (see below); without a wildcard the file/entry is implicitly forbidden.

entries is a list of dotted-path facts about the file’s content. Each entry carries the same mandatory and allowed semantics described above and adds a marker flag (default False): marker: True declares the path as the type marker for the surrounding type entry. utils.roles.type.get_role_types() walks every marker: True entry; each marker that resolves to a non-empty value in the role’s file adds the surrounding type to the role’s type set. application_id for application and system_service_id for system-service are the canonical markers.

Wildcard ROLE_TYPE_ALL collapses repetition: a single entry with "type": ROLE_TYPE_ALL applies to every concrete role type. A concrete-type entry that appears alongside the wildcard MUST take precedence so a contributor can spell out one per-type exception without losing the shared default.

utils.roles.mapping.role_is_stack(role_dir) → bool

True when the role renders a docker stack.

Parameters:

role_dir – path-like of the role directory (roles/<name>).

A role is a stack role iff it ships templates/compose.yml.j2, any templates/*.compose.yml.j2 sibling, or a templates/compose.*.yml.j2 variant such as compose.override.yml.j2.

utils.roles.meta_lookup module

Helpers for reading per-role metadata that lives on the role’s primary service entity in meta/services.yml.

Both fields used to live nested inside meta/main.yml.galaxy_info:

  • run_after — project-specific role-load-order list.

  • lifecycle — maturity marker filtered by cli meta roles lifecycle.

They live at meta/services.yml.<primary_entity>.{run_after,lifecycle} where <primary_entity> is the value returned by utils.roles.entity.name.get_entity_name() for the role’s directory name.

Both helpers degrade gracefully:

  • [] / None when meta/services.yml is absent OR the field is absent on the primary entity.

  • Raise a clear error only when meta/services.yml exists and parses into a wrong-shape document (non-dict root, non-dict primary entry).

utils.roles.meta_lookup.DEPLOY_MODES: tuple[str, ...] = ('compose', 'swarm')

The stack deploy modes the CI test-deploy matrix (get_role_skip) covers.

utils.roles.meta_lookup.MODES: tuple[str, ...] = ('compose', 'swarm', 'host')

Every deploy mode a role’s primary entity may toggle. compose/swarm target stack roles (own container stack); host targets invokable roles that configure the host instead of shipping a stack.

exception utils.roles.meta_lookup.MetaServicesShapeError

Bases: ValueError

Raised when meta/services.yml is malformed.

utils.roles.meta_lookup.get_role_lifecycle(role: str | Path, *, role_name: str | None = None) → str | None

Return the role’s lifecycle string (or None when absent).

utils.roles.meta_lookup.get_role_mode_enabled(role: str | Path, *, mode: str, role_name: str | None = None) → bool

Return whether the role opts into deploy mode (compose | swarm | host).

The SPOT is meta/services.yml.<primary_entity>.modes.<mode>.enabled. A missing modes block, a missing <mode> entry, or a missing enabled key all mean the role participates in that mode (default True).

utils.roles.meta_lookup.get_role_placement(role: str | Path, *, role_name: str | None = None) → str | None

Return the role’s placement string (or None when absent).

utils.roles.meta_lookup.get_role_provides(role: str | Path, *, role_name: str | None = None) → str | None

Return the service the role’s primary entity provides (or None).

utils.roles.meta_lookup.get_role_run_after(role: str | Path, *, role_name: str | None = None) → list[str]

Return the role’s run_after list (or [] when absent).

role may be a role name (relative to roles/) or an absolute path to a role directory. Pass role_name explicitly when role is an arbitrary path whose basename is not the canonical role name.

utils.roles.meta_lookup.get_role_skip(role: str | Path, *, role_name: str | None = None) → list[str]

Return the deploy modes the role is excluded from in test-deploy discovery (e.g. [compose, swarm]), or [] when it participates in all of them.

Derived from meta/services.yml.<primary_entity>.modes.<mode>.enabled: a mode is skipped iff its enabled flag is false.

utils.roles.meta_lookup.get_role_test_skips(role: str | Path, *, role_name: str | None = None) → list[str]

Return the deploy modes deactivated for test-deploy discovery via meta/tests.yml skip, or [] when absent.

modes in meta/services.yml states where a role RUNS; skip in meta/tests.yml deactivates TESTING a mode without touching that capability.

utils.roles.meta_lookup.iter_roles_with_placement(placement: str, *, roles_dir: str | Path | None = None) → list[str]

Sorted role names whose primary entity declares placement: <placement>.

utils.roles.order module

Resolve the roles of a group into their run_after order.

The play used to consume a task file per group, generated at setup time. The order is derived from data that already lives in the roles, so it is resolved here instead and read through the group_roles lookup.

utils.roles.order.build_dependency_graph(roles_dir: str | Path, prefixes: list[str] | None = None)

Return (graph, in_degree, roles) over the roles matching prefixes.

Only run_after edges pointing inside the selected set become edges, so a group resolves independently of the roles outside it.

utils.roles.order.find_cycle(roles: dict[str, dict[str, Any]]) → list[str] | None

Return a run_after cycle as a role list with the start repeated, or None.

utils.roles.order.find_roles(roles_dir: str | Path, prefixes: list[str] | None = None)

Yield (role_path, meta_file) for every role matching prefixes.

Parameters:

  • roles_dir – directory holding the role folders.

  • prefixes – name prefixes to keep; all roles when empty or None.

utils.roles.order.load_application_id(role_path: str | Path) → str | None

Return the application_id from the role’s vars/main.yml, or None.

utils.roles.order.load_run_after(meta_file: str | Path) → list[str]

Return the run_after list declared by the role owning meta_file.

utils.roles.order.ordered_roles(roles_dir: str, group: str) → tuple[dict[str, str], ...]

Return ({'role': …, 'app': …}, …) for group, in run_after order.

Parameters:

  • roles_dir – directory holding the role folders.

  • group – group name without the trailing dash, e.g. web-app.

Raises:

ValueError – when a role in the group declares no application_id.

utils.roles.order.topological_sort(graph, in_degree, roles=None) → list[str]

Return the role names in run_after order.

Raises:

RuntimeError – when the graph does not resolve, naming the cycle and the roles left unsorted so the failure is actionable.

utils.roles.required_by_coverage module

Static classifiers used by the required_by lint and verifier: role invokability (per categories.yml), required_by presence in meta/services.yml, and # nocheck: <id> opt-outs.

utils.roles.required_by_coverage.role_has_nocheck(role_id: str, check_id: str, roles_dir: str | Path | None = None) → bool

True if roles/<role_id>/meta/services.yml contains # nocheck: <check_id> (whole-word match).

utils.roles.required_by_coverage.role_has_required_by(role_id: str, roles_dir: str | Path | None = None) → bool

True if any entity in roles/<role_id>/meta/services.yml declares non-empty required_by categories/roles, either flat or nested under a compose:/swarm: mode block.

utils.roles.required_by_coverage.role_is_invokable(role_id: str, roles_dir: str | Path | None = None) → bool

True when any node along role_id’s category path has invokable: true.

utils.roles.stage module

Deploy-stage lookup SPOT.

meta/categories.yml declares the ordered stages (constructor -> workstation -> server -> destructor, matching tasks/stages/0*.yml) and a stage per top-level category. This module resolves a role to its stage and exposes the canonical stage order, so both the CLI (cli.meta.roles.order) and the stage_order / role_stage lookups read one source of truth.

utils.roles.stage.role_modes_defaults(role: str) → dict

Mode-enabled defaults a role’s services must declare, from the deepest categories.yml category carrying a modes mapping (parents apply when no deeper category declares one). Empty when no category declares modes.

utils.roles.stage.role_sort_key(role: str) → tuple

Sort key placing a role by (stage, category run_after, name). Used as the run_after tiebreak so ready roles fall out in call order.

utils.roles.stage.role_stage(role: str) → str

The stage a role is invoked in (deepest matching category wins).

utils.roles.stage.stage_groups(stage: str) → list[str]

Ordered role-group names that belong to stage, in intra-stage call order (category run_after, then name). The SPOT the stage plays consume so group membership follows categories.yml. Bootstrap-flagged categories are excluded (they run via explicit steps).

utils.roles.stage.stage_order() → list[str]

Canonical deploy-stage order.

utils.roles.type module

Role-type detection.

Role-type detection consumes the marker: True entries declared in utils.roles.mapping.ROLE_FILES so the schema and the predicate share one source of truth: adding a new type marker on the SPOT side makes the predicate recognise the new type without further code edits.

A role MAY belong to more than one type. svc-opt-ssd-hdd is the canonical example: it declares both application_id and system_service_id and is therefore an application AND a system service. get_role_types() returns the full set so callers that care about scope (e.g. lint tests asserting “this file is allowed for applications”) can use simple set membership.

Algorithm

For every marker entry declared anywhere in utils.roles.mapping.ROLE_FILES the predicate reads the role’s file at the entry’s dotted path. Each marker that resolves to a non-empty string adds the surrounding type to the result.

Roles without any marker fall back to:

  • utils.roles.mapping.ROLE_TYPE_USER for the bare user role and every user-<…> sibling (name-based);

  • utils.roles.mapping.ROLE_TYPE_TOOLING otherwise.

Conformance

A marker value MUST be a non-empty string. If the marker key is present but the value is of the wrong YAML type, empty, or whitespace- only, the role’s typing is broken: get_role_types() raises RoleTypeError so the misconfiguration surfaces at the first call instead of silently degrading to a different type.

Caching

Reads route through utils.cache.yaml.load_yaml_any() (already memoised per-path). get_role_types() caches its return per role-directory string so repeated calls during a single test sweep stay free.

exception utils.roles.type.RoleTypeError

Bases: ValueError

Raised when a role’s typing markers do not conform to the schema.

utils.roles.type.get_role_types(role_dir: Path) → frozenset[str]

Return the set of role types declared by role_dir’s markers.

A role MAY belong to several types simultaneously (for example an application that also ships a systemd unit). The returned frozenset contains every type whose marker matches; the user / tooling fallbacks fire only when no marker resolves.

Cached: repeated calls for the same role_dir reuse the first parse. Pass an absolute path so the cache key is stable across callers that resolve relative paths differently.

utils.roles.type.reset_cache_for_tests() → None

Drop the per-process role-type cache. Tests that mutate vars/main.yml (or another marker-bearing file) mid-run MUST call this between mutations.

Module contents