NLLB-200¶

Description¶

NLLB-200 is a sequence-to-sequence machine translation model covering 202 language-script combinations. It is a translation model rather than a chat model: the decoder is conditioned on a target language token, so it renders into a language instead of being asked to.

Overview¶

This role serves one NLLB-200 checkpoint over HTTP as an internal engine without a browser surface. The gateway web-svc-translate consumes it over the shared overlay network; nothing publishes it to the internet.

The surface speaks FLORES-200 codes (deu_Latn, bel_Cyrl, amh_Ethi), which is what the model itself carries. The ISO 639-1 codes the rest of the repository is keyed by are mapped in the gateway, in translate_gateway/flores.py, together with the languages of meta/languages.yml that NLLB has no code for at all.

Architecture¶

ARCHITECTURE.md draws NLLB-200 from its own metadata: the containers it runs under compose, what changes under swarm, how each volume is backed, and every role it depends on or is depended upon by.

Section	Drawn from
Compose	`meta/services.yml`
Swarm	`meta/services.yml` replicas and placement
Volumes	`meta/volumes.yml` against the NFS decision chain
Dependencies	`meta/services.yml` and `meta/main.yml` of every role

Features¶

Two routes: GET /languages lists the FLORES-200 codes the loaded checkpoint carries, POST /translate takes {"source", "target", "text"} and answers {"translation"}.
Local inference: the model runs inside the container, no request leaves the deployment.
Weights on a volume: the checkpoint is downloaded once into nllb_models and reused across restarts.
CPU or GPU: the build installs the CPU-only torch wheels unless the deploy resolved a usable GPU.

Quick Setup¶

Development¶

Clone, set up the workstation, and deploy NLLB-200 onto the local stack:

git clone https://github.com/infinito-nexus/core.git
cd core
make onboard
make compose-deploy mode=reinstall apps=svc-ai-nllb-engine full_cycle=false

Production¶

Run the published image to provision the inventory and deploy NLLB-200 to a managed server over SSH. The image logs in as root, so the server must accept root over a public key, and the two key variables are the two halves of one key pair: the private half authenticates the deploy, and the public half is what the deploy keeps authorized for root and installs for the administrator account. Naming a public half that does not belong to the private one locks the running deploy out at its next reconnect.

APP=svc-ai-nllb-engine
HOST="<your-server>"
DOMAIN="<your-domain>"
TLS_MODE=self_signed
SSH_PUBLIC_KEY="<your-ssh-public-key>"
SSH_PRIVATE_KEY="<your-ssh-private-key-file>"
WORKDIR="$HOME/.infinito.nexus"

mkdir -p "$WORKDIR/inventories"
docker run --rm \
  -v "$WORKDIR/inventories:/etc/infinito.nexus/inventories" \
  -v "$SSH_PRIVATE_KEY:/run/secrets/deploy_key:ro" \
  -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
  ghcr.io/infinito-nexus/core/debian:latest bash -c '
    INVENTORY=/etc/infinito.nexus/inventories/production
    infinito administration inventory provision "$INVENTORY" \
      --inventory-file "$INVENTORY/devices.yml" \
      --host "$HOST" \
      --include "$APP" \
      --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"SYS_SVC_SSHD_PERMIT_ROOT_LOGIN\": \"prohibit-password\", \"users\": {\"root\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}, \"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
    infinito administration deploy dedicated "$INVENTORY/devices.yml" \
      --password-file "$INVENTORY/.password" \
      -u root --private-key /run/secrets/deploy_key \
      --diff -vv'

The inventory under $WORKDIR is written by the container as root, including the vault password at inventories/production/.password. Run sudo chown -R "$USER" "$WORKDIR" afterwards to read it.

Credits¶

Kevin Veen-Birkenbach

Part of the Infinito.Nexus Project and maintained by Kevin Veen-Birkenbach. Licensed under the Infinito.Nexus Community License (Non-Commercial).

That licence covers this role. The software it deploys is a separate grant: CC-BY-NC-4.0, under which commercial use is prohibited. You owe its authors this much: use privately or for education; a commercial deployment needs a separate grant. Permission for one of the two is not permission for the other.

Checkpoints¶

services.nllb-engine.model selects the checkpoint. The sizes upstream publishes, with the resident memory the server needs for them in float32 and their relative quality:

Checkpoint	Parameters	Weights on disk	Resident	Quality
`facebook/nllb-200-distilled-600M`	600M	~2.5 GB	~3 GB	the floor; distilled from the 54B teacher and still a translation model on every pair
`facebook/nllb-200-distilled-1.3B`	1.3B	~5.5 GB	~6 GB	roughly +2 chrF++ over the 600M on the FLORES-200 average, most of it on low-resource pairs
`facebook/nllb-200-1.3B`	1.3B	~5.5 GB	~6 GB	the dense model the 1.3B was distilled towards; between the two distilled ones
`facebook/nllb-200-3.3B`	3.3B	~17.6 GB	~18 GB	roughly +1 chrF++ over the 1.3B, at three times the memory and latency
`facebook/nllb-moe-54b`	54B	~220 GB	out of reach	the research model, not deployable on a single node here

The role declares its resources for the 600M checkpoint. A larger one needs mem_reservation, mem_limit and min_storage of meta/services.yml raised with it.