LTEngine¶

Description¶

LTEngine is a machine translation server that speaks the LibreTranslate API and answers it with a local large language model through llama.cpp. Translation quality for many pairs is higher than a lightweight transformer model reaches, at the cost of memory and latency.

Overview¶

This role builds LTEngine from the upstream sources at a pinned commit and runs it as an internal engine without a browser surface. The gateway web-svc-translate consumes it over the shared overlay network; nothing publishes it to the internet.

Architecture¶

ARCHITECTURE.md draws LTEngine from its own metadata: the containers it runs under compose, what changes under swarm, how each volume is backed, and every role it depends on or is depended upon by.

Section	Drawn from
Compose	`meta/services.yml`
Swarm	`meta/services.yml` replicas and placement
Volumes	`meta/volumes.yml` against the NFS decision chain
Dependencies	`meta/services.yml` and `meta/main.yml` of every role

Features¶

LibreTranslate API: POST /translate, POST /detect and GET /languages in the published request and response shapes.
Local inference: the model runs inside the container, no request leaves the deployment.
Weights on a volume: the GGUF file is downloaded once into ltengine_models and reused across restarts.
Built from source: upstream ships no release image, so the role compiles the pinned commit against its own base images.

Quick Setup¶

Development¶

Clone, set up the workstation, and deploy LTEngine onto the local stack:

git clone https://github.com/infinito-nexus/core.git
cd core
make onboard
make compose-deploy mode=reinstall apps=svc-ai-ltengine full_cycle=false

Production¶

Run the published image to provision the inventory and deploy LTEngine to a managed server over SSH. The image logs in as root, so the server must accept root over a public key, and the two key variables are the two halves of one key pair: the private half authenticates the deploy, and the public half is what the deploy keeps authorized for root and installs for the administrator account. Naming a public half that does not belong to the private one locks the running deploy out at its next reconnect.

APP=svc-ai-ltengine
HOST="<your-server>"
DOMAIN="<your-domain>"
TLS_MODE=self_signed
SSH_PUBLIC_KEY="<your-ssh-public-key>"
SSH_PRIVATE_KEY="<your-ssh-private-key-file>"
WORKDIR="$HOME/.infinito.nexus"

mkdir -p "$WORKDIR/inventories"
docker run --rm \
  -v "$WORKDIR/inventories:/etc/infinito.nexus/inventories" \
  -v "$SSH_PRIVATE_KEY:/run/secrets/deploy_key:ro" \
  -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
  ghcr.io/infinito-nexus/core/debian:latest bash -c '
    INVENTORY=/etc/infinito.nexus/inventories/production
    infinito administration inventory provision "$INVENTORY" \
      --inventory-file "$INVENTORY/devices.yml" \
      --host "$HOST" \
      --include "$APP" \
      --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"SYS_SVC_SSHD_PERMIT_ROOT_LOGIN\": \"prohibit-password\", \"users\": {\"root\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}, \"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
    infinito administration deploy dedicated "$INVENTORY/devices.yml" \
      --password-file "$INVENTORY/.password" \
      -u root --private-key /run/secrets/deploy_key \
      --diff -vv'

The inventory under $WORKDIR is written by the container as root, including the vault password at inventories/production/.password. Run sudo chown -R "$USER" "$WORKDIR" afterwards to read it.

Credits¶

Kevin Veen-Birkenbach

Part of the Infinito.Nexus Project and maintained by Kevin Veen-Birkenbach. Licensed under the Infinito.Nexus Community License (Non-Commercial).

That licence covers this role. The software it deploys is a separate grant: AGPL-3.0-only, under which commercial use is allowed. You owe its authors this much: publish the source of changes once you offer it over a network, which a hosted deployment does. Permission for one of the two is not permission for the other.