TODOΒΆ

  • Add a post-deploy assert that the GPU actually reached the container, once feature/store-i18n is merged. The role already selects the path by itself: services.s1.gpu states the intent and S1_GPU ANDs it with lookup('gpu'), so a host sys-svc-container does not count as a GPU host builds the CPU image whatever the flag says. What is missing is the check that the device arrived, in the shape of web-svc-libretranslate/tasks/utils/assert_gpu.yml: ask the served container how many CUDA devices it counts rather than trusting the two stats. The runtime: nvidia line itself comes from sys-svc-container/templates/resource.yml.j2 on that branch. Upstream measures Router(preload=True) at 32.8 ms on GPU against 193 to 464 ms on CPU.

  • Cap torch’s intra-op threads on the CPU path. meta/services.yml limits the container to cpus: "2.0", but torch sizes its thread pool from the host’s core count, so a 16-core runner spawns 16 threads for 2 cores. laya reads LAYA_THREADS (serve.py, _apply_thread_limit) and upstream says to keep it at or below the physical core count; templates/env.j2 does not set it. Costs latency, breaks nothing, so no test catches it.