TODOΒΆ
Add a post-deploy assert that the GPU actually reached the container, once
feature/store-i18nis merged. The role already selects the path by itself:services.s1.gpustates the intent andS1_GPUANDs it withlookup('gpu'), so a hostsys-svc-containerdoes not count as a GPU host builds the CPU image whatever the flag says. What is missing is the check that the device arrived, in the shape ofweb-svc-libretranslate/tasks/utils/assert_gpu.yml: ask the served container how many CUDA devices it counts rather than trusting the two stats. Theruntime: nvidialine itself comes fromsys-svc-container/templates/resource.yml.j2on that branch. Upstream measuresRouter(preload=True)at 32.8 ms on GPU against 193 to 464 ms on CPU.Cap torchβs intra-op threads on the CPU path.
meta/services.ymllimits the container tocpus: "2.0", but torch sizes its thread pool from the hostβs core count, so a 16-core runner spawns 16 threads for 2 cores. laya readsLAYA_THREADS(serve.py,_apply_thread_limit) and upstream says to keep it at or below the physical core count;templates/env.j2does not set it. Costs latency, breaks nothing, so no test catches it.