Anyone who has moved a model off a laptop and onto a GPU box knows the failure. The script runs fine locally, then dies on the server with an import error, because the machine carries a slightly different PyTorch build. Patch that version and a second project breaks, because it wanted the old one. Setting up a Docker GPU server the right way ends this, since a container seals your model, its libraries, and the exact runtime into one image that behaves the same on any host.
That is the whole idea. PyTorch 2.1, CUDA 12.1, Python 3.10, pinned together and shipped as a single unit. Move it to any Linux machine with a GPU, and it runs the way it ran on yours, not almost the same way.
The GPU is where people get stuck. A fresh container cannot see the card. NVIDIA’s Container Toolkit is what changes that, and once it is installed, AI containers’ GPU access is a single flag at runtime. Your framework finds the hardware with nothing else to wire up.
This walkthrough starts on a bare Ubuntu GPU server and finishes with a container running your model on the GPU. You will install Docker, NVIDIA GPU support, confirm the card is visible from inside a container, and run both PyTorch and TensorFlow.
Also Read: How to Run Kubernetes on Cloud GPU – Complete Beginner Guide 2026
Why Docker for AI on GPU servers
Four things make Docker worth the setup for AI work, and the fourth is the one that actually matters on a GPU box.
Start with reproducibility, since it is why most teams reach for containers at all. A model that trains cleanly on your laptop can fall over on a server running an older framework build, and chasing that mismatch burns an afternoon you will not get back. The container carries its own versions, so the environment stops being a moving part.
Isolation comes next. One project needs TensorFlow 2.12, another needs 2.14, and on a shared host those two quietly overwrite each other in the same Python environment. Give each its own container and the conflict is gone. Nothing leaks across.
Scaling falls out of the first two. Once a job runs in a container, running it on ten servers is the same image ten times, with no per-machine drift to debug. What passed in testing is what runs in production, down to the byte.
The fourth reason is the hard one, and it is specific to GPUs: getting the card into the container. NVIDIA’s answer, shipped in the early days as docker nvidia-docker tooling and now folded into the NVIDIA Container Toolkit, makes the card visible from inside. Without it, your framework reports no CUDA device and quietly drops to the CPU while you wonder why training crawls. With it, the GPU just appears.
One thing this guide leaves out on purpose is writing your own Dockerfile. Here you run images that already exist, the official PyTorch and TensorFlow builds. Rolling your own is worth doing later. It is a different job.
Also Read: How to Host Server in China Without an ICP License in 2026 – Practical Guide for Foreign Businesses
Prerequisites and verification
Five checks before you install anything. Each takes seconds, and each heads off a specific, confusing failure down the line.
The host driver comes first. A container ships CUDA but never the driver, so the driver has to already be there:
| nvidia-smi |
A table of your GPU and its driver version means you are set. command not found means the driver is missing, and that is the thing to fix before anything else.
The CUDA Toolkit on the host is optional. Containers bring their own, so you only need this for compiling or debugging outside a container:
| nvcc –version |
If it fails, shrug and move on. Nothing here depends on it.
GPU containers want Linux. Ubuntu 20.04, 22.04, and 24.04 all work well; Windows and macOS run Docker but handle GPU passthrough poorly.
| lsb_release -a |
Keep an eye on disk. AI images are heavy, roughly 5 to 8 GB for PyTorch and 4 to 6 GB for TensorFlow, so leave at least 30 GB free:
| df -h / | grep -v Filesystem |
Last, the server needs the internet for the first pull. A large image can take 5 to 10 minutes to download; after that it is cached and starts instantly. Clear all five and Docker is next.
Also Read: Llama 405B vs DeepSeek: Which Cloud GPU Actually Handles Both in 2026?
Step 1: Install Docker
Docker installs from its own apt repository. The steps below use Docker’s current signed-keyring method, not the apt-key approach that newer Ubuntu releases now warn about.
Clear out any distro-packaged Docker first. On a clean server this does nothing, which is fine:
| for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove -y $pkgdone |
Now add Docker’s repository and install the engine:
| sudo apt-get updatesudo apt-get install -y ca-certificates curlsudo install -m 0755 -d /etc/apt/keyringssudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.ascsudo chmod a+r /etc/apt/keyrings/docker.ascecho \ “deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo “$VERSION_CODENAME”) stable” | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/nullsudo apt-get updatesudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin |
Most of that block is plumbing: pull in dependencies, register Docker’s GPG key so apt trusts the packages, point apt at Docker’s repository for your Ubuntu release, then install the engine along with the CLI, containerd, and the Buildx and Compose plugins. The only line worth memorizing is the last one.
Check the version:
| docker –version |
Something like Docker version 27.3.1 confirms it landed. Set the daemon to run now and on every boot:
| sudo systemctl enable –now docker |
If you would rather not type sudo before every command, add your user to the docker group:
| sudo usermod -aG docker $USERnewgrp docker |
Worth knowing: group membership is close to root access, so security-minded teams skip it and keep using sudo docker. Neither choice is wrong. It is a trade between typing less and shrinking the blast radius.
Finish with the smoke test everyone runs:
| docker run hello-world |
A “Hello from Docker!” line means the engine works. The card is next.
Also Read: Docker or Bare Metal on Cloud GPU? How to Choose the Right One in 2026
Step 2: Install the NVIDIA Container Toolkit
By default, a container is blind to the GPU. Your code asks for a CUDA device, the container has nothing to hand back, and the request fails. The NVIDIA Container Toolkit is the piece that gives the container a window onto the host card. It is the modern replacement for the old nvidia-docker wrappers you still see in stale tutorials.
Add NVIDIA’s repository and install the toolkit. This matches NVIDIA’s current guide:
| curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \ sudo gpg –dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpgcurl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \sed ‘s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g’ | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.listsudo apt-get updatesudo apt-get install -y nvidia-container-toolkit |
Then hook it into Docker and restart the daemon so the change takes effect:
| sudo nvidia-ctk runtime configure –runtime=dockersudo systemctl restart docker |
The nvidia-ctk command edits /etc/docker/daemon.json to register the NVIDIA runtime. Skip the restart, and Docker keeps running on its old config, which is a common reason the next test fails for people who did everything else right.
Now prove it. This launches a Docker CUDA container from NVIDIA’s official CUDA image and runs the GPU report from inside it:
| docker run –rm –gpus all nvidia/cuda:12.1.1-runtime-ubuntu22.04 nvidia-smi |
The –gpus all part is what hands every card to the container. If the same GPU table you saw on the host now prints from inside the container, the toolkit works and you can run GPU container jobs for real.
The error you are most likely to meet here is could not select device driver “” with capabilities: [[gpu]]. Nine times out of ten it means the daemon was never restarted after nvidia-ctk. Run the restart again and retry. If it sticks around, confirm the package is even installed with nvidia-ctk –version.
Also Read: Cloud GPU for AI Inference vs Training: Different Needs Explained
Step 3: Pull and run a PyTorch container
An image is the blueprint; a container is a running copy of it. For AI work, that blueprint is bulky, because a GPU Docker image stacks a full CUDA and cuDNN runtime on top of the framework and the base OS layers.
Pull the official PyTorch Docker container:
| docker pull pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime |
The first pull runs a few minutes on a normal connection. After that it is cached and starts at once. The tag spells out what is inside: 2.1.0 is the PyTorch version, cuda12.1 and cudnn8 are the GPU libraries it was compiled against, and runtime is the slim build. There is a devel tag too, far larger, that adds compilers and headers for building extensions. To run models, runtime is the one you want.
Confirm it downloaded:
| docker images | grep pytorch |
Roughly 3 to 5 GB should show up. Open an interactive shell with the GPU attached:
| docker run –rm –gpus all -it pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime bash |
Two flags carry the weight here. –rm throws the container away on exit so nothing piles up, and –gpus all is what actually grants the card. The -it pair just gives you a terminal you can type into.
From inside, ask PyTorch what it can see:
| python3 – <<‘PYEOF’import torchprint(f”GPU available: {torch.cuda.is_available()}”)print(f”GPU device: {torch.cuda.get_device_name(0)}”)print(f”CUDA version: {torch.version.cuda}”)PYEOF |
You want this:
| GPU available: True GPU device: NVIDIA H100 80GB HBM3 CUDA version: 12.1 |
A False on the first line means the passthrough did not happen. Run nvidia-smi inside the container to check whether the card is visible at all, then go back to Step 2.
Type exit to leave. For real work you never copy data into a container. You mount it from the host:
| docker run –rm –gpus all -v /path/to/your/data:/data -it \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime bash |
The -v flag maps a host folder onto a path inside the container. Your data and weights stay on the host at /data, the container reads and writes them there, and then the container disappears. Treat it as disposable and keep everything you care about outside it.
Step 4: Run your own AI model in a container
Interactive shells are for poking around. Real jobs run a script, unattended, so the pattern shifts: drop the script on the host, mount it, and let the container run it.
Save this as train.py. It is deliberately trivial, a stand-in that proves the GPU is doing the work:
| import torch import torch.nn as nn print(f”GPU available: {torch.cuda.is_available()}”) device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”) print(f”Using device: {device}”) model = nn.Sequential( nn.Linear(10, 64), nn.ReLU(), nn.Linear(64, 1), ).to(device) X = torch.randn(100, 10).to(device) y = torch.randn(100, 1).to(device) optimizer = torch.optim.Adam(model.parameters()) criterion = nn.MSELoss() for epoch in range(10): optimizer.zero_grad() output = model(X) loss = criterion(output, y) loss.backward() optimizer.step() if (epoch + 1) % 2 == 0: print(f”Epoch {epoch + 1}: Loss {loss.item():.4f}”) print(“Training complete!”) |
Swap in your real model and data and the run command does not change. That is the payoff of this whole workflow: the container ai models you build run on the GPU with zero host-side installation, here or on the next server you spin up.
Run it:
| docker run –rm –gpus all -v /path/to/your/scripts:/workspace \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python /workspace/train.py |
Mount the scripts folder, point Python at the file, and it runs. The output ends with “Training complete!”.
While it runs, watch the card from a second terminal on the host:
| watch -n 1 nvidia-smi |
Usage climbing and VRAM filling is your proof the container is on the GPU and not quietly grinding away on the CPU.
Two small jobs can share one card. Point both at the same device and let the driver interleave them:
| docker run –rm –gpus ‘”device=0″‘ -v /data:/data \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python /workspace/train_model_1.py & docker run –rm –gpus ‘”device=0″‘ -v /data:/data \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python /workspace/train_model_2.py & |
Fine for inference and light experiments. A serious training run should own its card, though, because two memory-hungry jobs on one GPU will collide in VRAM and one of them will crash. Picking the right card for the job is its own decision, and the breaks down which workloads suit which.
Also Read: Cloud vs. Dedicated Servers: The Decision Framework Every CTO Should Know
Step 5: Docker Compose for multi-container setups
A single docker run holds up fine until you have two or three containers that need to start together, share volumes, and come up in the right order. Docker Compose puts all of that in one file.
The GPU block below is the part to read closely. Compose v2 reserves cards through a deploy section, which replaced the old runtime: nvidia shortcut, and the capabilities: [gpu] line is mandatory. Leave it out and the service refuses to start.
| services: ai_training: image: pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime container_name: ai_train_job deploy: resources: reservations: devices: – driver: nvidia count: all capabilities: [gpu] environment: – PYTHONUNBUFFERED=1 volumes: – /path/to/data:/data – /path/to/models:/models – /path/to/scripts:/workspace working_dir: /workspace command: python train.py restart: unless-stopped inference_server: image: pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime container_name: ai_inference deploy: resources: reservations: devices: – driver: nvidia count: all capabilities: [gpu] environment: – PORT=8000 volumes: – /path/to/models:/models – /path/to/inference_code:/app working_dir: /app ports: – “8000:8000” command: python serve.py depends_on: – ai_training restart: unless-stopped |
Top to bottom, that defines two containers: a training job, and an inference service that waits for it through depends_on, exposes port 8000, and restarts itself if it dies. Older Compose files opened with a version: key. Drop it. Compose v2 ignores it.
Bring the whole thing up in the background:
| docker compose up -d |
Mind the space. docker compose is the plugin you installed in Step 1. docker-compose with a hyphen is the retired standalone tool, and confusing the two is a frequent source of “command not found.”
Watch what is happening:
| docker compose ps docker compose logs -f ai_training |
Tear it all down with docker compose down. Anything written to /path/to/models survives, because it lives on the host and not in the containers. For heavier multi-service stacks, the goes further.
Also Read: LLM Training in 2026: What Nobody Tells You About Infrastructure Costs
Getting containers production-ready
A container that runs once on your desk is not the same thing as one you trust to run overnight, restart cleanly, and not drag the whole server down with it. Four areas close that gap.
GPU memory
A myth worth killing first: –gpus cannot cap VRAM. The flag chooses which cards a container sees, and that is all it does. There is no memory= option on it, whatever older guides show. To stop one container from eating a whole card, you have three real levers. Pin it to a single device:
| docker run –rm –gpus ‘”device=0″‘ \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python train.py |
Cap the fraction from inside the framework:
| import torchtorch.cuda.set_per_process_memory_fraction(0.5, device=0) |
Or, on A100 and H100 cards, use MIG to split the GPU into hardware-isolated slices and give each container its own. MIG is the only one of the three that is a hard wall. The other two are cooperative and depend on the jobs’ behaviour.
Storage and limits
Trained weights should outlive the container that produced them. A named volume takes care of that:
| docker volume create ai_models docker run –rm –gpus all -v ai_models:/models \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python train.py docker volume ls |
Cap CPU and host memory so a runaway process cannot take the box down with it:
| docker run –rm –gpus all –cpus 4 –memory 32g \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python train.py |
And when something dies at 3 a.m., you want it back without you. In Compose:
| deploy: restart_policy: condition: on-failure max_attempts: 3 delay: 10s |
Security and isolation
The defaults lean permissive, so tighten them. Run as a normal user instead of root, which keeps a container breakout from landing on the host with root in hand:
| docker run –user 1000:1000 –rm –gpus all \ pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime python train.py |
For offline training on sensitive data, cut the network entirely with –network none. The container computes, writes to its volume, and talks to nothing else. There is more to lock down at the host level, and it covers it.
Housekeeping
Check GPU use in a live container without opening a shell:
| docker exec <container_id> nvidia-smi |
Read a stopped container’s output when you are hunting a crash:
| docker logs –tail 100 <container_id> |
And reclaim disk when images stack up. Read the confirmation before you agree, because -a removes every unused image, not just dangling layers:
| docker system prune -a |
Also Read: AI Video Generation 2026: Best GPUs, VRAM Guide, and Smart Setups That Work
Troubleshooting common errors
Most GPU and Docker trouble traces back to a short list of causes. Here are the ones you will actually run into.
docker: command not found means the install did not take, or the shell cannot find the binary. Confirm, then reinstall if it is genuinely missing:
| which dockersudo apt-get install -y docker-ce |
permission denied while trying to connect to the Docker daemon is the docker-group issue from Step 1. Add yourself and reload the group, or fall back to sudo:
| sudo usermod -aG docker $USERnewgrp docker |
could not select device driver “” with capabilities: [[gpu]] is the classic GPU failure. Either the toolkit is missing, or the daemon was never restarted after you configured it:
| nvidia-ctk –version sudo nvidia-ctk runtime configure –runtime=docker sudo systemctl restart docker docker run –rm –gpus all nvidia/cuda:12.1.1-runtime-ubuntu22.04 nvidia-smi |
no space left on device turns up mid-pull, when images have filled the disk. See what Docker is holding, then clear it out:
| df -h docker system df docker image prune -a docker container prune |
torch.cuda.is_available() returning False has two usual causes: the –gpus flag got left off, or the host driver is older than the CUDA version baked into the image. Check both, working from the outside in:
| nvidia-smi docker run –rm –gpus all nvidia/cuda:12.1.1-runtime-ubuntu22.04 nvidia-smi |
If the host nvidia-smi works but the container’s does not, the toolkit is your suspect. If the host driver is simply too old for the image, update it.
CUDA out of memory is a model-too-big problem, not a Docker one. Watch the VRAM as it climbs:
| docker exec <container_id> nvidia-smi |
Then shrink the footprint: smaller batch size, gradient checkpointing, or 8-bit or 4-bit quantization. Since –gpus will not cap memory for you, control it in code or give the job a card of its own.
A container that exits the instant it starts almost always ran a command that failed or pointed at the wrong directory. Read the logs first, then drop into a shell and look around:
| docker logs <container_id> docker run –rm –gpus all -it pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime bash |
When a GPU container will not cooperate, three suspects cover most cases: a missing NVIDIA toolkit, a driver that lags the image’s CUDA version, and a forgotten –gpus flag. Start there before you look anywhere else. If the host itself seems off, the team can check it from their side.
Also Read: What Is DDoS Protection on a Dedicated Server — And Why Your Business Needs It in 2026?
Running TensorFlow containers on the GPU
PyTorch got the long version. TensorFlow is the same shape, so this part is short. Everything you already set up, the toolkit and the –gpus flag, carries straight into TensorFlow Docker GPU work.
Pull the official image:
| docker pull tensorflow/tensorflow:latest-gpu |
Run it and check what it sees:
| docker run –rm –gpus all -it tensorflow/tensorflow:latest-gpu \ python3 -c “import tensorflow as tf; print(‘GPUs:’, tf.config.list_physical_devices(‘GPU’))” |
A non-empty list means the card is available. Running a script looks identical to the PyTorch case, just with a different image name:
| docker run –rm –gpus all -v /path/to/scripts:/workspace \ tensorflow/tensorflow:latest-gpu python /workspace/train.py |
What differs between the two frameworks is small. The image is tensorflow/tensorflow, and its GPU tag is a plain latest-gpu instead of the explicit CUDA and cuDNN numbers PyTorch pins into its tag. Both honor –gpus all and –gpus ‘”device=0″‘ the same way, and both land in the 4 to 7 GB range. Past that, nothing changes. Volumes, Compose, resource limits, monitoring, all of it behaves the same. Use whichever framework your team already writes in. The container workflow does not care which one you picked.
Also Read: How to Right-Size GPU Instances and Stop Overpaying in 2026
Running AI at scale on Hostrunway
That is the full path. Docker installed, the NVIDIA runtime wired in, and both PyTorch and TensorFlow reaching the GPU from inside a container. The same commands that ran one job here run a hundred across a fleet, unchanged, which is the real reason to bother with any of it.
The value shows up when a model moves from experiment to service. The container that trained on one card today deploys tomorrow with its environment intact, and rolling back a bad release is nothing more than running an older tag. The host stays clean, because every dependency lives in the image instead of scattered across the machine.
If you are still sizing the hardware, Hostrunway runs cloud GPU servers built for exactly this, with current NVIDIA cards and Ubuntu images that show up ready for the driver and toolkit setup above. You can provision one, follow this guide end to end, and have a container on the GPU the same afternoon. The line the cards up against workloads.
One last honest note before you scale it. A bare docker run is not a serving platform. Real request volume needs a model server such as Triton or TorchServe and an orchestrator such as Kubernetes to place and scale replicas across nodes. What Docker hands you is the reliable, identical unit sitting underneath all of that. Deciding what to build around it is the next problem, and a good one to have.
Frequently asked questions
Do I need CUDA installed on the host if I’m using Docker containers?
No, the container brings its own CUDA and cuDNN. The host only needs the NVIDIA driver plus the Container Toolkit. The one real constraint is that the host driver has to be recent enough for the CUDA version inside your image, since the driver is never bundled in the container. A full host CUDA Toolkit is optional and mostly handy for compiling or debugging outside containers.
Can I run multiple AI containers on one GPU simultaneously?
Yes. Point them at the same card with –gpus ‘”device=0″‘ and the driver time-slices between them. That works well for inference and small jobs, less well for training, where they fight over VRAM. On A100 and H100 cards, MIG gives each container an isolated slice and sidesteps the contention.
How do I version control my trained models inside Docker containers?
Keep the model out of the image entirely. Write checkpoints to a mounted volume so they sit on the host, then track them with a large-file tool like Git LFS or DVC, or push them to object storage with a version tag. The container stays a throwaway runtime; the weights and their history live on storage you control and can roll back.
What’s the difference between nvidia-docker and the –gpus flag?
nvidia-docker was the wrapper you needed before Docker 19.03 to get a GPU into a container. Since then that support moved into Docker itself as the –gpus flag, backed by the NVIDIA Container Toolkit. You no longer install a separate nvidia-docker binary. The toolkit supplies the runtime, and –gpus all or –gpus ‘”device=0″‘ is how you ask for cards.
Can I push a trained model to a Docker registry like Docker Hub?
You technically can, by baking the weights into an image and pushing that, but it is usually the wrong call. Images balloon in size, pulls turn slow, and every retrain spawns a fresh image to store and later clean up. Registries were built for code and environments, not gigabytes of data. Put the weights in object storage or a model registry such as MLflow or the Hugging Face Hub, and have the container pull them at startup from a mounted path or a short download step. Your images stay small and your model history stays somewhere tools can reason about it.
How do I monitor GPU memory usage inside a running container?
From the host, docker exec <container_id> nvidia-smi shows live use without opening a shell, and wrapping it in watch -n 1 gives a rolling view. Inside your code, torch.cuda.memory_allocated() reports what the framework is holding right now.
Is Docker secure enough for production AI workloads?
It runs in production everywhere, but the defaults lean permissive, so you harden it. Run as a non-root user, drop capabilities you do not use, mount read-only where you can, and never expose the Docker socket to a container. Keep base images patched. GPU work adds little risk on its own, though the NVIDIA runtime asks for some host trust, so limit who is allowed to launch privileged containers.
Can I use Docker to run AI inference at scale, thousands of requests per second?
Yes, though not from a single docker run. Wrap the model in an inference server such as Triton or TorchServe, then run many replicas behind a load balancer, usually with Kubernetes scheduling them across GPU nodes. Docker gives you one identical replica; the orchestration layer handles the volume, batching, and failover. Triton in particular is built for high-throughput GPU serving.
How much overhead does Docker add to GPU training performance?
Almost none. The GPU runs the math directly and the NVIDIA runtime passes the device through without a virtualization layer, so throughput lands within a point or two of bare metal. The measurable costs sit elsewhere: first-run image pulls, some storage-driver I/O, and startup time, all of which fade next to a long training job.
Can I build my own AI container images instead of using official ones?
Yes, and most teams get there eventually. Start a Dockerfile from an official base like pytorch/pytorch or nvidia/cuda, then layer your dependencies and code on top, which locks your exact versions into a reproducible image you own. Building up from bare Linux is rarely worth the effort, since the official CUDA and framework bases already solve the painful driver-compatibility part. That full build workflow is a guide of its own.
If a container crashes during training, does my model get saved?
Only what already reached disk. Anything held in the container’s memory or its writable layer is gone the moment it exits. Save checkpoints on a schedule to a mounted volume, so a crash costs you minutes instead of the whole run. Pair that with a restart policy and resume-from-checkpoint logic, and the job comes back close to where it stopped.
How do I pass environment variables to a containerized AI model?
Use -e on the command line, like docker run -e MODEL_PATH=/models/latest, or list them under environment: in a Compose file. When there are many, or when some are secrets, load them from a file with –env-file rather than typing them out. GPU jobs often set NVIDIA_VISIBLE_DEVICES to choose cards and read CUDA_VISIBLE_DEVICES inside the framework. Keep genuine secrets out of the image and out of version control, and inject them at run time.
