A financial services team pushing 100,000 AI queries a day is not making a technical decision when it picks a model. It is signing a recurring bill.
Run the numbers against flagship-tier API list pricing, roughly $5 per million input tokens and $30 per million output. A query averaging 1,000 input and 500 output tokens lands near two cents. At 100,000 queries a day, that is about $2,000 daily and $730,000 a year. Every one of those calls also carries customer records past your own network boundary.
DeepSeek R1 changed that arithmetic. Released in January 2025 under an MIT licence and updated in May as R1-0528, it was the first open-weight reasoning model to trade blows with OpenAI’s o1 on maths and code reasoning tests. You can run it on hardware you control. Ollama makes that practical: one binary, one pull command, a REST API on port 11434.
This Ollama GPU deployment guide walks the full path. Sizing GPUs against real DeepSeek R1 hardware requirements, installing and testing, exposing the API without leaving it wide open, tuning for concurrent traffic, and the cost maths finance will ask for before signing anything.
Also Read: Llama 405B vs DeepSeek: Which Cloud GPU Actually Handles Both in 2026?
Self-hosted AI models 2026: what actually changed
Three years ago, running an open model on your own hardware meant accepting a visible quality gap. That gap closed. The distilled 70B variant of R1 scores 86.7% (majority voting) on AIME 2024 and 94.5% on MATH-500, which puts most reasoning work within reach of a single 80 GB card.
Cost is where the argument gets settled. Cloud inference is a per-token bill that never stops. Running the model yourself is a capital purchase plus electricity. An A100 80GB PCIe card is rated at 300 W. Flat out for a year, that is roughly 2,600 kWh, somewhere between $250 and $500 depending on your rate. In that year, it serves billions of tokens.
Then there is data. Queries against an Ollama DeepSeek on-premises AI stack never cross your firewall, which removes a whole category of arguments about compliance. No third-party retention policy to audit, no residency question, no processing agreement to negotiate.
Ollama earns its place by being boring. One command installs it, it manages model versions like a package manager, and it answers over plain HTTP. Most teams end up hybrid anyway: R1 for high-volume routine work, a hosted frontier model for the rare hard question.
Also Read: Local LLM on Dedicated Servers vs Cloud APIs: TCO Comparison in 2026
DeepSeek R1 hardware requirements and sizing
Deployments fail here more than anywhere else, usually because someone sized against parameter count instead of what Ollama actually downloads.
| Model tag | Download (Q4_K_M) | VRAM to serve | Cards that fit | Suited to |
| deepseek-r1:7b | 4.7 GB | 6 to 8 GB | RTX 4060, A4000 | Testing, classification |
| deepseek-r1:14b | 9.0 GB | 12 to 14 GB | RTX 4080, A4000 | Light production |
| deepseek-r1:32b | 20 GB | 24 to 28 GB | RTX 4090, A5000 | Most production teams |
| deepseek-r1:70b | 43 GB | 48 to 56 GB | A100 80GB, H100 | High volume |
| deepseek-r1:671b | 404 GB | 450 GB and up | Multi-GPU cluster | Rare |
One thing vendors routinely blur: the 7 B through 70 B tags are distilled models built on Qwen 2.5 and Llama 3, not the 671B mixture-of-experts architecture. Only 671 B is the full model. Put that in your internal docs, because someone will eventually test the wrong thing.
The VRAM capacity, not clock speed, determines what can run. A 2020 A100 card with 80 GB of memory works better than a newer card with 24 GB of memory for a 70B model, because a model that does not fit spills into system RAM and generation speed collapses.
To calculate size: take the download number, add 20 to 30% for the KV cache and context window, then multiply the cache share by the planned number of parallel request slots you will run.
Storage should be NVMe. Pulling 43 GB off a SATA disk into VRAM adds minutes to every cold start. Network capacity is the wrong worry: 1,000 queries an hour at a 5 KB average response is under 2 KB/s.
Also Read: The Rise of Hybrid GPU Infrastructure: How AI Teams Are Balancing Performance and Cost in 2026
Step 1: Verify your infrastructure
| nvidia-smi # GPU model, VRAM, driver version lspci | grep -i nvidia # is the card visible to the OS at all nvcc –version # CUDA toolkit, if already installed df -h /usr/share/ollama # free space where models will land lsb_release -a # distribution and release |
Ollama’s hardware documentation requires NVIDIA compute capability 5.0 or higher and driver 531 or newer. Any card listed in the range 5.0 to 6.2 requires 570 or newer. Right from the start, check nvidia-smi for the driver version because an outdated driver is the one issue that stops a DeepSeek R1 local deployment on day one.
For free space, budget the model size plus half again. The 70B tag wants 43 GB downloaded and working room during the pull.
Ubuntu 22.04 or 24.04 LTS is the path of least resistance. Ollama runs on macOS and Windows, but production servers are Linux.
Also Read: Power Consumption & Electricity Costs: Cloud GPU vs Bare Metal Dedicated Servers in 2026
Step 2: Install Ollama
| curl -fsSL https://ollama.com/install.sh | sh |
Before sending the script to the shell on the production box, read the script. It automatically determines your distribution, places the binary in /usr/local/bin, creates a user named ollama to execute, and installs a unit that will start upon boot. Two minutes on a decent connection.
| ollama –version systemctl status ollama curl http://localhost:11434/api/tags |
That last command returns JSON with an empty models array on a fresh box. It confirms the API is listening.
Do not create the service user by hand. The installer already did it, and a second useradd either fails or leaves you with a mismatched home directory and models written somewhere you did not expect.
Configuration belongs in a systemd override, never an export in your login shell:
| sudo systemctl edit ollama.service |
| [Service] Environment=”OLLAMA_MODELS=/data/models” |
Then sudo systemctl daemon-reload && sudo systemctl restart ollama. Variables exported in a terminal never reach the service, which costs most teams a confusing hour at least once.
Also Read: How to Host Server in China Without an ICP License in 2026 – Practical Guide for Foreign Businesses
Step 3: Pull the DeepSeek R1 model
| ollama pull deepseek-r1:32b |
| Tag | Size | At 100 Mbps | At 1 Gbps |
| 7b | 4.7 GB | ~7 min | ~1 min |
| 32b | 20 GB | ~28 min | ~3 min |
| 70b | 43 GB | ~60 min | ~6 min |
| 671b | 404 GB | ~9 hours | ~55 min |
Ollama fetches layers in parallel, printing a hash per layer and then success. Interrupting with Ctrl+C is survivable since it resumes, but let it finish if you can.
Default quantisation is Q4_K_M. For multi-step reasoning, a 5-bit build holds accuracy noticeably better:
| ollama pull deepseek-r1:32b-q5_K_M ollama list |
Air-gapped sites can pull once on a staging host and copy /usr/share/ollama/.ollama/models across, or point at an internal registry mirror. Nobody needs to download 43 GB eight times.
Also Read: Rent NVIDIA H100 GPU on Cloud in 2026: Live Pricing, Availability, and How to Deploy
Step 4: Run DeepSeek R1 and test
| ollama run deepseek-r1:32b |
First load moves weights from disk into VRAM, which takes 30 seconds to two minutes.
R1 writes its reasoning inside <think> tags before answering. On a hard prompt, that phase can run 30 to 60 seconds. It is working, not hung. Anything user-facing needs to account for that.
From a second session:
| watch -n 1 nvidia-smi ollama ps |
ollama ps matters more than people expect. Its PROCESSOR column should read 100% GPU. Any CPU percentage means layers spilled out of VRAM, and speed will be poor no matter what else you tune.
Run four query shapes before trusting it: a plain factual lookup, an arithmetic problem, a code generation task, and a long reasoning prompt. Write down time to first token, tokens per second, peak VRAM and GPU utilisation.
Exit with /bye. The model stays resident for five minutes.
Also Read: Hidden Cloud GPU Costs That Are Secretly Draining Your Budget in 2026
Step 5: Expose the API and integrate
Ollama binds to 127.0.0.1:11434. Nothing outside the machine reaches it until you change that.
| curl http://localhost:11434/api/generate -d ‘{ “model”: “deepseek-r1:32b”, “prompt”: “Summarise this contract clause in plain English: …”, “stream”: false }’ |
The response carries response, done, and timings in nanoseconds. Divide eval_count by eval_duration for tokens per second, which is the number worth graphing.
| import requestsr = requests.post(“http://10.0.4.11:11434/api/generate”, json={ “model”: “deepseek-r1:32b”, “prompt”: “Classify this ticket: billing, technical, or sales.”, “stream”: False, “options”: {“num_ctx”: 8192}, }) print(r.json()[“response”]) |
To listen on the network, override the unit again:
| [Service] Environment=”OLLAMA_HOST=0.0.0.0:11434″ |
Read this part twice. Ollama ships no authentication whatsoever. An open 11434 is an open model server for anyone who can route to it, and people scan for it. Put a reverse proxy in front enforcing API keys, and close the port to everything else:
| sudo ufw allow from 10.0.0.0/8 to any port 11434 proto tcp sudo ufw deny 11434/tcp |
Ollama also answers on /v1/chat/completions, so most existing OpenAI SDK code works by changing the base URL and passing any non-empty key.
Also Read: Cloud GPU in USA 2026: Latency, Data Residency & Best Options
Enterprise tuning and scaling
Out of the box, one loaded model serves one request at a time and unloads after five idle minutes. Both defaults are wrong for a busy server.
| [Service]Environment=”OLLAMA_NUM_PARALLEL=4″Environment=”OLLAMA_MAX_LOADED_MODELS=2″Environment=”OLLAMA_KEEP_ALIVE=30m”Environment=”OLLAMA_MAX_QUEUE=512″ |
Every parallel slot needs its own KV cache. Four slots at 8K context multiplies that memory four times. Set the value, then confirm ollama ps still reads 100% GPU. If it dropped, you over-subscribed and made every request slower, including the ones running alone.
Past the queue ceiling, Ollama returns HTTP 503. Handle it client-side with retry and backoff rather than letting it reach a user as an error page.
Quantisation is the cheapest memory lever you have. The 70B tag at Q4_K_M needs roughly 48 GB against the 141 GB an FP16 build would ask for, at a modest accuracy cost.
Monitoring needs honesty. Ollama has no native Prometheus endpoint; curl http://localhost:11434/metrics returns 404, and the feature request has sat open since 2024. Run a proxy exporter in front of the API for request counts and durations, and scrape the cards separately with NVIDIA DCGM Exporter. Alert on GPU temperature above 85°C, VRAM that grows and never falls, and the 503 rate.
The issue of horizontal scaling is one of routing, not clustering. Place Nginx or HAProxy at the front to direct classification and summarisation tasks towards 7B or 14B instances while 32B or 70B carry out true reasoning. A single pool of large instances answering trivial questions wastes most of the GPU budget.
Backups are trivial because model files never change:
| tar -czf ollama-models.tar.gz /usr/share/ollama/.ollama/models |
Also Read: Cloud GPU Security & Compliance: What You Must Know in 2026
DeepSeek R1 vs cloud AI costs
Worked example: 100,000 a day with each 1,000 input and 500 output tokens. This translates to 36.5 billion input tokens and 18.25 billion output tokens annually.
| Line item | Flagship cloud API | Self-hosted R1 |
| Token charges, year 1 | ~$730,000 | $0 |
| Server, 2x A100 80GB | $0 | ~$50,000 once |
| Power and cooling | $0 | ~$4,000/yr |
| Share of an engineer | $0 | ~$25,000/yr |
| Year 1 | $730,000 | ~$79,000 |
| Year 2 | $730,000 | ~$29,000 |
| Year 3 | $730,000 | ~$29,000 |
| Three-year total | ~$2.19M | ~$137,000 |
Those figures move with your token mix, your electricity rate and what you pay engineers. The shape does not. At this volume, break-even arrives inside two months.
The cloud bill hides charges that never show in per-token pricing. Rate limits during a spike. Long-context requests are billed at roughly double the rate past the standard window. Egress. The cost of migrating off if you ever want to.
Self-hosting hides costs too. Somebody patches the OS, handles a dead card, carries the pager. Budget a fraction of a salary, not zero.
A rough decision rule: under about 10,000 queries a day, stay on the API. Above that, or anywhere data residency is written into a contract, the maths and the compliance argument point the same way.
Also Read: How to Run Kubernetes on Cloud GPU – Complete Beginner Guide 2026
Troubleshooting and common issues
CUDA out of memory. The model plus KV cache exceeds VRAM. Lower OLLAMA_NUM_PARALLEL, reduce num_ctx, drop to a smaller quantisation, or step down a size. Check ollama ps first; a forgotten second model is often sitting there.
Model loads, then crashes. Driver and CUDA mismatch. Confirm nvidia-smi reports 531 or newer, reboot after any driver change, and re-pull in case the download was corrupted.
Slow generation. Run ollama ps. Any CPU percentage in PROCESSOR is your answer. If it reads 100% GPU but the speed is still bad, check the iostat -x 1 command and make sure nothing else is fighting for the card.
The device has no available memory. When running as a service, Ollama stores the models in /usr/share/ollama/.ollama/models, not your home directory. Clear old tags with ollama rm deepseek-r1:7b, or repoint OLLAMA_MODELS at a larger volume.
Intermittent timeouts. Usually keep-alive expiry forcing a reload, or queue saturation returning 503. Which one did you hit before raising OLLAMA_KEEP_ALIVE and OLLAMA_MAX_QUEUE?
Works locally, not remotely. Check that OLLAMA_HOST is really 0.0.0.0 in the override, that you ran daemon-reload, and that the port is allowed by the firewall. ss -tlnp | grep 11434 indicates the current details of the bind.
Logs are in journalctl -u ollama -f. Most problems announce themselves there well before a user notices.
Also Read: How Much Does Cloud GPU Really Cost? The Hidden Costs Most People Miss in 2026
Production deployment checklist
Before go-live:
- VRAM sized at model requirement plus 20% headroom
- Driver 531 or newer, compute capability checked against Ollama’s supported list
- Model pulled and confirmed with ollama list
- Baseline recorded: tokens per second, time to first token, peak VRAM
- Load test of at least 1,000 queries at expected concurrency
- Model directory backed up
Network and security:
- Port 11434 reachable only from trusted subnets
- Reverse proxy enforcing API keys in front of Ollama
- Service running as the ollama user, never root
- Rate limiting configured at the proxy
Operations:
- GPU metrics via DCGM Exporter, API metrics via a proxy exporter
- Alerts on GPU temperature, VRAM growth, 503 rate, service restarts
- journalctl output shipped to central logging
- Runbook naming who responds out of hours
- Card refresh planned on a three- to five-year cycle
Documentation:
- API endpoint and auth documented for application teams
- Working examples in the languages your teams actually use
- Fallback path to a hosted API if the cluster fails
Also Read: Docker or Bare Metal on Cloud GPU? How to Choose the Right One in 2026
Where to run a DeepSeek R1 enterprise setup
All of the above assumes a GPU server already exists. Building one from bare metal, OS install through driver packages, CUDA, firewall rules, and monitoring agents, is a day or two before your first ollama pull.
Hostrunway GPU cloud servers arrive with drivers and CUDA already in place and performance-tested, so the sequence is SSH in, run the install script, pull the model. With 160+ locations, you can put inference near the users generating the queries, which does more for response time than any amount of local tuning. And because instances are provisioned on demand with no term contract, a seasonal spike does not mean buying hardware you will idle for the other ten months.
Whether the hardware is yours or rented, the sequence in this ollama gpu deployment guide is the same: pull, test, measure, then decide what deserves to run on it. Record the baseline numbers on day one. That file is the only thing that will settle the argument later.
Frequently asked questions
How does DeepSeek R1 compare with GPT-4o and newer frontier models?
Hosted frontier models are closed, billed per token, and carry the reliability that comes from a vendor running them at scale. R1 is open-weight, MIT licensed, and free to run on hardware you control. It holds its own on maths and code reasoning, with less production mileage behind it.
Do I need a special GPU, or will any card work?
NVIDIA, realistically. Ollama supports AMD through ROCm v7 and Apple Silicon through Metal, but NVIDIA is the best-supported path. You need compute capability 5.0 or higher and driver 531 or newer. An older A100 with 80 GB beats a newer 24 GB card for large models, because capacity decides what fits.
Can I run DeepSeek R1 on CPU only?
It runs, but not usefully. CPU inference on a 32B model is measured in minutes per response where a GPU answers in seconds. The 1.5B and 7B tags are tolerable on CPU for batch work with nobody waiting. Anything interactive needs a GPU.
How do I fine-tune DeepSeek R1 on our own data?
Ollama does not fine-tune. It serves models, and can import weights you trained elsewhere through a Modelfile. Training happens in frameworks like Unsloth or Axolotl, usually on the distilled variants. Most teams get further with retrieval-augmented generation, feeding relevant documents into the prompt, which is faster to build and cheaper to maintain.
Is DeepSeek R1 compliant with HIPAA, SOC 2 or GDPR?
The model is neither compliant nor non-compliant on its own. Your deployment is. Self-hosting removes the third-party processing question, but you still owe access controls, encryption at rest, audit logging and a retention policy. Have legal review the deployment rather than assuming on-premise settles it.
What happens when it gives a wrong answer in production?
R1 hallucinates like every other language model. The R1-0528 update reduced that, it did not remove it. Wrap outputs in validation: schema checks on structured responses, ground truth comparison where it exists, human review on anything with money or safety attached. Running it yourself means you add that logic today instead of filing a vendor ticket.
Can I use it for customer-facing chatbots?
Yes, with guardrails. Constrain it to a retrieved knowledge base rather than open generation, add a confidence threshold that escalates to a person, and hide the <think> output from users. Test against real transcripts first. Reasoning models are slow to first token, which changes how the interface should behave.
How often should we update to newer versions?
Model updates come from DeepSeek, not from Ollama. Pull the new tag, test it in staging against your own evaluation set, then move traffic. The old tag stays on disk, so rollback is instant. Quarterly suits most stable systems. Chasing every release costs more than it returns.
What is the environmental cost of running this around the clock?
An A100 80GB PCIe card is rated to 300 W, an H100 PCIe to 350 W. Continuous operation at 300 W is roughly 2,600 kWh a year before cooling overhead, landing between $250 and $500 per card at typical commercial rates. Running it yourself does let you pick a data centre on renewables.
Can several teams share one deployment?
Yes. One instance serves many applications over the same API. Match OLLAMA_NUM_PARALLEL to your VRAM, put a proxy in front issuing per-team keys, and log request counts against those keys. Without that logging you cannot allocate cost, and somebody’s batch job will eventually starve everyone else.
What is a sensible hybrid strategy?
Route by difficulty, not by preference. Classification, extraction, summarisation and internal search go to R1 on your own hardware. Anything where a wrong answer is expensive goes to a hosted frontier model. A cheap classifier at the front deciding which path a request takes usually pays for itself within weeks.
How do we measure the return against cloud APIs?
Track three numbers monthly: query volume, fully loaded cost per query including power and engineering time, and accuracy against a fixed evaluation set you wrote yourself. Compare against what the same volume would cost at current list rates. Keep the evaluation set stable, or the accuracy comparison means nothing.
