How Long Should Ollama Keep a Model Loaded?

Ollama can retain a model after a request, let it expire, or unload it immediately. Those choices trade warm-request latency for memory headroom. In an isolated Ollama v0.32.13 CPU run, the same pinned request reported load_duration of 858,726,192ns when cold and 77,440,653ns when warm. Immediate unload then changed /api/ps from one running model to none while the model remained installed.

That receipt does not produce one correct timer for every server. It gives an operator the state and measurements needed to choose. The official Ollama FAQ documents duration strings, seconds, negative pinning and zero unload. This article adds the missing control experiment: installed versus loaded state, a prior pin, fresh timer states, natural expiry, explicit unload and scoped cleanup.

For a bounded experiment, this lab sets the server default to 30 seconds instead of Ollama’s documented five-minute default so expiry remains quick to observe. It binds only to 127.0.0.1, runs as nobody, isolates both HOME and the model store, and uses a small smollm2:135m fixture. Do not expose an unauthenticated Ollama API publicly to reproduce a timer.

Separate Installed Models From Loaded Models

Two endpoints answer different questions. /api/tags lists models available in the model store. /api/ps lists models currently loaded for inference. A model can therefore appear in tags while the running-model array is empty. Unloading should change the second state without deleting the first.

Ollama’s current /api/ps reference exposes the model name, digest, expiry, loaded size, size_vram and context length. In this CPU-only run, size_vram was zero. That does not make /api/ps a whole-host memory meter; it makes it the authoritative Ollama residency check. Use Ollama CPU and GPU placement when processor placement, rather than the timer, is the unresolved question.

Stored model files measured 270,898,672 bytes, while the running-model size field was 295,782,316 bytes with a 512-token context. Do not subtract those values from free -h and call the difference reclaimed RAM. Model files, runner mappings, caches and Linux accounting describe related but different layers.

Pin One Loopback Runtime Before Comparing Timers

A timer comparison is credible only when it names the binary, model, listener and starting state. The first block expects the official ollama-linux-amd64.tar.zst archive in the current directory. It refuses an existing lab path, verifies the exact archive used for this article, starts one isolated server and records the owned PIDs. The archive is large; this is a controlled rehearsal, not an installation recommendation.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
MARKER="$LAB/.voxfor-owned"
ARCHIVE="$PWD/ollama-linux-amd64.tar.zst"
EXPECTED=0fd1dece38a1c6242e8013ce20b597345c5de072ae6b320160edb0e729ef1de1
BASE=http://127.0.0.1:11888

test ! -e "$LAB" || { printf 'Refusing existing path: %s\n' "$LAB" >&2; exit 1; }
command -v curl jq sha256sum zstd tar runuser ss pgrep >/dev/null
install -d -m 0750 "$LAB/runtime" "$LAB/home" "$LAB/models"
printf '%s\n' voxfor-ollama-residency-v1 > "$MARKER"
test "$(sha256sum "$ARCHIVE" | awk '{print $1}')" = "$EXPECTED"
zstd -t "$ARCHIVE" >/dev/null
tar --use-compress-program=unzstd -xf "$ARCHIVE" -C "$LAB/runtime"
test -x "$LAB/runtime/bin/ollama"
chown -R nobody:nogroup "$LAB"

runuser -u nobody -- env \
  HOME="$LAB/home" \
  LD_LIBRARY_PATH="$LAB/runtime/lib/ollama" \
  OLLAMA_HOST=127.0.0.1:11888 \
  OLLAMA_MODELS="$LAB/models" \
  OLLAMA_KEEP_ALIVE=30s \
  OLLAMA_CONTEXT_LENGTH=2048 \
  OLLAMA_NUM_PARALLEL=1 \
  OLLAMA_MAX_LOADED_MODELS=1 \
  OLLAMA_NO_CLOUD=1 \
  "$LAB/runtime/bin/ollama" serve >"$LAB/server.log" 2>&1 &
printf '%s\n' "$!" > "$LAB/wrapper.pid"

for attempt in $(seq 1 200); do
  SERVER_PID=$(pgrep -P "$(<"$LAB/wrapper.pid")" -f "$LAB/runtime/bin/ollama" | head -n1 || true)
  [[ -n ${SERVER_PID:-} ]] && curl -fsS "$BASE/api/version" > "$LAB/version.json" && break
  sleep 0.1
done
printf '%s\n' "$SERVER_PID" > "$LAB/server.pid"
test "$(readlink -f "/proc/$SERVER_PID/exe")" = "$LAB/runtime/bin/ollama"
test "$(jq -r .version "$LAB/version.json")" = 0.32.13
curl -fsS "$BASE/api/ps" | jq -e '.models == []' >/dev/null
curl -fsS "$BASE/api/tags" | jq -e '.models == []' >/dev/null

Loopback binding forms an evidence boundary. For remote maintenance, preserve the private listener and reach the host through private VPS administration rather than moving this unauthenticated lab endpoint onto a public interface.

Pull the fixture without calling it resident.

Pulling a model changes the model store. It should not by itself prove a running model. The next input captures the exact tag, digest and stored size, then requires an empty /api/ps. This is the negative baseline that many short recipes omit.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" '{model:$model,stream:false}')" \
  "$BASE/api/pull" | jq -e '.status == "success"' >/dev/null
curl -fsS "$BASE/api/tags" > "$LAB/tags-after-pull.json"
curl -fsS "$BASE/api/ps" > "$LAB/ps-after-pull.json"

jq -e --arg model "$MODEL" '
  any(.models[]; .name == $model and
    .digest == "9077fe9d2ae1a4a41a868836b56b8163731a8fe16621397028c2c76f838c6907" and
    .size == 270898672)
' "$LAB/tags-after-pull.json" >/dev/null
jq -e '.models == []' "$LAB/ps-after-pull.json" >/dev/null

Retain the tag check after every later unload. If it disappears, the test changed installation state and no longer answers the intended question.

Compare Identical Cold and Warm Pinned Requests

Ollama’s generate API returns total_duration and load_duration in nanoseconds when the response is not streamed. Pinning with a negative duration makes the next comparison easy: run one deterministic request from an empty /api/ps, then repeat the same body while the model remains loaded.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

jq -nc --arg model "$MODEL" '{
  model:$model,
  prompt:"Return the token RESIDENCY-188.",
  stream:false,
  keep_alive:-1,
  options:{temperature:0,seed:188,num_predict:8,num_ctx:512}
}' > "$LAB/cold-request.json"
curl -fsS -H 'Content-Type: application/json' \
  --data-binary @"$LAB/cold-request.json" "$BASE/api/generate" > "$LAB/cold.json"
curl -fsS "$BASE/api/ps" > "$LAB/pinned.json"

jq -e '.done == true and .load_duration > 0 and .total_duration >= .load_duration' \
  "$LAB/cold.json" >/dev/null
jq -e --arg model "$MODEL" '.models | length == 1 and .[0].name == $model' \
  "$LAB/pinned.json" >/dev/null

Response text is irrelevant here; the model is deliberately tiny and generation is capped at eight tokens. The receipt needs the timing fields and running-model state, not a quality judgment.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

curl -fsS -H 'Content-Type: application/json' \
  --data-binary @"$LAB/cold-request.json" "$BASE/api/generate" > "$LAB/warm.json"
jq -e '.done == true and .load_duration > 0' "$LAB/warm.json" >/dev/null
jq -e -s '.[1].load_duration < .[0].load_duration' \
  "$LAB/cold.json" "$LAB/warm.json" >/dev/null
jq -n \
  --argjson cold "$(jq .load_duration "$LAB/cold.json")" \
  --argjson warm "$(jq .load_duration "$LAB/warm.json")" \
  '{cold_load_ns:$cold,warm_load_ns:$warm,warm_lower:($warm < $cold)}'

In the selected run, load_duration measured about 859ms cold and 77ms warm. DatabaseMart’s Ollama performance guide also frames retained loading as a latency tradeoff. Neither measurement authorizes a universal speedup ratio: hardware, model, context, concurrent work and filesystem cache all matter. Use Ollama VPS capacity planning to size the whole workload, not this tiny fixture.

Reset a Negative Pin Before Testing a Shorter Policy

A subtle result appeared after the warm request. Sending another request without a keep_alive field did not shorten the already negative-pinned model in this v0.32.13 sequence. /api/ps still reported a far-future expiry in year 2318. Treat that as observed prior-state behavior, not a promise for every release.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" \
    '{model:$model,prompt:"",stream:false,options:{num_predict:1,num_ctx:512}}')" \
  "$BASE/api/generate" > "$LAB/omitted-after-pin.json"
curl -fsS "$BASE/api/ps" > "$LAB/omitted-after-pin-ps.json"
EXPIRY=$(jq -r '.models[0].expires_at' "$LAB/omitted-after-pin-ps.json")
DELTA=$(( $(date -d "$EXPIRY" +%s) - $(date -u +%s) ))
test "$DELTA" -gt 31536000

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" '{model:$model,keep_alive:0}')" \
  "$BASE/api/generate" >/dev/null
for attempt in $(seq 1 100); do
  curl -fsS "$BASE/api/ps" > "$LAB/reset-after-pin.json"
  jq -e '.models == []' "$LAB/reset-after-pin.json" >/dev/null && break
  sleep 0.05
done
jq -e '.models == []' "$LAB/reset-after-pin.json" >/dev/null

This reset is not housekeeping filler. Without it, the “server default” case would inherit the pinned state and produce a false comparison. Simplified Guide’s Ollama keep-alive recipe correctly emphasizes an observable final model list; the added lesson here is to make the starting list observable too.

Compare a Fresh Server Default With a Fresh Request Override

From empty running state, omit keep_alive and read expires_at. The lab configured OLLAMA_KEEP_ALIVE=30s, so it accepts an observed delta between 20 and 40 seconds to allow request and clock overhead.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1
curl -fsS "$BASE/api/ps" | jq -e '.models == []' >/dev/null

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" \
    '{model:$model,prompt:"",stream:false,options:{num_predict:1,num_ctx:512}}')" \
  "$BASE/api/generate" > "$LAB/default.json"
curl -fsS "$BASE/api/ps" > "$LAB/default-ps.json"
EXPIRY=$(jq -r '.models[0].expires_at' "$LAB/default-ps.json")
DELTA=$(( $(date -d "$EXPIRY" +%s) - $(date -u +%s) ))
test "$DELTA" -ge 20
test "$DELTA" -le 40
printf 'configured_default=30s observed_delta=%ss\n' "$DELTA"

Reset once more before the short override. That ensures the three-second request is not being compared with a still-loaded default case.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" '{model:$model,keep_alive:0}')" \
  "$BASE/api/generate" >/dev/null
for attempt in $(seq 1 100); do
  curl -fsS "$BASE/api/ps" | jq -e '.models == []' >/dev/null && break
  sleep 0.05
done
curl -fsS "$BASE/api/ps" | jq -e '.models == []' >/dev/null

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" \
    '{model:$model,prompt:"",stream:false,keep_alive:"3s",options:{num_predict:1,num_ctx:512}}')" \
  "$BASE/api/generate" > "$LAB/override.json"
curl -fsS "$BASE/api/ps" > "$LAB/override-ps.json"
EXPIRY=$(jq -r '.models[0].expires_at' "$LAB/override-ps.json")
DELTA=$(( $(date -d "$EXPIRY" +%s) - $(date -u +%s) ))
test "$DELTA" -ge 0
test "$DELTA" -le 8
printf 'requested=3s observed_delta=%ss\n' "$DELTA"

Fresh 30-second and three-second observations prove the intended precedence for these controlled states. They do not imply that every later request shortens an already longer residency; the negative-pin control showed exactly why prior state belongs in the receipt.

Wait for Expiry, Then Prove an Explicit Unload

A timer only matters if the model actually leaves the running set. Poll /api/ps until it is empty, then require the installed tag to remain. This separates natural expiry from deletion.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

for attempt in $(seq 1 100); do
  curl -fsS "$BASE/api/ps" > "$LAB/expired-ps.json"
  jq -e '.models == []' "$LAB/expired-ps.json" >/dev/null && break
  sleep 0.1
done
jq -e '.models == []' "$LAB/expired-ps.json" >/dev/null
curl -fsS "$BASE/api/tags" > "$LAB/tags-after-expiry.json"
jq -e --arg model "$MODEL" 'any(.models[]; .name == $model)' \
  "$LAB/tags-after-expiry.json" >/dev/null

For an immediate-release path, reload with the pinned request, capture only processes owned by the lab path, send keep_alive:0, and verify three different postconditions: no running model, installed tag retained and lower aggregate RSS for those owned processes. SumGuy’s Ollama memory-management overview usefully broadens the discussion to context and GPU tools; this bounded block avoids broad process kills and touches only the recorded lab executable.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
MODEL=smollm2:135m
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1

curl -fsS -H 'Content-Type: application/json' \
  --data-binary @"$LAB/cold-request.json" "$BASE/api/generate" >/dev/null
curl -fsS "$BASE/api/ps" > "$LAB/before-unload.json"
jq -e --arg model "$MODEL" 'any(.models[]; .name == $model)' \
  "$LAB/before-unload.json" >/dev/null
RSS_BEFORE=$(ps -eo rss=,args= | awk -v root="$LAB" 'index($0,root){sum+=$1} END{print sum+0}')

curl -fsS -H 'Content-Type: application/json' \
  --data-binary "$(jq -nc --arg model "$MODEL" '{model:$model,keep_alive:0}')" \
  "$BASE/api/generate" >/dev/null
for attempt in $(seq 1 100); do
  curl -fsS "$BASE/api/ps" > "$LAB/after-unload.json"
  jq -e '.models == []' "$LAB/after-unload.json" >/dev/null && break
  sleep 0.05
done
jq -e '.models == []' "$LAB/after-unload.json" >/dev/null
curl -fsS "$BASE/api/tags" | jq -e --arg model "$MODEL" \
  'any(.models[]; .name == $model)' >/dev/null
RSS_AFTER=$(ps -eo rss=,args= | awk -v root="$LAB" 'index($0,root){sum+=$1} END{print sum+0}')
test "$RSS_AFTER" -lt "$RSS_BEFORE"
printf 'owned_rss_before_kib=%s owned_rss_after_kib=%s\n' "$RSS_BEFORE" "$RSS_AFTER"

One selected receipt recorded 371,476KiB before unload and 58,196KiB afterward. /api/ps is the residency proof. The RSS change corroborates that the owned runner disappeared, but it is not an invoice for exact bytes returned to every future allocator. If the host is under pressure, trace a Linux OOM kill before treating any one process metric as the cause.

{
  "product": "Ollama 0.32.13",
  "model": "smollm2:135m",
  "installed_size_bytes": 270898672,
  "cold_load_ns": 858726192,
  "warm_load_ns": 77440653,
  "warm_lower_than_cold": true,
  "omitted_after_negative_pin_remained_pinned": true,
  "fresh_server_default_seconds": 30,
  "fresh_request_override_seconds": 3,
  "running_models_after_expiry": [],
  "running_models_after_explicit_unload": [],
  "model_still_installed": true,
  "owned_process_rss_pinned_kib": 371476,
  "owned_process_rss_after_unload_kib": 58196,
  "cleanup": {
    "listener_closed": true,
    "owned_directory_removed": true,
    "root_home_unchanged": true
  }
}

Choose the Residency Policy From Workload Gaps

Treat the timer as a workload policy, not a tuning badge. First measure the gap between requests for the same model. Then compare cold-load cost with the latency budget and retained memory with the host or cgroup budget. A model that receives another request every few seconds has a different answer from a batch model used twice a day.

Context and concurrency belong beside the timer. A larger context or more simultaneous sequences can change memory even when model weights do not. GPU memory planning for local inference covers weights, context and concurrent-request budgets. vLLM KV-cache budgeting shows why another inference server can reach a different answer under preemption and KV-cache pressure.

Accept a production residency policy only after representative traffic proves the idle-gap distribution, cold and warm service latency, request concurrency, /api/ps state, host or cgroup memory headroom and error rate remain inside explicit workload limits. Recheck after changing the model, quantization, context, parallelism or hardware. A lower warm load_duration in one isolated run proves an avoided load path; it does not prove universal capacity or user-visible latency.

Apply the Change With a Rollback Boundary

For a server-wide policy, change OLLAMA_KEEP_ALIVE in the service’s controlled environment, restart through the normal service manager, and verify version, bind address and /api/ps again. For a caller-specific policy, deploy the request field behind a measured cohort. Change one layer at a time; otherwise a server default and client override can hide each other.

Keep the rollback observable. Save the previous environment value or request body, define the acceptance window before rollout, and restore it when cold latency, memory pressure, queueing or errors cross the bound. The lab cleanup below verifies the recorded executable before stopping it and removes only the fixed marker-owned directory.

If the new policy fails its latency, residency, concurrency or memory acceptance criteria, restore the previous server value or request field. Use keep_alive:0 when immediate release is part of recovery, require the exact model to disappear from /api/ps, confirm /api/tags still contains it, and retain the before/after receipt. Never delete a broad model directory or stop an unverified PID as a timer rollback.

set -Eeuo pipefail
LAB=/tmp/voxfor-ollama-residency-reader
BASE=http://127.0.0.1:11888
test "$(<"$LAB/.voxfor-owned")" = voxfor-ollama-residency-v1
SERVER_PID=$(<"$LAB/server.pid")
test "$(readlink -f "/proc/$SERVER_PID/exe")" = "$LAB/runtime/bin/ollama"
kill "$SERVER_PID"
for attempt in $(seq 1 100); do
  ! ss -H -ltn 'sport = :11888' | grep -q . && break
  sleep 0.05
done
! ss -H -ltn 'sport = :11888' | grep -q .
case "$LAB" in /tmp/voxfor-ollama-residency-reader) ;; *) exit 1;; esac
test -f "$LAB/.voxfor-owned"
find "$LAB" -depth -mindepth 1 -delete
rmdir "$LAB"
test ! -e "$LAB"
! curl -fsS "$BASE/api/version" >/dev/null 2>&1

A defensible final decision fits one sentence: retain this model for this measured idle gap because the avoided load cost is worth this bounded memory residency. If that sentence cannot name both sides, the timer is still a guess.