6  Usage Practices

Last modified: 2026-02-08

This chapter expands on the basics from Getting Started with practical patterns for day-to-day work: function identity in depth, cache invalidation strategies, testing, REPL introspection, serialization constraints, and debugging tips.

Setup

(ns pocket-book.usage-practices
  (:require
   ;; Logging setup for this chapter (see Logging chapter):
   [pocket-book.logging]
   ;; Pocket API:
   [scicloj.pocket :as pocket]
   ;; Annotating kinds of visualizations:
   [scicloj.kindly.v4.kind :as kind]))
(def test-dir "/tmp/pocket-dev-practices")
(pocket/set-base-cache-dir! test-dir)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache dir set to: /tmp/pocket-dev-practices

"/tmp/pocket-dev-practices"

When to Use Pocket

Good use cases

  • Data science pipelines with expensive intermediate steps (data loading, preprocessing, feature engineering, model training)

  • Reproducible research where cached intermediate results let you iterate on downstream steps without re-running upstream computations

  • Long-running computations (minutes to hours) that need to survive JVM restarts, crashes, or machine reboots

  • Multi-threaded workflows where multiple threads may request the same expensive computation β€” Pocket ensures it runs only once

Comparison to alternatives

Feature Pocket clojure.core/memoize core.memoize
Persistence Disk + memory Memory only Memory only
Cross-session Yes No No
Lazy evaluation IDeref Eager Eager
Eviction policies LRU, FIFO, TTL, etc. None LRU, TTL, etc.
Thread-safe (single computation) Yes No Yes
Pipeline caching Yes (recursive) No No

Function Identity: Use Vars or Keywords

Pocket requires functions to be passed as vars (#'fn-name) or keywords (e.g., :train), not as bare function objects.

;; ❌ WRONG - bare function, unstable identity
(pocket/cached my-expensive-fn arg1 arg2)

;; βœ… CORRECT - var, stable identity
(pocket/cached #'my-expensive-fn arg1 arg2)

;; βœ… CORRECT - keyword, stable identity
(pocket/cached :train cached-split)

Why? Function objects have different identity across JVM sessions, making cache keys unpredictable. Vars provide stable symbol names that survive restarts. Keywords are inherently stable and work naturally as map accessors on cached values.

Pocket validates this and throws a clear error if you forget:

(defn example-fn [x] (* x x))
(try
  (pocket/cached example-fn 5)
  (catch Exception e
    (ex-message e)))
"pocket/cached requires a var or keyword (e.g., #'my-fn or :my-key), got: class pocket_book.usage_practices$example_fn"

Cache Invalidation Strategies

Pocket does not automatically detect when a function’s implementation changes. You must invalidate manually. Here are the strategies:

Strategy 1: Manual Invalidation

Use invalidate! for specific entries or invalidate-fn! for all entries of a function:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed false}
(defn transform [x] (* x 2))

Cache a computation:

(deref (pocket/cached #'transform 10))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10)

20

Function implementation changed β€” invalidate a single entry:

(pocket/invalidate! #'transform 10)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Invalidated: /tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10) existed= true

{:path
 "/tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10)",
 :existed true}

Or invalidate all entries for a function:

(deref (pocket/cached #'transform 1))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)

2
(deref (pocket/cached #'transform 2))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)

4
(pocket/invalidate-fn! #'transform)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Invalidated 2 entries for pocket-book.usage-practices/transform

{:fn-name "pocket-book.usage-practices/transform",
 :count 2,
 :paths
 ["/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)"
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)"]}

Strategy 2: Versioning Pattern

Add a version key to your function’s input. Bumping the version creates new cache entries while preserving old ones for comparison:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn process-data [{:keys [data version]}]
  {:result (reduce + data)
   :version version})

Version 1:

(deref (pocket/cached #'process-data {:data [1 2 3] :version 1}))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-data
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/58/(pocket-book.usage-practices_process-data {:data [1 2 3], :version 1})

{:result 6, :version 1}

After changing the function, bump version:

(deref (pocket/cached #'process-data {:data [1 2 3] :version 2}))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-data
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/64/(pocket-book.usage-practices_process-data {:data [1 2 3], :version 2})

{:result 6, :version 2}

Both versions coexist in cache β€” useful for A/B comparison.

Strategy 3: Full Cleanup

For a fresh start, use cleanup! to delete the entire cache:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}

Testing with Pocket

Tests should be isolated from production caches and from each other. Use binding and cleanup fixtures:

Test Fixture Pattern

(def test-cache-dir "/tmp/my-project-test-cache")

(defn cleanup-fixture [f]
  (binding [pocket/*base-cache-dir* test-cache-dir]
    (pocket/cleanup!)
    (try
      (f)
      (finally
        (pocket/cleanup!)))))

(use-fixtures :each cleanup-fixture)

This ensures: 1. Tests use a separate cache directory 2. Cache is cleared before and after each test 3. Tests don’t affect each other

Verifying Cache Behavior

Use an atom to track computation calls:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed false}
(def call-count (atom 0))
(defn tracked-fn [x]
  (swap! call-count inc)
  (* x x))

First call computes:

(reset! call-count 0)
0
(let [result (deref (pocket/cached #'tracked-fn 5))
      calls @call-count]
  {:result result :calls calls})
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/tracked-fn
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/4a/(pocket-book.usage-practices_tracked-fn 5)

{:result 25, :calls 1}

Second call uses cache (no additional computation):

(let [result (deref (pocket/cached #'tracked-fn 5))
      calls @call-count]
  {:result result :calls calls})
{:result 25, :calls 1}

REPL Development Workflow

Inspecting the Cache

Use introspection functions to understand cache state:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(deref (pocket/cached #'transform 1))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)

2
(deref (pocket/cached #'transform 2))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)

4
(deref (pocket/cached #'tracked-fn 3))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/tracked-fn
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e8/(pocket-book.usage-practices_tracked-fn 3)

9

See all cached entries:

(count (pocket/cache-entries))
3

Get aggregate statistics:

(:total-entries (pocket/cache-stats))
3

Visualize cache structure:

(pocket/dir-tree)
pocket-dev-practices
β”œβ”€β”€ db
β”‚   └── (pocket-book.usage-practices_transform 1)
β”‚       β”œβ”€β”€ meta.edn
β”‚       └── value.nippy
β”œβ”€β”€ e4
β”‚   └── (pocket-book.usage-practices_transform 2)
β”‚       β”œβ”€β”€ meta.edn
β”‚       └── value.nippy
└── e8
    └── (pocket-book.usage-practices_tracked-fn 3)
        β”œβ”€β”€ meta.edn
        └── value.nippy

Each directory contains a meta.edn file with metadata about the cached computation:

(-> (pocket/cache-entries)
    first
    :path
    (str "/meta.edn")
    slurp
    clojure.edn/read-string)
{:id "(pocket-book.usage-practices/transform 1)",
 :fn-name "pocket-book.usage-practices/transform",
 :args-str "[1]",
 :created-at "2026-03-01T14:31:42.241460726Z"}

This same information is available through the API:

(pocket/cache-entries)
[{:path
  "/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)",
  :id "(pocket-book.usage-practices/transform 1)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[1]",
  :created-at "2026-03-01T14:31:42.241460726Z"}
 {:path
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)",
  :id "(pocket-book.usage-practices/transform 2)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[2]",
  :created-at "2026-03-01T14:31:42.242730801Z"}
 {:path
  "/tmp/pocket-dev-practices/e8/(pocket-book.usage-practices_tracked-fn 3)",
  :id "(pocket-book.usage-practices/tracked-fn 3)",
  :fn-name "pocket-book.usage-practices/tracked-fn",
  :args-str "[3]",
  :created-at "2026-03-01T14:31:42.243660295Z"}]

Filter entries by function name:

(pocket/cache-entries (str (ns-name *ns*) "/transform"))
[{:path
  "/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)",
  :id "(pocket-book.usage-practices/transform 1)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[1]",
  :created-at "2026-03-01T14:31:42.241460726Z"}
 {:path
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)",
  :id "(pocket-book.usage-practices/transform 2)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[2]",
  :created-at "2026-03-01T14:31:42.242730801Z"}]

Checking Cached Status

Cached values print their status without forcing computation:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(def pending-value (pocket/cached #'transform 99))

Before deref:

(pr-str pending-value)
"#<Cached (pocket-book.usage-practices/transform 99) :pending>"
(deref pending-value)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/d8/(pocket-book.usage-practices_transform 99)

198

After deref:

(pr-str pending-value)
"#<Cached (pocket-book.usage-practices/transform 99) :cached>"

Debugging with Logging

Enable debug logging to see cache hits, misses, and writes. See the Logging chapter for setup.

With debug logging enabled, you’ll see:

  • Cache miss, computing: ... β€” computation triggered
  • Cache hit (memory): ... β€” served from in-memory cache
  • Cache hit (disk): ... β€” loaded from disk
  • Cache write: ... β€” written to disk

Long Cache Keys

When a cache key string exceeds 240 characters, Pocket falls back to using a SHA-1 hash as the directory name. This ensures the filesystem can handle arbitrarily complex arguments while maintaining correct caching behavior.

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn process-long-text [text]
  (str "Processed: " (count text) " chars"))
(def long-text (apply str (repeat 300 "x")))
(deref (pocket/cached #'process-long-text long-text))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-long-text
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/6d/6d7a9c06574c78354a8235a2e77b6ed27348e404

"Processed: 300 chars"

The entry is stored with a hash-based directory name:

(pocket/dir-tree)
pocket-dev-practices
└── 6d
    └── 6d7a9c06574c78354a8235a2e77b6ed27348e404
        β”œβ”€β”€ meta.edn
        └── value.nippy

But meta.edn inside still contains the full details, so cache-entries and invalidate-fn! work correctly:

(-> (pocket/cache-entries (str (ns-name *ns*) "/process-long-text"))
    first
    :fn-name)
"pocket-book.usage-practices/process-long-text"

Serialization Constraints

Pocket uses Nippy for serialization. Most Clojure data structures work, but some don’t:

βœ… Safe to Cache

  • Primitive types (numbers, strings, keywords, symbols)
  • Collections (vectors, maps, sets, lists)
  • Records and most deftypes
  • Java Serializable objects

⚠️ Requires Care

  • Lazy sequences β€” Nippy fully realizes them during serialization, which means infinite lazy seqs will hang or OOM. Force lazy seqs with doall inside your function (see below) to keep realization explicit and catch errors early. The round-trip type may also change (a LazySeq comes back as a regular seq).

❌ Cannot Cache

  • Open file handles, streams
  • Network connections, sockets
  • Functions, closures (use vars instead)
  • Atoms, refs, agents (stateful references)

Lazy Sequences

Use doall or vec to force evaluation inside your function:

(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn generate-data [n]
  ;; doall forces full evaluation of the lazy sequence
  (doall (range n)))
(deref (pocket/cached #'generate-data 5))
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/generate-data
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/79/(pocket-book.usage-practices_generate-data 5)

(0 1 2 3 4)

Known Limitations

  • No automatic cache invalidation β€” Pocket doesn’t detect when a function’s implementation changes. Use invalidate!, invalidate-fn!, or the versioning pattern described above.

  • Requires serializable values β€” Nippy handles most Clojure types, but you can’t cache functions, atoms, channels, file handles, or other stateful objects.

  • Disk cache grows indefinitely β€” The in-memory cache supports eviction policies (LRU, TTL, etc.), but the disk cache has no automatic cleanup. Use cleanup! or invalidate-fn! periodically if disk space is a concern.

  • No disk cache TTL β€” Cached values on disk never expire automatically. If you need time-based expiration, you’ll need to manage it externally or use cleanup!.

Summary

Practice Recommendation
Function identity Always use vars (#'fn-name)
Invalidation Manual, versioning, or full cleanup
Testing Use binding + cleanup fixtures
Debugging Enable logging, use introspection
Long cache keys Auto-handled with SHA-1 fallback
Serialization Avoid stateful objects; force lazy seqs with doall
Configuration Use pocket.edn β€” see Configuration 
(pocket/cleanup!)
[nREPL-session-120ee500-d4ba-4b41-bcdc-e26822f35e2b] INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
source: notebooks/pocket_book/usage_practices.clj