6  Usage Practices

Last modified: 2026-02-08

This chapter expands on the basics from Getting Started with practical patterns for day-to-day work: function identity in depth, cache invalidation strategies, testing, REPL introspection, serialization constraints, and debugging tips.

Setup

(ns pocket-book.usage-practices
  (:require
   ;; Logging setup for this chapter (see Logging chapter):
   [pocket-book.logging]
   ;; Pocket API:
   [scicloj.pocket :as pocket]
   ;; Annotating kinds of visualizations:
   [scicloj.kindly.v4.kind :as kind]))
(def test-dir "/tmp/pocket-dev-practices")
(pocket/set-base-cache-dir! test-dir)
10:06:34.519 INFO scicloj.pocket - Cache dir set to: /tmp/pocket-dev-practices

"/tmp/pocket-dev-practices"

When to Use Pocket

Good use cases

  • Data science pipelines with expensive intermediate steps (data loading, preprocessing, feature engineering, model training)

  • Reproducible research where cached intermediate results let you iterate on downstream steps without re-running upstream computations

  • Long-running computations (minutes to hours) that need to survive JVM restarts, crashes, or machine reboots

  • Multi-threaded workflows where multiple threads may request the same expensive computation β€” Pocket ensures it runs only once

Comparison to alternatives

Feature Pocket clojure.core/memoize core.memoize
Persistence Disk + memory Memory only Memory only
Cross-session Yes No No
Lazy evaluation IDeref Eager Eager
Eviction policies LRU, FIFO, TTL, etc. None LRU, TTL, etc.
Thread-safe (single computation) Yes No Yes
Pipeline caching Yes (recursive) No No

Function Identity: Use Vars or Keywords

Pocket requires functions to be passed as vars (#'fn-name) or keywords (e.g., :train), not as bare function objects.

;; ❌ WRONG - bare function, unstable identity
(pocket/cached my-expensive-fn arg1 arg2)

;; βœ… CORRECT - var, stable identity
(pocket/cached #'my-expensive-fn arg1 arg2)

;; βœ… CORRECT - keyword, stable identity
(pocket/cached :train cached-split)

Why? Function objects have different identity across JVM sessions, making cache keys unpredictable. Vars provide stable symbol names that survive restarts. Keywords are inherently stable and work naturally as map accessors on cached values.

Pocket validates this and throws a clear error if you forget:

(defn example-fn [x] (* x x))
(try
  (pocket/cached example-fn 5)
  (catch Exception e
    (ex-message e)))
"pocket/cached requires a var or keyword (e.g., #'my-fn or :my-key), got: class pocket_book.usage_practices$example_fn"

Cache Invalidation Strategies

Pocket does not automatically detect when a function’s implementation changes. You must invalidate manually. Here are the strategies:

Strategy 1: Manual Invalidation

Use invalidate! for specific entries or invalidate-fn! for all entries of a function:

(pocket/cleanup!)
10:06:34.524 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed false}
(defn transform [x] (* x 2))

Cache a computation:

(deref (pocket/cached #'transform 10))
10:06:34.525 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.526 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10)

20

Function implementation changed β€” invalidate a single entry:

(pocket/invalidate! #'transform 10)
10:06:34.528 INFO scicloj.pocket.impl.cache - Invalidated: /tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10) existed= true

{:path
 "/tmp/pocket-dev-practices/1c/(pocket-book.usage-practices_transform 10)",
 :existed true}

Or invalidate all entries for a function:

(deref (pocket/cached #'transform 1))
10:06:34.528 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.529 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)

2
(deref (pocket/cached #'transform 2))
10:06:34.530 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.530 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)

4
(pocket/invalidate-fn! #'transform)
10:06:34.531 INFO scicloj.pocket.impl.cache - Invalidated 2 entries for pocket-book.usage-practices/transform

{:fn-name "pocket-book.usage-practices/transform",
 :count 2,
 :paths
 ["/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)"
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)"]}

Strategy 2: Versioning Pattern

Add a version key to your function’s input. Bumping the version creates new cache entries while preserving old ones for comparison:

(pocket/cleanup!)
10:06:34.532 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn process-data [{:keys [data version]}]
  {:result (reduce + data)
   :version version})

Version 1:

(deref (pocket/cached #'process-data {:data [1 2 3] :version 1}))
10:06:34.534 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-data
10:06:34.535 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/58/(pocket-book.usage-practices_process-data {:data [1 2 3], :version 1})

{:result 6, :version 1}

After changing the function, bump version:

(deref (pocket/cached #'process-data {:data [1 2 3] :version 2}))
10:06:34.536 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-data
10:06:34.537 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/64/(pocket-book.usage-practices_process-data {:data [1 2 3], :version 2})

{:result 6, :version 2}

Both versions coexist in cache β€” useful for A/B comparison.

Strategy 3: Full Cleanup

For a fresh start, use cleanup! to delete the entire cache:

(pocket/cleanup!)
10:06:34.538 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}

Testing with Pocket

Tests should be isolated from production caches and from each other. Use binding and cleanup fixtures:

Test Fixture Pattern

(def test-cache-dir "/tmp/my-project-test-cache")

(defn cleanup-fixture [f]
  (binding [pocket/*base-cache-dir* test-cache-dir]
    (pocket/cleanup!)
    (try
      (f)
      (finally
        (pocket/cleanup!)))))

(use-fixtures :each cleanup-fixture)

This ensures: 1. Tests use a separate cache directory 2. Cache is cleared before and after each test 3. Tests don’t affect each other

Verifying Cache Behavior

Use an atom to track computation calls:

(pocket/cleanup!)
10:06:34.539 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed false}
(def call-count (atom 0))
(defn tracked-fn [x]
  (swap! call-count inc)
  (* x x))

First call computes:

(reset! call-count 0)
0
(let [result (deref (pocket/cached #'tracked-fn 5))
      calls @call-count]
  {:result result :calls calls})
10:06:34.541 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/tracked-fn
10:06:34.542 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/4a/(pocket-book.usage-practices_tracked-fn 5)

{:result 25, :calls 1}

Second call uses cache (no additional computation):

(let [result (deref (pocket/cached #'tracked-fn 5))
      calls @call-count]
  {:result result :calls calls})
{:result 25, :calls 1}

REPL Development Workflow

Inspecting the Cache

Use introspection functions to understand cache state:

(pocket/cleanup!)
10:06:34.545 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(deref (pocket/cached #'transform 1))
10:06:34.545 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.546 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)

2
(deref (pocket/cached #'transform 2))
10:06:34.547 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.547 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)

4
(deref (pocket/cached #'tracked-fn 3))
10:06:34.548 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/tracked-fn
10:06:34.548 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/e8/(pocket-book.usage-practices_tracked-fn 3)

9

See all cached entries:

(count (pocket/cache-entries))
3

Get aggregate statistics:

(:total-entries (pocket/cache-stats))
3

Visualize cache structure:

(pocket/dir-tree)
pocket-dev-practices
β”œβ”€β”€ db
β”‚   └── (pocket-book.usage-practices_transform 1)
β”‚       β”œβ”€β”€ meta.edn
β”‚       └── value.nippy
β”œβ”€β”€ e4
β”‚   └── (pocket-book.usage-practices_transform 2)
β”‚       β”œβ”€β”€ meta.edn
β”‚       └── value.nippy
└── e8
    └── (pocket-book.usage-practices_tracked-fn 3)
        β”œβ”€β”€ meta.edn
        └── value.nippy

Each directory contains a meta.edn file with metadata about the cached computation:

(-> (pocket/cache-entries)
    first
    :path
    (str "/meta.edn")
    slurp
    clojure.edn/read-string)
{:id "(pocket-book.usage-practices/transform 1)",
 :fn-name "pocket-book.usage-practices/transform",
 :args-str "[1]",
 :created-at "2026-02-09T08:06:34.545970813Z"}

This same information is available through the API:

(pocket/cache-entries)
[{:path
  "/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)",
  :id "(pocket-book.usage-practices/transform 1)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[1]",
  :created-at "2026-02-09T08:06:34.545970813Z"}
 {:path
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)",
  :id "(pocket-book.usage-practices/transform 2)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[2]",
  :created-at "2026-02-09T08:06:34.547336137Z"}
 {:path
  "/tmp/pocket-dev-practices/e8/(pocket-book.usage-practices_tracked-fn 3)",
  :id "(pocket-book.usage-practices/tracked-fn 3)",
  :fn-name "pocket-book.usage-practices/tracked-fn",
  :args-str "[3]",
  :created-at "2026-02-09T08:06:34.548536434Z"}]

Filter entries by function name:

(pocket/cache-entries (str (ns-name *ns*) "/transform"))
[{:path
  "/tmp/pocket-dev-practices/db/(pocket-book.usage-practices_transform 1)",
  :id "(pocket-book.usage-practices/transform 1)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[1]",
  :created-at "2026-02-09T08:06:34.545970813Z"}
 {:path
  "/tmp/pocket-dev-practices/e4/(pocket-book.usage-practices_transform 2)",
  :id "(pocket-book.usage-practices/transform 2)",
  :fn-name "pocket-book.usage-practices/transform",
  :args-str "[2]",
  :created-at "2026-02-09T08:06:34.547336137Z"}]

Checking Cached Status

Cached values print their status without forcing computation:

(pocket/cleanup!)
10:06:34.557 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(def pending-value (pocket/cached #'transform 99))

Before deref:

(pr-str pending-value)
"#<Cached (pocket-book.usage-practices/transform 99) :pending>"
(deref pending-value)
10:06:34.559 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/transform
10:06:34.560 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/d8/(pocket-book.usage-practices_transform 99)

198

After deref:

(pr-str pending-value)
"#<Cached (pocket-book.usage-practices/transform 99) :cached>"

Debugging with Logging

Enable debug logging to see cache hits, misses, and writes. See the Logging chapter for setup.

With debug logging enabled, you’ll see:

  • Cache miss, computing: ... β€” computation triggered
  • Cache hit (memory): ... β€” served from in-memory cache
  • Cache hit (disk): ... β€” loaded from disk
  • Cache write: ... β€” written to disk

Long Cache Keys

When a cache key string exceeds 240 characters, Pocket falls back to using a SHA-1 hash as the directory name. This ensures the filesystem can handle arbitrarily complex arguments while maintaining correct caching behavior.

(pocket/cleanup!)
10:06:34.562 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn process-long-text [text]
  (str "Processed: " (count text) " chars"))
(def long-text (apply str (repeat 300 "x")))
(deref (pocket/cached #'process-long-text long-text))
10:06:34.564 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/process-long-text
10:06:34.564 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/6d/6d7a9c06574c78354a8235a2e77b6ed27348e404

"Processed: 300 chars"

The entry is stored with a hash-based directory name:

(pocket/dir-tree)
pocket-dev-practices
└── 6d
    └── 6d7a9c06574c78354a8235a2e77b6ed27348e404
        β”œβ”€β”€ meta.edn
        └── value.nippy

But meta.edn inside still contains the full details, so cache-entries and invalidate-fn! work correctly:

(-> (pocket/cache-entries (str (ns-name *ns*) "/process-long-text"))
    first
    :fn-name)
"pocket-book.usage-practices/process-long-text"

Serialization Constraints

Pocket uses Nippy for serialization. Most Clojure data structures work, but some don’t:

βœ… Safe to Cache

  • Primitive types (numbers, strings, keywords, symbols)
  • Collections (vectors, maps, sets, lists)
  • Records and most deftypes
  • Java Serializable objects

⚠️ Requires Care

  • Lazy sequences β€” Nippy fully realizes them during serialization, which means infinite lazy seqs will hang or OOM. Force lazy seqs with doall inside your function (see below) to keep realization explicit and catch errors early. The round-trip type may also change (a LazySeq comes back as a regular seq).

❌ Cannot Cache

  • Open file handles, streams
  • Network connections, sockets
  • Functions, closures (use vars instead)
  • Atoms, refs, agents (stateful references)

Lazy Sequences

Use doall or vec to force evaluation inside your function:

(pocket/cleanup!)
10:06:34.569 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
(defn generate-data [n]
  ;; doall forces full evaluation of the lazy sequence
  (doall (range n)))
(deref (pocket/cached #'generate-data 5))
10:06:34.571 INFO scicloj.pocket.impl.cache - Cache miss, computing: pocket-book.usage-practices/generate-data
10:06:34.572 DEBUG scicloj.pocket.impl.cache - Cache write: /tmp/pocket-dev-practices/79/(pocket-book.usage-practices_generate-data 5)

(0 1 2 3 4)

Known Limitations

  • No automatic cache invalidation β€” Pocket doesn’t detect when a function’s implementation changes. Use invalidate!, invalidate-fn!, or the versioning pattern described above.

  • Requires serializable values β€” Nippy handles most Clojure types, but you can’t cache functions, atoms, channels, file handles, or other stateful objects.

  • Disk cache grows indefinitely β€” The in-memory cache supports eviction policies (LRU, TTL, etc.), but the disk cache has no automatic cleanup. Use cleanup! or invalidate-fn! periodically if disk space is a concern.

  • No disk cache TTL β€” Cached values on disk never expire automatically. If you need time-based expiration, you’ll need to manage it externally or use cleanup!.

Summary

Practice Recommendation
Function identity Always use vars (#'fn-name)
Invalidation Manual, versioning, or full cleanup
Testing Use binding + cleanup fixtures
Debugging Enable logging, use introspection
Long cache keys Auto-handled with SHA-1 fallback
Serialization Avoid stateful objects; force lazy seqs with doall
Configuration Use pocket.edn β€” see Configuration 
(pocket/cleanup!)
10:06:34.574 INFO scicloj.pocket - Cache cleanup: /tmp/pocket-dev-practices

{:dir "/tmp/pocket-dev-practices", :existed true}
source: notebooks/pocket_book/usage_practices.clj