Pocket

What is this about?

Pocket makes it easy to cache expensive function calls to disk and reuse results across sessions. If a computation takes longer than disk I/O, Pocket can help — wrap it once, and the result is saved for next time. This is especially useful for data science workflows with expensive intermediate steps that need to survive JVM restarts.

Under the hood, Pocket derives cache keys from the function identity and its arguments (so the same computation always maps to the same cache entry), uses Nippy for fast serialization, and provides an in-memory layer backed by core.cache with configurable eviction policies. Concurrent uses of the same computation are thread-safe — the computation runs only once.

Key features

Lazy evaluation — cached returns a deref-able value; computation runs only when needed
Pipeline caching — chain cached steps into pipelines with automatic provenance tracking
Storage policies — :mem+disk (default), :mem (no disk I/O), or :none (identity tracking only)
DAG introspection — reconstruct the full computation graph and render it as a flowchart
Thread-safe — concurrent uses of the same computation run it exactly once
Configurable — cache directory, eviction policy, and storage mode via pocket.edn, environment variables, or code

General info

Website	https://scicloj.github.io/pocket/
Source
Deps
License	MIT
Status	🛠alpha🛠

Quick example

(require '[scicloj.pocket :as pocket])

(pocket/set-base-cache-dir! "/tmp/my-cache")

(defn expensive-calculation [x y]
  (Thread/sleep 5000)
  (+ x y))

;; Create a lazy cached computation
(def result (pocket/cached #'expensive-calculation 10 20))

@result  ;; => 30 (computes and caches — takes 5 seconds)
@result  ;; => 30 (loads from cache — instant)

Discussion

The best places to discuss this project are:

a topic thread under the #data-science channel at the Clojurians Zulip (more about chat channels here)
- It is highly recommended to create separate topics for separate questions.
a github issue

License

Distributed under the MIT License — see LICENSE.

Chapters in this book

source: notebooks/index.clj

1 Preface