Clay Documentation

1 Clay

Clay logo

1.1 About

Clay is a minimalistic Clojure tool for data visualization and literate programming, compatible with the Kindly convention. It allows to conduct visual data explorations and create documents (HTML pages like this one, books, blog posts, reports, slideshows) from source code and comments.

Source: (GitHub repo)

Artifact: Clojars Project

Status: The project has moved into Beta stage (March 2024).

Clay is developed by Timothy Pratley & Daniel Slutsky in parallel and in coordination with Claykind, a tool with similar goals which is build in a more thoughtful process, aiming at a more modular structure.

1.2 Goals

  • Easily explore & share data visualizations and notebooks for others to easily pick & use.
  • Encourage writing Kindly-compatible notes for future compatiblity with other tools.
  • Flow with the REPL: encourage user interactions that flow naturally with the typical use of Clojure in editors and REPLs.

1.3 Getting started

Add Clay to your project dependencies: Clojars Project

(If you like to use aliases, you may add under it to the extra dependencies under an alias. See, for example, the deps.edn file of Noj. If you do not know what this means, just add it under the main :deps section of your deps.edn file.)

To render a given Clojure namespace, say "notebooks/index.clj", you may run the following in the REPL:

(require '[scicloj.clay.v2.api :as clay])
(clay/make! {:source-path "notebooks/index.clj"})

This will render an HTML page and serve it in Clay’s browser view. Note that Clay does not need to be mentioned in the namespace we wish to render.

See the API and Configuration subsections for more options and variations.

See the Setup section and recent Videos for details about integrating Clay with your editor so you do not need to call make! yourself.

1.4 Projects using Clay

1.5 Videos

Dec. 17th 2023 Cursive integration, API, configuration - blogging demo
Dec. 17th 2023 CIDER integration - image processing demo
Dec. 16th 2023 Calva integration - datavis demo
Dec. 12th 2023 Demo & Clay overview - London Clojurians - see Tim's part
Dec. 1st 2023 Kindly & Clay overview - visual-tools group - see Daniel's & Tim's parts
June 10th 2023 An early overview - babashka-conf

1.6 Setup

See the example project for a concrete example.

To enjoy Clay’s dynamic interaction, you also need to inform it about code evaluations. This requires some editor setup.

To use Quarto-related actions, it is necessary to have the Quarto CLI installed in your system.

See the suggested setup for popular editors below. If your favourite editor is not supported yet, let us talk and make it work.

1.6.1 VSCode Calva

If you now run a REPL with Clay version in your classpath, then Calva will have the relevant custom REPL commands, as defined here.

name function
Clay make Namespace as HTML will genenrate an HTML rendering of the current namespace.
Clay make Namespace as Quarto, then HTML will generate a Quarto .qmd rendering of the current namespace, then render it as HTML through Quarto.
Clay make Namespace as Quarto, then reveal.js will generate a Quarto .qmd rendering of the current namespace, then render it as a reveal.js slideshow through Quarto.
Clay make current form as HTML will generate an HTML rendering of the current form, in the context of the current namespace.

1.6.2 Emacs CIDER

See the clay.el package for the relevant interactive functions.

1.6.3 IntelliJ Cursive

Under preferences, search for “REPL Commands” (or use the menu IntelliJ -> Preferences -> Languages and Frameworks -> Clojure -> REPL Commands)

Add a global command, and edit it with these settings:

Name: Send form to Clay
Execution: Command

(do (require '[scicloj.clay.v2.api :as clay])
    (clay/make! {:single-form '~form-before-caret
                 :source-path ["~file-path"]}))

You might also like to create a command to compile the namespace:

(do (require '[scicloj.clay.v2.api :as clay])
    (clay/make! {:source-path ["~file-path"]}))

Or a top-level-form (replace form-before-caret with top-level-form).

You can then add keybindings under Preferences -> Keymap for the new commands.

For more information about commands, see the Cursive documentation on REPL commands and substitutions.

1.7 Example notebook namespace

This notebook is created by a Clojure namespace. Here is the namespace definition and a few examples of what such a namespace may contain.

(ns index
  (:require
   [scicloj.kindly.v4.api :as kindly]
   [scicloj.kindly.v4.kind :as kind]
   [scicloj.clay.v2.quarto.highlight-styles :as quarto.highlight-styles]
   [scicloj.clay.v2.quarto.themes :as quarto.themes]
   [scicloj.metamorph.ml.toydata :as toydata]
   [scicloj.hanamicloth.v1.api :as haclo]
   [tablecloth.api :as tc]))

A Hiccup spec:

(kind/hiccup
 [:div {:style {:background "#efe9e6"
                :border-style :solid}}
  [:ul
   [:li "one"]
   [:li "two"]
   [:li "three"]]])
  • one
  • two
  • three

A dataset using Tablecloth:

(-> {:x (range 5)
     :y (repeatedly 5 rand)}
    tc/dataset
    (tc/set-dataset-name "my dataset"))

my dataset [5 2]:

:x :y
0 0.43708737
1 0.97376624
2 0.85963333
3 0.67850567
4 0.39835792

A plot using Hanamicloth:

(-> (toydata/iris-ds)
    (haclo/plot haclo/rule-chart
                {:=x :sepal_width
                 :=x2 :sepal_length
                 :=y :petal_width
                 :=y2 :petal_length
                 :=color :species
                 :=color-type :nominal
                 :=mark-size 3
                 :=mark-opacity 0.2}))

1.8 API

(require '[scicloj.clay.v2.api :as clay])

The entry point of the Clay API is the scicloj.clay.v2.api/make! function. Here are some usage examples.

Evaluate and render the namespace in "notebooks/index.clj" as HTML and show it at the browser:

(comment
  (clay/make! {:format [:html]
               :source-path "notebooks/index.clj"}))

Do the same as above by default (since :format [:html] is the default):

(comment
  (clay/make! {:source-path "notebooks/index.clj"}))

Evaluate and render the namespace in "notebooks/index.clj" as HTML and do not show it at the browser:

(comment
  (clay/make! {:source-path "notebooks/index.clj"
               :show false}))

Evaluate and render the namespace in "notebooks/index.clj" and use the favicon at "notebooks/favicon.ico"

(comment
  (clay/make! {:source-path "notebooks/index.clj"
               :favicon "notebooks/favicon.ico"}))

Evaluate and render the namespaces in "notebooks/slides.clj" "notebooks/index.clj" as HTML and do not show it at the browser:

(comment
  (clay/make! {:source-path ["notebooks/slides.clj"
                             "notebooks/index.clj"]
               :show false}))

Evaluate and render a single form in the context of the namespace in "notebooks/index.clj" as HTML and show it at the browser:

(comment
  (clay/make! {:source-path "notebooks/index.clj"
               :single-form '(kind/cytoscape
                              cytoscape-example
                              {:element/style {:width "300px"
                                               :height "300px"}})}))

Evaluate and render a single form in the context of the current namespace (*ns*) as HTML and show it at the browser:

(comment
  (clay/make! {:single-form '(kind/cytoscape
                              cytoscape-example
                              {:element/style {:width "300px"
                                               :height "300px"}})}))

Render a single value as HTML and show it at the browser:

(comment
  (clay/make! {:single-value (kind/cytoscape
                              cytoscape-example
                              {:element/style {:width "300px"
                                               :height "300px"}})}))

Evaluate and render the namespace in "notebooks/index.clj" as a Quarto qmd file then, using Quarto, render that file as HTML and show it at the browser:

(comment
  (clay/make! {:format [:quarto :html]
               :source-path "notebooks/index.clj"}))

Evaluate and render the namespace in "notebooks/index.clj" as a Quarto qmd file and show it at the browser: (note the current browser view of this format it not so sophisticated and lacks live-reload on page updates).

(comment
  (clay/make! {:format [:quarto :html]
               :source-path "notebooks/index.clj"
               :run-quarto false}))

Evaluate and render the namespace in "notebooks/slides.clj" as a Quarto qmd file (using its namespace-specific config from the ns metadata) then, using Quarto, render that file as HTML and show it at the browser:

(comment
  (clay/make! {:format [:quarto :html]
               :source-path "notebooks/slides.clj"}))

Evaluate and render the namespace in "notebooks/slides.clj" as a Quarto qmd file (using its namespace-specific config from the ns metadata) then, using Quarto, render that file as a reveal.js slideshow and show it at the browser:

(comment
  (clay/make! {:format [:quarto :revealjs]
               :source-path "notebooks/slides.clj"}))

Evaluate and render the namespace in "notebooks/index.clj" as a Quarto qmd file with a custom Quarto config then, using Quarto, render that file as HTML and show it at the browser:

(comment
  (clay/make! {:format [:quarto :html]
               :source-path "notebooks/index.clj"
               :quarto {:highlight-style :nord
                        :format {:html {:theme :journal}}}}))

Evaluate and render the namespace in "notebooks/index.clj" as a Quarto qmd file with a custom Quarto config where the higlight style is fetched from the scicloj.clay.v2.quarto.highlight-styles namespace, and the theme is fetched from the scicloj.clay.v2.quarto.themes namespace, then, using Quarto, render that file as HTML and show it at the browser:

(comment
  (require '[scicloj.clay.v2.quarto.highlight-styles :as quarto.highlight-styles]
           '[scicloj.clay.v2.quarto.themes :as quarto.themes])
  (clay/make! {:format [:quarto :html]
               :source-path "notebooks/index.clj"
               :quarto {:highlight-style quarto.highlight-styles/nord
                        :format {:html {:theme quarto.themes/journal}}}}))

Evaluate and render the namespace in "index.clj" under the "notebooks" directory as HTML and show it at the browser:

(comment
  (clay/make! {:base-source-path "notebooks/"
               :source-path "index.clj"}))

Create a Quarto book with a default generated index page:

(comment
  (clay/make! {:format [:quarto :html]
               :base-source-path "notebooks"
               :source-path ["chapter.clj"
                             "another_chapter.md"
                             "a_chapter_with_R_code.Rmd"
                             "test.ipynb"]
               :base-target-path "book"
               :book {:title "Book Example"}
               ;; Empty the target directory first:
               :clean-up-target-dir true}))

Create a Quarto book with a specified favicon:

(comment
  (clay/make! {:format [:quarto :html]
               :base-source-path "notebooks"
               :source-path ["index.clj"
                             "chapter.clj"
                             "another_chapter.md"]
               :base-target-path "book"
               :book {:title "Book Example"
                      :favicon "notebooks/favicon.ico"}
               ;; Empty the target directory first:
               :clean-up-target-dir true}))

Create a Quarto book with book parts:

(comment
  (clay/make! {:format [:quarto :html]
               :base-source-path "notebooks"
               :source-path [{:part "Part A"
                              :chapters ["index.clj"
                                         "chapter.clj"]}
                             {:part "Part B"
                              :chapters ["another_chapter.md"]}]
               :base-target-path "book"
               :book {:title "Book Example"}
               ;; Empty the target directory first:
               :clean-up-target-dir true}))

Reopen the Clay view in the browser (in case you closed the browser tab previously opened):

(comment
  (clay/browse!))

1.8.1 Hiccup output

(experimental 🛠)

Render a notebook in Hiccup format and return the resulting Hiccup structure:

(comment
  (clay/make-hiccup {:source-path "notebooks/index.clj"}))

1.9 Configuration

Calls to the make! function are affected by various parameters which collected as one nested map. This map is the result of deep-merging four sources:

  • the default configuration: clay-default.edn under Clay’s resources

  • the user configuration: clay.edn at the top

  • the namespace configuration: the :clay member of the namespace metadata

  • the call configuration: the argument to make!

Here are some of the parameters worth knowing about:

Key Purpose Example
:source-path files to render ["notebooks/index.clj"]
:title sets the HTML title that appears in the browser tab bar "My Title"
:favicon sets a page favicon "favicon.ico"
:show starts the HTML server when true (the default) false
:single-form render just one form (inc 1)
:format output quarto markdown and/or html [:quarto :html]
:quarto adds configuration for Quarto {:highlight-style :solarized}
:base-target-path the output directory "temp"
:base-source-path where to find :source-path "notebooks"
:clean-up-target-dir delete (!) target directory before repopulating it true
:remote-repo linking to source {:git-url "https://github.com/scicloj/clay" :branch "main"}

When working interactively, it is helpful to render to a temporary directory that can be git ignored and discarded. For example: you may set :base-target-path "temp" at your clay.edn file. When publishing a static page, you may wish to target a docs directory by setting :base-target-path "docs" in your call to clay/make!. Creating a dev namespace is a good way to invoke a different configuration for publishing.

1.10 Kinds

The way things should be visualized is determined by the Kindly specification.

Kindly advises tools (like Clay) about the kind of way a given context should be displayed, by assigning to it a so-called kind.

Please refer to the Kindly documentation for details about specifying and using kinds.

In this documentation we demonstrate Kindly’s default advice. User-defined Kindly advices should work as well.

1.11 Examples

See the dedicated 📖 Examples chapter 📖 of this book.

1.12 Fragments

kind/fragment is a special kind. It expects a sequential value and generates multiple items, of potentially multiple kinds, from its elements.

(->> ["purple" "darkgreen" "brown"]
     (mapcat (fn [color]
               [(kind/md (str "### subsection: " color))
                (kind/hiccup [:div {:style {:background-color color
                                            :color "lightgrey"}}
                              [:big [:p color]]])]))
     kind/fragment)

1.12.1 subsection: purple

purple

1.12.2 subsection: darkgreen

darkgreen

1.12.3 subsection: brown

brown

(->> (range 3)
     kind/fragment)
0
1
2

Importantly, markdown subsections affect the Quarto table of contents.

1.13 Functions

kind/fn is a special kind. It is displayed by first evaluating the given function and arguments, then proceeding recursively with the resulting value.

(kind/fn
  [+ 1 2])
3
(kind/fn
  {:kindly/f (fn [{:keys [x y]}]
               (+ x y))
   :x 1
   :y 2})
3
(kind/fn
  [tc/dataset
   {:x (range 3)
    :y (repeatedly 3 rand)}])

_unnamed [3 2]:

:x :y
0 0.30103443
1 0.04917767
2 0.33611156 
(kind/fn
  {:kindly/f tc/dataset
   :x (range 3)
   :y (repeatedly 3 rand)})

_unnamed [3 2]:

:x :y
0 0.55549544
1 0.61657510
2 0.46215476

1.14 Delays

Clojure Delays are a common way to define computations that do not take place immediately. The computation takes place when dereferencing the value for the first time.

Clay makes sure to dererence Delays when passing values for visualization.

This is handy for slow example snippets and explorations, that one would typically not like to slow down the evaluation of the whole namespace, but would like to visualize them on demand and also include in them in the final document.

(delay
  (Thread/sleep 500)
  (+ 1 2))
3

1.15 Referring to files

In data visualizations, one can directly refrer to files places under "notebooks/" or "src/". By default, all files except of these directories, except for Clojure files, are copied alongside the HTML target.

This default can be overridden using the :subdirs-to-sync config option. E.g., :subdirs-to-sync ["notebooks" "data"] will copy files from the "notebooks" and "data" directories, but not from "src".

(kind/hiccup
 [:img {:src "notebooks/images/Clay.svg.png"}])
(kind/vega-lite
 {:data {:url "notebooks/datasets/iris.csv"},
  :mark "rule",
  :encoding {:opacity {:value 0.2}
             :size {:value 3}
             :x {:field "sepal_width", :type "quantitative"},
             :x2 {:field "sepal_length", :type "quantitative"},
             :y {:field "petal_width", :type "quantitative"},
             :y2 {:field "petal_length", :type "quantitative"},
             :color {:field "species", :type "nominal"}}
  :background "floralwhite"})

1.16 Hiding code

By default, a Clay notebook shows both the code and the result of an evaluated form. Here are a few ways one may hide the code:

  1. Add the metadata :kindly/hide-code true to the form (e.g., by preceding it with ^:kindly/hide-code).
  2. Add the metadata :kindly/hide-code true to the value.
  3. Globally define certain kinds (e.g., :kind/md, :kind/hiccup) to always hide code (on project level or namespace level) by adding theme as a set to the project config or namespace config, e.g., :kindly/options {:kinds-that-hide-code #{:kind/md :kind/hiccup}}.

1.17 Test generation

(experimental 🛠)

(+ 1 2)
3

We generate tests checking whether this last value is greater than 2.9. We can do it in a few ways.

We include the test annotations in the markdown text, since the annotations themselves are invisible.

(kind/test-last [> 2.9])

^kind/test-last
[> 2.9]

(kindly/check > 2.9)

See the generated test/index_generated_test.clj.

For a detailed example using this mechanism, see the source of the ClojisR tutorial.

source: notebooks/index.clj