Clay
Clay is a minimalistic Clojure tool for data visualization and literate programming, compatible with the Kindly convention.
Also see the Babashka Conf 2023 talk.
Status
This project is still alpha-stage and changing. As of October 2023, it is going through extensive refactoring, mostly seeking simpler and leaner implemenation and API. Clay is developed in parallel and in coordination with Claykind, a tool with similar goals which is build in a more thoughtful process, aiming at a more modular structure.
Goals
- Easily explore & share data visualizations and notebooks for others to easily pick & use.
- Encourage writing Kindly-compatible notes for future compatiblity with other tools.
- Flow with the REPL: encourage user interactions that flow naturally with the typical use of Clojure in editors and REPLs.
Setup
See the example project for a concrete example. To enjoy Clay’s dynamic interaction, you also need to inform it about code evaluations. This requires some setup at the your editor.
See the suggested setup for popular editors below. If your favourite editor is not supported yet, let us talk and make it work.
VSCode Calva
(coming soon)
Emacs CIDER
Please load clay.el at your Emacs config.
It offers the following functions, that you may wish to create keybindings for:
name | function |
---|---|
clay/start |
start clay if not started yet |
clay/make-ns-hmtl |
save clj buffer, render it as html, and show that in the browser view |
clay/make-ns-quarto-html |
save clj buffer, render it as quarto, render that as html, and show that in the browser view |
clay/make-ns-quarto-revealjs |
save clj buffer, render it as quarto, render that as a revealjs slideshow, and show that in the browser view |
clay/make-last-sexp |
render the last s-expression |
clay/make-defun-at-point |
render the defun-at-point |
IntelliJ Cursive
(coming soon)
API
The entry point of the Clay API is the scicloj.clay.v2.api/make!
function. Here are some usage examples.
comment
(;; Evaluate and render
;; the namespace in `"notebooks/index.clj"`
;; as HTML
;; and show it at the browser.
:format [:html]
(clay/make! {:source-path "notebooks/index.clj"})
;; Evaluate and render
;; the namespace in `"notebooks/index.clj"`
;; as HTML
;; and do not show it at the browser.
:format [:html]
(clay/make! {:source-path "notebooks/index.clj"
:show false})
;; Evaluate and render
;; the namespaces in `"notebooks/slides.clj"` `"notebooks/index.clj"`
;; as HTML
;; and do not show it at the browser.
:format [:html]
(clay/make! {:source-path ["notebooks/slides.clj"
"notebooks/index.clj"]
:show false})
;; Evaluate and render a single form
;; in the context of the namespace in `"notebooks/index.clj"`
;; as HTML
;; and show it at the browser.
:format [:html]
(clay/make! {:source-path "notebooks/index.clj"
:single-form '(kind/cytoscape
:style {:width "300px"
[{:height "300px"}}
cytoscape-example])})
;; Evaluate and render a single form
;; in the context of the current namespace (`*ns*`)
;; as HTML
;; and show it at the browser.
:format [:html]
(clay/make! {:single-form '(kind/cytoscape
:style {:width "300px"
[{:height "300px"}}
cytoscape-example])})
;; Evaluate and render
;; the namespace in `"notebooks/index.clj"`
;; as a Quarto qmd file
;; then, using Quarto, render that file as HTML
;; and show it at the browser.
:format [:quarto :html]
(clay/make! {:source-path "notebooks/index.clj"})
;; Evaluate and render
;; the namespace in `"notebooks/index.clj"`
;; as a Quarto qmd file
;; and show it at the browser.
:format [:quarto :html]
(clay/make! {:source-path "notebooks/index.clj"
:run-quarto false})
;; Evaluate and render
;; the namespace in `"notebooks/slides.clj"`
;; as a Quarto qmd file
;; (using its namespace-specific config from the ns metadata)
;; then, using Quarto, render that file as HTML
;; and show it at the browser.
:format [:quarto :html]
(clay/make! {:source-path "notebooks/slides.clj"})
;; Evaluate and render
;; the namespace in `"notebooks/slides.clj"`
;; as a Quarto qmd file
;; (using its namespace-specific config from the ns metadata)
;; then, using Quarto, render that file as a reveal.js slideshow
;; and show it at the browser.
:format [:quarto :revealjs]
(clay/make! {:source-path "notebooks/slides.clj"})
;; Evaluate and render
;; the namespace in `"notebooks/index.clj"`
;; as a Quarto qmd file
;; with a custom Quarto config
;; then, using Quarto, render that file as HTML
;; and show it at the browser.
:format [:quarto :html]
(clay/make! {:source-path "notebooks/index.clj"
:quarto {:highlight-style :nord}})
;; Evaluate and render
;; the namespace in `"index.clj"`
;; under the `"notebooks"` directory
;; as HTML
;; and show it at the browser.
:format [:html]
(clay/make! {:base-source-path "notebooks/"
:source-path "index.clj"})
;; Create a Quarto book
;; (to be documented better soon).
:format [:quarto :html]
(clay/make! {:base-source-path "notebooks"
:source-path ["index.clj"
"chapter.clj"]
:base-target-path "book"
:show false
:run-quarto false
:book {:title "Book Example"}})
,)
Configutation
Calls to the make!
function are affected by various parameters which collected as one nested map. This map is the result of merging four sources:
the default configuration: clay-default.edn under Clay’s resources
the user configuration:
clay.edn
at the topthe namespace configuration: the
:clay
member of the namespace metadatathe call configuration: the argument to
make!
Here are some of the parameters worth knowing about:
(to be documented soon)
Starting a Clay namespace
Now, we can write a namespace and play with Clay.
ns index
(:require [scicloj.clay.v2.api :as clay]
(:as kindly]
[scicloj.kindly.v4.api :as kind])) [scicloj.kindly.v4.kind
defonce memoized-slurp
(memoize slurp)) (
A few useful actions
Showing the whole namespace:
comment
("notebooks/index.clj")) (clay/show-doc!
Writing the document:
comment
(
(clay/show-doc-and-write-html!"notebooks/index.clj"
:toc? true})) {
Reopening the Clay view in the browser (in case you closed the browser tab previously opened by clay/start!
)
comment
( (clay/browse!))
These can be conveniently bound to functions and keys at your editor (to b documented soon).
Interaction
Clay responds to user evaluations by displaying the result visually.
+ 1111 2222) (
3333
In Emacs CIDER, after evaluation of a form (or a region), the browser view should show the evaluation result. In VSCode Calva, a similar effect can be achieved using the dedicated command and keybinding defined above.
Kinds
The way things should be visualized is determined by the advice of Kindly. In this namespace we demonstrate Kindly’s default advice. User-defined Kindly advices should work as well. Kindly advises tools (like Clay) about the kind of way a given context should be displayed, by assigning to it a so-called kind. Please refer to the Kindly documentation for details about specifying and using kinds.
Examples
Plain values
By default, when there is no kind information provided by Kindly, values are simply pretty-printed.
+ 4 5) (
9
str "abcd" "efgh") (
"abcdefgh"
Hiccup
Hiccup, a popular Clojure way to represent HTML, can be specified by kind:
(kind/hiccup:ul
[:li [:p "hi"]]
[:li [:big [:big [:p {:style ; https://www.htmlcsscolor.com/hex/7F5F3F
[:color "#7F5F3F"}}
{"hello"]]]]])
-
hi
-
hello
As we can see, this kind is displayed by converting Hiccup to HTML.
Reagent
(kind/reagentfn [numbers]
['(:p {:style {:background "#d4ebe9"}}
[pr-str (map inc numbers))])
(vec (range 40))]) (
From the reagent tutorial:
(kind/reagentfn []
['(let [*click-count (reagent.core/atom 0)]
(fn []
(:div
["The atom " [:code "*click-count"] " has value: "
@*click-count ". "
:input {:type "button" :value "Click me!"
[:on-click #(swap! *click-count inc)}]])))])
d3-require can be used to provide js dependencies:
(kind/reagentfn []
['(
(reagent.core/with-letnil)]
[*result (reagent.core/atom -> js/d3
(require "d3-array")
(.fn [d3-array]
(.then (reset! *result
(-> d3-array
(range 9)
(.pr-str)))))
:pre @*result]))]) [
HTML
Raw html can be represented as a kind too:
"<div style='height:40px; width:40px; background:purple'></div> ") (kind/html
(kind/html"
<svg height=100 width=100>
<circle cx=50 cy=50 r=40 stroke='purple' stroke-width=3 fill='floralwhite' />
</svg> ")
Markdown
Markdown text (a vector of strings) can be handled using a kind too.
(kind/md"This is [markdown](https://www.markdownguide.org/).")
This is markdown.
(kind/md"
[* This is [markdown](https://www.markdownguide.org/).
* *Isn't it??*"
"
* Here is **some more** markdown."])
- This is markdown.
- Isn’t it??
- Here is some more markdown.
When rendering through Quarto, LaTeX formulae are supported as well.
(kind/md"Let $x=9$. Then $$x+11=20$$")
Let \(x=9\). Then \[x+11=20\]
Images
Java BufferedImage objects are displayed as images.
import javax.imageio.ImageIO
( java.net.URL)
java.net.URL
defonce clay-image
(-> "https://upload.wikimedia.org/wikipedia/commons/2/2c/Clay-ss-2005.jpg"
(
(URL.) (ImageIO/read)))
clay-image
Plain data structures
Plain data structures (lists and sequnces, vectors, sets, maps) are pretty printed if there isn’t any value inside which needs to be displayed in special kind of way.
def people-as-maps
(->> (range 29)
(mapv (fn [i]
(:preferred-language (["clojure" "clojurescript" "babashka"]
{rand-int 3))
(:age (rand-int 100)}))))
def people-as-vectors
(->> people-as-maps
(mapv (juxt :preferred-language :age)))) (
take 5 people-as-maps) (
:preferred-language "babashka", :age 1}
({:preferred-language "clojurescript", :age 94}
{:preferred-language "clojure", :age 83}
{:preferred-language "clojure", :age 25}
{:preferred-language "clojure", :age 43}) {
take 5 people-as-vectors) (
"babashka" 1]
(["clojurescript" 94]
["clojure" 83]
["clojure" 25]
["clojure" 43]) [
->> people-as-vectors
(take 5)
(set)
"clojurescript" 94]
#{["clojure" 25]
["clojure" 43]
["clojure" 83]
["babashka" 1]} [
When something inside needs to be displayed in a special kind of way, the data structures are printed in a way that makes that clear.
def nested-structure-1
(:vector-of-numbers [2 9 -1]
{:vector-of-different-things ["hi"
(kind/hiccup:big [:big "hello"]])]
[:map-of-different-things {:markdown (kind/md ["*hi*, **hi**"])
:number 9999}
:hiccup (kind/hiccup
:big [:big "bye"]])}) [
-1 nested-structure
{
|
|
|
[
] |
|
{
} |
|
bye
|
}
Pretty printing
The :kind/pprint
kind makes sure to simply pretty-print values:
-1) (kind/pprint nested-structure
:vector-of-numbers [2 9 -1],
{:vector-of-different-things ["hi" [:big [:big "hello"]]],
:map-of-different-things {:markdown ["*hi*, **hi**"], :number 9999},
:hiccup [:big [:big "bye"]]}
Datasets
tech.ml.dataset datasets currently use the default printing of the library, Let us create such a dataset using Tablecloth.
require '[tablecloth.api :as tc]) (
nil
-> {:x (range 6)
(:y [:A :B :C :A :B :C]}
tc/dataset)
_unnamed [6 2]:
:x | :y |
---|---|
0 | :A |
1 | :B |
2 | :C |
3 | :A |
4 | :B |
5 | :C |
-> {:x [1 [2 3] 4]
(:y [:A :B :C]}
tc/dataset)
_unnamed [3 2]:
:x | :y |
---|---|
1 | :A |
[2 3] | :B |
4 | :C |
-> [{:x 1 :y 2 :z 3}
(:y 4 :z 5}]
{ tc/dataset)
_unnamed [2 3]:
:x | :y | :z |
---|---|---|
1 | 2 | 3 |
4 | 5 |
-> people-as-maps
( tc/dataset)
_unnamed [29 2]:
:preferred-language | :age |
---|---|
babashka | 1 |
clojurescript | 94 |
clojure | 83 |
clojure | 25 |
clojure | 43 |
babashka | 93 |
babashka | 15 |
clojurescript | 38 |
clojure | 91 |
clojurescript | 14 |
… | … |
clojurescript | 53 |
clojure | 97 |
babashka | 0 |
clojurescript | 92 |
clojure | 29 |
babashka | 54 |
clojure | 28 |
clojure | 23 |
clojure | 62 |
babashka | 58 |
babashka | 14 |
Tables
The :kind/table
kind can be handy for an interactive table view.
(kind/table:column-names [:preferred-language :age]
{:row-vectors people-as-vectors})
preferred-language | age |
---|---|
babashka |
1 |
clojurescript |
94 |
clojure |
83 |
clojure |
25 |
clojure |
43 |
babashka |
93 |
babashka |
15 |
clojurescript |
38 |
clojure |
91 |
clojurescript |
14 |
clojure |
77 |
babashka |
35 |
clojure |
9 |
babashka |
22 |
clojure |
63 |
babashka |
76 |
babashka |
44 |
clojure |
98 |
clojurescript |
53 |
clojure |
97 |
babashka |
0 |
clojurescript |
92 |
clojure |
29 |
babashka |
54 |
clojure |
28 |
clojure |
23 |
clojure |
62 |
babashka |
58 |
babashka |
14 |
(kind/table:column-names [:preferred-language :age]
{:row-maps people-as-maps})
preferred-language | age |
---|---|
babashka |
1 |
clojurescript |
94 |
clojure |
83 |
clojure |
25 |
clojure |
43 |
babashka |
93 |
babashka |
15 |
clojurescript |
38 |
clojure |
91 |
clojurescript |
14 |
clojure |
77 |
babashka |
35 |
clojure |
9 |
babashka |
22 |
clojure |
63 |
babashka |
76 |
babashka |
44 |
clojure |
98 |
clojurescript |
53 |
clojure |
97 |
babashka |
0 |
clojurescript |
92 |
clojure |
29 |
babashka |
54 |
clojure |
28 |
clojure |
23 |
clojure |
62 |
babashka |
58 |
babashka |
14 |
(kind/table:column-names [:preferred-language :age]
{:row-maps (take 5 people-as-maps)})
preferred-language | age |
---|---|
babashka |
1 |
clojurescript |
94 |
clojure |
83 |
clojure |
25 |
clojure |
43 |
-> people-as-maps
(
tc/dataset kind/table)
preferred-language | age |
---|---|
babashka |
1 |
clojurescript |
94 |
clojure |
83 |
clojure |
25 |
clojure |
43 |
babashka |
93 |
babashka |
15 |
clojurescript |
38 |
clojure |
91 |
clojurescript |
14 |
clojure |
77 |
babashka |
35 |
clojure |
9 |
babashka |
22 |
clojure |
63 |
babashka |
76 |
babashka |
44 |
clojure |
98 |
clojurescript |
53 |
clojure |
97 |
babashka |
0 |
clojurescript |
92 |
clojure |
29 |
babashka |
54 |
clojure |
28 |
clojure |
23 |
clojure |
62 |
babashka |
58 |
babashka |
14 |
Vega and Vega-Lite
defn vega-lite-point-plot [data]
(-> {:data {:values data},
(:mark "point"
:encoding
:size {:field "w" :type "quantitative"}
{:x {:field "x", :type "quantitative"},
:y {:field "y", :type "quantitative"},
:fill {:field "z", :type "nominal"}}}
kind/vega-lite))
defn random-data [n]
(->> (repeatedly n #(- (rand) 0.5))
(+)
(reductions fn [x y]
(map-indexed (:w (rand-int 9)
{:z (rand-int 9)
:x x
:y y}))))
defn random-vega-lite-plot [n]
(-> n
(
random-data vega-lite-point-plot))
9) (random-vega-lite-plot
Cytoscape
def cytoscape-example
(:elements {:nodes [{:data {:id "a" :parent "b"} :position {:x 215 :y 85}}
{:data {:id "b"}}
{:data {:id "c" :parent "b"} :position {:x 300 :y 85}}
{:data {:id "d"} :position {:x 215 :y 175}}
{:data {:id "e"}}
{:data {:id "f" :parent "e"} :position {:x 300 :y 175}}]
{:edges [{:data {:id "ad" :source "a" :target "d"}}
:data {:id "eb" :source "e" :target "b"}}]}
{:style [{:selector "node"
:css {:content "data(id)"
:text-valign "center"
:text-halign "center"}}
:selector "parent"
{:css {:text-valign "top"
:text-halign "center"}}
:selector "edge"
{:css {:curve-style "bezier"
:target-arrow-shape "triangle"}}]
:layout {:name "preset"
:padding 5}})
(kind/cytoscape cytoscape-example)
(kind/cytoscape:style {:width "100px"
[{:height "100px"}}
cytoscape-example])
ECharts
This example is taken from Apache ECharts’ Getting Started.
def echarts-example
(:title {:text "Echarts Example"}
{:tooltip {}
:legend {:data ["sales"]}
:xAxis {:data ["Shirts", "Cardigans", "Chiffons",
"Pants", "Heels", "Socks"]}
:yAxis {}
:series [{:name "sales"
:type "bar"
:data [5 20 36
10 10 20]}]})
(kind/echarts echarts-example)
(kind/echarts:style {:width "500px"
[{:height "200px"}}
echarts-example])
Plotly
(kind/plotly:data [{:x [0 1 3 2]
{:y [0 6 4 5]
:z [0 8 9 7]
:type :scatter3d
:mode :lines+markers
:opacity 0.5
:line {:width 5}
:marker {:size 4
:colorscale :Viridis}}]})
Leaflet
(experimental) This example was adapted from the Leaflet website.
(kind/reagent:deps [:leaflet]}
^{fn []
['(:div
[:div {:style {:height "200px"}
[:ref (fn [el]
let [m (-> js/L
(map el)
(.51.505 -0.09])
(.setView (clj->js [13))]
-> js/L
("https://tile.openstreetmap.org/{z}/{x}/{y}.png"
(.tileLayer
(clj->js:maxZoom 19
{:attribution "© <a href=\"http://www.openstreetmap.org/copyright\">OpenStreetMap</a>"}))
(.addTo m))-> js/L
(51.5 -0.09]))
(.marker (clj->js [
(.addTo m)"A pretty CSS popup.<br> Easily customizable.")
(.bindPopup (.openPopup))))}]])])
3DMol.js
Embedding a 3Dmol Viewer (original example):
(kind/reagent:deps [:three-d-mol]}
^{fn [{:keys [data-pdb]}]
['(:div {:style {:height "400px"
[:width "400px"
:position :relative}
:class "viewer_3Dmoljs"
:data-pdb data-pdb
:data-backgroundcolor "0xffffff"
:data-style "stick"
:data-ui true}])
:data-pdb "2POR"}]) {
Using 3Dmol within your code (inspired by these examples):
(kind/reagent:deps [:three-d-mol]}
^{fn [{:keys [pdb-data]}]
['(:div
[:style {:width "100%"
{:height "500px"
:position "relative"}
:ref (fn [el]
let [config (clj->js
(:backgroundColor "0xffffff"})
{
viewer (.createViewer js/$3Dmol el)]
(.setViewStyle viewer (clj->js:style "outline"}))
{"pdb")
(.addModelsAsFrames viewer pdb-data
(.addSphere viewer (clj->js:center {:x 0
{:y 0
:z 0}
:radius 5
:color "green"
:alpha 0.2}))
(.zoomTo viewer)
(.render viewer)0.8 2000)))}])
(.zoom viewer :pdb-data (memoized-slurp "https://files.rcsb.org/download/2POR.pdb")}]) {
Delays
Clojure Delays are a common way to define computations that do not take place immediately. The computation takes place when dereferencing the value for the first time. Clay makes sure to dererence Delays when passing values for visualization. This is handy for slow example snippets and explorations, that one would typically not like to slow down the evaluation of the whole namespace, but would like to visualize them on demand and also include in them in the final document.
delay
(500)
(Thread/sleep + 1 2)) (
3
Embedded Portal
:x (range 3)}) (kind/portal {
(kind/portal:img {:height 50 :width 50
[(kind/hiccup [:src "https://clojure.org/images/clojure-logo-120b.png"}])
:img {:height 50 :width 50
(kind/hiccup [:src "https://raw.githubusercontent.com/djblue/portal/fbc54632adc06c6e94a3d059c858419f0063d1cf/resources/splash.svg"}])])
(kind/portal:big [:big "a plot"]])
[(kind/hiccup [9)]) (random-vega-lite-plot
Nesting kinds in Hiccup (WIP)
(kind/hiccup:div {:style {:background "#f5f3ff"
[:border "solid"}}
:hr]
[:h3 [:code ":kind/md"]]
["*some text* **some more text**")
(kind/md
:hr]
[:h3 [:code ":kind/code"]]
["{:x (1 2 [3 4])}")
(kind/code
:hr]
[:h3 [:code "kind/dataset"]]
[:x (range 33)
(tc/dataset {:y (map inc (range 33))})
:hr]
[:h3 [:code "kind/table"]]
[
(kind/table:x (range 33)
(tc/dataset {:y (map inc (range 33))}))
:hr]
[:h3 [:code "kind/vega"]]
[9)
(random-vega-lite-plot
:hr]
[:h3 [:code "kind/reagent"]]
[
(kind/reagentfn [numbers]
['(:p {:style {:background "#d4ebe9"}}
[pr-str (map inc numbers))])
(vec (range 40))])]) (
:kind/md
some text some more text
:kind/code
{:x (1 2 [3 4])}
kind/dataset
_unnamed [33 2]:
:x | :y |
---|---|
0 | 1 |
1 | 2 |
2 | 3 |
3 | 4 |
4 | 5 |
5 | 6 |
6 | 7 |
7 | 8 |
8 | 9 |
9 | 10 |
… | … |
22 | 23 |
23 | 24 |
24 | 25 |
25 | 26 |
26 | 27 |
27 | 28 |
28 | 29 |
29 | 30 |
30 | 31 |
31 | 32 |
32 | 33 |
kind/table
datatables
x | y |
---|---|
0 |
1 |
1 |
2 |
2 |
3 |
3 |
4 |
4 |
5 |
5 |
6 |
6 |
7 |
7 |
8 |
8 |
9 |
9 |
10 |
10 |
11 |
11 |
12 |
12 |
13 |
13 |
14 |
14 |
15 |
15 |
16 |
16 |
17 |
17 |
18 |
18 |
19 |
19 |
20 |
20 |
21 |
21 |
22 |
22 |
23 |
23 |
24 |
24 |
25 |
25 |
26 |
26 |
27 |
27 |
28 |
28 |
29 |
29 |
30 |
30 |
31 |
31 |
32 |
32 |
33 |
kind/vega
kind/reagent
Nesting kinds in Tables (WIP)
(kind/table:column-names [:x :y]
{:row-vectors [[(kind/md "*some text* **some more text**")
"{:x (1 2 [3 4])}")]
(kind/code :x (range 3)
[(tc/dataset {:y (map inc (range 3))})
9)]]}) (random-vega-lite-plot
x | y | ||||||||
---|---|---|---|---|---|---|---|---|---|
some text some more text |
|
||||||||
_unnamed [3 2]:
|
|
More nesting examples
:plot (random-vega-lite-plot 9)
{:dataset (tc/dataset {:x (range 3)
:y (repeatedly 3 rand)})}
{
|
|
|
_unnamed [3 2]:
|
}
9)
[(random-vega-lite-plot :x (range 3)
(tc/dataset {:y (repeatedly 3 rand)})]
[
_unnamed [3 2]:
:x | :y |
---|---|
0 | 0.20138583 |
1 | 0.61248508 |
2 | 0.37419442 |
]
Referring to files
In data visualizations, one can directly refrer to files places under "notebooks/"
or "src/"
.
(kind/hiccup:img {:src "notebooks/images/Clay.svg.png"}]) [
(kind/vega-lite:data {:url "notebooks/datasets/iris.csv"},
{:mark "rule",
:encoding {:opacity {:value 0.2}
:size {:value 3}
:x {:field "sepal_width", :type "quantitative"},
:x2 {:field "sepal_length", :type "quantitative"},
:y {:field "petal_width", :type "quantitative"},
:y2 {:field "petal_length", :type "quantitative"},
:color {:field "species", :type "nominal"}}
:background "floralwhite"})
Coming soon
In the past, Clay used to support various data visualization libraries such as MathBox.
These have been disabled in a recent refactoring (Oct. 2023) and will be brought back soon.
source: notebooks/index.clj