37 Extensibility
Plotje is built on multimethods – open dispatch points that let you add new marks, statistics, position adjustments and output formats without modifying the core library. The scale types and the coordinate systems are fixed sets; their multimethods are listed below to show how the built-in ones are made.
This notebook catalogs every multimethod, shows its dispatch mechanism, lists the built-in implementations, and demonstrates how to extend each one.
(ns plotje-book.extensibility
(:require
;; Rdatasets -- standard datasets
[scicloj.metamorph.ml.rdatasets :as rdatasets]
;; Kindly -- notebook rendering protocol
[scicloj.kindly.v4.kind :as kind]
;; Tablecloth and dtype-next -- dataset and buffer operations, for the
;; custom stat below
[tablecloth.api :as tc]
[tech.v3.datatype :as dtype]
[tech.v3.datatype.functional :as dfn]
;; String splitting -- for reading drawn coordinates in one check
[clojure.string :as str]
;; Plotje -- composable plotting
[scicloj.plotje.api :as pj]
;; Layer-type registry -- for inspecting layer-type data
[scicloj.plotje.layer-type :as layer-type]
;; Implementation namespaces -- for extension points
[scicloj.plotje.impl.resolve :as resolve]
[scicloj.plotje.impl.stat :as stat]
[scicloj.plotje.impl.extract :as extract]
[scicloj.plotje.impl.position :as position]
[scicloj.plotje.render.mark :as mark]
[scicloj.plotje.render.svg :as svg]
[scicloj.plotje.impl.render :as render]
;; Membrane UI protocols -- read membrane width/height
[membrane.ui]))Overview
The pipeline from pose to plot has five stages, with the transitions between them governed by multimethods or fixed- function steps. The dotted plan->plot edge is an alternative direct path – skipping the membrane stage – that backends can register against if they build their figure directly from plan data.
| Multimethod | Namespace | Dispatches on | Purpose |
|---|---|---|---|
compute-stat |
impl/stat.clj |
:stat key |
Transform data (identity, bin, count, linear-model, loess, density, boxplot) |
extract-layer |
impl/extract.clj |
:mark key |
Convert a stat result into a plan layer descriptor |
layer->membrane |
render/mark.clj |
:mark key |
Render a plan layer as membrane drawables |
mark-clip-region |
render/mark.clj |
:mark key |
Name the panel region a mark’s geometry clips to |
plan->plot |
impl/render.clj |
format keyword | Convert a plan directly into a figure (the direct path) |
membrane->plot |
impl/render.clj |
format keyword | Convert a membrane tree into a figure (the membrane path) |
make-scale |
impl/scale.clj |
domain type + spec | Build a wadogo scale – for the built-in scale types only (see below) |
make-coord |
impl/coord.clj |
coord-type keyword | Build a coordinate function – for the built-in coordinate systems only (see below) |
apply-position |
impl/position.clj |
position keyword | Adjust group layout (dodge, stack, fill) |
compute-stat
Transforms raw data into a statistical summary. Each layer type uses a stat to prepare data for rendering.
Dispatch function: (fn [draft-layer] (or (:stat draft-layer) :identity))
(kind/table
{:column-names ["Dispatch value" "What it does"]
:row-maps
(->> (methods stat/compute-stat)
keys
(filter keyword?)
(remove #{:default})
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"What it does" (pj/stat-doc k)})))})| Dispatch value | What it does |
|---|---|
|
Bin numerical values into ranges |
|
2D grid binning (heatmap counts) |
|
Five-number summary + outliers |
|
Count occurrences per category |
|
Density -- 1D kernel density estimation (KDE) |
|
Density 2D -- 2D Gaussian kernel density estimation (KDE) |
|
Pass-through -- no transform |
|
Linear model -- OLS regression line + optional confidence band |
|
LOESS (local regression) smoothing |
|
Mean +/- standard error per category |
|
KDE per category (density profile) |
A stat from end to end
The stat below computes a running maximum – for each group, the largest y seen so far, which is the shape of a record-to-date line. Everything a stat has to do is here: read the layer it is handed, return the geometry, and declare what the axes must cover.
The x column passes through untouched, because a dataset column is already a buffer the renderer can read. dfn/cummax performs the scan as one vectorized operation instead of a sequence walk. The domains come from dfn/reduce-min and dfn/reduce-max over the concatenated buffers. A stat that copied its columns into Clojure vectors on the way through would work, and would allocate a second copy of every value for nothing.
(defmethod stat/compute-stat :running-max [{:keys [data x y group]}]
(let [subsets (if (seq group)
(vals (tc/group-by data group {:result-type :as-map}))
[data])
points (mapv (fn [ds]
(cond-> {:xs (ds x)
:ys (dfn/cummax (ds y))}
;; A grouped stat names each group by the value it
;; was split on. extract-layer turns that name into
;; a color and a legend entry.
(seq group) (assoc :color (first (ds (first group))))))
subsets)
all-xs (dtype/concat-buffers (map :xs points))
all-ys (dtype/concat-buffers (map :ys points))]
{:points points
:x-domain [(dfn/reduce-min all-xs) (dfn/reduce-max all-xs)]
:y-domain [(dfn/reduce-min all-ys) (dfn/reduce-max all-ys)]}))(defmethod stat/compute-stat [:running-max :doc] [_]
"Running maximum -- the largest y seen so far")Nothing else is needed – no mark, no extractor, no renderer. pj/lay-line already draws a group of :xs and :ys.
(These examples end in pj/plot rather than leaving a pose to render itself. A pose renders lazily, and this stat is removed a few forms below; drawing now, while the method exists, keeps them reproducible.)
(def rainfall
{:month [1 2 3 4 5 6 7 8 9 10 11 12]
:rain [42 30 55 20 61 48 35 70 25 58 44 66]})(-> rainfall
(pj/lay-point :month :rain {:color "#bbbbbb"})
(pj/lay-line :month :rain {:stat :running-max})
(pj/options {:title "Rainfall and its running maximum"})
pj/plot)Grouping needs no extra work either. :group arrives already worked out, and one group in gives one line out:
(-> {:month (concat (range 1 13) (range 1 13))
:rain [42 30 55 20 61 48 35 70 25 58 44 66
10 33 21 40 18 52 29 47 60 22 38 55]
:city (concat (repeat 12 "north") (repeat 12 "south"))}
(pj/lay-line :month :rain {:stat :running-max :color :city})
(pj/options {:title "Running maximum per city"})
pj/plot)A stat is called once per panel, with that panel’s rows. Faceted by half of the year, the running maximum starts again in the second panel, at the rain of July rather than at the higher rain of May:
(-> rainfall
(assoc :half (concat (repeat 6 "January to June")
(repeat 6 "July to December")))
(pj/lay-line :month :rain {:stat :running-max})
(pj/facet :half)
pj/plot)Cleanup – remove the example stat, and confirm it is gone:
(do (remove-method stat/compute-stat :running-max)
(remove-method stat/compute-stat [:running-max :doc])
(contains? (methods stat/compute-stat) :running-max))falseWhat a stat computes
The running maximum draws values of the :rain column, so its :y axis is the axis of that column. A histogram is different: its :y axis holds a count, and no column of the data holds that count. Plotje has to tell the two apart. Panels that share an axis share it only where their axes hold the same thing, and a count of a column is not the values of that column.
A stat declares what it computes with a method on [stat :computes], beside its [stat :doc]. The method takes the layer and returns a map from an axis to a name for what the stat computes on that axis. stat/computes reads the declaration:
(stat/computes {:stat :bin}){:y :count}The method takes the layer because the options of a layer can change what one stat computes. A histogram normalized to a density computes a density:
(stat/computes {:stat :bin :normalize :density}){:y :density}A stat with no such method draws the columns the layer names, and stat/computes returns nil for it:
(stat/computes {:stat :loess})nilThe built-in stats that declare what they compute, each asked with no options:
(kind/table
{:column-names ["Stat" "Computes"]
:row-maps
(->> (methods stat/compute-stat)
keys
(filter #(and (vector? %) (= :computes (second %))))
(map first)
sort
(mapv (fn [k] {"Stat" (kind/code (pr-str k))
"Computes" (kind/code (pr-str (stat/computes {:stat k})))})))})| Stat | Computes |
|---|---|
|
|
|
|
|
|
The names are keywords the stat chooses. An extension stat that computes the share of rows at or below each value would declare a name of its own.
Pseudocode:
(defmethod stat/compute-stat [:running-share :computes] [layer]
{:y :running-share})Plotje reads the declaration in three places:
Sharing an axis. Under
:share-scales, two panels share an axis where the axes hold the same thing: the values of one column, or the same computed name from one column. A count is shared with a count, and never with a density or with the values of a column.Layers of one panel. Where one layer draws a count and another draws a density or a column on the same axis, Plotje prints a Note that the axis reads in two units.
Columns a layer needs. A layer whose stat computes
:ydraws from an:xcolumn alone.
Where a layer’s stat comes from
compute-stat dispatches on a layer’s :stat, which is settled before it is called. Three cases decide it.
This helper reports what a pose’s first layer resolves to:
(defn mark-and-stat
"The mark and stat a pose's first layer resolves to."
[pose]
(-> pose
pj/draft
:layers
first
resolve/resolve-draft-layer
(select-keys [:mark :stat])))The layer type names the stat. (layer-type/lookup :histogram) carries :stat :bin, so a histogram bins whatever it is given:
(layer-type/lookup :histogram){:mark :bar,
:stat :bin,
:x-only true,
:accepts [:normalize :bins :binwidth],
:doc "Histogram -- bins numerical data into bars.",
:column-counts #{1}}The layer type names a mark but no stat. (layer-type/lookup :bar) fixes :mark :rect and leaves :stat empty:
(layer-type/lookup :bar){:mark :rect,
:stat nil,
:column-counts #{1 2},
:accepts [:bar-width],
:doc
"Bar -- counts categories (x only), or uses y as the bar height when a y column is given."}The mark is already decided, so the column-type rules below do not run. A narrower rule fills the gap instead, and it turns on one thing only – whether a y column was given. Without one, the bar counts its categories; with one, it takes that y as the bar height:
{"lay-bar with x only" (mark-and-stat (-> (rdatasets/datasets-iris)
(pj/lay-bar :species)))
"lay-bar with x and y" (mark-and-stat (-> {:city ["north" "south"] :rain [42 30]}
(pj/lay-bar :city :rain)))}{"lay-bar with x only" {:mark :rect, :stat :count},
"lay-bar with x and y" {:mark :rect, :stat :identity}}The shapes table below lists :rect twice, because a counting bar reads one shape and a value bar reads another.
Nothing names a mark at all. When no lay-* function has chosen a layer type – a bare pj/pose, or raw data handed straight to pj/plot – both mark and stat are inferred from the column types:
{"categorical x, numerical y" (mark-and-stat (-> (rdatasets/datasets-iris)
(pj/pose :species :sepal-width)))
"numerical x, numerical y" (mark-and-stat (-> (rdatasets/datasets-iris)
(pj/pose :sepal-length :sepal-width)))
"categorical x only" (mark-and-stat (-> (rdatasets/datasets-iris)
(pj/pose :species)))
"numerical x only" (mark-and-stat (-> (rdatasets/datasets-iris)
(pj/pose :sepal-length)))}{"categorical x, numerical y" {:mark :boxplot, :stat :boxplot},
"numerical x, numerical y" {:mark :point, :stat :identity},
"categorical x only" {:mark :rect, :stat :count},
"numerical x only" {:mark :bar, :stat :bin}}An explicit {:stat ...} on a layer overrides all three, which is what makes the cross-pairings later in this section possible.
What :identity is
:identity turned up in all three cases above – named by a layer type, filled in for a mark that named none, and the value the dispatch function itself falls back to.
:identity means the data is already what should be plotted. It applies no statistical transform, but it still does the preparation every layer needs: it drops rows missing an x, a y or a mapped aesthetic value, computes the domains, and splits the rows into groups. What comes out is one entry per surviving row, at the values the dataset holds.
It is what a scatter uses:
(layer-type/lookup :point){:mark :point,
:stat :identity,
:accepts [:size :shape :jitter :dx :dy],
:varies {:size :radius, :alpha :opacity},
:doc "Scatter -- individual data points.",
:column-counts #{2}}Writing {:stat :identity} keeps the rows as they are, with no aggregation. A bar chart of pre-computed totals needs no aggregation, and pj/lay-bar resolves to :identity on its own when a y column is present.
(The other fields in those lookups belong to the layer type rather than to the stat. :column-counts is the set of numbers of columns the layer type draws from: #{1} for an x column alone, #{2} for an x and a y column, #{1 2} for either. It is listed per layer type in Layer Types, and :x-only true is the older way to write #{1}. :column-counts decides which panels of a facet over columns, and which cells of an arrangement, a layer is drawn in, whether a layer naming one column is given the panel’s other column, and whether a column named on :y alone is drawn on its side (see The panel’s columns). :accepts lists the keys its mapping takes beyond the keys every layer type accepts, tabulated per layer type in Layer Types. :varies names the appearance aesthetics its mark varies from row to row – :point declares {:size :radius :alpha :opacity}. An aesthetic left out of :varies is one the mark draws once for the whole layer, so a column mapped to it varies nothing, and planning prints a warning rather than drawing a legend for an encoding the panel does not carry.)
One entry per row is visible in the output: a scatter of the 150-row iris dataset draws 150 marks.
(def grouped-scatter
(-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width {:color :species})))(-> grouped-scatter
pj/plot
pj/svg-summary
:points)150What a stat receives
A stat is handed one layer, as a map, holding the rows of one panel: under a facet the stat runs once per panel, so a running total or a rank starts again in each panel. The map is not the draft layer pj/draft returns: planning first infers the column types, resolves each aesthetic to either a column name or a fixed value, and works out the grouping, so that a stat can read those decisions instead of repeating them. resolve/resolve-draft-layer is the step that makes them, and a stat is never called with anything less – exercising one by hand means resolving first.
(def resolved-layer
(-> grouped-scatter
pj/draft
:layers
first
resolve/resolve-draft-layer))The keys a stat reads, on that layer (:data holds the dataset itself and is left out here so the rest prints readably):
(-> resolved-layer
(select-keys [:x :y :x-type :y-type :group :color :size :alpha :fixed-color])){:y :sepal-width,
:group [:species],
:color :species,
:fixed-color nil,
:size nil,
:alpha nil,
:x :sepal-length,
:x-type :numerical,
:y-type :numerical}Reading those:
:x,:y– resolved column names:x-type,:y-type–:numericalor:categorical:group– the columns to split the data by, as a vector; empty when the layer is ungrouped. A stat that ignores this produces one group where the user asked for several.:color,:size,:alpha– a column name when that aesthetic maps a column,nilwhen it does not. A constant lives in:fixed-colorand its siblings instead, and is applied later, so a stat can ignore it.:cfg– resolved configuration, where a stat finds its own options, such as:kde-bandwidthfor the density stats
What a stat returns
Two keys are required of every stat, whatever it computes:
:x-domain– the extent the x axis must cover:[lo hi]for a numerical axis, or the sequence of categories for a categorical one:y-domain– the same for y
Alongside them the stat returns at least one geometry shape: a key whose value follows a structure that some extract-layer method reads. The shape, not the stat’s name, is what decides which marks can draw the result.
| Shape | Carries | Produced by | Read by |
|---|---|---|---|
:points |
a vector of point groups | :identity, :linear-model, :loess, :density, :summary, :count |
the point-reading marks: :point, :line, :step, :area, :text, :rug, :lollipop, :errorbar, :pointrange, :interval-h, :segment |
:bins |
histogram bins, per group | :bin |
:bar |
:bars with :categories |
a count per category, per group | :count |
:rect, when counting |
:boxes with :categories |
a five-number summary per category | :boxplot |
:boxplot |
:violins with :categories |
a density profile per category | :violin |
:violin, :ridgeline |
:tiles with :fill-range |
a grid of cells and the range of their fill values | :bin2d, :density-2d |
:tile, :contour |
:rect and :tile each read a second shape as well: both fall back to :points when their own shape is absent. For :rect that is not a special case but the two-stat rule from above – a counting bar gets :count and so reads :bars, while a value bar gets :identity and so reads :points.
A stat may also return more than one shape, when its result means something to more than one family of marks. Two built-ins do: :linear-model returns :points for the fitted line and :ribbons for its confidence band, and :count returns :bars for the bar marks and :points for the point-reading ones. Some stats return values that are neither domain nor geometry: :bin reports :max-count, and :density-2d reports the :grid it estimated over.
The grouped scatter’s stat, computed by hand:
(def scatter-stat
(-> resolved-layer
(assoc :cfg {})
stat/compute-stat))(sort (keys scatter-stat))(:points :x-domain :y-domain)The :points shape
Most stats produce :points and most marks read it. It is a vector of groups – one per combination of the grouping columns, so three species give three groups – and each group is a map of parallel sequences.
(count (:points scatter-stat))3One group, with its sequences cut to three values so that it prints readably:
(-> scatter-stat
:points
first
(update :xs #(vec (take 3 %)))
(update :ys #(vec (take 3 %)))
(update :row-indices #(vec (take 3 %)))){:xs [5.1 4.9 4.7],
:ys [3.5 3.0 3.2],
:row-indices [0 1 2],
:data-positions #tech.v3.dataset.column<int64>[50]
:__pos-idx
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19...],
:group-key "setosa",
:color "setosa"}Every group carries:
:xs,:ys– parallel sequences of data-space coordinates, one entry per row in that group. The two must be the same length as each other; groups need not be the same length as one another. Any indexed sequence works, but a numeric buffer is what the library uses and what a stat should return: dataset columns here, and the result ofdfn/cummaxin the running-max stat above. A stat with no y to report, such as:binor:count, synthesizes one rather than omitting:ys.:row-indices– where each entry sat in the original dataset, which is what lets a tooltip name the row it came from.
The rest appear only when the group has something to put in them. Most are per-row aesthetics, attached by the preparation step :identity runs from the columns the layer maps – which is why {:size :petal-length} means something on a scatter and nothing on a histogram, since a stat that aggregates its input has no per-row values left to attach. A stat that computes a value per point may fill one in itself instead, as :count does for :labels:
| Key | Present when | Read by |
|---|---|---|
:color |
the layer is grouped | every mark, to choose the group’s color |
:labels |
the layer maps :text, or the stat computes one per point |
:text |
:ymins, :ymaxs |
:y-min and :y-max map columns |
:errorbar, :pointrange |
:sizes |
:size maps a column |
:point |
:alphas |
:alpha maps a column |
:point |
:shapes |
:shape maps a column |
:point |
:color-values |
:color maps a numerical column |
:point, :interval-h |
:x-ends |
:x-end maps a column |
:interval-h, :segment |
:y-ends |
:y-end maps a column |
:segment |
:color in a stat result is a category, not a color
A group’s :color holds the category value the group was split on – the string "setosa" – and not a color. Nothing is resolved against a palette until extract-layer runs, which is where the same key changes meaning: the plan layer’s :color is a resolved color, and the category moves to :label.
[(-> scatter-stat
:points
first
:color)
(-> grouped-scatter
pj/plan
:panels
first
:layers
first
:groups
first
(select-keys [:color :label]))]["setosa"
{:color
[0.8941176470588236 0.10196078431372549 0.10980392156862745 1.0],
:label "setosa"}]Which marks read which shape
The stat and the extractor are two halves of one contract, but they are not paired one to one. Most of the mark extractors read :points, and most of those go through one shared reader, extract-xy-groups, which is why they agree about what a group means. A stat returning :points is therefore readable by any of the :points extractors, not only by the layer type the stat was written for – and an explicit {:stat ...} is how a caller says so. extract-xy-groups is internal, so a mark of your own reads the groups itself.
A density drawn as a line, rather than as the filled area pj/lay-density would give:
(-> (rdatasets/datasets-iris)
(pj/lay-line :sepal-length {:stat :density}))And a per-category mean drawn as bare points, rather than as the point-with-error-bar pj/lay-summary would give – three species, three points:
(-> (rdatasets/datasets-iris)
(pj/lay-point :species :sepal-width {:stat :summary}))The converse holds too: a mark handed a shape it does not read finds nothing to draw. Four extractors guard against this – :bar, :boxplot, :violin and :ridgeline – and raise an error naming the key the extractor reads. The :points readers do not guard, so they produce a layer with no groups, and the plot comes out missing that layer rather than failing.
To write a stat in a shape other than :points, read the built-in stat and extractor that already produce and consume it as a matched template: :bin with :bar, :boxplot with :boxplot, :bin2d with :tile.
extract-layer
Converts a stat result into a plan layer descriptor – a plain map with data-space geometry and resolved colors. This is the half of the contract that reads the shapes tabulated above, and where a category value becomes a color.
Dispatch function: (fn [draft-layer stat all-colors cfg] (:mark draft-layer))
all-colors is the vector of categories of the column mapped to :color, or nil where no column is mapped to it. (extract/resolve-color all-colors color draft-layer cfg) turns one group’s category into its color.
The map an extract-layer method returns needs :mark and :style – a map, which may be empty. The plan is checked against its schema, and a plan layer without :style makes pj/plot throw an exception with the message Plan does not conform to schema. The geometry goes under :groups, or under keys of the mark’s own. A layer takes part in a :position adjustment such as :dodge only when it carries the draft layer’s :position – see apply-position.
(kind/table
{:column-names ["Dispatch value" "Output"]
:row-maps
(->> (methods extract/extract-layer)
keys
(filter keyword?)
(remove #{:default})
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"Output" (pj/mark-doc k)})))})| Dispatch value | Output |
|---|---|
|
Filled region under a curve |
|
Horizontal shaded band between two written y values |
|
Vertical shaded band between two written x values |
|
Vertical rectangles (binned) |
|
Box-and-whisker |
|
Iso-value polylines |
|
Vertical error bar |
|
Horizontal bars from x to x-end at categorical y |
|
Connected path |
|
Stem with dot |
|
Filled circle |
|
Point with error bar |
|
Positioned rectangles |
|
Stacked density curves |
|
Axis-margin tick marks |
|
Horizontal reference line at a written y |
|
Vertical reference line at a written x |
|
Straight lines from (x, y) to (x-end, y-end) |
|
Horizontal-then-vertical path |
|
Data-driven label, optionally on a background box |
|
Grid of colored cells |
|
Mirrored density shape |
A plan layer looks like this. Starting from a familiar iris scatter:
(-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width {:color :species}))The plan layer extracted from that pose:
(let [s (-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width {:color :species})
pj/plan)
layer (first (:layers (first (:panels s))))]
layer){:mark :point,
:style {:opacity 0.75, :radius 3.0},
:alpha-scale nil,
:y-domain [2.0 4.4],
:x-domain [4.3 7.9],
:size-scale nil,
:fixed-tooltip nil,
:alpha-drawn? nil,
:groups
[{:color
[0.8941176470588236 0.10196078431372549 0.10980392156862745 1.0],
:xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[5.100, 4.900, 4.700, 4.600, 5.000, 5.400, 4.600, 5.000, 4.400, 4.900, 5.400, 4.800, 4.800, 4.300, 5.800, 5.700, 5.400, 5.100, 5.700, 5.100...],
:ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.500, 3.000, 3.200, 3.100, 3.600, 3.900, 3.400, 3.400, 2.900, 3.100, 3.700, 3.400, 3.000, 3.000, 4.000, 4.400, 3.900, 3.500, 3.800, 3.800...],
:label "setosa",
:row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19...],
:group-key "setosa"}
{:color
[0.21568627450980393 0.49411764705882355 0.7215686274509804 1.0],
:xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[7.000, 6.400, 6.900, 5.500, 6.500, 5.700, 6.300, 4.900, 6.600, 5.200, 5.000, 5.900, 6.000, 6.100, 5.600, 6.700, 5.600, 5.800, 6.200, 5.600...],
:ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.200, 3.200, 3.100, 2.300, 2.800, 2.800, 3.300, 2.400, 2.900, 2.700, 2.000, 3.000, 2.200, 2.900, 2.900, 3.100, 3.000, 2.700, 2.200, 2.500...],
:label "versicolor",
:row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69...],
:group-key "versicolor"}
{:color
[0.30196078431372547 0.6862745098039216 0.2901960784313726 1.0],
:xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[6.300, 5.800, 7.100, 6.300, 6.500, 7.600, 4.900, 7.300, 6.700, 7.200, 6.500, 6.400, 6.800, 5.700, 5.800, 6.400, 6.500, 7.700, 7.700, 6.000...],
:ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.300, 2.700, 3.000, 2.900, 3.000, 3.000, 2.500, 2.900, 2.500, 3.600, 3.200, 2.700, 3.000, 2.500, 2.800, 3.200, 3.000, 3.800, 2.600, 2.200...],
:label "virginica",
:row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119...],
:group-key "virginica"}],
:size-drawn? nil}layer->membrane
Renders a plan layer descriptor into membrane drawable primitives. This is the “membrane path” – used when the target format goes through membrane, as SVG does.
Dispatch function: (fn [layer ctx] (:mark layer))
The rendering context ctx carries, among other keys:
:coord-fn– takes an:xand a:yvalue in data space and returns the point[px py]in drawing units, through the panel’s coordinate system. A renderer that places its marks throughcoord-fnis drawn correctly under(pj/coord :flip); a renderer that calls the scales directly is not. On a categorical axis a number is a place among the categories, counted from one, socoord-fnalso places the edges of a band.:sx,:sy– the two scales themselves, for a renderer that needs a scale object, such as the width of a band:coord-type–:cartesian,:flip,:polaror:fixed
The Waterfall Extension chapter draws its bars through coord-fn and shows them flipped.
(kind/table
{:column-names ["Dispatch value" "Membrane output"]
:row-maps
(->> (methods mark/layer->membrane)
keys
(filter keyword?)
(remove #{:default})
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"Membrane output" (pj/membrane-mark-doc k)})))})| Dispatch value | Membrane output |
|---|---|
|
Closed filled polygons with baseline |
|
One filled rectangle across the panel between two written y values |
|
One filled rectangle down the panel between two written x values |
|
Filled polygons (histogram bars) |
|
Box + whiskers + median line + outlier points |
|
Stroked iso-density polylines |
|
Vertical lines with caps |
|
Filled rectangles spanning x to x-end at categorical y |
|
Stroked polylines |
|
Stems with dots at category positions |
|
Translated colored rounded-rectangles |
|
Point at mean + vertical SE line |
|
Filled polygons (categorical/value bars) |
|
Overlapping filled density curves |
|
Short stroked tick marks at axis margins |
|
One stroked line across the panel at a written y |
|
One stroked line down the panel at a written x |
|
Straight lines from (x, y) to (x-end, y-end), with optional arrow heads |
|
Stroked step polylines |
|
Translated text labels, on a rounded background box when one is asked for |
|
Translated colored rectangles (heatmap cells) |
|
Mirrored filled density polygon |
Drawing an aesthetic the layer type declares it varies
A layer type says which appearance aesthetics its mark varies from row to row under :varies, and that declaration earns the layer a legend. Drawing them is the renderer’s half, and it has to agree with the legend: both read the column’s values through the same scale, and a mark that spreads the values itself instead will draw sizes its own legend does not explain.
layer-type/aesthetic-magnitude-fn is that shared function. It turns a value from the layer’s per-row buffers into the drawn size – for the :point mark, a radius for :size and an opacity for :alpha:
(def bubble-layer
(-> {:x [1 2 3] :y [1 2 3] :n [1 4 9]}
(pj/lay-point :x :y {:size :n})
pj/plan
:panels
first
:layers
first))(let [groups (:groups bubble-layer)
radius-of (layer-type/aesthetic-magnitude-fn bubble-layer :size
(keep :sizes groups))]
(mapv radius-of [1 4 9]))[2.0 5.674234614174766 8.0]It returns identity where the mapping gives the column’s values as they stand ({:scale false}), so a renderer applies the result either way, and nil where the layer carries nothing to draw.
A layer type of your own declares :varies, and lists the same aesthetic under :accepts, so that its mapping takes the key at all:
Pseudocode:
(layer-type/register! :blob
{:mark :point
:stat :identity
:varies {:size :radius}
:accepts [:size]})Without :accepts, {:size :n} on that layer prints a warning that the key is not recognized, and is dropped. The stat has to attach the per-row values to each group, as :sizes; :identity does. A mark that declares an aesthetic and then draws no per-row values for it prints a warning at pj/plan, naming the mark and the aesthetic.
How to extend: add a new mark type
Adding a new mark, such as a :dumbbell that joins two values per row, requires methods on both extract-layer and layer->membrane.
Pseudocode:
;; 1. Extract geometry from stat result
(defmethod extract/extract-layer :dumbbell [draft-layer stat all-colors cfg]
{:mark :dumbbell
:style {:opacity 0.8}
:position (:position draft-layer)
:groups (vec (for [{:keys [color xs ys]} (:points stat)]
{:color (extract/resolve-color all-colors color
draft-layer cfg)
:label color
:xs xs
:ys ys}))})
;; 2. Render to membrane drawables, placing each point with coord-fn
(defmethod mark/layer->membrane :dumbbell [layer ctx]
(let [{:keys [coord-fn]} ctx]
...))How to extend: register a layer type and create a pose-compatible layer function
After defining compute-stat and extract-layer for your custom mark, register a layer type and create a convenience function that works with the pose API. The Waterfall Extension chapter evaluates the same steps.
Pseudocode:
;; Register the layer type
(layer-type/register! :waterfall
{:mark :waterfall
:stat :waterfall
:doc "Waterfall -- running total with increase/decrease bars."})
;; Users can then call:
;; (pj/lay pose :waterfall)
;; Or create a convenience function using lay:
(defn lay-waterfall
([pose]
(pj/lay pose :waterfall))
([data x y]
(-> data
(pj/pose x y)
(pj/lay :waterfall)))
([data x y mapping]
(-> data
(pj/pose x y)
(pj/lay :waterfall mapping))))pj/lay given the key checks the mapping’s keys against the layer type’s, and prints a warning naming a key it does not recognize. A layer type whose mapping takes a key of its own lists the key under :accepts, as in {:mark :dumbbell :stat :dumbbell :accepts [:y-end]}; without it, {:y-end :after} on that layer prints the same warning.
Users can then call (lay-waterfall data :category :amount).
If your custom mark is not one of the built-in marks, you also need a layer->membrane defmethod for the SVG renderer. Without one, the library throws an error explaining which defmethod to add.
A mark placed at a written value
Four marks – :rule-h, :rule-v, :band-h, :band-v – draw one shape at a value written on the layer rather than at values read from its rows. They are ordinary marks: they travel on a panel’s :layers in the order they were written, and they have the two defmethods this chapter has been describing.
stat/written-extent reports the extent the mark covers – the written value on the axis the mark names, and the layer’s column on the axis it spans – instead of grouping rows into :points. Its extract-layer method then reads the written value straight off the draft layer, so the plan layer carries a number rather than a group of buffers:
(let [layer (-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width)
(pj/lay-rule-h {:y-intercept 3.0})
pj/plan
:panels
first
:layers
last)]
layer){:mark :rule-h,
:style {:stroke-width 1.5, :opacity 1.0},
:color [0.2 0.2 0.2 1.0],
:y-intercept 3.0,
:y-domain [2.0 4.4],
:x-domain [4.3 7.9]}A custom mark placed the same way follows the same three steps: a stat that reports an extent, an extract-layer method that carries the written value onto the plan layer, and a layer->membrane method that draws it.
mark-clip-region
Each panel clips its marks so geometry past the domain cannot paint outside the panel. The region a mark clips to is itself an extension point: mark-clip-region (in render/mark.clj, dispatching on the :mark key) names the region, and the panel renderer resolves it to a rectangle and clips the mark to it. (The clip itself is a Membrane scissor; the Membranes chapter shows the mechanism.)
Two regions exist:
:drawing-area(the default) – the grey panel background. Data marks clip here, so geometry past the domain is masked at the plotting edge and nothing spills into the axis margin.:panel-box– the wider panel rectangle including the axis margin, for marks that draw in the margin on purpose. The built-in:rugmark uses it so its ticks are not cut at the drawing-area edge.
(mark/mark-clip-region :point):drawing-area(mark/mark-clip-region :rug):panel-boxHow to extend: a mark that draws in the margin
A custom mark that paints in the axis margin (a marginal density, an axis-edge glyph) registers a mark-clip-region defmethod returning :panel-box, so the renderer does not clip it at the drawing-area edge. Without this, a margin-drawing mark defaults to :drawing-area and is cut off.
(defmethod mark/mark-clip-region :margin-glyph [_] :panel-box)(mark/mark-clip-region :margin-glyph):panel-boxCleanup – remove the example defmethod:
(remove-method mark/mark-clip-region :margin-glyph)#multifn[mark-clip-region 0xdc9823b](contains? (methods mark/mark-clip-region) :margin-glyph)falseplan->plot
Orchestrates the full path from plan to plot. The :svg implementation goes through the membrane path: plan, then membrane, then plot. Other renderers can skip membrane and go directly from plan to their target format.
Dispatch function: (fn [plan format opts] format)
| Dispatch value | Path |
|---|---|
:svg |
plan, then membrane, then membrane->plot :svg |
:bufimg |
plan, then membrane, then membrane->plot :bufimg (raster image) |
pj/save’s :png file format is not a separate dispatch value – it routes through the :bufimg path internally and encodes the resulting BufferedImage as PNG bytes on disk. The save-side keyword names the file format; the dispatch-side keyword names the JVM render target.
Using plan->plot directly:
(def my-plan
(-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width {:color :species})
pj/plan))(first (pj/plan->plot my-plan :svg {})):svgThe same plan can be rendered to different formats:
(def my-figure (pj/plan->plot my-plan :svg {}))(vector? my-figure)trueHow to extend: add a new direct format
To add a Plotly renderer that reads plan data directly (no membrane needed), register a plan->plot defmethod.
Pseudocode:
(ns mylib.render.plotly
(:require [scicloj.plotje.impl.render :as render]))
(defmethod render/plan->plot :plotly [plan _ opts]
;; Read plan domains, layers, legend, layout
;; Build a Plotly.js spec directly -- no membrane needed
{:data (mapcat plan-layer->plotly-traces
(:layers (first (:panels plan))))
:layout {:xaxis {:title (:x-label plan)}
:yaxis {:title (:y-label plan)}}})Then users opt in by requiring the namespace, and call the format through pj/plan->plot:
(require '[mylib.render.plotly])
(-> pose
pj/plan
(pj/plan->plot :plotly {}))pj/plot always renders through the membrane, so (pj/plot pose {:format :plotly}) reports an unknown format for a format registered on plan->plot alone. pj/save writes :svg and :png only.
membrane->plot
Converts a PlotjeMembrane into a plot for a given format. This is the extensibility point for membrane-based output formats – formats that consume the same drawable tree but walk it differently. The Membranes chapter walks the PlotjeMembrane record itself and the Membrane UI protocols it implements.
Dispatch function: (fn [membrane-tree format opts] format)
| Dispatch value | Output |
|---|---|
:svg |
SVG hiccup wrapped in kind/hiccup |
:bufimg |
Java BufferedImage wrapped in kind/buffered-image (raster) |
pj/plan->membrane builds the membrane, pj/membrane->plot converts it. The record carries plan-derived dimensions as fields and the title as :plotje/title, so pj/membrane->plot reads them off the value directly:
(def my-membrane (pj/plan->membrane my-plan))(pj/membrane? my-membrane)true(membrane.ui/width my-membrane)600(first (pj/membrane->plot my-membrane :svg {})):svgThe shorter shortcut pj/membrane runs the full chain from a pose to a membrane in one call – the natural starting point for a custom backend that consumes membranes:
(def shortcut-membrane
(pj/membrane (-> (rdatasets/datasets-iris)
(pj/lay-point :sepal-length :sepal-width
{:color :species}))))(pj/membrane? shortcut-membrane)trueHow to extend: add a new membrane-based format
To add a format that reuses the membrane tree, such as Canvas or PDF, register a membrane->plot defmethod. The defmethod reads canvas dimensions via the Membrane UI protocols and the title via :plotje/title, so it operates without the plan:
Pseudocode:
(ns mylib.render.canvas
(:require [membrane.ui :as ui]
[scicloj.plotje.impl.render :as render]
[scicloj.plotje.render.membrane :as membrane]))
(defmethod render/membrane->plot :canvas [membrane-tree _ opts]
(let [w (ui/width membrane-tree)
h (ui/height membrane-tree)
title (:plotje/title membrane-tree)]
;; Walk the drawable tree (via membrane.ui/children), emit
;; canvas draw calls at the correct canvas size
(canvas-walk membrane-tree w h title)))
;; Optionally register plan->plot for users who call
;; pj/plan->plot with this format; pj/plot and pj/membrane do not
;; need it -- they go through plan->membrane and membrane->plot.
(defmethod render/plan->plot :canvas [plan _ opts]
(-> plan
(membrane/plan->membrane opts)
(render/membrane->plot :canvas opts)))make-scale
Builds a wadogo scale from a domain and a drawing-space range.
The scale types are a fixed set: pj/scale and a scale spec accept the three types below and report an error for any other, so a defmethod on make-scale for a new type is never called. This section shows how the built-in scales are made.
(kind/table
{:column-names ["Dispatch value" "Scale type"]
:row-maps
(->> (methods scicloj.plotje.impl.scale/make-scale)
keys
(filter keyword?)
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"Scale type" (pj/scale-doc k)})))})| Dispatch value | Scale type |
|---|---|
|
Band scale (one band per category) |
|
Continuous linear mapping |
|
Logarithmic mapping |
Dispatch: inferred from the domain type and scale spec. Categorical domains dispatch to :categorical. Numerical domains default to :linear, overridden to :log by (pj/scale pose :x :log).
make-coord
Builds a coordinate function that maps data-space (x, y) to drawing units (px, py).
The coordinate systems are a fixed set: pj/coord accepts the four below and reports an error for any other, so a defmethod on make-coord for a new coordinate system is never called. This section shows how the built-in coordinate systems are made.
(kind/table
{:column-names ["Dispatch value" "Behavior"]
:row-maps
(->> (methods scicloj.plotje.impl.coord/make-coord)
keys
(filter keyword?)
(remove #{:default})
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"Behavior" (pj/coord-doc k)})))})| Dispatch value | Behavior |
|---|---|
|
Standard x-right, y-up mapping |
|
Fixed aspect ratio (1 data unit = 1 data unit) |
|
Swap x and y axes |
|
Radial mapping: x->angle, y->radius |
All four use the same scales – :flip swaps which scale maps to which drawing-space axis, and :polar maps x to angle and y to radius.
make-inverse
The other direction: canvas coordinates back to data-space x and y. pj/to-data reads it, and so does anything answering which value sits under a pointer. The built-in coordinate systems that can be inverted coordinate by coordinate have a method here:
(->> (methods scicloj.plotje.impl.coord/make-inverse)
keys
(filter keyword?)
(remove #{:default})
sort
vec)[:cartesian :fixed :flip]:polar registers none, so it falls to the default, which returns nil. A panel under a coordinate system with no inverse reports :invertible? false in pj/frames, and pj/to-data throws for it rather than answering with a value that is only sometimes the one asked about.
A flipped bar chart uses :flip coordinates:
(-> (rdatasets/datasets-iris)
(pj/lay-bar :species)
(pj/coord :flip))apply-position
Adjusts the plan layers of one panel that share a :position value – :dodge places groups side by side in a band, :stack and :fill pile them. It runs after extract-layer, on the layers that method returned.
Dispatch function: (fn [position layers] position). It returns the vector of adjusted layers.
(kind/table
{:column-names ["Dispatch value" "What it does"]
:row-maps
(->> (methods position/apply-position)
keys
(filter keyword?)
(remove #{:default})
sort
(mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
"What it does" (pj/position-doc k)})))})| Dispatch value | What it does |
|---|---|
|
Shift groups side-by-side within a band |
|
Stack normalized to [0, 1] (proportions) |
|
Plot at exact data coordinates (groups overlap) |
|
Pile groups cumulatively, the first group on top |
The adjustment reads the plan layer’s own :position, so an extract-layer method copies (:position draft-layer) onto the layer it returns. :dodge then writes a :dodge-idx on each group and a :dodge-ctx on the layer, and the renderer reads the two to place each group in its share of the band. A dodged point layer:
(-> {:site ["north" "north" "south" "south"]
:yield [3 5 4 6]
:year ["2024" "2025" "2024" "2025"]}
(pj/lay-point :site :yield {:color :year :position :dodge})
pj/plan
:panels
first
:layers
first
((juxt :position :dodge-ctx #(mapv :dodge-idx (:groups %)))))[:dodge {:n-groups 2} [0 1]]Self-Documenting Extensions
Every generated dispatch table in this notebook is built by introspecting multimethod keys at render time. When you extend a multimethod, your new dispatch value automatically appears in the table. You can also register a [:key :doc] defmethod to provide a description.
Adding a documented extension
Register both the implementation and a [:key :doc] defmethod:
(defmethod stat/compute-stat :quantile [draft-layer]
{:points [] :x-domain [0 1] :y-domain [0 1]})(defmethod stat/compute-stat [:quantile :doc] [_]
"Quantile regression bands")The doc helper picks it up immediately:
(pj/stat-doc :quantile)"Quantile regression bands"Missing documentation degrades gracefully
If you skip the [:key :doc] defmethod, the table still renders – the description falls back to “(no description)” instead of throwing an error. With the doc defmethod removed:
(remove-method stat/compute-stat [:quantile :doc])#multifn[compute-stat 0x59387a02](pj/stat-doc :quantile)"(no description)"Cleanup
Remove the example extension so it does not affect other tests:
(remove-method stat/compute-stat :quantile)#multifn[compute-stat 0x59387a02](count (remove #{:default} (filter keyword? (keys (methods stat/compute-stat)))))11Summary
| To add… | Extend… |
|---|---|
| A new statistical transform | compute-stat |
| A new mark type | extract-layer + layer->membrane |
| A new output format (direct) | plan->plot |
| A new output format (membrane-based) | membrane->plot + plan->plot |
| A new position adjustment | apply-position |
Background
- Architecture – the five-stage pipeline and key libraries
What’s Next
- Waterfall Extension – a worked example that uses the extension points above to add a waterfall chart
- Edge Cases – how the library handles unusual inputs