29 Extensibility

Plotje is built on multimethods – open dispatch points that let you add new marks, statistics, scales, coordinate systems, and output formats without modifying the core library.

This notebook catalogs every multimethod, shows its dispatch mechanism, lists the built-in implementations, and demonstrates how to extend each one.

(ns plotje-book.extensibility
  (:require
   ;; Rdatasets -- standard datasets
   [scicloj.metamorph.ml.rdatasets :as rdatasets]
   ;; Kindly -- notebook rendering protocol
   [scicloj.kindly.v4.kind :as kind]
   ;; Plotje -- composable plotting
   [scicloj.plotje.api :as pj]
   ;; Layer-type registry -- for inspecting layer-type data
   [scicloj.plotje.layer-type :as layer-type]
   ;; Implementation namespaces -- for extension points
   [scicloj.plotje.impl.stat :as stat]
   [scicloj.plotje.impl.extract :as extract]
   [scicloj.plotje.render.mark :as mark]
   [scicloj.plotje.render.svg :as svg]
   [scicloj.plotje.impl.render :as render]))

Overview

The pipeline from data to plot has several stages, each governed by a multimethod. The pose API adds a composable front-end that resolves into the same pipeline:

pose (pj/pose, pj/lay-*, pj/options, ...)
                       |
                  pose->draft
                       v
                     draft
                       |
                  draft->plan (compute-stat, extract-layer, ...)
                       v
                     plan
                       |
                   plan->plot (orchestrates full path)
                       v
            ----------------------
          membrane path      direct path
          plan->membrane     plan->plot
               |
          membrane->plot
               v
             plot                 plot

Multimethod	Namespace	Dispatches on	Purpose
`compute-stat`	`impl/stat.clj`	`:stat` key	Transform data (identity, bin, count, lm, loess, kde, boxplot)
`extract-layer`	`impl/extract.clj`	`:mark` key	Convert a stat result into a plan layer descriptor
`layer->membrane`	`render/mark.clj`	`:mark` key	Render a plan layer as membrane drawables
`plan->plot`	`impl/render.clj`	format keyword	Orchestrate the full path from plan to plot
`membrane->plot`	`impl/render.clj`	format keyword	Convert a membrane tree into a plot
`make-scale`	`impl/scale.clj`	domain type + spec	Build a wadogo scale
`make-coord`	`impl/coord.clj`	coord-type keyword	Build a coordinate function
`apply-position`	`impl/position.clj`	position keyword	Adjust group layout (dodge, stack, fill)

`compute-stat`

Transforms raw data into a statistical summary. Each layer type uses a stat to prepare data for rendering.

Dispatch function: (fn [draft-layer] (or (:stat draft-layer) :identity))

(kind/table
 {:column-names ["Dispatch value" "What it does"]
  :row-maps
  (->> (methods stat/compute-stat)
       keys
       (filter keyword?)
       (remove #{:default})
       sort
       (mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
                      "What it does" (pj/stat-doc k)})))})

Dispatch value	What it does
`:bin`	Bin numerical values into ranges
`:bin2d`	2D grid binning (heatmap counts)
`:boxplot`	Five-number summary + outliers
`:count`	Count occurrences per category
`:density`	Density — 1D kernel density estimation (KDE)
`:density-2d`	Density 2D — 2D Gaussian kernel density estimation (KDE)
`:identity`	Pass-through — no transform
`:linear-model`	Linear model — OLS regression line + optional confidence band
`:loess`	LOESS (local regression) smoothing
`:summary`	Mean ± standard error per category
`:violin`	KDE per category (density profile)

The stat is part of the layer type returned by layer-type/lookup. For example, (layer-type/lookup :histogram) returns a layer type with :stat :bin:

(layer-type/lookup :histogram)

{:mark :bar,
 :stat :bin,
 :x-only true,
 :accepts [:normalize :bins :binwidth],
 :doc "Histogram — bins numerical data into bars."}

(layer-type/lookup :bar) returns a layer type with :stat :count:

(layer-type/lookup :bar)

{:mark :rect,
 :stat :count,
 :x-only true,
 :accepts [],
 :doc "Bar — counts categorical values."}

(layer-type/lookup :point) returns a layer type with :stat :identity:

(layer-type/lookup :point)

{:mark :point,
 :stat :identity,
 :accepts [:size :shape :jitter :text :nudge-x :nudge-y],
 :doc "Scatter — individual data points."}

How to extend: add a new stat

To add a new statistical transform (e.g., :loess for local regression), define a new defmethod.

Pseudocode:

(defmethod stat/compute-stat :loess [draft-layer]
  ;; Compute LOESS smoothing from draft-layer's :data, :x, :y
  ;; Return {:points [...] :x-domain [...] :y-domain [...]}
  ...)

The return value must always include :x-domain and :y-domain. The rest of the shape depends on what the paired extract-layer expects – the stat and extractor are a matched pair. For point-like marks, return :points (groups of :xs, :ys). For other marks, study a similar existing pair as a template:

:identity returns {:points [...] :x-domain [...] :y-domain [...]}
:bin returns {:bins [...] :max-count ... :x-domain [...] :y-domain [...]}
:boxplot returns {:boxes [...] :categories [...] :x-domain [...] :y-domain [...]}

`extract-layer`

Converts a stat result into a plan layer descriptor – a plain map with data-space geometry and resolved colors.

Dispatch function: (fn [draft-layer stat all-colors cfg] (:mark draft-layer))

(kind/table
 {:column-names ["Dispatch value" "Output"]
  :row-maps
  (->> (methods extract/extract-layer)
       keys
       (filter keyword?)
       (remove #{:default})
       sort
       (mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
                      "Output" (pj/mark-doc k)})))})

Dispatch value	Output
`:area`	Filled region under a curve
`:bar`	Vertical rectangles (binned)
`:boxplot`	Box-and-whisker
`:contour`	Iso-value polylines
`:errorbar`	Vertical error bar
`:interval-h`	Horizontal bars from x to x-end at categorical y
`:label`	Label with background box
`:line`	Connected path
`:lollipop`	Stem with dot
`:point`	Filled circle
`:pointrange`	Point with error bar
`:rect`	Positioned rectangles
`:ridgeline`	Stacked density curves
`:rug`	Axis-margin tick marks
`:step`	Horizontal-then-vertical path
`:text`	Data-driven label
`:tile`	Grid of colored cells
`:violin`	Mirrored density shape

A plan layer looks like this:

(let [s (-> (rdatasets/datasets-iris)
            (pj/lay-point :sepal-length :sepal-width {:color :species})
            pj/plan)
      layer (first (:layers (first (:panels s))))]
  layer)

{:mark :point,
 :style {:opacity 0.75, :radius 3.0},
 :size-scale nil,
 :alpha-scale nil,
 :groups
 [{:color
   [0.8941176470588236 0.10196078431372549 0.10980392156862745 1.0],
   :xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[5.100, 4.900, 4.700, 4.600, 5.000, 5.400, 4.600, 5.000, 4.400, 4.900, 5.400, 4.800, 4.800, 4.300, 5.800, 5.700, 5.400, 5.100, 5.700, 5.100...],
   :ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.500, 3.000, 3.200, 3.100, 3.600, 3.900, 3.400, 3.400, 2.900, 3.100, 3.700, 3.400, 3.000, 3.000, 4.000, 4.400, 3.900, 3.500, 3.800, 3.800...],
   :label "setosa",
   :row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19...]}
  {:color
   [0.21568627450980393 0.49411764705882355 0.7215686274509804 1.0],
   :xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[7.000, 6.400, 6.900, 5.500, 6.500, 5.700, 6.300, 4.900, 6.600, 5.200, 5.000, 5.900, 6.000, 6.100, 5.600, 6.700, 5.600, 5.800, 6.200, 5.600...],
   :ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.200, 3.200, 3.100, 2.300, 2.800, 2.800, 3.300, 2.400, 2.900, 2.700, 2.000, 3.000, 2.200, 2.900, 2.900, 3.100, 3.000, 2.700, 2.200, 2.500...],
   :label "versicolor",
   :row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69...]}
  {:color
   [0.30196078431372547 0.6862745098039216 0.2901960784313726 1.0],
   :xs #tech.v3.dataset.column<float64>[50]
:sepal-length
[6.300, 5.800, 7.100, 6.300, 6.500, 7.600, 4.900, 7.300, 6.700, 7.200, 6.500, 6.400, 6.800, 5.700, 5.800, 6.400, 6.500, 7.700, 7.700, 6.000...],
   :ys #tech.v3.dataset.column<float64>[50]
:sepal-width
[3.300, 2.700, 3.000, 2.900, 3.000, 3.000, 2.500, 2.900, 2.500, 3.600, 3.200, 2.700, 3.000, 2.500, 2.800, 3.200, 3.000, 3.800, 2.600, 2.200...],
   :label "virginica",
   :row-indices #tech.v3.dataset.column<int64>[50]
:__row-idx
[100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119...]}],
 :y-domain [2.0 4.4],
 :x-domain [4.3 7.9]}

`layer->membrane`

Renders a plan layer descriptor into membrane drawable primitives. This is the “membrane path” – used when the target format goes through membrane (e.g., SVG).

Dispatch function: (fn [layer ctx] (:mark layer))

(kind/table
 {:column-names ["Dispatch value" "Membrane output"]
  :row-maps
  (->> (methods mark/layer->membrane)
       keys
       (filter keyword?)
       (remove #{:default})
       sort
       (mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
                      "Membrane output" (pj/membrane-mark-doc k)})))})

Dispatch value	Membrane output
`:area`	Closed filled polygons with baseline
`:bar`	Filled polygons (histogram bars)
`:boxplot`	Box + whiskers + median line + outlier points
`:contour`	Stroked iso-density polylines
`:errorbar`	Vertical lines with caps
`:interval-h`	Filled rectangles spanning x to x-end at categorical y
`:label`	Text label with filled background box
`:line`	Stroked polylines
`:lollipop`	Stems with dots at category positions
`:point`	Translated colored rounded-rectangles
`:pointrange`	Point at mean + vertical SE line
`:rect`	Filled polygons (categorical/value bars)
`:ridgeline`	Overlapping filled density curves
`:rug`	Short stroked tick marks at axis margins
`:step`	Stroked step polylines
`:text`	Translated text labels
`:tile`	Translated colored rectangles (heatmap cells)
`:violin`	Mirrored filled density polygon

How to extend: add a new mark type

Adding a new mark (e.g., :area for area charts) requires methods on both extract-layer and layer->membrane.

Pseudocode:

;; 1. Extract geometry from stat result
(defmethod extract/extract-layer :area [draft-layer stat all-colors cfg]
  {:mark :area
   :style {:opacity 0.5}
   :groups (vec (for [{:keys [color xs ys]} (:points stat)]
                  {:color (extract/resolve-color ...)
                   :xs xs :ys ys}))})

;; 2. Render to membrane drawables
(defmethod mark/layer->membrane :area [layer ctx]
  ;; Build filled polygon from xs/ys + baseline
  ...)

How to extend: register a layer type and create a pose-compatible layer function

After defining compute-stat and extract-layer for your custom mark, register a layer type and create a convenience function that works with the pose API:

;; Register the layer type
(layer-type/register! :waterfall
  {:mark :waterfall :stat :waterfall
   :doc "Waterfall -- running total with increase/decrease bars."})

;; Users can then call:
;; (pj/lay data (layer-type/lookup :waterfall))

;; Or create a convenience function using lay:
(defn lay-waterfall
  ([pose] (pj/lay pose (layer-type/lookup :waterfall)))
  ([data x y] (-> data (pj/pose x y) (pj/lay (layer-type/lookup :waterfall))))
  ([data x y opts] (-> data (pj/pose x y) (pj/lay (merge (layer-type/lookup :waterfall) opts)))))

Users can then call (lay-waterfall data :category :amount).

Note: if your custom mark is not one of the built-in marks, you also need a layer->membrane defmethod for the SVG renderer. Without one, the library throws an error explaining which defmethod to add.

Annotation marks live on the panel’s `:annotations`, not `:layers`

The four annotation marks – :rule-h, :rule-v, :band-h, :band-v – are split out of the per-panel :layers list during planning and rendered from a separate :annotations slot on each panel of the resolved plan. Extension authors building tooling that walks a plan should expect to find these on panel :annotations, not on panel :layers. The split is driven by scicloj.plotje.impl.resolve/annotation-marks, the canonical set of annotation mark keywords; if you add a custom annotation-style mark that should follow the same lifecycle, register it there.

`plan->plot`

Orchestrates the full path from plan to plot. The :svg implementation goes through the membrane path: plan, then membrane, then plot. Other renderers can skip membrane and go directly from plan to their target format.

Dispatch function: (fn [plan format opts] format)

Dispatch value	Path
`:svg`	plan, then membrane, then `membrane->plot :svg`
`:bufimg`	plan, then membrane, then `membrane->plot :bufimg` (raster image)

Using plan->plot directly:

(def my-plan
  (-> (rdatasets/datasets-iris)
      (pj/lay-point :sepal-length :sepal-width {:color :species})
      pj/plan))

(first (pj/plan->plot my-plan :svg {}))

:svg

The same plan can be rendered to different formats:

(def my-figure (pj/plan->plot my-plan :svg {}))

(vector? my-figure)

true

How to extend: add a new direct format

To add a Plotly renderer that reads plan data directly (no membrane needed), register a plan->plot defmethod.

Pseudocode:

(ns mylib.render.plotly
  (:require [scicloj.plotje.impl.render :as render]))

(defmethod render/plan->plot :plotly [plan _ opts]
  ;; Read plan domains, layers, legend, layout
  ;; Build a Plotly.js spec directly -- no membrane needed
  {:data (mapcat plan-layer->plotly-traces
                 (:layers (first (:panels plan))))
   :layout {:xaxis {:title (:x-label plan)}
            :yaxis {:title (:y-label plan)}}})

Then users opt in by requiring the namespace:

(require '[mylib.render.plotly])
(pj/plot pose {:format :plotly})

`membrane->plot`

Converts a membrane drawable tree into a plot for a given format. This is the extensibility point for membrane-based output formats – formats that share the same drawable tree but walk it differently.

Dispatch function: (fn [membrane-tree format opts] format)

Dispatch value	Output
`:svg`	SVG hiccup wrapped in `kind/hiccup`
`:bufimg`	Java BufferedImage wrapped in `kind/buffered-image` (raster)

pj/plan->membrane builds the tree, pj/membrane->plot converts it:

(def my-membrane (pj/plan->membrane my-plan))

(vector? my-membrane)

true

(first (pj/membrane->plot my-membrane :svg
                          {:total-width (:total-width my-plan)
                           :total-height (:total-height my-plan)}))

:svg

How to extend: add a new membrane-based format

To add a format that reuses the membrane tree (e.g., Canvas, PDF), register a membrane->plot defmethod.

Pseudocode:

(ns mylib.render.canvas
  (:require [scicloj.plotje.impl.render :as render]
            [scicloj.plotje.render.membrane :as membrane]))

(defmethod render/membrane->plot :canvas [membrane-tree _ opts]
  ;; Walk the same drawable tree, emit canvas draw calls
  (canvas-walk membrane-tree))

;; Also register plan->plot to orchestrate the full path:
(defmethod render/plan->plot :canvas [plan _ opts]
  (let [mt (membrane/plan->membrane plan)]
    (render/membrane->plot mt :canvas
                              {:total-width (:total-width plan)
                               :total-height (:total-height plan)})))

`make-scale`

Builds a wadogo scale from a domain and pixel range.

(kind/table
 {:column-names ["Dispatch value" "Scale type"]
  :row-maps
  (->> (methods scicloj.plotje.impl.scale/make-scale)
       keys
       (filter keyword?)
       sort
       (mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
                      "Scale type" (pj/scale-doc k)})))})

Dispatch value	Scale type
`:categorical`	Band scale (one band per category)
`:linear`	Continuous linear mapping
`:log`	Logarithmic mapping

Dispatch: inferred from the domain type and scale spec. Categorical domains dispatch to :categorical. Numerical domains default to :linear, overridden to :log by (pj/scale pose :x :log).

`make-coord`

Builds a coordinate function that maps data-space (x, y) to drawing units (px, py).

(kind/table
 {:column-names ["Dispatch value" "Behavior"]
  :row-maps
  (->> (methods scicloj.plotje.impl.coord/make-coord)
       keys
       (filter keyword?)
       (remove #{:default})
       sort
       (mapv (fn [k] {"Dispatch value" (kind/code (pr-str k))
                      "Behavior" (pj/coord-doc k)})))})

Dispatch value	Behavior
`:cartesian`	Standard x-right, y-up mapping
`:fixed`	Fixed aspect ratio (1 data unit = 1 data unit)
`:flip`	Swap x and y axes
`:polar`	Radial mapping: x→angle, y→radius

All four use the same scales – :flip swaps which scale maps to which pixel axis, and :polar maps x to angle and y to radius.

A flipped bar chart uses :flip coordinates:

(-> (rdatasets/datasets-iris)
    (pj/lay-bar :species)
    (pj/coord :flip))

Self-Documenting Extensions

Every generated dispatch table in this notebook is built by introspecting multimethod keys at render time. When you extend a multimethod, your new dispatch value automatically appears in the table. You can also register a [:key :doc] defmethod to provide a description.

Adding a documented extension

(defmethod stat/compute-stat :quantile [draft-layer]
  {:points [] :x-domain [0 1] :y-domain [0 1]})

(defmethod stat/compute-stat [:quantile :doc] [_]
  "Quantile regression bands")

The doc helper picks it up immediately:

(pj/stat-doc :quantile)

"Quantile regression bands"

Missing documentation degrades gracefully

If you skip the [:key :doc] defmethod, the table still renders – the description falls back to “(no description)” instead of throwing an error. Let us remove the doc defmethod and verify:

(remove-method stat/compute-stat [:quantile :doc])

#object[clojure.lang.MultiFn 0x5f2b7bdf "clojure.lang.MultiFn@5f2b7bdf"]

(pj/stat-doc :quantile)

"(no description)"

Cleanup

Remove the example extension so it does not affect other tests:

(remove-method stat/compute-stat :quantile)

#object[clojure.lang.MultiFn 0x5f2b7bdf "clojure.lang.MultiFn@5f2b7bdf"]

(count (remove #{:default} (filter keyword? (keys (methods stat/compute-stat)))))

Summary

To add…	Extend…
A new statistical transform	`compute-stat`
A new mark type	`extract-layer` + `layer->membrane`
A new output format (direct)	`plan->plot`
A new output format (membrane-based)	`membrane->plot` + `plan->plot`
A new scale type	`make-scale`
A new coordinate system	`make-coord`
A new position adjustment	`apply-position`

Background

Architecture – the five-stage pipeline and key libraries

What’s Next

Waterfall Extension – a worked example that uses the extension points above to add a new chart type
Edge Cases – how the library handles unusual inputs

source: notebooks/plotje_book/extensibility.clj

Overview

compute-stat

How to extend: add a new stat

extract-layer

layer->membrane

How to extend: add a new mark type

How to extend: register a layer type and create a pose-compatible layer function

Annotation marks live on the panel’s :annotations, not :layers

plan->plot

How to extend: add a new direct format

membrane->plot

How to extend: add a new membrane-based format

make-scale

make-coord

Self-Documenting Extensions

Adding a documented extension

Missing documentation degrades gracefully

Cleanup

Summary

Background

What’s Next

`compute-stat`

`extract-layer`

`layer->membrane`

Annotation marks live on the panel’s `:annotations`, not `:layers`

`plan->plot`

`membrane->plot`

`make-scale`

`make-coord`