38 Waterfall Extension
This notebook walks through building a custom chart type from scratch – a waterfall chart that shows running totals as colored bars (green for increases, red for decreases).
It uses all three extension points needed for a new mark:
compute-stat– transform raw values into cumulative barsextract-layer– convert stat output into plan geometrylayer->membrane– render bars as membrane drawables
(ns plotje-book.waterfall-extension
(:require
;; Kindly -- notebook rendering protocol
[scicloj.kindly.v4.kind :as kind]
;; Plotje -- public API
[scicloj.plotje.api :as pj]
;; Extension points -- the multimethods we will extend
[scicloj.plotje.impl.stat :as stat]
[scicloj.plotje.impl.extract :as extract]
[scicloj.plotje.render.mark :as mark]
;; Layer-type registry -- to register our new chart type
[scicloj.plotje.layer-type :as layer-type]
;; Membrane -- drawing primitives
[membrane.ui :as ui]
;; Tablecloth -- dataset operations
[tablecloth.api :as tc]))Sample Data
A simple profit-and-loss waterfall: revenue flows through costs to net income.
(def pnl-data
{:category ["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"]
:amount [500 -300 200 -120 -30 50]})Step 1: The Stat
compute-stat transforms raw data into the shape the extractor needs. For a waterfall, we compute running totals and determine the start and end of each bar.
The stat must always return :x-domain and :y-domain. Beyond that, the shape is a contract between the stat and the extractor that reads it – here we use :waterfall-bars.
(defmethod stat/compute-stat :waterfall [{:keys [data x y x-type] :as draft-layer}]
(let [clean (tc/drop-missing data [x y])
categories (vec (distinct (clean x)))
values (vec (clean y))
;; Running total: each bar starts where the previous ended
ends (vec (reductions + values))
starts (vec (cons 0 (butlast ends)))
bars (mapv (fn [cat s e v]
{:category cat
:start (double s)
:end (double e)
:value (double v)})
categories starts ends values)
y-min (min 0.0 (apply min (concat starts ends)))
y-max (apply max (concat starts ends))]
{:waterfall-bars bars
:categories categories
:x-domain categories
:y-domain [y-min y-max]}))Test the stat in isolation:
(stat/compute-stat {:stat :waterfall :data (tc/dataset pnl-data) :x :category :y :amount :x-type :categorical}){:waterfall-bars
[{:category "Revenue", :start 0.0, :end 500.0, :value 500.0}
{:category "COGS", :start 500.0, :end 200.0, :value -300.0}
{:category "Gross Profit", :start 200.0, :end 400.0, :value 200.0}
{:category "OpEx", :start 400.0, :end 280.0, :value -120.0}
{:category "Tax", :start 280.0, :end 250.0, :value -30.0}
{:category "Net Income", :start 250.0, :end 300.0, :value 50.0}],
:categories
["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"],
:x-domain ["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"],
:y-domain [0 500]}Step 2: The Extractor
extract-layer converts the stat output into a plan layer descriptor – a plain map with :mark, :style, and geometry data. Colors are resolved here using the library’s color system.
We color bars green for positive amounts and red for negative.
(defmethod extract/extract-layer :waterfall [draft-layer stat all-colors cfg]
(let [bars (:waterfall-bars stat)
green [0.2 0.7 0.3 1.0]
red [0.85 0.25 0.25 1.0]]
{:mark :waterfall
:style {:opacity 0.85}
:categories (:categories stat)
:bars (mapv (fn [{:keys [category start end value]}]
{:category category
:start start
:end end
:color (if (>= value 0) green red)})
bars)}))Step 3: The Renderer
layer->membrane turns the plan layer into membrane drawable primitives. The rendering context (ctx) provides:
:coord-fn– takes an:xand a:yvalue in data space and returns the point[px py]in drawing units, through the panel’s coordinate system, so a mark drawn through it is flipped under(pj/coord :flip):sx,:sy– the two scales, for a mark that needs a scale itself, such as the width of a band:coord-type–:cartesian,:flip,:polaror:fixed
On a categorical axis a number is a place among the categories, counted from one, so bar i spans the places from i + 0.6 to i + 1.4 – four fifths of its band. The renderer gives coord-fn the four corners of each bar in data space:
(defmethod mark/layer->membrane :waterfall [layer ctx]
(let [{:keys [bars style]} layer
{:keys [coord-fn]} ctx
{:keys [opacity]} style]
(vec
(for [[i {:keys [start end color]}] (map-indexed vector bars)
:let [[cr cg cb ca] color
place (inc i)
left (- place 0.4)
right (+ place 0.4)]]
(ui/with-color [cr cg cb (or opacity ca)]
(ui/with-style :membrane.ui/style-fill
(ui/path (coord-fn left start)
(coord-fn right start)
(coord-fn right end)
(coord-fn left end))))))))Step 4: Register and Plot
Register the layer type, so that pj/lay accepts :waterfall:
(layer-type/register! :waterfall
{:mark :waterfall :stat :waterfall
:doc "Waterfall -- running total with increase/decrease bars."}):waterfallNow we can plot it using the pose API. There is no built-in pj/lay-waterfall, so pj/lay takes the registered key.
The plan and the plot are both computed here, while the extension’s methods exist; the cleanup section at the end of this notebook removes them.
(def waterfall-bars
(-> pnl-data
(pj/pose :category :amount)
(pj/lay :waterfall)
pj/plan
:panels
first
:layers
first
:bars))(-> pnl-data
(pj/pose :category :amount)
(pj/lay :waterfall)
(pj/options {:title "Profit & Loss Waterfall"
:width 500 :height 350})
pj/plot)Six bars – one per category. Green for positive amounts (Revenue, Gross Profit, Net Income), red for negative (COGS, OpEx, Tax).
Because the renderer places each bar through coord-fn, (pj/coord :flip) turns the waterfall on its side:
(-> pnl-data
(pj/pose :category :amount)
(pj/lay :waterfall)
(pj/coord :flip)
(pj/options {:title "Profit & Loss Waterfall"
:width 500 :height 350})
pj/plot)Optional: Convenience Function
To make the call read like a built-in layer, wrap the pattern in a pose-compatible function:
(defn lay-waterfall
([pose]
(pj/lay pose :waterfall))
([data x y]
(-> data
(pj/pose x y)
(pj/lay :waterfall)))
([data x y mapping]
(-> data
(pj/pose x y)
(pj/lay :waterfall mapping))))Now the call matches any built-in layer type:
(-> pnl-data
(lay-waterfall :category :amount)
(pj/options {:width 500})
pj/plot)Optional: Self-Documenting Extension
Register [:key :doc] defmethods so the extension appears in the generated tables in the Extensibility chapter:
(defmethod stat/compute-stat [:waterfall :doc] [_]
"Compute running totals for waterfall bars")(defmethod extract/extract-layer [:waterfall :doc] [_ _ _ _]
"Colored bars (green/red), each spanning its start and end values")(defmethod mark/layer->membrane [:waterfall :doc] [_ _]
"Filled rectangles positioned by running total")(pj/stat-doc :waterfall)"Compute running totals for waterfall bars"Cleanup
Remove the extension so it does not affect other notebooks:
(remove-method stat/compute-stat :waterfall)#multifn[compute-stat 0x59387a02](remove-method stat/compute-stat [:waterfall :doc])#multifn[compute-stat 0x59387a02](remove-method extract/extract-layer :waterfall)#multifn[extract-layer 0x77e8f95d](remove-method extract/extract-layer [:waterfall :doc])#multifn[extract-layer 0x77e8f95d](remove-method mark/layer->membrane :waterfall)#multifn[layer->membrane 0x5982b3b6](remove-method mark/layer->membrane [:waterfall :doc])#multifn[layer->membrane 0x5982b3b6](swap! @(resolve 'scicloj.plotje.layer-type/registry*) dissoc :waterfall){:smooth
{:mark :line,
:stat :loess,
:accepts
[:confidence-band
:level
:bootstrap-resamples
:bandwidth
:size
:stroke-dash
:dx
:dy],
:doc
"Smoothed trend line -- defaults to LOESS; pass {:stat :linear-model} for OLS.",
:column-counts #{2}},
:tile
{:mark :tile,
:stat :bin2d,
:accepts [:fill :density-2d-grid],
:doc "Tile/heatmap -- 2D grid binning.",
:column-counts #{2}},
:errorbar
{:mark :errorbar,
:stat :identity,
:accepts [:y-min :y-max :size :cap-width :dx :dy],
:doc "Errorbar -- vertical error bars.",
:column-counts #{2}},
:histogram
{:mark :bar,
:stat :bin,
:x-only true,
:accepts [:normalize :bins :binwidth],
:doc "Histogram -- bins numerical data into bars.",
:column-counts #{1}},
:ridgeline
{:mark :ridgeline,
:stat :violin,
:accepts [:bandwidth :trim],
:doc "Ridgeline -- stacked density curves per category.",
:column-counts #{2}},
:interval-h
{:mark :interval-h,
:stat :identity,
:accepts [:x-end :interval-thickness],
:rejects [:position],
:doc
"Interval -- horizontal bars from x to x-end at categorical y. For Gantt-style timelines.",
:column-counts #{2}},
:bar
{:mark :rect,
:stat nil,
:column-counts #{1 2},
:accepts [:bar-width],
:doc
"Bar -- counts categories (x only), or uses y as the bar height when a y column is given."},
:band-h
{:mark :band-h,
:stat :identity,
:accepts [:y-min :y-max],
:rejects [:position :group :x-type :y-type :color-type],
:doc "Horizontal shaded band between y = y-min and y = y-max.",
:column-counts #{2}},
:rule-h
{:mark :rule-h,
:stat :identity,
:accepts [:y-intercept :stroke-dash :size],
:rejects [:position :group :x-type :y-type :color-type],
:doc "Horizontal reference line at y = y-intercept.",
:column-counts #{2}},
:rule-v
{:mark :rule-v,
:stat :identity,
:accepts [:x-intercept :stroke-dash :size],
:rejects [:position :group :x-type :y-type :color-type],
:doc "Vertical reference line at x = x-intercept.",
:column-counts #{2}},
:summary
{:mark :pointrange,
:stat :summary,
:accepts [:size],
:doc "Summary -- mean +/- standard error per category.",
:column-counts #{2}},
:lollipop
{:mark :lollipop,
:stat :identity,
:accepts [:size],
:doc "Lollipop -- stem with dot.",
:column-counts #{2}},
:line
{:mark :line,
:stat :identity,
:accepts [:size :stroke-dash :dx :dy],
:doc "Line -- connects data points in order.",
:column-counts #{2}},
:segment
{:mark :segment,
:stat :identity,
:accepts [:x-end :y-end :arrow :size :stroke-dash :dx :dy],
:rejects [:position],
:doc
"Segment -- a straight line from (x, y) to (x-end, y-end), one per row, optionally with arrow heads.",
:column-counts #{2}},
:label
{:mark :text,
:stat :identity,
:defaults {:box true},
:accepts
[:text
:font-size
:font-weight
:font-style
:box
:dx
:dy
:align-x
:align-y],
:doc
"Label -- text on a background box. The :text mark with :box preset on.",
:column-counts #{2}},
:band-v
{:mark :band-v,
:stat :identity,
:accepts [:x-min :x-max],
:rejects [:position :group :x-type :y-type :color-type],
:doc "Vertical shaded band between x = x-min and x = x-max.",
:column-counts #{2}},
:violin
{:mark :violin,
:stat :violin,
:accepts [:bandwidth :trim :size],
:doc "Violin -- mirrored density curve per category.",
:column-counts #{2}},
:point
{:mark :point,
:stat :identity,
:accepts [:size :shape :jitter :dx :dy],
:varies {:size :radius, :alpha :opacity},
:doc "Scatter -- individual data points.",
:column-counts #{2}},
:area
{:mark :area,
:stat :identity,
:accepts [:stroke :stroke-width :stroke-dash],
:doc "Area -- filled region under a line.",
:column-counts #{2}},
:contour
{:mark :contour,
:stat :density-2d,
:accepts [:levels :size],
:doc "Contour -- iso-density contour lines.",
:column-counts #{2}},
:density
{:mark :area,
:stat :density,
:x-only true,
:accepts [:bandwidth :trim :stroke :stroke-width :stroke-dash],
:doc "Density -- KDE (kernel density estimation) as filled area.",
:column-counts #{1}},
:density-2d
{:mark :tile,
:stat :density-2d,
:accepts [:density-2d-grid],
:doc
"2D density -- kernel density estimation (KDE) smoothed heatmap.",
:column-counts #{2}},
:step
{:mark :step,
:stat :identity,
:accepts [:size :stroke-dash],
:doc "Step -- horizontal-then-vertical connected points.",
:column-counts #{2}},
:boxplot
{:mark :boxplot,
:stat :boxplot,
:accepts [:size :box-width],
:doc "Boxplot -- median, quartiles, whiskers, outliers.",
:column-counts #{2}},
:rug
{:mark :rug,
:stat :identity,
:x-only true,
:accepts [:side :length],
:doc "Rug -- axis-margin tick marks.",
:column-counts #{1}},
:text
{:mark :text,
:stat :identity,
:accepts
[:text
:font-size
:font-weight
:font-style
:box
:dx
:dy
:align-x
:align-y],
:doc "Text -- data-driven labels, optionally on a background box.",
:column-counts #{2}}}Verify cleanup:
(nil? (layer-type/lookup :waterfall))trueBackground
- Extensibility – reference for every extension point
- Architecture – the five-stage pipeline in detail
What’s Next
- Edge Cases – how the library handles unusual inputs