38  Waterfall Extension

This notebook walks through building a custom chart type from scratch – a waterfall chart that shows running totals as colored bars (green for increases, red for decreases).

It uses all three extension points needed for a new mark:

(ns plotje-book.waterfall-extension
  (:require
   ;; Kindly -- notebook rendering protocol
   [scicloj.kindly.v4.kind :as kind]
   ;; Plotje -- public API
   [scicloj.plotje.api :as pj]
   ;; Extension points -- the multimethods we will extend
   [scicloj.plotje.impl.stat :as stat]
   [scicloj.plotje.impl.extract :as extract]
   [scicloj.plotje.render.mark :as mark]
   ;; Layer-type registry -- to register our new chart type
   [scicloj.plotje.layer-type :as layer-type]
   ;; Membrane -- drawing primitives
   [membrane.ui :as ui]
   ;; Tablecloth -- dataset operations
   [tablecloth.api :as tc]))

Sample Data

A simple profit-and-loss waterfall: revenue flows through costs to net income.

(def pnl-data
  {:category ["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"]
   :amount   [500       -300   200            -120   -30   50]})

Step 1: The Stat

compute-stat transforms raw data into the shape the extractor needs. For a waterfall, we compute running totals and determine the start and end of each bar.

The stat must always return :x-domain and :y-domain. Beyond that, the shape is a contract between the stat and the extractor that reads it – here we use :waterfall-bars.

(defmethod stat/compute-stat :waterfall [{:keys [data x y x-type] :as draft-layer}]
  (let [clean (tc/drop-missing data [x y])
        categories (vec (distinct (clean x)))
        values (vec (clean y))
        ;; Running total: each bar starts where the previous ended
        ends (vec (reductions + values))
        starts (vec (cons 0 (butlast ends)))
        bars (mapv (fn [cat s e v]
                     {:category cat
                      :start (double s)
                      :end (double e)
                      :value (double v)})
                   categories starts ends values)
        y-min (min 0.0 (apply min (concat starts ends)))
        y-max (apply max (concat starts ends))]
    {:waterfall-bars bars
     :categories categories
     :x-domain categories
     :y-domain [y-min y-max]}))

Test the stat in isolation:

(stat/compute-stat {:stat :waterfall :data (tc/dataset pnl-data) :x :category :y :amount :x-type :categorical})
{:waterfall-bars
 [{:category "Revenue", :start 0.0, :end 500.0, :value 500.0}
  {:category "COGS", :start 500.0, :end 200.0, :value -300.0}
  {:category "Gross Profit", :start 200.0, :end 400.0, :value 200.0}
  {:category "OpEx", :start 400.0, :end 280.0, :value -120.0}
  {:category "Tax", :start 280.0, :end 250.0, :value -30.0}
  {:category "Net Income", :start 250.0, :end 300.0, :value 50.0}],
 :categories
 ["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"],
 :x-domain ["Revenue" "COGS" "Gross Profit" "OpEx" "Tax" "Net Income"],
 :y-domain [0 500]}

Step 2: The Extractor

extract-layer converts the stat output into a plan layer descriptor – a plain map with :mark, :style, and geometry data. Colors are resolved here using the library’s color system.

We color bars green for positive amounts and red for negative.

(defmethod extract/extract-layer :waterfall [draft-layer stat all-colors cfg]
  (let [bars (:waterfall-bars stat)
        green [0.2 0.7 0.3 1.0]
        red [0.85 0.25 0.25 1.0]]
    {:mark :waterfall
     :style {:opacity 0.85}
     :categories (:categories stat)
     :bars (mapv (fn [{:keys [category start end value]}]
                   {:category category
                    :start start
                    :end end
                    :color (if (>= value 0) green red)})
                 bars)}))

Step 3: The Renderer

layer->membrane turns the plan layer into membrane drawable primitives. The rendering context (ctx) provides:

  • :coord-fn – takes an :x and a :y value in data space and returns the point [px py] in drawing units, through the panel’s coordinate system, so a mark drawn through it is flipped under (pj/coord :flip)

  • :sx, :sy – the two scales, for a mark that needs a scale itself, such as the width of a band

  • :coord-type – :cartesian, :flip, :polar or :fixed

On a categorical axis a number is a place among the categories, counted from one, so bar i spans the places from i + 0.6 to i + 1.4 – four fifths of its band. The renderer gives coord-fn the four corners of each bar in data space:

(defmethod mark/layer->membrane :waterfall [layer ctx]
  (let [{:keys [bars style]} layer
        {:keys [coord-fn]} ctx
        {:keys [opacity]} style]
    (vec
     (for [[i {:keys [start end color]}] (map-indexed vector bars)
           :let [[cr cg cb ca] color
                 place (inc i)
                 left (- place 0.4)
                 right (+ place 0.4)]]
       (ui/with-color [cr cg cb (or opacity ca)]
         (ui/with-style :membrane.ui/style-fill
           (ui/path (coord-fn left start)
                    (coord-fn right start)
                    (coord-fn right end)
                    (coord-fn left end))))))))

Step 4: Register and Plot

Register the layer type, so that pj/lay accepts :waterfall:

(layer-type/register! :waterfall
                      {:mark :waterfall :stat :waterfall
                       :doc "Waterfall -- running total with increase/decrease bars."})
:waterfall

Now we can plot it using the pose API. There is no built-in pj/lay-waterfall, so pj/lay takes the registered key.

The plan and the plot are both computed here, while the extension’s methods exist; the cleanup section at the end of this notebook removes them.

(def waterfall-bars
  (-> pnl-data
      (pj/pose :category :amount)
      (pj/lay :waterfall)
      pj/plan
      :panels
      first
      :layers
      first
      :bars))
(-> pnl-data
    (pj/pose :category :amount)
    (pj/lay :waterfall)
    (pj/options {:title "Profit & Loss Waterfall"
                 :width 500 :height 350})
    pj/plot)
Profit & Loss WaterfallamountcategoryRevenueCOGSGross ProfitOpExTaxNet Income0100200300400500

Six bars – one per category. Green for positive amounts (Revenue, Gross Profit, Net Income), red for negative (COGS, OpEx, Tax).

Because the renderer places each bar through coord-fn, (pj/coord :flip) turns the waterfall on its side:

(-> pnl-data
    (pj/pose :category :amount)
    (pj/lay :waterfall)
    (pj/coord :flip)
    (pj/options {:title "Profit & Loss Waterfall"
                 :width 500 :height 350})
    pj/plot)
Profit & Loss Waterfallcategoryamount0100200300400500RevenueCOGSGross ProfitOpExTaxNet Income

Optional: Convenience Function

To make the call read like a built-in layer, wrap the pattern in a pose-compatible function:

(defn lay-waterfall
  ([pose]
   (pj/lay pose :waterfall))
  ([data x y]
   (-> data
       (pj/pose x y)
       (pj/lay :waterfall)))
  ([data x y mapping]
   (-> data
       (pj/pose x y)
       (pj/lay :waterfall mapping))))

Now the call matches any built-in layer type:

(-> pnl-data
    (lay-waterfall :category :amount)
    (pj/options {:width 500})
    pj/plot)
amountcategoryRevenueCOGSGross ProfitOpExTaxNet Income050100150200250300350400450500

Optional: Self-Documenting Extension

Register [:key :doc] defmethods so the extension appears in the generated tables in the Extensibility chapter:

(defmethod stat/compute-stat [:waterfall :doc] [_]
  "Compute running totals for waterfall bars")
(defmethod extract/extract-layer [:waterfall :doc] [_ _ _ _]
  "Colored bars (green/red), each spanning its start and end values")
(defmethod mark/layer->membrane [:waterfall :doc] [_ _]
  "Filled rectangles positioned by running total")
(pj/stat-doc :waterfall)
"Compute running totals for waterfall bars"

Cleanup

Remove the extension so it does not affect other notebooks:

(remove-method stat/compute-stat :waterfall)
#multifn[compute-stat 0x59387a02]
(remove-method stat/compute-stat [:waterfall :doc])
#multifn[compute-stat 0x59387a02]
(remove-method extract/extract-layer :waterfall)
#multifn[extract-layer 0x77e8f95d]
(remove-method extract/extract-layer [:waterfall :doc])
#multifn[extract-layer 0x77e8f95d]
(remove-method mark/layer->membrane :waterfall)
#multifn[layer->membrane 0x5982b3b6]
(remove-method mark/layer->membrane [:waterfall :doc])
#multifn[layer->membrane 0x5982b3b6]
(swap! @(resolve 'scicloj.plotje.layer-type/registry*) dissoc :waterfall)
{:smooth
 {:mark :line,
  :stat :loess,
  :accepts
  [:confidence-band
   :level
   :bootstrap-resamples
   :bandwidth
   :size
   :stroke-dash
   :dx
   :dy],
  :doc
  "Smoothed trend line -- defaults to LOESS; pass {:stat :linear-model} for OLS.",
  :column-counts #{2}},
 :tile
 {:mark :tile,
  :stat :bin2d,
  :accepts [:fill :density-2d-grid],
  :doc "Tile/heatmap -- 2D grid binning.",
  :column-counts #{2}},
 :errorbar
 {:mark :errorbar,
  :stat :identity,
  :accepts [:y-min :y-max :size :cap-width :dx :dy],
  :doc "Errorbar -- vertical error bars.",
  :column-counts #{2}},
 :histogram
 {:mark :bar,
  :stat :bin,
  :x-only true,
  :accepts [:normalize :bins :binwidth],
  :doc "Histogram -- bins numerical data into bars.",
  :column-counts #{1}},
 :ridgeline
 {:mark :ridgeline,
  :stat :violin,
  :accepts [:bandwidth :trim],
  :doc "Ridgeline -- stacked density curves per category.",
  :column-counts #{2}},
 :interval-h
 {:mark :interval-h,
  :stat :identity,
  :accepts [:x-end :interval-thickness],
  :rejects [:position],
  :doc
  "Interval -- horizontal bars from x to x-end at categorical y. For Gantt-style timelines.",
  :column-counts #{2}},
 :bar
 {:mark :rect,
  :stat nil,
  :column-counts #{1 2},
  :accepts [:bar-width],
  :doc
  "Bar -- counts categories (x only), or uses y as the bar height when a y column is given."},
 :band-h
 {:mark :band-h,
  :stat :identity,
  :accepts [:y-min :y-max],
  :rejects [:position :group :x-type :y-type :color-type],
  :doc "Horizontal shaded band between y = y-min and y = y-max.",
  :column-counts #{2}},
 :rule-h
 {:mark :rule-h,
  :stat :identity,
  :accepts [:y-intercept :stroke-dash :size],
  :rejects [:position :group :x-type :y-type :color-type],
  :doc "Horizontal reference line at y = y-intercept.",
  :column-counts #{2}},
 :rule-v
 {:mark :rule-v,
  :stat :identity,
  :accepts [:x-intercept :stroke-dash :size],
  :rejects [:position :group :x-type :y-type :color-type],
  :doc "Vertical reference line at x = x-intercept.",
  :column-counts #{2}},
 :summary
 {:mark :pointrange,
  :stat :summary,
  :accepts [:size],
  :doc "Summary -- mean +/- standard error per category.",
  :column-counts #{2}},
 :lollipop
 {:mark :lollipop,
  :stat :identity,
  :accepts [:size],
  :doc "Lollipop -- stem with dot.",
  :column-counts #{2}},
 :line
 {:mark :line,
  :stat :identity,
  :accepts [:size :stroke-dash :dx :dy],
  :doc "Line -- connects data points in order.",
  :column-counts #{2}},
 :segment
 {:mark :segment,
  :stat :identity,
  :accepts [:x-end :y-end :arrow :size :stroke-dash :dx :dy],
  :rejects [:position],
  :doc
  "Segment -- a straight line from (x, y) to (x-end, y-end), one per row, optionally with arrow heads.",
  :column-counts #{2}},
 :label
 {:mark :text,
  :stat :identity,
  :defaults {:box true},
  :accepts
  [:text
   :font-size
   :font-weight
   :font-style
   :box
   :dx
   :dy
   :align-x
   :align-y],
  :doc
  "Label -- text on a background box. The :text mark with :box preset on.",
  :column-counts #{2}},
 :band-v
 {:mark :band-v,
  :stat :identity,
  :accepts [:x-min :x-max],
  :rejects [:position :group :x-type :y-type :color-type],
  :doc "Vertical shaded band between x = x-min and x = x-max.",
  :column-counts #{2}},
 :violin
 {:mark :violin,
  :stat :violin,
  :accepts [:bandwidth :trim :size],
  :doc "Violin -- mirrored density curve per category.",
  :column-counts #{2}},
 :point
 {:mark :point,
  :stat :identity,
  :accepts [:size :shape :jitter :dx :dy],
  :varies {:size :radius, :alpha :opacity},
  :doc "Scatter -- individual data points.",
  :column-counts #{2}},
 :area
 {:mark :area,
  :stat :identity,
  :accepts [:stroke :stroke-width :stroke-dash],
  :doc "Area -- filled region under a line.",
  :column-counts #{2}},
 :contour
 {:mark :contour,
  :stat :density-2d,
  :accepts [:levels :size],
  :doc "Contour -- iso-density contour lines.",
  :column-counts #{2}},
 :density
 {:mark :area,
  :stat :density,
  :x-only true,
  :accepts [:bandwidth :trim :stroke :stroke-width :stroke-dash],
  :doc "Density -- KDE (kernel density estimation) as filled area.",
  :column-counts #{1}},
 :density-2d
 {:mark :tile,
  :stat :density-2d,
  :accepts [:density-2d-grid],
  :doc
  "2D density -- kernel density estimation (KDE) smoothed heatmap.",
  :column-counts #{2}},
 :step
 {:mark :step,
  :stat :identity,
  :accepts [:size :stroke-dash],
  :doc "Step -- horizontal-then-vertical connected points.",
  :column-counts #{2}},
 :boxplot
 {:mark :boxplot,
  :stat :boxplot,
  :accepts [:size :box-width],
  :doc "Boxplot -- median, quartiles, whiskers, outliers.",
  :column-counts #{2}},
 :rug
 {:mark :rug,
  :stat :identity,
  :x-only true,
  :accepts [:side :length],
  :doc "Rug -- axis-margin tick marks.",
  :column-counts #{1}},
 :text
 {:mark :text,
  :stat :identity,
  :accepts
  [:text
   :font-size
   :font-weight
   :font-style
   :box
   :dx
   :dy
   :align-x
   :align-y],
  :doc "Text -- data-driven labels, optionally on a background box.",
  :column-counts #{2}}}

Verify cleanup:

(nil? (layer-type/lookup :waterfall))
true

Background

What’s Next

  • Edge Cases – how the library handles unusual inputs
source: notebooks/plotje_book/waterfall_extension.clj