29  Troubleshooting

Common mistakes and how to fix them.

(ns plotje-book.troubleshooting
  (:require
   ;; Rdatasets -- standard datasets
   [scicloj.metamorph.ml.rdatasets :as rdatasets]
   ;; Kindly -- notebook rendering protocol
   [scicloj.kindly.v4.kind :as kind]
   ;; Tablecloth -- dataset manipulation
   [tablecloth.api :as tc]
   ;; Plotje -- composable plotting
   [scicloj.plotje.api :as pj]))

Column Not Found

Symptom: "Column :foo (from :x) not found in dataset" error, listing the available columns at the end of the message.

Cause: The column reference does not match a dataset column name. Matching is strict – :foo matches keyword column :foo only, and "foo" matches string column "foo" only. The two forms do not interchange. Three common triggers:

1. Typo. A misspelled column name. Check the spelling against the dataset’s columns:

(tc/column-names (rdatasets/datasets-iris))
(:rownames
 :sepal-length
 :sepal-width
 :petal-length
 :petal-width
 :species)

2. Keyword or string. A CSV loaded without :key-fn keyword produces string column names; using a keyword reference against that dataset throws:

(try
  (-> (tc/dataset {"sepal_length" [5.0 6.0] "sepal_width" [3.0 3.5]})
      (pj/pose :sepal_length :sepal_width)
      pj/lay-point
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Column :sepal_width (from :y) not found in dataset. Available: (\"sepal_length\" \"sepal_width\") To place the mark at :sepal_width itself rather than read a column, write {:y {:value :sepal_width}}."

The fix is to either pass {:key-fn keyword} when loading the CSV (so the dataset has keyword columns) or to use string references everywhere.

3. Whitespace or punctuation mismatch. A column literally named "sepal length" (with a space) does not match :sepal-length (with a hyphen):

(try
  (-> (tc/dataset {"sepal length" [5.0 6.0] "sepal width" [3.0 3.5]})
      (pj/pose :sepal-length :sepal-width)
      pj/lay-point
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Column :sepal-width (from :y) not found in dataset. Available: (\"sepal length\" \"sepal width\") To place the mark at :sepal-width itself rather than read a column, write {:y {:value :sepal-width}}."

:key-fn keyword on "sepal length" produces :sepal length – a keyword whose printed form contains a space, not the hyphenated form a Clojure reader would normally produce. Spaces and other special characters in CSV headers usually need a custom :key-fn, such as (comp keyword #(clojure.string/replace % " " "-")).

Wrong Chart Type from Inference

Symptom: pj/pose produces a chart type that isn’t what you wanted – a boxplot when you wanted individual points, a line when you wanted a scatter.

Cause: pj/pose infers the layer type from column types. The defaults fit the most common use case for each column-type pair (see Inference Rules), but they can be overridden.

Fix: Use an explicit pj/lay-* function. For example, a categorical x with a numerical y defaults to a boxplot:

(-> (rdatasets/datasets-iris)
    (pj/pose :species :sepal-width))
sepal widthspeciessetosaversicolorvirginica2.02.53.03.54.04.5

Use pj/lay-point if you want the individual points instead:

(-> (rdatasets/datasets-iris)
    (pj/lay-point :species :sepal-width))
sepal widthspeciessetosaversicolorvirginica2.02.53.03.54.04.5

Numeric IDs Treated as Continuous Color

Symptom: You color by a subject/group ID column that contains numbers such as 1, 2 and 3, but instead of discrete colored groups you get a single continuous gradient.

Cause: Inference treats a numeric column as continuous. Continuous color means no grouping – all data stays in one group with a gradient legend.

(def subject-scores
  {:day     [1 2 3 4 1 2 3 4 1 2 3 4]
   :score   [3 5 4 6 6 7 5 8 8 9 7 10]
   :subject [1 1 1 1 2 2 2 2 3 3 3 3]})

Gradient (wrong for IDs) – one line for the whole dataset, drawn in the default color rather than in any color from the gradient: a line passes through rows of twelve different values and can take none of them. The legend still shows the gradient, and Plotje reports at the REPL that the column was not used, pointing at the fix:

(-> subject-scores
    (pj/lay-line :day :score {:color :subject}))
scoredaysubject1231234345678910

Fix: Add :color-type :categorical to override the inference – three discrete groups, one line per subject:

(-> subject-scores
    (pj/lay-line :day :score {:color :subject :color-type :categorical}))
scoredaysubject1231234345678910

See Inference Rules for the full mechanism.

Numeric Column Treated as Continuous Instead of Categorical

Symptom: A column of discrete numbers (hour of day, year, subject ID) is treated as a continuous axis. A categorical-axis mark like :boxplot, :violin, or :lollipop rejects it with an error like "requires a categorical column".

A pj/lay-bar given a numeric :x column and a value column draws a bar at each value and reports nothing; a counting pj/lay-bar given one numeric column reports the error. The fix below gives evenly spaced bands in both cases.

Cause: The column contains numbers, so column-type inference classifies it as :numerical. These marks need :categorical.

A boxplot keyed by hour runs into this – :hour looks like integers, so it is inferred numerical:

(try
  (-> {:hour [9 9 10 10 11 11] :value [1 2 3 4 5 6]}
      (pj/lay-boxplot :hour :value)
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-boxplot requires a categorical column on either :x or :y, but :hour is numerical and :value is numerical. Override with {:x-type :categorical} or {:y-type :categorical} to treat a numeric column as categorical."

Fix: Add :x-type :categorical (or :y-type :categorical for horizontal layouts) to override the inferred type. No need to convert the column itself:

(-> {:hour [9 9 10 10 11 11] :value [1 2 3 4 5 6]}
    (pj/lay-boxplot :hour :value {:x-type :categorical}))
valuehour91011123456

The override propagates into infer-column-types, so every downstream step (scale type, tick placement, domain) treats :hour as categorical. The same switch works for :y-type when a numeric column is on the y axis of a horizontal boxplot or similar layout. See Inference Rules for a worked example.

A Number Where a Category Was Meant

Symptom: A value written for a categorical axis reports "pj/plan got 2021 for :x, which is past the ends of this axis".

Cause: A number written for a categorical axis is a place among the categories rather than one of them, counted from one. On an axis built from the years 2020, 2021 and 2022 the number 2021 is a place far past the third and last category, which is what the error names. Placing Marks teaches the reading in full.

(try
  (-> {:cohort [2020 2021 2022] :n [3 5 4]}
      (pj/lay-bar :cohort :n {:x-type :categorical})
      (pj/lay-text {:x 2021 :y 5.5 :text "the 2021 cohort"})
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/plan got 2021 for :x, which is past the ends of this axis. Categories: [\"2020\" \"2021\" \"2022\"]. A number written for a categorical axis is a place counted from one -- 1 is the first category, 1.5 sits halfway to the second -- and this axis runs from 0.5 to 3.5. To move a mark by a distance on the page instead, use :offset-x / :offset-y, which work on any axis; to draw the number as a category, give the axis a column of them."

Fix: Name the category as a value. {:x {:value "2021"}} puts the mark on that band whatever numbers the categories are written from:

(-> {:cohort [2020 2021 2022] :n [3 5 4]}
    (pj/lay-bar :cohort :n {:x-type :categorical :color "#a6cee3"})
    (pj/lay-text {:x {:value "2021"} :y 4.5 :align-x :center
                  :text "on the band"}))
ncohorton the band2020202120220.00.51.01.52.02.53.03.54.04.55.0

Moving a Mark Clear of Another

Symptom: A value label sits on top of the bar it names, and the amount that separates them has to be guessed again on every chart.

Cause: A :dx or :dy is a distance in the data – a fraction of a band on a categorical axis, a value on a numeric one – so the number that clears a mark depends on what the axis holds. A label has to clear its mark by roughly the mark’s own size, which is a length on the page.

Fix: Use :offset-x/:offset-y, which are drawing units applied after the scales and so mean the same on every axis:

(-> {:species ["setosa" "versicolor" "virginica"] :pct [33.3 33.3 33.3]}
    (pj/lay-bar :species :pct)
    (pj/lay-text :species :pct {:text :pct :align-x :center :offset-y -6}))
pctspecies33.333.333.3setosaversicolorvirginica051015202530

To place a label relative to its own point, anchor it with :align-x/:align-y – :align-x :right puts the label’s right edge at the bar’s end, so the text sits inside the bar. (To spread overlapping marks on a categorical axis, use :jitter or :position :dodge.)

(-> {:species ["setosa" "versicolor" "virginica"] :pct [33.3 33.3 33.3]}
    (pj/lay-bar :species :pct {:color "#a6cee3"})
    (pj/lay-text :species :pct {:text :pct :align-x :right})
    (pj/coord :flip))
speciespct33.333.333.3051015202530setosaversicolorvirginica

Anchoring is covered in Placing Marks, and :dx itself in the same chapter.

Log Scale via :scale-x / :scale-y Options

Symptom: Passing {:scale-x :log} (or {:scale-y :log}) to a layer or to pj/options prints a warning – "does not recognize [:scale-x]" – and the chart comes out on a linear axis.

Cause: :scale-x and :scale-y are not option keys at all. A scale is set with pj/scale, called on the pose, or inside a mapping written out in full – {:x {:column :carat :scale :log}}.

The wrong form does not throw; it warns and falls back to a linear axis:

(with-out-str
  (-> (rdatasets/ggplot2-diamonds)
      (pj/lay-point :carat :price {:scale-y :log})
      pj/plan))
"Warning: pj/lay-point does not recognize [:scale-y].\n  Set by pj/scale, not by an options map: [:scale-y]\n  Accepted: [:alpha :color :color-type :data :dx :dy :group :in :jitter :mark :offset-x :offset-y :position :shape :size :stat :tooltip :x :x-type :y :y-type]\n"

Fix: Use pj/scale:

(-> (rdatasets/ggplot2-diamonds)
    (pj/lay-point :carat :price {:alpha 0.1})
    (pj/scale :y :log))
pricecarat0.00.51.01.52.02.53.03.54.04.55.030050010002000300050001000020000

pj/scale takes the pose, an aesthetic – an axis (:x, :y) or a visual one (:size, :alpha, :color, :fill, :shape) – and either a type keyword (:linear, :log) or a scale specification map with :type and an optional :domain override. See the Inference Rules chapter for how scale types and domains interact with column inference.

Two Columns Given to a One-Column Layer Type

Symptom: an error such as "pj/lay-histogram draws from one column, and was given :sepal-length on :x and :sepal-width on :y."

Cause: Histogram, density, and rug layer types draw from one column. Two columns given as arguments to one of them, as in (pj/lay-histogram data :sepal-length :sepal-width), report an error. (lay-bar is not among them – it uses a second column as the bar length.) One column named on :y is not an error: (pj/lay-histogram data {:y :sepal-width}) draws the histogram on its side.

This restricts the layer’s own columns and nothing else. A rule or a band goes on such a pose like any other layer, because its intercept is a written value rather than a column reference – see A reference line on a distribution.

(try
  (-> (rdatasets/datasets-iris)
      (pj/lay-histogram :sepal-length :sepal-width)
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-histogram draws from one column, and was given :sepal-length on :x and :sepal-width on :y. Name one column; to draw the layer on its side, name the column on :y, as in (pj/lay-histogram data {:y :sepal-length})."

Fix: Remove the y column:

(-> (rdatasets/datasets-iris)
    (pj/lay-histogram :sepal-length))
countsepal length4.55.05.56.06.57.07.58.00510152025

Categorical Column with Log Scale

Symptom: an error such as "Log scale on :x requires numeric data, but column :species is non-numeric."

Cause: Log scales only work with numerical columns. Categorical columns (strings, keywords) have no meaningful log transform.

(try
  (-> (rdatasets/datasets-iris)
      (pj/lay-bar :species)
      (pj/scale :x :log)
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Log scale on :x requires numeric data, but column :species is non-numeric."

Fix: Use a numerical column for the log-scaled axis, or drop the log scale on the categorical axis.

Polar Coordinates with Unsupported Marks

Symptom: "Mark :line is not supported with polar coordinates. Supported polar marks: (:bar :point :rect :rug :text)" (or the same message for :area and other unsupported marks).

Cause: Polar coordinates currently support a subset of marks: :bar, :point, :rect, :rug, and :text. Layer types built on these marks (such as :histogram, which renders as bars, and :bar with a y column, which renders as rectangles) work too.

(try
  (-> {:x [1 2 3 4 5] :y [2 4 3 5 4]}
      (pj/lay-line :x :y)
      (pj/coord :polar)
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Mark :line is not supported with polar coordinates. Supported polar marks: (:bar :point :rect :rug :text)"

Fix for now: Use a supported mark. A bar chart flipped to polar becomes a rose chart:

(-> (rdatasets/datasets-chickwts)
    (pj/pose :feed)
    pj/lay-bar
    (pj/coord :polar))
countfeedhorsebeanlinseedsoybeansunflowermeatmealcasein0510

Support for :line, :area, and other marks in polar is planned. See the Polar Coordinates chapter for the full set of currently supported marks and examples.

Tooltip and Brush Not Working

Symptom: You set {:tooltip true} but no tooltip appears when hovering over points.

Cause: Tooltip and brush interactivity use JavaScript that requires a compatible notebook viewer. Static HTML export or some viewers may not support it.

Fix: Use Clay or another Kindly-compatible tool that supports kind/hiccup with embedded scripts.

A second cause: the plot was rendered to a format that draws no interaction. A tooltip and a brush are drawn by a browser reading the figure, so SVG is the only format that draws them; {:format :bufimg} and pj/save to a PNG print a warning that the request draws nothing, naming the formats that do. Read the message the plot printed before looking at the viewer.

(with-out-str
  (-> (rdatasets/datasets-iris)
      (pj/lay-point :sepal-length :sepal-width {:color :species})
      (pj/options {:tooltip true})
      (pj/plot {:format :bufimg})))
"Warning: :tooltip asked for, and the :bufimg format draws no interaction. The formats that do: :svg. The request is accepted and draws nothing.\n"

The figure still renders – the request is dropped, not the plot. Fix: render to SVG, which is what a notebook shows by default:

(-> (rdatasets/datasets-iris)
    (pj/lay-point :sepal-length :sepal-width {:color :species})
    (pj/options {:tooltip true}))
sepal widthsepal lengthspeciessetosaversicolorvirginica4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

Faceting Keys in a Layer’s Mapping

Symptom: An error naming the panel aesthetic that was written in a layer’s mapping, when you put :col, :row, or one of the older :facet-col, :facet-row, :facet-x, :facet-y spellings inside a pj/lay-* mapping.

Cause: :col and :row are panel aesthetics, and a panel aesthetic is read from a pose’s mapping and not from a layer’s. A facet divides every layer of the pose, so Plotje cannot divide the panels by a column named on one layer. It reports an error rather than dropping the panel aesthetic.

(try
  (-> (rdatasets/datasets-iris)
      (pj/pose :sepal-length :sepal-width)
      (pj/lay-point {:facet-col :species})
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"A panel aesthetic facet-col was written in a layer's mapping. A facet divides every layer of the pose, and a panel aesthetic is read from a pose's mapping and not from a layer's, so write it on the pose: (pj/facet pose :species), or in the pose's mapping. On a composite, write it on the cell whose panels it divides."

Fix: write the panel aesthetic on the pose – (pj/facet pose :species) across, (pj/facet pose {:row :species}) down, or {:col ... :row ...} for both – or put :col and :row in the pose’s mapping:

(-> (rdatasets/datasets-iris)
    (pj/lay-point :sepal-length :sepal-width)
    (pj/facet :species))
sepal widthsepal length682.02.53.03.54.04.56868setosaversicolorvirginica

A facet over categories is a mapping and obeys the scope rules, so a facet written on a composite divides every cell, and a facet written on one cell divides that cell alone. Here the facet is written on an arrangement of two cells, and each cell is drawn as three panels:

(-> (pj/arrange [(pj/lay-point (rdatasets/datasets-iris) :sepal-length :sepal-width)
                 (pj/lay-point (rdatasets/datasets-iris) :petal-length :petal-width)])
    (pj/facet :species))
sepal widthsepal length682.02.53.03.54.04.56868setosaversicolorvirginicapetal widthpetal length2460.00.51.01.52.02.5246246setosaversicolorvirginica

Constant :x or :y in a Layer’s Options

Symptom: a note is needed at one fixed spot, and no column holds the x and y it belongs at.

This is not an error. :x and :y may be given as a value, the same way a color may be "red" rather than a column. The layer below places its text at x 6.5 and y 3.5 with no dataset of its own, and a string :text on such a layer is the text itself rather than a column name:

(-> (rdatasets/datasets-iris)
    (pj/lay-point :sepal-length :sepal-width)
    (pj/lay-text {:x 6.5 :y 3.5 :text "mean"}))
sepal widthsepal lengthmean4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

The other shape a value takes is beside a column. When :y is a column the layer describes the data, so the value given for :x repeats for every row – which is how a label at one fixed x is written. The five team names below line up at x 33 rather than sitting over their points, each at the revenue of its own row:

(-> {:team ["North" "South" "East" "West" "Central"]
     :spend [12 19 15 24 31]
     :revenue [30 45 38 62 74]}
    (pj/lay-point :spend :revenue)
    (pj/lay-text {:x 33 :y :revenue :text :team}))
revenuespendNorthSouthEastWestCentral12141618202224262830323430354045505560657075

:x and :y alone decide which of the two a value draws. A layer that gives both as values does not read the data at all, so it draws once; a value beside a column applies to each row, so it repeats. A string :text splits the same way: it is the text on a layer of values alone, and a column name once the layer has data to name a column in.

A value given for :x or :y is a data value like any other, so it takes part in the axis domains: a note placed beyond the data widens the axis to hold it. To place a mark on the panel instead – in drawing units from the corner of the panel background, leaving the axes alone – give the layer :in :drawing-area. The Placing Marks chapter covers that choice, and the two spaces the two readings belong to.

Reference lines remain their own layer types: pj/lay-rule-h with :y-intercept and pj/lay-rule-v with :x-intercept draw a line across the whole panel, which one :x and one :y cannot say.

A number is read against the data. A dataset built without column names is given integer ones, and a number in a mapping names such a column where the data carries it:

(-> (tc/dataset [[1 2] [3 4] [5 7]])
    (pj/lay-point 0 1))
Notestderr
[nREPL-session-c9b9fae5-165f-478a-a03b-662df66eca10] WARN tablecloth.api.dataset - Dataset creation behaviour changed for 2d 2-element arrays in v7.029. See https://github.com/scicloj/tablecloth/issues/142 for details.
1012345234567

Where the data carries no column of that name, the same number is a value to place a mark at, and the axis stretches to hold it. To say which reading you mean rather than letting the data decide, write the mapping in full: {:x {:column 0}} reads the column and {:x {:value 0}} places every mark at zero. Renaming the columns is the other remedy, and usually the clearer one:

(-> (tc/dataset [[1 2] [3 4] [5 7]])
    (tc/rename-columns [:x :y])
    (pj/lay-point :x :y))
Notestderr
[nREPL-session-c9b9fae5-165f-478a-a03b-662df66eca10] WARN tablecloth.api.dataset - Dataset creation behaviour changed for 2d 2-element arrays in v7.029. See https://github.com/scicloj/tablecloth/issues/142 for details.
yx12345234567

The Datasets chapter covers column names in full.

A Layer Left Without a Column

Symptom: an error such as "pj/lay-point draws from two columns, one on :x and one on :y, and this layer has no column on :x".

Cause: A layer reads its columns from its own mapping, from the pose’s mapping, and from the panel’s columns – those of the first layer naming any. Where none of the three names a column the layer type draws from, there is nothing to draw. Data of four or more columns gives no default mapping, so on such data every column has to be named somewhere. The message names the layer, the axis that is missing, what the layer does name, and the columns of the data:

(try
  (-> (rdatasets/datasets-iris)
      pj/pose
      (pj/lay-point {:y :sepal-width})
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-point draws from two columns, one on :x and one on :y, and this layer has no column on :x. It names :y :sepal-width. Name the :x column on the layer -- (pj/lay-point pose :x-column :sepal-width) -- or in the pose's mapping, which every layer of the pose reads. Available columns: [:petal-length :petal-width :rownames :sepal-length :sepal-width :species]"

Fix: Name the missing column on the layer, or in the pose’s mapping where every layer should read it:

(-> (rdatasets/datasets-iris)
    (pj/pose {:x :sepal-length})
    (pj/lay-point {:y :sepal-width}))
sepal widthsepal length4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

A Layer Naming a Column the Facet Lacks

Symptom: an error such as "pj/lay-point names :petal-width on :y, and no panel draws it".

Cause: A facet over columns draws a panel per column, and a layer naming a column is drawn in the panel of that column. A layer naming a column that is not among the facet’s fits no panel, so it is drawn in every panel – which works only where each panel also has a layer drawing its own column. Here no other layer does, so drawing the layer in every panel would leave the facet’s columns drawn nowhere, and the facet reports an error:

(try
  (-> (rdatasets/datasets-iris)
      (pj/lay-point :sepal-length :petal-width)
      (pj/facet {:y [:sepal-width :petal-length]})
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-point names :petal-width on :y, and no panel draws it: the panels draw :sepal-width and :petal-length on :y. Add :petal-width to the :y columns the panels are made from -- the facet's, or the cells given to pj/arrange -- and leave :y off the layer, or draw the layer on a pose of its own and put the two side by side with pj/arrange. A layer naming no column is drawn in every panel."

Fix: List the column in the facet, and name the varying columns in the facet alone:

(-> (rdatasets/datasets-iris)
    (pj/pose {:x :sepal-length})
    pj/lay-point
    (pj/facet {:y [:sepal-width :petal-length :petal-width]}))
sepal width234petal length246petal width4.55.05.56.06.57.07.58.0012sepal length

A layer naming another column is drawn in every panel where each panel also has a layer drawing its own column. Below, :petal-width is drawn in both panels beside each panel’s own column:

(-> (rdatasets/datasets-iris)
    (pj/pose {:x :sepal-length})
    pj/lay-point
    (pj/lay-point {:y :petal-width})
    (pj/facet {:y [:sepal-width :petal-length]}))
sepal width,petal widthsepal widthpetal width0246petal length,petal widthpetal lengthpetal width4.55.05.56.06.57.07.58.00246sepal length

One Panel Where Two Were Wanted

Symptom: two pj/lay-* calls on unrelated columns draw on one panel, and one axis holds both columns.

Cause: the layers of a pose share its one panel, whatever columns they name. Below, the second bar layer names a different value column from the first, and both are drawn against one x axis:

(-> {:cohort [:a :b :c] :growth [12 19 15] :tax [3 5 4]}
    (pj/lay-bar :growth :cohort {:color "#377eb8"})
    (pj/lay-bar :tax :cohort {:color "#e6550d"}))
cohortgrowth, tax024681012141618abc

Fix: split the pose over the columns with pj/facet. Each layer is drawn only in the panel of the column it names:

(-> {:cohort [:a :b :c] :growth [12 19 15] :tax [3 5 4]}
    (pj/lay-bar :growth :cohort {:color "#377eb8"})
    (pj/lay-bar :tax :cohort {:color "#e6550d"})
    (pj/facet {:x [:growth :tax]}))
growth051015abctax012345cohort

Separately built plots go side by side with pj/arrange. The Composition chapter shows a facet over columns and an arrangement of separate poses.

On an arrangement, a pj/lay-* call adds no cell. The layer is drawn in each cell it fits, and in every cell where it fits none – see Rules LP2 and LP3 in Pose Rules. So a second bar layer added to an arrangement of one cell is drawn in that cell, beside the first, and nothing is printed. To draw the layer on a cell of its own, give that cell to pj/arrange, which adds cells to an arrangement:

(-> (pj/arrange [(pj/lay-bar {:cohort [:a :b :c] :growth [12 19 15] :tax [3 5 4]}
                             :growth :cohort {:color "#377eb8"})])
    (pj/arrange [(pj/lay-bar {:cohort [:a :b :c] :growth [12 19 15] :tax [3 5 4]}
                             :tax :cohort {:color "#e6550d"})]))
cohortgrowth051015abccohorttax012345abc

A plot written for Plotje 0.16.1 or 0.17.0 can draw on one panel where that release drew two; see Upgrading from an Earlier Release.

Dataset Missing Columns a Template References

Symptom: An error like "Cannot attach data: pose references column(s) [:group] not present in the dataset. Available columns: [:x :y]" when calling pj/with-data on a dataless template pose.

Cause: pj/with-data validates at attach time – every keyword column reference in the template must exist in the dataset, or the attachment fails immediately.

(def template
  (-> (pj/pose nil {:x :x :y :y :color :group})
      pj/lay-point))
(try
  (-> template
      (pj/with-data {:x [1 2 3] :y [4 5 6]}))
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Cannot attach data: pose references column(s) [:group] not present in the dataset. Available columns: [:x :y]."

Fix: Either rename the dataset columns to match the template (tc/rename-columns), or adjust the template to reference the columns the dataset has.

(-> (pj/pose nil {:x :x :y :y})
    pj/lay-point
    (pj/with-data {:x [1 2 3] :y [4 5 6]}))
yx123456

Horizontal Ranking Bars Draw Biggest-at-Bottom

Symptom: A horizontal bar chart made with (pj/coord :flip) shows the first row of the data at the bottom of the chart. A descending-sorted “top-N” dataset ends up with the biggest bar at the bottom instead of the top.

Cause: coord :flip draws categories bottom-to-top in the order they appear in the data. ggplot2’s coord_flip() also runs bottom to top, in the order of the factor levels.

Descending data plotted as-is – “A” (the biggest) renders at the bottom, not the top:

(-> [{:category "A" :value 100}
     {:category "B" :value 50}
     {:category "C" :value 25}]
    (pj/lay-bar :category :value)
    (pj/coord :flip))
categoryvalue0102030405060708090100ABC

Fix: Sort the dataset ascending before plotting – the ascending order shows up top-to-bottom on the flipped axis, so the biggest value lands at the top:

(-> [{:category "A" :value 100}
     {:category "B" :value 50}
     {:category "C" :value 25}]
    (tc/dataset)
    (tc/order-by [:value] :asc)
    (pj/lay-bar :category :value)
    (pj/coord :flip))
categoryvalue0102030405060708090100CBA

Or leave the data as it is and give the category axis a :domain listing the categories in the order to draw them, bottom to top. A scale is named by the data’s column, not by the axis it is drawn on: under (pj/coord :flip) the categories are the data’s :x, so the scale is (pj/scale pose :x ...), though the categories run down the vertical axis:

(-> [{:category "A" :value 100}
     {:category "B" :value 50}
     {:category "C" :value 25}]
    (pj/lay-bar :category :value)
    (pj/coord :flip)
    (pj/scale :x {:domain ["C" "B" "A"]}))
categoryvalue0102030405060708090100CBA

Point Sizes Changed From an Earlier Release

Symptom: A plot with {:size :some-column} draws its middle values larger than it used to, and the difference between the smallest and largest points looks less dramatic.

Cause: A size scale spreads the square root of the value across the radii, so the area of a mark grows with the value rather than with its square. Earlier releases spread the value itself, which exaggerates the differences. The two ends of the range are unchanged; the values between them moved.

(-> {:x [1 2 3 4 5 6] :y [1 1 1 1 1 1] :n [1 4 9 16 25 36]}
    (pj/lay-point :x :y {:size :n}))
yxn5101520253035123456012

Fix: {:by :linear} restores the earlier reading, and {:by :area} gives the strict one, where equal steps in value are equal steps in ink:

(-> {:x [1 2 3 4 5 6] :y [1 1 1 1 1 1] :n [1 4 9 16 25 36]}
    (pj/lay-point :x :y {:size :n})
    (pj/scale :size {:by :linear}))
yxn5101520253035123456012

A Band Whose Two Edges Are the Same Value

Symptom: "lay-band-h requires :y-min < :y-max" error on a call that drew something, or nothing, in an earlier release.

Cause: A band shades the region between two values. Where the two are equal the region is empty, so the band covered nothing and drew a rectangle of zero thickness – present in the SVG, invisible on the plot, and named by no message. It reports an error now.

(try
  (-> (rdatasets/datasets-iris)
      (pj/lay-point :sepal-length :sepal-width)
      (pj/lay-band-h {:y-min 3.0 :y-max 3.0}))
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-band-h requires :y-min < :y-max, got both 3.0. A band with equal bounds covers nothing and draws nothing. For a line at that value use (pj/lay-rule-h pose {:y-intercept 3.0})."

Fix: A line at one value is a rule; a band needs two distinct edges. The error names the rule constructor for the axis in question.

(-> (rdatasets/datasets-iris)
    (pj/lay-point :sepal-length :sepal-width)
    (pj/lay-rule-h {:y-intercept 3.0}))
sepal widthsepal length4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

A Configuration Key From an Earlier Release

Symptom: A configuration key that used to set something prints a warning that it is not recognized, and the plot draws the default.

Cause: The key was renamed. Plotje warns about a key it does not read rather than merging it with no message, and where a release renamed a key the warning names the key that replaced it. :annotation-stroke sets the colour a rule draws in where its layer names none, and it is :rule-color now.

(with-out-str
  (pj/with-config {:annotation-stroke "firebrick"} nil))
"Warning: pj/with-config does not recognize configuration key(s): [:annotation-stroke].\n  :annotation-stroke was renamed to :rule-color\n  Accepted: [:band-color :band-opacity :bar-opacity :bin-method :brush :color-midpoint :color-range :color-values :decimal-separator :default-color :domain-padding :fill-midpoint :fill-range :fit-text-domain :format :grid-stroke-width :height :label-font-size :label-offset :legend-font-size :legend-position :legend-width :line-width :margin :min-panel-size :point-opacity :point-radius :point-stroke :point-stroke-width :rule-color :strict :strip-font-size :strip-height :strip-labels :theme :thousands-separator :title-font-size :title-offset :tooltip :validate :width :x-tick-angle :x-tick-label-pad :x-tick-spacing :y-tick-angle :y-tick-spacing]\n"

The same warning is printed for pj/set-config!, a plotje.edn file, the :config option and pj/options, so the key is named wherever it can be written.

Fix: Write the current name.

(-> (rdatasets/datasets-iris)
    (pj/lay-point :sepal-length :sepal-width)
    (pj/lay-rule-h {:y-intercept 3.0})
    (pj/options {:rule-color "firebrick"}))
sepal widthsepal length4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

A Layer Setting From an Earlier Release

Symptom: A layer setting that used to shift a mark prints a warning naming a different key, and the plot draws as it always did.

Cause: The layer setting was renamed. :nudge-x and :nudge-y are :dx and :dy. Plotje reads the old name as the new one, and the warning names the edit, where a renamed configuration key is dropped: dropping a shift would move a label back onto the mark it was written to clear. The warning is printed where the layer is built rather than where the plan is built:

(with-out-str
  (-> {:team ["red" "green" "blue"] :score [3 5 4]}
      (pj/lay-bar :team :score)
      (pj/lay-text {:x {:value "red"} :y 3 :nudge-x 0.5 :text "note"})))
"Warning: pj/lay-text: :nudge-x was renamed to :dx, and is read as :dx for now. Write :dx.\n"

Fix: Write the current name. Both names draw one picture, which is why the old name warns rather than reporting an error:

(-> {:team ["red" "green" "blue"] :score [3 5 4]}
    (pj/lay-bar :team :score {:color "#a6cee3"})
    (pj/lay-text {:x {:value "red"} :y 3 :align-x :center
                  :dx 0.5 :offset-y -10 :text "half a band"}))
scoreteamhalf a bandredgreenblue012345

Under :strict the old name reports an error instead of warning, so a notebook run that way names every old name still written in it. Configuration covers the setting.

A :size or :alpha Column That Changes Nothing

Symptom: {:size :some-column} on a line, boxplot or lollipop draws marks of one size, and a warning names the marks that vary the aesthetic.

Cause: Only marks that draw a size per row can read a size column. Among the built-in marks that is pj/lay-point. Every other mark draws one width or one opacity for the whole layer, so the column changes nothing. No legend is drawn for it either, since the legend would describe an encoding the panel does not show.

Fix: Write the value itself for a layer-wide size – {:size 2} on a line is a stroke width – and map the column on a layer whose mark varies it.

(-> {:x [1 2 3] :y [2 4 3] :r [1 2 3]}
    (pj/pose :x :y)
    (pj/lay-line {:size 2})
    (pj/lay-point {:size :r}))
yxr123123234

Dodge Has No Effect on Points Over Numbers

Symptom: Adding :position :dodge to pj/lay-point on a numeric x does not spread points apart by group – the plot looks identical to the version without :position :dodge.

Cause: A dodge divides a category’s band among the groups, and a numeric axis has no bands. On a numeric x the option is accepted and ignored. The marks that do not draw a dodge on any axis are listed in Known Limitations.

The points below are drawn with :position :dodge, and they sit at the same x values as they do without it:

(def points-data
  {:dose [1 1 2 2 3 3] :response [10 15 20 25 30 35] :batch ["A" "B" "A" "B" "A" "B"]})
(-> points-data
    (pj/lay-point :dose :response {:color :batch :position :dodge}))
responsedosebatchAB123101520253035

Fix: Where the x values are categories, read them as categories – a column of text is read that way already, and {:x-type :categorical} reads a numeric column that way. The points are then dodged within each band:

(-> points-data
    (pj/lay-point :dose :response {:color :batch :position :dodge :x-type :categorical}))
responsedosebatchAB123101520253035

Where x is a true quantity, distinguish the groups with :color or :shape, or pre-compute small offsets in the data.

Heatmap with Categorical Axes

Symptom: "Stat :bin2d requires a numeric column for :x, but :x is categorical" when passing a string column to pj/lay-tile.

Cause: Without :fill or :color, pj/lay-tile counts the rows that fall in each cell of a two-dimensional grid of bins, and a bin is an interval of numbers, so both columns have to be numeric.

(try
  (-> {:x ["a" "b" "c"] :y ["a" "b" "c"] :v [1 2 3]}
      (pj/lay-tile :x :y)
      pj/plan)
  (catch Throwable t (ex-message t)))
"Stat :bin2d requires a numeric column for :x, but :x is categorical. A stat that bins or smooths reads numbers; a column of text names groups instead. Dates written as text stay categorical until they are parsed with tc/convert-types."

Fix: map the column holding each cell’s value to :fill. The tile then draws one cell per row instead of counting rows, and a categorical axis gives each category one cell:

(-> (for [[i day] (map-indexed vector ["Mon" "Tue" "Wed" "Thu" "Fri" "Sat" "Sun"])
          hour (range 0 24)]
      {:day day :hour hour :v (+ (* 0.3 (Math/sin (* 0.5 hour)))
                                 (* 0.2 (mod i 3)))})
    (pj/lay-tile :day :hour {:fill :v}))
hourdayv-0.20.00.20.40.6MonTueWedThuFriSatSun05101520

Empty or All-Missing Column on a Grouping Layer

Symptom: "lay-boxplot requires a categorical column on either :x or :y, but :c has no rows and :v has no rows", or "has no values" where the column holds only nil.

Cause: The layer groups its data by category, and no row carries one. This is usually a data problem upstream of the plot – a filter that matched nothing, or a join that dropped every row:

(try
  (-> {:group [] :measurement []}
      (pj/lay-boxplot :group :measurement)
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-boxplot requires a categorical column on either :x or :y, but :group has no rows and :measurement has no rows. A layer that groups by category needs at least one row carrying a category."

A column holding nothing but nil reports having no values, which separates the two cases – rows that never arrived, against rows that arrived empty:

(try
  (-> {:group [nil nil] :measurement [nil nil]}
      (pj/lay-boxplot :group :measurement)
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-boxplot requires a categorical column on either :x or :y, but :group has no values and :measurement has no values. A layer that groups by category needs at least one row carrying a category."

The message names neither column a type, deliberately. An empty column and an all-nil one are both typed boolean by tech.ml.dataset, and Plotje reads such a column as numerical so the rest of the pipeline has something to work with – so a message naming the type would send you to check something that is not the problem.

:text Given a Value That Names No Column

Symptom: "Column :nope (from :text) not found in dataset", ending "It is not a label either -- a label is a string".

Cause: :text takes either a column reference, drawing one row’s value at each mark, or a literal string drawn at every mark. A keyword that names no column is neither:

(try
  (-> {:x [1 2] :y [1 2]}
      (pj/lay-text :x :y {:text :nope})
      pj/plot)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"Column :nope (from :text) not found in dataset. Available: (:x :y) It is not a label either -- a label is a string."

Write a string for constant text – {:text "note"} – or name a column the dataset carries.

An Axis :domain That Is Not Two Numbers

Symptom: "pj/scale :y :domain [0] is not a pair of two finite numbers, as [0 100] is".

Cause: What a :domain means depends on the column it is read for. Against a continuous column it is the interval the panel spans, which is two finite numbers; against a categorical one it is the list of categories, in the order they are to be drawn. Since the column decides, the check is made once the column is known:

(try
  (-> {:height [1 2 3] :weight [1 2 3]}
      (pj/lay-point :height :weight)
      (pj/scale :y {:domain [0]})
      pj/plan)
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/scale :y :domain [0] is not a pair of two finite numbers, as [0 100] is. :y here reads a continuous column, so its domain is the interval the panel spans. It has 1. To extend the interval the data gives rather than replace it, write :include."

Write both ends – {:domain [0 100]} – or, to extend the interval the data gives rather than replace it, write :include instead: {:include 0} puts zero on the axis and leaves the other end to the data. Scales works through the difference.

Upgrading from an Earlier Release

A plot written for an earlier release can draw differently, and most of these changes print no message. The cases below are listed by the release a plot was written for.

Coming from 0.16.1

Plotje 0.16.1 drew a panel for each pair of columns its layers named, and printed a note saying so. The layers now share the panel, so a plot written for 0.16.1 can draw something different, and no message says that the plot changed. Two of the cases below print a message about the new plot: a Note where one axis holds columns of very different size, and a warning where no layer draws the pose’s columns. The two retired spellings in the list, the :scales option and the direction argument of pj/facet, also print a warning. A plot written for 0.16.1 also meets every change listed under Coming from 0.17.0. An axis stretches to fit everything drawn on it, so the layers sharing a panel should agree on units. Check these cases:

  • Columns of very different size on one axis. The smaller one is drawn flat against the edge of the panel. Below, :hwy (miles per gallon) shares a y axis with :year. This case prints a Note, naming the columns and the pj/facet call that gives each a panel, where one column takes less than a twentieth of the axis.

  • Layers that specify their own colour. No layer gets a legend entry naming its columns; the axis title still names every column.

  • A layer beside a series, on another column. The layer is drawn on the panel of the series, on the axis of the values of the series. With no :color specified the layer takes a colour and a legend entry named after its column, and the axis title lists the layer’s column after the columns of the series.

  • pj/pose called again to add a panel. A later pj/pose replaces the pose’s columns, and every layer naming no columns reads the new ones, so the plot draws the new columns once for each such layer and the old columns nowhere. No message is printed. Write the panels with pj/arrange.

  • The :scales option. It is retired and warns. :share-scales replaces it and names the axes the panels share, where :scales named the axes left free: write {:share-scales #{:x}} for {:scales :free-y}, #{:y} for :free-x, and #{} for :free.

  • The direction argument of pj/facet. It is retired and warns. (pj/facet pose :species :row) draws as before; write (pj/facet pose {:row :species}).

  • A pose’s columns beside a layer naming other columns. The plot draws the layer’s columns only, and warns that no layer draws the pose’s :x and :y. Draw both pairs of columns with pj/arrange.

For columns of very different size on one axis, and for layers that specify their own colour, split the pose with pj/facet, or draw the layers as separate poses with pj/arrange. A series and a facet over columns on one pose report an error, so for a layer beside a series draw the series and the layer as separate poses with pj/arrange.

Some spellings 0.16.1 accepted report an error or warn in place of drawing something else. A vector of column pairs given to pj/pose or to a pj/lay-* call reports an error naming pj/arrange and pj/matrix, which crosses two lists of columns. pj/cross warns that it is deprecated and returns its pairs, so (pj/pose data (pj/cross cols cols)), the 0.16.1 way to write a scatter plot matrix, reports the same error; write (pj/matrix data cols). A vector of columns given alone to pj/pose or to a pj/lay-* call is a series, drawn on one panel, where 0.16.1 drew a panel per column; pj/arrange draws a panel per column. pj/overlay and {:overlay true} warn that they are ignored, and {:overlay false} reports an error. pj/facet-grid is retired too: it warns, names the pj/facet call to write, and draws the same panels. Here is the error for pairs of columns:

(try
  (pj/lay-point (rdatasets/datasets-iris)
                [[:sepal-length :sepal-width] [:petal-length :petal-width]])
  (catch clojure.lang.ExceptionInfo e (ex-message e)))
"pj/lay-point was given pairs of columns, [[:sepal-length :sepal-width] [:petal-length :petal-width]]. A pose draws one panel. For a panel per pair, write each pair as a mapping: (pj/arrange data [{:x :sepal-length, :y :sepal-width} {:x :petal-length, :y :petal-width}]); for every :x column against every :y column, write (pj/matrix data xs ys)."

The plot below draws :hwy and :year on one axis, the first case in the list:

(-> (rdatasets/ggplot2-mpg)
    (pj/lay-point :displ :hwy)
    (pj/lay-point :displ :year))
hwy, yeardisplhwyyear2345670200400600800100012001400160018002000

Coming from 0.17.0

A pose built inside pj/with-config and shown after the body returns – as a notebook shows the value of a form – is drawn under the configuration in effect when it is shown, not under the configuration pj/with-config gave. Plotje 0.17.0 kept that configuration on the pose. A notebook written for 0.17.0 that returns a pose from pj/with-config draws the default theme, and no message says so. To keep the configuration, end the body in pj/plot:

(pj/with-config {:theme {:bg "#1a1a2e"}}
  (-> (rdatasets/datasets-iris)
      (pj/lay-point :sepal-length :sepal-width)
      pj/plot))
sepal widthsepal length4.55.05.56.06.57.07.58.02.02.53.03.54.04.5

A pj/lay-* call naming columns, written on an arrangement, is drawn in the cells it fits – see Rule LP2 and Rule LP3 in Pose Rules. Plotje 0.16.1 and 0.17.0 added such a layer to the last of those cells alone, cells split with :where included. A plot written for those releases can draw the layer in more cells, and no message says so. The layer is drawn under the layers each cell holds of its own, where those releases drew it on top. To draw a layer in one cell, add it to that cell before arranging the cells. Here both cells draw :displ against :hwy, one for front-wheel drive and one for four-wheel drive, so the trend line is drawn in both, under the points:

(-> (pj/arrange [(-> (rdatasets/ggplot2-mpg)
                     (pj/pose :displ :hwy {:where {:drv "f"}})
                     pj/lay-point)
                 (-> (rdatasets/ggplot2-mpg)
                     (pj/pose :displ :hwy {:where {:drv "4"}})
                     pj/lay-point)])
    (pj/lay-smooth :displ :hwy))
hwydispl2345202530354045hwydispl23456121416182022242628

A pj/lay-* call naming columns that no cell fits is drawn in every cell, as written, under each cell’s own layers – see Rule LP3 in Pose Rules. Plotje 0.16.1 and 0.17.0 drew such a layer on a new cell appended to the arrangement. A histogram of :displ added to two scatter cells is drawn in both cells, and a Note says that each cell’s :y axis reads in two units, a count and :hwy or :cty. To draw the histogram on a cell of its own, give that cell to pj/arrange, which adds cells to an arrangement:

(-> (pj/arrange [(pj/lay-point (rdatasets/ggplot2-mpg) :displ :hwy)
                 (pj/lay-point (rdatasets/ggplot2-mpg) :displ :cty)])
    (pj/arrange [(pj/lay-histogram (rdatasets/ggplot2-mpg) :displ)]))
hwydispl24615202530354045ctydispl246101520253035countdispl246051015202530354045

A layer drawn in only some panels of a facet over columns prints no Note, and neither does a layer drawn in only some cells of an arrangement; Plotje 0.17.0 printed a Note naming the panels without the layer.

pj/lay-line and pj/lay-smooth under (pj/coord :flip) join their points in order of the data’s :x; to join them in order of the other column, swap the columns in the mapping. :margin-multi and :panel-size warn that they are retired, and the plot draws as if neither key were written: write :margin for the margin of every plot, and :panel-width or :panel-height for the size of each panel.

See Also

  • Core Concepts – the mapping and inference rules behind most of these symptoms

What’s Next

source: notebooks/plotje_book/troubleshooting.clj