{on hold} ds4clj: a data science course for Clojure devs

A data science course for Clojure devs
Published

July 12, 2022

black-eyed-pea

For various reasons of community priorities, this project was delayed. As of May 2024, we are reconsidering a revised version, which is probably a semi-structured series of talks alongside the more hands-on real-world-data group.

The content below is the old 2022-2023 draft.

exploration meetings preparing for the course

goals

  • provide Clojure devs with basic theory, practices, and tools for common data science tasks;
    • also welcome open-minded people who are new to Clojure;
  • create learning resources for future use;
  • encourage Clojurians to become active contributors to the emerging stack.

requirements

  • at least one of the following:
    • basic knowlede of Clojure
    • a very open mind towards new programming languages
  • being ready to put a few hours a week into learning and practice, for a few months

chat

time

  • We’ll have a class once a month.
  • We may have practice meetings in between.
  • Each class is 3 hours, containing a lecture, a short break, and then discussion and Q&A.

recording

  • The meetings will be recorded and shared at the Clojurians Zulip chat.
  • Some parts of the recordings (e.g. the lecture) will be shared publicly.

homework

Homework will be composed of:

  • exercises on class topics
  • personal projects (as individuals or in small groups) – for example:
    • exploring datasets
    • reproducing previously publushed research
    • contributing to the stack of relevant libraries
    • contributing to documentation

list of topics (tentative)

language

(mostly self learning)

  • Clojure
  • R

from today’s brainstorming:

theory & methods

(very basic intros)

  • hello world: a typical workflow
  • reshaping data (the “tidy” notion)
  • correctness: testing, reproducibility
  • descriptive statistics
  • frequentist statistical inference
  • supervised learning: principles & workflow, regression, classification
  • working with tree-structured data
  • probabilistic modelling through Bayesian statistics
  • unsuprevised learning: clustering, dimension reduction
  • linear algebra
  • deep networks
  • nlp
  • async data streams
  • graph data

libraries & tools

(some introduced briefly, some more thoroughly)

  • tables: tablecloth, tech.ml.dataset
  • arrays: dtype-next
  • transducers: xforms, injest
  • correctness (schemas): malli
  • data vis & notebook tooling: portal, oz, clerk, clay
  • data vis grammars: hanami, cljplot?
  • math stats: fastmath, kixi.stats
  • machine learning algorithms & pipelines: scicloj.ml
  • interoperation w/ other languages: libpython-clj, clojisr
  • Bayesian statistics: inferme, clj-stan
  • tree data: clojure.walk, core.match, specter, meander, tupelo
  • data ingestion: jdbc-next?, some web scraping, arrow?
  • linear algebra, deep learning: neanderthal, deep-diamond
  • parallel computing: geni, clojask
  • nlp: datalinguist?, spacy through interop
  • graph data: loom, asami, neo4j?

course plan (very tentative)

month topic libraries homework
core topics
1 common workflow tablecloth, fastmath, hanami, learn some R and Clojure
scicloj.ml
2 descriptive stats, fastmath, kixi.stats apply to a real-world dataset
frequentist stats
3 data visualization hanami, cljplot? apply to a real-world dataset
4 arrays & tables dtype-next, tech.ml.dataset, apply to a real-world dataset,
tablecloth run some speed comparisons
5 basic supervised learning workflow, scicloj.ml, malli reproduce some kaggle notebooks
reproducibility, tests
6 probability, Bayesian stats inferme, clj-stan reading in Statistical Rethinking,
reproducing some examples
7 advanced supervised learning workflows scicloj.ml reproduce some kaggle notebooks,
explore variations & improvements
8 unsupervised learning fastmath, scicloj.ml reproduce some kaggle notebooks,
explore variations & improvements
9 python and R interop libpython-clj, clojisr go through some tutorials by
interop
specialized topics
10 working with tree-sructured data, clojure.walk, specter, meander, scrape & analyse some
web scraping hickory unstructured data
11 linear algebra, basic image processing neanderthal process some images
12 deep learning deep diamond reproduce some tutorials
13 async data streams xforms, kixi.stats analyse some user events
14 natural language processing datalinguist, analyse some texts,
spaCy through interop write rules to capture intents
15 graph data loom, asami, neo4j? analyse some graph data
16 big datasets geni, clojask, tech.ml.dataset query and analyse a big dataset,
run some speed comparisons