Clojurists Together update - July 2025 - Siyoung Byun

TL;DR

• We are now more ready than earlier to explore the relevance of Clojure in new fields like education, research, academic papers, or art, thanks to recent progress around tooling, libraries, beginner resources, and community building.

• We propose proceeding in small steps through ad-hoc meetings, lightweight conferences, and short-term research projects.

• One space to support that is the new macroexpand gatherings series. We encourage you to mark you availability and preferences: Poll.

• If you have any thoughts or ideas: Please reach out.

Now, let us dive more into the details.

Why

There are many “Why Clojure” blog posts out there, and we will not try to reproduce their arguments here.

Let us just recognize that the efforts we are discussing here are not without a reason.

Many of us face technical challenges at work, and not only in engineering work. In research, teaching, and data exploration; in academia, and elsewhere; sometimes, programming is necessary. Often, programming can bother us with complexity, conceptual obstacles, and various kinds of friction.

Some of us believe that Clojure can offer a relief through simplicity, conceptual elegance, and lightweight exploration. Arguably, this can be relevant to anybody who has some data and is open-minded and curious to enjoy new ways of thinking.

Growth

In many business situations, this might not be the best time for non-mainstream engineering choices, which are perceived as too risky. For this and other reasons, it is not the easiest time to work on Clojure growth.

However, that is what we are actively working on now.

The first reason is that it is necessary. Without new and diverse fields and use cases, our open-source efforts cannot be made sustainable.

The second reason is that we are ready. As discussed below, a few parts of the solution have matured enough to proceed and finally meet more diverse crowds.

The third reason is the Scicloj perspective within the broader Clojure community. With our focus on data analysis, we offer a pathway as to how to create “quickstart” applications for various use cases (as demanded in the recent Clojurists-together survey).

We will keep looking inward and outward at the same time: creating small collaborations, helping each other, and reflecting, while reaching out to new crowds and acting more in the public sphere.

Community, collaboration, continuity, and communication

In our coming expansion to new fields, let us recognize a few of our usual open-source challenges. Arguably, knowing them is going to be even more important now.

Scicloj is a group of people who collaborate and support each other. Throughout the years, a few hundred people have been lightly involved in various ways. However, at any given point in time, typically a few dozens are actually involved, and only a handful of people are active in library maintenance and essential activities.

The people involved change frequently. All of us need to stop sometimes, then return after a few weeks or months. Such pauses are often unplanned. Free time is inherently variable, and that is always felt in volunteer-based communities.

Our goals, however, are long-term. We need to maintain continuity, and see beyond those waves by embracing uncertainty.

During the last year or so, this has been especially challenging. For a mix of reasons – global, local, and personal – it has not been the easiest time for a few of us. Alongside the beautiful progress in libraries, tooling, and community building, many of us had to slow down and focus on life challenges.

Our processes need to be loose enough and stable enough. It is a delicate balance. While we often collaborate tightly, we should avoid depending on each other’s tasks. We need to decouple dependencies across project timelines, recognize bottlenecks and readapt. We need to communicate frequently, and use our everyday communication as a form of light touch knowledge management. We need to support each other and help each other find comfort.

Of all the above, let us repeat the importance of communication. Let us keep chatting frequently so we can be together in those fluctuations.

Clojure in academia

One space we are gradually addressing more is academia.

Clojure is not very common in academic teaching, but throughout the years there have been some beautiful projects that used Clojure in university courses. Some well-known courses were taught by Lee Spector, Elena Machkasova, The University of Helsinki, and Attila Egri-Nagy. Some of these projects are still active.

While Clojure is not commonly cited in academic papers, during the last few years we have met many researchers who are curious about Clojure. Quite a few of them actively use it. Others avoid using it as it is considered too far from the mainstream and thus difficult to communicate with colleagues.

In certain fields, Clojure is still perceived as a niche. Some of the relevant people act alone and are not so visible to each other.

Hopefully, we can improve the situation by making communication easier, providing spaces to sync up and think, and helping the various projects, groups, and individuals become more visible.

The November meetings

Here we share a few notes from the discussions we had last November, where a few Clojure-friendly academics met and thought together for a while.

See also the #data-science>scicloj paper - initial meeting topic thread at the Clojurians Zulip Chat and following topic threads on related topics.

Many thanks especially to Blaine Mooers, who highly affected those conversations. As a biochemistry professor, Blaine offered a few thoughtful insights and observations regarding the ways various technologies have grown in academia and what we may learn from that.

Here are some observations and insights by a few of the people involved.

• We should learn from what other tech communities did when they were smaller. See the paper about Matplotlib, for example. The Tidyverse (and the publishing around it) can be a source of inspiration.
• Making Clojure easy is essential. For better reach to academic groups, tooling is important. Also, a cohesive stack of libraries.
• Possible technical aspects of the Clojure ecosystem stack could be documented in the form of academic papers: extending the grammar of graphics, nontrivial interop solutions, and perspectives on the history of Lisp in AI are some examples.
• At some point, we may initiate new implementation projects on certain topics that look promising nowadays. Two possible examples are Applied Category Theory and Topological Data Analysis.
• Alongside (1) technical aspects and (2) applications, a third type of academic writing can be: (3) uses in education.
• Finding reviewers for a Clojure paper might be a challenge. At some point, we may try to assemble a list of potential reviewers.
• Conference talks are a quicker and easier path than papers. We can organize our own online conferences. Our own conference proceedings would take more effort (and will mainly need a decent group of reviewers).
• Applications of Clojure in education (point (3) above) – this path seems to be in consensus as a preferred and accessible way to make Clojure visible in academia in the short term. A few of the people involved have been putting lots of thought into teaching Clojure for years.
• We need to create more visible spaces to share and discuss these observations and insights.

Relevant progress

Recently, we have had some progress in a few relevant directions.

1. A cohesive stack of libraries. The Noj toolkit for data science has been maturing. Its documentation and integration tests have received quite a few contributions by Carsten Behring and a few other contributors.

2. Beginner resources and workshops. The Noj getting started repo and 🎥 video provides a welcoming way for people who know some Clojure but are new to Noj. Kira Howe’s workshop (notes here) at the BobKonf 2025 conference introduced Clojure for data analysis to people who are new to Clojure. We are working on additional workshops of this kind for May 10th. Additional tutorials are evolving too.

3. Beginner-friendly tooling. Our growing set of tutorials uses the Kindly standard for data-visualization and notebooking. In addition to the existing support in Clay, Timothy Pratley and Carsten Behring have been working on the kindly-render engine, that is now almost complete and allows various Clojure tools to support the standard. They have been working with a few Clojure tool makers, and the result is that a few tools now offer decent support for Kindly visualizations (and thus Noj). A few notable examples are Clojupyter, VSCode Calva Webview, and Intellij Cursive inline HTML. We have been also working on beginner-friendly minimal-setup modes for Clay using a live-reload feature by Ken Huang. See a few recent video demos: 🎥 #1, #2, #3.

4. Meetups. We have moved more of the Scicloj activity at the various dev & study groups to the public sphere. Most recent meetups have been focusing on workflow demos and AI systems. You can follow them at the 🎥 video channel, 📣 Clojureverse announcements, and 📆 Caldendar feed.

5. Conferences. The SciNoj Light online conference in May 16-17 is a first attempt to explore our current conference-organizing approach. The conference is mostly a way to drive new collaborations around research, writing, and speaking. We support the speakers throughout the process of proposing and preparing the talks and the accompanying notes. It is already safe to say that the method works, and soon we will announce the amazing list of speakers. It is not a small effort, but we have now verified that it is doable. The talk topics include 🚌 public transportation networks, 🚓 road safety, 🌊 Computational Fluid Dynamics, ⛈ weather forecasting, 🎮 Cognitive Psychology of geometric games, 🧠 brain science, 🐛 program analysis, 📰 text analysis in political context, and more. More events will follow later this year.

6. Communication. The Clojurians Zulip Chat Zulip is an essential building block in both our communication and our knowledge management. We are thankful to Gert Goet who has been nurturing it throughout the years, and to the broader team of admins that has grown recently. During the last few months, we put even more care into helping people join the platform and settle in. We organized Zulip study sessions and discussed it in additional meetups. We are gradually shifting some of our announcements and discussions to a web-public channel: 🎉 #scicloj-webpublic, to make things more accessible to people who we not logged in.

How we continue

With our currently limited resources, we will have to focus on lightweight small steps in the short term.

SciNoj Light can serve as a model for the following steps. We will have additional target dates later this year. Toward those dates, people can work on small research, on writing tutorials, and preparing talks. As usual, we will work together and create spaces to conveniently brainstorm and share progress.

At the new macroexpand gatherings series, we will maintain broader discussions on Clojure’s expansion to new fields. Academia, research, education, data analysis, and art, are a few of the directions we will explore. Community members will share past experiences, discuss pain points, and come up with actionable ways to collaborate.

As usual, we are hoping to hear your thoughts. If you have any comments or wish to propose anything, let us talk.

Tentative details

• Date: Saturday, May 10th
• A few hours long
• We will start from scratch, including a beginner-friendly setup to run Clojure.
• We will assume some background in programming (say, at least 2 years with some programming language).
• We will not assume any background in Clojure or data analysis.
• The content will be inspired by Kira Howe’s great workshop at the BobKonf 2025 conference.

Clojurists Together update - November 2024 - Daniel Slutsky

Scicloj is a Clojure group developing a stack of tools and libraries for data science. Alongside the technical challenges, community building has been an essential part of its efforts since the beginning of 2019. Our current main community-oriented goal is making the existing data science stack easy to use through the maturing of the Noj library, mentioned below. In particular, we are working on example-based documentation, easy setup, and recommended workflows for common tasks.

All these, and the tools to support them, grow organically, driven by real-world use cases.

I serve as a community organizer at Scicloj, and this project was accepted for Clojurists Together funding in 2024 Q1 & Q3. I also receive regular funding through Github Sponsors, mostly from Nubank.

As some parts of November’s work have only matured during the beginning of December, I am writing this report a couple of weeks late. It overviews my involvement during November, the first half of December, and then comments about the proposed directions for the near future.

I had 31 meetings during November and 25 meetings during the first half of December. Most of them were one-on-one meetings for open-source mentoring or similar contexts.

All the projects mentioned below are done in collaboration with others. I will mention at least a few of the people involved.

Clojurists Together update - October 2024 - Daniel Slutsky

Scicloj is a Clojure group developing a stack of tools and libraries for data science. Alongside the technical challenges, community building has been an essential part of its efforts since the beginning of 2019. Our current main community-oriented goal is making the existing data-science stack easy to use through the maturing of the Noj library, mentioned below. In particular, we are working on example-based documentation, easy setup, and recommended workflows for common tasks.

All these, and the tools to support them, grow organically, driven by real-world use cases.

I serve as a community organizer at Scicloj, and this project was accepted for Clojurists Together funding in 2024 Q1 & Q3. I also receive regular funding through Github Sponsors, mostly from Nubank.

In this post, I am reporting on my involvement during October 2024, as well as the proposed goals for November.

I had 57 meetings during October. Most of them were one-on-one meetings for open-source mentoring or similar contexts.

All the projects mentioned below are done in collaboration with others. I will mention at least a few of the people involved. For consistency, I use people’s github handles when mentioning their work on the projects, and their full names when mentioning public talks.

Clojurists Together update - September 2024 - Daniel Slutsky

Scicloj is a Clojure group developing a stack of tools and libraries for data science. Alongside the technical challenges, community building has been an essential part of its efforts since the beginning of 2019. Our current main community-oriented goal is making the existing data-science stack easy to use through the maturing of the Noj library, mentioned below. In particular, we are working on example-based documentation, easy setup, and recommended workflows for common tasks.

All these, and the tools to support them, grow organically, driven by real-world use cases.

I serve as a community organizer at Scicloj, and this project was accepted for Clojurists Together funding in 2024 Q1 & Q3. I also receive regular funding from Nubank.

In this post, I am reporting on my involvement during September 2024, as well as the proposed goals for October.

I had 77 meetings during September. Most of them were one-on-one meetings for open-source mentoring or similar contexts.

All the projects mentioned below are done in collaboration with others. I will mention at least a few of the main people involved.

Clojurists Together update - April 2024 - Daniel Slutsky

April 2024 was the last of three months on the Clojurists Together project titled “Scicloj Community Building and Infrastructure”.

Scicloj is an open-source group developing Clojure tools and libraries for data and science. As a community organizer at Scicloj, my current role is to help make the emerging Scicloj stack easier and more accessible for broad groups of Clojurians. I collaborate with a few Scicloj members on this. 

While this is the last update under the Clojurists Together 2024 Q1 support, the project will, of course, continue.  Below are the sub-projects that were addressed during April 2024. They are listed by their proposed priorities for the coming month. 

The new real-world-data group is ranked highest for its impact on community growth. This means the following. Assuming this group will (hopefully) grow well and demand attention, the goals of other projects will receive less attention and will be delayed. However, some of them (e.g., required extensions or bugfixes to libraries) will receive more attention if the real-world-data group requires them.

The New Column API

Today we at scicloj deployed a new Column API (tablecloth.column.api) into the data processing library Tablecloth (available as of version 7.029.1). This new API adds a new primitive to the Tablecloth system: the column. Here’s how we use it:

(require '[tablecloth.column.api :as tcc])

(tcc/column [1 2 3 4 5])
;; => #tech.v3.dataset.column<int64>[5]
null
[1, 2, 3, 4, 5]

The new column is the same as the columns that comprise a dataset. It is a one-dimensional typed sequence of values. Underneath the hood, the column is just the column defined in tech.ml.dataset, the library that backs Tablecloth.

The difference is that now when you are using Tablecloth you have the option of interacting directly with a column using an API that provides a set of operations that always take and return a column.

Basic Usage

Let’s go through a simple example. Let’s say we have some test scores that we need to analyze:

(def test-scores (tcc/column [85 92 78 88 95 83 80 90]))

test-scores
;; => #tech.v3.dataset.column<int64>[8]
null
[85, 92, 78, 88, 95, 83, 80, 90]

Now that we have these values in a column, we can easily perform operations on them:

(tcc/mean test-scores)
;; => 86.375

(tcc/standard-deviation test-scores)
;; => 5.926634795564849

There are a many operations that one can perform. At the moment, the available operations are those that you would have previously accessed by importing the tech.v3.datatype.functional namespace from dtype-next.

To get a fuller picture of the Column API and how it works, please consult the Column API section in the Tablecloth documentation.

Easier Column Operations on the Dataset

The changes we’ve deployed also improve the expressive power of Tablecloth’s standard Dataset API. Previously, if you needed to do something simple like a group by and aggregation on a column in a dataset, the code could become unnecessarily verbose:

(defonce stocks
  (tc/dataset "https://raw.githubusercontent.com/techascent/tech.ml.dataset/master/test/data/stocks.csv" {:key-fn keyword}))


(tc/column-names stocks)

(-> stocks
    (tc/group-by [:symbol])
    (tc/aggregate (fn [ds]
                    (-> ds
                        :price
                        tech.v3.datatype.functional/mean))))
;; => _unnamed [5 2]:

| :symbol |      summary |
|---------|-------------:|
|    MSFT |  24.73674797 |
|    AMZN |  47.98707317 |
|     IBM |  91.26121951 |
|    GOOG | 415.87044118 |
|    AAPL |  64.73048780 |

With the new column operations within for datasets, you can now simply write:

(-> stocks
    (tc/group-by [:symbol])
    (tc/mean [:price]))

The same set operations available to be run on the column can be called on columns in the datasest. However, when operating a dataset, functions that would return a scalar value act as aggregator functions, as seen above.

Functions that would return a new column allow the user to specify a target column to be added to the dataset, as in this example where we first use the method above to add a column with the mean back to stocks:

(def stocks-with-mean
  (-> stocks
      (tc/group-by [:symbol])
      (tc/mean [:price])
      (tc/rename-columns {"summary" :mean-price})
      (tc/inner-join stocks :symbol)))


stocks-with-mean
;; => inner-join [560 4]:
;;    | :symbol | :mean-price |      :date | :price |
;;    |---------|------------:|------------|-------:|
;;    |    MSFT | 24.73674797 | 2000-01-01 |  39.81 |
;;    |    MSFT | 24.73674797 | 2000-02-01 |  36.35 |
;;    |    MSFT | 24.73674797 | 2000-03-01 |  43.22 |
;;    |    MSFT | 24.73674797 | 2000-04-01 |  28.37 |

Then we use a dataset column operation that returns a column – column division, in this case – to add a new column holding the relative daily price of the stock:

(-> stocks-with-mean
    (tc// :relative-daily-price [:price :mean-price]))
;; => inner-join [560 5]:
;;    | :symbol | :mean-price |      :date | :price | :relative-daily-price |
;;    |---------|------------:|------------|-------:|----------------------:|
;;    |    MSFT | 24.73674797 | 2000-01-01 |  39.81 |            1.60934655 |
;;    |    MSFT | 24.73674797 | 2000-02-01 |  36.35 |            1.46947368 |
;;    |    MSFT | 24.73674797 | 2000-03-01 |  43.22 |            1.74719814 |
;;    |    MSFT | 24.73674797 | 2000-04-01 |  28.37 |            1.14687670 |

For more information, on these operations, please consult the documentation here.

Thanks to Clojurist Together

This contribution to Tablecloth was supported by Clojurists Together through their Quarterly Fellowships for open source development.

Summary

There are a lot of links in this post. This table is an attempt to aggregate and summarize them. There are more details worth reading below, but in case you don’t have time, this is the gist of it. To make a very long story short, current efforts are heavily focused on consolidating all of these amazing libraries into one (or at least a small number of clearly delineated ones) that is/are easy-to-use, providing a comphrehensive toolkit for doing machine learning in Clojure.

In addition to all of these libraries, the post mentions the Clojurians Zulip, the main Clojure-for-data-science community discussion forum, where main contributors to the ecosystem are active daily.

Java ML Libraries

There are two (sort of four) popular Java libraries that implement many of the main algorithms and tools used in machine learning today (e.g. classification, regression, clustering, model development, etc.): Tribuo (including XGBoost, more on that in a second) and Smile. We count Smile as two libraries because Smile 2.x is LGPL-licensed, and Smile 3.x is GPL-licensed, which poses some potential conflicts for some end users. The community consensus is converging around moving away from Smile due to the GPL-relicensing issue, focusing instead on Tribuo and hand-rolled solutions.

There is also XGBoost for the JVM, mentioned above, which is an implementation of gradient boosting. XGBoost is a collection of algorithms whereas Tribuo is a more comprehensive framework (including things like data management, model evaluation, and experiment tracking). XGBoost can be used from Tribuo, so I don’t exactly count it as a standalone library, although it can also be used in that way.

Clojure wrappers

There are two main “families” of libraries that wrap these Java ML libraries in Clojure.

Fastmath includes statistical as well as machine learning tools for Clojure. Fastmath 2.4.0+ depends on Smile 2, and the forthcoming fastmath 3.x will have no Smile dependency at all. The clustering functionality in fastmath 2.x that depended on smile has been moved to the fastmath-clustering library, which will have a Smile 2.x dependency going forward. There is a strong preference in the community to avoid introducing GPL-licensed libraries into the ecosystem.

Clustering functionality will mostly be provided by scicloj.ml.tribuo going forward which, as you might expect, wraps the Tribuo Java library, and is likely to become the main source of ML algorithms for the ecosystem. This is one of a few libraries in the second family of libraries that wrap the Java libraries mentioned above. Other (self-explanatory) ones include scicloj.ml.smile, which wraps more of Smile than fastmath did (does), and scicloj.ml.xgboost.

It’s also worth mentioning tech.ml.dataset (the core dataframe/dataset library underlying tablecloth), which incorporates some of the functionality of tribuo, with the API centred around individual datasets. There also used to be a library called tech.ml, which implements some machine learning tools, but has been deprecated in favour of the various libraries discussed above.

The concept of orienting an API around individual datasets vs something else leads me to the next group of libraries.

Clojure ML Pipelines

Metamorph is a library that implements a function composition mechanism for composing ML pipelines. It arises from the common ML practice of repeatedly running the same set of functions with varied parameters. You might, for example, try many different test/train splits to see how that affects your results, or fit the same data using many different algorithms, or try training your model using different sets of features. This leads to an explosion of pipeline combinations, so it’s useful to have machinery to encapsulate the variable components of your ML pipeline into a single function. This is where metamorph.ml comes in.

Metamorph.ml is based on this concept of meta-functions and pipelines. It is currently the central library for orchestrating ML pipelines in Clojure. The API is stable, but there are currently many ways (10+) to achieve the same outcomes. This is great for power users who have complex needs and a clear understanding of the metamorph mental model, but it can be a bit daunting for newcomers, making it more challenging to pick a clear place to start. The community is actively discussing the best approach for consolidating and/or documenting these different approaches in the interest of making Clojure’s ML stack more accessible.

Collections/Frameworks

The community is well aware that it is difficult to know where to get started and several efforts have been made in an attempt to make the path more clear for people who want tools that Just Work. scicloj.ml is one such project. It’s a collection of libraries (mostly the ones mentioned above) with some lightweight wrappers and efforts in creating documentation.

The community is heading toward deprecating this library, though, in favour of noj, which we are hoping to stabilize in the near future. The goal is to have a single entry-point into the Clojure data science stack, gathering all the tools one would need to work with data consolidated in one place, with seamless interoperability akin to R’s tidyverse of libraries.

Interop

It wouldn’t be a complete roundup of the state of ML in Clojure without a mention of libpython-clj. This is a library that provides Python bindings for Clojure, so you can call Python code directly from Clojure if necessary. sklearn-clj makes use of this bridge to provide direct access to all of the estimators and models from Python scikit-learn in Clojure, so for cases where something is truly only available in Python, we can still access it.

It’s worth also briefly mentioning clojisr here, which is a similar kind of bridge from Clojure to R (and there exist libraries for Julia and Wolframite, too), but these are all less relevant for the specific area of ML, where Python is the overwhelmingly most popular current tool of choice.

More updates

These discussions all happen in the open, on the Clojurian’s Zulip instance, which has become the main gathering place of the Clojure-for-data-science community. The #data-science and #noj-dev streams are the most active on these topics at the time of this writing. You can follow along with developments in the trenches over there, or follow the key libraries on github for updates (scicloj.ml.tribuo, metamorph.ml, noj). I will also post periodic updates here and all the other corners of the internet where I lurk. Thank you for reading!

Discuss this post on Hacker News or Reddit