Clojure and Lisps in general have historically allowed very flexible formatting of s-expressions. This can aid readability, but adds a cognitive overhead for readers used to different styles. It can also be challenging to match existing source code formatting if you are using a different editor to the original author.
I believe it would be useful for the Clojure community to be able to develop (or adopt) a single source-code formatter which is able to format Clojure source code to a canonical format. The purpose of this thread is to help develop the problem space, hear from different stakeholders, and determine whether such a tool is desired, possible, and likely to be useful. It seems unlikely that 100% of the community would want such a tool, but I feel like there is enough desire for a common formatting tool that this could still be valuable.
The goals and thoughts put down here are a starting point for discussion, not the final word. I’ve been thinking about this for a while, but there are lots of other people who have also thought about this kind of thing. Many Clojurists bring valuable experiences from other language communities. I’m really interested in finding common ground to build a tool that the community can get behind.
This effort would be part of CLJ Commons, a community effort to build up the supporting infrastructure around Clojure to make a better experience for Clojurists.
Why not use “existing tool X”?
There are several existing tools for formatting Clojure source code: cljfmt, zprint, emacs, fipp, Cursive (and other editors have formatters too). Each of these doesn’t quite fit the goals I have for this project.
- zprint is extremely configurable which is great for some use-cases, but doesn’t move towards the goal of having a single common format. (zprint looks like a very good base to build this kind of tool on though)
- cljfmt doesn’t have a goal of providing a canonical format
- emacs and Cursive formatters are both part of the editors and don’t have an easy way to run outside of the tools.
- fipp isn’t currently suited for code formatting, but could be in the future with more work.
These are the stakeholders I’ve identified when thinking about building a tool like this. For a tool like this to get adoption it needs to have support from a wide section of the community, not just a single stakeholder or tool.
- Clojure developers, i.e. You!
- IDE authors: CIDER, Cursive, Counterclockwise, Calva, e.t.c.
- Other tooling authors, e.g. Parinfer, cljfmt, zprint
- Clojure Core may want to provide input
- Others? Who else should be involved here?
Here are some of the aspirational goals and outcomes I could imagine coming from this:
- Fast cold start time
- Fast to run - 10-100k LOC/second seems ambitious, but probably doable.
- The production of a reference implementation formatter
- The production of a specification which different editors and tools can use to implement a common code formatting style. This spec should be independent from the reference implementation, i.e. the formatting rules are not defined by the behaviour of the formatter.
- The spec should be opinionated over being flexible, providing the fewest config rules as possible, ideally none.
- Creating tooling to be able to report deviations in continuous integration, pre-commit hooks, e.t.c.
- Able to run on a single file, maybe even a subset of a file?
- Able to run without having to evaluate the Clojure code
- Works across Clojure, ClojureC, and ClojureScript
- A free service that can be installed as a Check in GitHub to check formatting and suggest formatting changes for open source projects.
- Able to provide the same output even in the face of many whitespace changes, i.e. whitespace (mostly) doesn’t matter
- An online playground for reformatting people’s Clojure code, similar to the Prettier playground
- Ability to use the CIDER indentation specification for controlling custom indentation.
- Ability for IDE authors to build tooling that follows the spec. The supporting tooling has to be a first-class citizen.
- Go, The Cultural Evolution of gofmt (this is particularly good on selling the benefits of gofmt)
- Black for Python
- Elastic Tabstops
- Formatting Clojure source code
I haven’t seen many of these kinds of very opinionated formatters for Lisps, but I’d be very interested if anyone knew of any. Does the Lisp culture select against these kinds of rigid tools?
Contexts where formatting needs to run:
Formatting happens in different places, we should design a solution which can work well for these different contexts.
- Running a reformatting command in an editor
- Typing in an editor
- Command line usage for detecting format deviation
- Command line usage for fixing format deviation
- In an online playground like environment?
- Consistent formatting when reading code
- Consistent code formatting amongst team members using different editors (or even the same editors!)
- Reduced git diff noise when making changes as formatting and whitespace is consistently applied
- Eliminate time spent worrying about formatting, or nitpicking it in code reviews
- Maintaining compatibility with any particular code base
Things to consider when making decisions about formatting rules:
- Readability of the code
- How it impacts common code idioms, i.e. look at examples of what it would do to real code
- Git diff impact when things change, e.g. lining up map values will often make extra whitespace changes if you add/remove map keys
- Community conventions, both written and unwritten.
- Pathological cases
- Difficulty to implement
- Lisp heritage
The purpose of this thread isn’t to figure out the answer to all of the different formatting decisions that would need to be made, but here are some of the kinds of big and little decisions that would need to be made.
- Do we only support UTF-8 files?
- Should we break lines at a certain line length or not? If so, should the line length be configurable? - https://news.ycombinator.com/item?id=17273616
- Should trailing whitespace be removed?
- Should we format to a single trailing newline at the end of a file?
- Do we want to reorder
nsforms to follow something like Stuart Sierra’s style guide? (I’m really in favour of this personally)
- Do you remove the whitespace on a blank line between two indented lines or keep it in?
- Should there be a space between
#_and the next form? What should happen with multiple
- How do we deal with comments? Is there a difference between
- How should reader conditionals be formatted? Should the conditional itself be outdented so that the code flows better, or just treat it as a regular form?
- What point on the continuum of formatter and linter do we want to hit?
Further tools that could be built from this
In the future, I could imagine this kind of tool may be useful for other tooling that works with source code like:
- Building automatic rewriters for upgrading libraries or tools, e.g. an automatic migrator from
- Linting Clojure code (e.g. Eastwood)
- Spotting better Clojure idioms (e.g. Kibit)
The process I imagined building this tool would work would be:
- Discussion continues on this thread from interested parties about the idea
- Find a group of people who want to be part of building it
- Do a survey of existing formatting tools to determine if one of the existing ones is suitable to adapt/modify/collaborate with on the goals of this project
- Work together to figure out the shared values and goals of the project, to make sure we are aligned before beginning work
- Start a GitHub repository in clj-commons and create some issues for the different formatting rules that would need to be decided
- In parallel, start building/adapting a formatter to implement the formatting rules. At this point it might be useful to do a few spikes in different directions to see what is the most promising route.
- Find common ground on formatting rules and incrementally add more and more rules over time, releasing early and often, developing a spec and a reference implementation. I had thought that starting with an ns formatter based on Stuart Sierra’s guide (or similar) would be a really useful starting point, as I don’t think anything like that currently exists.
What haven’t I thought about? Does this tool already exist in a form I’m unaware of? Are there other people who we should be talking to about this? Do Clojurists value flexibility over regularity so much that you would never use a tool like this? Is such a tool impossible to make in Clojure? Is this a tool that you’d like to use? Is this a tool you’d like to help build? What are your thoughts?