CLJ Commons: Building a formatter like gofmt for Clojure


#81

Maybe we should start listing some examples of what cljfmt does and doesn’t do?

For example, it has specific indent and block rules for certain forms here: https://github.com/weavejester/cljfmt/tree/master/cljfmt/resources/cljfmt/indents

But I don’t know what it defaults too otherwise.

I’m not sure I follow people’s objections against having more readable rules for common forms. But say that’s where we wanted to go, it could just be that we need to create a bundle of cljfmt with no specific rules.

It’s probably like a day’s work to wrap cljfmt into a container that freezes its options. So it would be easy basically to create a cljfmt bundle that prevents any configuration, and defaults it to a canonical one we all agree on. And then made a native-image of that for fast startup time.

Because of how simple and quick this option is, i feel it be nice to discuss cljfmt more. Can we not make it work for our use case? And if so, why not, give some examples, concrete reasons of its behavior, etc.


#82

If the prevention of configuration is not of value in itself, cljfmt can already be instructed to ignore its built-in rules using the ^:replace hint. So this project can provide its own rules.

As I am hoping we will go for Tonsky’s rules I tried to make cljfmt adhere to them, to build a formatter for VS Code where people can try those rules out, but run into what is probably a bug in cljfmt: https://github.com/weavejester/cljfmt/issues/154 I would greatly appreciate help with fixing that issue.


#83

I’m starting this project more from the end of defining a formatting spec that can meet the goals outlined in the initial post. Once we know what format we’d like to pick, we can then examine existing tools to see if any are close and can be modified to match the spec. I don’t want to just pick a tool and say “This is the tool”, because that defines the formatting spec as the implementation of that particular tool. Rather, we should think carefully about what an ideal world would look like, and then work towards it.

There are many different contexts that formatting Clojure code needs to operate in, I think it would be useful for there to be a spec and set of test cases defined so that people can write their own implementations. As an example, Cursive, CIDER, and Calva all have JVM parts to them, but I’m not sure if the architecture of these editors would suit just dropping in cljfmt.


#84

Could still be useful to start listing example code snippets, and use cljfmt defaults as the baseline. Then people could argue on a per-example basis what they dont like about the way its indented, aligned and line broken, and what they’d prefer.

By the way, I use indentation to mean how much space are in the beginning of the line. I use alignment to specify how much space to have in between symbols, and line breaks as where line breaks should be put. You probably also can have blank line adjustments, as the number of blank lines between forms. There’s also normalization, such as trimming of whitespace at the end, conversion of tabs to spaces, etc.


#85

Hey there!

For those of you who don’t know me - I’m the author of CIDER, the editor of the Community Clojure Style, and in the Ruby community I’ve spent years working on a formatter and linter for Ruby code (RuboCop) and a Ruby style guide that goes with it.
I think all of this gives me a somewhat unique perspective, as I’m both really passionate about setting up (community) standards and I’ve also experienced how painful all of this can be in practice.

I’m a couple of weeks late to the party, but here are my thoughts…

On the original proposal

I agree that ideally the tool should have no configuration options, but I doubt that’s feasible in practice. When I started work on RuboCop many years ago, it wasn’t configurable and many people were outraged by this. The Ruby community was 15 years old at the time, many formatting patterns existed and very few people cared about global code consistency - most cared about getting consistency in their projects, and of course - with their own style preferences. Luckily for us Lisp’s semantics are much simpler than Ruby’s. :slight_smile: Making RuboCop configurable was instrumental to its wide adoption in the Ruby community - we never got complete alignment in the style department, but we got some alignment and this definitely beats none. :wink: I know some people are still trying and maybe they’ll succeed at some point, but I lost my desire to participate actively in this, as I’m tired of endless debate over trivial matters.

I think that for the proposed formatter tool to be useful in editors it should certainly be able to operate on lines (and groups of lines), as reformatting the whole files all the time is somewhat annoying for users and not always an option in the first place. That’s probably the biggest reason why most editors have their own indentation engines - they give them the most flexibility with respect to whether you want to reformat everything, just a few lines, etc.

Some people might think that’s not a big concern, but few established projects would accept global changes to indentation and formatting just for the sake of uniformity. Clojure is well-known to be one of them. :slight_smile:

I guess for editors a great “API” would be - you send a filename and a range of lines (or characters if you want to be extra granular) and you get their formatted version. If it’s configurable it might be nice if editors can override some of its configuration options (prettier style).

Nikita’s Proposal

I’m an old-time Lisper and the proposal definitely made me grind my teeth. I understand just as well as everybody else that this is the simplest (and the only way without some extra formatting specs) way to achieve uniformity, but I think the price we’ll pay for this is
semantics.

“Special” indentation rules usually exist to related “special” semantics. Once you put everything under the same denominator you’re in effect saying that the semantics don’t really matter, which is always debatable. I might expand on this in a separate blog post if I find the inspiration to write one.

I don’t agree there are some rules in existing standards that probably can be simplified, but overall similar constructs have reasonably similar formatting rules and the only real problem is relaying formatters those semantics.

The approach of metadata has always been appealing to me, because:

  • it’s self-documenting
  • it’s easy to extract this info when doing static analysis
  • you have the info at your disposal when doing typically REPL-driven development (as CIDER does it)

It’s also something that Common Lisp and Emacs Lisp have proven to work well - but there we have universal consensus about the metadata and tools that understand it.

Community Uptake

I think that adopting an universal formatter/code style 10 years into the existence of a language is unlikely to (fully) succeed if it’s not driven from the top. There will always be strong opposition to whatever we decide, as people have built strong preferences at this point and they’ll need extremely compelling arguments to change them. Changes is hard, and no one really wants to deal with it, especially if they don’t have to.

gofmt succeeded mostly because it was pushed from the top, pushed from the start and everyone was expected to use it. I’m reasonably sure this ship has failed for Clojure, just as it has sailed for Ruby. I don’t know how successful the similar projects for other languages are. I know only that Prettier is quite successful, but it’s also configurable to some extent and it didn’t really propose anything novel in terms of formatting.

That’s why I think that the only way for a tool to gain much traction would be if it’s aiming to enforce something relatively close to what people are doing currently.

Conclusion

I think a formatting tool that can be used from editors would be quite useful, so I’m looking forward to hearing someone create one, although if we set on a simplistic indentation scheme obviously it renders much of the need for such tool redundant, as any editor can trivially implement this (Emacs/cloure-mode has been supporting this “consistent” indentation for years now).

I think we should also set our expectations accordingly about what can be achieved in a mature community - a common formatter would probably have some uptake (as proven by cljfmt), but it won’t be adopted by everyone (also proven by cljfmt).


#86

What about “align function arguments by default”. Am I the only one having trouble with this? I mean, it only works for really short functions, like or or and. Not for something like my-namespace/my-method. This would lead to a code indented way too much. Maybe defaults should be reversed here? No aligning is the default, aligning carries a special meaning?


#87

I think this came from the fact that people were looking for a way to differentiate a list literal for from some function call (and perhaps to highlight a function’s name), but that might be just a guess. I’ve noticed that people typically write multi-line literal lists with one element per line, but for functions they’d use the alignment you mentioned. If I had to speculate - perhaps this was influenced from how most Algol-like languages do such indentation. This arguably makes the name of function stand out and the code becomes easier to process by a human reader.


#88

If this existed, I would gladly use it.

Every few years, I try out all the latest Clojure IDEs, and switch to whichever one works best for me at the time. It has been frustrating for me when the IDEs disagree over how to format my existing code base. I would love to have one standard.


#89

I have strong opinions about code formatting, but even stronger is my desire to not think about it. I agree 100% with Stuart Sierra’s “How to ns,” but if there was a fast, standalone, zero config, Clojure code formatter that did the “wrong” thing, I would use it (probably with bitterness in my heart, but I would use it), because I care, but I want to not care.

Like, do I think that vertically aligning let forms is easier to read…probably…I think it could be problematic because it would shift things further to the right, but mostly I don’t do it because it is annoying to have to do that all manually. If there is a tool doing it, I am much happier.

Actually, my ideal world would be a pre-commit hook that condensed things down into some kind of whitespace minimal canonical format, and a post-checkout hook that automatically formatted everything the way I like it, others could have their own personal view. That combined with a tree differ. I don’t even want to diff lisp code as lines of text.

I want to encourage this effort and not discourage it, but I do want to point out that the fact that you have to understand code to format it is a feature not a bug. A file of Clojure code is essentially a REPL script. Each form interacts with all the lovely, living objects that were created in memory by all the previous forms. You can define a macro and then use it in the very next form. You can add metadata to things and use it for formatting. Clojure code is best written with an active connection to a REPL.

I know this makes static analysis hard, and causes heartache for tool maintainers like Colin, but that’s the way I see it. :smile:


#90

As a golang developer using gofmt I can’t imagine a language not having an canonical source code formatter with no settings at all. It’s simply part of the language: no controversies, no useless debates and tiresome and pointless style flame wars, no frustrating code reviews because my editor changed the formatting with respect to yours. What you call dictatorship, I see it as a practical, pragmatic move to make developers lifes easier when dealing large clojure code bases within a team of more than one developer.


#91

Does gofmt rewrites whitespace also? Like if I put a newline somewhere, would it get rid of it? So I can’t even control line wrapping and spacing?


#92

No it’s not very rigid wrt empty lines


#93

I’d argue that it is harder to read. You seldom start looking at the values, but instead read the map as an index table, scanning the index (keys) and then looking up the value. If the values are aligned, then you’d have to trace the line from the key to value. And there are no visual aids (like the grid lines in a spreadsheet) to help you with that tracing.

(edit: including the quote I’m replying to)