Nice error messages?

rmschindler · April 15, 2024, 8:34pm

what follows is an error message I get when my precondition assertion fails . I wonder, do y’all have anything in place that gives nicer error messages? . is there something I’m missing here which those-in-the-know utilize? . in this case, eg, I would love it if it left out the stack trace (— is that what it’s called, a ‘stack trace’?)

and please nobody make a case for why the error messages as they are are great . they are not great

------ REPL Error while processing ---------------------------------------------
(app.dom/evaluate "borf")
Encountered error when macroexpanding app.dom/evaluate.
AssertionError: Assert failed: (even? (count forms))
	app.dom/evaluate (dom.cljc:10)
	app.dom/evaluate (dom.cljc:10)
	clojure.core/apply (core.clj:671)
	clojure.core/apply (core.clj:662)
	cljs.analyzer/macroexpand-1*/fn--3704 (analyzer.cljc:4025)
	cljs.analyzer/macroexpand-1* (analyzer.cljc:4023)
	cljs.analyzer/macroexpand-1* (analyzer.cljc:4010)
	cljs.analyzer/macroexpand-1 (analyzer.cljc:4074)
	cljs.analyzer/macroexpand-1 (analyzer.cljc:4070)
	cljs.analyzer/analyze-seq (analyzer.cljc:4107)
	cljs.analyzer/analyze-seq (analyzer.cljc:4087)
	cljs.analyzer/analyze-form (analyzer.cljc:4296)
	cljs.analyzer/analyze-form (analyzer.cljc:4293)
	cljs.analyzer/analyze* (analyzer.cljc:4349)
	cljs.analyzer/analyze* (analyzer.cljc:4341)
	shadow.build.compiler/analyze/fn--13434 (compiler.clj:266)
	shadow.build.compiler/analyze (compiler.clj:254)
	shadow.build.compiler/analyze (compiler.clj:211)
	shadow.cljs.repl/repl-compile/fn--15150/fn--15151 (repl.clj:515)
	shadow.cljs.repl/repl-compile/fn--15150 (repl.clj:488)
	shadow.cljs.repl/repl-compile (repl.clj:486)
	shadow.cljs.repl/repl-compile (repl.clj:483)
	shadow.cljs.repl/process-read-result (repl.clj:574)
	shadow.cljs.repl/process-read-result (repl.clj:550)
	shadow.cljs.repl/process-input (repl.clj:713)
	shadow.cljs.repl/process-input (repl.clj:691)
	shadow.cljs.devtools.server.worker.impl/fn--15676 (impl.clj:698)
	shadow.cljs.devtools.server.worker.impl/fn--15676 (impl.clj:688)
	clojure.lang.MultiFn.invoke (MultiFn.java:234)
	shadow.cljs.devtools.server.util/server-thread/fn--15350/fn--15351/fn--15359 (util.clj:283)
	shadow.cljs.devtools.server.util/server-thread/fn--15350/fn--15351 (util.clj:282)
	shadow.cljs.devtools.server.util/server-thread/fn--15350 (util.clj:255)
	java.lang.Thread.run (Thread.java:840)

p-himik · April 15, 2024, 8:49pm

precondition assertion

Those are not meant to be friendly. It’s more like a last resort check in the “it’s better to go up in flames than to proceed” sense.
A nicer error message can be achieved when using the (assert ...) form manually: (assert some-condition "some message").
But assertion are meant to be a developer-only tool and can be disabled. If you want to provide an error message under all circumstances, it’s better to use an explicit exception:

(when-not some-condition
  (throw (ex-info "some message" {:some "data"})))

I would love it if it left out the stack trace (— is that what it’s called, a ‘stack trace’?)

Yes, a stack trace. There are some libraries that do that, e.g. some links here, along with a discussion: How to display better errors in clojure stacktraces? is this planned to be attacked on a new clojure version? - Clojure Q&A
But I would caution against using them. Becoming fluent at reading a Clojure stack trace is much more useful. And when a library hides some detail that it considers to be minute but that ends up being crucial, you’re bound to have a bad time.

and please nobody make a case for why the error messages as they are are great . they are not great

Except when the line number is screwed up (there are some tracked issues on the matter), the error messages are adequate when you learn how to work with them.

p-himik · April 15, 2024, 8:58pm

Just as a tiny sample of what I mean by being “fluent at reading Clojure stack traces”. Which are, of course, just Java stack traces with a bit of extra stuff on top.

When reading the error in your original post, you don’t even need to go further than the first four lines.
The zeroth line (added later, too lazy to change the following numbering) is not actually a part of the error message itself - it’s a part of the error report as implemented by shadow-cljs. And it just tells you that the form you’ve evaluated in the REPL didn’t work.
The first line tell you what code has failed exactly.
The second line tells you that app.dom/evaluate is actually a macro that has failed during its expansion.
The third line tells you what assertion has failed, and even without looking at the code of the macro I can guess with high certainty that app.dom/evaluate has a signature of [& forms] and expects an even number of arguments. So it should be something like (app.dom/evaluate "foo" "bar") or whatever.
The fourth line (and the last one that’s actually useful) tells you where you need to navigate to see that assertion.

And there’s absolutely no need to read that stack trace further for this particular error, so you can just ignore all those lines in their entirety.
However, there are most definitely cases when the whole stack trace is needed. In those case, omitting even things like clojure.core/apply can easily lead to an extra half an hour of investigation.

thheller · April 16, 2024, 9:41am

My question would be: How would you improve this error message?

I’d second that learning to read stack traces is essential. I don’t think this error message is all that bad. It shows you were to look and what failed. Granted it is far from perfect, but I also don’t know what could be done to improve it.

The challenge with stacktraces is that its not always easy to know when to cut them off and omit the rest from the tooling side. I personally get frustrated if anything is cut off at all, since most often some clue is missing when trying to diagnose the causing error. I also know what to ignore though, so I’m not the best judge.

teodorlu · April 17, 2024, 3:50pm

I’ve used Elm quite a bit, and absolutely love the error messages in Elm. They are consistently short and helpful.

But I think the reason Elm’s error messages can be so awesome is that Elm is a walled garden. You can use what’s in the language.

Clojure is designed as an open system. If you’re on the JVM, you can call into Java libraries. If you’re on Javascript, you can call into Javascript libraries. With that in mind, I don’t think it’s possible to consistently provide both short and helpful error messages.

thheller · April 18, 2024, 6:56am

Here is the perfect case study why dropping stacktraces is almost always bad.

@lilactown reported an error with a reproduction with a seemingly weird error, which turns out to be a bug.

 File: /Users/lilactown/Code/shadow-ns-map/src/app/a.cljs:9:15
--------------------------------------------------------------------------------
   6 | #_(def data {::b/x 1 ::b/y 2 ::b/z 3})
   7 |
   8 | ;; broken
   9 | (def data #::b {:x 1 :y 2 :z 3})
---------------------^----------------------------------------------------------
No namespace: app.b found

Nothing else in that error message, so everything we could do was guess. Which rarely leads somewhere useful on the first few tries.

As the first step I re-enabled the full stacktrace in shadow-cljs and that yielded this, which enabled me to pin down this error in 2 minutes.

  30 | (def data #::b {:x 1 :y 2 :z 3})
---------------------^----------------------------------------------------------
No namespace: app.b found
	clojure.tools.reader/read* (reader.clj:953)
	clojure.tools.reader/read* (reader.clj:917)
	clojure.tools.reader/read-delimited (reader.clj:198)
	clojure.tools.reader/read-delimited (reader.clj:191)
	clojure.tools.reader/read-list (reader.clj:209)
	clojure.tools.reader/read-list (reader.clj:205)
	clojure.tools.reader/read* (reader.clj:935)
	clojure.tools.reader/read* (reader.clj:917)
	clojure.tools.reader/read (reader.clj:988)
	clojure.tools.reader/read (reader.clj:961)
    ...
Caused by:
Exception: No namespace: app.b found
	clojure.core/the-ns (core.clj:4163)
	clojure.core/ns-name (core.clj:4165)
	clojure.core/ns-name (core.clj:4165)
	clojure.tools.reader/read-namespaced-map (reader.clj:762)
	clojure.tools.reader/read-namespaced-map (reader.clj:753)
	clojure.tools.reader/read-dispatch (reader.clj:72)
	clojure.tools.reader/read-dispatch (reader.clj:68)
	clojure.tools.reader/read* (reader.clj:935)
	clojure.tools.reader/read* (reader.clj:917)
	clojure.tools.reader/read-delimited (reader.clj:198)
	clojure.tools.reader/read-delimited (reader.clj:191)
	clojure.tools.reader/read-list (reader.clj:209)
	clojure.tools.reader/read-list (reader.clj:205)
	clojure.tools.reader/read* (reader.clj:935)
	clojure.tools.reader/read* (reader.clj:917)
	clojure.tools.reader/read (reader.clj:988)
	clojure.tools.reader/read (reader.clj:961)

This has all the clues needed to follow back where it goes wrong. It is at clojure.tools.reader/read-namespaced-map (reader.clj:762). I didn’t even need a debugger, just a quick skip through the code following the mentioned locations.

I shortened the stacktrace since actually it is a lot longer, but I could do that since I know how to read them. I could not do this from the code in shadow-cljs, since this is a Exception. By definition this is unexpected, so not something the code could account for beforehand. A generic filter would filter all clojure.tools.reader traces would have hidden this again, although in most cases it would probably be fine to do that.

I’m not saying that stacktraces should always be shown, often they do not yield any additional useful infos. Problems often may not be directly in your code, but rather some library where you maybe passed in some wrong data and it only manifesting way down the line.

So the gist is that it is tricky to do this right. Does the end user get something useful out of this stacktrace? Absolutely not, but without it is also really hard to debug for anyone else.

hlship · April 18, 2024, 3:18pm

I think there is a middle ground - which is why I created Pretty when I first started using Clojure professionally.

The stack trace is a wealth of information, in an unwieldy, intimidating jumble.

Pretty reformats the exception, demangles the names, puts things in readable and easily scan-able columns, combines the individual traces together, and flips it all into a better order. Further, most exceptions carry data to help identify the problem … pretty pretty-prints that with the exception.

For my mind, doing this in your head, every single time, is exhausting - pretty mechanizes much of the work you need to to do analyze an exception.

corbathegeek · April 21, 2024, 5:43pm

Thanks for the link to ‘pretty’, hlship. Very nice!

It reminds me of a mainframe addon that a retired friend once described to me. Mainframe “abends” (abnormal end) were notoriously large and hard to analyze. They were called “dumps.”

Along came a company that created a software hook that intercepted the abends and digested and reported the essential parts. If that wasn’t enough, a mainframe developer could always access the full dump. The product was named AbendAid and became massively popular.

On a personal note, I have been programming in Java for 15+ years and Clojure for ~4. In spite of my familiarity with stacktraces, I still find Clojure’s error messaging off-putting. So I have to agree with rmschindler.

habruening · April 29, 2024, 6:43pm

To me the question looks like misunderstanding for what exceptions and assert are made for.

You should never present exceptions to the user.
You should never use exceptions for things, which you ever expect to happen.

Let’s see an example. You ask the user for a file name and then you want to open that file. In case that file does not exist, you may think that you just catch the IO Exception. The majority of people in OO languages would tell you, this is absolutely wrong and not acceptable. Using exceptions for error reporting is not how exceptions should ever be used. Exceptions should never participate in your normal control flow. Exceptions are not a programming technique. It is not so unusual, that a user gives you a file name, that does not exist. So you must foresee this in your regular program code with explicit checks.

Exceptions are not a mechanisms for error reporting. They are a mechanism for bug reporting. An exception or a failing assert is always a programming error. So it should be helpful for programmers and not for users. And this is why they look how they look. They give you the information, that the software developer need: Where did the problem occur and in what context. And this is the stack trace. Exceptions are not a tool for telling the end user what he did wrong.

Maybe you find it sometimes comfortable to use exceptions for other purposes. You have that choice. And maybe Clojure people are not so strict. But then they use exception for purposes for which they were never designed. So they cannot expect user friendly error messages.

seancorfield · April 30, 2024, 4:47am

No, this is simply not true in the Java ecosystem.

There are two subclasses of Throwable:

Error – these are things which should never happen: “unexpected” failures,
Exception – these are things which you absolutely should expect to happen, and should expect to catch and handle.

Sorry, but this is just wrong advice. Java is very explicitly designed to have “expected exceptions” and “unexpected exceptions”. This stuff is baked right into the signatures of library functions: you absolutely must take this into account in your “normal” control flow.

seancorfield · April 30, 2024, 5:12am

And just in case anyone thinks I’m being a bit harsh here, the Oracle Java documentation says:

An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.
The class Exception and its subclasses are a form of Throwable that indicates conditions that a reasonable application might want to catch.

habruening · April 30, 2024, 2:39pm

As this is the Clojure forum, we can be happy, that we don’t have to discuss or agree on such topics. For OOP you read thick books until you think, that you understand something. And then when you try to reproduce it, someone comes and claims the opposite.

You may be right. There are indeed things, where an Exception is the only way to check something. If you want to check if an XML file is valid XML, the only way is to parse it and see if you get an Exception. However, this does not mean, that this is good or bad. But that is not relevant for the question. So please ignore my last answer! After your reply I would have to reflect my answer more.

But I suppose, we all agree, that in Java end user should not see exceptions and that this explains why exception looks like they look.

seancorfield · May 1, 2024, 4:10am

Yes, absolutely agree: both Exception and Error should not escape to your users. Exception probably shouldn’t ever make it to the top-level, Error likely will, but in both cases something at the top-level needs to catch Throwable and present the user with “Welp, sorry, I tried…”

thheller · May 1, 2024, 7:12am

Agreed, although I don’t quite see the point in the context of this discussion.

This isn’t about end users. This is about a development tool showing an exception to the developer, which IMHO should absolutely see all of it.

hlship · July 3, 2024, 9:45pm

I do know that there has been abuse of exceptions for control flow in the past; people have used exceptions almost as a goto, to jump up a few stack frames when certain conditions occur. This is a very, very bad practice, as exceptions are extraordinarily expensive to create, throw, and catch.

ShalokShalom · December 23, 2024, 7:39am

First: Thanks for creating and updating this package, it looks incredibly useful to me.

(Why is this not merged into the language yet )

Second: I found this eye opening statement:

And this accompanying paper on it