Using :pre and :post for error handling and type checking

[ralf]

Hello,

Reading about error handling I was thinking about my use of try/catch and about proper use of :pre and :post.

I don’t want to use any special libraries for error handling and I would like to stick with Clojure basics. But in my case I have two options.

I’m chaining a couple of tiny functions in a main function. I don’t want to have multiple try/catch in the main function for every tine function that needs it. So I want to throw in those tiny functions.

Example of tiny function without error handling:

(def price-levels
  {:expensive 300
   :moderate 200
   :cheap 100})

(defn get-price-level [level-name]
  (level-name price-levels))

(get-price-level :moderate)
;; => 200
(get-price-level :wrong-name)
;; => nil

Basically it’s just an abstraction around :key map.

But the main function HAS to get a number from the tiny function since I’m not going to consider / handle / unit-test possibility of nil or anything else except number.

So I can write the tiny function as:

(defn get-price-level-2 [level-name]
  (if-let [price (level-name price-levels)]
    price
    (throw (ex-info "Unable to get price." {:level-name level-name}))))

(get-price-level-2 :moderate)
;; => 200
(get-price-level-2 :wrong-name)
; Execution error (ExceptionInfo) at user/get-price-level-2 (REPL:15).
; Unable to get price.

or I can leverage :post:

(defn get-price-level-3 [level-name]
  {:post [(number? %)]}
  (level-name price-levels))

(get-price-level-3 :moderate)
;; => 200
(get-price-level-3 :wrong-name)
;; Execution error (AssertionError) at user/get-price-level-3 (REPL:34).
;; Assert failed: (number? %)

or maybe also :pre but that is probably overkill.

(defn get-price-level-4 [level-name]
  {:pre [(keyword? level-name)]
   :post [(number? %)]}
  (level-name price-levels))

(get-price-level-4 :moderate)
;; => 200
(get-price-level-4 :wrong-name)
;; Execution error (AssertionError) at user/get-price-level-4 (REPL:33).
;; Assert failed: (number? %)
(get-price-level-4 "wrong input")
;; Execution error (AssertionError) at user/get-price-level-4 (REPL:33).
;; Assert failed: (keyword? level-name)

I like the :pre and :post syntax but one thing is bugging me - Clojure is dynamic. And I basically turned it into statically typed function.

Could you please tell me what is the Clojure-way? And what are Clojure-way approved use cases for :pre and :post?

Thank you, Ralf

[jogo3000]

I think your example is fine. That’s what they are for. Just be careful, AssertionError is not caught by catching Exceptions, as it is an Error. So you might want to catch Throwable when you handle it.

Usually static typing means that types are checked at compile time. That isn’t the case with :pre and :post so it is still a dynamically typed function.

[didibus]

This seems like unnecessary error handling to me. What exactly are you trying to do with it? Why not just let it return nil if there are no price-level at that level?

(defn get-price-level
  "Returns the price at the given level or nil if none are defined for that level."
  [level-name]
  (level-name price-levels))

Which is also why maybe this whole function is useless and you should just have the main function do:

(level-name price-levels)

[ralf]

@didibus

What exactly are you trying to do with it? Why not just let it return nil if there are no price-level at that level?

Thank you for asking that question @didibus, I didn’t explain it well.

I think I understand the benefits of nil-punning. The problem is that in this case I know for sure that the main function will not work if price-level is nil. So for example this can happen:

(filter #(when (= (:price %) price-level) %)
  [{:price 1500} {:price 2500} {:price 3000}])

and if the price-level is nil, the filter will return nil. The problem is that there will be more filters, maps, reduces and at the end something will fail. And then I’ll spend hours testing database, checking values in atoms, reading JSONs in files… since my assumption will be that something mutated since my pure functions are perfect. Not realizing that I made a stupid typo in :expensve.

So If I know for sure that the whole thing will not work when price-level is nil then I think I should fail at that point (fail early) with informative ex-info.

But I’m new to Clojure so I’m happy to be corrected . Thank you.

@jogo3000

Just be careful, AssertionError is not caught by catching Exceptions

Ouh, I didn’t know that. Thank you.

[seancorfield]

If you want a process to fail fast, it’s reasonable to use assert in some form (just don’t turn it off when you go to production!) and it’s reasonable to not catch an Error and just let it propagate to the top-level (and stop the process, or at least stop that request).

If you plan to “fail fast” but also recover from that somewhere, it’s better to use an exception, such as ex-info, with an explicit conditional check in the (production) code.

If you have a condition that is an expected “bad” value and you intend to handle that fairly directly, in the calling code, don’t use an exception – use a flag value of some sort: either nil (a natural choice) or some qualified keyword (such as ::not-found) that code can easily check for.

[ralf]

@seancorfield

If you have a condition that is an expected “bad” value and you intend to handle that fairly directly, in the calling code, don’t use an exception – use a flag value of some sort: either nil (a natural choice)

Thank you @seancorfield, in my simple example I would not even have to define a function for that since as @didibus correctly pointed out, I could just (level-name price-levels). The problem is that the calling function would look like this:

(defn my-calling-fn []
  (let [x (:key map)
        y (:key map)])

  ;; I really need x and y, it's pointless to continue without them
  (when (nil? x)
    (throw ...))

  (when (nil? y)
    (throw ...))

  ;; rest of code
  )

If I write the simple (and stupid) functions mentioned in the first post that combines something simple like (:key map) with (when (nil? x) ... )then I can structure the calling function.

(defn my-calling-fn []
  (->
   stupid-fn-dealing-with-x
   stupid-fn-dealing-with-y
   cleaver-fn-1
   cleaver-fn-2))

Which in my opinion is much nicer. But I’m also a beginner so what do I know

Thank you both @didibus and @seancorfield

[seancorfield]

In a pipeline like that, you could have each function return nil if it is passed nil (or whatever values it can’t deal with). It depends on what you need from the overall pipeline: an exception, a value, nil…

Figuring out the most idiomatic solution for each specific problem is something that comes with time and practice.

[didibus]

So if I understand, you’re worried something returns a nil somewhere and then you will have a hard time debugging which of the functions were the one to do so and cause the chain to return nil.

I think before we get there, I need to know, why would your main function pass a non existent level-name?

You need to distinguish you being worried of a bug in your code, and your program being given invalid input at runtime which you need to handle.

So, does level-name get derived from external input? If it doesn’t, then you don’t need to throw or anything like that, you just need to test your code and make sure it works and there are no cases where it would fail. So test for mistyped keys and all that. In that case, you can Spec your functions and have them instrumented at the REPL and when your tests run.

If level-name is derived from external input, what I recommend is to fail even faster. What is the external input you are given which will cause level-name to be wrong? Can you validate that at the point where the external input is given to you and error there?

Alright, now say you still want to be extra careful, and you’re not too sure why your functions could fail, and you just want to make it easy to debug in case something does. There’s a few approaches I’d recommend.

First off, what you did in get-price-level-2 is fine. The other use of pre and post don’t seem related, they fail on type errors, not on failing to find a price for the given level.

Using pre/post/assert to validate certain input and output are correct is also fine, but generally you would do it only at the REPL and test time, to make sure you don’t have bugs, not to assert things about the external input at runtime. That’s why it throws Error and not Exception, because it is designed to catch bugs, not exceptions. That said, it’s a bit annoying to disable at production time, as it can be tricky to do right, and I think it’s one reason their use isn’t very popular.

Another thing you can do is have your functions return nil like I said, but then use some-> as your threading macro. Then your my-calling-fn will short-circuit on the first nil returned and the whole thing will return nil. So you won’t get a strange exception somewhere else about a NullPointerException or anything like that you didn’t account for.

Now it doesn’t tell you where you first got the nil, but it let you handle that case.

Beyond that, (throwing an ex-info in your functions or short-circuiting on nil), you can venture in using a different threading mechanism either with a macro or higher order functions. I’m not gonna go there though, it’s a vast space, lots of people end up writing their own which work how they prefer.

[ralf]

@didibus

I think before we get there, I need to know, why would your main function pass a non existent level-name?

My mistake or mistake of somebody who might be using my library.

@didibus

You need to distinguish you being worried of a bug in your code, and your program being given invalid input at runtime which you need to handle.

Yes, this won’t be during runtime. It won’t be handling external input.

@didibus

So test for mistyped keys and all that

Can do.

So if I don’t explicitly throw on wrong input the calling function might:

• return nil - for example (= (:price %) price-level) will return nil on nil
• gives an exception - for example (< (:price %) price-level)

So that means I can test that the calling function works with :expensive and doesn’t work with :wrong-name and in that case returns nil OR has non-explicit (without me adding throw) exception.

But this feels like I’m not in charge… I’m just adding unit test results based on what I see that the calling function is doing in REPL which is not actual testing. That’s just making sure that the result doesn’t change after refactoring from nil to exception or from exception to nil.

And yes, I should know that (< 50 nil) gives an exception and (= 50 nil) just nil. I’m just thinking that with more complicated implementation (-> a b c d e f) I’ll get lazy and just observe what happens in REPL on wrong input instead of knowing that the it terminates since I explicitly throw.

@didibus

use some-> as your threading macro

I didn’t know that one, thank you. I like it, that feels that I’m in charge again ;-).

@didibus

Now it doesn’t tell you where you first got the nil, but it let you handle that case.

It’s different what I’m used to but that’s OK, Clojure / FP is different and I need to get used to that. Thank you.

@didibus

I’m not gonna go there though, it’s a vast space

Thank you, I wouldn’t want to get there either till I have a lot of code written and I really understand why I need it.

---

So my observation from your fantastic description @didibus:

1. Choose the simplest possible solution (the least amount of code) to get the work done.

2. A lot of conditionals with throw is a lot of code and therefore choosing something that can almost get you there (like some->) is the right way since -> and some-> is basically the same amount of code. But -> would be OK too, that’s the Clojure style to make is simple / short.