Beginner trouble with spec fdef

Questions & Help How to?
#1

I’m reading the guide on how to do instrumenting of functions via spec, but I am kind of stuck. The spam function is just made up, its contents is not important.

I got the :args to check for the types, but I really wanted to compare the length of the data to the stop parameter. My attempts there failed to compile.

:fn and :ret in the spec does not seem to do anything. I can change the :ret int? to :ret string? to no effect.

I want the spec to fail if the data is too short.

Anybody got any ideas?

Edit:
I got a working version that uses :pre in the function itself. But I wanted the check in the spec.

(ns test-fdef
  (:require [clojure.spec.alpha :as s]
            [clojure.spec.test.alpha :as stest]))

(s/fdef spam
  :args (s/cat :start int?
               :stop int?
               :buf bytes?)
  :fn #(>= (-> % :args :buf alength) (* 3 (-> % :args :stop)))
  :ret int?)


(defn spam [start stop data]
  (println start stop data)
  1)

(stest/instrument `spam)

(spam 1 10 (byte-array [1 2 3])) ; I want this to fail (passes)
(spam 1 2 (byte-array [1 2 1 2 1 2])) ; I want this to pass
#2

instrument is for verifying that calls to a given function are correct (via :args).

check is for verifying that a given function is internally correct (via :fn and :ret).

The former is good for enabling during development and (unit) testing.

The latter is good for standalone use during development and longer-running tests, since it relies on generative testing.

This section of the docs explains this difference: https://clojure.org/guides/spec#_instrumentation_and_testing

1 Like
#3

And if like most Spec users, you’d want convenient instrumentation of ret and fn spec as well, because you’re often lazy and don’t re-run check or haven’t setup a generative test for your function, well you can use this library as a drop-in replacement for instrument: https://github.com/jeaye/orchestra which will instrument functions so they validate the full function spec, not just the arg spec.

#4

I would definitely challenge that. I think “most Spec users” use Spec the way it was (very deliberately) designed and sensibly keep call verification separate from behavior verification :slight_smile:

That said, I gather than fdef is an area that is going to see a redesign in Spec 2 (which has a number of improvements over Spec 1 already but is still being revised in hammock mode :slight_smile: ).

#5

I was definitely being a little cheeky on purpose there. I can’t say the true statistics, but Orchestra has been downloaded half a million time on Clojars, so it’s definitely popular.

Personally I didn’t agree with the change to instrument, I understand wanting to encourage generative testing, but since instrument is meant to be used only for testing and development time anyways, I’d rather get faster feedback loop from it. Yes maybe I’ll eventually re-run check on my function, but I will much faster catch a small bug at the REPL if instrumentation instrumented ret and fn as well. And for some functions, check takes forever, and has a really high level of ceremony (and effort) involved in getting working properly with the right set of generators and all that.

#6

Then this is on me, for not reading thoroughly enough. Thanks

#7

Turns out this functionality was what I wanted. I’m also lazy :slight_smile:
But on that note, I still don’t quite get why there is all the options of :args, :ret, and :fn, when :fn kind of has it all. But maybe that is another discussion altogether.

#8

A lot of the time, folks just use :args and instrument to check calls are made correctly.

If you can describe (aspects of) the behavior of the function, using :fn makes sense for generative testing, but it’s not always possible to describe the behavior without re-implementing it (which defeats the purpose).

If you can’t easily describe the behavior of the function but you can state properties of the result value, using :ret makes sense for generative testing.

#9

:fn does have it all, but its not logically grouped. So I think it is easier to think in term of:

  1. What are valid arguments? (:args)
  2. What are valid return values? (:ret)
  3. What are known properties of input to output? (:fn)

Since Specs are also meant as documentation, separating things like this I think is much cleaner. Also on failure, you know which of those 3 things you did wrong at a glance, since it’ll say: Failed in :ret or Failed in :fn, etc.

1 Like