Implementing spec for arrays

clojure_spec

#1

Hi everyone!

I’m just starting to use clojure.spec and I’m curious how to write spec for an array of numbers. Here’s what I have so far.

(ns playground.spec
  (:require [clojure.test :refer :all]
            [clojure.spec.alpha :as s]
            [clojure.spec.test.alpha :as stest]
            [clojure.spec.gen.alpha :as gen]
            [clojure.spec.alpha :as spec]))

(defn number-array? [o]
  (let [c (class o)]
    (and (.isArray c)
         (or
           (identical? (.getComponentType c) Long/TYPE)
           (identical? (.getComponentType c) Integer/TYPE)
           (identical? (.getComponentType c) Double/TYPE)
           (identical? (.getComponentType c) Short/TYPE)
           (identical? (.getComponentType c) Float/TYPE)))))

(defn insertion-sort!
  [arr]
  (doseq [i (range 1 (alength arr))
          :let [val (aget arr i)]]
    (do
      (let [j (loop [j i]
                (if (and (> j 0)
                         (> (aget arr (- j 1)) val))
                  (do
                    (aset arr j (aget arr (- j 1)))
                    (recur (- j 1)))
                  j))]
        (aset arr j val))))
  arr)

(s/fdef ::insertion-sort!
  :args (s/with-gen
          (s/and (spec/cat :val number-array?)
                 #(not (nil? (:val %))))
          #(gen/fmap into-array (s/gen (s/coll-of int?))))
  :ret #(and
          (number-array? (:ret %))
          (= (-> % :args :val seq sort)
             (-> % :ret seq))))

(stest/instrument `insertion-sort!)

;; Unable to construct gen at: [:val] for: number-array?...
(stest/check `insertion-sort!)

(defn sorted-array?
  [arr]
  (->> (range (alength arr))
       (partition 2)
       (every? (fn [[x y]] (<= (aget arr x) (aget arr y))))))

(deftest sort-test
  (is (true?
         (sorted-array?
           (insertion-sort! (int-array [5 2 4 3 1]))))))

Running (stest/check 'insertion-sort!) produces:

...
 :sym getting-clojure.sort/insertion-sort!,
 :failure #error{:cause "Unable to construct gen at: [:val] for: number-array?",
                  :data #:clojure.spec.alpha{:path [:val], :form getting-clojure.sort/number-array?, :failure :no-gen},
                  :via [{:type clojure.lang.ExceptionInfo,
                         :message "Unable to construct gen at: [:val] for: number-array?",
                         :data #:clojure.spec.alpha{:path [:val],
                                                    :form getting-clojure.sort/number-array?,
                                                    :failure :no-gen},
                         :at [clojure.spec.alpha$gensub invokeStatic "alpha.clj" 282]}],
...

Any advice is much appreciated!


#2

You’re on the right track. Let’s take a closer look at the s/fdef form. Notice that values produced by the generator don’t satisfy the specification.

(let [gen (gen/fmap into-array (s/gen (s/coll-of int?)))
      spec (s/and (spec/cat :val number-array?)
                  #(not (nil? (:val %))))]
  (s/explain spec (first (gen/sample gen))))
;; val: #object["[Ljava.lang.Long;" 0xcfdd8cb "..."]
;; fails predicate: (cat :val number-array?)

According to the specification, the array should be put in a collection. Let’s adjust the generator.

(let [gen (gen/fmap (comp vector into-array)
                    (s/gen (s/coll-of int?)))
      spec (s/and (spec/cat :val number-array?)
                  #(not (nil? (:val %))))]
  (s/explain spec (first (gen/sample gen))))
;; In: [0] val: #object["[Ljava.lang.Long;" 0x7b4fe70f "..."]
;; fails at: [:val] predicate: number-array?

We’ve got the outer shape right, but the object in the collection doesn’t satisfy the number-array? predicate. The reason is into-array returning an array of java.lang.Object. Replacing it with long-array should satisfy the specification.

(let [gen (gen/fmap (comp vector long-array)
                    (s/gen (s/coll-of int?)))
      spec (s/and (spec/cat :val number-array?)
                  #(not (nil? (:val %))))]
  (s/explain spec (first (gen/sample gen))))
;; Success!

Note also that the predicate passed as :ret should be provided using :fn, as explained in the s/fdef docstring.

I hope that helps.


#3

Thanks for your help, Jan!

I managed to get tests to work with the following

(s/def ::double
  (s/double-in :NaN? false))

(s/def ::number-array
  (s/with-gen (s/and
                (spec/cat :val number-array?)
                #(not (nil? %)))
              #(gen/fmap (comp vector double-array)
                         (s/gen (s/coll-of ::double)))))

(s/fdef insertion-sort!
  :args ::number-array
  :ret number-array?
  :fn #(-> % :ret sorted-array?))

What I don’t understand is why I can’t use ::number-array as the spec for :ret?

(s/fdef insertion-sort!
  :args ::number-array
  :ret ::number-array             <- doesn't work, but works with number-array? fn
  :fn #(-> % :ret sorted-array?))

When using ::number-array instead of number-array? I get:

{:type clojure.lang.ExceptionInfo,
                         :message "Specification-based check failed",
                         :data {:clojure.spec.alpha/problems [{:path [:ret],
                                                               :pred (clojure.core/fn
                                                                      [%]
                                                                      (clojure.core/or
                                                                       (clojure.core/nil? %)
                                                                       (clojure.core/sequential? %))),
                                                               :val #object["[D" 0x33ef1413 "[[email protected]"],
                                                               :via [],
                                                               :in []}

:ret expects a spec for function’s return value, and since insertion-sort! returns same data structure, array of doubles, why doesn’t ::number-array work?

Again, appreciate your help!


#4

Note that, as currently specced, ::number-array matches a sequence that has only one element (and that element must satisfy number-array), whereas number-array? returns true if the arg is a number array.

This might be clearer in code :smile:

playground.spec=> (s/explain number-array? (int-array []))
Success!
nil
playground.spec=> (s/explain ::number-array (int-array []))
val: #object["[I" 0x4409e975 "[[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
nil
playground.spec=> (s/explain ::number-array [(int-array [])])
Success!
nil

You can fix this by removing the s/cat from your ::number-array definition. Instead, I think you want your args spec to look like this: :args (s/cat :val ::number-array). Hope that helps.


#5

Another way to get feedback on your spec is to generate values and see if they match the spec. Assuming you have test.check as a dep in your project, you can do:

(doseq [vals (map first (s/exercise ::number-array))] (s/explain ::number-array val))
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: ;; :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)
;; val: #object[clojure.core$val 0x75504cef "[email protected]"] fails spec: :playground.spec/number-array predicate: (cat :val number-array?)