Spec riddle - parsing [::key1 val ::key2 val [::key3 val] ...]

[ryantate]

Let’s say you have specs like so:

(s/def ::string string?)
(s/def ::number number?)
(s/def ::map map?)
... [many more] ...

And let’s say you want to spec a data format of vectors of uncontained key-value pairs and other such vectors like so:

[::number 42
 ::string "foo" 
 [::map {:bar "bar"}]
...]

Conundrum/riddle: How to spec the above in spec1 without using s/conformer or knowing all the possible “key” specs (::string, ::number, etc) in advance?

With s/conformer it’s possible like so:

(defmulti tuple-spec first)
(defmethod tuple-spec :default
  [[k v]]
  (s/tuple qualified-keyword? (s/get-spec k)))

(s/def ::free-kv (s/& (s/cat :key qualified-keyword? :value any?)
                      (s/conformer (juxt :key :value))
                      (s/multi-spec tuple-spec first)))

(s/def ::free-kvs (s/and vector? 
                         (s/* (s/alt :entry ::free-kv
                                     :free-kvs ::free-kvs))))

The thing is, people really seem to cast aspersions on s/conformer, like using it is a sign of weakness somehow, or non separation of concerns. The library expound, which makes friendlier spec explains, does not even support its use, with the author reasoning, “Although using conformers in this way is fairly common, my understanding is that this is not an intended use case.” The “use case” is “using conformers to transform values,” as I do here. (Nothing against this author or their reasoning, by the way, this is just an example of why I want to avoid using s/conformer.)

There is a way to use s/conformer here without transforming the value (which I guess made my original version maximally Bad, so this is Better):

(s/def ::free-kv (s/& (s/cat :key qualified-keyword? :value any?)
                      (s/conformer #(s/conform (:key %) (:value %)))))

This may avoid the sin of converting values but gives much worse s/explain output - it merely explains the conformer failed, whereas my original version gives the core error from the original spec (::string or ::number or whatever)

Anyway maybe there’s a way for me to solve this problem without using the dreaded conformers. I don’t see it, though, and I think this traces back to the decision to make s/cat conform to a map. In the context of spec, the shape of a map’s values are always dictated by the keys. Without conforming it back to a sequence I don’t see where else to meaningfully go after a cat, unless I know all the possible pair types in advance, which perhaps in many use cases one does. In my case I definitely do not, and don’t want to (extensible system).

Maybe I’m just missing something, thanks for any tips!

(Update - Made the use case slightly more elaborate and like my actual use case to show why I can’t just use s/keys*…)

[thheller]

Why does the vector format break the key/value pair convention? Keys are keywords, values are any.

Why

[::number 42
 ::string "foo" 
 [::map {:bar "bar"}]
...]

and not

[::number 42
 ::string "foo" 
 ::map {:bar "bar"}
...]

Or if the goal is to express any map as a vector

[::number 42
 ::string "foo" 
 ::map [:bar "bar"]
...]

?

Seems like the spec problems arise from a weirdly defined data format?

[didibus]

@ryantate

[::number 42
 ::string "foo" 
 [::map {:bar "bar"}]
...]

Would something like this work? (not tested):

(s/def ::structure
  (s/* (s/alt :kv-pair (s/cat :key qualified-keyword? :value any?)
              :weird-vector (s/tuple qualified-keyword? (s/map-of keyword? any?)))))

[ryantate]

Yes, the format is admittedly quirky.

Thanks didibus. I’m trying to make sure the “value” side of each pair validates to the “key” side. Although perhaps your more basic approach is a clue I should defer this until later.

[didibus]

I think you could achieve what you want maybe like so:

(s/def ::structure
  (s/* (s/alt :number (s/cat :key #{::number} :value number?)
              :string (s/cat :key #{::string} :value string?)
              :sub-structure (s/every ::structure :kind vector?))))

Again untested.

I’m not sure if every will work, you could also try:

(s/def ::structure
  (s/* (s/alt :number (s/cat :key #{::number} :value number?)
              :string (s/cat :key #{::string} :value string?)
              :sub-structure (s/spec ::structure))))

I feel like one of these or a small variation should do the trick.

Ok I quickly tested the following and it seems to work:

(s/def ::structure
 (s/*
  (s/alt
   :number 
   (s/cat
    :key
    #{::number}
    :value 
    number?)
   :string
   (s/cat 
    :key 
    #{::string}
    :value 
    string?) 
   :map
   (s/cat
    :key #{::map}
    :value (s/map-of any? any?)) 
   :sub-structure
   (s/spec 
    ::structure))))

But you have to evaluate it twice. I’m not too sure how you avoid having to do that.

[ryantate]

These are great, thanks didibus. Unfortunately I cannot presume advance knowledge of the keys (like I said, extensible). But I appreciate the thoughtful approach.

I’m starting to realize my need is perhaps more than a touch exotic definitely I don’t fault spec (I do think this shows a nice (non Bad) use of conformers to alter values fwiw)

[didibus]

Ah I see what you want now. I think conformer is fine here. Conformers are not meant to be used to coerce values, but they are meant to conform them for the purpose of further validation in my opinion at least, which is what you’re using it for here.

That said, any predicate is a spec, so you could also simply implement what you want as a predicate. The downside with more custom specs like that is you would need to provide your own generators, and they don’t always report errors as clearly.