Hi, I was wondering if anyone knows the answer or could point me at a resource that would help me solve this problem.
I want to generate a list of entity-ids and then a list of entities, the catch is I want each entity in the list to be generated with an id that corresponds to a list from the entity id.
(s/def ::id (s/with-gen string? #(s/gen #{"a" "b" "c" "d"})))
(s/def ::entity-ids (s/coll-of ::id :count 2 :distinct true))
(s/def ::entity (s/keys :req-un [::id]))
(s/def ::entities (s/coll-of ::entity :count 4 :distinct true))
(s/def ::manifest (s/keys :req-un [::entity-ids]))
(s/def ::manifests (s/coll-of ::manifest :count 2 :distinct true))
(s/def ::foo (s/keys :req-un [::manifests ::entities]))
(gen/generate (s/gen ::foo))
;; output
{:manifests [{:entity-ids ["d" "a"]}
{:entity-ids ["a" "b"]}],
;; c is in neither of the manifests entity lists
:entities [{:id "a"}
{:id "d"}
{:id "c"}
{:id "b"}]}
;; But what I want is below.
;; Each entity has an id that maps to at least one of the entity-id lists
{:manifests [{:entity-ids ["a" "b"]}
{:entity-ids ["c" "d"]}],
:entities [{:id "a"}
{:id "d"}
{:id "c"}
{:id "b"}]}
If I haven’t made the problem clear enough, please let me know and I’ll try and elaborate. Thanks in advance! 
Hello!
You might want a custom generator for that.
Note that your ::foo spec is valid even when the ID’s aren’t consistent (as you’re noting). You could use a predicate for that. Something like
(defn consistent-foo? [foo]
;; ...
)
(s/def ::foo
(s/and consistent-foo?
(s/keys :req-un [::manifests ::entities])))
Teodor
That’s the part that I was missing! I completely forgot that s/and uses the first predicate/spec as a generator and those after it can narrow down the results. It makes perfect sense, like you said my spec for foo wasn’t capturing the constraint that I wanted, so it obviously wouldn’t generate the correct data.
Here’s what I ended up with if anyone is interested:
(s/def ::id (s/with-gen string? #(s/gen #{"a" "b" "c" "d"})))
(s/def ::entity-ids (s/coll-of ::id :count 2 :distinct true))
(s/def ::entity (s/keys :req-un [::id]))
(s/def ::entities (s/coll-of ::entity :count 4 :distinct true))
(s/def ::manifest (s/keys :req-un [::entity-ids]))
(s/def ::manifests (s/coll-of ::manifest :count 2 :distinct true))
(defn consistent-foo? [{:keys [manifests entities]}]
(= (set (mapcat :entity-ids manifests))
(set (map :id entities))))
(s/def ::foo
(s/and (s/keys :req-un [::manifests ::entities])
consistent-foo?))
(gen/generate (s/gen ::foo))
;; output
{:manifests [{:entity-ids ["b" "c"]}
{:entity-ids ["d" "a"]}],
:entities [{:id "b"}
{:id "a"}
{:id "d"}
{:id "c"}]}
Thanks Teodor!
This is useful stuff… I have a similar (but a bit larger scope) problem, and this looks like a good starting point for solving it. Thanks guys!
Posting this in case its useful to anyone.
I started running into issues where spec was unable to generate large consistent sets in 100 tries. The solution I found was to use a custom generator with single item set that guaranteed that the manifest and the items would have the same keys. I don’t feel like this is a particularly elegant solution but it does work.
The data format is slightly different to that in my initial question.
{:manifest {:item-order ["b" "c"]},
:items [{:id "b" :type "food"}
{:id "c" :type "drink"}]
Here’s my approach.
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.gen.alpha :as gen])
(defn large-shuffled-vec-of-ids []
(->> (range 1000)
(map (partial str "xid"))
shuffle
vec))
(s/def ::id string?)
(s/def ::item-order
(s/with-gen
(s/coll-of ::id :distinct true)
#(s/gen #{(large-shuffled-vec-of-ids)})))
(s/def ::manifest (s/keys :req-un [::item-order]))
(s/def ::type #{"food" "drink"})
(s/def ::item (s/keys :req-un [::id ::type]))
(s/def ::items
(s/with-gen
(s/coll-of ::item :distinct true)
#(s/gen #{(let [ids (large-shuffled-vec-of-ids)]
(map (fn [item id] (merge item {:id id}))
(apply concat (s/exercise ::item (count ids)))
ids))})))
(defn consistent-ids? [{:keys [manifest items]}]
(= (set (manifest :item-order))
(set (map :id items))))
(s/def ::items-with-manifest
(s/and
(s/keys :req-un [::manifest ::items])
consistent-ids?))
(comment
(gen/generate (s/gen ::items-with-manifest)))
Is there a better approach that I’m just missing?
Hey @andersmurphy,
Just read the PurelyFunctional.tv newsletter, (read issue 330 in full), which mentioned this. You’ve got your solution, but if you’re interested in more context, the newsletter might give you that.
@ericnormand provides an example for E-mail generation. Snipped from the newsletter (hope that’s okay):
(def email-re
#"(([^<>()\[\]\.,;:\s@\"]+(\.[^<>()\[\]\.,;:\s@\"]+)*)|(\".+\"))@(([^<>()\[\]\.,;:\s@\"]+\.)+[^<>()\[\]\.,;:\s@\"]{2,})")
(def email-char-re #"[^<>()\[\]\.,;:\s@\"]")
(def gen-email-char
(gen/such-that #(re-matches email-char-re (str %)) gen/char))
(def gen-email-char-string
(gen/fmap #(apply str %)
(gen/not-empty (gen/vector gen-email-char))))
(def gen-email-name
(gen/fmap #(str/join "." %)
(gen/not-empty (gen/vector gen-email-char-string))))
(def gen-email-domain
(gen/fmap (fn [[domain-segs tld1 tld2]]
(str domain-segs "." tld1 tld2))
(gen/tuple gen-email-name
gen-email-char-string
gen-email-char-string)))
(def gen-email
(gen/fmap (fn [[username domain]]
(str username \@ domain))
(gen/tuple gen-email-name gen-email-domain)))
(gen/sample gen-email)
;; => ("‡@¥.ú .²e" "Ø.@ñ.NÎ" "¿G.Ø@ñwßÃ..ëxÂ!" "š¨.Ðâë.¼/.Âù@òÉ.Â.…ë.‚ù"
;; "vu.BÖ¯¢.ƒÍ@þÛü..´Ê!6" "Â.L¹¹èf.Zô.æ@\b*î.0¡6.½.ƒŒ¶"
;; "ÏÂÆ{É.ûÈÉu.âàWN4@7.j1€WÂ.µR*A.Hcõ&E.PE$.òIÛOÚäû"
;; "ž&9ÅÂÇ.Où¸.r¾.Þa&.auj@#mõê.cÓ\\æ÷"
;; "Yù.íï®±|y.ÐNFÕÊ!š.TŒÔéã.ŒP.ipÂðêc}.PŒ@^P.I˸j¤ñ³.óZÀËa.ú}#zvá.Šm.^}öSFjØ.bžÀÊEšcÖ¡"
;; "æ}¯.sÇ.F4.Ý*fÝBº.V.ø.L–¥hôít².³ÒbžÕ@wHföª†«.â·cÔq¬.Yí-§ª.cÞÇ”\bñĽö.Æ×`’.í Oâ.ƒ³«˜Œ.šØ$¨ã§8ÿú/•")
Awesome thanks for sharing! (signed up to the newsletter too) 