Using Schema-like schemas in malli

I’ve just refactored my reitit API to use malli instead of Clojure spec through spec-tools. However, I sorely missed the Schema-like syntax in malli. Even though the hiccup-like syntax in malli is more powerful, I like how the similarity between the Schema-syntax and the data they’re describing. So I made a function that converts Schema-syntax to malli-syntax.

(ns to-malli
  (:require [clojure.walk :as walk])
  (:import #?(:clj (clojure.lang MapEntry))))

(defn ->m
  [schema]
  (->> schema
       (walk/postwalk
         (fn [form]
           (cond
             (= MapEntry (type form))
             form

             ;; leave [:keyword ...] vectors untouched
             (and (vector? form) (keyword? (first form)))
             form

             ;; leave forms with ^:m meta untouched
             (:m (meta form))
             form

             ;; one-element map with non-keyword key -> :map-of
             ;; NOTE: {int? :int} will be interpreted as [:map-of int? :int]
             ;; whereas {:int :int} will be interpreted as [:map [:int :int]]
             ;; This is to allow {:x :int} to be interpreted as map with
             ;; only one key.
             ;; Perhaps :map-of should require explicit malli syntax
             ;; and not be inferred.
             (and (map? form) (= 1 (count form)) (not (-> form first key keyword?)))
             [:map-of (-> form first key) (-> form first val)]

             ;; map -> :map
             (map? form)
             (into [:map] form)

             ;; one-element vector -> :vector
             (and (vector? form) (= 1 (count form)))
             [:vector (first form)]

             ;; one-element list -> :sequential
             (and (list? form) (= 1 (count form)))
             [:sequential (first form)]

             ;; > 1 element vector -> :tuple
             (and (vector? form) (< 1 (count form)))
             (into [:tuple] form)

             ;; one-element set -> :set
             (and (set? form) (= 1 (count form)))
             (into [:set] form)

             :else
             form)))))

Works surprisingly well (don’t know why this code isn’t properly formatted)

(->m {:map1 {:y int?
             :z [:maybe string?]}
      :map2 {:min-max     [:int ^:m {:min 0 :max 10}]
             :tuple       [int? string?]
             :set-of-maps #{{:e int?
                             :f string?}}
             :map-of-int  {int? string?}}})
; =>
; [:map
;  [:map1
;   [:map
;    [:y int?]
;    [:z [:maybe string?]]]]
;  [:map2
;   [:map
;    [:min-max [:int {:min 0, :max 10}]]
;    [:tuple
;     [:tuple
;      int?
;      string?]]
;    [:set-of-maps
;     [:set
;      [:map
;       [:e int?]
;       [:f string?]]]]
;    [:map-of-int
;     [:map-of
;      int?
;      string?]]]]]

(actually the int? is converted to #object[clojure.core$int_QMARK_ 0x2542240d "clojure.core$int_QMARK_@2542240d"] and same for string? but I edited for clarity)

This saved me a lof of time because I could reuse my previous spec-tools schemas with very little change.

Caveats:

  • Since ->m assumes that all maps should be converted to [:map ...], properties like {:min 0 :max 10} need to be protected using ^:m.

  • The inferring of [:map-of ...] does not work if one uses a type schema (e.g., :int instead of int?) because {:int int?} may be a valid one-element map with :int as key, whereas {int? int?} can be interpreted as a map with integer keys (like spec-tools does).

  • EDIT: A similar problem for [:int] which is interpreted as [:int] while [int?] is interpreted as [:vector int?], because of the second cond. That could probably be changed to only return vectors beginning with a key with at least two elements, but I haven’t thought that through.

Comments welcome!

(Disclaimer: I am completely self-taught in Clojure so I have no idea what I’m doing :slight_smile: and I realize that this cannot replace the full power of using hiccup-like syntax in malli).

3 Likes

Thanks for sharing! Malli (and spec-tools) lead here. Agree that data-specs are handy when used with libs like reitit, as long as the needs are simple.

Suggestions:

  • It might be better not to support automatically interpreted vectors or map-of’s to avoid ambiguity. simple syntax should be simple
  • by default, meta-data is not visible when printing the form, so debugging the syntax is harder as one can’t know which form has the ^:m
  • like data-specs, adding functional helpers for optional keys, maybes, and ors might be a good idea.

Here is an alternative implementation:

(ns malli.experimental.lite
  (:refer-clojure :exclude [set vector and or])
  (:require [malli.core :as m]))

(declare schema)

(defrecord -Optional [value])
(defn -schema [t & xs] (schema (into [t] (map schema xs))))
(defn -entry [[k v]]
  (let [[v optional] (if (instance? -Optional v) [(:value v) true] [v])]
    (cond-> [k] optional (conj {:optional true}) :always (conj (schema v)))))

(defn schema [x] (m/schema (if (map? x) (into [:map] (map -entry x)) x)))

(defn optional [x] (->-Optional x))
(defn maybe [x] (-schema :maybe x))
(defn set [x] (-schema :set x))
(defn vector [x] (-schema :vector x))
(defn map-of [k v] (-schema :map-of k v))
(defn tuple [& xs] (apply -schema :tuple xs))
(defn and [& xs] (apply -schema :and xs))
(defn or [& xs] (apply -schema :or xs))

defining schemas:

(require '[malli.experimental.lite :as l])

(def Schema
  (l/schema
   {:map1 {:x int?
           :y [:maybe string?]
           :z (l/maybe keyword?)}
    :map2 {:min-max [:int {:min 0 :max 10}]
           :tuples (l/vector (l/tuple int? string?))
           :optional (l/optional (l/maybe :boolean))
           :set-of-maps (l/set {:e int?
                                :f string?})
           :map-of-int (l/map-of int? {:s string?})}}))
;[:map
; [:map1
;  [:map
;   [:x int?]
;   [:y [:maybe string?]]
;   [:z [:maybe keyword?]]]]
; [:map2
;  [:map
;   [:min-max [:int {:min 0, :max 10}]]
;   [:tuples [:vector [:tuple int? string?]]]
;   [:optional {:optional true} [:maybe :boolean]]
;   [:set-of-maps [:set [:map [:e int?] [:f string?]]]]
;   [:map-of-int [:map-of int? [:map [:s string?]]]]]]

works normally with rest o malli:

(require '[malli.core :as m])
(require '[malli.error :as me])

(-> Schema
    (m/explain
     {:map1 {:x 1
             :z "kikka"}
      :map2 {:min-max 12
             :tuples [[1 "a"] [2 "b"]]
             :set-of-maps #{{:e 1, :f "f"}}
             :map-of-int {1 {:s "seppo"}
                          2 {:s :sanni}}}})
    (me/humanize))
;{:map1 {:y ["missing required key"]
;        :z ["should be a keyword"]}
; :map2 {:min-max ["should be between 0 and 10"]
;        :map-of-int {2 {:s ["should be a string"]}}}}

What do you think?

EDIT: I pushed the key optionality into value, instead of special wrapper for key (l/opt, l/req like in data-specs), I think it’s simpler that way, yup does it like this too,

3 Likes

Pasted the above code into malli and merged in master. As the experimental namespace suggests, not 100% sure if that belongs to malli. But, it’s just 18 lines of code, so easy to inline into client projects (or into a separate library) if for some reason would not be part of the final 1.0.0 release.

1 Like

Awesome, so glad if I have contributed to an idea of further development of Malli! I’ve read your previous post and have some thoughts - I’ll write more this weekend.

1 Like

I agree that vectors, tuples, etc. should not be inferred because of the ambiguity. But I think I would prefer to not use helper functions but instead have the full set of malli schemas available (e.g., :fn, :enum). I can then use {} which are inferred as maps and otherwise rely on malli docs

But one caveat with not using helper functions is that the {...}[:map ...] isn’t recursively applied. So (l/schema {:key [:vector {:key :int}]}) gives an error instead of [:map [:key [:vector [:map [:key :int]]]]. My function is recursive, but is pretty dumb so it needs the ^:m meta to preserve property maps. I can’t write parsers but it should be possible to infer from placement when a map should be preserved and when it should be changed to [:map] (like, properties always come after the type/predicate schema)? Nevertheless, your solution gets the job done in terms of allowing a syntax that is very close to data-specs.

Supercool that this is part of master now!

As maps present properties already in malli, you must either:

  1. tag the properties (meta-data or a wrapper function) and enable recursion with lite-syntax
  2. tag explicitly map-schemas (w.g. l/schema)

As one can mix and match both normal schemas and the lite-ones, I like the explicit option (2) better, as with 1, you can get surprises like:

(def Location
  "normal malli schema"
  [:tuple {:clj-kondo/type :any} int? int?])

;; lite-schema
(l/schema
 {:name :string
  :location Location})
;[:map 
; [:name :string] 
; [:location [:tuple [:map [:clj-kondo/type :any]] int? int?]]]

here, the properties map of the :tuple is interpreted as :map, which is not right.

anyway, thanks for bringing this up to attention, I’ll release the feature after user feedback and enable it later with reitit.

Btw, found the placeholder issue for this in reitit: https://github.com/metosin/reitit/issues/434

1 Like

Yeah, I see what you mean. Look forward to seeing this in a release!