Any means to fill in holes in deep EDN data?


#1

Suppose I got a piece of data from API server which is:

{:name "demo"
 :projects []
 :children {"ab4a34" {:name "child"
                      :projects nil
                      :children nil}}}

but what I wanted is I don’t get those nil values, instead I want:

{:name "demo"
 :projects []
 :children {"ab4a34" {:name "child"
                      :projects []
                      :children {}}}}

Is there any tools in the Clojure(Script) ecosystem that may help me correct those nils? Is Spec capable of doing this?


#2

Maybe fnil? https://clojuredocs.org/clojure.core/fnil


#3

You can use clojure.walk. Or I believe specter can do that too.

Specter would be a better investment if you’re looking to do more and more data transformation and manipulation of deeply nested structures. But learning its DSL initially is more short term overhead.

While clojure.walk to write a single remove nil util function might be simpler at first, it won’t scale the same way in terms of usability. But if that’s all you need, at least its standard in Clojure and doesn’t depend on a lib.

http://cljs.github.io/api/clojure.walk/


#4

You need to define the scope more clearly.

If you just want to do some custom walk and replace operations, clojure.walk is good although it is
relatively complicated to reason about.

If you have the shape of the data and you want the input data conform to your shape of data, clojure.spec is good for this task, but remember it is not fast and increase file size for cljs.

(def data 
  {:name "demo"
   :projects []
   :children {"ab4a34" {:name "child"
                        :projects nil
                        :children nil}}})

(require '[clojure.spec.alpha :as s])

(s/def ::fund
  (s/keys :req-un [:fund/name :fund/projects :fund/children]))

(s/def :fund/name string?)
(s/def :fund/projects (s/* (s/and ::nil->vec :fund/project)))
(s/def :fund/children (s/and ::nil->map (s/map-of string? ::fund)))

(s/def ::nil->vec (s/conformer (fnil identity []) identity))
(s/def ::nil->map (s/conformer (fnil identity {}) identity))

(s/conform ::fund data)
;{:name "demo"
; :projects []
; :children {"ab4a34" {:name "child"
;                      :projects []
;                      :children {}}}}

#5

You could also consider if it was worth accepting nil? A lot of clojure fns keep on trucking when exposed to nil:

(assoc nil :foo 42) => {:foo 42}
(merge nil {:foo 42} => {:foo 42}

etc …


closed #6

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.