Is it efficient to nest/compose eductions?

[ryantate]

Very short version of my question, I am wondering if something like this is going to be efficient, without intermediate collections created by the eductions and with execution of the eduction transducers deferred until the end. Oversimplified approximation of what I’m actually doing:

(->> (myapp/retrieve-posts) ;intermediate coll here
     (eduction (map assoc-post-access-level)
               (filter post-is-public?)) ;eduction, nothing happens yet
     (eduction (map myapp/assoc-tags)) ;eduction passed to eduction, what happens?
     (into [] (comp (map myapp/assoc-view-stuff) (take 10)))) ;both eductions called here right? efficient since reducible context?
     myapp/template-render

eduction produces a reducible application of the transducers it takes over the coll it takes. From what I can tell looking at clojure.core source, eduction itself also provides a reducible application since it calls transduce on the coll in a reducible context. So neither eduction should have its trasnducers called until we get to into?

---

This slightly longer example hopefully shows why I want to do this, again a very simplified version of my actual code:

(let [posts (myapp/retrieve-posts)
      posts (if anonymous-user?
              (eduction (map assoc-post-access-level)
                        (filter post-is-public?)
                        posts)
              posts)
      posts (if need-tags?
              (eduction (map myapp/assoc-tags) posts)
              posts)]
  (myapp/template-render (into []
                               (comp
                                (map myapp/assoc-view-stuff)
                                (take 10))
                               posts)))

---

Longer explanation:

Basically what I’m doing is building a site with Pedestal. In Pedestal a web response is built up using a pipeline of narrowly-targeted “interceptors,” which are basically just functions inside of a map.

The idea is to keep each interceptor simple and focused so it can be composed with other interceptors like Lego bricks. So let’s say I’m building up a collection of content posts. I might want to compose a series of functions that do approximately

1. Retrieve post maps →
2. filter public posts unless user logged in →
3. enrich posts (map assoc) with tags if we’ll be displaying tags →
4. enrich posts (map assoc) with attributes needed for this particular view template →
5. render response (with somewhat generic code reusable in other contexts)

It seems to me like eductions are a good use case for this because I’d like to avoid creating intermediate collections in steps 2, 3, and 4. It seems like eduction offers a way to buid up a transduction over a specific collection after that collection is bound into the transduction process early on.

But I am not sure - I am new to eduction and trying to learn where it might be useful. I feel more comfortable with other transducing functions like transduce, sequence, and into.

So I thought I’d ask you folks if anyone knows if this seems like a good/reasonable way to use eductions. Thanks very much in advance for any feedback.

[p-himik]

You’re right - nothing will happen till the execution gets to into. (Your first code block is broken though - it has a few extra posts there.)

Somewhat beside the point - you can use cond->> to write things like (if x (... coll) coll) as (cond->> coll x (...)).

Back to the topic - you can also use cond-> to conditionally chain transducers instead of creating eductions (note that it’s cond-> and not cond->> - the order of composition is important):

(->> (myapp/retrieve-posts)
     (into []
           (cond-> (comp (map myapp/assoc-view-stuff)
                         (take 10))
             anonymous-user?
             (comp (map assoc-post-access-level)
                   (filter post-is-public?))
             
             need-tags?
             (comp (map myapp/assoc-tags))))
     (myapp/template-render))

[ryantate]

Thanks!! Super good to know.

Ah you are right about the extra posts, I will fix those typos.

cond-> is useful too.

On your last suggestion, the second, longer code example I put is still simplified, right now those steps are split up across different functions inside different interceptors, each one modifying (context :posts), so it’s useful to put something there that actually has the current state of the posts and can be grabbed, at least in a form that can be used to reduce. But yes if it was all in one function I would just build up a transducer, it’s a good approach.

(Another approach with interceptors might be to just build up a transducer in the context, as (context :posts-xf), and then don’t touch (context :posts) until the very end. That’s just ever so slightly more complicated and with eduction it seems like I can keep things a bit simpler and just have (context :posts).)