Expanding a vector to it's individual elements with a macro

I am trying to construct an SQL query with dynamic parameters in the in sections. I do not want to use honeysql here. I am trying to do this with a macro. Note: apply will not work here.

(defmacro expand-second-arg
  [q b]
  `[~q [email protected]])

(defn find [ids]
  (let [query "SELECT * FROM foo WHERE id in (?,?)"]
        (jdbc/execute! datasource (expand-second-arg query ids))))

But I get this error

error while macroexpancing..
Don't know how to create ISeq from: clojure.lang.Symbol

But it works fine when I use the macro with values directly. For example this

(expand-second-arg "SELECT * FROM foo WHERE id in (?,?)" ["id_1" "id_2"])

works and returns ["SELECT * FROM foo WHERE id in (?,?)" "id_1" "id_2"].

It’s not working because macros are evaluated at read-time, not run time. Since ids is not known at read time you can’t expand it with a macro. You want to use the function apply, as in (apply vector q ids)

1 Like

my bad. Yes, vector works fine. Thank you:)

1 Like

I would probably use (into [query] ids) here. I think it’s clearer and I would expect it to be faster. I try to avoid apply unless it’s absolutely necessary.

1 Like

I think clarity is subjective here, but why would you expect it to be faster? Is apply that bad or is into that optimized?

Just looking at the source of the two functions apply and into makes me think into would be faster, so I ran a quick benchmark with Criterium and, with an ids list of 10 numbers, into is about twice as fast:

user=> (b/quick-bench (into [10] ids))
Evaluation count : 1872594 in 6 samples of 312099 calls.
             Execution time mean : 364.756664 ns
    Execution time std-deviation : 80.701919 ns
   Execution time lower quantile : 310.080901 ns ( 2.5%)
   Execution time upper quantile : 474.054445 ns (97.5%)
                   Overhead used : 7.683004 ns
nil
user=> (b/quick-bench (apply vector 10 ids))
Evaluation count : 971178 in 6 samples of 161863 calls.
             Execution time mean : 683.204222 ns
    Execution time std-deviation : 94.196118 ns
   Execution time lower quantile : 614.400740 ns ( 2.5%)
   Execution time upper quantile : 808.153578 ns (97.5%)
                   Overhead used : 7.683004 ns
nil
user=> 

(I ran those several times – the results were pretty consistent)

This is the relevant arity from apply:

  ([^clojure.lang.IFn f x args]
     (. f (applyTo (list* x args))))

and that arity of list*:

  ([a args] (cons a args))

So it’s going to build a (lazy) sequence of all the arguments and then call .applyTo (from IFn). The relevant Java code for that can be seen here: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/AFn.java#L227-L239

That walks all 11 elements of the arguments to call .invoke, and then then relevant arity of vector for 11 args is going to be:

  ([a b c d e f & args]
     (. clojure.lang.LazilyPersistentVector (create (cons a (cons b (cons c (cons d (cons e (cons f args))))))))))

which builds up another (lazy) sequence of all the arguments(!) and finally creates a vector from them.

By contrast, into does this:

  ([to from]
     (if (instance? clojure.lang.IEditableCollection to)
       (with-meta (persistent! (reduce conj! (transient to) from)) (meta to))
       (reduce conj to from)))

[10] is an IEditableCollection so we take the (fast) transient route to reduce with an init value, which is an optimized path that makes a single pass over the collection (although you pay the cost of constructing a persistent version of the transient vector at the end).

2 Likes