Expanding a vector to it's individual elements with a macro

[ashwinbhaskar]

I am trying to construct an SQL query with dynamic parameters in the in sections. I do not want to use honeysql here. I am trying to do this with a macro. Note: apply will not work here.

(defmacro expand-second-arg
  [q b]
  `[~q ~@b])

(defn find [ids]
  (let [query "SELECT * FROM foo WHERE id in (?,?)"]
        (jdbc/execute! datasource (expand-second-arg query ids))))

But I get this error

error while macroexpancing..
Don't know how to create ISeq from: clojure.lang.Symbol

But it works fine when I use the macro with values directly. For example this

(expand-second-arg "SELECT * FROM foo WHERE id in (?,?)" ["id_1" "id_2"])

works and returns ["SELECT * FROM foo WHERE id in (?,?)" "id_1" "id_2"].

[bsless]

It’s not working because macros are evaluated at read-time, not run time. Since ids is not known at read time you can’t expand it with a macro. You want to use the function apply, as in (apply vector q ids)

[ashwinbhaskar]

my bad. Yes, vector works fine. Thank you:)

[seancorfield]

I would probably use (into [query] ids) here. I think it’s clearer and I would expect it to be faster. I try to avoid apply unless it’s absolutely necessary.

[madbonkey]

I think clarity is subjective here, but why would you expect it to be faster? Is apply that bad or is into that optimized?

[seancorfield]

Just looking at the source of the two functions apply and into makes me think into would be faster, so I ran a quick benchmark with Criterium and, with an ids list of 10 numbers, into is about twice as fast:

user=> (b/quick-bench (into [10] ids))
Evaluation count : 1872594 in 6 samples of 312099 calls.
             Execution time mean : 364.756664 ns
    Execution time std-deviation : 80.701919 ns
   Execution time lower quantile : 310.080901 ns ( 2.5%)
   Execution time upper quantile : 474.054445 ns (97.5%)
                   Overhead used : 7.683004 ns
nil
user=> (b/quick-bench (apply vector 10 ids))
Evaluation count : 971178 in 6 samples of 161863 calls.
             Execution time mean : 683.204222 ns
    Execution time std-deviation : 94.196118 ns
   Execution time lower quantile : 614.400740 ns ( 2.5%)
   Execution time upper quantile : 808.153578 ns (97.5%)
                   Overhead used : 7.683004 ns
nil
user=> 

(I ran those several times – the results were pretty consistent)

This is the relevant arity from apply:

  ([^clojure.lang.IFn f x args]
     (. f (applyTo (list* x args))))

and that arity of list*:

  ([a args] (cons a args))

So it’s going to build a (lazy) sequence of all the arguments and then call .applyTo (from IFn). The relevant Java code for that can be seen here: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/AFn.java#L227-L239

That walks all 11 elements of the arguments to call .invoke, and then then relevant arity of vector for 11 args is going to be:

  ([a b c d e f & args]
     (. clojure.lang.LazilyPersistentVector (create (cons a (cons b (cons c (cons d (cons e (cons f args))))))))))

which builds up another (lazy) sequence of all the arguments(!) and finally creates a vector from them.

By contrast, into does this:

  ([to from]
     (if (instance? clojure.lang.IEditableCollection to)
       (with-meta (persistent! (reduce conj! (transient to) from)) (meta to))
       (reduce conj to from)))

[10] is an IEditableCollection so we take the (fast) transient route to reduce with an init value, which is an optimized path that makes a single pass over the collection (although you pay the cost of constructing a persistent version of the transient vector at the end).