Luminus: java.lang.IllegalArgumentException: Key must be integer

I have a luminus project and in the route/home.clj I have a test function for updating my database:

(defn save [req]
  (db/addb {:name "jonas" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 323232 :omk 43433434 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34}))

migration file:


CREATE TABLE rent
(id INTEGER PRIMARY KEY,
 name TEXT,
 org INTEGER,
 parea INTEGER,
 barea INTEGER,
 tarea INTEGER,
 hyra FLOAT,
 omk FLOAT,
 tid INTEGER,
 start INTEGER,
 stop INTEGER,
 notes TEXT,
 kost FLOAT,
 kost2 FLOAT);

and the corresponding queries.sql function:

--:name addb
--:doc adds another booking
INSERT INTO rent
(name,org,parea,barea,tarea,hyra,omk,tid,start,stop,notes,kost,kost2)
VALUES (:name, :org, :parea, :barea, :tarea, :hyra, :omk, :tid, :start, :stop, :notes, :kost, :kost2)

When I try this from the repl, i.e just calling addb everything works ok. But when I try this from my webbrowser, the I get this error: java.lang.IllegalArgumentException: Key must be integer

I simply don’t understand why I get this error. Any ideas?

Don’t know the exact cause, but the error is typically related to using a vector as a map and applying it to a key since vectors have hash map semantics (but only over integer keys):

user=> (def x [1 2 3 4])
#'user/x
user=> (x 0)
1
user=> (x :a)
Execution error (IllegalArgumentException) at user/eval3 (REPL:1).
Key must be integer
user=>

Somewhere in your application this is likely happening. It is unclear if the cause is in your code, or inside of a luminus interaction somewhere.

that may be the case, but I am working with {:key value :key value} structures that are supposed to be a hash map. Then there must be a conversion somwhere to a vactor. strange though that calling addb from repl works fine

Is it possible to share the code base or a reproducible example so that additional forensics could be provided? I have no idea other than the cause of said error (maybe in routing somewhere, no idea).

the code base is a bit fragmentet so it’s hard so show it. However, when saving and checking with sqlite3, the row is actually saved so the error message has nothing to do with the actual saving process. It may be a bug in luminus

ok, I have got rid of the error message but now I have the problem with

Parameter Mismatch: :name parameter data not found.

even though it in the map. But I will post that as an separate question.

how did you do fix it?

I submitted it as follows:
(db/addb (:latest (:params req)))

but I encountered other problems as described in another post.
But I do not know the origin of this problem, there is a lack of documentation about simple crud methods in luminus.

Ah so you had to unpack the argument from the request map.
It looks like the query infrastructure expects the params map entry to be exposed:

Why did you include :latest ? I would think (req :params) would have been sufficient based on the examples.

because latest is the map with the values and it is the only entry in params.
trying the example code from the documentation

(defn par [{:keys [params]}]
  (db/addb params))

gives the same error:

Parameter Mismatch: :name parameter data not found.

Maybe wrap it in an exception handler to trap the error with some more forensics beyond just that message:

(defn par [{:keys [params] :as m}]
(try (db/addb params)
  (catch Exception e (throw (ex-info "failed DB commit" {:in m})))

If this is happening on the repl thread, then you should be able to grab the full error, to include the actual input for the response map, and compare notes to make sure it has the keys/structure you actually expect. You can alternately save the input in an atom somewhere, or if you go the exception route, you should be able to access it via the *e repl var; the stack trace should help determine where in the guts of the library calls things are going wrong, and the exception’s :in value should show the input that was passed. It could be the input is violating some expectation.

Thanks, trace and *e gives null. Stack info is a long list of called libs but nothing from the actual files.
I am a bit confussed over all this, crud operations should be well documentet in any framework.
I am including the output for clarity:

user=> clojure.lang.ExceptionInfo: failed {:in nil}
	at rental.routes.home$par.invokeStatic(home.clj:87)
	at rental.routes.home$par.invoke(home.clj:85)
	at reitit.ring$ring_handler$fn__14633.invoke(ring.cljc:329)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:384)
	at ring.middleware.reload$wrap_reload$fn__4564.invoke(reload.clj:39)
	at selmer.middleware$wrap_error_page$fn__4579.invoke(middleware.clj:18)
	at prone.middleware$wrap_exceptions$fn__4821.invoke(middleware.clj:169)
	at ring.middleware.flash$wrap_flash$fn__8069.invoke(flash.clj:39)
	at ring.adapter.undertow.middleware.session$wrap_undertow_session$fn__8612.invoke(session.clj:77)
	at ring.middleware.keyword_params$wrap_keyword_params$fn__8706.invoke(keyword_params.clj:53)
	at ring.middleware.nested_params$wrap_nested_params$fn__8764.invoke(nested_params.clj:89)
	at ring.middleware.multipart_params$wrap_multipart_params$fn__8896.invoke(multipart_params.clj:171)
	at ring.middleware.params$wrap_params$fn__8923.invoke(params.clj:75)
	at ring.middleware.cookies$wrap_cookies$fn__8385.invoke(cookies.clj:214)
	at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__9111.invoke(absolute_redirects.clj:47)
	at ring.middleware.resource$wrap_resource_prefer_resources$fn__8959.invoke(resource.clj:25)
	at ring.middleware.content_type$wrap_content_type$fn__9059.invoke(content_type.clj:34)
	at ring.middleware.default_charset$wrap_default_charset$fn__9083.invoke(default_charset.clj:31)
	at ring.middleware.not_modified$wrap_not_modified$fn__9025.invoke(not_modified.clj:61)
	at ring.middleware.x_headers$wrap_x_header$fn__8646.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8646.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8646.invoke(x_headers.clj:22)
	at rental.middleware$wrap_internal_error$fn__9185.invoke(middleware.clj:18)
	at ring.adapter.undertow$undertow_handler$fn$reify__16740.handleRequest(undertow.clj:40)
	at io.undertow.server.session.SessionAttachmentHandler.handleRequest(SessionAttachmentHandler.java:68)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1423)
	at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
	at java.base/java.lang.Thread.run(Thread.java:844)

So, it looks like a couple of things are going on:

There’s a bunch of middleware that is being applied to the handler (pretty normal). I am curious what the effect of all the middleware is, but the end state is that you are getting a nil input instead of a map of parameters. So that’s where to start. Perhaps the request is malformed and there are no parameters (or the request map itself is nil somehow)…

It looks like you are also wrapping reitit in there as well.

Again, this is like doing remote surgery while blind folded. The process could probably be advanced much more rapidly if there was a repo to examine and walk through.

Perhaps the problem isn’t with CRUD operation documentation or lack thereof (I don’t have any context to judge by at the moment), but how the requests are being handled and passed through various middleware to arrive at your function to enact the CRUD stuff. Your db operations aren’t getting input to work on (from the looks of things).

ok, I did two things: I’ve created a resipory in git, here is the link: GitHub - md1frejo/rental: boking system
To replicate the error, run it, click on btable and then on test.
I copied in verbatim the example code that uses input tags to enter data and I got stuck with the following error.

2022-06-12 15:40:18,240 [XNIO-1 task-1] ERROR rental.middleware - contains? not supported on type: java.lang.Integer 
java.lang.IllegalArgumentException: contains? not supported on type: java.lang.Integer

ok, so now it works using all the input fields as input. Interestingly, the error

ERROR rental.middleware - contains? not supported on type: java.lang.Integer 
java.lang.IllegalArgumentException: contains? not supported on type: java.lang.Integer

only arrives if I call addb with two paranthesis like this:

(defn par [{:keys [params]}] 
  (db/addb params))

but if I call with one instead, the error disappear:

(defn par [{:keys [params]}] 
  (db/addb params)
..)

so paranthesis has an extrem importance here…
The only thing left now is to get rid of all input fields…

Is the key for the parameters really :params?

I think at some point, ring or some other library (remember this is not Luminus, Luminus is just a convenient way of putting a bunch of libraries together) changed or expanded, to differentiate query parameters from form params from POST params etc.

Might be worth doing some debugging/logging to see what keys are available in your request, and what they contain…

some quick notes, no answers yet, but we almost never use Float/parseFloat, or floats in general (outside of interop of other specific use cases where you are specifically controlling for space/size) like I see here, since clojure defaults to double and long types, e.g. Double/parseDouble (now in clojure 1.11 there are core functions for common parsing like parse-double ). Unless you are storing stuff in the database in this way, or interoperating with a java library that specifically uses floats, I would not recommend them. They can provide counter-intuitive equality stuff like

user=> (hash-set 1.1 (float 1.1))
#{1.1 1.1}

I don’t think it’s necessary or idiomatic to specify the version of cider in your plugins either, e.g. in project.clj, although it is unclear if you are even using cider (looks like vscode is there…)

 :project/dev  {:jvm-opts ["-Dconf=dev-config.edn" ]
                  :dependencies [[org.clojure/tools.namespace "1.2.0"]
                                 [pjstadig/humane-test-output "0.11.0"]
                                 [prone "2021-04-23"]
                                 [ring/ring-devel "1.9.5"]
                                 [ring/ring-mock "0.4.0"]]
                  :plugins      [[com.jakemccrary/lein-test-refresh "0.24.1"]
                                 [jonase/eastwood "0.3.5"]
                                 [cider/cider-nrepl "0.26.0"]]  ;<----maybe remove this.....
...

I can’t get the project running at the moment due to config/environment expectation mismatch. I probably need to dig into luminus more (aside from getting a repl connected). dev profile isn’t helping, etc.

regarding the observation of invoking with 2 parens, that should be meaningless, e.g.

(defn some-fn [x] 
  (blah x))
;;vs 
(defn some-fn [x]
  (blah x)
)

That sounds extremely suspect - if you think it is what is isolating a particular error.

edit - got it somewhat working, specifying the sqlite db connection and correcting the dev config (I think); enough to reproduce your error in the /test2 route, and to try stuff out at the repl:

rental.routes.home> (db/addb! {:name "Bilbo" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 3232
                                  :omk 4343 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34})
1
rental.routes.home> (->> (db/getrows) (map :name) set)
#{"noname" "Bilbo" "jonas"}

So the db layer works fine. Something is getting jacked up in the middleware.

this is the actual request map with all the middleware junk tacked on. I just changed the route for /test2 to:

(fn [req]
     (pr req)
     (db/addb! {:name "jonas" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 3232
                :omk 4343 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34}))]])

to print out the request map prior to invoking our problem function. So now we can dissect it a bit. There are prettier ways to do it (like using the cider debugger or other stuff), but this worked fine.

-Final edit

I get it now. If you look at the result of add-db, you will notice it returns an integer. I am pretty sure this is the number of records created (or a 0/1 indicator of success, not 100% sure, need to check the docs). So you have a defined a route with some middleware on the /test2 route, that invokes this function. The problem is, the result you return from this function - 1 - is then handed off to the middleware, which is expecting (per the ring methodology) to have a hash map of all the data for the response. So you are returning 1, the ring.middleware.flash handler (which is wrapped up in your middleware somewhere in the configs), now tries to apply its work to the response map, except it’s not a map. So when it invokes contains? against an integer, 1, we get a runtime type error. As a result the response bombs out and you get the stack error you noticed.

So the reason it “works at the REPL” but not the routing of an HTTP response, is that your handler is yielding a non-map response. I am uncertain if there is some expectation that a middleware would pick up on these values an wrap them for you, or if this is just an oversight (or learning stuff). In this case, if I change the handler to:

["/test2" (fn [req]
               (do (db/addb! {:name "jonas" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 3232
                              :omk 4343 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34})
                   {:status 200, :headers {}, :body  "Added jonas!"}))]

I invoke the db addition as a side effect, but I return a map indicating a success response (maybe it should be 201, I am not a web dev…). In any case, if I reload the new code, and invoke user/restart to ensure the libraries (I think they are using mount) pick up all the new changes, then navigate back to /test2, I get a response and no error.

If you look at all of the other routing functions, they all return an HTTP response via the layout/render function, which is:

(defn render
  "renders the HTML template located relative to resources/html"
  [request template & [params]]
  (content-type
    (ok
      (parser/render-file
        template
        (assoc params
          :page template
          :csrf-token *anti-forgery-token*)))
    "text/html; charset=utf-8"))

where ok is from the [ring.util.http-response :refer [content-type ok]] requires, so a convenience function to wrap the body in a map…

(defn ok
  "200 OK (Success)
  OK"
  ([] (ok nil))
  ([body]
   {:status 200
    :headers {}
    :body body}))

I think if you remember to return response maps for these handlers (perhaps there are more sophisticated possibilities with reitit routing), then you will avoid these kinds of errors. I am unsure what the idiomatic path is to modify the db - you probably want to return a response conditioned on whether the commit actually went through.

4 Likes

Thanks for the long reply, I willl look into to all this.
Just a quick note, the double parens was producing the error, I just wrote it a bit lazy:

(defn par [{:keys [params]}] 
  (db/addb params))

but adding code and then the closing parens removed the error:

(defn par [{:keys [params]}] 
  (db/addb params)
some other code and then finaly closing parens)

so the two dots in my original answer was “some other code”.

I also removed the !n flag from addb but that did not solve anything.
note that most of the code is from the example from the webpage where !n actually is in place.

another remark. double parens gives the error:

(fn [req]
     (pr req)
     (db/addb! {:name "jonas" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 3232
                :omk 4343 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34}))]])

but if I remove the pr function after addb! then the error goes away, because we get rid of the double paren:

(fn [req] (db/addb! {:name "jonas" :org "sdf" :parea 23 :barea 34 :tarea 65 :hyra 32 :omk 43 :tid 23 :start 43 :stop 56 :notes "notes" :kost 65 :kost2 34}) (pr req))

once again, thanks to joinr for a great reply.

Again, I am really dubious that the double parens itself is changing the evaluation semantics, enough to invoke a specific error path. There are times when unintentional parens can create unintended evaluation like

user=> ((+ 1 2 3))
Execution error (ClassCastException) at user/eval1 (REPL:1).
java.lang.Long cannot be cast to clojure.lang.IFn
;;vs
(+ 1 2 3)

Outside of something like that, I don’t really see what the context is here that is mapping imbalanced or out of place parens into the error you specified. I was able to walk the path through my analysis though, and arrive at an explanation (hopefully that works out for you).

I also removed the !n flag from addb but that did not solve anything.

In clojure, there aren’t really many reserved characters for symbols. There is no functional difference between the symbol add and add!, or *add or <add> etc. You have a high degree of flexibility in naming stuff. The convention is to use ! to denote mutable or side-effecting operations (like writes to db, or altering an array, or messing with an atom).

another remark. double parens gives the error:

In that example, you still did not return a response map - but the integer result from before - which is what the final bit of my post is indicating as the cause of error. So I would expect the error to be there regardless of where you space the parens.

but if I remove the pr function after addb! then the error goes away, because we get rid of the double paren:

This is not quite the whole story, and I think you are inferring causality where there is none (w.r.t. the double paren being the independent variable of consequence). What happened when you moved the pr function call to the end, is that you are now returning nil for the function handler. So your function takes a request map, and returns nil (since nil is the result of pr, where pr will cause the side-effect of printing to *out* but it returns nil as a result of the function call). In clojure, the last value in the function body is what is returned, so

user=> (pr {:a 2})
{:a 2} nil  ;;looks odd, but {:a 2} is printed to *out*, and nil is returned
user=> (println {:a 2})
{:a 2} ;;this is printed to *out*
nil     ;;more obvious that nil is returned.

So since your “new” function now returns nil, it turns out the prior error (trying to invoke contains? on an integer, which is what addb! returns) is not going to get triggered. In fact, most of the clojure core protocols and functions behave with nil without throwing an error:

user=> (contains? nil :a)
false

This has nothing to do with the parens, but everything to do with the value you are returning from the request handler, which is then being passed down to middleware. It was an integer before (not compatible with the successive call to contains?), it is now a nil (compatible, treated as empty).

2 Likes

What I meant was that I removed the n flag in defining addb! so there was no return value, below is the definition with n that returns number of rows added:

--:name addb! :! :n
--:doc adds another booking
INSERT INTO rent
(name,org,parea,barea,tarea,hyra,omk,tid,start,stop,notes,kost,kost2)
VALUES (:name, :org, :parea, :barea, :tarea, :hyra, :omk, :tid, :start, :stop, :notes, :kost, :kost2)