java.lang.NoClassDefFoundError in scheduled function

Community Center Beginners
1

I have a background task that pings the API of Scalegrowth.ai and imports the data, once a day. If any of the HTTP calls fail with an Exception, then I call my “failure” function with the same arguments and I have it retry in the future.

I use at/at for scheduling tasks on the thread pool:

I use cli-http for the HTTP calls:

The initial calls to ScaleGrowth mostly work. About 1% of the calls fail, so the “failure” function gets called.

My “fetch” function seems to work perfectly so I won’t show it here. My “failure” function simply calls “fetch” again and schedules the call for the future.

But I am getting the NoClassDefFoundError on the last line of this code:

(declare fetch)


(defn failure

  ([url item-type tp]
   (log/warn :scalegrowth "failure scheduled for retry: " url " " item-type)
   (Thread/sleep (long (rand-int 180000)))
   (at/at (+ 180000 (at/now)) #(fetch url item-type tp) tp))

  ([url item-type tp next-page]
   (log/warn :scalegrowth "failure scheduled for retry: " url " " item-type " next-page: "  next-page)
   (Thread/sleep (long (rand-int 180000)))
   (at/at (+ 180000 (at/now)) #(fetch url item-type tp next-page) tp)))

Why is that?

2

What is the full message and stacktrace of that exception?

3

It seems like I was able to fix the error by calling the function fetch directly instead of scheduling it in an anonymous function. In other words:

(declare fetch)


(defn failure

  ([url item-type tp]
   (log/warn :scalegrowth "failure scheduled for retry: " url " " item-type)
   (Thread/sleep (long (rand-int 180000)))
   (fetch url item-type tp))

  ([url item-type tp next-page]
   (log/warn :scalegrowth "failure scheduled for retry: " url " " item-type " next-page: "  next-page)
   (Thread/sleep (long (rand-int 180000)))
   (fetch url item-type tp next-page)))

Is there some rule that I shouldn’t declare a function name if I am going to later use it inside of an anonymous function?

4

It’s the last line of code that was causing problems, and so now the last line of code seems to be fixed. But I wish I understood why the error happened.

5

It most definitely has nothing to do with anonymous functions and rather with at/at or at/now. That’s why I asked for the stack trace and the full error message - where exactly did the error originate from (it might be some place other than the last line in the quoted code), which class was not found?

6

Interesting. I will investigate this tomorrow. For now I am calling the function directly, but failure is called from fetch and failure then calls fetch so I am worried about the risk of stack overflow. That is why I was using at/at. If I scheduled it on the thread pool, I would not have worry about stack overflow.

I can post the stack trace but is that useful with anonymous functions called on some thread? I’ve looked at the stack trace and with anonymous functions on some thread it seems like the useful information is lost.

7

Need more information. Just post the stack trace and the actual exception; answer the pending questions. That will advance the discovery process.

1 Like
8

I’m baffled by this one. I added the at/at call back to the code, but the error has no recurred in days. I’ve no idea what caused it, or what solved it.

(declare fetch)


(defn failure
  [url item-type tp next-page verb]
  (log/warn :scalegrowth "failure scheduled for retry: " url " " item-type " next-page: "  next-page)
  (Thread/sleep (long (rand-int 3000000)))
  (at/at (+ 1000 (at/now)) #(fetch url item-type tp next-page verb) tp))