Exception when creating a release in shadow-cljs

Questions & Help Troubleshooting
1

Hi!
Clojure(script) newbie here, trying to migrate a lein+figwheel project to a lein+shadow-cljs one. I’m in the process of the debugging the thousands of errors that I’m getting, but I’m interested in fixing why the command lein shadow release app is raising an exception that I don’t have a clue what it could be about. This is the error:

. . .

-> build target: :browser stage: :compile-finish
-> Checking used npm package versions
<- Checking used npm package versions (19 ms)
<- build target: :browser stage: :compile-finish (25 ms)
-> build target: :browser stage: :optimize-prepare
<- build target: :browser stage: :optimize-prepare (0 ms)
-> Closure - Optimizing ...
IllegalStateException: DecomposeExpression depth exceeded on:
CALL 317 [length: 35378] [free_call: 1] [source_file: my-company/commons/props_from_fn.cljc]
    NAME my-company$commons$props_from_fn$question_STAR_ 317 [length: 35378] [source_file: my-company/commons/props_from_fn.cljc]
    NAME question$jscomp$27 317 [length: 35378] [source_file: my-company/commons/props_from_fn.cljc]
    NAME s_id$jscomp$28 317 [length: 35378] [source_file: my-company/commons/props_from_fn.cljc]
    NAME answers$jscomp$inline_2838 317 [length: 35378] [source_file: my-company/commons/props_from_fn.cljc]

	com.google.javascript.jscomp.ExpressionDecomposer.maybeExposeExpression (ExpressionDecomposer.java:114)
	com.google.javascript.jscomp.FunctionInjector$CallSiteType$6.prepare (FunctionInjector.java:520)
	com.google.javascript.jscomp.FunctionInjector.maybePrepareCall (FunctionInjector.java:593)
	com.google.javascript.jscomp.InlineFunctions.decomposeExpressions (InlineFunctions.java:794)
	com.google.javascript.jscomp.InlineFunctions.process (InlineFunctions.java:152)
	com.google.javascript.jscomp.PhaseOptimizer$NamedPass.process (PhaseOptimizer.java:317)
	com.google.javascript.jscomp.PhaseOptimizer$Loop.process (PhaseOptimizer.java:462)
	com.google.javascript.jscomp.PhaseOptimizer.process (PhaseOptimizer.java:232)
	com.google.javascript.jscomp.Compiler.performOptimizations (Compiler.java:2417)
	com.google.javascript.jscomp.Compiler.lambda$stage2Passes$1 (Compiler.java:802)
	com.google.javascript.jscomp.CompilerExecutor.runInCompilerThread (CompilerExecutor.java:129)
	com.google.javascript.jscomp.Compiler.runInCompilerThread (Compiler.java:829)
	com.google.javascript.jscomp.Compiler.stage2Passes (Compiler.java:799)
	com.google.javascript.jscomp.Compiler.compileModules (Compiler.java:744)
	sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethodAccessorImpl.java:-2)
	sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
	sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke (Method.java:498)
	clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:167)
	clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:102)
	shadow.build.closure/compile-js-modules (closure.clj:1038)
	shadow.build.closure/compile-js-modules (closure.clj:1027)
. . .

What could be wrong with the file my-company/commons/props_from_fn.cljc? What is the error about? Any way to fix it?
Thank you very much for your help

2

I’ve never seen this error. Would help to see the related code. No guesses otherwise.

debugging the thousands of errors

Any other examples? Maybe some are related?

Might be a dependency issue, make sure you are getting all the correct versions. See more info here. Probably not though since it seems to compile fine otherwise and just fails on the closure-compiler :advanced optimizations. Did you not use those before?

3

The dependencies of shadow-cljs should be correctly updated… I’ll post my whole project.clj here so maybe you can spot what is wrong right away (using lein-shadow so shadow-cljs config is here).

(defproject my-company "0.1.0-SNAPSHOT"
  :description "My-company"
  :url "http://domain.com/FIXME"
  :license {:name "Commercial"
            :url "http://domain.com/licence.html"}
  :dependencies [[org.clojure/clojure "1.10.1"]
                 ;;[org.clojure/core.async "0.2.385"]
                 [org.clojure/core.async "1.2.603"]
                 [org.clojure/core.match "0.2.2"]
                 [com.datomic/datomic-free "0.9.5390" :exclusions [org.slf4j/slf4j-nop
                                                                   org.slf4j/log4j-over-slf4j
                                                                   joda-time
                                                                   com.google.guava/guava
                                                                   commons-codec]]
                                        ;[com.datomic/datomic-pro "0.9.5786" :exclusions [org.slf4j/slf4j-nop
                                        ;                                                  org.slf4j/log4j-over-slf4j
                                        ;                                                  joda-time
                                        ;                                                 com.google.guava/guava]]

                 [org.clojure/tools.nrepl "0.2.12" :exclusions [[org.clojure/clojure]]]
                 [com.draines/postal "1.11.4"]
                 [com.cognitect/transit-clj "1.0.324"]
                 [com.stuartsierra/component "0.3.1"]
                 [liberator "0.14.0"]
                 [buddy "0.8.2"]
                 [cheshire "5.6.3"]
                 [compojure "1.5.1"]
                 [garden "1.3.0"]
                 [hiccup "1.0.5"]
                 [http-kit "2.2.0"]
                 [org.clojure/data.json "1.0.0"]
                 [clj-time "0.11.0"]
                 [ring/ring-core "1.8.1" :exclusions [org.clojure/tools.reader]]
                 [ring/ring-json "0.4.0"]
                 [ring/ring-devel "1.4.0"]
                 [ring/ring-defaults "0.1.5"]
                 [ring-transit "0.1.4"]
                 [bouncer "0.3.3"]

                 ;;Read and write office docs
                 [dk.ative/docjure "1.12.0"]
                 ;; Pdf
                 [clj-pdf "2.2.30"]
                 ;; Logging
                 [org.clojure/tools.logging "0.3.1"]
                 [org.slf4j/slf4j-log4j12 "1.7.12"]

                 ;; Client
                 [org.clojure/clojurescript "1.10.758" :exclusions [org.clojure/data.json]]
                 [com.google.javascript/closure-compiler-unshaded "v20200504"]
                 [org.clojure/google-closure-library "0.0-20191016-6ae1f72f"]
                 [org.clojure/google-closure-library-third-party "0.0-20191016-6ae1f72f"]
                 [reagent "1.0.0-alpha2"]
                 [re-frame "1.0.0-rc2"]
                 ;;[reagent "0.6.0"]
                 ;;[re-frame "0.8.0"]
                 [day8.re-frame/http-fx "0.1.2"]
                 [com.andrewmcveigh/cljs-time "0.5.2"]
                 [day8.re-frame/undo "0.3.2"]
                 [cljs-ajax "0.5.8"]
                 [cljsjs/pikaday "1.5.1-2"]
                 [caesium "0.10.0"]
                 [com.taoensso/nippy "2.14.0"]
                 [google-apps-clj "0.6.1" :exclusions [com.google.guava/guava-jdk5]]

                 [com.google.apis/google-api-services-gmail "v1-rev83-1.23.0" :exclusions [com.google.guava/guava-jdk5]]
                 [org.clojure/data.codec "0.1.1"]
                 [clojurewerkz/quartzite "2.1.0"]
                 [org.clojure/tools.cli "1.0.194"]
                                        ;[aramis "0.1.1"]
                 [day8.re-frame/tracing "0.5.5"]
                 [thheller/shadow-cljs "2.9.10"]]

  :plugins [[lein-shadow "0.2.0"]
            [lein-shell "0.5.0"]
            [cider/cider-nrepl "0.25.0-alpha1"]
            [lein-codox "0.10.4"]]

  :main my-company.core


  :repositories
  {"my.datomic.com" {:url "https://my.datomic.com/repo"
                     :username "whatever"
                     :password "whatever-pass"}}

  :repl-options {:timeout 180000
                 }

  :codox {:language :clojure
          :output-path "target/doc/clj"}

  :profiles {:dev {:source-paths ["dev"]
                   :dependencies [[org.clojure/tools.namespace "0.3.0-alpha3"]
                                  [org.clojure/java.classpath "0.2.3"]
                                  [criterium "0.4.3"]
                                  [binaryage/devtools "1.0.0"]
                                  [day8.re-frame/re-frame-10x "0.6.5"]
                                  ]}
             :codox-cljs {:codox {:language :clojurescript
                                  :output-path "target/doc/cljs"}}
             :prod {}}


  :source-paths ["src/my-company/client" "src"]

  :clean-targets ^{:protect false} ["resources/public/cljs-runtime"
                                    "resources/public/app.js"
                                    "target"]
  :test-paths ["test"]

  :aliases {"dev"          ["with-profile" "dev" "do"
                            ["shadow" "watch" "app"]]
            "prod"         ["with-profile" "prod" "do"
                            ["shadow" "release" "app"]]
            "build-report" ["with-profile" "prod" "do"
                            ["shadow" "run" "shadow.cljs.build-report" "app" "target/build-report.html"]
                            ["shell" "open" "target/build-report.html"]]
            "karma"        ["with-profile" "prod" "do"
                            ["shadow" "compile" "karma-test"]
                            ["shell" "karma" "start" "--single-run" "--reporters" "junit,dots"]]}

  :shadow-cljs {:nrepl {:port 8777}


                :builds {:app {:target :browser
                               :output-dir "resources/public/"
                               :asset-path "/dev"
                               :modules {:app {:init-fn my-company.client/render-app
                                               :preloads [devtools.preload
                                                          day8.re-frame-10x.preload
                                                          ]}}
                               :dev {:compiler-options {:closure-defines {re-frame.trace.trace-enabled? true
                                                                          day8.re-frame.tracing.trace-enabled? true}}}
                               :release {:build-options
                                         {:ns-aliases
                                          {day8.re-frame.tracing day8.re-frame.tracing-stubs}
                                          :par-timeout 6000000}}

                               :devtools {:http-root "resources/public"
                                          :http-port 8280
                                          :after-load my-company.client/render-app
                                          ;;:http-handler my-company.client/init-user-data
                                          }}
                         :browser-test
                         {:target :browser-test
                          :ns-regexp "-test$"
                          :runner-ns shadow.test.browser
                          :test-dir "target/browser-test"
                          :devtools {:http-root "target/browser-test"
                                     :http-port 8290}}

                         :karma-test
                         {:target :karma
                          :ns-regexp "-test$"
                          :output-to "target/karma-test.js"}}}




  ;;:cljsbuild {:builds [{:id "dev"
  ;;:source-paths ["src/"]
  ;;:figwheel {:on-jsload "my-company.client/render-app"}
  ;;:compiler {:main "my-company.client"
  ;;:asset-path "dev"
  ;;:output-to "resources/public/app.js"
  ;;:output-dir "resources/public/dev"}}
  ;;{:id "prod"
  ;;:source-paths ["src"]
  ;;:compiler {:output-to "resources/public/app.js"
  ;;:optimizations :advanced
  ;;:externs ["externs.js"]
  ;;:pretty-print false}}]}
  ;;:figwheel {:css-dirs ["resources/public/css"]})
  )

(I left part of the previous config that was using figwheel commented out at the end)

In the process of porting to shadow-cljs I had to bump the version of some of my dependencies, although I left others untouched. I left most of the exclusions that were previously defined as well, even though they might not still be necessary.

About the errors, I get around 50 warnings related to the use of the complete namespace in order to call a function, but I gave up on them when I saw that they were raising circular dependency issues when replaced by "require"s. Are these warnings very important? The errors look like this:

------ WARNING #47 - :undeclared-var -------------------------------------------
 File: /home/jbarren/workspace/my-company/src/ubikare/client/user_profile/util.cljs:237:15
--------------------------------------------------------------------------------
 234 |   ::appdb-add-client-profile-data
 235 |   (fn [{:keys [db]} [_ client result]]
 236 |     (let [previews-nav @(re-frame/subscribe [:nav])]
 237 |       (reset! my-company.client.user-profile.profile-read/selected-tab 0)
---------------------^----------------------------------------------------------
 Use of undeclared Var my-company.client.user-profile.profile-read/selected-tab
--------------------------------------------------------------------------------
 238 |       {:db (-> db
 239 |                (assoc :previews-nav previews-nav)
 240 |                (assoc :client-profile-data result)
 241 |                (assoc :loading? false)
--------------------------------------------------------------------------------

------ WARNING #48 - :undeclared-var -------------------------------------------
 File: /home/jbarren/workspace/my-company/src/my-company/client/questions.cljs:48:14
--------------------------------------------------------------------------------
  45 |   To do so, first the systems is prepared turning a flag to say there are answers to save (`s-save/arm`)
  46 |   Then, current answers location is combined with the given `path` to pass it to the function that will add the answers."
  47 |   [path new-value]
  48 |   (if (not= @my-company.client.hps.util/selected-subtab :antec-meos) (s-save/arm))
--------------------^-----------------------------------------------------------
 Use of undeclared Var my-company.client.hps.util/selected-subtab
--------------------------------------------------------------------------------
  49 |   (re-frame/dispatch [::assoc-in-answers
  50 |                       (apply merge [:survey-answers] path)
  51 |                       new-value]))
  52 | 
--------------------------------------------------------------------------------

What I’m currently experiencing is that when I run lein shadow watch up those warnings are raised, but when I try to load localhost:8280 the cpu goes crazy and there is nothing loaded in the screen nor any error logs. So my only way of going forward has been compiling the app, running lein run to serve it from the backend and check the errors in console.

The client side code lives under src/my-company/client.

4

Well, circular requires are not allowed and working arround them by using the fully qualified name is not exactly supported. The warnings are there for good reason. Why is that stuff not just part of the re-frame db state? Why does it have to live in a separate ns/atom?

The warnings are sticky and will prevent everything else (eg. hot-reload, caching) from working. You’ll need to fix those. shadow-cljs is much stricter in this area and it will keep bugging you about that.

 :source-paths ["src/my-company/client" "src"]

This is also kinda invalid since "src/my-company/client" is a child of src so you’ll end up with duplicated files on the classpath. my-company is also invalid and should likely be my_company as the convention for CLJ(S) dictates that - in namespaces is replaced by _ for file names.

5

Thanks for the quick response.

Well, circular requires are not allowed and working arround them by using the fully qualified name is not exactly supported.

All right, so I guess I’ll have to work on fixing this first before anything else then.

Why is that stuff not just part of the re-frame db state? Why does it have to live in a separate ns/atom?

I don’t know the answer to this, but I’ll look into it. I just joined the team a couple of weeks ago, so I’m not familiar with the codebase yet.

my-company is also invalid and should likely be my_company as the convention for CLJ(S) dictates that - in namespaces is replaced by _ for file names.

Yeah, sorry about that. I just renamed the instances of my company’s name by my-company after pasting it here, so this is not the exact content of the code.

Thanks again for the help!

6

I assumed so. Just mentioning it in case there is an actual - in the file path. The important bit is not adding :source-paths that contain each other.

7

After fixing all those circular dependency errors I’m afraid I’m still getting the same error when I try to create a release, and what is worse for me at the moment is that I’m not able to run a shadow-cljs watch app correctly and therefore don’t have a REPL nor hot-reloading. When I run the watch everything goes without errors until the end, but when I try to load the page at localhost:8280 the cpu goes to 100% usage and nothing gets loaded.

The initial page gets created via hiccup from a views.clj file, where it contains a div with the “app” id, and the client.cljs file hooks that id with a main reagent component. Pretty standard. But my guess is that the development server needs to stick to serve a index.html file from the root, because in this case the web is created from the backend. I created this index.html file, replicating what it is being created by hiccup and the backend, but I don’t see any change.

What am I missing in the config? Could the error creating the release be related to the error using the watch?

EDIT: Forgot to mention that the config is still the same as the one above in the post, and that when I compile and do a lein run I can load the app and everything works fine (without a REPL nor hot-reloading of course)

8

Once thing that I noticed is that after running shadow-cljs watch app --verbose in the last line y always end up with something like this:

[:app] Build completed. (703 files, 0 compiled, 0 warnings, 5,41s)

0 compiled.

The app.js is created (17Mb of bundle), and it outputs the same even if I run lein clean and delete the app.js file beforehand.

9

That is likely your code doing this and has nothing to do with shadow-cljs. Does the browser console have anything to say? You are using :init-fn my-company.client/render-app in your config. Maybe comment out whatever the function is doing so it just does nothing. If you still get to 100% then something weird is going on.

It is completely irrelevant to shadow-cljs how your HTML/JS is loaded. They are just static files. All the dynamic stuff at runtime is done over websockets at runtime. There is no requirement to do anything at all when serving the files except serving the files. You do not need to use the shadow-cljs server for it, literally any webserver will do. Including webservers which generate the initial HTML dynamically, there is no requirement for a static index.html.

Config looks fine.

No, maybe. The error you initially posted is coming from the Closure Compiler doing the :advanced optimizations to minify your code. watch does not do this so it doesn’t exhibit this error. Whatever code is causing this issue however may still be present in development watch builds. Don’t bother trying to create a release build until watch or compile work though.

This is because it used the cache and didn’t need to compile anything. You can delete the .shadow-cljs/builds directory to wipe the cache but that should never be required.

10

Ok, so what I understand from this is that for development I should use lein run in which I deploy the app in the port 3000 and works fine, and then in another terminal run lein dev which is a (shadow-cljs watch app).

I can see that after removing every instance of a web server config from the :devtools section (:http-root, :http-port etc) looks like the lein run deployment wants to connect with the server via websockets, and complains when the ẁatch` is not running.

With this setup it looks like both the deployment and the watch are connected, but as soons as I make a change in the code I can see a recompilation, but no refresh in the browser running localhost:3000.

My devtools section is empty now. Is there any extra config that I might be missing? There is no error in any of the terminals nor in the browser…

11

Sounds like you also removed :after-load my-company.client/render-app? This tells shadow-cljs to call that function when it is finished re-loading your code and re-rendering is then up to you.

See
Shadow CLJS User’s Guide (you may also use the metadata variant instead of the :devtools config).
Hot Reload in ClojureScript

12

I’ve tried using the metadata variant and the devtools variant and neither of them seem to call my render-app function. Added a prn in that function and I’m not getting any log in the browser at all.

Following an example from a leiningen re-frame template, I’ve tried creating a :http-handler that points to a handler like this:

(def dev-handler (-> #'routes wrap-reload push-state/handle))

Where wrap-reload is a function from ring.middleware and push-state/handle is shadow-cljs function. This didn’t seem to work either.

Tried to load a semi-empty index.html file, with no link to the created js just to test that the web server is capable of loading the index file, but that didn’t work either. No error in console, just a 100% cpu.

That made me think that the problem must be somewhere in my code. There were a couple of warnings at the beginning of the process which I didn’t pay much attention to, and they looked like they were caused by the upgrade of some libraries so they needed to be upgraded too. So now that there is no error (not entirely true… I’m getting this warning too: “JSDoc annotations are not supported on return” caused by React-big-calendar component, but I don’t think this would be causing all this), I’m getting this error:

[:app] Build failure:
no output for id: [:shadow.build.classpath/resource "goog/base.js"]
{:resource-id [:shadow.build.classpath/resource "goog/base.js"]}
ExceptionInfo: no output for id: [:shadow.build.classpath/resource "goog/base.js"]
        shadow.build.data/get-output! (data.clj:196)
        shadow.build.data/get-output! (data.clj:192)
        shadow.build/enhance-warnings (build.clj:26)
        shadow.build/enhance-warnings (build.clj:22)
        shadow.build/extract-build-info/fn--66649 (build.clj:97)
        clojure.core/map/fn--5866 (core.clj:2753)
        clojure.lang.LazySeq.sval (LazySeq.java:42)
        clojure.lang.LazySeq.seq (LazySeq.java:51)

Just in case, and I know it’s an overkill, I’m running this command:

rm -rf .shadow-cljs/builds && rm -f shadow-cljs.edn && lein clean && lein deps && lein shadow compile app && lein run

This way I can run and deploy the application, but I’m getting the error above when I run : lein shadow watch app --verbose.

Not sure what could be causing this, but I have the dependencies for shadow-cljs correctly defined (shown in a previous post).

EDIT: Looks like the error is gone after some more retries, but still can’t get hot-reload nor repl connection to work

EDIT2: Just in case it helps, this is what I get in the console when I use a watch with --debug --verbose when I make a change and it recompiles. In the browser I don’t get anything in the console (remember that I launch the app from the backend with a lein run, so a ring handler generates the initial html. Then I launch the watch):

[:app] Build completed. (703 files, 2 compiled, 0 warnings, 0,70s)
[:app] Compiling ...
-> Resolving Module: :app
<- Resolving Module: :app (29 ms)
-> build target: :browser stage: :compile-prepare
<- build target: :browser stage: :compile-prepare (0 ms)
-> Compile CLJS: company/client/core.cljs
-> Compile CLJS: company/client.cljs
<- Compile CLJS: company/client.cljs (16 ms)
-> Cache write: company/client.cljs
<- Compile CLJS: company/client/core.cljs (107 ms)
-> Cache write: company/client/core.cljs
<- Cache write: company/client.cljs (128 ms)
<- Cache write: company/client/core.cljs (141 ms)
-> build target: :browser stage: :compile-finish
<- build target: :browser stage: :compile-finish (10 ms)
-> build target: :browser stage: :flush
-> Flushing unoptimized modules
-> Flush: company/client/core.cljs
-> Flush: company/client.cljs
<- Flush: shadow/module/app/append.js (1 ms)
<- Flush: company/client.cljs (2 ms)
<- Flush: company/client/core.cljs (3 ms)
<- Flushing unoptimized modules (216 ms)
<- build target: :browser stage: :flush (226 ms)
[:app] Build completed. (703 files, 2 compiled, 0 warnings, 0,63s
13

This error has been reported. Haven’t figured out what is causing it but just compiling again seems to fix it.

What does the browser console say? shadow-cljs will tell you what it is doing. The compile log looks fine.

14

That’s the problem. The console doesn’t say anything. This is a screenshot of the application with a recompilation in the watch performed (the logs belong to the function that starts the rendering):

image
image1915×323 31.1 KB

And this is what shows when closing the watch process:

image
image1884×290 85.1 KB

Is there any way that I can get more logs about the process? I only get to see the shadow-cljs logs in the console when there is a disconnection like this

15

Now I’m super confused. If you close the watch process than it is expected that the websocket connection dies because the watch provided that?

16

Yes, I showed that just to probe that it is connected to the server of the watch process.

17

This is also confusing. This is server side and has absolutely no effect whatsoever on shadow-cljs hot-reload. It is not required that you do anything at all on your server-side for hot-reload to work. Anything you do in that area is irrelevant.

The only parts playing a part in hot-reload is the shadow-cljs watch instance which provides its own server independent of yours and the websocket connection from your page to that server.

18

And you also showed the “Render-app called!!!” message which I took for “this is all working as expected”?

19

Yes, but that message is shown the first time it gets deployed via ring and lein run. It gets rendered twice apparently, but it is as part of the first deployment, not as part of a re-rendering after a change was made in the code. I see now that showing the log twice might have been confusing…

Whenever I save a file and the code gets recompiled in the console I don’t see any change in the browser console.

20

I’m sorry. I’m way too confused and don’t know what your actual problem is anymore.

You need to have YOUR server running as well as the shadow-cljs watch. They run separately, independent of each other. Hot-reload only works while the watch is running. You should be loading the code served by YOUR server. You are not supposed to make any adjustments to the shadow-cljs server.

When your code loads initially it should log a “Websocket connected” message. If it doesn’t do that then something in your code prevents that from happening. You mentioned 100% CPU use so maybe some of your code runs in an infinite loop or something? I have not seen any of your code and cannot say what you are doing.