Hey everyone,
over the next few weeks I’ll be working on a couple of updates for the react-native integration in shadow-cljs. While it already works quite well it has some issues that can be improved and I wanted to give a quick overview of my understanding of them. I still don’t do any real react-native development myself so I’m mostly going by what others have told me. So please if you have found some rough edges or things that don’t work please let me know.
Upgrade #1: Source Maps
To explain why they don’t work it is probably best to provide some basic background in how all the tools work together. First the CLJS compiler (with or without shadow-cljs) will translate all CLJS code to JS. In doing so it’ll create the first set of source maps that map the generated parts of JS back to the original CLJS code. So after compilation you typically end up with a cljs/core.js and cljs/core.js.map file in some fashion. The source map could also be inlined in which case it’ll be part of the cljs/core.js without an extra .map file.
In the second step all .js files are processed by the react-native bundler called metro. It’ll process all code and transpile unsupported features out or provide polyfills for some. It’ll also process all import directives and require calls and provide the actual JS code for those. In turn metro will also produce source maps for the JS files it compiled. facebook invented their own source map format for reasons I don’t know but the problem here is that it doesn’t read “input source maps” at all. So it won’t read the source maps created by the CLJS compiler at all and only map errors and so on back to the generated JS but not the actual CLJS source. Other compile to JS languages suffer the same fate and there is an open issue about that.
Luckily there is a hacky way around that since react-native has a built-in “Debug JS remotely” option which will load all the JS in a remote Chrome instance which means we gain access to the capabilities of Chrome by “cheating”. re-natal uses a figwheel-bridge “cheat” where it basically first loads a dummy placeholder app which is processed by metro normally. At runtime it’ll then load the actual generated JS code dynamically and when done loading replace the placeholder with the actual app. Since Chrome properly respects the // #sourceMappingUrl directives in the source the code loaded this way will be mapped properly in the Chrome devtools.
This means the generated JS is never actually seen by metro though which means some manual work needs to be done if you want to include other JS dependencies. re-natal solves this with its command line tools I believe or you just manually repeat all the require calls you plan on using in the CLJS code.
The approach also adds an asynchronous step to the loading process. Since initialization by default it is not async you are forced to use the placeholder app. This is fine bug somewhat different than production builds where this async loading step does not occur.
So far shadow-cljs only produces JS code that could be fully processed by metro. This worked fine and no extra manual work needed to be done to extract js/require calls since metro would take care of them. It is mostly a zero-conf setup and doesn’t require a “placeholder” app. I’m told however that it is near impossible to debug issues at runtime given the lack of source maps.
The Plan
As a first step I adjusted the way the code is generated to essentially employ the same tricks as the figwheel-bridge did. “Hiding” the actual sources from metro and instead loading it dynamically. Instead of loading the code via HTTP requests however I embedded the sources inline as strings and they will just be evaluated directly on load. This means no extra async layer is introduced on startup. I first created this loader-mode for the browser a while ago and it works very well there. Source maps are also directly embedded inline so no extra request is necessary there either.
This gets source maps into a usable state as long as “Debug JS remotely” is used. The Chrome devtools correctly map (js/console.log "foo") calls and “Pause on Exceptions” also correctly show CLJS sources and lets you step through them.
The react-native “RedBox” and “YellowBox” Components to show Errors/Warnings still don’t show mapped sources but I think this cannot be cheated in any way until react-native actually supports input source maps.
The price for getting the partial source maps however is that all the generated JS code is no longer processed by metro at all which means it also won’t fill in manual js/require calls at all. However this is also a small benefit since metro can be quite slow processing all the generated JS code.
The Question
We now need a way to “gather” all JS require calls so they can be put into the sources that metro actually processes and made available to the CLJS parts. For JS code this is done completely automatically if you require code from the ns form.
(ns my.app
(:require
["react-native" :as rn]
["some-react-native-lib/foo/Bar" :as X]))
;; use via aliases normally
rn/View
rn/Text
X
;; etc.
No extra config is needed for those. They map directly to how you’d use them in JS and the compiler will take care of adding the appropriate code where necessary.
There however can be “rogue” (js/require "something") calls which the compiler will not “find” currently. That still need to be processed since require is not available at runtime.
So while there are a couple of options to get this done properly I don’t know which one people would prefer?
I’m not using react-native myself and so probably shouldn’t be the one to decide this. I’ll just list the options and listen to the feedback.
Option 1: Manual via Config
Similar to re-natal or figwheel-bridge you just list the “extra” requires you need in the config file and shadow-cljs will make sure they are available at runtime. Just need to repeat every js/require call done in the code (eg. for assets) in the config again.
Option 2: Utility Macros
I used this approach before and it works well. A macro is created which “records” all require uses and makes that info directly available to the build tool. So instead of (js/require "../some/asset.png") you’d call (some-alias/require-asset "some/asset.png"). The parts needed for this look very similar to the shadow.resource example. The problem with this is that shadow-cljs is currently the only tool to process this info and therefore if you couldn’t use code using this with other build tools. And you have that extra ns :require in each ns needing this.
Option 3: Compiler Magic
It would be sort of simple to automatically process js/require calls at the compiler level and basically make it a “special form”. So basically automated Option 1. I’m always hesitant to adding shadow-cljs only stuff to the compiler though.
Option 4: Use the ns
Technically the ns form already does everything we need and this is the way the JS world does everything after all. In JS require is used to load other JS code as well as static assets (eg. images).
In JS(X) you’ll see examples like
<Image source={require('/react-native/img/favicon.png')} />
which mapped to something like reagent looks like
[rn/Image {:source (js/require "/react-native/img/favicon.png")}]
and could be also
(ns my.fancy.component
(:require
["react-native" :as rn]
;; some kind of prefix to "mark" as not-code
["asset:/react-native/img/favicon.png" :as favicon]))
[rn/Image {:source favicon}]
I think this is getting into dangerous territory since it would be shadow-cljs specific again but it moves the configuration to the place it is actually used. No need to configure anything else again.
I also kind of like this approach since it makes things declarative again. The ns form is basically just static information that can be processed separately. All other options require actual compilation and puts more burden on the user.
But this is moving things into a dangerous messy direction the JS world is in right now and I kinda want to keep this out of CLJS. Pure developer convenience isn’t always the best idea.
Anyways. I’m curious what actual react-native developer think and which approach you’d prefer.
- Manual Config
- Utility Macros
- Compiler Magic
- Use the ns
0 voters
