Generating reflection-free Java wrappers

The Weird Java Interop issues thread sent me down a bit of a rabbit hole, figuring out if it would be possible to automatically generate Clojure wrappers for Java classes, which include all possible type information, so you never have to type hint again.

Like I said I got a bit sucked in, the result I present: defwrapper. It’s likely this will become a library, but I first want to use it a bit more myself to get a feel for it. But I also wanted to share it to see what people think.

The idea is simple. Say I want to call methods from the java Synthesizer interface, then I do this,

(defwrapper javax.midi.Synthesizer "synth-")

Then this generates (among others) functions like

(defn synth-get-default-soundbank
  {:arglists '([javax.sound.midi.Synthesizer])}
  (^javax.sound.midi.Soundbank [^javax.sound.midi.Synthesizer this13886]
    (.getDefaultSoundbank this13886)))

It will handle multiple arities, static methods, method overloads.

And the result (try it, it’s fun :slight_smile: )

(defwrapper javax.sound.midi.MidiSystem "midi-sys-")
(defwrapper javax.sound.midi.Synthesizer "synth-")
(defwrapper javax.sound.midi.MidiChannel "chan-")

  ;;; Play some tunes

(def synth (midi-sys-get-synthesizer))

(synth-open synth)

(def chan (first (synth-get-channels synth)))

;; play some chords
(do
  (chan-note-on chan 60 600)
  (chan-note-on chan 64 600)
  (chan-note-on chan 67 600)

  (Thread/sleep 1100)

  (chan-note-on chan 60 600)
  (chan-note-on chan 64 600)
  (chan-note-on chan 67 600)
  (chan-note-on chan 71 600))

So is this a good idea? I’m still not sure. It was definitely more challenging than I thought it would be, and a good refresher on the finer points of type hints. (and metadata in macros, oh my). Primitives are a pain, as Clojure only allows hinting long and double, and only in some places (in others you need to cast, (long ...)). There’s no void (I used Object instead). There are some cases that are hard to handle, e.g. what if two method with the same arity but different argument types have different return types? (hopefully that’s not too common).

So the result is still pretty rough, it uses boxed primitives in lieue of the actual primitive types in some cases, which works but might introduce extra boxing/unboxing. Varargs still requires creating a Java array and passing that in. Constructors aren’t wrapped yet.

But… I also think it’s pretty cool. It causes issues like the one in the “Weird Java Interop” thread to disappear, I think it can take away some of the sharp edges of dealing with Java interop. Does everyone really need to know that sometimes you need to type hint to the public interface in order for things to work? Does anyone enjoy hunting down reflection warnings because someone wants to use their library on Graal?

I’ll be playing with this a lot more I think. Even when not using it (yet) in production code I would 100% use it today when I quickly want to use a Java library from the REPL.

Some more examples of the generated code:

(defn channel-note-off
  {:arglists '([javax.sound.midi.MidiChannel int int]
               [javax.sound.midi.MidiChannel int])}
  (^java.lang.Object [^javax.sound.midi.MidiChannel this13946 ^java.lang.Integer int13947 ^java.lang.Integer int13948]
   (.noteOff this13946 int13947 int13948))
  (^java.lang.Object [^javax.sound.midi.MidiChannel this13949 ^java.lang.Integer int13950]
   (.noteOff this13949 int13950)))

(defn midi-sys-get-midi-file-format
  {:arglists '([javax.sound.midi.MidiSystem java.io.InputStream]
               [javax.sound.midi.MidiSystem java.io.File]
               [javax.sound.midi.MidiSystem java.net.URL])}
  (^javax.sound.midi.MidiFileFormat [G__13976]
   (cond
     (and (instance? java.io.InputStream G__13976))
     (let [G__13976 ^java.io.InputStream G__13976]
       (javax.sound.midi.MidiSystem/getMidiFileFormat G__13976))

     (and (instance? java.io.File G__13976))
     (let [G__13976 ^java.io.File G__13976]
       (javax.sound.midi.MidiSystem/getMidiFileFormat G__13976))

     (and (instance? java.net.URL G__13976))
     (let [G__13976 ^java.net.URL G__13976]
       (javax.sound.midi.MidiSystem/getMidiFileFormat G__13976)))))

And the current state of things, do try it out and report back!

7 Likes

I don’t understand how you would disambiguate ambiguous types? My understanding is that type hints are only needed when it just simply isn’t possible to know, because there are multiple possibilities and only the programmer knows which one is needed.

Like try it on say:

public class Foo {
    public String bar(int a) {
        return "a";
    }
    public int bar(String a) {
        return 10;
    }
    public String bar(String a) {
        return "b";
    }
}

I like where this is going!

I made something similar for mapping to and from data beans – could be a nice compliment for some APIs:

1 Like

In this case you would get

(defn bar [^Foo this g__123]
  (cond
    (instance? java.lang.Integer g__123)
    (.bar this ^java.lang.Integer g__123)

    (instance? String g__123)
    (.bar this ^String g__123)))

(which still needs to be improved to prevent unnecessary boxing)

Which does have some type checks but it doesn’t reflect on the class and method, which is a big difference. In this case your return types differ, which is why there is no return type hint. But like I said I think that is a pretty rare scenario.

Oh interesting… Is that actually better then reflection? In terms of performance and compatibility with graalVM?

Also, what about scenarios where there are multiple concrete classes or interfaces?

I do believe it’s better. With reflection you have to at runtime

  • get the class of the object you’re calling a method on
  • list its methods
  • filter them by arity and type of the arguments
  • filter them by visibility
  • find the one that is left, invoke it through the reflection API

in the above cases each call to .bar is unambiguous, the compiler can know which one it is, and emit byte code that calls it directly.

Also, what about scenarios where there are multiple concrete classes or interfaces?

When you create the wrapper you need to pick the class or interface you’re wrapping, only objects that are assignable to that type will be able to be used with those generated functions. If there’s an interface then you should prefer that over the concrete object.

Nice! I never had to deal much with the Bean stuff (except for a project in uni involving enterprise java beans, still gives me the shivers), but this looks really useful.

Many Java APIs are replete with them, if only to model data with. This library was motivated by using the AWS SDK from Clojure, for example. (Today I just use Cognitect’s AWS lib for this case.)

2 Likes

I see. Ya, actually that’s pretty neat, I think you’re on to something.

Would be nice if you could also have it so the macro throws when there is ambiguity, and allows you to explicitly type hint them. That would achieve maximal performance.

Or we should benchmark the different approach. Cause it is true the JIT might be able to optimize the instance of away.

I like the idea of just lifting functions into a namespace (inverting the OOP stuff basically).
I did something like this out of necessity, but for a narrower use-case: selectively exposing private fields with accessor functions that had no business being private (from jfreechart I think, maybe piccolo2d). I came off of that thinking "why isn’t this the default for interop? just have a library of functions automatically generated. Then you went and did it, hah!

1 Like

This is great, data hiding can be so overrated :slight_smile:

Does this still work on Java 9+?

This seems pretty cool and exactly what I need at the moment!!! :smile:

Nice! Do let me know what you think once you’ve managed to use it a bit!

I haven’t tested it, but it’s definitely on the way out due to the breaking changes they introduced (you know…not allowing you to be a consenting adult in favor of modularity - silliness IMO).

It should work (with warnings and/or jvm startup flags) for the time being, but at some point using reflection to crack open object vaults will be verboten. Thankfully, I’m likely to still be on java 1.8 by then due to the nature of my work :wink:

This also reminds me of an old lib I wrote to get better transparency of Java objects: gavagai. It creates objects that expose a lazy map-like interface for java beans and bean-like objects in a recursive way. You can customize precisely what you want to expose and how you optionally want to transform values. It emits functions that are annotated to do no reflection on access.

By the way, it has had no commit in over 4 years, and still works perfectly with clojure 1.10.1 and Java 12, kudos for stability!

1 Like

This would be a great foundation for implementing datafy/nav for all kinds of objects.

Hi,

this looks pretty interesting. I have done a similar-ish thing for java.time, the output of which is in this lib: https://github.com/henryw374/cljc.java-time. It’s a bit different in that it is .cljc and has to deal with a few irregularities in the underlying js impl of java.time.

I also found type hinting a bit of a struggle and so quite a bit is still tbd… I might leverage some of your code to help with that :-).

Btw I had a quick go with e.g. java.time.ZoneId and see the overloaded ‘of’ method is missing the (String,bool) version.

@henry_w where are you seeing this ZoneId/of method, I can only find (String) and (String,Map) versions.

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/ZoneId.html

ah sorry, it has default visibility, not public. so not expected

Hey, so I have taken bits of this to generate better type hinting in my java.time lib - thanks again.

One case that is still problematic is java.time.Year#isLeap. There’s a static and non-static versions with the same name - urghh. it remains special-cased for now.

btw I’ve been referring to the java.time lib as a ‘vacuum wrapper’, as in this kind of thing https://images.app.goo.gl/x8ctNQBPYewXTLy3A - in case you’re looking for names/analogies.