Analyzing Java sources / .class files using javaparser or javap

[PEZ]

@colinfleming

I think that it’s a far more common use case in Clojure projects to want Java info for dependencies used via interop. Not as many projects have mixed Clojure/Java source.

This. With the strong interop story that Clojure has, looking up docs and signatures, while using it, is super convenient.

As for the whole initiative. Love it! Let me throw in some inspiration:

F(by) 2016 - Bozhidar Batsov - Speech 2 - Deep Dive into CIDER

[colinfleming]

I’ll see your inspiration, and I’ll raise you one. Bozhidar says:

There are two potential classes that implement the toUpperCase method. One is java.lang.String and the other is java.lang.Character. We cannot possibly know which one you are trying to evaluate here. … It’s not ideal, but it simply cannot be done in any other way.

Ummm… in the example, he’s literally calling it with a String receiver. Here’s Cursive:

I hear you say: but that’s a trivial example, and real code doesn’t look like that! Ok:

Cursive runs local type inference in the editor, mimicking what the Clojure compiler does. This allows you to have almost Java-level completions. The main issue is that in Clojure, when you’re typing out your code, the method comes before the receiver. No problem, you can do this:

Put the receiver first and use completion for your interop, and since Cursive knows the type of the receiver, you’ll only get completions relevant for that type. When you actually complete the method, Cursive will swap them around for you:

You don’t need to do that if your code is naturally structured in a way which allows the type to be determined:

Since the receiver comes first in the threading form, Cursive knows its type with no swapping required.

Additionally, note that neither List nor Iterator are imported in this ns, but Cursive knows their types due to the inference. If you want to know the inferred type at any time, you can just ask:

There’s plenty more, I could go on… this all works for Kotlin and Scala code too, you can rename Java/Kotlin/Scala methods from your Clojure code, finding usages of the JVM methods will find the usages in your Clojure code and vice versa, etc etc.

There’s lots that can be done in this space!

[borkdude]

Thanks for sharing Colin, that’s really cool. Clj-kondo has a basic form of type inference too that could be leveraged here. Still contemplating which route to go for Java bytecode/source analysis.

[didibus]

I also think just listing out the completion of all possible types grouped by each type would be great.

Like I’m smart enough to quickly find the section of the type I know I have. Bonus point if it’s ordered by child-most type.

One question, what it it came from a global or a function parameter that had a type hint, would completion also work then?

[colinfleming]

@didibus

One question, what it it came from a global or a function parameter that had a type hint, would completion also work then?

Yes, it does. The subs example above knows about the type because subs has a type hint saying what it will return. Local function arg hints and local binding hints also work, as well as various places where things are implicitly type hinted (e.g. this args in reify/deftype/extend-type method implementations).

[didibus]

Ok, maybe I’m pushing my luck, but how far does this the inference goes?

Example 1:

(defn hello
  []
  "hello"

(.toUpperCase (hello))

No type hint on hello, but local inference of hello should know the type from the literal and infer the return type of hello. Is that then able to be used external to the function by the call to .toUpperCase on (hello)

Example 2:

(defn hello
  [name]
  (str "hello " name)) 

(.toUpperCase (hello))

Would clojure.core functions, even though not type hinted, have their common types be known by Cursive magically (probably hard-coded somewhere)

Example 3:

(defn make-info
  [^String name]
  {:name name})

(.toUpperCase (:name (make-info)))

This one might be a tougher one, but basically can it infer the type of values on maps? Like would it know this is a Variant [^Keyword :name ^String name] and then know that the Keyword fn :name returns the value of key?

[colinfleming]

No, this inference currently only does what Clojure itself does. I could potentially do more, but it would only be in order to suggest to the user where they might want to add further type hints. Cases like the function return types are relatively easy to implement if I decide to go that far. The map value one is probably going beyond what would be worth it, though.

Would clojure.core functions, even though not type hinted, have their common types be known by Cursive magically (probably hard-coded somewhere)

Currently I’ve avoided magic hard-coding, but most core functions (e.g. str) are properly type hinted for interop purposes anyway.

[PEZ]

So awesome!

[bbatsov]

Just to clarify what I meant - in CIDER’s case, as we’re 100% REPL-powered (no static analysis at all), we can’t know the type of the receiver unless it’s a literal or we have evaluated it. As type method hints come solely from resolving the method names without any context (we don’t send the whole expressions to the backend, just the method symbol) we’re forced to do some guesswork. We may consider adding some context down the road, at least for the trivial cases, but evaluating receivers is dangerous and potentially slow, so that’s definitely one limitation of our approach, at least for the Java interop. Clearly that’s not an issue for Clojure code, but on the Java front Cursive’s approach is way superior, that’s undeniable.