On Polylith (from Amazing Ideas thread)

Continuing the discussion from Amazing Ideas that blew your mind:

I’ve spent quite a bit of time chatting with @tengstrand over the last few days and exploring Polylith with a lot of feedback in both directions. There’s a lot to take in about Polylith and I think some of the communication done about it so far has caused some folks – me included – to focus on parts of it that are not really core to the ideas behind it.

Joakim has already updated the Polylith website to clarify the core problems Polylith is trying to address and expanded the Polylith FAQ to answer several questions that cropped up on Slack and other places recently.

We’ve talked quite a bit about the naming and structure of Polylith projects and he’s made it clear to me that a larger degree of flexibility is allowed than the documentation currently suggests, and it’s possible that Polylith will soon explicitly allow some variations in naming and structure.

After addressing that, I voiced the opinion that Polylith’s tooling seemed to be a very powerful and a really important part of the whole package: the poly info command provides a lot of information about the workspace structure and poly test provides incremental testing because it understands the relationship between the various parts of a Polylith project. Joakim said the team had worked with the architecture for some years before building the tooling. It’s impressive tooling but, clearly, again not core to the ideas behind it.

There were a lot of small nuances about Polylith that I didn’t “get” at first but having talked with Joakim and also experimented with Polylith for a few days, I’m appreciating the concepts more:

  • bases and components don’t need to specify their interdependence in their deps.edn files, only their external dependencies – it’s in the projects that the actual combination of bases and components comes together (this is key to the Lego-like aspect of Polylith)
  • projects are deliberately separated from both the bases and the root deps.edn file (used for development) – so you can both separate production from development (I got that bit) and so that you can build different deployable artifacts from similar collections of bases and components
  • the “strict” adherence to “interfaces” in components is what allows for the mix’n’match approach that lets you build those different deployable artifacts

I still feel the emphasis on functional delegation in several places is a bit of “boilerplate” but at least now I understand why that delegation is there and what it enables.

So I’ve decided to add a built-in polylith template to clj-new so that you can quickly get a minimal working Polylith project running locally:

$ clojure -Sdeps '{:deps {seancorfield/clj-new 
                          {:git/url "https://github.com/seancorfield/clj-new" 
                           :sha "d38b731d4d52ab57647c4462576197997c729491"}}}' \
          -X clj-new/create :template polylith :name myname/myproject
$ cd myproject
$ clojure -M:poly info
  ... information about the workspace ...
$ clojure -M:poly test :all :dev
  ... run all the tests ...
$ (cd projects/myproject && clojure -X:uberjar)
  ... builds an application ...
$ java -jar projects/myproject/myproject.jar Lisa
Hello, Lisa!
$ (cd projects/myproject-lib && clojure -X:jar)
  ... builds a library ...
$ (cd projects/myproject-lib && clojure -X:deploy)
  ... deploys that new library to Clojars ...
$ clj -A:dev:test
  ... starts a REPL for development and testing against the workspace ...
Clojure 1.10.3
user=> (require '[myname.myproject.greeter.interface :as greeter])
nil
user=> (greeter/greeting {:person "ClojureVerse"})
"Hello, ClojureVerse!"

And, just like the poly create workspace command, this also sets up git and commits the initial version of the workspace so that Polylith can track changes etc. See the git section in the Polylith README.

This is a work-in-progress, tracking changes being made to Polylith on its issue-66 branch, but it will be part of the next clj-new release once the Polylith team has completed that work.

12 Likes

Yes, when the weekend was over, we finally managed to understand each other :slightly_smiling_face:

I also want to add this:

The default naming convention for the interface namespaces will still be “interface”, e.g. com.mycompany.invoicing.interface for the invoicing interface, but we will also support giving the interface namespace the same name as the interface as you suggested: com.mycompany.invoicing.invoicing.

We will also be more clear in the documentation that the API of a base can be given any name (not only api which we use in the videos, or core which is the default used by the poly tool) and that the implementing namespace for a component can have any name (not just core).

We will not support any variations in the directory structure, that will stay the same as today.

I think we worked for about a year without any tooling, before we created the first Leiningen based version. Then we used that for about two years, and then Furkan created a bare minimum version of a tools.deps based version (around 500 lines of Clojure code) that he used for a year. After that I “took over” and created the version we have today (> 5000 loc + tests).

I can see that you get the core ideas now, and that’s awesome!
It’s also cool that you are testing Polylith and contribute to it, so thanks for that!

/Joakim

5 Likes

Ah, that wasn’t quite what I’d taken away from that conversation but, yes, more options are good :slight_smile:

I’m migrating my User Manager example app to Polylith by following the Transitioning to Polylith guide – see the polylith branch in my repo – as a way to get more familiarity and I’m currently using api instead of interface and I think I like that better, at least for now.

Being able to control it via a setting in workspace.edn is nice flexibility since even if every Polylith project you encounter uses a different name, that file – and the poly tool – means you can quickly get your bearings. I’ll probably make that a “variable” in clj-new so you can choose its value when you create a new project, and maybe other settings as I get more of a feel for that :slight_smile:

The big reason why interface would be preferable is that this is what we call that concept everywhere, e.g. in the doc and when you run the info command. We use API when we talk about base’s public API.

A correction. We will support the namespace pattern that you suggested, e.g. com.mycompany.invoicing for the interface and com.mycompany.invoicing.interface.subns for interface sub namespaces. I thought for a while that we couldn’t support it, that’s why my previous answer said something else. Sorry for that.

1 Like

bases and components don’t need to specify their interdependence in their deps.edn files, only their external dependencies

I got a different impression from the docs (libraries section) and example apps.

Specifically it seems to say that external deps (aka libraries) are only specified at a project level, whether in the top-level deps.edn (the development project), or one of the runtime projects inside the projects folder.

The benefits might be that it explicitly lets you configure it consistently once at a top level (for dev), at a cost of duplicating it for your runtime projects and figuring out specifically what each component depends on.

Am I misunderstanding?

I’m working with the issue-66 branch: Polylith - Libraries:

Libraries are specified in deps.edn in each brick and project:

  • Bases: bases/BASE-DIR > deps.edn > :deps
  • Components: components/COMPONENT-DIR > deps.edn > :deps
  • The development project: ./deps.edn > :aliases > :dev > :extra-deps
  • Other projects: projects/PROJECT-DIR > deps.edn > :deps

My usermanager example web app (Component, Ring, Compojure, Selmer, next.jdbc) now has a complete Polylith version: seancorfield/usermanager-example at polylith (github.com).

1 Like

Also worth updating the note about libraries on the issue-66 branch as a new enhancement just dropped this evening (which was prompted by my usermanager example):

Libraries are specified in deps.edn in each component, base, and project:

Entity Type Location
Components src components/COMPONENT-DIR > deps.edn > :deps
test components/COMPONENT-DIR > deps.edn > :aliases > :test > :extra-deps
Bases src bases/BASE-DIR > deps.edn > :deps
test bases/BASE-DIR > deps.edn > :aliases > :test > :extra-deps
Dev project src ./deps.edn > :aliases > :dev > :extra-deps
test ./deps.edn > :aliases > :test > :extra-deps
Other projects src projects/PROJECT-DIR > deps.edn > :deps
test projects/PROJECT-DIR > deps.edn > :aliases > :test > :extra-deps

The tool parses each deps.edn file and looks for library dependencies, which are then used by the libs and test commands.

1 Like

The Polylith team has decided to not support having the interfaces in e.g. com.mycompany.invoicing.clj but only as today in com.mycompany.invoicing.interface.clj.

There are several reasons for this:

  • There is a value in that all Polylith codebases look the same.
  • It will add a big burden to the maintenance cost of the Polylith codebase. Many more tests and more complexity added.
  • The number of examples will increase significantly, and it’s simpler to say “this is how it works” instead of letting the users choose which style to use.
  • It may also be a matter of taste what style you prefer, and some users now prefer the current style. It has some advantages and it is a bit simpler to have all the .clj files at the same level.

I have also talked with @seancorfield the other day and he is fine with this decision and said that he and his team is now used with the new style.

/Joakim

4 Likes

I just want to give a big hat tip to @tengstrand and the Polylith team. I haven’t yet adopted Polylith in production, but even if you don’t, the thought that they’ve put into Polylith (and the excellent project docs) can be really eye-opening.

Polylith motivated me to dig in and learn how Clojure projects (and classpaths) are built, and dispelled a lot of the magic of the build system for me. It showed me what was possible and gave me the inspiration to rework my build in exactly the way I wanted. I definitely owe you all a beer or two if we cross paths. :beers:

4 Likes

Thanks for the kind words!
I wouldn’t say no to a beer or two :wink:

3 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.