Clojure Microservices

[duzunov]

Hi all, I was wondering what is the opinion of the community on micro services in something like Kubernetes? I know of Datomic Ions which something really interesting, but it is not open-source and tightly coupled with AWS which unfortunately limits the market a lot.

Is there a preferred approach, libraries or a framework to interop with?

Also, are micro services even as desirable in Clojure as in imperative languages without a concurrency strategy? My impression of most micro services I’ve seen is that they just remotely executed pure functions that benefit from an orchestrator for concurrency and a lot of the organisational benefits come from what you will get built in Clojure, without complicating the deployment story. Do micro services + k8s partially even out the field for languages like JavaScript and Python?

[simongray]

The way I see it, microservices are mostly a solution to an organisational problem. Software gets harder to build the bigger the group building it becomes (see: The Mythical Man Month). Microservices encapsulate a specific portion of responsibility to one piece of code which can be managed independently by a team and integrated with code from other teams by adhering to an API contract. They help formalise organisational cooperation, but other than that they are pretty much a step down from a regular integrated codebase (which for some reason is called a monolith) in every other way.

Aside from organisation size and structure, the only other reason to go with a microservice architecture by default would be if you have some gigantic scaling needs from the very beginning.

[vvvvalvalval]

I agree with what @simongray said - microservices are a response to an organisational problem, not a technical one. If you’re not sure why you’d want to do microservices, don’t do microservices.

You should consider microservices when your human coordination problems become so bad that it might become interesting to trade them for distributed systems problems (which are hard, and will add significant technical hurdles). As such, the choice of microservices is mostly language-agnostic.

That said, Clojure and it’s ecosystem encourage several things that make it easier to move towards a service-oriented architecture, by being a “Language of the System”: programming with immutable values and generic data structures that are easily transmitted over the wire, using queues, separating program components via small interfaces, etc.

OTOH, there is another downside of micro-services that is specific to Clojure: once your program is distributed over several machines that do different things, the reach of your REPL becomes significantly reduced.

For the sake of completeness, I should mention that a Clojure team has developed something called Polylith, which is supposed to give you the best of both worlds. I’m not particularly encouraging you to use it; having looked over it, I’m pretty sure it has more drawbacks than advertised, so be careful.

I can’t answer for k8s unfortunately.

[didibus]

I wouldn’t call Ions a micro-service framework. It is more in-line with some form of serverless design.

Don’t be fooled by all the internet fog around micro-services. A micro-service is just a service which doesn’t fully deliver your user requirement, thus needing to be combined with and make use of other services to do so.

Let’s take clojureverse.com for example. If the frontend communicates with a single web server whose requests are handled on the same machine as the web server, and where to handle each request the only external call to this process that needs to be made is to a database (possibly running on the same or a seperate machine), that is a monolithic application. The entire set of user requirements for users of clojureverse.com is handled by one service implemented on top of some web server and some database.

Now if some of the requests the frontend sends to the web server when handled by it make a call to some other service running in another process (possibly on some other machine but could be same machine) well it would be a micro-service. I say make a “call”, but it could be any method of messaging, technically the more “loosely coupled” the more inline with micro-service idioms.

That’s really all there is to it. So any framework/lib that lets you build a service will do just fine. So you could use ring and HTTP APIs, so ring can serve APIs over HTTP as the messaging method. You could also just run a Clojure application and communicate through a message broker like AWS SQS, Kafka or ActiveMQ.

Now how you package, deploy and run these is up to you, dockerise them and deploy them on k8 if you want.

[duzunov]

Some great replies so far… I understand there is justified and healthy scepticism for things on the top of the hype curve.

[GKlijs]

I’ll experimented a bit with event sourcing, which is not necessarily microservices. But it does tend to break things up in tiny pieces which could be run together or seperated without much difference.

I don’t think microservices is just a hype. I’ve seen it used successfully, by grouping together functionality, it’s easier to be able to change/add things. But at the same time almost always something ‘leaks’ to other parts which need to be closely coordinated.

[didibus]

There’s some good reasons to break up a system into one or more smaller services.

Wikipedia does a pretty good job here:

Modularity: This makes the application easier to understand, develop, test, and become more resilient to architecture erosion. This benefit is often argued in comparison to the complexity of monolithic architectures.

What this is saying is that in a monolithic system, the boundaries in the application architecture are easy to break and leak, which can cause monolithic systems that start with clean seperation of concerns into well designed modules layered or composed together can over time erode where those seperation are slowly broken over time and everything becomes one big ball of mud. I’d say Clojure is less prone to this, since its primitives tend to be simpler: pure functions and immutable data, but it can still happen.

Scalability: Since microservices are implemented and deployed independently of each other, i.e. they run within independent processes, they can be monitored and scaled independently.

Think that’s self-explanatory.

Integration of heterogeneous and legacy systems: microservices is considered as a viable mean for modernizing existing monolithic software application. There are experience reports of several companies who have successfully replaced (parts of) their existing software by microservices, or are in the process of doing so. The process for Software modernization of legacy applications is done using an incremental approach.

Sometimes it can be easier to refactor a big system by breaking it piece by piece into smaller services.

Distributed development: it parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently. It also allows the architecture of an individual service to emerge through continuous refactoring. Microservice-based architectures facilitate continuous integration, continuous delivery and deployment.

This is the most important one in my opinion, in that with care you can build a monolithic system that doesn’t have its architecture erode, you can manage to measure and scale appropriately, and you can successfully refactor a legacy one without breaking it down into seperate services, but with a monolithic system you really can’t achieve distributed development, especially with regards to distributed independent deployments/delivery.

All from: Microservices - Wikipedia