DSLs in the Wild

The Fringe Other Languages
1

I feel that Domain Specific Languages have long been touted as a benefit of Lisp’s powerful macro system. While a Clojure library like Overtone arguably has a few domain specific macros for creating synths, I wouldn’t call it a domain specific language. Curious - is there a full-fledged DSL built in Clojure?

I have met two folks in the last couple years that have built their businesses around DSLs. They admit that it can be difficult to sell to corporate decision makers, but their businesses persevere and they are both busy.

Neither work in Lisp or Clojure. I don’t find this particularly surprising, but I am wondering

a) is building a DSL in Clojure as uncommon as I think it is?
b) does anyone believe that interest in DSLs are growing - either for toolmakers (such as make) or for businesses?

1 Like
2

In Clojure we try to represent DSLs using built-in data structures for the same reason we represent domain concepts in built-in data structures, rather than macros.

Data > function > macro.

You could call hiccup a DSL, but we can still use all of our tools to manipulate it. The Perlis quote also comes to mind.

5 Likes
3

Ah this is a good point. If XML is a DSL, which I have seen argued, then hiccup and edn certainly qualify.

These are as Lispy as any macro system because they take advantage of the homoiconic nature of the language.

4

I think your definition of DSL is too narrow. So many things are DSLs in Clojure.

Like you said, Hiccup, Garden, EDN are all DSLs.

But even things like Spec is a DSL. Even within it things like the s/keys macro is a DSL for defining a spec about a map with definition of what keys it should have.

Even in Clojure core, you have many functions or macros whose input are DSLs. Think about for? That’s a DSL for mapping over a sequence.

You could literally argue every macro is a DSL, since they provide custom semantics to specific restricted non Turing complete behavior.

And many functions could also count as DSL. For example, functions that take optional key value args.

The customizability of a lot of Clojure core functions is because their inputs are mini-dsl into defining the behavior of the function. Anytime a function does something different based on the interpretation of the passed in arguments, you can consider it a DSL.

So like other notable DSLs, you have ns, you have require, you have def, defn, gen-class, try, reify, update-in, etc.

1 Like
5

As a little bit of a an anecdote by me when I answered this question for myself in 2016:

In 2010 I built a tool where, using my self-developed DSL, I could specify and generate ‘crud webapps’, with hooks/plugin model to further modify the generated code.

Built it with EMF framework from Eclipse, doesn’t really matter.

But, fast forward 6 years, when I already had abandoned this code generator, I tried to, as an exercise build the thing in clojure (just proof of concept for myself).

And I started with macro’s etc, then realized: this should all just be EDN. So I ended up with a sort of ‘hiccup’ but then with more high level constructs.

Writing the code generator wasn’t hard at all too (I just went the recursive descent parser / interpreter route really leaning on multi-methods and destructuring).

So, for me that answered your question why in clojure we don’t build DSL’s in the traditional sense, with custom grammars etc, but just use EDN there.

It’s really great. So ‘for free’, in Hiccup, you can use the complete clojure programming language to manipulate it.

1 Like
6

This must be a powerful perspective for language designers. At least that’s the context I’ve seen internal DSLs discussed in the wild. I wonder if this is also a useful delineation for teaching a language?

Overtone is a middle-of-the-road example: the library provides a language for making music within Clojure, but it doesn’t have its own interpreter and editor like Supercollider. Supercollider certainly has all the features to fully qualify as an external DSL.

Thanks for sharing. It’s a clarifying anecdote. There is a nugget in here that I’m starting to see. Via Lambda the Ultimate:

External DSLs on the other hand are like puppies, they all start out cute and happy, but without exception turn into vicious beasts as the grow up (make, XSLT, regular expressions, …).

I’ve always been skeptical of DSLs (and to be more precise, I am talking about external DSLs) because it seems that a well-designed DSL requires an immense amount of domain knowledge. The edge cases of any reasonably complex domain are just too sharp.

While data modeling also has its pitfalls, far fewer decisions are being made early on if all of the processing is happening with the generic functions of a well-known language. I also think this makes it easier to teach someone about the project.

Another example of how the Clojure philosophy seems to be a good starting point when working on a domain problem:

2 Likes
7

Reading the replies in this thread, I’m reminded of Denotational design. In fact, I’m having a hard time differentiating “a good denotational design” from “a well-designed internal DSL”.

Conal Elliot on denotational design, with examples in Haskell:

Or @ericnormand, who argues that Denotational Design paves the way to “algebraic thinking”.

Very interesting question. Thanks for bringing this up, @schmudde!

1 Like
8

I think something missing is that denotational semantics try to convey meaning to the computer, while a DSL in my opinion often tries to convey meaning to the programmer.

You might want to find a way to map your programming language, which is designed for humans, into a formal denotation most likely inspired by an existing mathematical formalism (because inventing a new one is super hard), and once you’ve done that, you can then more easily write programs that can understand the semantics and validate them, modify them, translate them, etc.

And then, you get into a place where… a place I think Haskellers often find themselves in, where you start to wonder, why teach the computer my semantics, when I could instead learn the computer’s semantics? And that is how programming started, when we didn’t have higher level programming languages, you just learned the operational semantics of a Turing machine and described your program directly in it, assembly instructions one after another. Until someone made a “higher level” programming language where “higher” really means closer to human language and thought.

Now with denotational semantics, I often see this happening, hey, why not just learn some existing mathematical semantics theory, like say category theory, or set theory, or hoare logic, etc. And why not write our programs in those languages instead? Since they are better understood by the computer? It does make sense, sure, but now you’ve again shifted the burden to the human to learn some other language that is “lower level”, where “lower” means further remote to the natural way our language and thoughts work.

And maybe that’s what a programmer needs to do in order to write correct programs that they can have the computer proove its behavior, etc. Or programs that the computer can split up and re-combine in other equivalent orders for performance, compatibility, flexibility, etc.

But, there’s another approach, that which I think is more what DSLs are about. To target a specific targeted domain, that which is smaller in scope than all Turing computations, and to create a language which appeal more to the human. That is of a higher level in that sense. Easier for a human to understand. Now, yes, you might not be able to have your computer statically prove programs of such language, or automatically parallelize them, or all kind of other such neat tricks. But maybe you’d have made it easier for the programmer to quickly write a program that does what they want, and you’d have lowered their chances of making mistakes in semantics because the burden to translate them was eased as well.

That said, I think there is a middle ground here, as a programmer, it does help to learn of other models of thoughts, because now it broadens your tools for how to think about problems. Language can do that. So it might be that learning set theory, and then translating your problem to it, actually helps you better understand the problem and find solutions to it. So denotational semantics can help in that way, where it gives you frameworks of thoughts and language. I just think we shouldn’t forget that we are humans and our thoughts are not that straightforward, so I think DSLs also very much have their place.

Just think about for versus map. Seems most human begin with for and eventually transition to map. for being a DSL, a higher level language of map that is closer to our thoughts about it. Until we learn the modeling semantics of map and then suddenly no longer feel like map is any more obscure than for.

3 Likes
9

Agree with other answers.

My understanding is that when Hickey decided against having reader macros in Clojure he was explicitly rejecting Lisp’s tradition of DSLs in the way, say, Racket thinks of them, as custom syntaxes.

(Although he cared enough about data to beef up Clojure’s data DSL in the form of EDN.)

Instead the Clojure way is do the DSL in EDN data (so as not to have to worry about custom parsers etc.) Because EDN is pretty expressive anyway, and completely interoperable with Clojure code and macros, that seems to work out in Clojure world.

And it’s very different from using XML as a DSL, when XML is horrid, verbose and hostile to humans. EDN is basically as readable as Lisp.

4 Likes
10

Thanks for the thorough reply, @didibus! Would you agree with the following?

Domain specific launguages are aimed up towards the user, to a higher level of abstraction. Denotational design is a means of organizing stuff that needs doing, interspersed with generic pieces of logic.

1 Like
11

Ya I’d agree with that. Is that a quote from somewhere?

Denotational design is great for composition, because it’s kind of what it’s all about. Rules of decomposition and composition over atoms. Or as algebra says, the study of symbols and the rules for manipulating them.

DSLs are kind of terrible at composition, they rarely compose between themselves. And not all DSL are done in a way that their own parts can be composed easily either.

But I think that’s kind of inherent to their essence. Like denotational semantics are all about generality, finding similarities between things, and all that. While DSLs are domain specific, they don’t pretend to try to be general in any way, they are all about specificity.

But, one could translate their DSL into some underlying denotonational representation and allow it to compose with others. I think Racket does that for its DSLs, where different syntax can coexist as long as you can rewrite them in Racket.

12

Just my attempts at synthesis yesterday!

1 Like
13

While I agree that a lot of „the domain“ is encoded in EDN, I think there’s another layer to this discussion. Lisps are already pretty much a DSL for DSLs (not arguing that that’s Lisps primary purpose), making it all the harder to talk about. Is core.async a DSL? Is honeysql? I‘d call reitit, bidi and compojure DSLs, as I would many of the lower level libraries like integrant or mount. The „D“ varies of course, but in a language like Clojure I feel it’s sort of hard to write non-DSL code at all. There’s not much else to do in terms of boilerplate or glue code, which by definition are not domain-specific and rarely tend to incur their own language-like semantics.

14

As others have noted, using data to create a DSL is generally seen as the preferred approach. Data is easier to reason about than a macro, you can serialize it, send it across the wire, etc. A good example of how data DSLs compare to a macro based DSLs would be to look at Compojure and Reitit or Malli and Spec.

There are cases where macros can be preferable to data. Generally, I find macros make sense in a situation where you’re extending the language semantics the way Overtone is doing. A couple of other notable DSLs would be core.async, [Typed Clojure], Specter, and Clara Rules.

1 Like
15

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.