DSL and passing immutable data

Hi,

I’m a Clojure beginner and trying to learn by implementing a DSL that I have worked on in another language. Part of my motivation for using Clojure is to compare the different approaches of an internal DSL (using Clojure) vs an external DSL (written using tools like Xtext or JetBrains MPS or even via a hand written compiler). Clojure in particular appeals to me because of its emphasis on using generic immutable data structures, and the availabilty of macros to help extend syntax. Plus, Rich Hickey seems pretty smart and persuasive !

So, I’ve had a go at implementing it and the main thing that I’m struggling with is (invisibly) threading the state through all the statements of my DSL. I’ve got some things working, but there are still some issues - but before I continue down this path I’d like to get some opinion on whether I’m going about this the right way or whether I’ve missed the idiomatic Clojure way of doing things.

Any advice would be greatly appreciated :slight_smile:

I’ll try to briefly describe the DSL I am trying to implement, and how I’m writing it so far.

The basic idea is that the DSL allows rules to be written to process and validate data, eg customer and order information. For example :-

(rule ApplyDiscountIfGoldCustomer
(iff (= (customer :STATUS) :GOLD)
(set-value (order) :TOTAL (* (order :TOTAL) 0.95)))
(log-message "order total amended to " (order :TOTAL))
)

The infrastructure of the system would step through all the data records and apply all the rules - in this particular case, apply the rule to each order record that it comes across.

For users of the DSL it’s implicit that things like (order) and (customer) refer to the current order that is being processed, and the customer who made that order. They shouldn’t have to worry about passing the current data around as arguments to functions - it should be invisible to them.

The DSL for the example above could generate Clojure code something like :-

(defn ApplyDiscountIfGoldCustomer
[data]
(as-> data x
(x-iff x (= ((customer x) :STATUS) :GOLD)
(x-set-value x (order x) :TOTAL (* ((order x) :TOTAL) 0.95)))
(x-log-message x "order total amended to " ((order x) :TOTAL)))
)

So to make this happen I’m taking the following approach :-

  • rules need to take in and return the current state of the data - which actually needs to be all the data records, not just the current order. This is because there could be a rule which for example wants to find the total number of orders for all customers. The updated data can then be passed to the next rule.

  • each DSL statement also needs to take in and return the current state of the data - so that as one rule returns an updated state of the data, the next statement has access to that.

  • rule needs to be a macro which generates code that defines a function and adds in the data parameter. The data can be threaded through all the statements in the rule, either via a threading macro or by let statements.

  • the parameters to DSL statements may also need a data parameter, eg the order function in the log-message statement. I’ve experimented with making each DSL statement a macro which adds in the data parameter to each function that “needs” it. At first I was just defining “needs” as any function with no parameters, eg (order), but this doesn’t work for things like (order :TOTAL). I was looking into whether I could use metadata to store some information to determine whether I need to add the data parameter, but it seemed difficult to retrieve this information. Perhaps the rule macro could take care of adding the data parameter to statements, instead of the statement macros themselves.

  • I’m currently defining the data using a vector of maps. This lets me loop through the vector, keeping track of where we are up to via a current record index. Not sure whether a map of maps might be better, because I do need to do some searching for records based on a key. I typically wouldn’t have more than 10,000 records at a time.

I’ve written the basic infrastructure to loop through all the data records and apply all the rules (using explicit recursion, rather than map, as sometimes I need to apply a rule repeatedly, or change the current record to a previous record etc) but am still having various issues, eg properly adding in the data parameter to each parameter in a DSL statement (something wrong with my recursion, I ended up defining functions that were too big). Also smaller issues like my IDEs (both Cursive and Calva) complaining about rule names not being defined.

But for now, I really just want to get an understanding on whether my approach is at all reasonable, or whether I should be taking a completely different approach. Sorry if I’ve rambled on for too long, I hope I’ve clearly outlined the problem and approach.

Once again, any feedback greatly appreciated, thanks very much !