Separating implementation from contracts

Is there a good way to separate implementation codes from the contracts?

Testing can be separated out, because they don’t have to live in the same namespace. Specs don’t really work with defmulti, they’re not appropriate anyway. Protocols are a great idea, unfortunately completely lacking in many needed qualities. One can separate namespaces, that requires the consumer to refer to the implementation package in addition to the contract file

Part of the problem is lisp is very flat. The clojure cheat sheet must segment the documentation with boxes rather than anything.intrinsic to the libraries. Most data structures are maps. One can make a public declaration of the expected shape, or specify it in documentation. Unfortunately, there’s not a logical place to make the documentation. The data structure is used across bunches of functions. Both private and public. There isn’t a single logical owner.

I know many lispers view the code as the best documentation. It’s actually the worst. Over time, implementations change. Either because systems change or bugs are fixed. A consumer relying on a particular implementation is slicing their throat. Or the throat of someone else on their team or some poor smuck left with the task of maintaining it. I learned my lesson in '81 or so. It’s true in Fortran, lisp, C., prolog. It’s true with every other development environment I’ve used since '85 as well. It also clutters up source files.lots of private functions and data structures live in the file. Bulking up the source, making monolithic files for version control to deal with.

There is a real need to be able to specify contracts without requiring the consumer to slog through lots of implementation. It would allow finer grained version control as well. Unfortunately (for me) I don’t have a clue how to pull it off.

I’m not sure which area you are focusing on.

If it is about external boundaries, basically IO, I find Spec is the best that Clojure offers. You could use schema or malli as well there. Almost nothing else will allow you to specify in as much detail and formally your IO contracts then these.

If you’re talking internally, or for libraries, what I’ve seen is either to use private to hide away the non interface functions and vars (that’s what I do), or have a convention where namespace suffixed with -impl are where the non contractual functions and vars live. You can add some spec as well to define the input/output structures of your functions and vars and possibly even throw in some fn specs for communicating some of their semantic properties. Though that can be a lot of work, for not as much value.

Lastly, I’ll say, the logical owner of data in Clojure is the system using it. So each function owns their input shape and value semantics, similarly for macros. And I said system, because your database owns its own input/out out shape and value semantics as well. Same for your file storage, etc.

This is very different from OO. In OO, objects store state, and they are shared across systems. That means the object itself owns the shape and value semantics, and it is everything that uses the object that must comform.

For example, in OO, you might have a User class. That class defines the shape and values of a User through field declarations, generally private. Now, it has methods, those all conform to the fields of the class. Now you have components which operate on User objects generally only through the public methods. Once again, those components must conform to the User API.

In Clojure (and this is my opinion), I believe it is different. You will not have a User. You will have functions, all of which might take a user argument. But, it isn’t because they share a named argument that they actually all expect to receive this singular global User structure. No, the name is simply indicative of the information domain this function was written for.

What this means is that the function itself defines the shape and value semantics of the user argument it takes as input. Not some global definition of User like in OOP.

So maybe you have 10 functions all taking a user, yet that’s a different user. Maybe one takes a map of :name and :password. Maybe the other takes a vector of [id preferences…]. Etc.

These functions never break, since they don’t depend on a global User definition. There’s nothing you can change outside of them that breaks them. Once written, they are forever useful. They do what they do, a function of input to output.

So the documentation for the user structure they take must be either on their doc-string, their destructuring arg, their spec, or their code.

How do you use these functions together then? Simple, by composing them and transforming their inputs/outputs along the way as see fit so that they can be composed.

In OOP, the methods are composed through a global convention that they all expect this global User structure to be exactly as they assume. You can think of it as the called method handles the composition for you. That’s why you can call the methods one after the other without needing to connect their ins and outs, in an imperative style.

In Clojure, the caller is responsible for the composition. And thus everything becomes a data pipeline with transformartions in between function calls.

Now, you can choose to share structure at first. For convenience. For example, you can have them all use the :app/user spec. But, this is where things get interesting and the motivation for Spec2 comes in.

For example, function-last is called last, and it expects the user structure it gets to have a session-id on it. But, it is function-login that creates the session-id and returns a user with a session-id on it. So what would be the definition of a global :app/user that they would both share?

You can’t make session-id mandatory, because prior to calling function-login it won’t be possible to have it. And making it optional is wrong, since function-last requires a session-id to do what it does.

And here’s the truth, User is an abstract concept. It is only indicative of a domain information. But the actual data associated with it, and the shape of that data changes over time and circumstances.

Even if you put function-login and function-last on the same User class, you create a user object with a null session-id, you call function-login it throws a null pointer exception. So now there’s all these implicit pre-requisite things that must happen to the user object before you can call function-last, and those are not defined anywhere… Or we’re back at what Clojure does, it can be mentioned on the doc-string.

In OO, someone will tell you, you need two class, and maybe one should inherit the other. A User class and a LoggedUser which extends User with an added session-id and the function-last method.

And now, the caller must know that there are actually two type of Users, each with different shapes and values, and different methods expect different Users. Keep going like that and maybe every method needs its own User class :stuck_out_tongue: And now you just have Clojure’s style again.

If you realize their is no global User. There is the domain concept of a User, and functions in the user namespace are related by this concept. But each function has different expectations of the concrete set of data they need to operate on such a user. Now you can design independent functions, test them, define their inputs/outputs in terms of the function only.

And then you can just have a workflow, maybe doX which gathers the data it needs to call function-login and structure it as it needs to be, then get the session-id returned by it and passes it to function-last.

This is what is sometimes meant by saying that Clojure favors a data flow style of programming.

In that style, you can mix and match independent functions as you see fit. And it isn’t driven by a noun, such as a User and its assumed form, but by a process (or workflow, or just flow or wtv you want to call it). It is thus the process definition that is more important. And each process have a total set of data they’ll need over their lifetime. But at different steps (or states since it can be thought of as a state machine as well), it might need to gather more data or create new data from existing data in order to find what the next step requires.

For now, these processes are defined in Clojure as code. Generally they have a top level function which puts together a particular process by composing a bunch of functions.

Spec2 is trying to make it so you can spec a schema, which is a view of all the data the process could require over its lifetime. And then each step could define a selection of the subset that they require. Which would be a nice way to define more formally the processes.

And I could go on, but I think that’s enough to let everyone ponder.

5 Likes

Data interfaces (data standards, data specifications) are better than code interfaces.

In industry, the product standard is the interface, the production method (code implementation) is not limited, input the raw materials (data) that conform to the standard, and output the products (data) that conform to the standard, that’s all.

Taking Clojure as an example, the req-map and resp-map specification of the ring are the product standard (interface), the ring is the warehouse, and the functions on both sides of the C / S are the production workshop (code), and the raw materials (hash-map) are transferred to each other through warehouse (ring) through interactive activities. .

Therefore, I recommend functions with single hash-map type parameters.This parameter can be mapped to standards, databases (with constraints, stored procedures, schemas, etc.), tables as needed.The Clojure’s immutable persistent data structure does not cause data cloning, which is suitable for this.
In addition, when using ->> to write a pipeline dataflow, you can omit the parentheses and make Clojure the language with the fewest parentheses. :grinning:

This is The Pure Function Pipeline Data Flow v3.0 with Warehouse/Workshop Model