File-Oriented Programming - [Are We There Yet?]

Sometimes I like to cook. And I never needed a separate container to store each item/piece/slice of food. I just have it.
On other hand I sense “a files obsession” in our current state [of software development].

Codeq has a good example of this mismatch http://blog.datomic.com/2012/10/codeq.html

Is a tree chunk representation (aka filesystem, folders) a good way to help with your current problem/project/solution?

Are there any [real] reasons to use files (instead of database) to retrieve and store our code (aka data)?

UPDATE about Codeq from Rich:

4 Likes

One of the oldest programming languages, SmallTalk, was developed independent of the “file-oriented” model of programming, relying instead on an “image” that held both the source and the state of the running program. Granted, this is a bit more than you’re asking for (what would a SmallTalk image that holds source, but not state, be like?), but I think we can still learn a valuable lesson from SmallTalk’s experience.

That is, in general the “Unix philosophy” states that we should “treat everything as a file”. While we can argue about whether or not this is the best paradigm, and about how far we should take it (Plan 9 took it way further than even Unix or Linux ever did), the bottom line is that the path of least resistance on any *nix system is going to be working with files.

That said, the Internet doesn’t have the same philosophy baked in. (One could argue whether it has any philosophy, but that’s another debate.) So, if you were going to try and transcend the file-based workflows of old, I suspect you might have the most success doing so in the Cloud.

1 Like

Git works on files really well, don’t see it, and thus files going anywhere anytime soon.

That said, I believe that the way we organize files containing functions, doc strings, specs, tests, etc could be split up, and then combined into a better working experience than the current text editors we have.

For instance doc-strings in the same file as code could be better. At present in the same source file leads to readability/formatting issues. Why not have a function per file. Then clojure doc-strings could be located in a markdown formatted docs file side by side with the function file. Whatever UI/IDE is in use could display the docs when and how you want to look at it.

Same could be done for specs, code examples, resource files, etc…

Your description is similar to IBM’s Visual Age (Eclipse’s predecessor), It has the disadvantages that it is difficult to expand and the underlying technology is weak.

And, indeed, Clojure’s real unit of organization is the namespace, thus it is relatively easy to build a Smalltalk-like browser:

6 Likes

I’ve been thinking about this for a bit, and it extends on my previous idea of decomplecting human names with machine reference ids.

TL;DR: An IDE which works directly with Clojure semantics, and saves top level constructs into a database, any change to them creates a new version, meaning they are all immutable. You would need to specify which version you want on a per-use basis, or maybe per-namespace basis. When you actually wanted to run the code, it would create a Clojure source file projection so that you can bundle it and package it. You wouldn’t need git, since the IDE itself has all code versioned and immutable at the level of vars, macros, functions, and namespaces.

One could create the following environment:

  1. An IDE which would allow you to create namespaces, gobal functions and vars, not with text, but as part of the UX.
  2. You could move global vars between namespaces, maybe by drag an drop.
  3. Under the hood, the IDE would give the var a uuid, and the namespace would also be a uuid. It would add to the doc-string of each the name you gave it, using some convention. Could also be meta, but doc would work.
  4. At this point, it means that every namespace and global var you create have a unique reference-id of namespace-uuid/var-uuid.
  5. The IDE would not let you type text. The namespace and functions vars or value vars are UX construct, always balanced. You can add arguments, add doc, add pre/post, but the UX maintains consistent syntax.
  6. The implementation of the functions or the value to go in a var would be free-form text, with paredit or pareinfer.
  7. But, THE MOST IMPORTANT PART, each time you save, the namespaces, functions, value vars, and macros get stored in a database.
  8. You can freely change the doc-string, doc-name and meta, and it gets updated everywhere, but try to change the implementation, arguments, or pre/post conditions and when you save, it creates a new version of that construct.
  9. When you use one of these, you’d need to specify which version you want.
  10. I’m not sure namespaces are useful in that sense, but maybe it could ease the burden of having to define per-function dependency versions.
  11. Once you’re done, to run the code, it would create a source file projection, including only the constructs that are used.
  12. There could be a global database, so libraries wouldn’t really make sense anymore. No need for packages. Just top level constructs and their specific dependencies.

My only hesitation is that it greatly reduces the tools one can use, and its possible that dependency management would become a huge burden. For example, what if a function gets changed to fix a bug, woudn’t you want to automatically get the bug fix?

Maybe we could mix this with spec. You could attach a spec to a function and it could run generative testing on it after the change is saved. If it still passes the spec, reference to it would get auto-promoted to the new version. If it fails the spec, it would be flagged, and reference to it would continue to point to the old version.

Anyways, just some of my ideas for now.

2 Likes

For example, what if a function gets changed to fix a bug, woudn’t you want to automatically get the bug fix?

Not at all. One of the big advantages to the sort of system you’re talking about here is that the dependency graph can support multiple versions of the same function at the same time, so if you have two deps that use two different versions of some underlying library they won’t fight.

@mhuebert and I have talked a great deal about this sort of architecture, but it does require some very special tooling to be practical.

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.