Amazing Ideas that blew your mind

What Ideas did you encounter lately which changed your view about programming or were just so fresh you wondered why not everyone was talking about them?

I am reading a lot on clojureverse lately; here and there I stumble upon some ideas in the clojure-world that simple shake my current view on programming.
Basic Clojure stuff like homoiconicity, immutability and data-orientation. Or that you can simply call a map (coming from a different programming language, I find this to be just extremely elegant), or the concept of transducers :smiley:

Databases: Datomic (or Datahike) Wait, a Database can be immutable?!

I found myself reading a lot about interesting ideas in the web-development:
Datascript Wait, a Database on the Client might make sense?!
Pathom I can leverage the power of graphs to save me from a lot of function calls
Fulcro I can co-locate my components with their data needs and use a “client db” and pathom to do the heavy lifting for me

So what ideas did you came across? I am curious :slight_smile:

5 Likes

Recently, I finally understood the value of rules engines. It was the the videos on O’Doyle Rules and - a couple of weeks after that - minikusari that I found to be great introductions to the practicality of a rules engine.

Prior to that, I only really knew about Clara Rules and I found its API too boilerplaty and the code quite hard to read. I think it’s a lot easier to understand the value when the code has mostly been stripped down to Just Data™.

5 Likes

I was recently watching Stuart Halloway’s talk “Debugging with the Scientific Method” and at the end he is talking about the method of using git bisect to find bugs, and has a call to action for building upon that method for automatic cause detection. I’d love to know if anyone has been building on those ideas, my last position gave me my first real taste of the power of generative testing and I’ve been starting to dive deeper into the material and possibilities that Clojure provides for building these systems.

edit: link to the talk queued to this part of the discussion https://youtu.be/FihU5JxmnBg?t=2731

2 Likes

There’s a lot of cool ideas I found from Clojure itself, like some of the ones you mentioned already, that all blew my mind, macros, transducers, core.async, refs, EDN, core.logic, dynamic bindings, repl connected editor, etc.

Some other cool things are probabilistic programming languages, array based languages, stack based languages, and machine learning.

Condition restart systems are neat as well. Actor model, structured concurrency, event driven programming, monads, dependent and linear types, etc.

All these things blew my mind at some point or another.

1 Like

Hash Array Mapped Tries

There exist “fast” persistent arrays, sets, and maps, fast enough that you can use them for a much larger subset of programming than previously thought. Prior to HAMTs and friends, the choice between immutable FP and mutable imperative/OOP/procedural was very well delineated. ex. You go with immutable maps, you’re working with binary trees and living with that performance envelope, or singly linked lists, or copy-on-write structures.

The design of HAMT itself (an extension on Phil Bagwell’s earlier work with some novel optimizations) [and later RRB-Vectors] really blew my mind. We take it for granted since it’s baked into the language, but that data structure substantially blurred the performance/efficiency line that existed for decades prior. There are still plenty of spaces where the gains from mutability are worth the trade off in correctness and persistence, but I’m surprised to see how far I can get with just the built-in HAMT-based stuff. They also enable value semantics for collections, which in turn enables efficient comparisons (for the not= case) and enables the immutable thread-safe reference types.

Stepping Debuggers Are Optional, Maybe Even Distracting

I went years without touching a stepping debugger after having crutched on them in prior languages. Impetus toward lack of mutation, a focus on immutable values, and a nice repl meant inspecting about pieces of the program and function dependencies was faster than stepping through a program to watch some boxes. As the Cider debugger came online, I found myself using it maybe 3x in 5 years, and I was happy to have it, but it wasn’t necessary per se. I contrast this with the seeming Rube Goldberg Machines other languages encourage.

All You “Really” Need is a REPL

Everything else (all the nice tooling and fancy IDEs, etc) are really just window dressing. If all you have is a REPL, you have access to the entire language, the ability to inspect everything, the ability to redefine stuff live, compile things, extend the language, etc. I came up in the early days when the tooling wasn’t so great, so I ended up living in the repl primarily, and using it for things like documentation, searching namespaces, etc. I still do that these days, although the tools are very nice.
This isn’t a feature unique to clojure (arguably other lisps like Common Lisp and non-lisps like SmallTalk have even better introspective/live coding capability), but it was pretty mind-expanding when I encountered it.

clojure.core.async/go

The go macro flexes multiple technologies in Clojure to simultaneously enable a new programming paradigm in 2 (maybe 3 if they port to CLR) language targets. It’s a showcase in metaprogramming where the macro expansion is actually leveraging a tools.analyzer implementation with multiple custom passes (written in clojure), to transform the input into something with parking i/o. They basically lifted Go’s communicating sequential processes implementation into a macro as a library, and implemented a little compiler along the way. It reminds me of some of the stuff Paul Graham talks about in On Lisp, where he describes programs that spend the majority of their time in macroexpansion just doing ad hoc compilation without ever leaving the host lisp.

Transducers (preceded by reducers)

I think Guy Steele’s talk gets the majority credit, which formed the basis for reducers and then the later generalization into transducers. Very cool idea, relatively simple to implement in hindsight.

Will Byrd Implements Eval in miniKanren and Uses it for Program Snythesis

magic. Let’s find all the programs that could eval to our input expression…

14 Likes

The Unison language for sure, its been a long time since I’ve come accross something so novel and organic-feeling before. https://www.unisonweb.org/

2 Likes

I have been programming for ~15 years, but I have never really used a heap. A heap is a data structure that basically lets you insert things in log(N) time and remove the smallest/largest item from it in log(N) time. I could have parrot quoted that to you at any time in the past, and yet I never really saw how to use them in reality.

It finally clicked with me that I should use a heap when I need to maintain a dynamic collection of things in a sorted order. It is surprising how many problems become much simpler once you have a heap and a key/comp/comparator/whatever function that you can use to compare your values by. Previously I would go to torturous lengths by using sorted arrays and bisection (which was still O(N) for insertion). :slight_smile:

3 Likes

Priority queues make the world go round :slight_smile: Wait until you start looking into all the heap variants (binomial, fibonacci, etc.).

Funny enough about the push toward arrays + binary search, Alexander Alexandrescu had a great talk on optimizing the sorting implementation in C++, and focusing on mechanical sympathy that ended up being counterintuitive to algorithmic analysis.

3 Likes

People here may find this amusing, but the concept of property-based testing is new to me (this year). Prior to learning this concept, “testing” was synonymous with example-based testing to me.

You mentioned it, but homoiconicity is itself absolutely amazing, We’re still discovering new ways to take advantage of this language feature decades on.

3 Likes

I was a programmer for 5 years, using imperative languages then I took 30 years off and have restarted with Clojure. I feel like I’m having my brain rewired but I’ve slowly realised that some of the things I’ve learned aren’t Clojure specific.

Learning about collections and Clojure’s abstract data abstractions, it hit me explicitly that ‘ordered’ and ‘sorted’ are different things but we use the words very carelessly. Humans classify and put things in order in experimental ways that help us think and make decisions. That isn’t easy(/possible?) to do with algorithms. ‘Sorting out your old clothes for the charity shop’ is a personal-values-based filter, though you might do it through prioritization with a cut-off point. This is a similar process to using relative-value ordering to select the user-stories for a Scrum sprint.

4 Likes

This blew my mind. I’m still trying to pick my brain together.

8 Likes

So many golden nuggets in 30 minutes :slight_smile:
It’s worth to take the time and print out some papers to read (sorry paper you still rule). It certainly helped me understand the tools I’m using better and create better things with them.

1 Like

You know, that is not a bad idea. You need a printer like a few times a year, and so I have always been too cheap to purchase one. But if I was going to view it as a tool to make me read more papers…

I can recommend getting a printer with refillable ink tanks. The cost will shift from cartridges to paper. They are also great at printing photos which can delight dinner guests or non-tech savvy relatives. Much cheaper over the long run over cartridge ink-jet printers.

1 Like

Polylith. The idea is brilliant and the execution is just plain beautiful. I say without having put it to use yet, but I was given a demo during lunch today and my mind was blown for the second time in short succession. (I had been watching David Nolen’s talk just before the demo.) Got reminded by this reddit post:

5 Likes

I’ve been thinking about buying something like a Papyr or a Remarkable for a similar reason, like can they really replace paper?

@PEZ Can you elaborate on why Polylith was so brilliant/beautiful for you?

1 Like

@didibus I got a Remarkable2 for my wife some time ago, and she uses it all the time. It really has a great feel for writing, and if you’re used to doodling diagrams and taking notes, it’s a very viable replacement for pen and paper.

I can at least try. It is all a bit over my head still. What’s extra cool is that the idea, at its core, is super simple, if you arrange your dependencies and interdependencies in a certain way, whole categories of problems never arrive at your doorstep. There seem to be zero drawbacks with it. I know it was the architect of Polylith that gave me the demo, but anyway.

And the way it separates development from deployment, and thus lets you keep deployment options open, and deployment decisions deferred until really needed, is yummy for me who think development is fun and deployment issues dreadful. Then those well crafted CLI commands giving you overviews of your full project.

Speaking about developer fun. I too want to be able to use the same REPL for my whole project. I am not going to settle for something less than that next time I build something new.

Also, I think the ability to treat your other sub-projects (components) as if they were libraries, without having to make them libraries, enables 100% healthy reuse of your code.

I’m going to spend the weekend opening this box a bit more, maybe I will. be able to articulate this feeling of sensation a bit clearer after that. :smiley:

3 Likes

There are two episodes of the ClojureScript podcast where Jacek Schae interviews the team behind Polylith, btw. Here’s Part 1: