Hey, everybody!
Don’t know if I started a topic in correct sub-forum, if not — I hope moderator will move it to the right place.
As a partially self-taught programmer, I noticed that there is an excess of blogs, infos, books and articles about concrete implementations, algorithms, solutions of whatever you need.
And this is fine, but very few books are about the bread-and-butter of every programmer - using abstractions. What is the difference between indirection and abstraction? What kind of abstractions there are? What are the tradeoffs of adding abstraction, what are the situations when one should abstract something away? Should I move some functionality to a separate function? Why? Etc.
It’s supposed that one can internalize these decisions by experience and develop some sort of intuition to these questions. But I’m looking for more thoughtful, clear and direct approach. Less of “this is my experience and how I do things”, more of a focused research and direct thinkings on this subject.
One of the books I’m aware of is Zach Tellman’s Elements Of Clojure. I haven’t got myself a copy yet, but I read his short excerpt about abstractions and namings, it is quite profound in my opinion.
I’m looking exactly for this kind of books/articles: high-level, abstract book about abstractions (sorry), functions, namings and all the general stuff that is not linked to any programming language or algorithm in particular. Are those books even on a programming/engineering level, or should I go to the bottom of something like systems theory etc?
It’s my belief that the more fundamental, general knowledge and awareness of what are you doing give you more than any particular method, pattern or framework can ever give you.
Answers, recommendations and links are highly appreciated, thanks!
Thanks, I’m already familiar with “Out of the Tar Pit”, going to read your other recommendations as well!
By the way, could you tell me what was the impact of those books on you as a programmer?
It is suggested to make full use of the subject knowledge that has been studied before as a metaphor.
You’ve read my blog, purefunctionpipelinedataflow, and you should be able to find that there are many metaphors and ideas that come from the accounting knowledge and statistical knowledge I’ve learned before.
OO is a rudimentary and shoddy metaphor for the real world.
…also a book about algorithms and data structures is good (sorry; can’t think of suggestions here) once you’re ready for this. Understanding data structures etc. is crucial when doing software design; it is more connected than it might perhaps seem at first.
The Pragmatic Programmer was the first good book i read about programming in general that really clicked. It presents a principle, expands on it and provides examples. A few years have passed since I read it the first time, and I still get plenty out of it.
I would also like to point to the Clojure ecosystem in general. There are just so many thoughtful resources. The ClojureTV YouTube channel has a good collection. A few personal favorites:
Figwheel shows how tight a feedback loop you could build yourself. I’ve totally changed the process by which I develop Python as a result – I make myself a reloading script, then reload my code in-place on file changes. I can also cache results using an equivalent of defonce. Many other languages allow for a Clojure-inspired development flow, and once you know the idioms, replicating key parts may be less hard than you expect.
The Clojure tools.cli example program shows how you can build really tight code that handles all errors gracefully without resorting to advanced error systems or some option/failure monad. If you manage to expose the core part of your application as pure data (see the cli-options), you can build a really simple, really flexible API.
The tools.deps.alpha source code shows using multimethods for an extensible system. The clojure.tools.deps.alpha ns imports all the extensions. Each included extension has its own namespace, and a defmethod after some local convenience functions. Functionality can be added simply by creating new extensions.
You ask for “direct”, analytic-brain, research-backed works on the subject. I am skeptical of any research in this topic, because I do not believe the field is developed enough to make practical conclusions from the few axes we investigate through usually tiny-n experiments with ill-defined-outcomes. It’s a fundamentally difficult topic to study, the field has been moving faster than research can reasonably keep up with, and finding good metrics and use cases is a terrifyingly hard problem. So right off the bat I think you should lower your expectations. Science will not save you from the need for intuition here. I’m willing to be convinced otherwise but the science would have to be very convincing.
Kernighan & Pike’s The Practice of Programming addresses some of your specific questions about abstraction, and is overall a good read. I warn you that most of their advice is rules of thumb based on precisely the internalized experience and intuition that you wish wasn’t the basis for our knowledge.
Related to your question, I have a long-standing search for a book similar to the Gang of Four’s Design Patterns, but oriented towards functional programming (and even lisp) instead of object oriented programming. My current hypothesis is that such a book sadly does not exist. (Please prove me wrong.)
I agree completely, the field is not mature enough to have solid research about this topic. I wasn’t asking for some bulletproof theories for writing programs though, I don’t even think there could be any. I was curious is there more of a research (informal is ok too) on specific topic of using abstractions in programming, may be even some “esoteric” things bordering on linguistics or systems thinking or cybernetics. Hell, why not, after all our code is blend of text and system
Thanks for all your recommendations!
By the way, there is a half-serious Clojure design patterns, I guess you’re already familiar with it
I haven’t read them, but I have seen them mentioned often. They come from the ML and Haskell world, so you may have to go through a syntactic adjustment period.
I think that’s mostly because the classic Design Patterns we see for OOP are “just” (higher order) functions in FP. Many of the “Core J2EE Design Patterns” specifically exist as workarounds for problems that Java-style OOP introduces and even several of the “Gang of Four” patterns are really little more than workarounds for limitations of OOP’s abstractions (everything is a class/interface – and functions were not first class in the languages where Design Patterns arose).
The hype around Design Patterns came at a time when the world had gone “all in” on OOP and was beginning to feel the pains specific to OOP, after the honeymoon period where OOP had dramatically improved how we designed and implemented solutions to problems. AT&T’s Cfront E (education edition of their C++ compiler) was released in '84 and within half a dozen years C++ rocketed into the mainstream, and then half a dozen years later Java appeared, a year after the Design Patterns book had been published. That period – 20-30 years ago – was an insane time for the hype around OOP and Design Patterns and “The Language of Patterns”. We had whole conferences dedicated to patterns and research/thinking around them but it was all about OOP. Looking back on that time, and the myriad conferences I attended, I think we were “inventing” ourselves out of yet another tar pit!
I would love to hear more about your Clojure-inspired Python work flow! I have been trying to understand why this is generally not done, is there a practical reason or is it cultural?
Cool that you’re interested. I think it’s mostly cultural, when you get that work flow you typically choose other options. Also, I think Clojure developers value a dynamic workflow over simplicity, compared to Python devs in general.
In principle, I watch files for changes and use importlib.reload. I plan to put some more work in the API, and open source if there’s interest.
Interesting. I have been trying to put together a talk on Clojure for pythonistas and the bit I’ve been struggling with is explaining how awesome proper repl driven development is. I was toying around with using Jupyter notebooks to try to do this
