Worried about types + Clojure Elitism


#41

Ah yes that’s it :slight_smile: It looks cool, I’ll definitely check it out. I think it might be a bit basic compared to what I am looking for but batteries included and everything is a great start and I’d say that might be looking like the best place to start for a beginner at square one.

You’re welcome for the feedback - yeah I just feel the setup of all the tooling to get a dev setup going with Clojure is the biggest impediment to it’s adoption by beginners, it’s certainly what stopped me in my tracks, and that applies from beginner environments all the way through to fully tooled pro dev setups. Most other languages are much easier to get started with a dev setup in comparison. It looks like Zach Oakes is making some good progress with the beginner dev setups but at the more pro end it could still really do with some kind of 'Visual Studio" for Clojure.


#42

Thank you for courageously sharing your experiences to kick off this vital, but potentially sensitive topic. I appreciate that you don’t assume your perceptions are 100% accurate but that you are confident enough in them to raise this issue as a possible concern, which I would summarize as: Does elitism exist in Clojure and is it harming our community?

I’ve been thinking a lot lately about the paradox of communication. On the one hand, it’s so easy. Words roll off the tongue, often automatically. On the other, it can be extremely difficult!

We can talk about talking, but it’s a different thing to be there in that moment in time, in that group of people, perceiving the interactions through our own lens, life experiences and goals. All of which differ from those of each member of the group, the group and its network of sub-groups. It’s complicated. We work and interact within schedules and deadlines and people are naturally interested in participating, understanding, and getting their points across.

Compounding the difficulty, in our industry, we often work alone for long stints, coming out of our hermit phases, unpracticed in the finer nuances of communicating gracefully with others.

I’m keenly interested in this topic because I think our community’s success will be directly proportional to how effectively we learn to help newcomers learn Clojure’s unique approaches to problem-solving. If memory serves, according to surveys done by FreeCodeCamp, some 65% of programmers are self-taught. Every individual learner matters, but to exclude such a massive share of the market of minds would be foolish, at best.

But isn’t that easy for me to say, today. I’m not the one who authored Clojure, or who painstakingly supported its growth from its earliest days, doing the long, hard work of caring for the details which have made it as successful as it is today. I didn’t suffer through decades of programming in Java, working around the object impedance mismatch issues of traditional database systems, or developing web applications with pervasive, cyclical object mutation.

Those who did suffer through these stages are naturally more expert in Clojure than I. Relative to me, they are elite in our given field of endeavor. I hang on their every word to refine my mental model of solving problems with Clojure.

Each of them only ever had 24 hours in a day, a portion of which to dedicate to improving the Clojure experience for future Clojurians. Trade-offs had to be made! Should they have marketed Clojure to someone who didn’t know what OOP is, or wait for them to “get it”. How much time in their day did they have to explain rudimentary concepts? Or could they get more traction by focussing on professional developers who can immediately understand the significance of Clojure’s basic value propositions?

After re-reading Eric’s quote above I can see I’ve spent the last two paragraphs emoting the stages of “crossing the chasm”. I’m trying to use that emotion to make a point.

How easy is it for us to throw around the language “elitist” while leaving out a potential host of positive adjectives to describe those selfsame conversations among Clojurians. Are we the elitists, ascribing the worst of attributes to those who simply learned the most efficient path to get us here and took it?

Please don’t read into my point that I’m suggesting we shut down this conversation. On the contrary, I assume your perceptions have an essence of validity, the conscientious exploration of which can improve our outreach.

Moreover, I think this conversation is a positive sign and an indication that we might be ready to cross the chasm by standing on the shoulder’s of giants and polishing the Clojure tooling to make getting started with Clojure mind-blowingly awesome, a no-brainer for the average aspiring programmer. But we have other work to do too. And you’re tapping that vein of gold by airing your inner-most thoughts on fragile ground. And doing so with care.

At a recent Clojure meetup in Raliegh Durham, Stuart Halloway gave a talk about Spec, during which he used his jovial and light-hearted sense of humor to make some comments about deciphering Rich Hickey’s ultra-concise language around Clojure. I thought to myself, “Thank goodness, it’s not just me!” It was an almost gleeful moment of relief, probably because I was imagining in my head Stuart hanging onto every one of Rich’s words like I was his in that moment, and giggling to myself at the universality of the challenge of communication, even between two great minds like Rich and Stu. After all, there’s only so much time in a day and eventually Rich has to get off the phone with Stu and go about his own affairs.

Why was I so gleeful? We invest ourselves so fully into this shared pursuit. I’m no exception. I want to master this craft so badly! I look up to these people. To imagine Stu flailing about like me to expand his understanding was a comic relief and cathartic on a deeply personal level. This is the place from which I’m hearing you say we likely have something to improve on as a community.

In watching a handful of Lambda Island videos, I’ve gained a sense of how thoroughly you approach a subject and a respect for your analytical skills, which is why I look forward to unpacking this problem further, together.

It sounds like my experience (or the perception thereof) has been somewhat different than yours. I’m not formally trained or educated in programming. I’m one of the 65%. I first learned some JavaScript, then Python. I had a prototype for my web app in Python but my code was distorted with ORM concerns to keep the database happy. I didn’t want to be forced to alter my data model to satisfy performance.

My relentless research into the impedance mismatch problem led me to Datomic, upon which I discovered Clojure. I watched Rich’s talk Simple Made Easy and I knew in that moment that I was in the right place with the right people learning the right things. That feeling has never left me.

I’ve felt incredibly welcome and supported at every Clojure event and group I’ve plugged into. From the Google Group to the beginner’s channel on Clojurians Slack, the Clojure user groups to the 2016 Austin Conj, I’ve been blown away by the incredible people I’ve met and their willingness and desire to help me learn. I can’t list everyone here, but experienced developers from Clojure’s inner rank, Russ Olsen, Michael Nygard, Luke Vaderhart, have all invested one-on-one time to mentor me into a better developer.

So much so, that I have caught myself feeling guilty for wasting their time with trivial questions which I would know with a more formal education in our field. To be clear, no one ever did or said anything to make me feel this way, it’s just that I realized the inefficiency I was party to.

In fact, these feelings got me thinking about how to improve that situation. I think we could do a better job of developing, and more importantly, organizing curriculum paths which would spare both mentors and mentees from the distracting minutia, thus freeing both parties to interact around more core fundamentals.

And maybe we need to set aside a time and place for experts to convene, to interact unimpeded by trivial education distractions. Just as we need an outright revolution in education itself. We open source our code. Maybe its time to open source our education, en masse. ClojureBridge is a step in the right direction. I propose we need a sibling effort open to all persons.

In conclusion, I believe the sentiment behind your concerns about elitism is an important catalyst for positive change because we are at a natural stage of community evolution, not that we have an inherent elitism problem preventing our growth. It feels like we are at a critical juncture for moving Clojure into the limelight and I’m ecstatic that you are the people I’m playing that game with. Looking forward to your critiques, rebuttals, and proposals. My best : )


#43

Hi,

Clojure beginner perspective (~1 year). Worked in python, js, coffescript & mysql about 10 years experience, never seen a lisp before and I don’t really like math :slight_smile:

Pretty surprised how good most clojure developers are. And to be honest I really think most are “ellite”.
Clojure isn’t that popular and the number of jobs isn’t that big. So it kinda makes sense that most are experienced developers, that have decided to spend there nights in search of a “better way of crafting software”. Think you have to be pretty crazy as a beginner to not learn Java/python, javascript well (be able to find a decent payed job) and just start with learning clojure.

The thing I find really strange is there’s not really a big focus on “clojurescript” & reactjs. From my perspective clojure (java) is a lot better than python… But still don’t think it adds enough value to justify changing technologies. Python is decent enough, the number of developers that know it is pretty big, just not enough value if there isn’t a strong FP community in your city.

Clojurescript on the other hand, paired together with libs like: fulcro, re-frame. I can really make a case it makes all the sense in the world to ditch javascript and go for it especially you’re already using react-js. If you have the front-end in clojurescript ditching something like python in favor of clojure starts to make a lot more sense (same language, shared code, etc…).

The think I find strange is that I know a lot of people that do react-js and none of them have heard about clojurescript (they heard about elm some even did an app in elm). Not really much buzz about how awesome react-js is with clojure. Egghead.io for example, has everything but the kitchen sync (reactjs, rxjs, purescript, elm, etc…) but no clojurescript.

I was lucky that a colleague played a bit with clojure and suggested it, so I decided to “give this alien looking ugly language 1 month” since I wasn’t happy with the end results of using: react, redux, graphql, immutablejs, python/nodejs.

Clojure is an amazing language, clojurescript tooling & libs are really impressive.
But marketing & outreach to javascript developers, it’s really not that great… or at least that was/is my experience :slight_smile:


#44

Hey @claudiu,

Thanks for sharing your experience. Sometimes it’s hard for us who live in the ecosystem to know how it’s perceived.

I think you’re right that ClojureScript is not as well-known as some other languages, though it is really mature. I gave a ClojureScript talk at a local JavaScript conference and when I listed all of the features, they were very much like “we have that with webpack”. Live reloading, for instance. But then when I dug deep, it turned out that they had it sometimes, after two weeks of trying. How do you market that? Because sometimes we feel like webpack is stealing from us (and the stealing is welcomed), but then doing it badly and giving the feature a bad rap. Same goes for React. And Redux. And Immutable. We need better marketing, for sure.

Giving me some ideas. I’m not that great at online community building or social media. But it does seem like some effort in that direction would be useful.

Rock on!
Eric


#45

@ericnormand

Static typists believe that untyped programs are a subset of typed programs […] dynamic typists see typed programs as a subset of dynamic.

This is a super salient observation!

But, IMO, therein lies the value-add of dynamic languages: You can always impose static strictness on a dynamic system - the other way, not necessarily so easy.

Might any sufficiently complicated set of Haskell or Idris type definitions contain an ad-hoc, under-specified, bug-ridden, slow verification of half of Clojure’s semantics? :joy:


#46

But, IMO, therein lies the value-add of dynamic languages: You can always impose static strictness on a dynamic system - the other way, not necessarily so easy.

That’s right. But everything is parallel. They can make types as loose as they want, e.g., use Strings for everything.

Dynamic: Loose now —> Strict later
Static: Strict now —> Loose later

Where later usually means never.

It sometimes feels that way :slight_smile:

Eric


#47

Dynamic: Loose now —> Strict later
Static: Strict now —> Loose later

Is not strict-to-loose inherently more difficult than loose-to-strict? Perhaps not inherently. Though I can’t imagine how more constraints could increase the expressiveness… Could be my failure of imagination.

But your observation brings up some other question for me:

Is not the base type of everything a 1 or a 0?

Then we have higher composed types like words, ints, longs, etc.

And whether we call them statically typed or not, aren’t all modern languages statically typed in this way? Where the base byte formats are all marshaled, checked and managed by the language, often at compile time?

In that sense, untyped programs would be a subset of typed programs, in the sense of byte-layouts being managed.

Wouldn’t it be unfortunate if, in our high level languages, we still had to always handle marshaling data between byte formats manually?

So it would seem that formal type definitions are extremely useful when a problem domain is well specified / defined, especially when you want to build a rigorous model that can abstract over those minutia internal to that domain, like byte-layouts. In other words, strict typing seems to allow for the creation of more generic, loose abstractions.

If that were true, then it would seem zealotry on the strict typing side would be missing the forest for the trees - types should allow us to abstract over the semantics of the objects they define, not to be imposed on the rest of the universe.

Do those intuitions ring true to you or do you think I’m off base?


#48

I my experience, strict to loose is easier. You can always make types that encode something without the benefits of type safety. For instance, you can encode stuff in Strings without type safety.

Loose to strict, however, requires you to write a type checker. Not a gargantuan task, but not trivial either.

I think that’s what they try to do in the best cases. They’re building abstractions on top of abstractions, usually for stuff that are discovered in mathematics. That would be where category theory enters the conversation.

They will acknowledge that static types have a cost. But they perceive the benefit to be much greater. They’re willing to appease the pedantic type checking for the benefits of reasoning at a higher level of abstraction.

Eric


#49

It wasn’t so much that Evan Borden succeeded in the details—it was the shift in perspective (“what would Clojure typing look like in Haskell”) that was interesting and new (to me).

Taking that a step further, it is interesting to think about what happens with interoperability between languages/type systems. Re-watching Wadler’s Propositions as Types led me to his blame calculus paper (with Findler): Well-typed programs can’t be blamed.

In that paper, if I may butcher the terminology, they develop a hybrid language with untyped exprs and typed exprs bridged with casts. They introduce a “blame” concept that captures two kinds of error that can occur at a cast. They then, I think, go on to show how such a language can be pretty good. (I say “I think” because the notation gets pretty dense in the body of the paper, so I have to admit to very skimming those bits.)

They conclude with:

Can we promote the evolution of programming languages by
some mechanism other than ‘survival of the fittest’? Integrating two
competing designs into a single language may provide a better basis
for comparing the strengths and weaknesses of each

(my emphasis)

The model in the Wadler-Findler paper is contract-based. Contracts à la Eiffel and specs à la Clojure are strongly connected of course. It makes me wonder:

  • what is the real relationship between specs and types?
  • could we use specs to support the vaunted IDE refactorings so many of us seem to care so much about?
  • is there something to be learned by exploring the use of specs at the FFI between e.g. Clojure and Eta or ClojureScript and TypeScript—in particular would that exploration lead us to a better “understanding” between the Clojure community and the static typing communities?

#50

This is almost certainly something simple. If you drop a message on the mailing list or on the Clojurians Slack, or just email me on [email protected] I’m sure that we can get it going quickly.


#51

Interesting. Any source or more information on this?

I might also ask this on Twitter or somewhere else because I’m curious:

Do we want a larger community?

For instance, do we want more Javascript programmers to be Clojure/Clojurescript programmers? If that really seems like the overall best thing for Clojure , then I’d make it a priority for myself to do things that might help achieve that.

I’m young programmer who came to Clojure from JS this year. I still have to use JS for work and it positively kills me, so if there’s something positive that could come out of that by the suffering informing a pitch to JS devs or whatever it is, that’d be wonderful.


#52

This thread seems to address two different topics: types and elitism. I want to add my two cents to the former. I agree with some of the comments that elitism is not a great word to use to describe the problem, even though this might well capture the feelings of a newcomer to the language. I think the problem is that Clojure is just not that friendly to newcomers. There have been great efforts in this direction, with ClojureBridge and the work of Eric Normand and Daniel Higginbotham. But in my view the problem lies deeper, in the culture of Clojure.

My most recent experience in learning another language was with Elm. The contrast between the vibe you get from the two communities is stark. The Elm community is pervaded by the spirit of wanting to help new users. This comes from the top down, beginning with the language author, Evan Czaplicki. He talks constantly about the experience of new users to the language. This emphasis is most manifest in Elm’s famously helpful error messages. Users talk about the sense of having the compiler’s error messages guide them in using the language. Error messages have been one of the long running complaints of Clojurists, and clojurists have looked to the core maintainers to address this issue.

Stuart Holloway addressed this concern again in some of his recent talks about Spec. He has pointed out that specs can generate code which can then be used to expose better error messages. He then asked rhetorically what the community needs from the core team to build to make error messages better. Answer: nothing.

While this is technically true, it seems to side-step the truth that a programming language is not just the language itself, but the community and the larger eco-system. There are significant challenges to a community-based solution to better error messages. First, efforts in this direction are equally susceptible to the lisp curse, where many different people roll their own solution and no single one gains enough of the support of the whole community. This then leads to the second challenge where one solution fails to gain wide-enough adoption to become available to beginners by being integrated into the tooling, rather than as a library they they would need to learn about and adopt. If there are many different solutions, it also becomes harder for community members to justify the investment of their time and energy into a solution that will not be widely adopted.

While it is admittedly unlikely that Clojure will be a user’s first programming language, there is alot to be gained by assuming new users come to the langauge with much less experience. Assuming a knowlege of the Java ecosystem can be off-putting or daunting to someone who has been user Ruby or Python for years. There seems to be a resistance to “dumb down” Clojure for newcomers. An example of this is the introduction to ClojureScript (https://clojurescript.org/guides/quick-start) where it takes the beginner through the steps of creating each file and calling to the ClojureScript and Google Closure compilers, even though this is not how most users of the language work. Now, I understand the motivation for this approach, a pulling back of the curtains to show that there is no magic here. But it would be more beginner friendly if the quickstart was a quick intro to how the average user compiles their code, leaving this step-by-step walkthrough until the user has some code compiling and is ready for more.

Having said all of this, I am not sure what the answer is. The Rust core team devoted most of 2017 to improve the usability of the language which the goal of furthering adoption. This has galvanized the whole comminity and channeled their efforts. However, this seems an unlikely thing to hope for in Clojure, given the comments of the core team. This is not where their interest lies. This is not to say that nothing is being done by the core team. I am very excited about the cli tool and deps in Clojure 1.9. This is certainly a step in the right direction. But if they are not interested, it leaves the wider community to address the problem. Zach Oakes is generating a lot of good ideas. But how do you get momentum for this kind of thing in such a wide-ranging community? Perhaps Clojurists Together could raise money and fund an effort for better error messages and tooling. Maybe the core team could deputize a documents committee or error messages working group that could be a central clearing house for ideas and efforts. They would benefit from the authority of the core team without drawing from the limited resources of Alex Miller and others.

Clojure is an wonderful language. Rich, Cognitect and company have done an excellent job, and should be thanked and praised early and often. I would love to see its adoption increase.


#53

Any source or more information on this?

I was specifically referring to David’s talk at EuroClojure 2016, which I took as a rallying cry to go forth and get people using ClojureScript, because CLJS tooling and workflow has matured and now matches the awesomeness it has had since the beginning in areas like lispiness and immutability.

Do we want a larger community?

Yes. :slight_smile: To expand on that only a tiny bit, more people means more contributors and users, which in turn means more and better libraries and tooling and discovery.


#54

Thanks for digging that up. Watched the whole thing and agree with your assessment


#55

For instance, do we want more Javascript programmers to be Clojure/Clojurescript programmers?

Personally, I do want this, for selfish and unselfish reasons! :slight_smile:

Selfishly, a larger community increases the number of possible Clojure(script) jobs. Whether it’s an accurate concern or not, a common concern for companies considering Clojure is whether or not they’ll be able to hire developers. Also, when considering a language to invest their time in, developers do look at whether a language is growing, shrinking, or remaining steady.

Less selfishly, Clojure(script) is an excellent language and I think it would add tremendous value to many projects. Since I think using Clojure(script) produces better software in many domains, I want to get many more developers using the language.

I’m young programmer who came to Clojure from JS this year. I still have to use JS for work and it positively kills me, so if there’s something positive that could come out of that by the suffering informing a pitch to JS devs or whatever it is, that’d be wonderful.

I think that you (and developers like you) possess one of the most valuable attributes for growing our community: concrete experiences about specific pain points you encountered in JS and an explanation about how CLJS solved those problems. For instance, this is one of my favorite posts on the subject because it focuses on specific problems and how CLJS (specifically re-frame) avoids the problem.

This is a valuable approach because even though CLJS tooling is good and improving (due to the hard work of many - thanks!!!), it’s still a big jump from JS. If we try to motivate JS devs to switch based on minor or abstract features, they will be more likely to abandon learning CLJS when they run into the rough edges that still exist. But if we can motivate them by demonstrating how CLJS can solve their trickiest problems, their hardest pain points, then they are more likely to be motivated to continue learning.

Personally, I came from the Ruby world. I think Ruby to Clojure is a tougher sell just because there are so many languages competing for the mindshare of ex-Rubyists on the backend. Still, my transition followed a similar path: I had been bit really hard by bugs in mutable state in Ruby (as well as countless hours spent on minor adjustments to class hierarchies). Those pain points motivated me to continue to learn, even though the transition was really, really tough for me.

I agree that tooling, error messages, etc are all problems for beginners. But for me, the biggest barrier was failing to understand the process of writing Clojure. Reading source code doesn’t reveal anything about how it was made. I probably spent a year trying to write Clojure the way I wrote Ruby - using a debugger and running tests by invoking lein test over and over. As a result, I spent a lot of time trying to get a Clojure debugger working and trying to main lein boot faster. It was much much later that I started to learn how REPL-driven development could change all that. So, as we evangelize to the JS community, we need to remember to not only focus on problems solved, but demonstrate the processes we use to write Clojure. I’m not sure the best way do this online, but certainly if we get the chance to do in-person demos, we should show off this interactivity!


#56

Right, and more jobs (as @bbrinck pointed out), and more money from companies to buy dev tools, sponsor conferences, etc etc.


#57

OK thanks for the offer of help Colin, it’s been a number of months since I was last trying it out so I can’t really remember much at all right now. I just tried it out now and I can’t even seem to open the src file for editing in IntelliJ now. When I double-click on it it brings up a Project Structure window and when I drag it into the window beside it to open it nothing happens so I’ve actually regressed :smiley:

I’ll make sure to drop you a line by email or Slack sometime within the next week; I’ll look into it a bit more first, maybe time having passed it’ll simply work for me now :slight_smile:


#58

Do we have some actual data like repository activity/# of contributors/-ions from GitHub/BitBucket/… and even Clojars/Maven/… to support the (apparently rather ubiquitous) impression that ‘we’ are not growing, or that growth is somehow stagnating?


#59

But for me, the biggest barrier was failing to understand the process of writing Clojure. Reading source code doesn’t reveal anything about how it was made. … I probably spent a year trying to write Clojure the way I wrote Ruby - using a debugger and running tests by invoking lein test over and over.

Yeah, very good point! I’ve given demos of this interactive style. Don’t know if it changed any minds, but maybe it was useful for someone trying out Clojure who still needed to cast off old ideas.

I think it’s hard to demo a full dev process. Even at most Clojure-using companies, it’s hard to win the freedom to develop naturally. For instance, my dev is intertwined with sleep & thinking. Already, I feel I don’t take enough showers—where everyone says they get the best ideas!

Whenever anyone argues with me about Lisps, I feel we have alien notions of programming. And I know their notion better than they know mine, because theirs dominates the industry and requires energy to reject.

Yet a REPL’s exactly what we demand from DBs and OSes. We require interactivity at some level: to experiment and inspect. A game-like approach where you mold a live universe.

Maybe its significant that mainstream REPL languages are weak, like bash. (What’d happen if a REPL language was powerful enough for general programming?)


#60

There’s also the interpretation that every dynamically typed language can be seen as a singly-typed one.

The clmap type signature could arguably seen as: clmap :: EDN

Exactly, I agree. When we work in any dynamically typed language we constantly think of the types and what we have in mind is often not very formal and it’s not precise. With that mindset we do the type-checking ourselves and we may think about the types that are used to back a certain constructor of an EDN value. You explained it well, we know that map takes a function as first arg. Even a function that has one parameter - as long we map over exactly one seq. If we map over three then our function should have three parameters. This is something that is trivial to express in Clojure but more difficult in Haskell. There we need zipWith and zipWith3 and zipWith7. Haskell is still evolving and it will take possibly 1-3 more years until one single map function in Haskell can do the same (via Dependend Types).

Though to be fair: map is a relict from older days and it’s kept because it makes for a great teaching utility. What we want is something that is even richer, and that would be

fmap :: Functor f => (a -> b) -> f a -> f b

The clmap that you suggested is not complete enough because

(map #(+ %1 %2 4) [1 2 3] [100 200 300])

is a valid usecase of map.

Without dependend types I wouldn’t know how I could even represent this variable-arity map function.

The Seqable can not be heterogeneous. It’s all values of the type EDN. You just think of it as heterogeneous. In Haskell it’s the same if you were to use the EDN type. Besides that: you can express heterogeneous collections with advanced type system features. This is something that we can’t do in dynamically typed languages (although we think about it like that).

And in such a case, if you just want to print things then several dynamically typed languages offer a quick and simple way of making this possible. In Haskell it will likely take a bit more time. It’s Pros and Cons, depending on the situation. If you do this in Haskell either with an EDN type or with a true heterogeneous type-safe collection then you may also benefit from this and actually save development time.


Your first scenario would probably rather be met with a specific type in Haskell. Values that we know can be handled more comfortably and efficient with our own record type.

In your second scenario you can write a Clojure program that can merge two maps with unknown keys. But if you really don’t know what your keys are you can never really do anything of interest with them. You may try to print them but I could supply a .toString implementation that crashes the program. So printing is also not safe for truly unknown keys.

And you can’t do #2 with them, that is, treat them as generic containers.

You could do this with existential quantification. But you couldn’t “do” anything useful to them.

I am not sure here if you refer to https://hackage.haskell.org/package/row-types