Clojure tradeoffs and how to mitigate them?

Community Center Beginners
1

Disclaimer: Posted this on ask.clojure.org but thought asking in a more community-centric sphere would be beneficial and potentially more conducive to the nature of this discussion.

Earlier this week I conversed with a person who had decided to stop using Clojure, I have summarized their points. Before we grab our pitchforks to go after any party here, I am more curious about what we can do to mitigate these tradeoffs if you agree with them. What have you tried? What’s your experience? What’s your advice?

I feel it’s important to state my goal is not to be critical of anyone’s work here. I LOVE Clojure and I find this person’s insight motivates me to learn about Clojure’s tradeoffs and explore how to address them using the plethora of tools already at our disposal.

Below is a summary of that person’s issues followed by my own thoughts.

REPL driven development

  • Felt like a work-around to the slow startup problem
  • REPL dev was not productive. Often encountered broken REPL states requiring a restart.

Clojure

  • No static types.
  • How do you know what keys and values are within a map?

Spec

  • Too verbose. It seems to contradict the conciseness that Clojure typically embraces.
  • A lot of legwork required to get value from spec requiring you to write many predicates, s/ forms, checkers, etc…

My thoughts:

  • Startup time of CLJS or CLJ hasn’t felt like much of a barrier to me in development but GRAAL could be a good solution.
  • I haven’t encountered as much broken REPL state with Clojure but it definitely happens frequently with CLJS and Shadow. Is this something we can mitigate using libraries like Integrant or Component? What often leads to an inconsistent state?
  • As for static typing, I can see why it’s a goto solution for knowing what a map contains but aside from spec how can we address it in our code?
  • Since speaking to this person I have observed there are many times where I’m executing in the REPL just to find out what data I have.
  • Should we be adding more docs to our functions to specify which keys we need? Could we make better use of destructuring to help emphasize expected keys? Are there better tooling mechanisms to help with this?
  • I don’t have any issues with spec myself but I’m not the most experienced with it.
  • What I like is that you are constructing a DSL to specify the relationships in your domain logic which seems advantageous to me over static typing which brings all problems down to a comp-sci academic level.
  • Do I really need to know if this lead I’m trying to send to a vendor in my marketplace is a Record Type? My understanding is that spec is designed to instead describe the nature of the data.

What are your thoughts?

3 Likes
2

Please report shadow-cljs bugs if you have a specific set of commands that gets the REPL into a broken state. This is obviously not intended and will be fixed. I just don’t know about any specific broken cases and if things aren’t reproducible or don’t happen to me I cannot do much.

Graal will NOT solve any issues regarding startup time during development. You are trading a lot of the dynamic features for the improved startup times which is not something you want during development. Startup time is also generally not a problem as soon as you embrace a REPL based workflow and rarely restart. If you fight the REPL and constantly restart its horrible but if you only “start” once every couple of days it doesn’t matter.

I don’t really have any general comments about the other points raised.

3

Feel free to file this under controversial opinions, but here goes:

Felt like a work-around to the slow startup problem

You’re supposed to live/develop in your program and process, which brings us to

REPL dev was not productive. Often encountered broken REPL states requiring a restart.

Which makes me suspect usage of one of the state-management frameworks. I only had a chance to develop with Clojure (not script) and I’ve found that the state management frameworks cater so much to the object-oriented style of writing, that you get back all the object-oriented problems, mainly, messed up state, the only solution to is killing everything and starting again

No static types

Some people like them, some don’t. Seems more like personal preference, that.

How do you know what keys and values are within a map?

When you receive a JSON object from outside, how do you know which keys and values are in it? You don’t. You can have a contract, you can validate the schema, but in the end you don’t know. For your internal functions, you should be able to know, so the question should mostly be about the borders of the system.
That’s where I lean to use Malli at the borders and spec internally.

spec

I see the point but it seems like a powerful enough tool. There are also 3rd party solutions like Malli and schema. There’s also clojure.typed which is back in development.

But all of that doesn’t mean there’s anything wrong with Clojure or with your friend. Could be they think in a way which doesn’t lend itself well to Clojure / lisps in general, like I have a hard time with the ALGOL family.

2 Likes
4

I welcome discussions like this. Can I get some background info on that person? What prior experience with programming? Which language did they come from? How much time they spent trying to learn Clojure? Are we talking about Clojure or ClojureScript?

I think that can help me understand a bit better the perspective.

I don’t know what to address here. This seems a misconception? The REPL isn’t a workaround for slow startup times. Even if Clojure started in under 10ms you’d still want a connected in-IDE REPL driven development workflow. For example, babashka starts in under 10ms and has just added support for nRrepl a highly demanded feature!

If we want to address slow startup time for use cases that need it. I’d say for all things scripting GraalVM, Babashka, Joker and self-hosted ClojureScript are pretty straightforward full featured workarounds. Scripting is a solved problem with those in my book.

For non scripting use cases slow startup time is still a problem. I’m thinking of someone wanted to make an Android app, serverless functions like AWS Lambda, a small GUI desktop app, etc. Those I think there’s still to wait for startup time improvements. The workaround is just to live with it or use ClojureScript for now.

I rarely get into these, like really rarely. I think the workaround is just to learn how to use the REPL better. Maybe we need more guides? But also I suspect users must just put in the time and effort to learn.

That’s not enough information. It’s like saying I didn’t like the logo. We’d need to understand what pain the lack of static type were causing them? Did they find there code had more bugs? Did they find they were less productive, because of lack of types driven development? Did they find runtime performance suffered because of it? We’re they just missing the auto-completion of keys in records? We’re they missing a rename refactoring? Etc.

The answer to this is you know the same way that you know what a for loop does when looking at it. By reading the code you can figure out what keys are in a map. This comes with practice, getting better at reading Clojure code.

It helps to know how to leverage the REPL to try things out sometimes.

Ideally, the code author provided good documentation, good variable names, maybe used destructuring, might have specced their entities, etc. Otherwise you know by reading the code.

You could also make heavy use of records, but you learn eventually that you don’t need too, once you become comfortable enough with Clojure.

That’s somewhat true, and Spec2 I think helps a little bit. But this comment also makes me feel fundamentally they didn’t understand Spec.

Spec is all about individual fields. This is a big difference from some other statically typed languages or from schema. It is also why it is more verbose, it forces you to spec each field independently. This is also why it’s awesome.

Again, I think this needs more details, what kind of “value” are they looking for?

If you need to perform data validation, Spec saves you a ton of time compared to custom validation. Same if you need to generate data, or perform generative tests, or stub functions, etc. If you were trying to get static type checking you missed the point and didn’t understand what Spec is for.

1 Like
5

I feel that if someone wants static types, fast start-up, and is unwilling or unable to change their workflow to match Clojure idiom, then maybe Clojure is a bad fit for that person. That’s okay.

That said, I’m always in favor of more tutoring/pair-programming and materials — GIF tutorials, livecoding streams, pre-recorded screencasts, blog posts sharing tiny tooling/dev idioms — that explain the REPL workflow. It’s a weird and unusual and different way of interacting with your code. We can always do more to help folks make that transition.

I also think we should be willing to admit that slow start-up is a trade-off. (Though it’s not the reason the REPL exists.) Faster Clojure start-up would be better, and I’ve been glad to see efforts to improve it. Onequestion is whether this is a project requirement (which is valid) or a workflow issue.

6 Likes
6

First off, let me just say that Clojure isn’t for everyone and we shouldn’t be terribly concerned about any subjective reasons for leaving a language and moving on to another.

This is something that is absolutely fundamental to effective Clojure development – but we, as a community, still don’t seem to have mastered how to explain this well to folks who are new to Clojure. Lots of existing Clojure developers still haven’t embraced this. Part of the problem is that it’s a complete mental shift in how you develop programs that has no corresponding practice in other, more mainstream languages. Some of the old Lisp systems worked this way. Smalltalk worked this way. Almost everything since then has been either edit-compile-run or a very simple interactive (and often interpreted) “REPL” that bears very little similarity to what we have in Clojure.

As above, the REPL is a core part of Clojure’s development workflow. I run my main development REPL for days, sometimes weeks. I can dynamically load new dependencies into it (via the add-lib branch of tools.deps.alpha; but boot also supports that and I believe there are solutions for lein too – although those do not seem widely adopted?). If someone feels it’s any sort of “workaround” then they’re just not using it correctly, which brings me to…

…feeling that “REPL dev” isn’t productive goes hand-in-hand with feeling it’s a “work-around”. It’s an alien workflow for folks coming from other languages and it takes a huge shift in thinking for it to become core to how you develop. As for the “broken REPL states” that’s just not something I run into. I’ll also add that I don’t use any of the “reloaded” or “refresh” tooling that some people feel necessary to “work-around” state issues in the REPL (I do use Component heavily tho’, so I can start/stop lifecycle-based things like web servers, database connection pools, cache system setup, etc).

I use Chlorine with Atom and it’s a deliberately simple workflow: it connects to any Socket REPL, without nREPL/CIDER or any sort of “completion” libraries; it provides eval form, eval top-level form, load file, show docs, go to definition, run tests, and a handful of other commands that I use a lot less. I can connect it to any local process, to a remote process on QA or even production, and I have the exact same workflow in all cases (well, I run Cognitect’s REBL locally so that is a difference when working with a remote REPL).

I eval every top-level form ctrl-; B as I edit it so my REPL state is always fresh – without even needing to save files. I rely heavily on “Rich Comment Forms” for code that sets up, or tears down, state for testing things in the REPL. Chlorine shows evaluation results inline, so my REPL window is tiny – just a couple of lines at the bottom of my screen so I can confirm an evaluation finished (or produced a stack trace). For the few things in Clojure that can get a REPL into an odd state (protocols, records, multi-methods, etc), I have recently added hotkeys bound to remove-ns and require ... :reload-all but I hardly ever need them (I would invoke them manually on occasion but rarely enough that having the hotkey now is more of a luxury than a need).

If someone really likes type systems, they’re just not going to like Clojure. And that’s fine. Over the three and a half decades that I’ve been programming, I’ve used a mix of static and dynamic type system including, early on, various ML-family languages before Haskell appeared, and I’m just plain old’ happier in a dynamic language. There are pros and cons to both and each person will make their choice based on the trade-offs they’re happiest with.

That question applies to any external data consumed, even in very strong statically typed languages. Clojure’s design is that maps are inherently open and you have plenty of choice about how to deal with missing required keys: you can let code fail/blow up, you can assert they’re present, you can use Spec or simple predicates directly, or you can handle missing required keys as part of normal program flow and deal with them, gracefully, via error messages/codes.

That’s definitely a subjective opinion! :slight_smile: We use Spec very heavily at work – and have done ever since the early alphas appeared with it in 1.9 – and Specs form a very small portion of our overall codebase so they don’t seem very verbose to us (see my blog post about the many ways we use Spec).

Again, that seems very subjective to me. As my blog post above shows, there are many ways to get value from Specs – and a single Spec describing the structure of your data lends itself to reuse in development, testing (both example-based and generatively), and also in production.

In summary…

Language choice is pretty subjective – if there were truly objective metrics by which to measure how Language A is “better” than Language B, we would all have consolidated around the top one or two (or maybe three) and almost no new languages would be created at this point. That’s clearly not how things are working out so we all make choices based on what we prefer in a language. For folks who are very attached to static type systems, there is a wide variety of options for them to choose from, but Clojure is not one of them. Similarly, for folks who are really into OOP, they probably won’t like Clojure with its policy of data being all open and not encapsulated.

I think one area that your friend’s concerns really point to as needing more/better work from the Clojure community is the area of REPL-driven development. It’s one of those things that, when you “get” it, you’re truly productive, but we seem to be doing a very poor job of showing the “how” and the “why” of this to others.

5 Likes
7

Looking at the responses here, I may have unintentionally misrepresented the person a bit. People seem to be under the impression that this was someone who tried Clojure for a week and didn’t like it. That is very far from the truth.

Instead, this person has used Clojure longer than I have and has gone much further with it than I have in terms of challenges they have tackled.

The awkward bit here is I wouldn’t say they’re a friend. The context here is I was looking at a Clojure library I use and in a support question, someone else asked, the maintainer responded that they had moved on from Clojure in 2017 but would accept a pull-request. I then reached out to get their insight and felt there was some merit to it worth discussing and getting a counter-perspective on. I respect this person, I like discussing such things with this person, but I can’t say we’re close or that I’m well versed in their background.

@didibus
Going by their public github they started using Clojure in 2014, produced about 10 Clojure repos. Some general libraries\frameworks, some one-off projects before they stopped using it around 2017.

Given that history, I am confident they knew how to use a REPL and how to work with it.

@seancorfield I agree with you that these are all subjective concerns and language choice is subjective. While I see some merit to some of their thoughts, it could simply be that they changed, their tastes changed, and another language was better suited there. I still think it’s worth discussing as I’ve encountered some of those issues myself and wanted insight to improve upon. I’ll try to elaborate on my experience with an example from the other day.

More importantly, those are some great links and helpful tips. It definitely makes me want to give Chlorine a shot!

Though since you’re able to run your REPL for days at a time, I wonder if it has a lot to do with project type? For web apps, I’m betting REPL issues are somewhat rare given the maturity of the tools and how much mindshare there is in that problem space for Clojure. For some of the lesser established problem spaces out there, maybe it’s harder to get those guarantees? Especially depending on extensive interop with JS and\or Java?

@thheller I appreciate you reaching out to offer your support. This discussion got me thinking that the issues in my experience are not likely Shadow related, though there is some funkiness with macros I may be able to report. As for the messed up REPL state, the project I’m working on at work is running an express server + feathersjs + mongoose wrapped in ClojureScript. Express is fine, but feathersjs and mongoose are very OOPy with a lot of state. Mongoose is particularly odd in that you have to define schemas against a connection and if you try it again it throws an error. I was able to work around it by using Shadow’s meta tags for async reloading to delete the schema from the connection on the stop lifecycle async function. Every once in a while though, something odd happens where it doesn’t quite reload cleanly and requires a full restart. I’ll try to report if I can reproduce it consistently. However, that’s usually resolved by rerunning node target/app.js so not a huge deal.

@seancorfield Here’s an example that made me see more merit in what this contact said which inspired this post:

(defn- oauth-hash [base key]
  (-> crypto
    (.createHmac "sha1" key)
    (.update base)
    (.digest "base64")))

(defn- mkoauth [key secret]
  (oauth.
    (clj->js
      {:consumer {:key key :secret secret}
       :signature_method "HMAC-SHA1"
       :hash_function oauth-hash})))

(defn- oauth-headers [odata {:keys [key secret] :as token}]
  (if-let [oauth (and key secret (mkoauth key secret))]
    (.toHeader oauth
      (.authorize oauth (clj->js odata) (clj->js token))
    (throw (js/Error. "Unable to create oauth-1.0a client."))))

While it’s working ClojureScript, it took 3 of us over 3 hours to learn why the API provider we’re writing against kept giving us “You don’t have permission to see view this page” errors. It turns out that the docs for the oauth1.0a library specify a token as the second arg to the oauth.authorize function. Unfortunately, the API provider doesn’t like that token arg. We then found that the docs of the oauth-1.0a library casually mentions that the token arg is sometimes optional for APIs.

Now I know that a big part of that is on us for not catching that. However, I can see how types would have helped us catch that but I’m curious what tools or processes we could apply in Clojure(Script) to avoid mistakes like that in the future. Having ran into that issue, I can understand where that contact’s coming from a bit better.

1 Like
8

@borkdude Come to think of it, you likely work on a lot of projects without as much prior art on. Do you encounter any of those issues my contact described in the top post?

9

@eccentric_j I’m interpreting your question as follows: do I have similar issues with Clojure as the person you describe, when working on new tooling?

I do see where he/she is coming from: dirty REPL state, slow startup, lack of static typing. But the alternative might be giving up on interactive development, which is a style I very much like about Clojure. It’s easy to capture some values that flow through a function in development, inspect it, make some changes, and build stuff out that way. No distracting write/compile/run cycles. Subjectively, I feel more productive in a language like Clojure than say, Haskell or Scala. Clojure is a tangible language whereas other languages feel more like watching things from behind a window once you have compiled the program. I do like some of the aspects of these languages and the guarantees they can give you. But some of those perceived benefits can also get in your way.

The tools I’ve been writing recently (clj-kondo, babashka, jet) are mostly command line tools. From an application development perspective, they are not so different from other Clojure projects, if not easier to hold in your head.
Clj-kondo helps me prevent or recover from dirty REPL state in many cases. Babashka and GraalVM partially solve the startup problem, at least for a significant subset of Clojure.

4 Likes
10

If the argument is optional, I don’t see how the type system would have helped here. One argument and two arguments would both be accepted. And you’re talking about this in the context of JS which isn’t even as strongly typed as Clojure at runtime :slight_smile:

Type systems can’t prevent/detect semantic errors. Clojure Spec is more able to detect those sorts of things since it can be written to support any semantics of Clojure itself.

Going back to the person’s background: we’ve seen quite a few folks over the years, who come into Clojure, spend a few years being very active, producing libraries, etc, but then decide they don’t really like Clojure after all and move on. Such folks often spend a few years in different languages, looking for options they like better. I’ve done that with Groovy and Scala before arriving at Clojure so I don’t think this is particularly surprising/unusual – nor is it indicative of anything those languages could do differently (and still be those languages).

I’m a bit puzzled by this assertion so I’d like to drill into your reasoning behind it. I don’t believe I’m using different tools here than anyone building non-web apps (except that I’m not using ClojureScript, which would seem even more web-related). Can you expand on your thinking here?

1 Like
11

Great way to put it.

4 Likes
12

You’re probably not wrong but I was under the impression a good type system would require you to mark which args are optional or you would be creating a function signature with multiple definitions for the different arity definitions like you do in Clojure. I recognize it doesn’t completely solve the problem but it could give you a quick, direct reference to look at to understand what’s required or not. However, I don’t know much about static typing so I could be wrong here.

Sure thing! For the dirty REPL state, I don’t mean specifically the tools we use to send code to the REPL and receive results in our editor. I mean the app-specific issues resulting from evaling code that causes the program and subsequent, sometimes unrelated parts of the app to stop working. The times where you’re evaling some code and it just doesn’t seem to be working no matter what you do. Finally, you restart the REPL then it works correctly. It doesn’t happen to me that often, but I have observed it from time to time.

My thought is that if you want to build a web-app you can search “How to build a web-app in Clojure” and find numerous frameworks, videos, articles, discussions, all kinds of great content and resources. Say you want to write a new ClojureScript transpiler or a new type of tool or project that doesn’t exist yet in Clojure where there isn’t as much mindshare on how to do it. Like “How do you write a babashka?” may not yield many results if at all.

So my hypothesis is that if you’re working on a web app, the frameworks, supporting libs, and tools are well developed to work really well with REPL driven development so you’re less likely to encounter those dirty REPL state issues that require restarts whereas treading new territory you’re more likely to hit dirty REPL states as you figure things out and could be a more prevalent concern.

@vlaaad If you care to share your experience: Have you experienced any of those issues mentioned in the top post while working on cljfx?

You’re right, it’s not uncommon for people to get excited by a language, take it as far as they can, then find interest in something else. I hope this discussion doesn’t come off as too reactionary in the sense that we need to change everything because one person doesn’t enjoy it as much as before. This was my first time talking to someone who had worked with Clojure extensively then migrated elsewhere, while I understand it’s subjective, I have encountered some of these issues so I was curious if people have also experienced them and what they did about it.

13

I don’t see how that’s relevant here. You’d still be able to call it with one argument or with two arguments so your call would still be valid. See, for example, this post about optional parameters in Scala: Optional Parameters in Scala – Anil G Kurian Am I misunderstanding what you’re trying to say?

the frameworks, supporting libs, and tools are well developed to work really well with REPL driven development

I think it’s much more about working practices than about tools (a la “bad workman always blames his tools”). I see people getting into all sorts of trouble working in the web application space with Clojure and when you drill into it, it seems to be about how they are working – and web development is the most common use case for Clojure (according to the State of Clojure survey results, it’s been #1 for a decade). I’ll be interested to hear what others say about working in more uncharted territory.

14

I am wondering if a solid type system would have helped us learn that the second argument to that function was optional or if it’s lack of types in a type system would have made me more curious earlier on to catch the issue earlier. There are likely other ways to accomplish this, for instance writing a spec for the oauth-1.0a lib may have helped us learn it more too which could have prevented it as well. Either way, it’s not fun hunting for a bug for 3 hours only to find we were providing an extra argument. But maybe there’s only so much that can be automated\systematic about that?

There’s at least been a few cycles of desired working practices influencing the tool design influencing even better working practices in the web development domain than a lot of others. All I can think to do is to listen to those working in more uncharted territory and see if the theory holds.

15

Ah, because perhaps in your editor you would have seen code assist that indicated something like:

(.authorize oauth |
            oath.authorize( data : Object [, token : Token] )

where | indicates the cursor position and that second line is popup code assistance?

We get that with multiple arities in Clojure with most editors tho’, yes? The equivalent in multiple arities would be:

(defn authorize oauth
  ([this data] ...)
  ([this data token] ...))

and code assist based on that would clearly indicate that token was optional?

JavaScript is particularly nasty in that regard because you can write a function that appears to take no arguments but then dynamically looks at what has been passed in and plucks arguments out as it wants. I’ve worked with other languages that do that too and, yes, that can make it hard to figure out what arguments to pass if the documentation is poor (which it sounds like it was for this library).

16

Hum, well that makes it more confusing to me :yum: I don’t really know how to frame those issues, they seem to indicate a lack of understanding of how to leverage the REPL effectively and how to leverage Spec effectively. My only instinct is maybe they come from a Lisp background, from that background, you can feel like the Clojure REPL (due to Java limitations) is restrictive and the slow startups are legitimate compared to CL.

All I can say personally is that, I don’t have any of these issues, but I did have some of them at first. Got over all of them.

For me, the REPL was an instant: “Oh my God this is amazing!”. I can’t relate to people that don’t have this reaction to be honest.

Being able to run your code as you write it, immediately see the result, change it re-run live! This idea of live programming, I don’t know for me it is transformative.

I honestly don’t understand how other languages don’t try and move there. How is it not the future of programming? When do people go: I want things to be less interactive, less immersive?

I tried Clojure and immediately had more fun. The REPL was half the reason for it. But the fun kept going, Clojure has every single toy you might dream of, packaged up in a way that you can use those toys for work! How cool is that!

Anyways, I digress.

There are many things still I think the community could improve on. When it comes to the language itself honestly, I can’t even think of anything missing or that needs major improvements or modifications, I find it close to perfect. I let it to Rich Hickey to surprise me again with something else I never knew I needed in my life, but can’t live without after learning about it.

So tooling, guides, tutorials could definitely improve, and are improving actively as well!

Let’s get back to the REPL. What kind of weird state can it actually get in? I think documenting this could be a good start.

What can’t be unloaded and reloaded basically? I actually don’t know, I thought everything can be. Maybe with the exception of Java dependencies.

17

Well, that code you wrote, it’s not even Clojure, you’re basically just doing JavaScript with a Clojure syntax :stuck_out_tongue_closed_eyes:

All that this oauth library needed to do is validate their arguments and return appropriate errors.

Static Types sometimes help and sometimes don’t. It’s hard to say in your situation.

I emphasized static, because in your case, there also isn’t any runtime type checking in place, and there is no runtime data validation in place. Both of those things would have made you catch this bug going from 3h to less then 1min.

Static type on top of that would have maybe gone from 1min to catch the bug to 1 second.

Here’s how I normally deal with this:

  1. Editor auto-completion, it doesn’t show you the types, but it’ll list out the possible arities and the argument names. I refer to that a lot as I code, I look what are the names of the args to know what needs to go in which position. The names should have been enough here, if it just said: authorize(auth, token, data)

  2. When it comes to interop, because auto-complete often isn’t reliable, and listing of arities and argument names even less, I always look up the doc to see what are the arguments and their order.

Doing that, I can’t remember the last time I had a bad case of: “used the wrong argument in the wrong place”

Other aspect to consider, always have to see the flip side. You wasted 3 hours, how much hours you saved using Clojure though?

Another aspect, this did not cause a production defect it seems. So we still don’t have an example of lack of static types resulting in more faulty software.

  1. I would have normally tried calling oauth-headers in the REPL, I try calling every function after writing them, I don’t mock anything either, my REPL is always connected to a real Devo environment. Don’t know if that could have helped in your case, but what does oauth-headers return when authorize is called with token and data reversed? Nothing that looks weird to the eye?

Beyond that, I do think we can blame the oauth lib a bit in your case for having poor validation and poor errors. Or for using positional arguments that overlap and mixed with optionals, maybe they should have gone for named parameters or a better design in the API.

Finally, I want to say that I don’t find the statically typed languages I used prior (Scala, C#, C++, Java) to be better than Clojure, I find myself less productive in them and often have even harder to debug issues, and I get the feeling the code often ends up much more difficult to understand and modify, with more defects making it through production. Not to mention so boring in comparison :stuck_out_tongue_closed_eyes: That said, I do think there might be an opportunity here for Clojure to do better.

In your case, it was a JS lib. But let’s say authorize was a Clojure function from some Clojure lib. I’d like to see where a little bit of typing, and a little bit of static schemas could take us. Maybe there’s a way to reduce these issues even more without much compromise.

18

Oh yeah, we logged, and logged, and logged the loggers attempting to find out why it wouldn’t work. Postman = worked
Ruby example from API Provider = worked
JS example devs sent us = worked
Our codebase NOT WORKING

We outputted the oauth-headers function directly in the REPL as well, nothing looked off as that data is factored into the signing. Oh! Maybe that’s why this issue was extra annoying? There was no direct output to check against. A hashed string looks like a hashed string whether it’s hashed correctly or not.

That’s a very fair counter-point. I think it will pay off well for growing it over time since mechanically the project is pretty simple. I think the process of writing a spec for the authorize function could have forced us to learn the API better than take the docs at face value.

That’s definitely what I’m on about here. Does it require new tooling, or applying the tools we already have better? Or is it a matter of processes?

1 Like
19

Exactly! It could have helped a bit to recognize or at least question if we needed to add it, but there could be other ways to get there as well.

Agreed, multiple arities of Clojure defs works but the issue was that it was a JS lib vs a clojure function so I don’t think the tooling will report their signatures?

20

I have to say, I like that you brought up a concrete example. I think this is the first thing, we need to see first hand real issues, not just impressions. Bringing concrete examples allows us to think of concrete solutions to specific problems.

I think for every concrete issue, we should also try and have some understanding of the weight it has. How big a deal is it? Do you face this issue once per 100kLOC? Or once every 100loc? Does it happen to you once a year, everyday? And how painful was it? 3 hours of debugging like you mentioned, but could be days, could be minutes? And what was the impact? Production? Or was it just that it got pushed to tests or QA?

Okay, now second thing, analyze the actual issue. I still think I’m not fully capturing the problem you faced. After re-reading, it sounds like authorize is a function of 1 arg, but with an optional arg that comes first?

authorize(optional token, data)

Is that it?