I’m curious about the variety of methods used by developers for testing in [some of] your primary project[s], and would value your participation in a little survey. Please comment with any further explanation or questions you have!
What is the scale of your project (not counting libraries)?
100 or fewer fns
101-500 fns
501-1000 fns
1001+ fns
0voters
What flavor of platform are you targeting?
CLJ
CLJS
CLJC
Other
0voters
What forms of testing do you do?
TDD
Coverage testing
Unit Testing
Generative Testing
Property-based Testing
Other (comment below)
0voters
What testing frameworks/libraries do you leverage?
We use HtmlUnit to perform UAT-style end-to-end testing (as well as clojure.test, expectations.clojure.test, generative testing, and property-based testing).
In terms of scale, at work, we have 72,953 lines of source code and 21,182 lines of test code.
Good question. To be honest, I just included “property-based testing” because it’s a phrase I see a lot. In response to your question I looked at the following:
From these I gather that property-based testing originates with Haskell’s Quick Check system, and that Clojure’s generative testing via spec can be used to implement property-based checking. Reading online, it seems there isn’t a major difference.
There is a difference between property and generative.
Generative testing relates to the test input. Which is generated instead of hard-coded. While property based testing relates to the assertions made, which asserts properties instead of hard-coded outputs.
So if you assert things like that your input/output always follow certain patterns, like that they are of an expected range, type, commutative, etc. That’s property based testing. You can do that even if you have hard-coded inputs.
With generative testing, you will most likely need property based assertions as well. Since if the input is generated, you can’t really have an assertion on the output that is an exact equality for the given input. So you’ll most likely won’t be able to assert it produced the exact expected output, but that at least it followed some generally applicable properties as they are expected.
So often times, generative testing implies property based tested, but not the other way around.
Those were my intuitions as to the difference between the two, but I hadn’t managed to articulate it so well to myself. Thanks for an excellent explanation!
We also to Specification by Example / BDD leveraging the Java SbE testing framework Concordion via clj-concordion.
We have a ± smallish batch job of 180 functions with critical, money-related business functionality so testing the business rules and being able to have “living documentation” that shows the system is working correctly and that we can talk through with the business people is very important.
I think unit does not need to equal to a function.