Which libraries are in need of accessible API docs?


#1

I was looking at Enlive (see also Best library for querying HTML?), which I’ve used in the past and actually quite like, but then realized that I couldn’t find any up-to-date API docs anywhere. Certainly not linked from the project. There’s even a 4 year old issue about the lack of docs, with no resolution.

Apparently clojuredocs.org used to host API docs for libraries, but now it seems to only do core.

The thing is though, if you have your library on Github then it’s extremely easy to just host API docs youreslf, using codox and this script:

Show generate_docs script
#!/bin/bash

# Keep a separate branch of generated API docs.
#
# This script generates API documentation, commits it to a separate branch, and
# pushes it upstream. It does this without actually checking out the branch,
# using a separate working tree directory, so without any disruption to your
# current working tree. You can have local file modifications, but the git index
# (staging area) must be clean.

# The git remote to fetch and push to. Also used to find the parent commit.
TARGET_REMOTE="origin"

# Branch name to commit and push to
TARGET_BRANCH="gh-pages"

# Command that generates the API docs
DOC_CMD="lein with-profile +codox codox"

# Working tree directory. The output of $DOC_CMD must end up in this directory.
WORK_TREE="gh-pages"

if ! git diff-index --quiet --cached HEAD ; then
    echo "Git index isn't clean. Make sure you have no staged changes. (try 'git reset .')"
    exit
fi

git fetch $TARGET_REMOTE
rm -rf $WORK_TREE
mkdir -p $WORK_TREE

echo "Generating docs"
$DOC_CMD

echo "Adding file to git index"
git --work-tree=$WORK_TREE add -A

TREE=`git write-tree`
echo "Created git tree $TREE"

if git show-ref --quiet --verify "refs/remotes/${TARGET_REMOTE}/${TARGET_BRANCH}" ; then
    PARENT=`git rev-parse ${TARGET_REMOTE}/${TARGET_BRANCH}`
    echo "Creating commit with parent ${PARENT} ${TARGET_REMOTE}/${TARGET_BRANCH}"
    COMMIT=`git commit-tree -p $PARENT $TREE -m 'Updating docs'`
else
    echo "Creating first commit of the branch"
    COMMIT=`git commit-tree $TREE -m 'Updating docs'`
fi

echo "Commit $COMMIT"
echo "Pushing to $TARGET_BRANCH"

git reset .
git push $TARGET_REMOTE $COMMIT:refs/heads/$TARGET_BRANCH
git fetch
echo
git log -1 --stat $TARGET_REMOTE/$TARGET_BRANCH

Copy the script to the root of your project, and in your project.clj configure Codox (for boot just update the DOC_CMD in the script).

(defproject ,,,
  ,,,
  :profiles {:codox {:plugins [[lein-codox "0.10.3"]]
                     :codox {:project {:name "my-project"}
                             :doc-paths ["doc"]
                             :output-path "gh-pages"}}}
  )

That’s it. Now when you run ./generate_docs it will run codox, create a new commit on the gh-pages branch, and push it to Github. The script is designed to be super robust. It uses lower level Git operations so it doesn’t have to mess with your current branch, switch branches, or change the working tree. It really should just work. Your docs will be available at https://<username>.github.io/<projectname>. Don’t forget to link to them from your README.

You can add extra Markdown files which become part of the generated site, which is really great feature for adding e.g. a getting started guide or more reference material, and there are several themes available to adjust the look of Codox to your taste.

Here are some examples

I created a PR to add this to Enlive, I hope Christophe accepts it.

Now my question is: which other projects could really benefit from this? Who else should I send a PR to?


#2

A few that come to mind (disclaimer: some are my own :innocent:):


#3

I opened issues for tick and s3-deploy, and created a PR for derivatives.

With that there’s now also an example of using boot with generate_docs. It’s pretty straightforward still, just add codox in your build.boot, and update the DOC_CMD and WORK_TREE variables in the script like so

# Command that generates the API docs
DOC_CMD="boot codox -n derivatives -o gh-pages -s src target"

# Working tree directory. The output of $DOC_CMD must end up in this directory.
WORK_TREE="target/gh-pages"

#4

As part of adding the script to Derivatives Martin and I made several improvements, the current version is now here

Some improvements

  • You can specify some of the variables from the command line, instead of always having to change the script
  • The commit message on the gh-pages branch shows the state of your repo, so you know which code you actually generated docs for
  • if there is no change in output then it won’t create and push an empty commit, instead it shows a warning
  • Added a lot more comments since some of this stuff is pretty arcane

I’ve turned it into a Gist so I can push updates if needed. In the long run we’ll need a better way to distribute this so people can more easily upgrade. Maybe it should actually be a lein/boot package? Or something something npm?

Of course even better would be a public service that can generate and host these docs for multiple libraries across multiple versions, but I personally have enough going on right now to start on that :slight_smile:


#5

That’s awesome, thanks Arne! :heart:

So here are the brand-new derivatives docs. It’s interesting to see this come together in this form. Makes it quite a bit easier to see where documentation could be improved.

Of course even better would be a public service that can generate and host these docs for multiple libraries across multiple versions, but I personally have enough going on right now to start on that :slight_smile:

I think that’s a very cool idea. I also have a bunch of other things in my queue but I believe that this could become a thing. I’m sure if such project came to be hosting would be a solvable issue as well. So many cool things to do… :scream_cat:


#6

I just keep seeing more uses for this, like I want to have some org-mode files, and always have the latest HTML version on-line as well. So far I’ve been copying this script from repo to repo, but it keeps improving, so I finally gave it a proper home. I tweaked it a bit more, and now it’s called autodoc (monospace font for the name is part of the branding :-P)

With this you can simply put this in your repo

#!/bin/bash
export AUTODOC_CMD="lein codox"
\curl -sSL https://raw.githubusercontent.com/plexus/autodoc/master/autodoc.sh | bash

You like? I like!


#7

And… now it’s self-hosted. Converted the README to org-mode so I can convert it to HTML with emacs.

Result: https://plexus.github.io/autodoc/

Also made the output a bit nicer, added a version number and CHANGELOG, added some more checks so that if warns you if something seems out of whack, like the DOC command not creating any output.