[WIP] A visual blocks engine for Clojure -- need your input!

Inspired by Parinfer and Paredit: embrace or avoid?, I whipped up a really basic frame-based code viewer for Clojure/ClojureScript.

You can see it live at http://lincoln-b.com/blocks.

All it does right now is turn code into a basic box representation like this:

03 PM

Obviously, quite a bit of information is lost, and that’s where I need some help. What do you think is the best way to represent semantic blocks of code using colored boxes without losing any data?

Currently regular forms are blue, vectors are red, maps are green, and everything else is black. I’ve toyed with different border decorations (dotted, squiggly, double), as well as box-shadows and labels, although including the labels sort of defeats the purpose.

11 Likes

Since I can only put two links in a post, I couldn’t include the GitHub repo, but here it is: https://github.com/lincoln-b/blocks

Nice start! Take a look at Shaun Lebron’s Inspiring a future Clojure editor with forgotten Lisp UX if you haven’t seen it already:

It has some ideas about ways to maintain semantics without using colour alone and losing the brackets – the “filling” approach is one way:

Glad someone started prototyping something already! I think you found the boundaries of the problem here pretty quickly. Here’s a full screenshot overview of your prototype for reference:

I think your question on what to do next is hard to answer, but maybe some context on what we’re dealing with exactly will help.

Here we go!

delimiter types

A frame is obviously associated with delimiters. So if we look, Clojure(script) has three types of them—but each with their own set of possible prefixes:

  • () lists
  • [] vectors
    • #queue [] queues
    • #js [] js arrays
  • {} maps
    • ^{} metadata
    • #{} sets
    • #js {} js objects
    • #:{} namespace maps
    • #::{} alias-namespace maps

Further still, most can also be prefixed with quoting characters too:

(And I only include the following for completeness, should we want frames for strings or unreadables that can still be printed)

  • "" strings
    • #"" regex
  • #<> unreadable

Part of the difficulty here is that Clojure syntax may continue growing, since it is a superset of edn afterall (e for extensible), particularly for tagged literals like #js, #queue, #inst, #uuid.

frame construction

If we construct frames from what we know in the previous section, it seems feasible to start by determining a style for the three main delimiters—as you have done. Then to deal with all the prefixes, maybe insert them as labels, at least at first.

I think that reduces the problem of constructing frames to choosing styles for each:

  • () box
  • [] box
  • {} box
  • prefix label

how to style

Some methods:

  • Frame colors might be used for distinguishing () [] {}, unless it clashes with colors used for possible depth-shading or syntax-highlighting. For example, Greenfoot elides syntax colors and shades frame backgrounds instead.
  • Frame corners might be shaped according to () [] {}—round, square, curly?
  • Frame labels might be shown as a bubble somewhere on the frame.

Some sketches using corners and labels, but not colors:

how to style nested borders

Showing code in Frames makes code more visual, but tracing lines to connect parentheses into boxes might be more overwhelming than the parens themselves—if done without some attention to aesthetic, I mean.

To avoid overwhelming, I think nested border styles are usually controlled using two variables:

  • border presence: which borders do you show, and when?
  • border padding: should borders be padded or collapsed, and when?

Some examples below to make this concrete.

examples

  1. @jiyinyiyong’s Cirru editor only draws some of the borders on each frame according to some heuristic:

  2. Matthew Chadwick’s editor prototype shows all borders by virtue of its background shading, and has slight padding:

  3. A more nuanced example is Antonin Hildebrand’s prototype “Plastic” (below). If you can make out the subtle shading borders, here’s how I see them:

    • borders drawn for some forms—(defn), (fn), (let), #(condp)
    • no borders for others—(if), (log-render), vectors [], maps {}
    • used for subtle left border on every textual line inside a multiline delimited form

    What’s also interesting is that frames are still present for every multiline delimited form, as evidenced by the dangling placement of some close-parens ) } ] near where the borders would be drawn if visible:

  4. Jack Rusher sent me a quick screenshot of a concept he was exploring. There’s a dotted left border for list forms, and full borders for vectors and maps—which I think may be distinguished by context of the form, though I’m not sure:

  5. Nassi–Shneiderman diagrams from 1973, I was shown recently. The triangle at the top denotes a condition that can branch to any form below it, and flow descends vertically from there. Notice that all borders are collapsed, except for the DO block showing padding on left to denote a LOOP:

  6. I did some experiments around adding SVG borders as an annotation layer for a text editor. It drew frames that would collapse at the bottom and right sides

    As we saw with Jack’s example, we can really just show the left borders, and that gives us the standard “Indent Guides”:

  7. To bring up Greenfoot again, they have a notion of pinning the head of the top frame as you scroll through it. For non-top-level nested frames, sideways annotations are used instead:

conclusions

There’s a lot that we can borrow from to build a Frame-based Clojure editor. Greenfoot is a good start, but there’s some extra problems to solve that are Clojure-specific—and some people have already started laying ground work here.

I haven’t been able to read through this recent journal yet, but that might help here too.

To answer your actual question, maybe using the “prefix bubbles” on the frame would be a good next thing to try.

19 Likes

I made Cirru Editor. Time for shout out my video again :blush:

1 Like

There are also some screenshots of old explorations of Cirru Project, despite that it’s written in Chinese: https://segmentfault.com/a/1190000009737250

2 Likes

I hadn’t seen that @nick, thanks for the link. Lotta good stuff in there, leans toward the practical side too.

Holy crap @shaunlebron looks like you’ve done your research.

Looks like I got all those delimiter types except queues, (alias) namespace maps, and JS stuff. Here’s my list:

;   "(" ")"     form
;   "[" "]"     vector
;   "{" "}"     map
;   "'(" ")"    quoted form
;   "#{" "}"    hash set
;   "`(" ")"    syntax quoted form
;   "^{" "}"    metadata map
;   "#(" ")"    anonymous function
;   "#?(" ")"   reader conditional
;   "'#(" ")"   quoted anonymous function
;   "`#(" ")"   syntax quoted anonymous function
;   "#[email protected](" ")"  splicing reader conditional

I got that from https://clojure.org/reference/reader, but I purposely ignored namespace map syntax because I don’t understand it. Relatively new clojurist still.

I now realize that colors shouldn’t be used to distinguish between block types – it’s too imprecise. Color is better for indentation a la LispWorks or Greenfoot. Frame corner styling is a good idea but I worry that it might be too subtle.

I like the prefix bubble idea. I was toying with labels above the boxes but they made the things a bit too unwieldy.

And I like @jiyinyiyong’s use of the frame cursor and border displays, that seems like a good way to do it.

Speaking to the earlier point of distinguishing the three main frame types, I suppose you could use left-right borders for forms, bottom borders for vectors, and maybe top-bottom dotted borders for maps. That would make the meaning more immediately transparent AND remove visual noise. Huh.

Thanks for your feedback, that’s by far the most useful input I’ve gotten so far. I didn’t realize so much work had already been done in this area.

2 Likes

I like the idea.
The green box for maps is a bit difficult to differentiate when they’re boxed inside a list which is blue
When the two lines are close together it is really hard to distinguish

I’ve pasted in some pretty hairy code and came to a realization that this is a great tool for getting a handle on a new codebase or revisiting a codebase you haven’t touched in a while.

It really lets you get a birds eyeview of how the code is structured.

I’m interested to see a discussion about what would be more helpful; a standard scheme or scheme customization
Standard scheme advantage is that after a while you can quickly scan some code and quickly understand the structure, the types being used, and scopes.

Scheme Customization, or rather, not having a set in stone way of colorizing data structures means that if you get sick of the color scheme you can just change it without having to create an RFC and bump the version. But if you have to learn a new color scheme over and over again, you lose the speed of understanding.

I’m very excited about this.

I’m thinking that one day it would be cool to embed this in Marginalia so when someone generates documentation for their code they can have the code portion blockified

1 Like

Color is a tricky visual dimension — in most situations it’s easier for people to quickly interpret relationships encoded in distance and proximity.

Unfortunately, spatial dimensions aren’t always free to use for encoding — everything on this thread sofar except the Nassi–Shneiderman diagram seems to be retaining the same spatial dimensions as “plaintext” code.

Cartographers (map makers) also have fixed spatial data, so they’ve studied questions of color extensively.
Their work should be helpful for folks designing editors.

Cynthia Brewer’s research is particularly accessible in an online color picker that helps you choose colors with respect to the data you are trying to communicate:

  • categories (maps, vectors, sets)
  • sequential or diverging relationships (e.g. depth of forms from toplevel or perhaps depth “up” or “down” from current cursor position)
  • resilance under black and white printing or common forms of color blindness
5 Likes

I’ve recently been experimenting with Spec for building UIs. The idea is to recurse though your data structure and its Spec, using the spec to inform the UI where possible (and if not, use type). You don’t always need a full spec, you can use (s/* any?) or something for unspec’d structure, but there you do have a spec it helps. For example, if you have a map representing a distribution {:mu 0.5 :sigma 0.1} you could render that as 2 sliders. But if you have the Spec for it, you could render it as an SVG graphic with controls to directly manipulate the distribution.
You can also do a kind of visual ellipsis where your spec says a data structure is a repeated pattern (to outline what the rest of the data would look like) – I mean, as you can imagine there are loads of possibilities.
The code I’ve written is hideous so I don’t want to release it yet, but it’s enough to know it can be done.

1 Like

More thoughts: I’ve recently read Where Mathematics Comes From (Lakoff & Nuñez) and it struck me that Clojure code perfectly embodies two of the fundamental metaphors described: the container metaphor (as (in (like this))) and the trajectory metaphor, (-> as in like this)!
Readers of this book will realize all kinds of possibilities for the display of code.

1 Like

I’ve been thinking about a design most visually similar to Matthew Chadwick’s, leveraging transparency and nesting, and the idea of the current screen being represented by a “context” node in the AST, and visualizes navigation up and down the context tree with zooming. Also a small + (conj) button at the bottom of the currently-focused node.

I think the most important part of any Clojure editor is that it offer low-friction self-modification (Emacs). This allows for quick visual prototyping in the course of regular programming, the same way that a hackable text editor allows encourages the accumulation of invisible customizations. Once that starts to get steam, the textual syntax underneath will disappear as far as many are concerned and many forms of programming will go fully-visual fairly quickly.

I’ve also been working on a generic interpreter for exactly such an editor (warning, shameless plug): https://github.com/notduncansmith/factfold

Basically, it’s a way to sneak Lisp onto other platforms, by structuring code in such a way as to get most of the correctness and succinctness of Clojure from other programming environments. An editor for this structure works near-identically as a specialized Clojure editor, but can have just about any language at the bottom, as the interpreter is straightforward to port to most platforms.

EDIT: Also in the vein of reading recommendations, I wanted to mention the fantastic “Notation as a Tool of Thought” paper by Ken Iverson.

4 Likes

Great ideas! Would love to see a better alternative to rainbow parens (colourblind :frowning: )

Here’s some more inspiration. It’s Shbobo’s Fish, a drag and drop sexp editor for Shlisp (a DSL for computer music). :slight_smile:

4 Likes

Hey, nice discussion thread you have going here.

Just wanted to add some comments to the Plastic screenshot:

  1. it uses Crockford’s context coloring
  2. visible frames are only for forms which introduce a new lexical scope (new color)
  3. is not used as a border but it is a post-new-line token, the editor is structural but it does not auto-format the code, so these tokens must be there to encode line-breaks for formatting/saving. For some conceptual reasons it is at the beginning of a line instead of at the end.

I explored some other things like having visible frames at each level, but it looked too “busy” for real-world code:

6 Likes

I would really like to see more and weirder variations on programming notation. Sadly, I’m as hemmed into text-y presentations as everyone else, having started with Lisp on dumb terminals in the 80s.

One idea I’ve been meaning to chase for awhile is the introduction of nested style sheets defined as metadata on each macro/function. The idea here is that not all forms want the same formatting, so it should be possible to hint the editor that cond, for example, wants a vertically positioned series of pairs. If the presentation metadata were combined with something like spec, the editor could also provide constraints/hints on legal input as well.

1 Like

Lots of interesting ideas and research here, I thought I would add some mention of blocks as visual aids to understanding and experiencing live code.

Dave Griffith’s did some work on the use of visual scheme closure blocks to visualise the live building and running of a musical performance. Pulsing with beats, clock pulses and activity. The project Scheme Bricks:

http://www.pawfal.org/dave/index.cgi?Projects/Scheme%20Bricks

Once you have the visual tools in place to aid development, new ways of communicating the liveliness of a system become possible…

4 Likes

Scheme blocks are one of my favorites from an aesthetic perspective. :slight_smile:

1 Like