Datalevin powering environment and runtime?

This is quick (10 min) draft exploring the idea of storing code (what we usually store in files) in a database. Datalevin is a good fit because it natively stores EDN.

Why is this interesting?

  • Datalevin could be the backing store for an IDE
    • code analysis: functions bodies can be inspected for their dependencies (this could leverage clojure.spec) and hashed. Any time the spec/hash for a function body changes one can immediately run a figure out which other functions might now be broken / behave differently
  • using d/listen! change in data can trigger a query and execution of functions depending on that data – a reactive runtime.
  • ship it! Datalevin compiles to native using GraalVM. This can act as a fast, standalone backend for spreadsheet-like (relationship-heavy) apps.
(require '[datalevin.core :as d])
(require '[sci.core :as sci])

(def code-schema
  {:var {:db/unique    :db.unique/identity
         :db/valueType :db.type/symbol}})

(def code-conn (d/create-conn "data/datalevin/code-db" code-schema))

;; FIXME these functions could be defined and transacted by a defn-like macro
(def foo-fn
  {:var  `foo
   :body '(fn [a]
            ["I'm foo and here's my arg: " a])})

(def bar-fn
  {:var  `bar
   :body '(fn [] (conj ["I'm bar and I invoke foo: "] (invoke-fn `foo :arg-from-bar)))})

(defn transact-code! [txs]
  (d/transact! code-conn txs))

(transact-code! [foo-fn bar-fn])

(defn get-fn [var]
  (d/entity @code-conn [:var var]))

(defn invoke-fn [var & args]
  (when-let [{:keys [body]} (get-fn var)]
      (sci/init {:bindings {'invoke-fn invoke-fn}})
      `(~body [email protected]))))

(invoke-fn `bar)
;; => ["I'm bar and I invoke foo: " ["I'm foo and here's my arg: " :arg-from-bar]]

Have you heard of Codeq?

1 Like

Another thing that came to mind, is that in this context, time-travel might be a useful thing, and Datalevin doesn’t support it (but e.g., datahike does, and it has essentially the same API)

Yes I’ve heard of codeq and familiar with datahike, though can it store EDN?

I’m thinking in a slightly different direction. We all def, derive or read/write some data from files during development. Derived data disappears whenever we restart the session. Files don’t provide the right abstraction/granularity to view large amounts of data. Therefore, whenever we want to make something long-lived and interactive we throw our REPL away and start building UI’s and/or API’s for end users.

But what if we had a perpetual REPL session powered by a graph database? Something more akin to spreadsheet without a UI or where UI can be tacked on on demand?