Include interactive Clojure/script code snippets in a web page with SCI & friends

September 26, 2023

Demo!
Gimme the code!
Building a custom CodeMirror 6 editor with Clojure support

1. code_editor.cljs

Requires
Code evaluation
Code evaluation “extension”
CodeMirror extensions and setup
Displaying the editor
Bonus: Output display panel
Complete source code

2. Building the custom editor

Tips for Cryogen & Asciidoctor users
Credits
TODO

I have long dreamt about having interactive code snippets of Fulcro in my teaching materials. Showing people code they could modify and see it render right next to it. Fulcro is a ClojureScript library, but it uses some heavy macros - and those typically require JVM Clojure. Well, not anymore. I was able to rewrite them into Borkdude’s Small Clojure Interpreter (SCI) dialect of Clojure. I.e. I can ask SCI to evaluate a piece of code with these macros, which SCI will macro-expand into more cljs, and execute. With SCI, my Fulcro sci.configs, CodeMirror, and Nextjournal’s clojure-mode, I can have a beautiful in-page editor with code evaluation. And I will show you how to do the same, for your blog.

I won’t spend much time on how to write SCI configs to expose a library. The core is that you need to tell it about all the namespaces and public vars that you want available in your SCI scripts, and you need to rewrite macros into functions annotated with ^:sci/macro. There are few more things, such as dealing with dynamic variables, and you can read all about it in SCI’s Readme. You might want to look at existing configs for Reagent and Fulcro for inspiration.

Demo!

I couldn’t possibly write a post about including an interactive code snippet without actually using it, could I? So here it is, my beloved Fulcro displaying a message to you:

In-browser editable Fulcro app

(ns test (:require
          [com.fulcrologic.fulcro.dom :as dom]
          [com.fulcrologic.fulcro.mutations :as m]
          [com.fulcrologic.fulcro.application :as app]
          [com.fulcrologic.fulcro.react.version18 :refer [with-react18]]
          [com.fulcrologic.fulcro.algorithms.normalize :as norm]
          [com.fulcrologic.fulcro.components :as comp :refer [defsc]]))

(defonce app (-> (app/fulcro-app) with-react18))

(defsc Counter [this {:ui/keys [n] :as props}]
  {:query [:ui/n :counter/id]
   :ident :counter/id
   :initial-state {:counter/id 1 :ui/n 1}}
  (dom/button {:onClick
               (fn [evt]
                 (m/set-integer! this :ui/n :value (inc n)))}
              (str "Likes: " n)))

(def ui-counter (comp/factory Counter))

(defsc Root [this {:keys [counter]}]
  {:query [{:counter (comp/get-query Counter)}]
   :initial-state {:counter {}}}
  (dom/div
   (dom/h3 "Hello from Fulcro!")
   (dom/p "The awesome framework for full-stack webapps")
   (ui-counter counter)))

(app/mount! app Root "demo-app") ; <1>

1	Mount it to the div just below the editor

Gimme the code!

If you want to skip reading all the text and just have a look at all the changes necessary to add support for live code snippets to my Cryogen-powered blog, go to the details of the commit range 'Interactive code-editor for Fulcro powered by SCI'.

Building a custom CodeMirror 6 editor with Clojure support

The in-browser Clojure editor requires some assembly:

Configure CodeMirror with all the extensions you want, such as history, line numbers, a key map to run actions on key combinations, etc. This could be done in JavaScript.
Build the clojure-mode extension. Currently, it is only available as a git repo with ClojureScript code, i.e. as a Clojure Deps dependency, though people are reportedly working on making it available as an npm library.
Create a SCI build with all the configs you want.
Put all the parts together, and use them in a web page, such as this blog post.

It may sound scary, but it is rather simple.

I only build the code editor JS manually, and keep it in git. No need to rebuild it whenever I write new content.

1. code_editor.cljs

Let’s look at the heart of it all, the code-editor ns, which combines 1. - 3. above to produce code-editor.js. This .js is the complete solution, with a custom CodeMirror build including Clojure support, and with SCI-powered evaluation.

Requires

To understand the code snippets later down, remember that clojure-mode is required as cm-clj, SCI as sci, and most other stuff is CodeMirror objects and functions.

Requires, complete listing

[sci.core :as sci]
[sci.configs.fulcro.fulcro :as fulcro-config]

["@codemirror/commands" :refer [history historyKeymap]]
["@codemirror/language" :refer [syntaxHighlighting defaultHighlightStyle]]
["@codemirror/state" :refer [EditorState]]
["@codemirror/view" :as view :refer [EditorView lineNumbers showPanel]]
[nextjournal.clojure-mode :as cm-clj]

Code evaluation

First, the evaluation itself:

Code evaluation with SCI

(defonce sci-ctx (doto (sci/init {:classes {'js js/globalThis :allow :all}}) ; (1)
                   (sci/merge-opts fulcro-config/config))) ; (2)

(defn eval-code [code]
  (try (sci/eval-string* sci-ctx code) ; (3)
       (catch :default e
         {::error (str (.-message e))})))

1	Allow calling `js/alert` etc.
2	The first key line - include the parts of sci.configs or your custom sci configs you want
3	The second and last key line - evaluate code with SCI

Code evaluation “extension”

Next, we want to trigger evaluation on a key press, and thus need to create a CodeMirror keymap extension:

Code eval extension

(defn eval-all [on-result x] ; (1)
  (on-result (some->> (.-doc (.-state x)) str eval-code))
  true)

(defn sci-extension [on-result]
  (.of view/keymap ; (2)
       #js [#js {:key "Mod-Enter" ; Cmd or Ctrl
                 :run (partial eval-all on-result)}]))

1	`x` is some CodeMirror object, which contains the editor’s text, and `on-result` is our callback, to communicate the result of the evaluation (to display it in an output “panel”). It just stores the value in an atom.
2	We are creating a keymap extension, to run code on a key press

CodeMirror extensions and setup

In the extensions below, we add some custom styling, support for history and syntax highlighting, line numbers, some keymaps, and support for clojure via cm-clj, i.e. the clojure-mode’s extensions.

Create CodeMirror instance with the desired extensions

;; "Static" extensions
(defonce extensions ; (1)
  #js[theme     ; optional, see below
      (history)
      (syntaxHighlighting defaultHighlightStyle)
      (view/drawSelection)
      (lineNumbers)
      (.. EditorState -allowMultipleSelections (of true))
      cm-clj/default-extensions
      (.of view/keymap cm-clj/complete-keymap)
      (.of view/keymap historyKeymap)])

(defn bind-editor! [el code] ; (2)
  (let [last-result (atom nil) ; (3)
        exts (.concat extensions ; (4)
                #js [(output-panel-extension last-result) ; optional
                     (sci-extension (partial reset! last-result))])]
    (new EditorView
         #js {:parent el
              :state (.create EditorState #js {:doc code
                                               :extensions exts})})))

1	We define “static” extensions, which we can reuse for multiple editor instances
2	`bind-editor` will insert the editor as a child of the given DOM element (the real bind-editor I use is little more feature-rich)
3	An atom we use to display evaluation results (see below)
4	We add a few “dynamic” extensions, which are unique to each editor instance, because they depend on the editor-specific result atom.

Displaying the editor

Let’s assume you have an element such as <div id="code1"><pre>…, containing the code you want to make editable and evaluable. We can replace it with the editor like this:

Display the editor

(let [el (js/document.getElementById "code1")
      target-el (js/document.createElement "div")
      code (-> (.getElementsByTagName el "pre") (.item 0) .-textContent)]
  (.replaceWith el target-el)
  (bind-editor! target-el code))

Bonus: Output display panel

CodeMirror has the concept of panel extensions you can add to the top or bottom of the editor. This is a good place to show the output of the evaluation, so let’s do that:

An output display panel extension

(defn output-panel-extension [result-atom] ; (1)
  (let [dom (js/document.createElement "div")]
    (add-watch result-atom :output-panel
               (fn [_ _ _ result]
                 (if (::error result)
                   (do
                     (.add (.-classList dom) "error")
                     (set! (.-textContent dom) (str "ERROR: " (::error result))))
                   (do
                     (.remove (.-classList dom) "error")
                     (set! (.-textContent dom) (str ";; => " (pr-str result)))))))
    (set! (.-className dom) "cm-output-panel")
    (.of showPanel (fn [_] #js {:dom dom}))))

(def theme ; (2)
  (.theme
   EditorView
   #js {".cm-output-panel.error" #js {:color "red"}}))

1	This fn builds our aforementioned output panel extension, displaying whatever appears in the result atom
2	Here we build the previously mentioned theme extension, essentially just defining a new CSS class, added to the default theme. We could well also do it in plain old CSS but this was more fun.

Complete source code

See my blog/code-editor/holyjak/code_editor.cljs.

2. Building the custom editor

Prerequisites: yarn install and correct deps.edn + package.json (explored later on)

shadow-cljs.edn

{:deps {:aliases [:code-editor]}
 ;; :dev-http {8118 {:root "themes/lotus", :push-state/index "html/dev-editor.html"}}
 ;; :nrepl {:port 9000}
 :builds {:code-editor {:compiler-options {:output-feature-set :es8
                                           :optimizations :advanced}
                        :target :browser
                        :output-dir "themes/lotus/js"
                        :asset-path "js"
                        :modules {:code-editor
                                  {:init-fn holyjak.code-editor/render}}}}}

With this configuration, I can run npx shadow-cljs -A:code-editor release code-editor to build ./themes/lotus/js/code-editor.js (which Cryogen, my blog generator, will copy so that it is available as /js/code-editor.js). The render function will be called when the code is loaded, and will detect and replace all relevant pieces of code with the interactive editor.

The commented-out lines make it possible to run npx shadow-cljs -A:code-editor watch code-editor and access my dev-editor.html at http://localhost:8118/ so that I can develop it interactively.

package.json

{  "devDependencies": {
    "@codemirror/autocomplete": "^6.0.2",
    "@codemirror/commands": "^6.0.0",
    "@codemirror/lang-markdown": "6.0.0",
    "@codemirror/language": "^6.1.0",
    "@codemirror/lint": "^6.0.0",
    "@codemirror/search": "^6.0.0",
    "@codemirror/state": "^6.0.1",
    "@codemirror/view": "^6.0.2",
    "@lezer/common": "^1.0.0",
    "@lezer/generator": "^1.0.0",
    "@lezer/highlight": "^1.0.0",
    "@lezer/lr": "^1.0.0",
    "@nextjournal/lezer-clojure": "1.0.0",

    "react": "18.2.0",
    "react-dom": "18.2.0",

    "shadow-cljs": "2.25.6"
  }
  ...}

I need to add all the (codemirror and lezer) npm dependencies of clojure-mode, plus React for my Fulcro app, and shadow-cljs itself.

deps.edn

{;...
 :aliases
 {:code-editor
  {:replace-paths ["code-editor"]
   :replace-deps
   {thheller/shadow-cljs {:mvn/version "2.25.6"}
    org.babashka/sci {:mvn/version "0.8.40"}
    io.github.babashka/sci.configs {:git/sha "bf9769c7b9797ac764f4f2fb48fbf342f78c0477"}
    io.github.nextjournal/clojure-mode {:git/sha  "7b911bf6feab0f67b60236036d124997627cbe5e"}
    com.fulcrologic/fulcro {:mvn/version "3.7.0-SNAPSHOT"}
    com.wsscode/pathom3 {:mvn/version "2023.01.31-alpha"}}}}}

Here too we need shadow (same version!), of course SCI and its configs, clojure-mode, and whatever dependencies my editable code snippets need.

Tips for Cryogen & Asciidoctor users

You might have noticed some Cryogen-specific things in the whole setup:

The compiled code-editor.js lives under theme/<name>/js/ and is copied to public at build time. It can’t be kept in public, because Cryogen wipes it out. Similarly, the dev-editor.html lives under the theme (though somehow it doesn’t make it into ./public; but it doesn’t need to be there anyway)
I already use deps.edn for the blog itself, so I made a dedicated alias that completely ignores and replaces the project’s paths and dependencies
I have modified my template’s base.html so that I can add :extra-js to the post’s preamble EDN and have it included in the page. I use it to load the code-editor.js for this post.
The code listing I want to bring alive must have [source,text,role="code-editor",subs="-callouts"]
1. source of course because it is a code listing
2. text as the language, so that highlight.js doesn’t mess up with it
3. role="code-editor" for code-editor.js to find it
4. subs="-callouts" to prevent Asciidoc from replacing callouts with fancy HTML, which would break the code (since I have :icons: font by default)

Have a look at code_editor.cljs to see how I find and replace the Asciidoctor-rendered code listing.

You may also want to have a look at the .asc source of this post.

Credits

I have heavily copied from clojure-mode/demo.cljs and am indebted to Borkdude for his awesome tools and invaluable help.

TODO

Add proper support for a dark theme, so that the code is actually readable in the evening. Fixed by 1153a1b.

Tags: Fulcro