Holy Dev Newsletter Sep 2023

👓 candera/causatum: A Clojure library for generating streams of events based on stochastic state machines. [clojure, library, testing]
A Clojure library for generating streams of timed events based on stochastic state machines - i.e. like a Finite State Automaton (FSM), but each possible transition has a probability (and also delay, which I read as "cost ~ time to perform the transition"). The event-stream then gives you a stream of [current state, time counter, info about transition taken (weight, delay)]. You could use this e.g. to model user's behavior on a webpage. It gives you more control than randomly generating actions with test.check.
(For test.check, there is also fsm-test-check, described in the post Verifying State Machine Behavior Using test.check).

👓 Tree-sitter｜Introduction [library, parsing]
"Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited." Intended, it seems, primarily for editors. Written in C++ but with Java and other bindings.

👓 xdrop/fuzzywuzzy: Java fuzzy string matching implementation of the well known Python's fuzzywuzzy algorithm. Fuzzy... [library, java, search]
No dependencies fuzzy string matching for java based on the FuzzyWuzzy Python algorithm, using a fast implementation of Levenshtein distance to calculate similarity between strings. Simple to use, lightweight.
Similar but heavier: clj-fuzzy (a native clj(s) implementation of a bunch of popular algorithms dealing with fuzzy strings and phonetics), org.apache.commons.text.similarity.

👓 API Hub - Free Public & Open Rest APIs | Rapid [SaaS, startup]
API providers (such as startups) can leverage RapidAPI to easily add auth*, billing, monitoring, test sandbox UI for users, etc. I have no experience with it but it could be useful to speed up building/testing you API-focused startup.

👓 carp-lang/Carp: A statically typed lisp, without a GC, for real-time applications. [programming languages, learning]
Carp - if Rust and Clojure had a baby :-) A Clojure-inspired, statically typed & compiled Lisp without garbage collection (but with ownership tracking instead). Interesting and already usable, though with a bunch of bugs. Marked as a research project and a warning not to use it for anything important just yet, v 0.5. Written in Haskell, started in 2016.

🎥 A great talk by Christian Johansen, a Clojurist and highly experienced frontend dev (one time author of the 📘 Test-Driven...
A great talk by Christian Johansen, a Clojurist and highly experienced frontend dev (one time author of the 📘 Test-Driven JavaScript Development) about framework-agnostic best practice for building maintainable web applications (and a little about his Portfolio tool). The main points? First, UI should be a "pure" function of data (what he calls data-driven UIs). I.e. components only get data from their parent - they do not fetch anything themselves with useEffect or a Re-frame subscription. Second, the UI is written in the language of the UI and does not use any domain terms - the incoming domain data passes first through a pure prepare function, which translates them into the UI domain. E.g. you don't have FacilitySelector with meter-name and street, but a Dropdown with a title and details - which you can reuse for displaying highly different kinds of data. The argument is that a frontend dev is in the domain of building UIs and not in the business domain of e.g. providing electricity to end consumers. And I trust Christian's experience on this point.

👓 Norris Numbers [opinion, productivity, software development]
A great post, posing the hypothesis that applications at different scale (LoC) need (radically) different approaches to write. With a particular approach, you will eventually hit a wall of how much complexity you can manage - e.g. ± 2000, 20k, 200k, ... . Just as running will eventually plateau and require you to switch to a car, then a plane, ... .

👓 Kitemaker [SaaS, project management]
"End-to-end product development from user feedback to shipping things people want." A new SaaS tool for managing project work. Contrary to Jira, it aims for the whole lifecycle, from specifying what to do, through individual work items, to user feedback. Having that in a single, integrated tool is attractive. I haven't tried it but would like to.

👓 Announcing HoneyEQL 1.0 - Tamizhvendan [clojure, library, database] - HoneyEQL is a Clojure library that enables you to query the database declaratively using the EDN Query Language(EQL). It aims to simplify th
This library looks interesting for when you use a RDBMS with next.jdbc and use joins to fetch an entity with sub-entities. With HoneyEQL, you specify an EQL query such as [:presentation/title {:presentation/slides [:slide/title ...]}} and HoneyEQL will generate the correct SQL with joins and then post-process the with group-by or something to turn the tables back to the tree you actually want: {:presentation/title "Clj Rocks", :presentation/slides [{:slide/title "Into"},{:slide/title "Conclusion"}]}.

I work in gaming, so I cannot speak to your specific experiences. Entity Component Systems are extremely performant, really good science, and shipping in middlewares like Unity. However, in order to ship an ECS game, in my experience, you have to have already made your whole game first in a normal approach, in order to have everything be fully specified sufficiently that you can correctly create an ECS implementation. In practice, this means ECS is used to make first person shooters, which have decades of well specified traditions and behavior, and V2 of simulators, like for Cities Skylines 2 and Two Point Campus. - doctorpangloss at Hacker News

👓 Stateless, data-driven UIs [opinion, webdev, productivity]
The article Stateless, data-driven UIs explores how to deal with the complexity of frontend development by separating UI from state management and event handling. Christian uses simple, generic components, and all the state management logic is in a pure, testable function. Event handling is also described with data, and thus testable. ❤️

👓 Polylith in a Nutshell - Polylith [clojure, tool, architecture]
A new and great introduction page for Polylith. Just remember that "software architecture" in this context refers to how your code is structured into modules (which differs from how it is deployed). Polylith is an approach + tooling for structuring your code into small, separate modules that can easily use each other, and be combined in somewhat arbitrary ways to produce deployable artifacts.

👓 Introducing runes [webdev, framework, javascript]
Interesting development in upcoming Svelte 5, introducing "runes, which unlock universal, fine-grained reactivity." Universal = not limited to .svelte files, more explicit (with $state(val) rather then (only) top-level let ...) and thus easier to control and understand, extending from compile-time detection of dependencies to also runtime. With the pre-5 approach "[..] code is hard to refactor, and understanding the intricacies of when Svelte chooses to update which values can become rather tricky beyond a certain level of complexity." All this is, under the hood, based on Signals, used e.g. by Knockout since 2010, and more recently by Solid. Reportedly, "Signals unlock fine-grained reactivity, meaning that (for example) changes to a value inside a large list needn't invalidate all the other members of the list."
Reading the list of syntax/features that becomes obsolete with runes, it sounds as a great simplification. And once again shows that explicit is better than unreliably magical.
There is also a 12 min intro video, if you prefer.

👓 OrbStack 1.0: Fast, light, easy way to run Docker containers and Linux [productivity, devops, tool]
OrbStack 1 is here! This is great, although paid for commercial use ($8/month), replacement for Docker Desktop. We run our dev stack (some 8-10 containers) via Docker, and it used to eat quite a bug chunk of my CPU regularly, under Docker Desktop. Since I switched to OrbStack few months back, I had not noticed any such issues. It brands itself as "the fast, light, and easy way to run Docker containers and Linux. Develop at lightspeed with our Docker Desktop alternative." You can read more on its what/ why page. And it can also run Kubernetes, though I haven't tried that.

👓 Pagefind | Pagefind — Static low-bandwidth search at scale [tool, authoring]
Pagefind is a single binary you run on your HTML files to produce an index and add a JS + CSS file, which enable you to search the content. I have used it on my blog's search to replace Custom Google search, with its ads. Pagefind is trivial to use - you can just npx -y pagefind --site and add their built-in search UI to any page with a few lines of code. Or build your own UI with the JS API. Remember, all this is static, and the search runs fully in the browser. Pagefind is reportedly efficient and can search even large sites, downloading chunks of the index as needed. You can even influence the indexing with few data- attributes, or they full control with their JS indexing API. You can check out the few changes I had to do to adopt Pagefind. Or go to my search and type e.g. "fulcro". (And of course it is written in Rust! It indexed my 575 pages / 25303 words in ± 5s.)

👓 Roadmap to Tauri 2.0 [library, GUI, Rust, javascript]
Tauri is the attractive, secure-by-default and far more efficient alternative to Electron for building cross-OS desktop apps. The key difference is that Tauri leverages OS's native WebView instead of bundling Chromium and Node, and its focus on security. You might want to check out this Electron vs Tauri comparison. The upcoming v2 brings support for creating also mobile apps, much more powerful plugin system (dogfooding FTW!), and support for Swift and Kotlin plugins.

👓 The State of Async Rust: Runtimes [rust, criticism]
An insightful article about the state of async in Rust, warts and all. Key point: Async is hard, especially multi-threaded, only use it if truly necessary. Often, multithreading without async has much better cost/benefit ratio. My highlights:
Tokio is the absolutely most used async runtime. Async-std, the would-be async replacement for stdio, is effectively abandoned. But Tokio is much more than a runtime, with its extra modules for fs, io, net, etc. That makes it more of a framework for asynchronous programming than just a runtime. The author's main concern with Tokio is that it makes a lot of assumptions about how async code should be written and where it runs. Quote:
The Original Sin of Rust async programming is making it multi-threaded by default. If premature optimization is the root of all evil, this is the mother of all premature optimizations, and it curses all your code with the unholy Send + 'static, or worse yet Send + Sync + 'static, which just kills all the joy of actually writing Rust. — Maciej Hirsz
Multi-threaded also means you need Arc / Mute for most state. The choice to use Arc or Mutex might be indicative of a design that hasn't fully embraced the ownership and borrowing principles that Rust emphasizes. It's worth reconsidering if the shared state is genuinely necessary or if there's an alternative design that could minimize or eliminate the need for shared mutable state.
Going beyond Tokio, several other runtimes deserve more attention. These runtimes are important, as they explore alternative paths or open up new use cases for async Rust:

• smol: A small async runtime, which is easy to understand.
• embassy: An async runtime for embedded systems.
• glommio: An async runtime for I/O-bound workloads, built on top of io_uring and using a thread-per-core model.

👓 Choosing a more optimal `String` type [rust, performance, experience]
Intriguing post from a Sentry Rust SDK maintainer, exploring alternatives to String, that have performance characteristics better suited to their use case, i.e.: immutable, copied often, small, Option-al, etc. There is a bunch of alternatives to choose from, such as Arc and smol_str. The author concludes that there is no free lunch, but smol_str with its O(1) clone, small string optimization, wrapping in Option adding nothing to its size, and its heavy use in rust-analyzer seems to be the best candidate for them.
A liked another observation there:

🎥 A good 30 min overview of key developments in Java since v7, starting with syntax improvements (try with resources, var with...
A good 30 min overview of key developments in Java since v7 (slides), starting with syntax improvements (try with resources, var with derived type, records, sealed interfaces [i.e. only fixed number impls], switch as an expression - possibly on records, etc.) and concluding with virtual threads, problems with go-like concurrency (which also applies to core.async) and avoiding them with "structured concurrency".

Handle conflicting files when uberjar-ing in Leiningen

Context: Groovy libraries may include /META-INF/groovy/org.codehaus.groovy.runtime.ExtensionModule to register some extensions to install into JDK. If you uberjar multiple such libraries, you will end up with just a single one of these (last wins). In our case, this is not acceptable and we want to filter out one we don't care about, and fail if there are still multiple. We use Leiningen, and (mis)use its :uberjar-merge-with for this. It takes a map of file pattern -> [fn1: stream->X, fn2: X+X->X, fn3: stream, X -> void].

:uberjar-merge-with
{#"^META-INF/groovy/org\.codehaus\.groovy\.runtime\.ExtensionModule$"
 [slurp
  concat
  (fn [out contents]
    (if (> (count contents) 1)
      (throw (ex-info "More than 1 groovy extensions in uberjar! (see :uberjar-merge-with in project.clj)"
                      {:cnt (count contents)
                       :modules (mapv #(first (clojure.string/split-lines %)) contents)}))
      (spit out (first contents))))]}
            

👓 Mock Service Worker [library, testing, javascript]
We use this at work in frontend tests, to mock the REST backend. The nice thing is that it intercepts requests on the network level, so it doesn't care about which library or API you use to issue http requests.

👓 anteoas/broch: A library for handling numbers with units. [clojure, library]
A library for handling numbers with units - conversion between compatible units, comparison and arithmetic, data literals. Ex: (b/> #broch/quantity[1 "km"] (b/meters 999)) ;=> true

👓 Understanding htmx [webdev, opinion, productivity, architecture]
Intro to HTMX, why Biff uses it, when it is not suitable (lot of complex state as in Google Sheets or superfast response, as in G. Maps).
Thomas Heller in his critique of HTMX argues (I believe) that you can relatively easily roll your own htmx, thus gaining full control and making sure it will 100% fit your unique needs, while with HTMX you will run into the design walls, since it is made for general use, without knowledge of your needs. Jacob and Thomas have a discussion about it here, where Thomas asserts that the flexibility of your own solution is worth the trade-offs (mentioning "HTMX itself seems a bit too limited for most stuff I do, which often involves some more interactivity than HTMX is capable of."). Jacob reasonably proposes that htmx is superior for people who are still getting up to speed with Clojure web dev and aren't already familiar with CLJS / JS. Though Thomas objects that learning htmx + likely hyperscript is nontrivial, and it is better to spend the time to learn cljs + DOM properly. In Jacob's own apps, he did not run into htmx's limitations (and uses Hyperscript to get more interactivity where needed.) You might also want to check out htmx's own When Should You [not] Use Hypermedia? (meaning htmx).

👓 Intro to Running LLMs Locally [learning, llm, ai]
What, how, and why of running LLMs (Large Language Models - think ChatGPT & friends) locally, from Clojure. Reportedly, many models are available to download and run locally even with modest hardware. Conclusion: LLMs really only have one basic operation (~ given a seq of tokens, predict probabilities of tokens coming next) which makes them easy to learn and easy to use. Having direct access to LLMs provides flexibility in cost, capability, and usage.
I only skimmed the article, it seems as something useful to have in hand for when I need it.