Holy Dev Newsletter January 2023 & December 2022
Welcome to the Holy Dev newsletter, which brings you gems I found on the web, updates from my blog, and a few scattered thoughts. You can get the next one into your mailbox if you subscribe. This time it is a double issue because I obviously forgot to publish in December 😅.
I have long felt that software development is wrong. It takes too much effort to build and maintain stuff. Why??? I see three primary causes:
Lack of knowledge, in two areas:
(intrinsic) the problem itself - we do not understand it well enough and need to experiment and research; this is an intrinsic part of the job
(extrinsic) our tools - i.e. we do not understand enough the programming language and the technology we use; I believe/hope there must be much better way of preventing this waste
this is exacerbated by the lack of observability, i.e. our inability to see clearly what is happening in the system and our code; something Bret Victor talked a lot about
Language not fit to talk about the solution and problem - a topic I explore in a recent post
Complexity, both intrinsic (real world is a dirty beast) and extrinsic (related closely to #2) - I have mused about complexity in SW development before
If you have any thoughts on this topic, please let me know!
A lot! Primarily I have been pretty busy building a simple ERP system for a friend, built on Fulcro RAD, Asami, and running on Fly.io. A considerable fraction of the time was spent on writing fulcro-rad-asami v2 with support for Pathom 3 and transactional resetting of singular attributes, as well as on learning RAD and Pathom better and improving their code (e.g. a picker with support to create/edit entities) and docs. Why, you migt wonder? I feel that to support my current and future mentee clients better I need to maintain my hands-on experience. I like to help friends. And I wanted something that could have been turned into a teaching tool, showing people how to create a real app from 0 (though I am not sure I will figure out how to do this). It is also fun to be responsible for everything, from UI through security to operations and resillience.
In other news, in April I am going to the US for Conj, THE Clojure conference. I am excited about both the conference and meeting some online friends in the real life for the first time.
A mixture of SW dev, Clojure, and Fulcro stuff came out of my print press recently: Get insight into Pathom 3 in Fulcro with Pathom VizFirst craft a language then build your software - one of the most important things I’ve ever written, about the topic that is bothering me for years - why do we write so much code and spend so much time doing that. Though already Paul Graham has indicated the answer years ago and Bret Victor has things to say here. My year 2022 in reviewFulcro Lesson: Detached Root Component (Form)Clojure is in fact a trinity - of language, interactive development, and structural editing - a slightly controversial piece aimed at newcomers to Clojure and the approach to learning it, triggered by chats with my colleagues
A series on error handling providing good food for thought. Explores different kinds of errors, ways to represent them, what information you might want to include with an error, and finally how to work with them in the code (this installement). Something to ponder when you design error handling for your project!
Search across GitHub repos, filtering for a particular language etc. Useful to find examples of the usage of a library etc.
Task scheduling libs for Clojure
Libraries for scheduling of tasks and batch jobs that are Clojure friendly.
- Chime - "create the simplest possible scheduler" - "[it] is really just a promise to execute a function at a (possibly infinite) sequence of times" I suppose "done" but with occasional updates (including mine!)
- Tea-time (last updated 2019)
- at-at (last updated 2014) - built on ScheduledThreadPoolExecutors. Not smart enough to see that a task is already scheduled.
- cronut - a data-first Clojure wrapper for the popular, feature-rich Java Quartz Job Scheduler. The README describes how it differs from older wrappers such as Quartzite and Twarc.
- goose - "A Reliable [avoid data loss in face of HW/net/... failure], Pluggable [Message Broker, metrics backend] & Scalable background job processing library for Clojure."
Recursively compare Clojure or ClojureScript data structures, and produce a colorized diff of the result - intended for human consumption.
Bring datafy and nav to commonly used mechanism types. Currently covers some of Java (socket, thread stuff, ...), core.async (channels, buffers), and reify. For use with REBL, Portal, etc.
jlink is a JDK tool to create bespoke, minimal JREs for your applications. For a simple hello world example it reduces the JRE size from 140 to 30MB.
Though it might not be always so simple to figure out what modules are necessary - as Maarten explains in Creating the smallest JVM microservice deployment: "Unfortunately modularisation is not that popular because it’s hard to implement. The maintainers of Spring don’t think its worth implementing modularisation because of the costs. So, to use the jlink command, we have to manually add all the modules to the runtime, one by one, until our application runs."
Here is an article with a two stage Dockerfile build leveraging jlink to create a custom JRE (with all modules included by default, read it to find out how to limit that).
You could also consider using GraalVM to build a fast-starting, possibly smaller image. Gene Kim has done that for a Fulcro webapp (reducing startup time from 25-30s in an uberjar to 8 seconds) (code here).
Currently JVM-based lambdas running on the Correto JVM can reduce startup time by ~ 90% thanks to a memory & disk snapshot taken after initialization and used during subsequent starts.
Perhaps the most important text I have read this year. The author argues eloquently that “little languages” — small languages designed to solve very specific problems (think SQL or RegExp) — are the future of programming. Do we really need 30 million lines of code to implement an operating system, or is 20k enough? An example is Nile, a little language for computer graphics, that in 300 lines of code can do what Cairo needs 40 000 lines of general purpose programing language code. How does this apply to your software project? This way: "The idea is that as you start to find patterns in your application, you can encode them in a little language—this language would then allow you to express these patterns in a more compact manner than would be possible by other means of abstraction. Not only could this buck the trend of ever-growing applications, it would actually allow the code base to shrink during the course of development!"
"Benchmark" to compare GUI toolkits in terms of their notation. To that end, 7GUIs defines seven tasks that represent typical challenges in GUI programming.
A Ring<_->Jetty adapter for multiple versions of Jetty, even 5he newer one still lacking official Ring adapter.
Generate docs about the shape of function arguments by instrumenting them by the Flow Debugger and exercising them (typically by running the test suite). Produces an EDN and can display it in the debugger.
Great practice for more efficient, collaborative software coding. Key points: use a monorepo to prevent dependency hell & more, use WIPs, not feature branches, use feedback early and often.
WIPs are small, focused, frequently updated, short-lived branches, merged eventually as a single, squashed commit.
It should be possible to ask for feedback at any time, not only when the work is finished. Getting feedback on an initial design is most valuable. And we want to be able to get a quick feedback on a piece of code at any time.
A library for the generation of relational test data. A fork of Specmonstah that supports more schema providers than just Spec, has fixture lifecycle management, first-class support for inserting fixtures into databases, and is user friendlier.
Thank you for reading!