Holy Dev Newsletter Oct 2023

Welcome to the Holy Dev newsletter, which brings you gems I found on the web, updates from my blog, and a few scattered thoughts. You can get the next one into your mailbox if you subscribe.

What is happening

This has been a month of HTML and CSS tinkering. I have fixed my online Clojure & Fulcro editor to properly support dark mode, fixed headings to show the self-link icon only on hover, and added a submenu to the "Tumblr" menu item to search through its content (more on that below).

I have long desired the ability to search through my "gems from the web" links, and finally I have implemented it thanks to PageFind and a custom indexer. The index thus produced is copied to my site and searchable at /devlinks. Fun fact: currently, PageFind only supports custom indexers in JavaScript and not Rust (yet) 😭 but JS has weird syntax and misplaced parentheses 😹 so I have used Borkdude’s Squint to be able to write Clojure syntax that gets compiled to JS. (Originally, I hoped it had built-in EDN, which it doesn’t, and I had to pull in the edn-data lib, but I stuck with it anyway, because Clojure.) It took some effort to find how Clojury I could write the code and when to just use JS functions but it turned out fine in the end: build-pagefind-index.cljs. It takes under 5s to index my over 1k entries. Sweet. The search and UI could be improved but it is pretty decent, and so much better than grep-ing through the EDNs. I can search for a topic and use tags to narrow down to e.g. "clojure" and "library" and "webdev".

On the Fulcro side, I made a small fix to fulcro-rad-datomic to make it possible to get the db/id of entites. I extract the time portion of that to sort them by time when I back up orders to a Google Sheet so that newer entities are always added at the end (the thought being that I could eventually implement incremental backup, instead of wipe out & insert all).

I have also spent some time learning about and playing with Rama’s Clojure API, but far less than I wanted to.

On the other hand, I had fun time at work, where we finally managed to unify the two sources of data we use for dashboards, thus making it simpler and more consistent, and enabling us to delete a lot of code, and even more data. (Since the common abstraction enables us to store aggregated summaries in the DB, instead of large raw data files.) It has been in the making for a year, often blocked by urgent product work, and it feels great to be finally done with it.

Gems from the world wide web

👓 HyperLogLog - Wikipedia [algorithm, data]
HyperLogLog is an algorithm for the count-distinct problem, approximating the number of distinct elements in a multiset. Supported in PostgreSQL via the postgresql-hll, where it is written: "HyperLogLog is a fixed-size, set-like structure used for distinct value counting with tunable precision. For example, in 1280 bytes hll can estimate the count of tens of billions of distinct values with only a few percent error." This sounds very useful.

👓 Cilium - Cloud Native, eBPF-based Networking, Observability, and Security [tool, devops, networking]
"Cilium is an open source, cloud native solution for providing, securing, and observing network connectivity between workloads, fueled by the revolutionary Kernel technology eBPF (sandboxed, user-provided kernel extensions)" - i.e. a service mesh without sidecar, with a single proxy per node (instead of per container). Via ThoughtWorks Tech Radar.

👓 Devbox: Portable, Isolated Dev Environments [tool, productivity]
"Devbox is a terminal-based tool that provides an approachable interface for creating reproducible, per-project development environments, leveraging the Nix package manager without using virtual machines or containers." No knowledge of Nix required it seems; you can install a bunch of stuff via provided plugins. Via thoughtworks radar

👓 The Collaborative API Development Platform [testing, tool]
An alternative to Postman, "Insomnia is a free and open-source desktop application designed for API testing, development and debugging." They have also a SaaS for collaboration, but you can use the free dekstop tool locally.
Also interesting is HTTPie with its free terminal tool (the "simple yet powerful command-line HTTP and API testing client for the API era") and now also a desktop app. "Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers. The http & https commands allow for creating and sending arbitrary HTTP requests. They use simple and natural syntax and provide formatted and colorized output."

👓 remotemobprogramming/mob: Tool for smooth git handover. [tool, collaboration]
From thoughtworks/radar: "mob is a command-line tool for seamless git handover in remote pair or mob programming. It hides all the version control paraphernalia behind a command-line interface that makes participating in mob programming sessions simpler. [..] Several of our teams highly recommend mob, and it has become an integral part of our toolchain in remote pair or mob programming."

👓 Mocks Server [testing, tool, productivity]
From thoughtworks/radar: "Mocks Server is a Node.js-based API mocking tool valued by our teams for its ability to replicate intricate API responses, headers and status codes. The dynamic response generation supports the simulation of diverse scenarios, which allows for the rigorous testing of API interactions. Mocks can be described as YAML or JSON and managed through the CLI, REST API or JavaScript code. Mocks Server's features include request matching, proxying and record-playback features, which facilitate the emulation of realistic API interactions. [..] ts straightforward approach aligns with our emphasis on simplicity and efficiency in development processes."

👓 The Web After Tomorrow [webdev, opinion, future]
Interesting, deep thoughts about the state and future of web dev from 2015. Perhaps frontend can talk directly to (a future, security-aware) DB? We want: Consistent view of the data, instead of it being in varying stages of stalesness. Always fresh data. Effect of user actions should be displayed immediately. Handle network failures. Offline support (do local changes, merge in when online again). No low-level connection management, retries, deduplication.
The article further explores what would we need to make this possible, such as having queries that can both fetch the initial data and stream updates. We are getting to the domain of distributed databases, where browser plays the role of one of the peers. "For example, DB could provide a strongly consistent event log, and DB-client synchronization protocol can take advantage of that. Or we might choose eventual consistency, CRDTs and anti-entropy measures." Arguing for how Clojure, Datomic and DataScript could help here.

👓 MineREPL [clojure, learning]
Create interactive 3D world in your browser, while learning Clojure! A super cool idea. You have a character, and can create movable boxes with different size, color, and physics properties. The tutorial is still under construction.

👓 clj-async-profiler 1.1.0: right-click menu and startup profiling [performance, tool, clojure]
clj-async-tool is a super useful, low-overhead way to profile your code using flamegraphs. V1.1 comes with menus so that you can more easily filter and modify the flamegraph to zoom in on the problem. This article explains and demonstrates them. Awesome!

👓 iconmonstr - Free simple icons for your next project [authoring, webdev, visuals]
Reportedly they have 4784+ free icons in 316 collections. Simple, clean, black&white icons (the few I saw). E.g. the cloud icon has 34 variants (cloud, cloud of music, ...).

Huff is essentially Hiccup 3.0, with extensible syntax, a bunch of conveniences, better performance, babashka support.
👓 escherize/huff: Juicy hiccup in pure Clojure [clojure, library]

👓 nubank/nodely [clojure, library, architecture]
Similarly to the abandoned Plumbing Graph, nodely allows you to define a ± DAG of dependant computations and then execute the one you need, transitively executing its dependencies... Contrary to P.G., it has support for conditional dependencies and for "sequence nodes," i.e. map over a sequence. Nodely enables you to maximally decouple pure logic from effectful computation and to separate data dependency declaration from the data fetching execution model.
A nice thing is that it supports multiple execution engines, so that you can e.g. switch to a simpler one to debug a problem.
There is also a London Clojurians talk about Nodely, but it mostly just repeats the readme and tutorial, though at the end there is an example of replacing a Ring handler from side-effecting one to one that returns a Nodely graph with a :response key.

👓 Phosphor Icons [webdev, authoring, visuals, library]
Another free icon set, over 7k. These are more round and soft. You can pick color, size, and style (thin ... bold, filled, duotone). Looks nice. There is also a phosphor icons as Hiccup library.

👓 mhjort/trombi: Load testing library for testing stateful apps with Clojure [clojure, library, testing, performance]
A successor to clj-gatling for load testing stateful applications needing scripted logic in tests. You define scenarios with sequential steps, and may run multiple scenarios in parallel. Steps may pass data to subsequent ones. Configure the number of virtual users, their rate of arival... . Returns stats and may render graphs of these. (Trombi is Finnish for Tornado.) I've used Gatling and it was awesome.

👓 Synth [tool, testing] - Generate test data programmatically. Need fake data for application testing or anything else? Just use Synth.
Synth enables you to generate complex test data with interdependencies. You specify a schema as json in various collections, possibly import data from databases, and ask synth to generate. It leverages Faker providers to generate valid addresses, telephone numbers, etc.

Four Ways to Create High-Quality Connections at Work

Great tips here! The 4 ways are communicating respect, task enabling (ie. help them, remove obstacles,..), trusting (e.g. give others control over resources or decisions, share information openly), and "playing", i.e. creating fun or enjoying pleasant experiences (humor,..). But you can't really get these without reading the article.


Thank you for reading!

Tags: newsletter

Copyright © 2023 Jakub Holý
Powered by Cryogen
Theme by KingMob