Holy Dev Newsletter July 2022

👓 datafold/data-diff at console.dev [tool, database]
data-diff splits the table into smaller segments, then checksums each segment in both databases. When the checksums for a segment aren't equal, it will further divide that segment into yet smaller segments, checksumming those until it gets to the differing row(s). A CLI and Python library.

👓 BrianChevalier/radiant: Write full featured CSS as Clojure data structures, inline [clojure, library, clojurescript, webdev]
Similar to the older Garden, Radiant enables you to write CSS as clojure data structures and generates css classes from these. Built-in support for Hiccup and Reagent and DIY using the API. Programatically create and reuse styles similar to tailwindcss with merge. Radiant aims to be easily replaceable and not be coupled to your code. It also tries to stay as close to vanilla CSS as possible with as little Radiant-specific learning as possible.

👓 ‎ClojureStream Podcast: E72 Data Processing with Chris Nuernberger (part 1) on Apple Podcasts [clojure, library, data processing] - ‎Show ClojureStream Podcast, Ep E72 Data Processing with Chris Nuernberger (part 1) - Jun 7, 2022
I really enjoyed this deep dive into tech.ml.dataset and the underlying dtype-next and learning about efficient computations on the JVM and via external, native libraries (such as those from Python or Julia). It turns out you can thing about these is essentially a high-performance, in-memory, column-oriented database. (There is also Part 2.)

👓 nubank/matcher-combinators: Library for creating matcher combinator to compare nested data structures [clojure, library, testing]
Flexible matchers for testing, supporting both primitive values (incl. regexp vs. string) and collections. Use value or a predicate, order-less comparison, maps only check specified keys. It looks really useful.

In-process libs for a processing pipeline as a DAG of actions

From Clojurians Slack, key contributions by Adrian Smith, respatilized, Ben Sless.
Ben Sless' more.async/dataflow for composing simple core.async processes, inspired by Onyx.
Bob CI (uses XTDB for persistence)
Onyx - written with distributed computation in mind but as far as I know nothing prevents running it entirely in memory on a single node. Designed exactly for the type of dynamic workflow definition you describe. The docs are extremely thoughtfully written, describing exactly why an information model is superior to an API
Dativity - a 'stateless process engine' that allows DAG-like computation by dispatching functions on the basis of what the data looks like at a given point in time. Not sure how dynamic it can be at runtime, though.
Daguerreo- explicitly patterned after Apache Airflow but designed to run in-process.
Matrix ("Fine-grained, transparent data flow between generative objects.") - it's kind of at the boundary between reactive programming and DAG computation.
Overseer - (see above description, basically - but stores job state in Datomic)
Titanoboa (though not in-process)
Note: Dynamic manipulation of the DAG and full state inspection are generally missing in these.
An article by respatialized on the topic:  Organizing Computation With Finite Schema Machines, where he outlines his thinking about how to solve a similar problem using malli schemas without pulling in a library like one of those.

👓 Bun is a fast all-in-one JavaScript runtime [webdev, tool, javascript]
A would-be replacement for Node.js and a competitor to Deno, build in Zig with focus on performance and fast start (aiming to run in edge functions) and being an all-in-one tool, with built-in transpiling and bundling (so TypeScript and JSX work out of the box). It implements 90% of Node.js API (contrary to Deno), making it a viable drop-in replacement. Built on JavaScriptCore rather than V8. It will be interesting to see where this goes.

👓 ykskb/phrag: GraphQL API created from an RDBMS connection for instant, flexible and customizable CRUD operations. [clojure, library, webdev, graphql]
GraphQL API from an RDBMS Connection (leveraging the DB's metadata), with focus on simplicity. CRUD: tables and/or views become queryable as root objects including nested objects of n-ary relationships with aggregation, filter, sorting and pagination supported. Mutations (create, update and delete) are also created per table. Customizable processing via interceptors - configured per table & operation type and at pre/post DB operations. Tries to create a single SQL joining multiple tables when necessary. It comes with a demo project you can explore. Currently supports SQLite and PostgreSQL.

👓 About Calico [devops, cloud, networking]
Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. From Kubernetes to bare metal. It provides developers and cluster operators with a consistent experience and set of capabilities whether running in public cloud or on-prem, on a single node, or across a multi-thousand node cluster.

Can I un-chunk lazy sequences to realize one element at a time?

No. If you need control over when elements are realized, you cannot use lazy sequences (which are chunked by default, realizing 32 elements at a time). You are better off using transducers.
Transducers allow us when each element is processed but what about the input sequence? Even clojure.core/sequence realizes the first argument prematurely from a pragmatic standpoint. So in practice you need a sequence impl that doesn't do that and which relies on a concrete type that specifically realizes one element per call to first/next, which in practice lazy-seq with a single cons call in it whose rest is a recursive call does this, but that relies on an implementation detail of lazy-seq so could more correctly be implemented with a reify.
(From a Slack discussion with contributions from Alex Miller, Joshua Suskalo, and others.)

👓 Tauri 1.0 Release | Tauri Apps [library, rust, gui]
Tauri, the high-security, cross-platform toolkit for building dekstop GUIs powered by Rust and web technologies has reached version 1.0. (A little like Electron but faster, smaller, leveraging the OS' native web view, and allowing you also to use Rust instead of JavaScript.)

👓 Brendonovich/prisma-client-rust: An autogenerated and fully type-safe database client for Prisma [rust, library, database]
Prisma Client Rust is an autogenerated query builder that provides simple and fully type-safe database access without macros or domain-specific languages. It is an alternative to ORMs like Diesel and SeaORM and tools like SQLx. Prisma Client Rust provides a much more consistent and understandable API than other ORMs, which typically hide a lot of code with macros
(SQLx is also interesting: async, pure Rust SQL crate featuring (optionally) compile-time checked queries without a DSL; also not an ORM.)

👓 The Babashka Toolbox [clojure, learning]
A categorised directory of libraries and tools for Babashka, with relevant entries tagged as pod or built-in. The ultime resource for powering up your bb scripts!

👓 WebAssembly Everywhere [wasm]
From WebAssembly Weekly #175:
This is a really good all-in-one primer on WbeAssembly. It covers what it is, and the features that have contributed to its success (simple runtime, portable, security). From there, it discusses various use cases, many of which are non-browser. There’s an interesting section on companies and projects that use WebAssembly, and why, and finally, what the future might look like. It is also very balanced, with a small section looking at competing technologies, and a section that discusses what WebAssembly is not good at.

👓 Majesticons premium icons for designers and developers [webdev, styling]
Majestic looking iconset with 720 free MIT License icons in two styles. If it is not enough then you can get 3200+ icons for $59.

👓 ReadySet | Same database, (much) faster queries [performance, database, tool]
ReadySet connects to an upstream Postgres or MySQL instance, automatically maintaining a cache for specific queries by listening to the database replication stream. Rather than forcing developers to switch to a key-value store or copy their database to a new provider for high performance, ReadySet adds an ultra-low latency cache to your existing database. Alpha.

👓 Sparqlify — Agile Knowledge Engineering and Semantic Web (AKSW) [tool, database]
SPARQL on top of a RDBMS (tested with PostgreSQL, H2), in development since 2011. Implements a subset of the SPARQL 1.0.

👓 JSON Viewer - JSON Hero [tool, webdev]
I haven't tried it but it looks as a nice tool for exploring potentially large json files. They're is also browser extension and a VS Code plugin. Open-source.

👓 Efficient partitioning – Inside Clojure [clojure, performance]
A fascinating deep-dive into details of Clojure regarding drop and partition[-all]. Key poinst: 1.12 brings partitionv[-all] and splitv-at that is faster thanks to the new IDrop interface and using take transducer (no intermeddiaries, leverages self-reducing collections) instead of the slower seq version.

👓 Announcing LocalStack 1.0 General Availability! - LocalStack [aws, tool]
Local emulation of numerous AWS services, including Lambda and API Gateway has reached version 1.0. It supports around 80 services distributed across the community and pro versions as well as a number of advanced team collaboration features in the new product tier.

👓 clj-fuzzy - Clojure [clojure, library]
A handy collection of algorithms dealing with fuzzy strings and phonetics. Reportedly simple and useful. Includes:
clj-fuzzy.metrics:Sorensen / Dice coefficient, Levenshtein distance, Hamming distance, Jaccard / Tanimoto distance, Jaro-Winkler distance, MRA Comparison, Tversky Index
clj-fuzzy.stemmers: Lancaster stemmer Lovins stemmer Porter stemmer Schinke stemmer
clj-fuzzy.phonetics: Metaphone Double Metaphone Soundex NYSIIS Caverphone Cologne Phonetic MRA codex

👓 Fleet Below Deck, Part III — State Management | The JetBrains Fleet Blog [architecture]
JetBrains, the company behind a number of most popular IDEs such as IntelliJ, share the design of their next generation IDE. Interestingly we can see a strong inspiration by Clojure and Datomic, including a link to a Rich Hickey's talk.
State consists of ordered immutable snapshots. Readers might get a slightly outdated but always consistent view of it (similar to the "db value" in Datomic). Internally the data is stored as subject-attribute-value triplets.
It also adopts reactivity (thanks to knowing which UI elements query for what data) and React-like approach of re-rendering only the minimal changed subset of the UI.
I am sure other parts of the series are also worth checking out.

👓 How to: Make your multilingual website suitable for RTL [webdev, styling] - Making a multilingual website? Here are 4 knowledgeable things to make your website suitable for RTL and LTR languages with just HTML and CS
Leverage CSS and html to the fullest to support right to left and top to bottom languages. Stop using properties such as top and right and use block/inline + start/end instead, as in padding-inline-end instead of padding-right. (Block start/end is top/bottom in English, inline start/end is left/right).