Holy Dev Newsletter December 2023

👓 Cosmograph: Visualize big networks within seconds | Cosmograph [tool, library, data visualization]
A GPU-powered web app and React library to visualize huge network graph data and machine learning embeddings, to gain insights into complex relationships and patterns that are often challenging to discern. Leverage components such as Timeline, Histogram, Search, and more. Free for non-commercial use.

👓 Tracebit | Use decoys to detect and respond to cloud intrusions in minutes [security, saas]
Not sure how well it works in practice and the cost/benefit ratio, but it sounds very interesting. I'd want such "honeypots" in my infrastructure.

👓 Why is Jepsen Written in Clojure? [clojure, opinion]
Why is the database testing tool Jepsen written in Clojure? Because of: Great support for concurrency (JVM + immutable data, atoms, etc.), clients for all DBs thanks to JVM, REPL great for interactive explorations, best library for transforming data, very stable language. Cons: smaller community, no broadly accepted static type system.

👓 deadeyejoe/fabrikk: An entity generation library for Clojure [clojure, testing, library]
A wonderful small library by a colleague for generating (and optionally persisting) graphs of test data. Its awesome, brief yet knowledge-packed and highly digestible tutorial will give you a very good idea about everything it can do. In short: create inter-related entities, where some values may be derived from other values or generated by no-arg functions, persist them to a database, and propagate DB-generated values (e.g. IDs) to all effected test data. Easily build your test data stepwise in exactly the shape and quantities you need, mixing randomness and control as you want, overriding defaults where necessary. Check out Why Fabrikk? for better understanding and alternatives.

👓 ont-app/graph-log: A logging utility built on the IGraph protocol [clojure, library, troubleshooting]
An interesting idea: log a message + values with Timbre but also add each log entry, with arbitrary key-value entries, to an in-memory graph, which can be queried and explored via RDF triplets and everything IGraph offers.

👓 pangloss/pattern: Pattern lets you transform data structures in amazing ways. [clojure, library]
Transform data structures based on powerful patterns and substitutions. Based upon the core ideas described in the excellent book Software Design for Flexibilty by Hanson and Sussman.
See the Nov 2022 talk for London Clojurians.

👓 Simple sabotage for software [productivity, process]
An amusing but also sobering tips for sabotaging ones company as any kind of technical or product leader. Are you sure your company is not doing one of these things? (Thanks, Erik!)

👓 The Third Web [future, criticism, web3] - (This text is very long. Maybe too long. You can find a PDF and an EPUB of it below.The current version of this text will always live at htt
A conscise introduction to blockchain, NFTs, and Web3, and an eloquent argument for why they are bad solutions. Written by a programmer, and once expert for the German Bundestag (parliament) on the topic of Blockchains and their value and regulation.
Web3 is an unclear concept, but often includes decentralisation, leveraging blockchains.Explains blockchains (and Proof of Work, Proof of Stake), NFTs, DAOs (“decentralized autonomous organizations”, basically “basically smart contracts with a mission”) . The hard problem of blocchains: get consensus in a decentralised network to solve conflicts. Web3 criticism:

• bad engineering - blockchains neither perform (e.g. 5, 30 transactions / second, compared to Visa’s 2000, with support for 24k) nor scale
• a security disaster - no consumer protection, if you lose a token (or it gets stolen, defrauded), it is lost
• NFTs don’t really do what they claim to do; you own a thing that says you own a thing.
• The Oracle Problem: from within a system you cannot determine the truth of statements about the outside of that system. Thus a blockchain can’t function on its own for anything in the real world (such as ownership of a physical object), you’d need “oracles” you trust for that => there goes the no authority/decentralization approach.
• Climate cost
• Web3 doesn’t deliver - economics and societal pressure will produce centralisation (e.g. there is already only few exchanges for tokens)

👓 Exploring flamegraphs [performance, tool, learning]
A tiny intro into what flame graphs are, then we dig into interpreting a real one, filtering, sorting, slicing, etc. Highly useful, especially if you use clj-async-profiler. We learn why we fight want to use sorting by width or the reversed view, how to remove irrelevant data, collapse recursive calls, and re-merge unnecessarily split code paths (in this case, by lazy-seq), to get a clearer picture.

👓 Differential flamegraphs [performance, tool, learning]
Learn to compare profiling results in two flamegraphs by using differential flamegraphs (from the two f.g.’s .txt files)

📜 Programming as Theory Building, an essay by Peter Naur

The wonderful 1985 (yes still very relevant) essay Programming as Theory Building argues that a program is a shared mental construct (a “theory”) that lives in the minds of the people who work on it. If you lose the people, you lose the program. The code is merely a written representation of the program, and it’s lossy, so you can’t reconstruct a program from its code.
I.e. programming is about achieving a certain kind of insight, a theory, of the matters at hand - not about artifacts. It is about building knowledge. Source code + all available documentation can never capture this knowledge fully, as demonstrated by a few experiences from the real world. The programmers’ knowledge should be regarded as a theory, which is something that characterizes an intellectual, knowledge-based activity,: “[..] a person who has or possesses a theory in this sense knows how to do certain things and in addition can support the actual doing with explanations, justifications, and answers to queries [..]”. A key factor is that the person is able to apply the knowledge to novel situations. The theory includes mapping between the program and aspects of the real world, knowing why the program is the way it is, and ability to extend the program to meet new needs based on understanding it and similarities between the new and existing needs. This can’t be reduced to a set of rules. If we adopt the Theory Building View, as opposed to programming as text editing, then we have no ground to expect a program modification to be cheap. Building-in flexibility is not an answer, because it is itself very costly, and relies on the (severely lacking) ability to predict what future modifications will be needed. You need a person with a live insight into the program, to find similarities between existing and the newly requested capability and determine the best way to add it. “For a program to retain its quality it is mandatory that each modification is firmly grounded in the theory of it.” The theory of a program, is something that could not conceivably be expressed, but is inextricably bound to human beings.
Building a theory to fit and support an existing program text is a difficult, frustrating, and time consuming activity. You have a better chance of success if you build a new program, which will not cost more, and likely less.
Programmer education should also focus on building their ability to formulate theories [in this sense].

👓 A Management Maturity Model for Performance - Infrequently Noted [performance, management, best practices]
Delivering acceptable performance is not a technical problem per se — it's a management issue, and one that teams can conquer with the right frame of mind and support. Performance is about reducing latency and variance across interactions in a session, with a particular focus on the tail of the distribution (P75+). Bad performance => lost users and thus revenue, reduced engagement, etc.
Teams progress through a hierarchy of performance management practice phases:
Level 0: fully unaware.
Level 1 (firefighting, starting to track): pick and balance (all) the right metrics => look at industry standard (e.g. Web Vitals) & get advice. Build a strong model of the user [needs] and understanding of your systems to know what matters most.
Level 2: Global Baselines & Metrics - found objective, industry-standard metrics / reference points that correlate with their business success. A sense of shared ownership over performance, performance work framed in terms of business value. Continual reporting against these standard metrics. Do strive to uncover what matters most to you, not to drown in metrics.
Level 3: P75+, Site-specific Baselines & Metrics - realise that global metrics/values can’t fully fit your product’s UX and find what really matters for you. Begin to map key user journeys and track the influence of performance across the full conversion funnel => add custom, relevant metrics. Percentile thinking: median (P50) isn’t the most important, P75, P90, and P95 are. Histograms are key. Teams at Level 3 begin to understand their distributions are nonparametric, and they adopt more appropriate comparisons in response. Enable slicing the data by percentile, geography, device type, etc. Integrate metrics with experimentation frameworks to track the effect of new changes. Build a lab for accurate measurements, correlate lab and production metrics. Management support for consistent performance.
Level 4: Variance Control & Regression Prevention - realise the impact that variance has on UX and start managing the tail latency (i.e. P75+). Automated tests check the performance of PRs and block those that impacts user flows badly <_> must understand which flows and scenarios are worth the effort (cost x benefit of developing and running them). Watch for slow, cumulative performance degradation, watch trends over longer time. Start with latency budgeting and attribute slowness to product features. Start "performance team", or a group of experts whose job it is to run investigations and drive infrastructure to better inform inquiry.
Level 5: Strategic Performance - fully institutionalise performance management and come to understand it as a strategic asset, and it becomes a part of the culture. “Teams that reach top-level performance have management support at the highest level. Those managers assume engineers want to do a good job but have the wrong incentives and constraints, and it isn't the line engineer's job to define success — it's the job of management.”
The article closes with a great list of Questions for Senior Managers, including “Is there a shared understanding in the leadership team that slowness costs money/conversions/engagement/customer-success?”, and about the constraints and support given to teams.

👓 An API that returns the requested number or random people data, with a name, email, address, coordinates, and more. Supports... [tool, testing]
An API that returns the requested number or random people data, with a name, email, address, coordinates, and more. Supports various query params, such as results=1000&seed=123.

👓 Transforming Data With Malli and Meander - Metosin [clojure, data processing]
“Transforming data programmatically is great, but we don't have to stop there. We can describe also the data models and data transformations as data and write an interpreter or compiler for it” - with Malli (schema def, data coercion) and Meander (a great library for creating transparent data transformations). A neat idea to convert between source and target data formats in a declarative way, with schema validation. Meander does the conversion based on a pattern -> expression you manually write (with any custom inline functions, so it is not purely declarative), while Malli validates and coerces the in/output data.