Challenging Myself With Coplien's Why Most Unit Testing is Waste
Perhaps the most important lesson is this: Don't blindly accept fads, myths, authorities and "established truths." Question everything, collect experience, judge for yourself. As J. Coplien himself writes:
Be skeptical of yourself: measure, prove, retry. Be skeptical of me for heaven’s sake.
I am currently fond of unit testing so my mission is now to critically confront Coplien's ideas and my own preconceptions with practical experience on my next projects.
I would suggest that the main thing you take away isn't "minimize unit testing" but rather "value thinking, focus on system testing, employ code reviews and other QA measures."
I'll list my main take-aways first and go into detail later on:
Continue reading →
Running JavaScript Tests On a CI Server With Karma, Chrome And Fake X
In code:
Continue reading →
Focus & Do the Simplest Thing Possible
Credit: Dennis Jarvis, CCAre you tired of days spent in front of the screen, with no results to show? Have you once again engaged in yak shaving? Today, after having failed previously, I have finally managed to solve a problem while avoiding this trap by following rigorously two guidelines preached by grandmaster programmers. Be warned: Following this approach, you will get a working solution - but you won't like it. It will be ugly, stained by compromises, far from the elegant solution you wish for. But if your resources are limited and you want to avoid death by too many yaks, this is your only option. But first, what are these guidelines?
One: Maintain a laser-sharp focus. A great programmer is constantly aware of what she is trying to achieve and never strays far from it. If the path leads away, she backs up. If something else pops up, she writes it down for later and gets back to the job. This is essentially about deciding what not to do. (Many thanks to Kent Beck for sharing his focus secret!)
Continue reading →
Notes On Automated Acceptance Testing (from the Continuous Delivery book)
These are my rather extensive notes from reading the chapter 8 on Automated Acceptance Testing in the Continuous Delivery bible by Humble and Farley. There is plenty of very good advice that I just had to take record of. Acceptance testing is an exciting and important subject. Why should you care about it? Because:
We know from experience that without excellent automated acceptance test coverage, one of three things happens: Either a lot of time is spent trying to find and fix bugs at the end of the process when you thought you were done, or you spend a great deal of time and money on manual acceptance and regression testing, or you end up releasing poor-quality software.
Continue reading →
Continuous Delivery Digest: Ch.9 Testing Non-Functional Requirements
Digest of chapter 9 of the Continuous Delivery bible by Humble and Farley. See also the digest of ch 8: Automated Acceptance Testing.
("cross-functional" might be better as they too are crucial for functionality)
- f.ex. security, usability, maintainability, auditability, configurability but especially capacity, throughput, performance
- performance = time to process 1 transaction (tx); throughput = #tx/a period; capacity = max throughput we can handle under a given load while maintaining acceptable response times.
- NFRs determine the architecture so we must define them early; all (ops, devs, testers, customer) should meet to estimate their impact (it costs to increase any of them and often they go contrary to each other, e.g. security and performance)
- das appropriate, you can either create specific stories for NFRs (e.g. capacity; => easier to prioritize explicitely) or/and add them as requirements to feature stories
- if poorly analyzed then they constrain thinking, lead to overdesign and inappropriate optimization
- only ever optimize based on facts, i.e. realistic measurements (if you don't know it: developers are really terrible at guessing the source of performance problems)
A strategy to address capacity problems:
Continue reading →
The blog's year 2014 in review
Continue reading →
Connect Tunnelblick to VPN automatically after wake up
Partial solution: Make sure VPN is always running with "connect when computer starts" and using an AppleScript to connect after waking up from sleep. Disconnect manually when at work.
Future: Check the current location (wifi name? IP?) and do not connect when at work.
Continue reading →
Book Review & Digest: Capital In The Twenty-First Century
Capital in the Twenty-First Century by Thomas Piketty is together with Thinking, Fast and Slow, the most important book on society I've ever read. And together with Rawls' A Theory of Justice it has shaped my opinions on society and justice. All politicians and good people interested in politics and inequality should study it. It is unique in leveraging over 200 hundred years of data from different countries and using those as the base of the discussion of the evolution, laws, and future of capital and (in)justice in its distribution.
Key points:
- Growth cannot be 4-5% forever. In the long term, 1-1.5% is more realstic (and still far more than in the past). Higher growth is typical of countries catching up to more advanced economies (such as Europe to US, UK after WW2, China to the West nowadays).
- Return on capital, typically 4-5% (before taxes), is thus far higher than the growth of economy and salaries. The result is that the rich get ever richer, taking ever more of the national income. (Consequently, the poorer half has ever less of it.)
- To keep democracy and have a stable society, this has to be adressed politically.
Continue reading →
Tiny, Tiny Steps - Experience Report Developing A Feature In Minimal Value-Adding Increments
@lukew: Start with the simplest version you can. It's much easier to add complexity than to remove it.
Once upon time, there was a webshop portal with hundreds of partner webshops displayed on the front page. Potential users wanted to find out if their favorite webshops or a particular type of goods were available, existing users wanted to find a shop quickly. Therefore it was decided to implement search. But how to do that?
Continue reading →
Notes From CodeMesh 2014
Continue reading →
Most interesting links of July '14
Recommended Readings
- Video: The Unreasonable Effectiveness of Dynamic Typing for Practical Programs - a static-typing zealot turned friend of dynamic typing under the experience of real-world projects and problems shares thoughts about the limits of type systems (f.ex. both energy and torque are measured in N*m yet cannot be combined) and their cost: according to the Hanenberg's experiment about static and dynamic typing => the time required to handle the time chacker > time to debug the errors that it would have caught. According to a review of issues at GitHub, only 2% of reported issues for JS, Clojure, Python, and Ruby are type errors and for a large, closed-source Python project type/name/attribute errors were 1%. "I have come to believe that tests are a much better investment [than static typing]." Rigorous type system/model => limited applicability (due to different needs) <=> modelling some things with types doesn't cut it. "Are the costs of static typing offset by a few percent fewer defects? Is agility more important than reliability?" "Static types are anti-modular" - too a tight coupling. "Static type checking comes at the expense of complexity, brittleness and a tendency to monoliths." (Personally I miss static typing - but that is perhaps due to having relied on it for so long.)
- ThoughtWorks Tech Radar July 2014 (pdf): f.ex. Ansible in Adapt, Masterless Chef/Puppet in Trial, Machine image as a build artifact: Trial, PostgreSQL for NoSQL: Trial, Adopt Dropwizard (Rest 4 Java), Go lang, Reactive Extensions across langs [JH: RxJava, RxJS, ..]; Asses Property-based (generative) testing, ... . Other highlights: Mapbox (open-source mapping platform), OpenID Connect as a less complex and thus promising alternative to SAML/generic OAuth, Pacto/Pact for Consumer-Driven Contracts (contract => simulate consumers/stubb producers => test your REST clients against the contract so that the rest of tests can assume it is correct and use a stubbed client), Swagger for REST documentation.
- The madness of layered architecture - a nice critique of over-designed "enterprise" apps, why that is a problem (SRP, cost of code, unclear where to do a change, ....), why it is different from the successful layered network stack of Ethernet/IP/TCP/... (because in an app, all layers are on the same level of abstraction); bottom line: do not add a layer unless you have a really good reason (hint: the advice of a consultant/speaker does not count as one)
- Key Takeaway Points and Lessons Learned from QCon New York 2014 (viz @RiczWest) - "[..] deep insights into real-world architectures and state of the art software development practices, from a practioner’s perspective." - architectures of Fb, Foursquare etc., continuous delivery, creating culture, real world functional programming, ... .
- Questioning the Lambda Architecture (J. Kreps of LinkedIn) - maintaining the same processing in two very different systems (one batch, one stream & real-time) is a maintenance nightmare => improve the RT/stream processing to handle re-processing and thus both (using e.g. Kafka to store the data and thus be able to re-play them)
- Google: Checklist for mobile website improvement
- Google Dataflow and the transition from batch to stream processing - G. Dataflow might not be a Hadoop killer due to requiring that the data are in the Google Cloud but the trend is clear, going away from batch processing to more stream-oriented processing with tools like Spark, Flume etc. that are faster thanks to using memory better and more flexible thanks to not being limited to the rigitd two-stage model of map-reduce. (Reportedly, Google - the one that made Map-Reduce popular - doesn't use it anymore.)
- OS X: Extract JDK to folder, without running installer
Continue reading →
Most interesting links of June '14
Recommended Readings
- The emperor’s new clothes were built with Node.js - I know sadly little about Node.js but this goes against the hype and is thus interesting. So what does Node.js give us? Performance 1-5x slower than Java [like Clojure] according to the Benchmarks Game (contrary to other benchmarks with the opposite result as mentioned in the comments), use of a single CPU/core on our multi-cpu, multi-core machines, callback hell. At the same time, there are good non-blocking servers available in other languages (Clojure's http-kit, Vert.x, etc.) (Update: From the comments it seems that f.ex. the "callback hell" situation is geting better with 0.11, fibers and other things I do not know anything about. Also Sandro has a nice anti-comment (No. 36).) The Node.js Is Bad Ass Rock Star Tech 5 min video is a nice companion :)
- The Expert (Short Comedy Sketch) (7 min) - you've certainly seen this one but I had to put it here; a young engineer is hammered into being an "Of course I can do it, I am an expert" 'expert/consultant' during a business meeting. Maybe you too have experienced a dialog with the business where your true expert opinion was crushed by the business people's insistence on their absurd requirements?
- Reset The Net - Privacy Pack - privacy-enhancing apps for PC/mobile
- The Dyslexic Programmer (via Kent Beck) - interesting read about quite a different way to percieve and think about code, the advantages of IDEs.
- It’s Here: Docker 1.0 => more stable from now on
- Kent Beck: Learning About TDD: The Purpose of #isTDDDead - what is the purpose and value of TDD? Where are the limits of its value? "I recognize that TDD loses value as tests take longer to run, as the number of possible faults per test failure increases, as tests become coupled to the implementation, and as tests lose fidelity with the production environment."
- Failure & Cake: A Guide to Spotify’s Psychology of Success - want to be innovative and successfull? Learn to embrace failure, nurture the "growth mindset" (failure as opportunity to improve) rather than the "fixed mindset" (I do not learn and every failure shows I have no value). Read this if you want your org to be a better place to work!
Continue reading →
Review: Clojure for Machine Learning (Ch 1-3)
Pack Publishing has asked me to review their new book, Clojure for Machine Learning (4/2014) by Akhil Wali. Interested both in Clojure and M.L., I have taken the challenge and want to share my impressions from the first chapters. Regarding my qualification, I am a medium-experienced Clojure developer and have briefly encountered some M.L. (regression etc. for quantitive sociological research and neural networks) at the university a decade ago, together with the related, now mostly forgotten, math such as matrices and derivation.In short, the book provides a good bird-eye view of the intersection of Clojure and Machine Learning, useful for people coming from both sides. It introduces a number of important methods and shows how to implement/use them in Clojure but does not - and cannot - provide deep understanding. If you are new to M.L. and really like to understand things like me, you want to get a proper textbook(s) to learn more about the methods and the math behind them and read it in parallel. If you know M.L. but are relatively new to Clojure, you want to skip all the M.L. parts you know and study the code examples and the tools used in them. To read it, you need only elementary knowledge of Clojure and need to be comfortable with math (if you haven't worked with matrices, statistics, or derivation and equations scare you, you will have a hard time with some of the methods). You will learn how to implement some M.L. methods using Clojure - but without deep understanding and without knowledge of their limitations and issues and without a good overview of alternatives and the ability to pick the best one for a particular case.
Continue reading →
Most interesting links of May '14
Recommended Readings
- Monolith - from The Codeless Code - fables and koans for the SW engineer - the Monad monolth #Haskell #fun
- http2 explained (pdf, 27 pages) - cons of http 1 (huge spec / no full impl., wasteful use of TCP <=> latency [x spriting, inlining, concatenation, sharding]) => make it less latency sensitive, fix pipelining (issue a req before previous one finished), stop the need for ever increasing # connections, remove/reduce optional parts of http. Http2 is binary; multiple "streams" over 1 connection => much less conns, faster data delivery; header/data compression; [predictive] resource pushing; . Inspired by SPDY. Chrome and Mozilla will only support it over TLS, yay! (see also Is TLS Fast Yet? [yes, it is]) Promise: faster, more responsive web pages & deprecation of http/1 workarounds => simplified web dev.
Continue reading →
Fixing clojurescript.test failing with "ReferenceError: Can't find variable: cemerick"
cemerick.cljs.test) may fail with this confusing exception:ReferenceError: Can't find variable: cemerickdue to couple of reasons:
- Your test namespaces do not require
cemerick.cljs.test(and thus it is missing from the compiled .js; requiring macros is not enough) - cljsbuild has not included any of your test files (due to wrong setup etc.; this is essentially another form of #1)
- You are trying to test with the node runner but have built with
:optimizations:noneor:whitespace(for node you need to concatenate everything into a single file, which only happens if you use:simpleor:advancedoptimizations)
Example failures from all the runners:
Continue reading →