The first Continuous Delivery and DevOps Conference in Oslo is over. It was nice to see so many people interested in the topic. I would have preferred more practical talks of the "how we did it" type over the "why" type but it was OK, though next year I would prefer flatMap. Here are my highlights:
One of the annoying things with Jest is that while it enables you to run only a single test by using it.only, it does not report this in any noticeable way. Thus you can end up in the same situation as we did, not running many tests without knowing it. (Oh yeah, if we only did review the code properly ...).
James O. Coplien has written in 2014 the thought-provoking essay Why Most Unit Testing is Waste and further elaborates the topic in his Segue. I love testing but I also value challenging my views to expand my understanding so it was a valuable read. When encountering something so controversial, it's crucial to set aside one's emotions and opinions and ask: "Provided that it is true, what in my world view might need questioning and updating?" Judge for yourself how well have I have managed it. (Note: This post is not intended as a full and impartial summary of his writing but rather a overveiw of what I may learn from it.)
Perhaps the most important lesson is this: Don't blindly accept fads, myths, authorities and "established truths." Question everything, collect experience, judge for yourself. As J. Coplien himself writes:
Be skeptical of yourself: measure, prove, retry. Be skeptical of me for heaven’s sake.
I am currently fond of unit testing so my mission is now to critically confront Coplien's ideas and my own preconceptions with practical experience on my next projects.
I would suggest that the main thing you take away isn't "minimize unit testing" but rather "value thinking, focus on system testing, employ code reviews and other QA measures."
I'll list my main take-aways first and go into detail later on:
Are you tired of days spent in front of the screen, with no results to show? Have you once again engaged in yak shaving? Today, after having failed previously, I have finally managed to solve a problem while avoiding this trap by following rigorously two guidelines preached by grandmaster programmers. Be warned: Following this approach, you will get a working solution - but you won't like it. It will be ugly, stained by compromises, far from the elegant solution you wish for. But if your resources are limited and you want to avoid death by too many yaks, this is your only option. But first, what are these guidelines?
One: Maintain a laser-sharp focus. A great programmer is constantly aware of what she is trying to achieve and never strays far from it. If the path leads away, she backs up. If something else pops up, she writes it down for later and gets back to the job. This is essentially about deciding what not to do. (Many thanks to Kent Beck for sharing his focus secret!)
These are my rather extensive notes from reading the chapter 8 on Automated Acceptance Testing in the Continuous Delivery bible by Humble and Farley. There is plenty of very good advice that I just had to take record of. Acceptance testing is an exciting and important subject. Why should you care about it? Because:
We know from experience that without excellent automated acceptance test coverage, one of three things happens: Either a lot of time is spent trying to find and fix bugs at the end of the process when you thought you were done, or you spend a great deal of time and money on manual acceptance and regression testing, or you end up releasing poor-quality software.
("cross-functional" might be better as they too are crucial for functionality)
f.ex. security, usability, maintainability, auditability, configurability but especially capacity, throughput, performance
performance = time to process 1 transaction (tx); throughput = #tx/a period; capacity = max throughput we can handle under a given load while maintaining acceptable response times.
NFRs determine the architecture so we must define them early; all (ops, devs, testers, customer) should meet to estimate their impact (it costs to increase any of them and often they go contrary to each other, e.g. security and performance)
das appropriate, you can either create specific stories for NFRs (e.g. capacity; => easier to prioritize explicitely) or/and add them as requirements to feature stories
if poorly analyzed then they constrain thinking, lead to overdesign and inappropriate optimization
only ever optimize based on facts, i.e. realistic measurements (if you don't know it: developers are really terrible at guessing the source of performance problems)