;; sec.ns | memory | CPU %
1541052937.882172509 59m 0.0
1541052981.122419892 78m 58.0
1541052981.625876498 199m 85.9
1541053011.489811184 1.2g 101.8

The tools

Gnuplot 5

• Simple, no-downtime deployment and automatic roll-back based on user-provided health-check (either one subset of nodes at a time or blue-green deployment)
• Autoscaling
• Managed updates - security fixes and other improvements installed automatically
• Built-in HTTP Proxy with caching in front of your application
• Monitoring dashboard with alerting and access to logs without the need for SSH
• A list of past versions & ability to roll-back
• Support for many runtimes (Java, Node.js, Docker to name just a few)

Re-posted from Telia's tech blog.

Solution: Use toxiproxy and its timeout "toxic" with the value of 0, i.e. the connection won't close, and data will be delayed until the toxic is removed.

The steps:

1. Start toxiproxy, exposing the port 6666 that we intend to use as localhost:6666:

docker pull shopify/toxiproxy
docker run --name=toxiproxy --rm --expose 6666 -p 6666:6666 -it shopify/toxiproxy

(If I was on Linux and not OSX then I could use --net=host and wouldn't need to expose and/or map the port.)

2. Tell toxiproxy to serve request att 6666  via an upstream service:

docker exec -it toxiproxy /bin/sh
/ # cd /go/bin/
/go/bin # ./toxiproxy-cli create upstream -l 0.0.0.0:6666 -u google.com:443

3. Modify your code to access the local port 6666 and test that everything works.

Since we want to access Google via HTTPS, we would get a certificate error when accessing it via localhost:6666 (e.g. "SSLHandshakeException: PKIX path building failed: [..] unable to find valid certification path to requested target" in Java or (much better) "(51) SSL: no alternative certificate subject name matches target host name 'localhost'" in curl) so we will add an alias to our local s /etc/hosts:

127.0.0.1 proxied.google.com

and use
https://proxied.google.com:6666 in our connecting code (instead of the https://google.com:443 we had there before). Verify that it works and the code gets a response as expected.

Note: google.com is likely a bad choice here since it will return 404 as you must specify the header "Host: www.google.com" to get 200 OK back; without it you will get 404.

4. Tell toxiproxy to have an infinite timeout for this service

Continuing our toxiproxy configuration from step 2:

./toxiproxy-cli toxic add -t timeout -a timeout=0 upstream

(Alternatively, e.g. timeout=100; then the connection will be closed after 100 ms.)

5. Trigger your code again. You should get a timeout now.

Tip: You can simulate the service being down via disabling the proxy:

./toxiproxy-cli toggle upstream

Aside: Challenges when proxying through Toxiproxy

The host header

Servers (e.g. google.com, example.com) don't like it when the Host header (derived normally from the URL) differs from what they expect. So you either need to make it possible to access localhost:<toxiproxy port> via the upstream server's hostname by adding it as an alias to /etc/hosts (but how do you then access the actual service?) or you need to override the host header. In curl that is easy with -H "Host: www.google.com" but not so in Java.

In Java (openjdk 11.0.1 2018-10-16) you need to pass -Dsun.net.http.allowRestrictedHeaders=true to the JVM at startup to enable overriding the Host header (Oracle JVM might allow to do that at runtime) and then:

(doto ^HttpURLConnection (.openConnection (URL. "https://proxied.google.com:6666/"))
  (.setRequestProperty "Host" "www.google.com")
(.getInputStream)

SSL certificate issues

As described above, when talking to HTTPS via Toxiproxy, you need to ensure that the hostname you use in your request is covered by the server's certificate, otherwise you will get SSL errors. To apply the solution described here, i.e. adding e.g. proxied.<server name, e.g. google.com> to your /etc/hosts works, provided the certificate is valid also for subdomains, i.e. is issued for <server> and *.<server>, which is not always the case.

Alternatively, you can disable certificate validation - trivial in curl with -k but much more typing in Java.

Cross-posted from the TeliaSonera tech blog

Cross-posted from the TeliaSonera tech blog

Cross-posted from TeliaSonera Tech blog.

Failed attempt one: Let tools do it

1. An experienced speaker once recommended sharing personal experiences (even - or especially - if they make me vulnerable) as it is much easier for people to relate to them than to general statements.
2. A Cognicast eposide mentioned storytelling as a great tool for introductory guides. We humans are natural storytellers, we think in stories and relate to them much more easily - so a story should be great also to communicate the value of a change.
3. My ex-colleague Therese Ingebrigtsen gave an inspiring talk presenting some points from The Switch - mainly that we need to address the recipient's minds with rational arguments, but also their hearts to involve their emotion (e.g. by drawing a picture of the new bright future), and that it is important to show a clear path forward.

Extra Links

• Release It! slides: http://gotocon.com/dl/jaoo-sydney-2009/slides/MichaelT.Nygard_FailureComesInFlavoursPart2.pdf
• https://github.com/Netflix/Hystrix/
• Netflix's Dependency Command talks about using circuit breakers and a thread pool limit http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
• MSDN Guidance for Cloud Applications: Design Patterns - Circuit Breaker and many more highly useful patterns

Stability

Stability antipatterns

Integration points

• Practical CSS tips & tricks for backend developers - really useful tips! video
• No Estimates, Let's Explore the Possibilities video
• Form with Function: Adding Behavior with CSS - recommended by a friend (modal dialogs, tab switching, ...) video
• ? Anti-fragile and feedback. Trying to make up for the failures of "agile." video
• ? JavaScript Forensics - not sure what this is about but it might be interesting
• ? Designing and Programming Accessible Website and App UIs video

• ? Declarative REST: State Machines for the Web video
• Continuous Delivery for Architects - Neal Ford video
• ? Desktop applications using JavaScript and Electron video
• ? High Performance in the Critical Rendering Path - how to make pages to load fast video
• This is Water - Neal Ford - an excursion into a strange, fantastical world with things like immutable database server, phoenix machines, and lambdas. video
• Securing Web APIs – Patterns & Anti-Patterns video
• ? Functional Data - event sourcing & FP video
• Authentication and authorization in modern JavaScript web applications – how hard can it be? video
• ? Taking other peoples money: A guide to online payments video
• ? Running Docker and Containers in Development and Production video
• Not Even Close: The State of Computer Security

• 595 billions income - untouched by human hands video
• Boosting security with HTTP headers video
• The rest of ReST - we'll look at the challenges of building usable real-world ReST APIs: Hypertext Application Language (HAL), HTTP Patch, ... video
• ? How do you scale a logging infrastructure to accept a billion messages a day? - DB -> ELK -> ELK + Kafka video
• ? Learning Client Hypermedia from the Ground Up - how to move specific knowledge of 1) addresses, 2) inputs, and 3) workflow out of the client app and place it into the message => a more robust, adaptable, and resilient client video
• ? Make it Faster - Lessons Learned from Benchmarking NoSQL on the AWS Cloud - best practices for performing database benchmarking on the AWS cloud & how to get more speed and efficiency in your production workloads video
• ? Crafting Evolvable Web API Representations - like structuring for evolution, sizing for optimum caching, the different ways to include metadata, ... video
• ? Mob Programming, A Whole Team Approach video
• ? Removing barriers - JetBrains's good and bad expericences with minimizing management video

Keynote Data and Goliath ☆☆☆☆

Troy Hunt (@troyhunt, blog) had a great, very hands-on 2-day workshop about webapp security at NDC Oslo. Here are my notes.

Highlights - resources

Personal security and privacy

• https://www.entropay.com/ - a Prepaid Virtual Visa Card
• mailinator.com - tmp email
• f-secure VPN
• https://www.netsparker.com/ - scan a site for issues (insecure cookies, framework disclosure, SQL injection, …) (lot of $k)

Site security

• https://report-uri.io/ - get reports when CSP rules violated; also displays CSP headers for a site in a human-friendly way
• https://securityheaders.io/ check quality of headers wrt security
• free SSL - http://www.startssl.com/, https://www.cloudflare.com/ (also provides web app firewall and other protections) ; 
• SSL quality check: https://www.ssllabs.com/ssltest/ 
• https://letsencrypt.org/ - free, automated, open Certificate Authority (Linux Found., Mozilla)
• HSTS Preload - tell Chrome, FF that your site should only be ever loaded over HTTPS - https://hstspreload.appspot.com/

Breaches etc.

• http://arstechnica.com/security/2015/06/hack-of-cloud-based-lastpass-exposes-encrypted-master-passwords/
• https://twitter.com/jmgosney - one of ppl behind http://passwordscon.org . http://password-hashing.net  experts panel. Team Hashcat. 
• http://arstechnica.com/security/2012/12/25-gpu-cluster-cracks-every-standard-windows-password-in-6-hours/ 

To follow

• ! http://krebsonsecurity.com/
• ! http://www.troyhunt.com/
• ! https://www.schneier.com/
• ! https://twitter.com/mikko (of F-Secure) also great [TED] talks
• kevin mitnick (jailed for hacking; twitter, books)

The game

The imperative solution

1. Build duplicity and its dependencies (since WD Debian v04 switched to page size of 64kB, all pre-built binaries are unusable)
2. Configure S3 to move the data files to Glacier after 0 days
3. Create your backup script - see backup-pictures-to-s3.sh
4. Schedule to run incremental backups regularly via Cron
5. Preferably test restore manually

Be skeptical of yourself: measure, prove, retry. Be skeptical of me for heaven’s sake.

• f.ex. security, usability, maintainability, auditability, configurability but especially capacity, throughput, performance
• performance = time to process 1 transaction (tx); throughput = #tx/a period; capacity = max throughput we can handle under a given load while maintaining acceptable response times.
• NFRs determine the architecture so we must define them early; all (ops, devs, testers, customer) should meet to estimate their impact (it costs to increase any of them and often they go contrary to each other, e.g. security and performance)
• das appropriate, you can either create specific stories for NFRs (e.g. capacity; => easier to prioritize explicitely) or/and add them as requirements to feature stories
• if poorly analyzed then they constrain thinking, lead to overdesign and inappropriate optimization
• only ever optimize based on facts, i.e. realistic measurements (if you don't know it: developers are really terrible at guessing the source of performance problems)

We know from experience that without excellent automated acceptance test coverage, one of three things happens: Either a lot of time is spent trying to find and fix bugs at the end of the process when you thought you were done, or you spend a great deal of time and money on manual acceptance and regression testing, or you end up releasing poor-quality software.

• Growth cannot be 4-5% forever. In the long term, 1-1.5% is more realstic (and still far more than in the past). Higher growth is typical of countries catching up to more advanced economies (such as Europe to US, UK after WW2, China to the West nowadays).
• Return on capital, typically 4-5% (before taxes), is thus far higher than the growth of economy and salaries. The result is that the rich get ever richer, taking ever more of the national income. (Consequently, the poorer half has ever less of it.)
• To keep democracy and have a stable society, this has to be adressed politically.

@lukew: Start with the simplest version you can. It's much easier to add complexity than to remove it.

ReferenceError: Can't find variable: cemerick

1. Your test namespaces do not require cemerick.cljs.test (and thus it is missing from the compiled .js; requiring macros is not enough)
2. cljsbuild has not included any of your test files (due to wrong setup etc.; this is essentially another form of #1)
3. You are trying to test with the node runner but have built with :optimizations :none or :whitespace (for node you need to concatenate everything into a single file, which only happens if you use :simple or :advanced optimizations)

Update: Graphite data gotchas that got me

1. Graphite shows aggregated, not raw data if the selected query period (24h by default) is greater than the retention period of the highest precision. F.ex. with the schema "1s:30m,1m:1d,5m:2y" you will see data at the 1s precision only if you select period less than or equal to the past 30 minutes. With the default one, you will see the 1-minute aggregates. This applies both to the UI and whisper-fetch.py.
2. Aggregation drops data unless by default at least 50% of available data slots have values (xFilesFactor=0.5). I.e. if your app sends data at a rate more than twice slower than Graphite expects them, they will never show up in aggregates. F.ex. with the schema "1s:30m,1m:1d,5m:2y"  you must sends data at least 30 times within a minute for them to show in an aggregate.

Lesson learned: Always send data to Graphite in *exactly* same rate as its highest resolution

0. Create a project:

$ mvn archetype:generate \
   -DarchetypeCatalog=http://repository.excilys.com/content/groups/public/archetype-catalog.xml
   -Dfilter=io.gatling:

1. Import to IntelliJ

2. Record a simulation

1. Run the src/test/scala/Recorder.scala (right-click - Run 'Recorder')
2. Set the port it should listen on, f.ex. 8000 (maybe you also need to set port for HTTPS, f.ex. 8001), set the target app (perhaps localhost, <some port>, <some https/dummy port>)
3. Optionally set the class name of the recorded simulation and the target package (the output goes to src/test/scala/<package>/<name>.scala)
4. Click [Start !]
5. Go to your browser and configure it to use the recorder as its HTTP[s] proxy
6. Browse localhost:8000/your_app as you want for the test
7. Click [Stop and save] in the Recorder UI

• the synchronize module, using rsync
  • cons: does not support group=, owner=
  • -C and --diff - it does not print diff of the files changed; when running ansible with -v, it will print output of rsync's --itemize-changes, i.e., for each changed file/dir, something like "<f.st...... conf/httpd.conf\n" (< = file uploaded, s = size changed, t = timestamp changed, . = this attribute has not been changed)
• the copy module
  • -C --diff - it only reports "changed" without naming the changed files or showing diffs (unless there is only one changed file)
• the local_action module, used to run rsync manually (essentially the same as synchronize but more control)
• So the only way to do a kind of recursive copy with working diff is to use copy with with_fileglob for each directory and subdirectory :-(

Use ./hacking/test-module

https://github.com/jakubholynet/ansible-example-with-vm

1. A non-trivial Ansible configuration that demonstrates couple of useful features and tricks
2. A Vagrant/VirtualBox virtual machine with Ansible & co. to make it easy to run it (even on Windows)
3. Another VM that can be used to test the configuration

1. Create an all-in one uberjar via "lein with-profile production ring uberjar" (using the lein-ring plugin; a simple lein uberjar would suffice for an app with a main- method)
2. Create an upstart <service name>.conf file in /etc/init/
3. Run sudo start/stop/status <service name>

;; core and charts are the incanter namespaces
(defn plot-power []
  (let [fun #(Math/pow 10 %)
        y-axis (log-axis :label "log(x)")
        chart (charts/function-plot fun 0 5)]
    (set-axis chart :y y-axis)
    (core/view chart :window-title "LogAxis Test: Incanter fun plot")))

• Real Options—a Mindset - an intro into the Real Options approach, which has been quite a hot topic and a transformational way of thinking for a number of inspiring people (Dan North, Liz Keogh etc.). "Real Options help us to better make decisions and commitments with three simple principles: Options have value. Options expire. Never commit early unless you know why." We can "pay" to keep our options open longer, i.e. to avoid commiting prematurely.
• Demystifying the CHAOS report's claim of ~ 1/2 features being unused: the Standish Group's CHAOS report has been often quoted for its "finding" that a large percentage of features in applications is never/rarely used. However this claim seems to have never been confirmed, their "research" is reportedly not very scientific and not publicly available for scrutiny.  Critique by Laurent Bossavit (2013), Jorge Aranda's Standish, the CHAOS report, and science. Thx to @smalltalk80 for pointing this out! However there is one research, Online Experimentation at Microsoft, that supports the claim, in a different context but the same problem applies to features: "Evaluating well-designed and executed experiments that were designed to improve a key metric, only about one-third were successful at improving the key metric!"
• Why Yammer believes the traditional engineering organizational structure is dead - small teams, small projects (2-10 people, 2-10 weeks), no separation into front/middle tier/backend team (=> communication, design obstacle); have instead people specializing in these areas and construct feature teams from them based on the actual needs; engineers, not managers do eng. decisions; all aligned via focusing on the same 3 key metrics. Small projects => constant sense of urgency (and excitement): Often very long projects cause engineers to lose track of the end goal.  Think of it in terms of hiking: start fresh & excited, get tired and losing track of the goal, excited again at the end => cut out the middle part, keep them in the exciting state where they can measure progress and see it visually; it’s the only way to maintain urgency and morale. Focus: people alwasy work only at one (short) project at a time (there are special bug-fixing teams for maintenance tasks with people rotating in&out).
• Agile development is more culture than process - Why thinking of agile as culture and not just process explains resistance and difficulty in teaching and learning the approach - and should be taught so => 1. Underscore agile values that motivate practice; 2. Identify organization values that compete with agile values, conflict of values; 3. Be sensitive to culture shock.
• Mark Zuckerberg's Letter to Investors: 'The Hacker Way' (quite long, you might want to read only "The Hacker Way" part at the end) - about Facebook's "unique culture and management approach" - "Hackers believe that something can always be better, and that nothing is ever complete." "Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once." "Instead of debating for days [..], hackers would rather just prototype something and see what works." "Hacker culture is also extremely open and meritocratic." "Many of our most successful products came out of hackathons, [..]." <=> five core values: Focus on Impact (focus on solving the most important problems, be good at finding the biggest problems to work on); Move Fast ("[..] if you never break anything, you’re probably not moving fast enough."); Be Bold ("Building great things means taking risks."); Be Open (=> effort to make as much info as possible visible to all); Build Social Value ("[..] Facebook exists to make the world more open and connected, and not just to build a company. ")
• Dave Nicolett: I know how to tie my shoes - on the difficulty of convincing people to try unfamiliar software development techniques - "People change the way they operate when they are experiencing some sort of inconvenience or negative feedback. As long as things are going along reasonably well, people don’t go out of their way to change the way they work." (with few exceptions) You can learn to tie your shoes in a split second, but why to invest the effort? You'd need to set aside assumptions, suppress habits, practice. You can argument there are many inconveniences (bugs, criticism for slow delivery, ...) but "Unfortunately, that’s all pretty normal, and most people in the software field are accustomed to it.  They don’t see it as a problem that calls for them to change their practices. Most of them probably have a hard time visualizing a different reality." => Maybe that’s the reason there’s been no satisfactory answer to the question of how to convince people to adopt different practices. We shouldn’t be trying to convince people to do anything. We should be helping people solve their problems and achieve their goals. If they are satisfied with the outcomes they achieve using their current methods, then there is no problem to solve.
• Kent Beck: Pace of Progress = Pace of Feedback - '"The pace of my progress is completely constrained by the pace of my feedback". If I want to go faster, it's hard to achieve by going faster. I can almost always optimize my feedback loop, though.' "The second lesson from this episode is that it's not just the duration of the feedback loop that matters, it's also the quality. All week I was working in tiny little iterations. Without producing useful information, though, those iterations could be as small or as large as I liked, I was still just going to spin my wheels." => "The next time I seem to be going slow, I'm going to look at my whole feedback loop--duration, quality and my ability to respond to the information."
• What Google Has Learned About How to Hire People - interview results have no relation to actual performance on the job: "We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship. It’s a complete random mess." "Instead, what works well are structured behavioral interviews, where you have a consistent rubric for how you assess people, [..]" 'Behavioral interviewing also works — where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.”' Link to an interesting book, Hiring Geeks That Fit.

• The Elixir language - Clojure + Ruby + Erlang - a functional meta-programming aware language built on top of the Erlang VM; a dynamic language with flexible syntax with macros support that leverages Erlang's abilities to build concurrent, distributed, fault-tolerant applications with hot code upgrades. First-class support for pattern matching, polymorphism via protocols, etc. (via @bodil)
• Random Testing seems to be gaining popularity and looks very interesting; at NDC Oslo, John Hughes has presented how QuickCheck, which generates random sequences of API calls, has been successfully used to find bugs in the Riak DB and a file system that a human would never think of, and Stuart Halloway has presented simulation testing with Simulant, which runs predefined actions according to a probabilistic model (e.g. 100 traders, each having 1h mean time between trades and mean traded amount 100, the test runs for 4 simulated hours). Something worth exploring!
•  Dmytro Navrotskyy's collection of Frontend Development resources and learning materials for tools (grunt, unused css detection,..), best practices (Atomic Design, ...), JS/CSS frameworks, typography, animation, visualization, useful on-line services,  and many more (via Herman Schistad)
• The Secret To 10 Million Concurrent Connections - The Kernel Is The Problem, Not The Solution: To have really fast SW, you need to implement your own core services (FS, net driver (packet handling), thread scheduling, ..) tuned for your app. You need to be aware of the clock-time cost of cache misses, memory access etc.. Custom solutions are times faster than what the general OS kernel can offer. => "data plane oriented system" Core areas and solutions for them: packet scalability, multi-core scalability (locks are expensive), memory scalability.

• M. Fowler: EmbeddedDocument - a pattern for working with JSON flowing in/out of our services (REST <-> JSON-friendly DB) without unnecessary conversions but with good encapsulation; naive approach: json -> object graph -> (processing) -> json; "In many of these situtiations a better way to proceed is to keep the data in a JSONish form, but still wrap it with objects to coordinate manipulation." - use a lib to parse the JSON into a generic structure (e.g. a structure of lists, and maps/dicts) and store in a field of an object defining methods that encapsulate it - f.ex. for an Order we could have a method returning the customer and another computing the cost, accessing the underlying generic structure. The user of the wrapper object doesn't need to know/care about the underlying structure. "The sweet spot for an embedded document is when you're providing the document in the same form that you get it from the data store, but still want to do some manipulation of that data. [..] The order object needs only a constructor and a method to return its JSON representaiton. On the other hand as you do more work on the data - more server side logic, transforming into different representations - then it's worth considering whether it's easier to turn the data into an object graph."
• ThoughtWorks' Approach To Big Data Analytics - an inspiring, brief read. Some really good points such as "It’s not about Data. It’s about Insight and Impact" => "focus on the questions you’d love to answer for your business" => "changing big data from a technological problem to a business solution." Also "The value of data is only realised through insight. And insight is useless until it’s turned into action." Measure the value you gain at each step. See  Introducing Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing by Ken Collier
• Wired.com, Nassim Taleb: Beware the Big Errors of 'Big Data' - in big data, noise has much stronger effect and in a large enough dataset we will always find spurious (i.e. false) relationships => beware! "Well, if I generate (by simulation) a set of 200 variables — completely random and totally unrelated to each other — with about 1,000 data points for each, then it would be near impossible not to find in it a certain number of “significant” correlations of sorts. But these correlations would be entirely spurious."
• A Taste of Salt: Like Puppet, Except It Doesn’t Suck - a deescription of Salt and the tools around by an enthusiastic user with deep experience with Puppet. Highlights: Light-weight communication over ZeroMQ, very active community, simplicity, configuration is YAML, Salt-cloud can spin instances in EC2/Openstack/..., Salt-virt does the same for virtual machines (KVM/Xen/...), Salt-vagrant, Salt-monitor (work in progess) can ask all the server for their stats. "Having stood up a number of different configuration management systems across a wide variety of environments, I’ve yet to find a solution that’s as rapid to deploy, simple to scale, or as well architected as Salt."
• Trash Your Servers and Burn Your Code: Immutable Infrastructure and Disposable Components - leveraging the lectures of PF to have a stable infrastructure - instead of updating servers, throw them away and create a new one from scratch (requires virtualization/cloud); this is something that Netlfix is doing and also Comoyo is moving towards
• Robin Ward: AngularJS vs Ember - a nice overview of the different approaches of the two; the author is strongly pro-Ember, claiming that AngularJS is much closer to low-end libraris like Backbone/Knockout and that you will often need the additional features of Ember. The comments provide the right countrweight to the biased post and form thus a good whole together.
• Scala Productivity. A Survey of the Community - people (that asnwered) seem to be productive with Scala right from the start

• After Your Job Is Gone - an interesting essay on the future, which, according to the author, we can already see happening, when technology will take away most of our work and we will not need to work all day. Not very optimistic, though (the author predicts few reach and many poor people).

Clojure Corner

• Clojure Cup 2013, Sept 28-29 - create something cool with Clojure/ClojureScript within 48h and perhaps win a price! #fun
• Clojure use in the industry - examples at an e-mail forum - Netflix, Puppet Labs (e.g. PuppetDB), UBS (talk), Deutsche Bank (talk, some details), Citigroup (reportedly "the largest private sector deployment of Clojure to date," 11/2012), getprismatic.com (with frontend moving to ClojureScript; -> Why Prismatic Goes Faster With Clojure), Roomkey.com (details in a Relevance podcast), MastodonC.com (big data), Trend Micro, Walmart, beanstalkapp.com, ReadyForZero.com (50kLoC),  www.cognician.com (20kLoC), World Singles (13kLoC) and more... (another similar thread)
• Stuart Sierra's My Clojure Workflow, Reloaded (6/2013) - mainly about reloading changes into REPL, working around things that are not reloaded/left over => restart the app from scratch after significant changes => the app as a transient object => no global state, careful management of resources, :dev profile with :source-paths to a dir with user.clj (autoloaded by repl, pre-loading useful stuff) and dev util deps
• Adam Bard's walk-through useful Clojure libs - f.ex. clojure.[data.[csv xml json] inspector java.shell java.browse xml], tools.logging, clojure.core.[match logic typed contracts ...]
• Juxt.pro: Jon Pither's and Malcolm Sparks' "network of experienced IT professionals who specialise in the Clojure programming language," providing training, consulting, talks
• Anthony Grimes: The Clojure Community and Me (2011) - an exciting insight into the embracing and supportive Clojure community
• In Clojure-based Machine Learning: "Our backend is 99.4% coded in Clojure, and 66% of the team [of 3] had never programmed seriously in any Lisp, let alone Haskell or Prolog (heck, not even I (the remaining 33%) had actually tried anything non-mainstream for real in a big project!) Maybe some Ruby, and lots and lots of Java and C and C++. But they accepted the challenge after reading around and learning the basics, and 3 months later you couldn’t take Clojure from their prying hands."
• J. Pither: TDD and Clojure - "If you were to create a shopping list of things you really want for your development experience then what would you put at the top?" => 1. rapid feedback on changes, 2. REPL (place to explore and to play with your code <=> TDD), 3. FP and Immutability ("FP and dynamic languages lead to a lot less code. There’s less ceremony, less modeling. Because you’re managing less code you do less large scale refactorings." => TDD needed less), 4. Regression Tests ("It’s my current opinion that what you get left out of TDD once you have amazingly fast feedback and a REPL is regression testing.")

Tools/Libs

• More beautiful and colorful git log, by Filipe Kiss (via @lcdutoit) You may also want to have a look at tig, which is a text-based UI for git with the default view similar to git log.
• jq (via lcdutoit) - sed/awk/grep for JSON - slice, filter, map, transform structured data
• SemanticMerge (Windows) - a Java-aware merge tool - free beta (I haven't tried it)

• JetLang - a high performance java threading library for in-memory messaging, based upon Retlang (via @tastapod, used likely in a trading SW).

Favorite Quotes

Agile [is] in NOT a process — it’s a philosophy.

• Patterns - strategies that work in a particular context - and not in another (too often we forget the context and to consider the context where a strategy doesn't work / is contra-productive); beware: a part of the context is the experience of the developer; for unexperienced devs it might be better to just stick to a process and applying TDD etc. all the time than trying to guess when they are appropriate and when not without having the experience to decide it right
• Effective - optimize for something: volume of SW produced? time to market? learning/discovery? certanity? user experience?
• Delivery - get stuff that is useful out of the door; software is not important, the utility it provides is; know why you write the SW to be able to get better at it

• ThoughWorks Technology Radar May 2013 - Maven replaced by Gradle, Clojure and Scala on adopt, big enterprise SW and WS-* out, lot of interesting stuff to adopt or assess
• Straw Man TDD: debunking 6 common TDD myths (via M.Fowler) - such as TDD = {no upfront design, 2*longer development, # hard-to-change test code}.
• Programmer Competency Matrix - a nice overview of a programmer needs to know in different areas (computer science: data structures etc., SW engineering: SCM etc., programming: defensive coding, IDE etc.) + what experience and knowledge they need, split into 4 levels. Where are you?
• The 12 most disruptive technologies of the next 10 years according to McKinsey - 1. mobile internet, 2. automation of knowledge work, 3. internt of things, 4. cloud, 5. advanced robotics, 6. [near-]autonomous vehicles, 7. next-generation genomics, 8. energy storage, 9. 3D printing, 10. advanced materials, ...; the full 178 page report.
• Your login form posts to HTTPS, but you blew it when you loaded it over HTTP - summary: without https, anybody between the server and the client can inject anything (e.g. a key logger) into the web page; exploit example
• Older Is Wiser: Study Shows Software Developers’ Skills Improve Over Time (from an analysis of StackOverflow)- and older developers have much broader skill set and know about subjects

• Discussion: World's Biggest 'Agile' Software Project Close To Failure - what is/isn't agile, agile vs. waterfall etc; a nice collection of all the possible opinions and misunderstandings. Some of my favorites: waste: '[..]An "agile" project cannot fail and cost Billions because it must always deliver runnable software with a maximum of a few weeks delay[..]' (runnable = delivering value), separation: '[..] my experience has been that separating the designer/architect role from the developer role is fraught with pitfalls. The people writing the code should be the ones designing it, [..]', stability: '[..] On the successful projects that I've worked with in Agile, there's strong stakeholders, good architecture keeping the vision in place and project management that keeps things well orchestrated. Without those in the mix, it'll fail just like all software projects. [..]', waterfall success, people issue: 'The problem here isn't waterfall/agile. The problem here isn't .Net/Linux. The problem here is the parties involved. [politicians and IT dinosaurs]' (learn what's needed from drunk, bitching employees; ignore official nonsense requirements),: simplicity '[..] It would probably be a lot easier if they started by making a simpler tool - instead of trying to calculate everybody's entitlements everywhere [..]', agile suitability: '[..] You cannot use Agile to build a 100-mile canal, as the whole thing would be useless even if you completed 99 miles. [..]'. Some people seem to believe that agile means no architecture and no/too little planning. Some believe that agile = hack now, fix later.

• Why do we worry about scaleability on Day 1? (via @simenfur) -  you can fake/do many things manually to get started to validate your service and eventually to provide it to real, paying customers without lot of features/automation that you might believe to be essential; many real-world examples of this. Quoting 42Floors: “In fact, when you look at each aspect of our operations, you will find a fairly sophisticated manual operations process predating every piece of technology that we’ve built.” Buffer launched a business without 5 seemingly essential ingredients, f.ex. upgrade from a free to paid account. ... Try the “service-first” approach!
• Confessions of a lean startup: how I got my first customers without having a product (in 7 weeks) - an interesting story of verifying that an idea is worht pursuing by getting paying customers before there was any product (doing it manually for them), the tools and approaches to use and mistakes to avoid (attract people - signups - interviews & pitches - paying customers - development)

Clojure Corner

• B. Batsov's Clojure Style Guide (based on JoC etc.) at GitHub
• Code quaterly interview with R. Hickey (2011) - motivation behind Clojure (simplicity,..), reusability in C. (vs. Java); many valuable things
• Clojure in the Enterprise? - about the differences between the Clojure and the enterprise java  (with heavy frameworks such as JPA, JSF, mutable state) ways and difficulties of introducing Clojure due to old-fashioned thinking, limited skills, etc. "Take away objects, mutable state, variables, loops... a lot of Java developers are immediately all at sea and have no idea how to solve even basic problems. They'll try to bend Clojure to their OOP way of thinking and they'll most likely fail." World Singles' experience with Clojure: "We love Clojure. It's made development a lot more fun. We're able to solve harder problems, make changes faster, leverage multi-core concurrency more effectively, and we have a much smaller code base to maintain."
• Replace Temp with Query in Clojure - in this post we can follow an interesting refactoring from a deeply nested if & let code to a much flatter one (featuring cond to test multiple conditions (instead of guard conditions used in imperative languages) delay to be ably to bind an expression to a local variable without evaluating it yet)
• The Why and How of Clojure on Android (4/2013) - about the experience of using Clojure on Android, which is little slow and crazy but fun. From the author of the Nightweb app. Key components:  neko Clojure wrappers for Android and for lein-droid building w/o an IDE. Nicer than Java! (This GSoC proposal shows the limitations/future of neko.)
• Reconstructing Clojure Macros With Speclj - a good idea to learn macros by trying to create one's own implementation of existing Clojure macros, driven by tests that define the expected behavior of the macro (using macroexpand, macroexpand-1, macroexpand-all)
• Advanced inspector middleware for Clojure nREPL and Emacs (nrepl-inspector on GitHub) - C-c C-i or nrepl-inspect to inspect interactively the value of a var (beware: (require 'nrepl-inspect) fails, however calling nrepl-inspect as a function is ok). Related: Clojure Debugging ’13: Emacs, nREPL, and Ritz (May 17th) - what works, what is missing, how to set up
• Pithering About » REPL bootstrap pimpage - neat tricks with a custom bootstrap REPL namespace - print last few commits and git branch, pre-loading common libs

Tools

• Fish Shell 2.0 finally released (among others adding the ability to set the title of iTerm tabs properly); release notes, 1m demo of some coolness

Favorite Quotes

C, C#, and Java: Applying some of the best ideas of the 1970s to the problems of today.

Benchmarks are important for rational technological choices yet it is very hard if not impossible to perform them in a sensible way.

Events & side jobs

• Kent Beck: When Worse Is Better: Incrementally Escaping Local Maxima - Kent reintroduces his Sprinting Centipede strategy ("reduce the cost of each change as much as possible so as to enable small changes to be chained together nearly continuously" => "From the outside it is clear that big changes are happening, even though from the inside it's clear that no individual change is large or risky.") and advices how to deal with situations where improvements have reached a local maxima by making the design temporarily intentionally worse (f.ex. by inlining all the ugly methods or writing to both the old and the new data store); strongly recommended
  • Related: Efficient Incremental Change - transmute risk into time by doing small, safe steps, then optimize your ability to make these steps quickly and thus being able to achieve large changes
• Researchers: It is not profitable to outsource development - the Scandinavian research organisation SINTEF ICT has studied the effects of outsourcing and discovered that often it is more expensive than in-country development due to hidden costs caused by worse communication and cultural differences (f.ex. Indians tend not to ask questions and work based on their, often incomplete, understanding) and very high people turn-over; even after the true cost is discovered, companies irrationally stay there. However it is possible to succeed, in some cases.
• Bjørn Borud: Tractor pulling and software engineering - very valuable and pragmatic advices on producing good software (i.e. avoiding accumulating so much crap that the software just stops progressing). Don't think only about the happy path. Simplify. Write for other developers, i.e. avoid too "smart" solutions, test & document, dp actually think about design and its implication w.r.t performance etc. Awake the scientist in you: "Do things because you know they work, not because it happens to be the hip thing to do." (Note: I see the good intention behind "design for the weakest programmer you can think of" but plase don't take it too far! Software should be primarily simple, not necessarily easy.
• Know your feedback loop – why and how to optimize it - to succeed, we need to learn faster; the only way to do that is to optimize our feedback loops, i.e. shorten the path our assumptions travel before they are (in)validated, whether about our code, business functionality, or the whole project idea. Conscise, valuable.
• Code quality is the least important reason to pair program - the author argues, based on his experience, other benefits of pair programming are more important than code quality: "[..]  the most important reasons why we pair: it contributes to an amazing company culture, it’s the best way to bring new developers up to speed, and it provides a great way to share knowledge across the development team."
• You Can’t Refactor Your Way Out of Every Problem - refactoring can’t help you if the design is fundamentally wrong, you need to rewrite it; know when it can or cannot help and act accordingly (related to how much design is needed upfront since some design decision cannot be reverted/improved upon)

• Josh Bloch: Java - the good, bad and ugly parts (video, 15 min); summary: right design decisions (VM, GC, threads, dynamic linking, OOP, static typing, exceptions, ...), some bad details (signed byte, lossy long-> double, == doesn't cal .equals, ability to call overriden methods from constructors, ...); Mr. Bloch has also given a longer talk examining the evolution of Java from 1.0 to 1.7 in The Evolution of Java: Past, Present, and Future.
• True Scala complexity - a thoughtful criticism of the complexity of Scala, based on code samples; "[it is true that] Scala is a language with a smaller number of orthogonal features than found in many other languages. [...] However, the problem is that each feature has substantial depth, intersecting in numerous ways that are riddled with nuances, limitations, exceptions and rules to remember. It doesn’t take long to bump into these edges, of which there are many."; however, its possible to avoid many of the problems mentioned by resorting to less smart, more clumsy and verbose Java-like code; also, the author still likes Scala.
• Scala or Java? Exploring myths and facts (3/2012) - a balanced view of Scala's strengths and weaknesses; "[..] the same features that makes Scala expressive can also lead to performance problems and complexity. This article details where this balance needs to be considered." Topics: productivity, complexity, concurrency support, language extensibility, Java interoperability, quality of tooling, speed, backward compatibility. Plenty of useful links.

• Dean Wampler's slides from Beyond Map Reduce - 1) Hadoop Map Reduce is the EJB 2 of big data but there are better APIs such as Cascading with Scala/Clojure wrappers; there are also "alternative" solutions like Spark and Storm; 2) functional/relational programming with simple data structures (lists, sets, maps etc.) is much more suitable for big data than OOP (for we do mostly stateless data transformations)
• Apache HBase vs Apache Cassandra - comparison sheet - if you want to decide between the two
• Optimizing MongoDB on AWS - 20 min talk about the current state of the art. Simplicity: Mongo AMIs by 10gen, Cloudformation template etc. Stability & perf.: new storage options - EBS with provisioned IOPS volumes (high I/O) + EBS Optimized Instances (dedicated throughput to EBS), High IO instances (hi1.4xlarge - SSD)); comparison of throughput (number of operations, MBs) of these storages; tips for filesystem config. Scalability: scale horizontally and vertically, shrink as needed.
• Getting Real About Distributed System Reliability by Jay Kreps, the author of the Voldemort DB: distributed software is NOT somehow innately reliable; a common mistake is to consider only probability of independent failures but failures typically are dependent (e.g. network problems affect the whole data center, not a single machine); the theoretical reliability "[..] is an upper bound on reliability but one that you could never, never approach in practice"; "For example Google has a fantastic paper that gives empirical numbers on system failures in Bigtable and GFS and reports empirical data on groups of failures that show rates several orders of magnitude higher than the independence assumption would predict. This is what one of the best system and operations teams in the world can get: your numbers may be far worse." The new systems are far less mature (=> mor bugs, worse monitoring, less experience) and thus less reliable (it takes a decade for a FS to become mature, likely similar here). Distributed systems are of course more complex to configure and operate. "I have come around to the view that the real core difficulty of these systems is operations, not architecture or design." Some nice examples of failures.

• Talks To Help You Become A Better Front-End Engineer In 2013 (tl;dr) - topics such as mobile web development, modern web devel. workflow, current/upcoming featrues of CSS3, ECMAScript 6, CSS preprocessors (LESS etc.), how to write maintainable JS, modular CSS, responsive design, JS debugging, offline webapps, CSS profiling and speed tracer, JS testing
• On Being A Senior Engineer - valuable insights into what makes an engineer "senior" (i.e. mature; from the field of web operations but applies to IT in general): mature engineers seek out constructive criticism of their designs, understand the non-technical areas of how they are perceived (=> assertive, nice to work with etc.), understand that not all of their projects are filled with rockstar-on-stage work, anticipate the impact of their code (on resource usage, others' ability to understand & extend it etc.), lift the skills and expertise of those around them, make their trade-offs explicit when making decisions, do not practice "Cover Your Ass Engineering," are able to view the project from another person’s (stakeholder's) perspective, are aware of cognitive biases (such as the Planning Fallacy), practice egoless programming, know the importance of (sometimes irrational) feelings people have.

Clojure Corner

• Polymorphism in Clojure - Tim Ewald's 1h live coding talk at Øredev conference introducing mechanisms for polymorphism (and Java interoperability) in Clojure and explaining well the different use cases for them. Included: why records, protocols & polymorphism with them (shapes, area => open, not explicit switch) (also good for Java interop.: interfaces), reify, multimethods.
• Stuart Sierra: Thinking in Data (1h talk) - Sierra introduces data-oriented programming, i.e. programming with generic, immutable data structures (such as maps), pure functions, and isolated side-effects. Some other points: Records are an optimization, only for perforamnce (rarely) or polymorphism (ot often); the case for composable functions;  testing using simulations (generative testing) etc.; visualization of state & process

Tools & Libs

• Netflix' Hysterix: library to make distributed systems more resilitent by preventing a single slow/failing dependency from causing resource (thread etc.) exhaustion etc. by wrapping external calls in a separate thread with clear timeouts and support for fallbacks, with good monitoring etc. Read "Problem Definition" on the page to understand the problem it tries to solve.

Favorite Quotes

if you build something that is fundamentally broken it isn't really interesting that you followed the plan or you followed some methodology -- the thing you built is fundamentally broken.

• James Roper: Scaling Scala vs Java (recommended by M. Odersky) - writing scalable apps in Scala is much easier then Java because idiomatic Scala uses immutable structures and lends itself naturally to asynchronous processing while doing these things in Java is possible but very unnatural and laborious. "It [Scala] is biased towards scaling, it encourages practices that help you scale."
• Exception Handling Antipatterns (2006, still valuable) - Log and Throw, Throwing Exception (instead of a suitable subclass), Throwing the Kitchen Sink (declaring many exceptions in method signature), Catching Exception (instead of a particular subclass), Destructive Wrapping (not including the exception as cause), Log and Return Null, Catch and Ignore (swallowing the exception), Throw from Within Finally, Multi-Line Log Messages (via repeated log calls instead of \n), Ignoring InterruptedException (instead of breaking the loop etc.), Relying on getCause().
• The GitHub way: Your team should work like an open source project - a provocative article about the development process in GitHub that strongly prefers asynchronous and on-line communication over face-to-face meetings and communication, which, according to the author, leads to increased productivity. That is quite the opposite of what is usually practiced. I can think of situation where direct interaction is invaluable but, on the other hand, I could certainly live with less meetings. (Comments on Hacker News)

Clojure Corner

• Chas Emerick's screencast Starting Clojure is a great example of Clojure development and interactive Clojure web development without restarts, with live code changes and direct access to the running app via REPL. It makes also a good job of introducing the Eclipse Clojure plugin Counterclockwise and the popular web framework Compojure with the template engine Enlive and HTTP abstraction Ring. Highly recommended! (I would however recommend to already know a little about the language.)
• Results of the 2012 State of Clojure survey (and, for comparison, 2010 results) - some interesting facts are what people use Clojure for (math / data analysis 35%, web development 70%), 60% people evaluating ClojureScript, answers to "What have been the biggest wins for you in using Clojure?", the fact that ~ 20% use Eclipse, around 60% Emacs, only 10% IntelliJ, 23% vim. Also interesting is "What has been most frustrating for you in your use of Clojure" (with 30% mentions of documentation, being now improved by clojure-doc.org, 23% "future stuffing concerns")

Favorite Quotes

You can reach a point with Lisp where, between the conceptual simplicity, the large libraries, and the customization of macros, you are able to write only code that matters.

• David Veksler: Some lesser-known truths about programming - things newcomers into the field of IT don't know and don't expect, true and an interesting read. Not backed by good data but anyway. F.ex.: "[..] a programmer spends about 10-20% of his time writing code [..] much of the other 90% thinking, researching, and experimenting". "A good programmer is ten times more productive than an average programmer. A great programmer is 20-100 times more productive than the average [..]" "Bad programmers write code which lacks conceptual integrity, non-redundancy, hierarchy, and patterns, and so is very difficult to refactor." "Continuous change leads to software rot, which erodes the conceptual integrity of the original design." "A 2004 study found that most software projects (51%) will fail in a critical aspect, and 15% will fail totally."
• Brett L. Schuchert: Modern Mocking Tools and Black Magic - An example of power corrupting - interesting for two reasons: a good analysis of a poorly written piece of code and discussion of the code injection black magic (JMockIt) vs. actually breaking dependencies to enable tests.  The author presents a typical example of low-quality method (mixing multiple concerns, mixing different levels of abstractions, untestable due to a hardcoded use of an external call) and discusses ways to improve it and to make it testable. Recommended to read.
• It’s Not About the Unit Tests - Learning from iOS Developers: iOS developers don't do much testing yet they manage to produce high quality. How is that possible? The key isn't testing itself, but caring for the code. (Of course, iOS is little special: small apps, no legacy, a powerful platform that does lot for the apps, very visual apps.) "It’s not about the practices. It’s about the spirit and intent behind them, and how they are applied." (M. Fowler had a similar observation about a team that used mock-based testing exclusively and thus lacked integration tests yet all worked. [I've lost the link to the post and would be grateful for it])
• Java Code Quality Tools – Overview - brief descriptions of 44 quality-related tools including some interesting tools and Eclipse plugins I didn't know or knew but forgot. F.ex. analysis of dependencies with JBoss Tattletale or JarAnalyzer, Clirr to check libraries for source and binary backwards compatibility, JDiff generates JavaDoc-based report of removed/added/changed in an API. Spoon - read and check or transform Java code. Java PathFinder (NASA) - special JVM capable of checking all execution path to discover concurrency defects etc.

Tools

• DirB, Directory Bookmarks for Bash (home) - moving efficiently among favourite directories (s <name> to create a bookmark for pwd, g <bookmark | relative/abs dir path> to enter a dir (=> works both for bookmarks and as a replacement for cd); also support for relative path bookmarks & more; sl lists bookmakrs in the last used order) (You might also want to check out Autojump, described in Dec 11; bashmarks is another similar project. Another similar project is rupa's z and j2 and the fish clone z-fish)

Clojure Corner

•  Jon Pither: Clojure at a Bank – Moving from Java -  the justification (productivity, dynamism, FP a better match for the domain) and process behind moving from Java to Clojure with a monolithic 1M LOC Spring/Hibernate app. (Random quotes: "I had used some dynamical languages before and it was quite obvious that we were essentially forcing lots of schema and type definition on to a problem domain that just didn’t want or need it." "[..] it [dependency injection] just looks redundant in retrospect now that I’m working 95% with FP code.") There is also a EuroClojure talk about their experiences one year later (35 min).
• Prismatic's "Graph" at Strange Loop - an interesting desing problem, its solution, and a resulting OSS library. The problem: How to break a large function into independently usable small ones that might depend on each other without ever needing to recompute a value once the function producing is called. The solution: Graph - "Graph is a simple, declarative abstraction to express compositional structure." (Enabling explicit declaration of data dependencies and pluging in different implementations.)
• The Oblong: Blog about 2/3 D game programming in Clojure, starting from scratch (w/o an engine); interesting experiences
• Ironclad: Steam Legions – Clojure game development battle report (the game on Github)
• Building the Wishlisted.org webapp in Clojure - experiences from learning Clojure for real by building a webapp in Noir
• Clojure vs. Scala smackdown ("Just kidding with the title of this post :-)") - a short post with interesting discussion. Dmitri Sotnikov's opinion resonates with me: "I found that for me Clojure wins on simplicity and consistency. While it looks more alien initially, once you learn the basics, you just reuse the same patterns everywhere." Some more comments: "One major concern was maintainability, since it's fairly easy to write very dense code. This turned out to not be a problem in practice. Because Clojure code is written as a tree, refactoring it is very easy." REPL seems to be a big win (applies to Scala too). Scala's type system might get tedious and learning its quirks takes time but there is lot of potential and both have they strong sides.
• Code Fatigue - discussion of the advantages of learning, using, and combining the (many) standard Clojure functions instead of a "basic solution" using recursion etc. The argument is in favor of higher-level code with less complexity in the form of branching, recursion, nested expressions etc. and thus less mental fatigue.

Favorite Quotes

A classic test only cares about the final state - not how that state was derived. Mockist tests are thus more coupled to the implementation [emphasis mine] of a method. Changing the nature of calls to collaborators usually cause a mockist test to break.

1. Run lein pom to generate a Maven pom.xml from project.clj
2. Import the project as a Maven project (File - New Project... - Import project from external model - Maven - browse to the directory - ...)

1. Using Native Libs in Java

Calling Native Libs

1. Do not start the datanode service on the node
2. If you've configured Hadoop to allow only nodes on its whitelist files to connect to it then add it to the file pointed to by the property mapred.hosts but not to the file in dfs.hosts.
3. Otherwise add the node to the DFS' blacklist, i.e. file pointed to by the property dfs.hosts.exclude and execute hadoop dfsadmin -refreshNodes on the namenode to apply it.

• M.Fowler: ORM Hate - Why ORM is actually a good solution - a very valuable article where Fowler opposes the popular trend of criticising Object-Relational Mappers such as Hibernate. Yes, using an ORM is difficult and a leaky abstraction - but that's because the problem of mapping from a rich in-memory object model to a relational store is inherently difficult (and you need to do it with or w/o an ORM tool) and because those last 10-20% of DB access require human intelligence. If you can avoid the need for ORM by using the relational model also in memory or by using a NoSQL database with a data model that fits your in-memory model then it's great to do so but often you can't and then using ORM is the best solution. You certainly don't want to code your own "lightweight" ORM.
• Alan Quayle on WebRTC, the HTML5 standard for in-browser video/text communication - intro & status - this is an exciting technology coming to our browsers. Some quotes: "WebRTC enables applications such as voice calls, video chat, file sharing, messaging, white-boarding, gaming, human computer interaction, etc. without any client or plug-in download to run from a browser using simple HTML and JavaScript APIs.  Real time communications becomes pervasive on the internet. … Essentially any browser becomes a SIP end point, a telephone, an 'open' Skype client, an end point for any real-time communication and control. … Likely by the end of this year we'll see Chrome and Firefox running WebRTC."
• Communicating Sequential Processes: Theory for reasoning about concurrent, interacting processes - an inspirational reading about a much better way to do concurrency than Java threads; "... [CSP] is a language for describing patterns of interaction between concurrent objects. It is supported by an elegant, mathematical theory, a set of proof tools, and an extensive literature." The beauty is that thanks to the theory behind, you can actually reason about the interactions and verify their correctness, contrary to the feared mess of Java threads. CSP is broadly similar to the popular Actors model and is implemented in Occam while it also influenced Erlang's concurrency model and Go. The library JSCP brings it to Java. I guess we're better of using Actors due to their popularity and maturity though the mathematical backing of CSP with the potential of formal proofs of correctness is indeed attractive. Any of the two is better than using threads directly because:

  The monitor-threads model provided by Java, whilst easy to understand, proves very difficult to apply safely in any system above a modest level of complexity. One problem is that monitor methods are tightly interdependent, so that their semantics compose in non-trivial ways [...]

• Rich Hickey introduces the Reducers library: simplicity in practice - a beautiful example of simplifying something by taking appart all the unrelated but mingled concerns and focus only on those really needed. Whether you're interested in Clojure or not, you should read the beginning of the post where Hickey explains how the current collection functions based on first (returns 1st element) and rest (returns the remaining ones) mix too many things (ordering, output representation, etc.) and how this "new super-generalized and minimal abstraction for collections" avoids that and thus provides e.g. for doing things in parallel and composing transformation without producing intermediate collections. Beautiful! (PS: I've blogged about more examples of pursuing simplicity & gaining power.)
• M. Fowler: Cannot Measure Productivity - a thoughful discussion of why the productivity of programmers is hard/impossible to measure (i.e. you should concentrate on measuring other, more useful metrics) "[..] false measures only make things worse."
• Gojko Adzic: Redefining software quality - an obligatory read that introduces a holistic view of SW quality and the quality pyramid. The key idea is that there are multiple, vertically organized facets of quality and once a more basic facet is saturated, you should move and and concentrate on the next facet and level of quality. The quality pyramid: Deployable & functionally OK > Performant & secure > Usable > Useful > Successful. Once a particular level is satisfied, it is wasteful to put more effort into it and you'll bring much more value to the customer by focusing on the next higher level. Gojko: "Yet from what I see most software teams invest, build and test only at the lowest two levels, gold-plating things without a way to explain why that is bad."
• Is Pair Programming for Me? - the author, who claims to have taught pair programming to 200+500 people, points out that pair programming is a skill that must be (consciously) learned, or actually a number of inter-personal skills. He also describes the cycle people go through when learning it, including a temporary downswing in productivity and negative view of pairing (therefore people should do it at least for 3-4 weeks to overcome the problems and gain the benefits).

Videos

• Bret Victor: Inventing on Principle (55 min, see at least the first 5 min) - very inspiring! Victor firmly believes that "creators need an immediate connection to what they create" and demonstrates how this can be achieve when coding image rendering, a game, an algorithm, when designing a circuit. After watching it for few minutes you will think: How could we have been working with such crappy tools without realizing how limited they are? Fortunately people started to apply the idea of an immediate connection between code and the result, f.ex. in LightTable and Bikeshed's IDE. On the other hand, there is an evidence that this may be too hard with the current programming languages.
• Eric Ries: Evangelizing for the Lean Startup - entertaining and enriching introduction into an approach for bringing innovation to life - f.ex. in startups - withou failing unnecessary, demonstrated on the example of the author's failed and successful startup. Many innovators fail because they don't realize that their key challenge is that they don't now neither the problem (who are our customers and what they need) nor the solution (the product to satisfy the need) and thus what they need to do is to experiment and learn in the shortest cycles possible. If you wander what the buzz about lean startup is or how to build innovations, this is the ultimite source you should watch. The video has 1h but the first 20-30 min will give you a sufficient overview. The key points summarized by the Iterate lean guru Anders Haugeto: dd

  - There is only one way to measure progress: Progress == The amount of things you have learned from your real customers - Hence, you need to work a continuous loop to build, measure and learn as fast as possible. Typical iterations, like sprints, are too long, hence inefficient - Until you have an established product, even recognized engineering practices like TDD, sprints, refactoring, and all the XP-stuff are less important than this feedback cycle - Even the perfect agile method is nothing, if you're using it to build the wrong product: How can you know you are heading in the right direction?

• WebRebels 2012 conference talks - I'd especially recommend the awakening talk by Zed Shaw pointing out that we're building amazing things - but on top of crapy technologies without realizing anymore that the technologies are crappy and could/should be much better. Erlend Oftedal's talk about webapp security was an (scary) eye-opener for me. If you're considering offline webapps with HTML5's webapp cache and/or local storage then you must listen to Jake Archibald's painful story of various pitfalls hidden there. Christian Johansen's Pure, functional JavaScript is a pleasure to listen to. Check out the program.

Links to Keep

• E. King: Maximizing the Value of Your Stand-up - interesting techniques to try out at your stand-ups - Speed Scrum, Pass-the-Conch Scrum (passing a token randomly to define the order), Time-Box Scrum, Challenge Scrum (the team may ask 1 question each presenter), Impediments-Only Scrum, Award Scrum (reward for best articulation of his/her information), Business Value-Focused Scrum, No-Board Scrum, Whiteboard Scrum, Buddy Scrum (report for sb. else)

Useful Tools

• puppet-lint - check code style of your Puppet files
• Guard - cross-platform tool that can watch for file changes and execute actions ("guards") when a file is changed, useful e.g. to execute tests/style checks only on the files being modified. Includes support for many testing/checking tools and multiple notification means such as Growl.
• ThreadLogic - Thread dump analyzer that understands common patterns found in application servers and enabling the definition of custom patterns. Supports Sun, IBM, and JRockit.
• Dumbster - mock SMTP server for unit testing (start in @Before, get sent messages in the test, stop afterwards)

Quotes

Accurate estimation is impossible for complex technical projects, but keeping to agreed budgets, and deadlines is achievable by using feedback and change.

Clojure Corner

• StackOverflow: Clojure Performance Benchmarks - links to various discussions and benchmarks (beware that older results and facts are likely to be outdated). And of course you must keep in mind that 1) benchmark only measure what they measure, e.g. the outcomes cannot be generalized and that 2) benchmark results aren't relevant for your problem unless you're doing exactly the kind of operations being benchmarked (e.g. who cares that X is 100 ms slower if your code spends 1s waiting for a XML file download?) (Craig Andera had a pretty good experience report from webapp performance testing including what (not) to do)
• A good wrapup of the EuroClojure conference by Deon Garrett. Such a pity I missed it!
• Goldberg (at GitHub) - Johann Sebastian Bach's Goldberg Variations in Overtone by @ctford; using Overtone and Clojure to build up mathematical and functional definitions of canons. Deon Garrett: "Go right now and download the code from Chris’ talk. If you don’t know Clojure, use this as an excuse to learn it – it’s that good."

Reducing incidental complexity is a primary focus of Clojure, and you could dig into how it does that in every area.

• ThoughtWorks Technology Radar 3/2012 - including apps with embedded servlet containers (assess), health check pages for webapp monitoring, testing at the appropriate level (adopt), JavaScript micro-framewors (trial, see Microjs.com), Gradle over Maven (e.g. thanks to flexibility), OpenSocial for data & content sharing between (enterprise) apps (assess), Clojure (before in asses) and CoffeeScript on trial (Scala very close to adopt), JavaScript as a 1st class language (adopt), single-threaded servers with aync I/O (Node.js, Webbit for Java [http/websocket], ...; assess).
• Jez Humble: Four Principles of Low-Risk Software Releases - how to make your releases safer by making them incremental (versioned artifacts instead of overwritting, expand & contract DB scripts, versioned APIs, releasing to a subset of customers first), separating software deployment from releasing it so that end-users can use it (=> you can do smoke tests, canary releasing, dark launching [feature in place but not visible to users, already doing something]; includes feature toggles [toggle on only for somebody, switch off new buggy feature, ...]), delivering features in smaller batches (=> more frequently, smaller risk of any individual release thanks to less stuff and easier roll-back/forward), and optimizing for resiliance (=> ability to provision a running production system to a known good state in predictable time - crucial when stuff fails).
• The Game of Distributed Systems Programming. Which Level Are You? (via Kent Beck) - we start with a naive approach to distributed systems, treating them as just a little different local systems, then (painfully) come to understand the fallacies of distributed programming and start to program explicitely for the distributed environment leveraging asynchronous messaging and (often functional) languages with good support for concurrency and distribution. We suffer by random, subtle, non-deterministic defects and try to separate and restrict non-determinism by becoming purely functional ... . Much recommended to anybody dealing with distributed systems (i.e. everybody, nowadays). The discussion is worth reading as well.
• Shapes Don’t Draw - thought-provoking criticism of inappropriate use of OOP, which leads to bad and inflexible code. Simplification is OK as long as the domain is equally simple - but in the real world shapes do not draw themselves. (And Trades don't decide their price and certainly shouldn't reference services and a database.)
• Capability Im-Maturity Model (via Markus Krüger) - everybody knows CMMI, but it’s useful to know also the negative directions an organization can develop in. Defined by Capt. Tom Schorsch in 1996, building on Anthony Finkelstein's paper A Software Process Immaturity Model.
• Cynefin: A Leader’s Framework for Decision Making - an introduction into the Cynefin cognitive framework - the key point is that we encounter 5 types of contexts differing by the predictability of effects and each of them requires a different management style, using the wrong one is a recipe for a disaster. Quote:

  The framework sorts the issues facing leaders into five contexts defined by the nature of the relationship between cause and effect. Four of these—simple, complicated, complex, and chaotic—require leaders to diagnose situations and to act in contextually appropriate ways. The fifth—disorder—applies when it is unclear which of the other four contexts is predominant.

• Et spørsmål om kompleksitet (Norwegian). Key ideas mixed with my own: Command & control management in the traditional Ford way works very well - but only in stable domains with clear cause-and-effect relationships (i.e. the Simple context of Cynefin). But many tasks today have lot of uncertanity and complexity and deal with creating new, never before seen things. We try to lead projects as if they were automobile factories while often they are more like research - and researchers cannot plan when they will make a breakthrough. Most of the new development of IT systems falls into the Complex context of Cynefin - there is lot of uncertanity, no clear answers, we cannot forsee problems, and have to base our progress on empirical experience and leverage emergence (emergent design, ..).
• The Economics of Developer Testing - a very interesting reflection on the cost and value of testing and what is enough tests. Tests cost to develop and maintain (and different tests cost differently, the more complex the more expensive). Not having tests costs too - usually quite a lot. To find the right ballance between tests and code and different types of tests we must be aware of their cost and benefits, both short & long term. Worth reading, good links. (Note: We often tend to underestimate the cost of not having good tests. Much more then you might think.)

Links to Keep

• Proved Patterns for Creating Responsive Web UIs (adapting to the user device size - PC, mobile, ..): Mostly Fluid (fluid grids, down-scaling images,decreasing margins and eventually stacking columns below each other on smaller screens), Column Drop, Layout Shifter (diff. layout for large/medium/small screens), ... .
• 16 Linux Server Monitoring Commands You Really Need To Know - some elementary such as top, iostat, and netstat and some more dashboard-like such as nmon (Nigel's Monitor) and the stats collecting sar.

Quotes

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it. I do tend to make sense of test errors, so I'm extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

• List of open source projects at Twitter including e.g. their scala_school - Lessons in the Fundamentals of Scala and effectivescala - Twitter's Effective Scala Guide
• M. Fowler & P. Sadalage: Introduction into NoSQL and Polyglot Persistence (pdf, 11 slides) - what RDBMS offer and why it sometimes isn't enough, what the different NoSQL incarnations offer, how and on which projects to mix and match them
• Two phase release planning - the best way to plan something somehow reliably is to just start doing it, i.e. just start the project with the objective of answering "Can this team produce a respectable implementation of that system by that date?" in as short time as possible (i.e. few weeks). Then: "Phase 2: At this point, there’s a commitment: a respectable product will be released on a particular date. Now those paying for the product have to accept a brute fact: they will not know, until close to that date, just what that product will look like (its feature list). What they do know is that it will be the best product this development team can produce by that date." Final words: "My success selling this approach has been mixed. People really like the feeling of certainty, even if it’s based on nothing more than a grand collective pretending."
• Tumblr Architecture - 15 Billion Page Views A Month And Harder To Scale Than Twitter - what SW (Scala, Finagle, heavily partitioned MySQL, ...) and HW they use, the architecture (Firehose - event bus, cell design), lessons learned (incl. "MySQL (plus sharding) scales, apps don't."
• Jay Fields' Thoughts: Compatible Opinions on Software - about teams and opinion conflicts - there are some areas where no opinion is really right (e.g. powerful language vs. powerful IDE) yet people may have very strong feeling about them. Be aware of what your opinions are and how strong they are - and compose teams so that they include more less people with compatible (not same!) opinions - because if you team people with strong opposing opinions, they'll loose lot of productivity. Quotes: "I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software." "I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost."
• Jay Fields' Thoughts: Lessons Learned while Introducing a New Programming Language (namely Clojure) - introducing a new language and winning the hearts of (sufficient subset of) the people is difficult and requires lot of extra effort. This is both an experience report and a pretty good guide for doing it.
• Jay Fields' Thoughts: Life After Pair Programming - a proponent of pair-programming comes to the conclusion that in some contexts pairing may not be beneficial, i.e. the benefits of pair-programming don't overweight the costs (for a small team, small software, ...)
• The Why Monitoring Sucks (and what we're doing about it) - the #monitoringsucks initiative- what tools there are, why they suck, what to do, new tools, what metrics to collect, blogs, ...
  • Related: Getting started with the Sensu monitoring framework
• JBoss Byteman 2.0.0: Bytecode Manipulation, Testing, Fault Injection, Logging - a Java agent which helps testing, tracing, and monitoring code, code is injected based on simple scripts (rules) in the event-condition-action form (the conditions may use counters, timers etc.). Contrary to AOP, there is no need to create classes or compile code. "Byteman is also simpler to use and easier to change, especially for testing and ad hoc logging purposes." "Byteman was invented primarily to support automation of tests for multi-threaded and multi-JVM Java applications using a technique called fault injection." It was used e.g. to orchestrate the timing of activities performed by independent threads, for monitoring and statistics gathering, for application testing via fault injection. Contains a JUnit4 Runner for easily instrumenting the code under test, it can automatically load a rule before a test and unload it afterwards:

  @Test
  @BMRule(name="throw IOException at 1st call",
  targetClass = "TextLineProcessor",
  targetMethod = "processPipeline",
  action = "throw new java.io.IOException()")
  public void testErrorInPipeline() throws Exception { ... }

• How should code search work? - a thought-provoking article about how much better code completion could be if it profited more from patterns of usage in existing source codes - and how to achieve that. Intermediate results available in the Code Recommenders Eclipse plugin.

• What Makes Jersey Interesting: Parameter Classes (by Coda Hale, 5/2009) - brief yet rich and very practical introduction into Jersey (the reference implementation of JAX-RS. i.e. REST, for Java) including error handling, parameter classes (automatic wrapping of primitive values). The following article, What Makes Jersey Interesting: Injection Providers, might be of interest too.
• How to GET a Cup of Coffee, 10/2008 - good introduction into creating applications based on REST, explained on an example of building REST workflow for the ordering process in Starbucks - a "self-describing state machine". The advantage of this article is that it presents the whole REST workflow with GET, OPTIONS, POST, PUT and "advanced" features such as the use of If-Unmodified-Since/If-Match, Precondition Failed, Conflict. The workflow steps are connected via the Location header and a custom <next> link tag with rel and uri. Other keywords: etag, microformats, HATEOS (-> derive the next resource to access from the links in the previous one), Atom and AtomPub, caching (web trades latency for scaleability; if 1+s latency isn't acceptable than web isn't the right platform), URI templates (-> more coupling than links in responses), evolution (-> links from responses, new transitions), idempotency. "The Web is a robust framework for integrating systems at local, enterprise, and Internet scale."

Links to Keep

Tools, Libraries etc.

• ClusterSSH - whatever commands you execute in the master SSH session are also execute in the slave sessions - useful if you often need to execute the same thing on multiple machines (requires Perl); to install on Mac: "brew install csshx"
• HTML5 Boilerplate (H5BP) - customizable initial HTML5 project template for a website; can be combined e.g. with Bootstrap, the HTML/JS/CSS toolkit (there is even a script to set them both up). Includes server configs for optimal performance, "delivers best practices, standard elements".
• High performance libraries in Java - disruptor, Java Chronicle (ultra-fast in-memory db), Colt Matrix library (scientific computations), Javolution (RT Java), Trove collections for primitives, MG4J (free full-text search engine for large document collections), some serialization & other banchmarks links.
• Twitter Finagle - "library to implement asynchronous Remote Procedure Call (RPC) clients and servers. Finagle is flexible enough to support a variety of RPC styles, including request-response, streaming, and pipelining; for example, HTTP pipelining and Redis pipelining. It also makes it easy to work with stateful RPC styles; for example, RPCs that require authentication and those that support transactions." Supports also failover/retry, service discovery, multiple protocol (e.g. http, thrift). Build on Netty, Java NIO. See the overview and architecture.
• Eclipse Code Recommenders - interesting plugin in incubation that tries to bring more more intelligent completion based more on context and the wisdom of the crowds (i.e. patterns of usage in existing source codes) to Eclipse

Clojure Corner

• Clojure/huh? - Clojure's Governance and How It Got That Way - an interesting description how the development of Clojure and inclusion of new libraries is managed. "Rich is extremely conservative about adding features to the language, and he has impressed this view on Clojure/core for the purpose of screening tickets." E.g. it took two years to get support for named arguments - but the result is a much better and cleaner way of doing it.
• Clojure Monads Series - comprehensive explanations of monads starting with Monads In Clojure

Quotes

A language that doesn't affect the way you think about programming, is not worth knowing-

 - Alan Perlis

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

Eric S. Raymond, "How to Become a Hacker"

@RunWith(BMUnitRunner.class)
public class BytemanJUnitTests {
@Test(expected=MyServiceUnavailableException.class)
   @BMRule(name="throw timeout at 1st call",
   targetClass = "Socket",
   targetMethod = "connect",
   action = "throw new java.io.IOException()")
   public void testErrorInPipeline() throws Exception {
      // Invokes internally Socket.connect(..):
      new MyHttpClient("http://example.com/data").read();
   }
}

1. Aligned business, software, and test models => small change in business requires only a similarly small change in the software and a small change in tests (Gojko Adzic explains that very well in his JavaZone 2012 talk Long-term value of acceptance tests)
   • The key to gaining the alignment is to use business language in all the three models from the very start, building them around business concepts and relationships
2. Testing under the surface level, if possible
   • Prefer to test your application via the service layer or at worst the servlet layer; only test on the UI level if you really have to and only as little as possible for UI is much more brittle (and also difficult to test)
   • The more you want to test the more you have to pay for it in the terms of maintenance effort. Usually you decide so that you cover the part(s) of the application where the most risk is - the best thing is to do cost-benefit evaluation.
3. Isolating tests from implementation by layers of test abstraction
   • Top layer: Acceptance tests should only describe "what" is tested and never "how" to test it. You must avoid writing scripts instead of specifications.
   • Layer 2: Instrumentation - right below the acceptance test is an instrumentation layer, which extracts input/output data from the test and defines how to perform the test via a high-level API, provided by the next level (we could say a test DSL) such as "logInUser(X); openAccountPage();"
   • Layer 3: High-level test DSL: This layer contains all the implementation details and exposes to the higher layer high-level primitives that it can use to compose the tests without depending on implementation details (ex.: logInUser may use HtmlUnit to load a page, fill a form, post it). See the PageObject example below.

1. Tests tell a story
2. True unit tests + decoupled higher-level integration tests (-> Mike Cohn's Layers of the Test Automation Pyramid)
3. More functional composition of the processing

Extract

• Steve Yegge's Execution in the Kingdom of Nouns - I guess you've already read this one but if not - it is a well-written and amusing post about why not having functions as first class citizens in Java causes developers to suffer. Highly recommended.
• Reply to Comparing Java Web Frameworks - a very nice and objective response to a recent blog summarizing a JavaOne presentation about the "top 4" web frameworks. The author argues that based on number of resources such as job trends, StackOverflow questions etc. (however data from each of them on its own is biased in a way) JSF is a very popular framework - and rightly so for even though JSF 1 sucked, JSF 2 is really good (and still improving). Interesting links too (such as What's new in JSF 2.2?). Corresponds to my belief that GWT and JSF are some of the best frameworks available.
• Using @Nullable - use javax.annotation.Nullable with Guava's checkNotNull to fail fast when an unexpected null appeares in method arguments
• JavaOne 2011: Migrating Spring Applications to Java EE 6 (slides) - nice (and visually attractive) comparison of JavaEE and Spring and proposal of a migration path. It's fun and worthy to see.
• xUnitPatterns - one of the elementary sources that anybody interested in testing should read through. Not only it explains all the basic concepts (mocks, stubs, fakes,...) but also many pitfalls to avoid (various test smells such as fragile tests due to Data Sensitivity, Behavior Sensitivity, Overspecified Software [due to mocks] etc.), various strategies (such as for fixture setup), and general testing principles. The materials on the site were turned into the book xUnit Test Patterns: Refactoring Test Code (2007), which is more up-to-date and thus a better source.
• Eclipse tip: Automatically insert at correct position: Semicolon, Braces - in "while(|)" type "true {" to get "while(true) {|" i.e. the '{' is moved to the end where it belongs, the same works for ';'
• Google Test Analytics - Now in Open Source - introduces Google's Attributes-Components-Capabilities (ACC) application intended to replace laborous and write&forget test plans with something much more usable and quicker to set up, it's both a methodology for determining what needs to be tested and a tool for doing so and tracking the progress and high-risk areas (based not just on estimates but also actual data such as test coverage and bug count). The article is a good and brief introduction, you may also want to check a live hosted version and a little more detailed explanation on the project's wiki.
• JSF and Facelets: build-time vs. render-time (component) tags (2007) - avoid mixing them incorrectly
• StackOverflow: What are the main disadvantages of Java Server Faces 2.0? Answer: The negative image of JSF comes from 1.x, JSF 2 is very good (and 2.2 is expected to be just perfect :-)). Nice summary and JSF history review.
• Ola Bini: JavaScript in the small - best practices for projects using partly JavaScript - the module pattern (code in the body of an immediately executed function not to polute the global var namespace), handling module dependencies with st. like RequireJS, keeping JS out of HTML, functions generating functions for more readable code, use of many anonymous functions e.g. as a kind of named parameters, testing, open questions.

Talks

• Kent Beck's JavaZone talk Software G Forces: The Effects of Acceleration is absolutely worth the 1h time. Kent describes how the development process, practices and partly the whole organization have to change as you go from annual to monthly to weekly, daily, hourly deployments. What is a best practice for one of these speeds becomes an impediment for another one - so know where you are. You can get an older version of the slides and there is also a detailed summary of the talk from another event.
• Rich Hickey: Simple Made Easy - Rich, the author of Clojure, argues very well that we should primarily care for our tools, constructs and artifacts to be "simple", i.e. with minimal complexity, rather than "easy" i.e. not far from our current understanding and skill set. Simple means minimal interleaving - one concept, one task, one role, minimal mixing of who, what, how, when, where, why. While easy tools may make us start faster, only simplicity will make it possible to keep going fast because (growing) comlexity is the main cause of slowness.  And simplicity is a choice - we can create the same programs we do today with the tools of complexity with drastically simpler tools. Rich of course explains what, according to him, are complex tools and their simple(r) alternatives - see below. The start of the 1h talk is little slow but it is worth the time. I agree with him that we should much more thing about the simplicity/complexity of the things we use and create rather than easiness (think ORM). Read also Uncle Bob's affirmative reaction ("All too often we do what’s easy, at the expense of what’s simple. And so we make a mess. [...] doing what is simple as opposed to what is easy is one of the defining characteristics of a software craftsman.").

Random Notes from Rich's Simple Made Easy Talk:

I like writing unit tests but Java doesn't make it particularly easy. Especially if you need to create objects and object trees, transform objects for checking them etc. I miss a lot a conscise, powerful syntax, literals for regular expressions and collections, conscise, clojure-based methods for filtering and transforming collections, asserts providing more visibility into why they failed. But hey, who said I have to write tests in the same language as the production code?! I can use Groovy - with its syntax being

Few examples

def testBean = new Customer(fname: "Bob", sname: "Newt", age: 42)
// Java: c = new Customer(); c.setFname("Bob"); c.setSname("Newt"); c.setAge(42);

assert test.method() == new File("expected.txt").getText()
// Java: buffered reader line by line ...; Note: == actually uses equals()

assert customerFinder.findAll().collect {it.sname}.sort() == ["Lizard","Newt"]
// Java: too long to show here (extract only surnames, sort them, compare ...)
assert getCapitalsMap() == ["UK" : "London", "CR" : "Prague"]

assert ("dog1-and-dog2" =~ /dog\d/).getAt([0,1]) == ["dog1", "dog2"]

• Or more fail-safe regexp:

  
  assert ("dog1-and-dog2" =~ /dog\d/).iterator().toSet() == ["dog1", "dog2"].toSet()
  

• With a match group:

  
  assert ("dog11-and-dog22" =~ /dog(\d+)/).iterator().collect({it[1]}).toSet() == ["11", "22"].toSet()