Holy on Dev

Archive for December, 2011

Most interesting links of December

Posted by Jakub Holý on December 31, 2011

Recommended Readings

  • The Netflix Chaos Monkey – how to test your preparedness for dealing with a system failure so that you won’t experience nasty wakeup when something really fails in Sunday 3 am? Release a wild, armed monkey into your datacenter. Watch carefuly what happens as it randoly kills your instances. This is exactly what Netflix does with their with their cloud infrastructure – also a great inspiration for my recent project. Do you need to be always available? Than consider employing the chaos monkey – or a whole army of monkeys!
    (PS: There is also a post with a picture of the scary monky.)

Links to Keep

  • BDD: Write specifications, not scripts (from the Concordion site) – relatively brief yet very enriching practical description of how to do behavior-driven development a.k.a. Specification by Example right, the key point here being “Write specifications, not scripts.” It says why not (for scripts overspecify -> are brittle, specs should be stable) and how to do it (decouple the stable spec and the volatile system via fixture code, expose minimal stuff to the spec, perhaps evolve a DSL between the fixture and the system). It also lists common “smells” of BDD done wrong. If it still isn’t clear to you, read the Script to Specification Makeover example (or perhaps read it anyway). BTW, Concordion is a new tool for doing BDD based on JUnit and HTML, which was created as a response to the weaknesses of Fit[Nesse], i.e. exactly the tendency to do scripting instead of specifications. It looks very promissing to me!

SW Utilities

  • (Linux/Mac) Autojump – superfast navigation between favorite directories in the command line (via Jake McCrary) – it keeps track of how much time you spend in each directory and when you execute j <substring of directory path/name>, it jumps into the most frequently used one matching the substring. It awesome! (You can also run jumpstat to see the statistics.)
  • (Linux) Tcpkill – service/network outage testing (via Jake McCrary) – kill connections to or from a particular host, network, port, or combination of all – useful e.g. when you want to test that your software is resilient to the outage of a particular service or server – less brutal than actually killing the database etc. instances. We need to test that our application recoveres properly when one of our MongoDB nodes dies so this may be quite useful.
  • Manik Hot Deploy Plugin for Maven Projects (v1.0.2 in 5/2011; older version in the Marketplace) – plugin that can do hot and incremental deployment to any app server (simply by copying to its hotdeploy directory or the directory of an installed webapp) whenever you run mvn install or automatically whenever sources change, multi-module support

Clojure Corner

  • Jake McCrary: Continuous Testing With Clojure and Expectations – continuous test runner lein-autoexpect for Clojure tests written using the library expectations by Jay Field.
  • Jake McCrary: Quickly Starting a Powerful Clojure REPL – Clojure REPL only two steps away: 1) Run Emacs, 2) Execute M-x clojure-swank (no more need to open an existing Leinigen project) – the trick is to install the Leinigen plugin swank-clojure and use Jake’s elisp function clojure-swank that automatically starts the swank-clojure server. (I had to hack the function for the clojure-swank output contained “null” instead of “localhost”, likely due to incorrect DNS setup.)

Posted in eclipse, General, Testing, Top links of month | Tagged: , , , , , , , | Comments Off on Most interesting links of December

AWK: Extract Logs for the Given Date(s) from a Log File

Posted by Jakub Holý on December 18, 2011

If your log file has entries like these:

2011-12-10T22:00:27.996+0000 [http-8080-1] INFO  my.package.MyClass Hello, I'm alive!
2011-12-11T17:05:46.811+0000 [http-8080-15] ERROR my.package.MyClass  - Error caught in DispatcherServlet
        at my.package.MyServiceClass(MyServiceClass.java:36)
2011-12-11T17:06:10.120+0000 [http-8080-14] DEBUG my.package.MyClass Whoo, that has been a long day!

Then you can use the following bash script snippet to extract logs only for a particular day or consecutive days, including everything – even lines not starting with the date such as stacktraces – between the first log of the date up to the first log of a subsequent date (default: yesterday):

if [ -z "$LOGDAY" ]; then LOGDAY=$(date +%F -d "-1 days"); fi
if [ -z "$AFTERLOGDAY" ]; then AFTERLOGDAY=$(date +%F -d "$LOGDAY +1 days"); fi
echo "Extracting logs in the range (>= $LOGDAY && < $AFTERLOGDAY) into $LOGFILE ..." awk "/^$LOGDAY/,/^$AFTERLOGDAY/ {if(!/^$AFTERLOGDAY/) print}" $LOGFILE_ORIG > $LOGFILE

This date format works on Linux. Date is very flexible and can provide dates in any format, not only yyyy-mm-dd. You may also want to read more about Awk ranges and other tips.

You would run it in one of the following ways:

$ ./analysis.sh /path/to/logfile.log
$ LOGDAY=2011-12-12 AFTERLOGDAY=2011-12-17 ./analysis.sh /path/to/logfile.log

Posted in General, Tools | Tagged: , , | Comments Off on AWK: Extract Logs for the Given Date(s) from a Log File

Quiz: What’s the Best Test Method Name?

Posted by Jakub Holý on December 13, 2011

Which of the following names of test methods do you think to be the best?

(Notice that we could leave out “payment_” from the last name if it is clear from the context, i.e. from the fixture [a fancy name for test class] name.)

According to the holy book of Clean Code, the code should make visible the intent as much as possible. According to the testing guru Kent B., a test should be telling a story to its reader – a story about how the code should be used and function. According to these two and my own experiences from reading a lot of (test) core written by other people, the last one is absolutely the best. However you have the right to disagree and discuss 🙂

PS: I firmly believe that calling a test method “test()” should be punishable.

Posted in General, Languages, Testing | Tagged: , , | 3 Comments »

Getting Started with Amazon Web Services and Fully Automated Resource Provisioning in 15 Minutes

Posted by Jakub Holý on December 7, 2011

While waiting for a new project, I wanted to learn something useful. And because on many projects we need to assess and test the performance of the application being developed while only rarely there is enough hardware for generating a realistic load, I decided to learn more about provisioning virtual machines on demand in the Cloud, namely Amazon Web Services (AWS). I’ve learned a lot about the tools available to work with AWS and the automation of the setup of resources (machine instances, security groups, databases etc.) and automatic customization of virtual machine instances in the AWS cloud. I’d like to present a brief introduction into AWS and a succinct overview of the tools and automation options. If you are familiar with AWS/EC2 then you might want to jump over the introduction directly to the automation section.

Read the rest of this entry »

Posted in General, Testing, Tools | Tagged: , , , , | 1 Comment »