The power of batching or speeding JDBC by 100 We all know that one coarse-grained operation is more efficient than a number of fine-grained ones when communicating over the network boundary but until recently I haven't realized how big that difference may be. While performing a simple query individually for each input record proceeded with the speed of 11k records per hour, when I grouped each 100 queries together (with "... WHERE id IN (value1, .., value100)), all 200k records were processed in 13 minutes. In other words, using a batch of the size 100 led to the speed-up of nearly two orders of magnitude! Exposing a POJO as a JMX MBean easily with Spring JMX is a great way to check or change state variables or invoke a method in a (remote) running application via a management GUI such as JConsole. And Spring makes it trivial to expose any POJO as a JMX MBean with only little configuration in a few minutes. The Spring JMX documentation is very good, however there are few points that I struggled with for a while and would therefore like to record here the right solutions. I needed to monitor a command-line java application using Spring 2.5 on IBM JVM 1.4 1.5 running on a server with a jconsole on Sun JVM 1.6 as the JMX client on my PC. Implementing retrial with a MDB or an MQ batch job? (WAS 7, MQ 6) We need to listen for messages distributed via Websphere MQ to get informed when an employee joins or leaves IBM. And because the resources used in the processing (a database, a webservice) may be temporarily unavailable, we must be able to deal with such outages, which may range from minutes to hours, by repeatedly retrying the processing after some delay. And we must be also able to deal with "poison messages", that means messages whose processing always fails either because their content is invalid or because their data isn't consistent with the database. The question is whether this would be better implemented as a Message-Driven Bean (MDB) or a batch job regularly checking its queue given that we have Websphere Application Server 7 (and thus Java EE 5) and Websphere MQ 6, which both have some important changes compared to the previous versions. It turns out that it depends - both approaches have some advantages and disadvantages and so it's a question of the likelihood of particular problems and business requirements and priorities.
SSH magic: Authorize only once for multiple ssh/scp invocations OpenSSH has a nice feature that makes it possible to open one "master connection", which can be shared by multiple subsequent ssh/scp/sftp "slave connections". The advantage is that you need to supply password only when opening the master connection and thus you can easily perform a sequence of remote commands without constant re-authentication. Let's see how to do it in such a way that it can be used in a script.
Jetty-maven-plugin: Running a webapp with a DataSource and security This post describes how to configure the jetty-maven-plugin and the Jetty servlet container to run a web application that uses a data source and requires users to log in, which are the basic requirements of most web applications. I use Jetty in development because it's fast and easy to work with. Most interesting links of August I hope everybody is enjoying the holiday and not spending hours on tech blogs and sites. At least I do :-) and thus this month's list is a short one:
Working With Static Imports in Eclipse - how to make working with static imports (nearly) as easy as with the normal ones (especially useful for fluent interfaces and "DSLs"), mainly by adding types like JUnit's Assert and Mockito to your favorite impo An encrypted backup of a disk/partition to a Samba share with Clonezilla You will learn how to customize Clonezilla Live (v. 1.2.5-24) for an easy backup of a partition (or a disk) to an encrypted file stored on a remote Samba server and how to test the backup by restoring it to a VMware virtual machine. We will few scripts to simplify the task, including a custom Clonezilla startup script to mount a TrueCrypt volume on a Samba share. Content: What is Clonezilla? | Preparing Clonezilla for a custom backup | Backup | Encryption of the backup | Restoration and testing of the backup | The complete backup - encrypt - test cycle | Summary PS: If you are scared by the length of this post then read only "The complete backup – encrypt – test cycle" :-) Update 2010-09-23: Added "The complete backup – encrypt – test cycle", little reorganization.
What is Clonezilla?
Clonezilla is a live Linux distribution containing tools for performing backup and restoration of disks and partitions. It is basically a collection of various open-source tools such as partimage and gzip and custom scripts that "glue" them together to create a single backup tool driven by a wizard-like user interface. You install it to a CD or USB flash disk, boot from that medium, answer few questions and a backup or a restoration may start. Most interesting links of July This month about performance, the Java language and patterns, the development process, and a few interesting news. Mobile learning application MiniPauker 1.1.05 released - please test! My path to SCEA 5 I'd like to share with you my experience with the Sun Certified Enterprise Architect for the Java Platform, Enterprise Edition 5 (SCEA 5) [ 1] certification. There was a couple of unclear things regarding the assignment and its deliverables and I've learned some interesting things (mostly about hardware estimation and deployment environments such as the "clouds"), both of which may be of an interest to somebody aiming at this certification. I only know that I passed but not how well, so my way of doing things, though sufficient, may not be the best one. Most interesting links of June The last month's list of interesting articles is a bit shorter due to off-line holidays, which I enjoyed much more then reading articles :-)
Boy Scout Rule in practice - improving the code on the go with respect to the Clean Code's and similar best practices: 1) If you spot a piece of code to improve, don't add Implementing build-time bytecode instrumentation with Javassist If you need to modify the code in class files at the (post-)build time without adding any third-party dependencies, for example to inject cross-cutting concerns such as logging, and you don't wan't to deal with the low-level byte code details, Javassist is the right tool for you. I've already blogged about " Injecting better logging into a binary .class using Javassist" and today I shall elaborate on the instrumentation capabilities of Javassist and its integration into the build process using a custom Ant task. Booting from a USB stick in VMware Player It isn't possible to boot from a USB stick under VMware player 3.0.0, because its BIOS is ignorant of USB, but fortunately there is an easy workaround:
Download the PLoP boot manager (e.g. plpbt-5.0.10.zip), which supports usb booting
Decide whether you want to install PL Webservice testing with JMeter: Passing data from a response to another request JMeter is great for functional and performance testing of many things, including web services (and to my surprise also LDAP). It also provides means for extracting data from a response and passing them to a subsequent request, which is exactly what I needed. There is already a good tutorial on testing a WS with JMeter, so I won't repeat the basic setup here. The steps are:
Create a webservice (WS) test plan, as described in the tutorial (in my case it contains two WS calls)
Add the User Defined Variables config element to the test plan and define there a variable for transferring the response data
Add an XPath Extractor Post Processor to the first WS call to extract the value of interest into the user defined variable (beware namespaces!)
Add a BeanShell Pre Processor to the second call, which will replace a placeholder in the WS call's XML data with the value of that variable
Most interesting links of May The most interesting stuff I've read in May, in no particular order. You can easily guess I've been working on performance troubleshooting this month :-)
NoSQL is About… - all the things NoSql databases are said to be about (and perhaps are not) and a good overview of the different goals and thus also features of the various implementations
Previous (20) Next (4)