[DRAFT] Maven "change" project extending a non-maven web project

April 10, 2009

Disclaimer: I'm rather new to Maven and thus my solution is likely not the best one. I welcome any improvement suggestions.

My current project here at IBA CZ extends a legacy non-maven web project and I had to devise how to organize the extension modules and other new modules depending upon them. The final goal was to create a portlet face for this servlet-only web application that is written in JSP and a strange presentation framework and then create specific portlets for specific needs.

Key factors and motivations:

The source WAR project is not under our control.
The source WAR project has been imported into an external maven repository but only with a minimal Maven POM without any definitions of dependencies etc.
I need to extend this WAR to work in our specific environment (Liferay portal) and to satisfy our specific requirements, which will require some but hopefully not many modifications of its source codes and other resources.
I want to be able to migrate my changes easily to a new version of the source WAR.
I want to use Maven because its our preferred build tool, integrates well with our Continuous Integration (CI) server (Hudson), and - last but not least - handles dependencies so that I can execute svn co http://ibacz.eu/my/project && cd project && mvn package to build a working, deployable package.

My idea was to create only a "change" project, which would only include our changes to the original WAR (additions/deletions/modifications) and merge it with the original project fetched from its maven repository to create a final deployable artifact. Then I'd create other artifacts depending on this.

The source artifacts

As I've said, the source application - jpivot.war - is maven agnostic but is imported into an external Maven repository (http://repository.pentaho.org/artifactory/).

Web app artifact: jpivot.war

The web application that we want to customize. It doesn't include any classes directly - they're in WEB-INF/lib/jpivot.jar.

Its pom.xml:

<?xml version="1.0" encoding="UTF-8"?><project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.tonbeller</groupId>
  <artifactId>jpivot</artifactId>
  <packaging>war</packaging>
  <version>1.8.0-081008</version>
  <description>Auto generated POM</description>
</project>

Classes artifact: jpivot.jar

The same jar as in jpivot.war/WEB-INF/lib/jpivot.jar but as a standalone maven artifact, also without any dependencies definitions. Its pom.xml:

<?xml version="1.0" encoding="UTF-8"?><project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.tonbeller</groupId>
  <artifactId>jpivot</artifactId>
  <version>1.8.0-081008</version>
  <description>Auto generated POM</description>
</project>

Handling dependencies: jpivot-libs-combined.jar

For compilation and comfortable development of our change and other projects we will certainly need the source project's dependencies. Unfortunately they're not listed in its maven POM and it would be a tedious task to do that. Also we do not control the WAR's POM and it's likely that some of the dependencies also aren't mavenized, which gives us a nice recursive trouble.

Therefore I'll go for the simplest solution and combine all libraries' classes from jpivot.war/WEB-INF/lib/*.jar excluding jpivot.jar (for it has already its own imported maven artifact) into a single "uberjar" jpivot-libs-combined.jar and deploy it to the company-wide thirdparty maven repository.

An Ant build.xml that merges all JARs except of jpivot.jar from the working directory together:

<?xml version="1.0" encoding="ISO-8859-1"?>
<project name="jpivot-libs-combined" default="combine" basedir=".">
  <target name="combine">
        <zip destfile="jpivot-libs-combined.jar">
                <zipgroupfileset dir="." includes="*.jar" excludes="jpivot.jar"/>
                <fileset dir="." excludes="*.jar"/>
        </zip>
  </target>
</project>

A pom.xml that shall be included with the combined JAR:

<?xml version="1.0" encoding="UTF-8"?><project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.tonbeller</groupId>
  <artifactId>jpivot-libs-combined</artifactId>
  <version>1.8.0-081008</version>
  <dependencies>
    <dependency>
        <!-- Dummy dependency to show what has this been generated from -->
        <groupId>com.tonbeller</groupId>
        <artifactId>jpivot</artifactId>
        <version>1.8.0-081008</version>
        <type>war</type>
        <scope>provided</scope>
        <optional>true</optional>
    </dependency>
  </dependencies>
  <description>
  Dependencies extraced from jpivot.war/WEB-INF/lib (excluding jpivot.jar that already has a maven artefact) and merged into a single .jar.
  </description>
</project>

Commands to create the uberjar and install it into a maven repository:

/tmp$ unzip /path/to/jpivot.war WEB-INF/lib
/tmp$ ant -f /path/to/the/build.xml
/tmp$ mvn install:install-file -Dfile=jpivot-libs-combined.jar -DgroupId=com.tonbeller -DartifactId=jpivot-libs-combined -Dversion=1.8.0-081008 -Dpackaging=jar -DgeneratePom=true

Now we should deploy the artifact to the company-wide Maven repository for thirdparty stuff and replace the pom.xml generated by the install command with the one provided above.

Our artifacts

Finally the artifacts that we do develop.

A root POM

I like a parent POM for my artifacts that defines common configuration including properties, repositories and SW versions.


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <parent>
        <artifactId>ibacz-root-pom</artifactId>
        <groupId>eu.ibacz.maven</groupId>
        <version>1.1</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <groupId>eu.ibacz.pbns</groupId>
    <artifactId>pbns-root-pom</artifactId>
    <packaging>pom</packaging>
    <name>PBNS - Root project</name>
    <version>0.0.1-SNAPSHOT</version>
    <description/>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <executions>
                    <execution>
                        <id>enforce-versions</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <requireMavenVersion>
                                    <version>2.0.9</version>
                                </requireMavenVersion>
                                <requireJavaVersion>
                                    <version>1.5</version>
                                </requireJavaVersion>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>pentaho</id>
            <name>iba copy of pentaho repo at http://repository.pentaho.org/artifactory</name>
            <url>http://w3.ibacz.cz/nexus/content/repositories/pentaho</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
        <repository>
            <id>thirdparty</id>
            <name>Third party repository</name>
            <url>http://w3.ibacz.cz/nexus/content/repositories/thirdparty</url>
        </repository>
    </repositories>

    <properties>
        <wcf.version>1.8.0-070305</wcf.version>
        <jpivot.version>1.8.0-081008</jpivot.version>
        <jpivot.source.version>1.8.0-081008snapshotsrc</jpivot.source.version>
    </properties>

</project>

The "change" artifacts

As mentioned, these reflect the source artifacts and contain changes to them.

jpivot-classes-customized [jar]

Depends on jpivot.jar and contains classes changed from the original jar. They're combined together for deployment, overriding original classes with the modified ones.

The combination and overriding is done by using the maven-dependency-plugin and its undeploy goal to unpack jpivot.jar to the target compilation prior to running the compile phase, which will thus override original jpivot's classes with our modified ones.


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <parent>
        <artifactId>pbns-root-pom</artifactId>
        <groupId>eu.ibacz.pbns</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <groupId>eu.ibacz.pbns</groupId>
    <artifactId>jpivot-classes-customized</artifactId>
    <name>Customized JPivot Classes</name>
    <version>1.8.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>com.tonbeller</groupId>
            <artifactId>jpivot</artifactId>
            <version>${jpivot.version}</version>
            <type>jar</type>
            <scope>provided</scope> <!-- don't propagate further as it will be merged into this project's jar -->
        </dependency>
        <dependency>
            <groupId>com.tonbeller</groupId>
            <artifactId>jpivot-libs-combined</artifactId>
            <version>${jpivot.version}</version>
            <scope>compile</scope> <!-- with 'provided' it doesn't get as a transitive dep. of this project's dependants. -->
        </dependency>
    </dependencies>
    <description>modified classes of jpivot.jar</description>
    <build>
        <finalName>jpivot-classes-customized</finalName>
        <plugins>
            <plugin>
                <!--
                    Merge this project's classes with the original jpivot.jar. You can
                    safely ignore reports about duplicates - this happens for classes
                    that we override here.
                -->
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.1</version>
                <executions>
                    <execution>
                        <id>unpack</id>
                        <phase>process-resources</phase> <!-- To run before compile and this be overriden by compiled classes -->
                        <goals>
                            <goal>unpack</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>com.tonbeller</groupId>
                                    <artifactId>jpivot</artifactId>
                                    <version>${jpivot.version}</version>
                                    <type>jar</type>
                                </artifactItem>
                            </artifactItems>
                            <!--excludes>**/AClassIDontLike.class</excludes> -->
                            <outputDirectory>${project.build.outputDirectory}</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

jpivot-web-customized [war]

Depends on and includes jpivot-classes-customized.jar and depends on jpivot.war, contains changed and new resources and is merged with jpivot.war replacing its jpivot.jar with our jpivot-classes-customized.jar during packaging.

The combination and overriding is done by using overlays of the maven-war-plugin. We also exclude jpivot.jar and replace it with our dependency jpivot-classes-customized.jar (which mixes the original jar with our changes).

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>eu.ibacz.pbns</groupId>
    <artifactId>jpivot-web-customized</artifactId>
    <packaging>war</packaging>
    <version>1.8.0-SNAPSHOT</version>
    <name>jpivot-web-customized Maven Webapp</name>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.tonbeller</groupId>
            <artifactId>jpivot</artifactId>
            <version>${jpivot.version}</version>
            <type>war</type>
        </dependency>
        <dependency>
            <groupId>eu.ibacz.pbns</groupId>
            <artifactId>jpivot-classes-customized</artifactId>
            <version>1.8.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
    <description>Our modification of the original jpivot 1.8.0 webapp. This project should only include the classes, JSPs etc. that we do change.
    A package is generated by combining this with the original war.</description>
    <parent>
        <artifactId>pbns-root-pom</artifactId>
        <groupId>eu.ibacz.pbns</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <build>
        <finalName>jpivot-web-customized</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.1-alpha-2</version>
                <configuration>
                    <!--
                        exclude jpivot.jar, replace it with jpivot-classes-customized.jar
                    -->
                    <packagingExcludes>**/jpivot-libs-combined-*.jar</packagingExcludes>
                    <overlays>
                        <overlay>
                            <groupId>com.tonbeller</groupId>
                            <artifactId>jpivot</artifactId>
                            <excludes>
                                <exclude>WEB-INF/lib/jpivot.jar</exclude>
                                <!-- Mondrian stuff we do not needed. -->
                                <exclude>WEB-INF/mondrian.properties</exclude>
                                <exclude>WEB-INF/datasources.xml</exclude>
                            </excludes>
                        </overlay>
                    </overlays>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

New dependant artifacts

jpivot-portlet-liferay-classes [jar]

Classes necessary to implement a jpivot portlet working under the Liferay portal (5.2.2).

Depends on jpivot-classes-customized.jar for compilation only. It depends also on some Liferay libraries that have been imported into our thirdparty maven repository.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <parent>
        <artifactId>pbns-root-pom</artifactId>
        <groupId>eu.ibacz.pbns</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <groupId>eu.ibacz.pbns</groupId>
    <artifactId>jpivot-portlet-liferay-classes</artifactId>
    <name>JPivot Liferay Portlet Classes</name>
    <version>1.8.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>eu.ibacz.pbns</groupId>
            <artifactId>jpivot-classes-customized</artifactId>
            <version>1.8.0-SNAPSHOT</version>
            <type>jar</type>
        </dependency>

        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <version>2.3</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>javax.portlet</groupId>
            <artifactId>portlet-api</artifactId>
            <version>2.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.liferay</groupId>
            <artifactId>portal-kernel</artifactId>
            <version>5.2.2</version>
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.liferay</groupId>
            <artifactId>portal-service</artifactId>
            <version>5.2.2</version>
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.liferay</groupId>
            <artifactId>portal-impl</artifactId>
            <version>5.2.2</version>
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
        
        <!--
            Actually included in jpivot's libs, but made explicit here; Apache
            FOP for printing
        -->
        <dependency>
            <groupId>fop</groupId>
            <artifactId>fop</artifactId>
            <version>0.20.5</version>
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
    </dependencies>
    <description>Classes for a jpivot portlet ready for Liferay</description>
    <build>
        <finalName>jpivot-portlet-liferay-classes</finalName>
    </build>
</project>

bns-portlets [war]

A set of JPivot portlets, usually consisting of JSPs, CSSs, and JSs.

Extends further jpivot-web-customized and is merged with this WAR during packaging, same as jpivot-web-customized itself is merged with jpivot.war. Only it doesn't include any changes but only additions.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>eu.ibacz.pbns</groupId>
    <artifactId>bns-portlets</artifactId>
    <packaging>war</packaging>
    <version>0.0.1-SNAPSHOT</version>
    <name>BNS Portal Portlets</name>

    <dependencies>
        <dependency>
            <groupId>eu.ibacz.pbns</groupId>
            <artifactId>jpivot-web-customized</artifactId>
            <version>1.8.0-SNAPSHOT</version>
            <type>war</type>
        </dependency>
        <dependency>
            <groupId>eu.ibacz.pbns</groupId>
            <artifactId>jpivot-portlet-liferay-classes</artifactId>
            <version>1.8.0-SNAPSHOT</version>
            <type>jar</type>
        </dependency>
    </dependencies>
    <description>
  Portlets for BNS Portal panels based on our modified JPivot and jpivot liferay portlet support.
  </description>
    <parent>
        <artifactId>pbns-root-pom</artifactId>
        <groupId>eu.ibacz.pbns</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <build>
        <finalName>bns-portlets</finalName>
        <plugins>
            <plugin>
                <!-- Merge this WAR with the jpivot war stuff. -->
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.1-alpha-2</version>
                <configuration>
                    <!-- Exclude the transitive jpivot libs dependency - the WAR has it. -->
                    <packagingExcludes>WEB-INF/lib/jpivot-libs-combined-*.jar</packagingExcludes>
                    <overlays>
                        <overlay>
                            <groupId>eu.ibacz.pbns</groupId>
                            <artifactId>jpivot-web-customized</artifactId>
                        </overlay>
                    </overlays>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

The build process summarized

jpivot-classes-customized is compiled and merged with the original jpivot.jar into a single archive.
jpivot-web-customized is merged with the original jpivot.war, replacing its WEB-INF/lib/jpivot.war with the already built jpivot-classes-customized.jar. The output is a standalone and deployable standard web application.
jpivot-portlet-liferay-classes is packaged in the standard way and the produced JAR includes only its own classes.
bns-portlets is merged with our already built jpivot-web-customized.war to produce a deployable portlet application.

A development project for IDEs

The "change projects" are not complete on their own and are therefore not very suitable for use for development in an Integrated Development Environment (IDE). If we used an IDE to work on a change web project, it would complain about missing TLDs and other resources from the WAR to be merged in and it wouldn't be able to deploy it thus loosing the advantage of hot deploy.

The proposed solutions is as follows:

Build the final, complete bns-portlets.war (cd bns-portlets; mvn package).
Import the WAR as a project into an IDE.

Eclipse: File - Import - Web - WAR file.
NetBeans: Unpack the WAR somewhere then File - New Project - Java Web - Web Application with Existing Sources - go to the unpacked directory.

You should have defined a Liferay runtime environment for the server. If not, you will need to add classpath entries for the Liferay libraries it uses to be able to compile it.

Delete WEB-INF/lib/jpivot-portlet-liferay-classes.jar.

// If WEB-INF/classes hasn't precedence over lib/ then delete classes that we've modified from jpivot-classes-customized.jar.

Add 2 new source folders that are linked to the source folders of the jar projects, namely one to jpivot-classes-customized/src/main/java/ and the other to jpivot-portlet-liferay-classes/src/main/java/ .

Eclipse: Project - Properties - Java Build Path - Source - [Link Source]

Develop.

Now you can modify the sources if needed and commit their modifications to SVN. Because we actually work on the original projects' directories, the proper svn metadata will be picked up and the changes will be commited to the respective projects.

Unfortunately Eclipse ignores the SVN information of the linked sources. To be able to commit etc. them you'll need either to use another svn client or have the projects owning the sources open in Eclipse. An alternative is may be to drop the links and define project dependency.

If you need to modify another source code or resource file, copy it first to the approrpiate change project from the original jpivot.war or jpivot.jar's sources.

All changes will be commited to the corresponding SVN projects so this is ok but there is now the risk that a developer will do something in this development project, commit it to SVN, but forgets to build and deploy the corresponding real project's artifact. Well, this is a risk we have to live with. Such an omission could be detected by our continuous integration server (indicated by a failed build) and perhaps we could add a test which verifies that deployed artifacts aren't older than the corresponding SVN project.