Ivy: How to Retrieve Source Codes of Dependencies

Summary: To download sources you must map the dependency's conf also to 'sources' (ex.: conf="myScope->default,sources").

If you are using Apache Ivy (2.2) for maintaining you dependencies, you may also want to retrieve source codes (or javadocs) of those dependencies, e.g. to get nice help pop-ups in Eclipse or to actually browse the sources. Provided that you're retrieving the dependencies from Maven repositories, the configuration required to enable retrieval of source codes is to map to the Ivy "configuration" ("scope", or perhaps more exactly classifier, in the Maven language) called "sources" (or "javadoc" to fetch JavaDocs).



To find out what target configurations are defined, you may check the generated Ivy metadata for a downloaded Maven artifact, for example ~/.ivy2/cache/org.springframework/spring-context/ivy-2.5.6.xml - check <configurations> for (obviously) the configurations and <publications> to find out what artifacts are available in which configurations (ex.: ...name="spring-context" type="source" ext="jar" conf="sources"...).

Example: Ivy.xml with non-standard configurations

<!-- ivy.xml -->
...
   <configurations>
      <conf name="oracle-war" description="Runtime dependencies for deploying to an Oracle container." />
      <conf name="compile" extends="oracle-war" visibility="private" description="Additional compile time dependencies required to build the application." />
   ...
   </configurations>

   <dependencies>
      <dependency org="org.springframework" name="spring-webmvc" rev="2.5.6" conf="oracle-war->default,sources" />
...

Notes:

• We first define some custom configurations (scopes), including oracle-war
• Next we define a dependency belonging into the configuration oracle-war and map it to the target configurations default and sources, i.e. we depend on anything which is published either in the default configuration (-> spring-webmvc-2.5.6.jar) or in the sources configuration (-> spring-webmvc-2.5.6-sources.jar)
• Notice that we could use configurations' attributes defaultconf (e.g. ="oracle-war") and defaultconfmapping (e.g. ="*->default,sources") to avoid unnecessary duplication in the file

Retrieving sources or binaries with Ant

To retrieve only the binary dependencies and put them inside the lib/ folder (notice type="jar") with ivy:retrieve:

<!-- build.xml -->
	<target name="ivy-get-binaries">
		<ivy:settings file="ivysettings.xml" />
		<ivy:retrieve pattern="lib/[artifact].[ext]" type="jar" />
	</target>

To retrieve source codes of the dependencies and put them inside the libsources/ folder:

<!-- build.xml -->
    <target name="ivy-get-sources" description="Fetch JARs with source codes for libraries that have them">
        <ivy:settings file="ivysettings.xml" />
        <ivy:retrieve pattern="libsources/[artifact]-[type].[ext]" type="source" />
    </target>

To verify that some artifacts have been found, the resolution report in the log is useless for it sums all types of artifacts (i.e. binary and source ones) together. You have to check the last line, which shows count only for the artifacts of the requested type:

[ivy:retrieve]     0 artifacts copied, 4 already retrieved (0kB/170ms)

Limitations

Ivy ignores some source attachements

Ivy sometimes ignores source artifacts, e.g. in the case of org.springframework:spring-webmvc:2.5.6 - when you check the Ivy-generated .ivy2/cache/org.springframework/spring-webmvc/ivy-2.5.6.xml, you will see that the publications section only lists ...type="jar" conf="master" even though spring-webmvc-2.5.6-sources.jar certainly exists - I suppose that this is due to the MD5 checksum of the file being incorrect.

Ivy can't fetch attachements for transitive dependencies

I don't know any way how to force Ivy to retrieve source/javadoc attachements for transitive dependencies. The problem is that the dependencies always depend only on other binaries, never on sources/javadocs so Ivy has no reason to download those.

Additional Notes

The Eclipse Ivy plugin IvyDE does download sources/javadocs automatically.

References

Documentation of the Ivy pattern syntax , description of how to define configuration mappings (I've always troubles finding these).

Tags: tool