Dwins’s Weblog


Workaround: SBT/Maven “Inconsistent module descriptor”

Posted in Development,Open Source Software by dwins on October 15, 2011
Tags: , , ,

Here’s an issue I’ve been dealing with in GeoScript.scala for a while: when fetching dependencies I get an error message:

update: sbt.ResolveException: unresolved dependency: xml-apis#xml-apis-xerces;2.7.1: java.text.ParseException: inconsistent module descriptor file found in ‘http://download.osgeo.org/webdav/geotools/xml-apis/xml-apis-xerces/2.7.1/xml-apis-xerces-2.7.1.pom’: bad module name: expected=’xml-apis-xerces’ found=’xml-apis’; bad revision: expected=’2.7.1′ found=’xerces-2.7.1′;

Sure enough, when I go to http://download.osgeo.org/webdav/geotools/xml-apis/xml-apis-xerces/2.7.1/xml-apis-xerces-2.7.1.pom I can see that even though the URL suggests the artifact would be (group: xml-apis, artifact: xml-apis-xerces, version: 2.7.1), it is in fact listed as (group: xml-apis, artifact: xml-apis, version: xerces-2.7.1).  Apparently Maven doesn’t verify those details when fetching dependencies, but Ivy (by default) does.  According to this JIRA issue I found there is an option to disable it, but SBT doesn’t seem to expose it (I fiddled around a bit with an ivysettings.xml file and the ivyXML setting but to no avail.)

Finally I just added an entry to my libraryDependencies setting like this:

“xml-apis” % “xml-apis-xerces” % “2.7.1” from “http://download.osgeo.org/webdav/xml-apis/xml-apis-xerces/2.7.1/xml-apis-xerces-2.7.1.jar

This doesn’t avoid the warning but does let me go ahead with my build.  Good enough I guess.

But recently I’ve been hearing some complaints from folks checking out my code (that’s right, potential contributors!) who’ve been confused by the error.  Since first encountering this issue I’ve been informed that xerces is actually not even needed, the API it implements is included in modern JVMs or something, so I tried just adding a dependency exclusion to my SBT build file.  Success! Not only was I able to run the ‘update’ command with no scary errors, but the test suite even runs.  Ticket closed.

Except, as it turns out, Ivy needs that metadata to handle the exclusion (or something, I didn’t dig in too deep.)  I already had it in a local cache, thanks to having run the build with that explicit URL, but when benqua from Github attempted a build from scratch he ran into the same issues again.  ARRGH.

Finally I ended up sticking with the exclusion, but now the GeoScript build includes a dummy subproject that has the xml-apis dependency with the explicit URL.  The intermodule dependencies are set up so that it always runs before the main “library” subproject, meaning that the xml-api-xerces project, with correct metadata, is in the local cache before any dependency resolution involving GeoTools begins.  Not incredibly elegant, but it does work.

I’ll be in touch with the GeoTools guys to see about just fixing the metadata in the repository – since Maven appears to ignore it I don’t see this breaking any existing builds.

Advertisements

You Don’t Need Java to use the JVM

Posted in Open Source Software by dwins on November 30, 2009
Tags: , , , , ,

Not too long ago I was looking for a decent web application framework for an application we’re working on at OpenGeo. It’s based on GeoServer and maybe some other Java servers, but the team expressed some concerns about being able to quickly turn around and maintain Java code. So I checked out some alternative JVM languages during my search. Here are my thoughts on the ones I looked at, Jython, Rhino, Groovy, and Scala.

Jython

Jython is an implementation of Python that runs on the JVM. It lets you call Java constructors and methods, as well as extend Java classes and interfaces, and it even maps bean properties to keyword arguments on constructors.  However, the interoperability is not entirely seamless in either direction.  When using overloaded Java methods from Python code, the method is selected on the runtime type of the arguments (nulls are especially troublesome here.)  The recommended way to work around this seems to be just to perform the appropriate conversions manually (not particularly problematic.)  In the other direction, things are a bit more of a hassle: since the Python code is not compiled, classes defined in Python do not exist until the script has been run (and disappear once the JVM exits.)  So, there’s a fair bit of boilerplate required to get a hold of an instance of a Python class, and integration with frameworks that rely on reflection (such as Spring) will require even more (a wrapper class to create the Python object and then delegate to its methods).  Additionally, the base types (String, Integer, etc.) used in Python scripts are not the standard Java classes, so occasionally there is an impedance mismatch due to that.  (The interpreter automatically wraps and unwraps the objects so most of the time it is not a problem.)

Rhino

Rhino is an implementation of JavaScript that runs on the JVM. It also allows you to call Java constructors and methods and extend Java classes and interfaces.  There is also some nice sugar:

  • bean properties are mapped to simple properties in JavaScript code (obj.foo = “new value”; instead of obj.setFoo(“new value”); )
  • The method overloading problem is handled by providing access through longer keys which include the type signature, as well as the short name.
  • If a method expects an interface argument and the interface only declares one method, you can pass in a function object instead of explicitly implementing the interface. (var button = new javax.swing.JButton(); button.addClickListener(function(){…}); )
  • Rhino implements the E4X extension for embedded XML literals, so manipulating XML documents is pretty painless.

Rhino has also been around for the longest of any of the languages I looked at and is a fairly robust implementation.  There is a compiler so you can generate real, reflection-friendly classes with it, and, although the JavaScript standard library leaves a bit to be desired, there are projects like Narwhal and JSAN to improve that situation.

Groovy

Groovy is a language designed specifically for use on the JVM, with an eye to interoperability with existing Java libraries, but also providing a more dynamic language.  It provides facilities like optional typing and the “elvis operator” (which works like || in other scripting languages for providing default values in expressions that would otherwise return null):

javascript: var foo = baz.mightBeNull || "defaultFoo";
groovy:     def foo = baz.mightBeNull ?: "defaultFoo";

On the other hand, valid Java code is a mostly valid Groovy syntax, so transitioning an existing codebase is (supposedly) easy.  There are some gotcha’s though.  For example, if you use type checking, it is only implemented at runtime, so you can have the headache of a compile phase followed by a type error.  Again, compiled code is fully accessible to the Java runtime, including reflection and direct instantiation.

Scala

Scala is another language designed with JVM interoperability in mind, although it is much more of a deviation from Java than Groovy.  We’re not using it for the project that inspired this research, but I’ve been working with it on a little side project for some time now.  It is fully interoperable in the sense that Scala classes are Java classes too, but it resorts to some clever compile-time tricks to implement some of its niftier features, like operator overloading and traits (similar to Java’s interfaces, but with the ability to provide default implementations for methods.)  However, it is fully type-checked, using type inference to avoid type declarations all over the code.  It also has a number of features that don’t translate well to Java code:

  • Extractors, used for pattern matching; implemented as functions which return instances of a class from the Scala library.  These are usable from Java, but without Scala’s syntactic sugar for using them, they are much less useful.
  • Classes may have companion objects (which serve as the holder for any “global” methods; the stuff that you would declare static in Java.)  The way that the Scala compiler implements these means that static methods show up in Java code as part of a second class with a $ at the end of its name (ie, Foo would have a Foo$ class with all the static helpers.)
  • Functions as values.  In scala code, calling a function stored in a variable is just like calling a function normally, and there is specialized syntax for function variable types: “var foo: (Int, Int) => Int = (a, b) => a + b”, “foo(1, 2) == 3”.  In Java, such functions show up as instances of scala.Function2<scala.Int, scala.Int, scala.Int> and must be called through their apply() method.

The upshot is that Scala works quite well for working with existing Java code.  But if you are designing a library for use by general Java developers and not just Scala developers, then writing in Scala means that you need to avoid certain language features to avoid making the API cumbersome.

There are, of course, plenty of other languages available for use on the JVM.  JRuby, Clojure, and Jaskell are a few I’ve heard of, but not looked into yet.  However, of the languages I looked at, Rhino and Jython seem better suited as user-facing scripting interfaces, while Scala is a nice alternative to Java for core implementation.  If you are designing a library for consumption by Java developers though, Java itself gives you the best control over the Java classes and method signatures.

For the record, we ended up choosing Django on CPython for the current round of development, with plans to investigate moving to Jython if and when consolidating the code onto the JVM becomes more necessary.