Fluent Interface for Regular Expressions

Some years ago when we migrated Radeox - a wiki engine - to a new Regular Expression engine, we had to change lots of Regular Expressions. Today I would do it in a different way, with Fluent Interfaces. Joshua has a solution for building Regular expressions with a Fluent Interface, about which I wrote before.

Regex socialSecurityNumberCheck =
  new Regex(Pattern.With.AtBeginning
    .Digit.Repeat.Exactly(3)
    .Literal("-").Repeat.Optional
    .Digit.Repeat.Exactly(2)
    .Literal("-").Repeat.Optional
    .Digit.Repeat.Exactly(4)
    .AtEnd);

Not only is this solution easier to understand than using a String for regular expressions, migrating from ORO Regex to JDK Regex for example would be easy with this solution. The solution we chose was to create two engine implementations for ORO and JDK regex: Compiler (JdkCompiler,OroCompiler), MatchResult (JdkMatchResult,OroMatchResult), Matcher (JdkMatcher,OroMatcher) and so on. For example:

public class OroCompiler extends Compiler {
  private Perl5Compiler internalCompiler;

  private boolean multiline;

  public OroCompiler() {
    this.internalCompiler = new Perl5Compiler();
  }

  public void setMultiline(boolean multiline) {
    this.multiline = multiline;
  }

  public Pattern compile(String regex) {
    org.apache.oro.text.regex.Pattern pattern = null;
    try {
      pattern = internalCompiler.compile(regex,
         (multiline ? Perl5Compiler.MULTILINE_MASK : Perl5Compiler.SINGLELINE_MASK) | Perl5Compiler.READ_ONLY_MASK);
    } catch (MalformedPatternException e) {
      // handle later
    }

    return new OroPattern(regex, multiline, pattern);
  }
}

The actual regular expressions are stored in Java .properties files, to make them easily exchangable. The bold expression looks like this for example:

filter.bold.match=(^|>|[\\p{Punct}\\p{Space}]+)__(.*?)__([\\p{Punct}\\p{Space}]+|<|$)

With the migration of Radeox to a new regex implementation I had to write and implement several interfaces and it wasn’t very easy to both support ORO and the JDK implementation. It wan’t easy either to change the slightly different pattern dialact from ORO to JDK. With a Fluent Interface, just change the Fluent Interface, no need to think about different Java Regex APIs and to think about different syntaxes.

Thanks for listening.

Java and Apple: Fallen from love

As several people have written before, there is no Java 1.6 in the new Mac OS. What a stupid move. At most developer conferences concerning Java people have been and are using Apple laptops. Now in the last release, Leopard, Apple doesn’t even mention Java any more. But the page shouts “Work in a developer’s dreamland.” They said this some years ago about Java. Now it’s Ruby on Rails. Apple not even once mentions Java on their feature page, but “Leopard is the perfect platform for Ruby on Rails development, with Rails, Mongrel, and Capistrano built in.” Don’t they know the Rails hype is over? Really. It is. That the enterprise will move to Grails, if they have need for a dynamic lanugage? And move to Scala, when they want to innovate? And that today most big RoR applications are into legacy land. Funny that three companies with Ruby on Rails recently offered me CTO jobs - I guess you have to bring in some maintenance talent. Companies have to deal with new problems after some years of RoR. That’s another story. Back to Java. There are so many more companies buying Apple laptops for Java development than for Rails development. And as Alex said:

“But my boss has said that if Macs don’t support Java 6, moving forward it’s just not going to be an option anymore. We need to run and test with Java 6 and not having access to it is a big issue.”

There is so much more interest in Java than in Rails. But I guess we should have seen this coming with a Steve Jobs quote like: ““Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain.”.

Our love was nice. I fear it’s over. “Love takes hostages. It gets inside you. It eats you out and leaves you crying in the darkness.” Together with my current burned and smoking MacBook Pro power adapter, which they didn’t want to take back, my love for Apple decreased a lot (don’t mention the broken batteries in several iBooks and MacBooks). Rant over.

Thanks for listening.

Update: Yes, I know about the inaccurancies of Google trends, especially when you do not hit the right search terms. So one needed to include RoR, or perhaps even Ruby. In this case though, looking at the long, declining, but flat line of Java I think the search term “Java” more reflects the language than the island, there are no big spikes which should be there when it would reflect that people search that much for the island. Of course, this is debatable. But most often people confuse the “news” section of Google trends and the “search terms” section of Google trends. The one has nothing to do with the other. When you actually do a “Java” Google search, there is no hit for Java as an island on the first several pages.

Graeme - the Grails bug buster

While testing Reposita with the Grails repository, I was amazed about Graeme who is the Grails project lead and recently became the CTO of G2One.

Scanning the Grails subversion commits and searching for bug fixes, Graeme seems to be a very very active bug buster. One in four commits is a bug fix. Perhaps this directly contributes to the Grails success story. I hope he keeps up fixing bugs, now that he’s CTO.

See for yourself:

More Hibernate JPA troubles

When running JPA together with Grails (not tested on it’s own), with a Hibernate ORM either Hibernate, or with a pool the pool (c3p0), runs out of connections. The architecture of Reposita has a Grails frontend with a (Java) JPA backend to enable switching the frontend and backend independently. The MySQL database barks about to many connections. I do some persist() calls to the database, I guess around 50. Each call gets an EntityManager, does a transaction and closes the EM. Shouldn’t closing the EM close the connection and return it to the pool? Or is this a problem together with Grails, which also runs Hibernate to a MySQL database? Any ideas? I’ve run out of mine.

Update: I’m not sure what I’m doing wrong,

em = getEntityManager();
em.getTransaction().begin();
em.persist(entity);
em.getTransaction().commit();
...
em.close();

should work. But even when using with c3p0 as a pool, when calling this code several times, the connection count to the database increases until it reaches the limit, where the MySQL database barks about too many connections.

This might be caused by create-drop of my testing setup.

Update 2: As Dirk pointed out, of course em.close() should go into a finally block and I might add there should be some exception handling and rollback() action.

JPA hangs Grails, Part 2

As I wrote before, using Java JPA in my Grails application, the Grails application (0.6) hanged. After not much help from the mailing list, I updated to Grails RC1.0 without success. Adding the newest Hibernate JARs and Annotations JARs to the grails/lib directory solved my issue though. Grails no happily uses my JPA/Dao backend. Puh.

@License(name = “Apache”,version = “2.0″) annotations and GPL

Is your Open Source project correctly licensed? When I was thinking about my open source projects I remember I was shocked. They probably haven’t been correctly licensed (they were in the end) What license you can use depends on the third party open source projects you use. But not only that, it depends on what links to what.

(Sorry for all the license typos :-)

Suppose I have two servers, one licensed under the Apache 2.0 license and one licensed under the GPL. And the GPL version uses the Alfresco GPL CIFS server, no problem. I can mix those licenses in my project and use the GPLed Alfresco server, although parts of my project are Apache 2.0 (the first server). But when my Apache 2.0 server directly uses the GPL Alfresco server, I’m no longer allowed to distribute my project with the GPL Alfresco code because I violate their license.

To prevent problems some people do license audits by looking at the Java code in the repository, the imports and then looking at the dependent libraries and their licenses. Reposita will have this feature soon. Maven is working on it by analysing dependencies. But in the end it depends on what code is calling what code, not what code is packaged with what library, or what code imports what code (some people claim a Java import equals linking though) or what license files might be packaged with the project.

So we need a way to annotate code with the license it has, so we can at runtime - for example with aspects - check if Apache 2.0 code directly calls GPL code. Then we’re in trouble.

Java 5 to the rescue. With Java 5 Sun introduced annotations. We now can write a @License annotation and annotate our code with the @License() annotation.

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface License {
  String name();
  String version();
}

Annotating our code like this:

@License(name = "Apache",version = "2.0")
public class Example {
    public void doWhatever() { ... }
}

helps us in the end to determine the license of the code programmatically. Either by wrapping method calls in aspects and checking the license of the caller and the callee or for example by checking all classes in a jar. The first example could be used during a QA test or during code audits. The aspects then would be removed for deployment. The second variant works great for license audits, merger and acquisitions, new projects, open source projects etc. This would enable us to more directly audit classes, code and their license and not depend on the JavaDoc license header (where sometimes the JavaDoc or license header gives the wrong information, because the license has changed several times).

For this to work most open source projects need to annotate their code though ;-) I guess we need a JSR and some standard annotations and conventions to make this work for all.

Thanks for listening.

Update: Some Feedback: analysing import does not help. With more and more dependency injection (Spring XML, Guice annotations) there is no longer an import relationship between your code and the third party code. License violations can then only be determined at runtime.

Creating a fluent interface for Google Collections

Chris wrote about functional programming in Java. As an example he used Google Collections which “is a suite of new collections and collection-related goodness for Java 5.0″.

Looking at his post he gives the following code on how to use Goolge Collections

files = Iterables.transform(
  Iterables.filter(
      Iterables.filter(getPagesList(),WebPage.class),undeletedPages),
  new PagesToFilesTransformation()))

This is very verbose and hard to read. Mostly due to the static method calls and the seperation of the first “filter” and the “new PagesToFilesTransformation()” parameter. Using a static import we can reduce this to:

import static Iterables.*;

transform(
   filter(
      filter(getPagesList(),WebPage.class), undeletedPages),
   new PagesToFilesTransformation()))

Better. I’ll show you how to make this even more readable with a fluent interface. I use the following example for this:

List names = Arrays.asList("Stephan", "Chris", "Mike", "Miha", "Katrin");

Iterable filtered =  Iterables.filter(
  Iterables.filter(names, or( isEqualTo("Katrin"), isEqualTo("Miha"))),
    lengthLessThan(5)) ;

We want to move this code to a fluent interface. Martin Fowler wrote about Fluent Interfaces on his bliki and there is a entry on Wikipedia. They were made popular by JMock as a way to specify mock expectations.

logger.expects(once()).method("setLoggingLevel").with(eq(newLevel));

What is a fluent interface though? For me it’s an interface to an API which is more consistent and readable. Which helps you do several steps with an API in a correct way. It’s often implemented with speaking method names and call chaining. These together create the illusion of natural, descriptive language. When the chained methods return the correct type, most often it’s impossible to call the wrong methods or methods in the wrong sequence. Therefor fluent interfaces create better readable code with less bugs. Which distinguishes them from domain specific languages (like in Ruby on Rails), macro programming (Lisp) or plain language constructs (methods which take closures) like “[1..5].each { … }”. The differences is fluent ;-) “5.times.each {…}” could be considered a simple fluent interface.

The most recent fluent interface in Java with some fame is Quaere, a kind of LINQ for Java for querying object graphs. There even is a promising Quaere API for JPA. I really think fluent interfaces are useful, although others think they are stupid. I keep to myself what I think is stupid and we see where this all is going to.

Back to our example. I’ve recently shown how to create a fluent interface for object creation. One for Google Collections might look like this:

Iterable<T> filtered =
    with(names)
   .filter( or(isEqualTo("Chris"), isEqualTo("Miha")))
   .filter(lengthLessThan(5));

This example isn’t optimal, but much more readable than the one given by Chris. The or() part could be improved because it doesn’t look very nice. This will change with closures. Fluent interfaces will boom with closures in Java 7.

In the comment to his blog post, Chris mentions “Making a fluent interface to do this might be a little risky, because I think there’s an expectation* that a method like Iterable.filter() wouldn’t modify the source iterable itself, but rather would return a new, filtered iterable.” We do not need to modify the Iterable, we only hold a reference and create new fluent interfaces on the fly as needed. Our code to enable a fluent interface for Google Collections this is rather simple:

public class FluentIterable<T> implements Iterable<T> {
  private Iterable<T> iterable;

  public FluentIterable(Iterable<T> iterable) {
    this.iterable = iterable;
  }

  public static <T> FluentIterable<T> with(Iterable<T> iterable) {
    return new FluentIterable<T>(iterable);
  }

  public FluentIterable<T> filter(Predicate<? super T> predicate) {
    return new FluentIterable<T>(
        Iterables.filter(this.iterable, predicate)
    );
  }

  public Iterator<T> iterator() {
    return this.iterable.iterator();
  }
}

This example has several main points. The with() static method call is descriptive and creates a fluent interface object for us to hold the Iterable object. Second most methods in fluent interfaces return the object itself. We create a new fluent interface object in filter to wrap the filtered iterable and enable chaining. My last example with object creation did hold the same bean and did not create new objects on the fly. This really depends on what should be done. We could have set the new iterable instead of creating a new wrapper like this:

  public FluentIterable<T> filter(Predicate predicate) {
    Iterable filtered = Iterables.filter(this.iterable, predicate);
    this.iterable = filtered;
    return this;
  }

This mostly depends on your style and if you think immutable objects are good or bad. For this I prefer the immutable Fluent Interface object over the mutable. Perhaps this is a pattern.

Conculsion:: A fluent interface for Google Collections would be useful. It’s more readable, less error prone and flows. Although my example works and is usable, there needs to go more work into a fluent interface for Google Collections to make it useful. For example support more Predicates and support transformations.

Thanks for listening.

Grails, Hibernate and JPA: hanging application

When developing an application in Grails I’ve been adding a backend written for JPA. This should be no problem - you think. But while the unit test for the JPA backend work fine for themselves, when adding the backend to the Grails application, Grails hangs when accessing the EntityManager. It does not help that all my efforts to join the Grails mailing list with my gmail address have failed so far. I have no clue how to fix this problem which took most time out of my last weekend :-) Perhaps Hibernate does not like to act as a Grails ORM and at the same time as a JPA provider. Or I have JAR conflicts. Any ideas?

Update: It has been suggested that the problem might be that there are several incompatible ASM jars in the classpath.

Mozy on Mac does not work :-(

I’ve been trying out Mozy as a backup service for my Mac. Granted it’s beta, but for me it does not work. When I change files or add ne files to a folder, those changes are not detected by Mozy (0.7.0.0). It tells me “no files changes” when I do a backup. Sigh. Either I’m doing something wrong or this is a bug. Their FAQ says it depends on Spotlight, but after checking Spotlight does detect the new files. I guess I either need to way for a final Mozy version, or I’ll look for another service. Sad, because Mozy looked good and was cheap.

JavaRebel impressions - Java Reload just like in Rails

JavaRebel promises Ruby on Rails and PHP reloadability for Java. It is a library that works as an agent to the Java Virtual machine and helps the VM dynamically reload Java classes when they have changed. The setup I used was a fairly large application with dozends of modules, tomcat 6 and Eclipse as my IDE. When installing JavaRebel be sure to change both paths to the JAR in your JAVA_OPTS. I only changed one and the VM couldn’t load the JAR. With some friendly and speedy help from the JavaRebel developers the problem was found in only a few minutes.

There isn’t much of a review here. JavaRebel works as advertised. Adding code, extracting methods and changing your class, then save it with auto compile in Eclipse and press reload in your browser. The file is changed. Just as with PHP or Rails. Great news. With some large tomcat setups that need several minutes for startup, this saves a lot of time (and essentially money). Why not use the Hotswap function with Eclipse? HotSwap with the VM currently only supports changing code in methods. JavaRebel supports refactoring like adding and removing methods, fields or constructors. With HotSwap you also need to attach a debugger and switch to the debugger view. One click too much.

I asked them for a open source license some days ago and now they provide one. Thanks.

As a first impression: Excellent tool. We’ll see how it does in a real development situation for some days. I’ll keep you informed.

Fluent Interface and Reflection for Object Building in Java

A post by Randy Patterson on creating fluent interfaces in C# got me thinking. Randy shows how to create a Fluent Interface to build an object. He uses a Person class (POJO) with setFirstName and a PersonFluentInterface class with firstName. Creating an object and setting some attributes needs quite a lot of code:

Person person = new Person();
person.setFirstName("Pat");
person.setLastName("Frank");
person.setAge(30);
person.setActive(true);

The alternative and the usual way is to use a constructor and set the attributes. Often this results in unreadable constructor calls when creating objects (no paramter names in Java):

Person person = new Person("Frank", "Pat", 30, true);

What is “Frank”, “Pat” and especially “true” ? In Groovy and other languages we could write:

Person person = new Person(firstName: "Frank",  lastName: "Pat", age: 30, active: true);

Indeed this is one of the best Groovy features and can be utilized when writing unit tests for Java classes. With a fluent interface the PersonFluentInterface class, which calls the setters on the POJO for him, Randy can write something similiar:

Person person = new Person();
person.Set.FirstName("Pat").LastName("Frank").Age(30).IsActive();

The set method in Person returns the PersonFluentInterface wrapped around the Person class. This looks more readable than calling lots of setters on an object and more readable than a large constructor call. We can do even a little better than Randy. With an additional method call we can combine the two lines in his example into one:

    Person person = Person.with()
      .firstName("Stephan")
      .lastName("Schmidt")
      .create();

Randys approach requires the developer to implement a PersonFluentInterface class which calls the correct setters on the object. The only thing we would need is the interface description of the Fluent Interface to enable the IDE to support code completion and provide static checking.

public interface PersonFluentInterface {
  public PersonFluentInterface firstName(String firstName);
  public PersonFluentInterface lastName(String lastName);
  public Person create();
}

I thought about a class which then maps the interface to a POJO with reflection. Some minutes later I was done. Voila!

public class FluentInterface<T> implements InvocationHandler {
  Object obj;

  public FluentInterface(Object obj) {
    this.obj = obj;
  }

  public static <T> T create(Object object, Class fluentInterfaceClass) {
    FluentInterface handler = new FluentInterface(object);
    @SuppressWarnings("unchecked")
    T fluentInterface = (T) Proxy.newProxyInstance(
       fluentInterfaceClass.getClassLoader(),
        new Class[]{fluentInterfaceClass},
        handler);
    return fluentInterface;
  }

  public Object invoke(Object proxy, Method m, Object[] args) throws Throwable {
    try {
      String name = m.getName();
      if ("create".equals(name)) {
        return obj;
      } else {
        String setter = "set" + name.substring(0, 1).toUpperCase()
          + name.substring(1);
        Statement stmt = new Statement(this.obj, setter, args);
        stmt.execute();
      }
    } catch (Exception e) {
      e.printStackTrace();
    } finally {
    }
    return proxy;
  }
}

Together with the interface we can now extend our Person POJO with an additional static method and we’re ready to use a fluent interface to create our POJOs.

public class Person {
  private String firstName;
  private String lastName;

  public static PersonFluentInterface with() {
    return FluentInterface.create(
      new Person(), PersonFluentInterface.class);
  }

  public Person() {
  }

  public void setFirstName(String firstName) {
    this.firstName = firstName;
  }

  public void setLastName(String lastName) {
    this.lastName = lastName;
  }
}

Some future ideas: when using byte code instrumentation instead of a proxy we could use a abstract class which extends Person and calls the setter on itself. This would allow us the drop the create() call and add custom builder methods like “isMemberOfGroup(group)”.

I’m quite satisfied with the results. Some lines could be tweaked and optimized, but overall the solution works. I’m going to use this for some time to decide if it’s worth writing and maintaining the additional interface. But most probably it will increase the readability of large constructor calls for others and for me - after I haven’t read the code in question for some weeks.

Thanks for listening.

Update: There is a small bug in the examples ;-)

IDEA license

After Ilias comment on my last blog post about the open source IDEA license we solved the problem with my mail and I’m a proud owner of an IDEA license. Thank you very much IntelliJ. Some money saved and the hell of Eclipse prevented :-)

Drop IDEA, move to Eclipse? No open source license from IntelliJ

I need a tough decision to make. I’ve been using and evangelizing IDEA from the first IDEA release several years back. Whenever I came to a new company I introduced IDEA as an IDE and kept using it for commercial and open source use since then. Because I’m less productive with other IDEs I moved several projects where I arrived to IDEA ;-) I evangelized nearly the whole German Java IRC channel into IDEA. When I left my last employeer, I lost access to an IDEA license. Thinking that IDEA is the best IDE around - with quite some margin - I wanted to get an Open Source License for IDEA to develop Reposita, Messages, Meaxure and Radeox. I have a difficult case as there is no community for those projects yet because they just started. Writing to Intellij and pleading for a license, I got no answer.

So now I have three options: Buy a $249/year personal license, move to Eclipse (which I use at work, but which is far from being a good IDE) or learn something new and go with NetBeans? What does that mean for other contributers? Some of them will have access to an IDEA license. What are your experiences with different IDEs in on open source project? What would you do?

The IDE should support Groovy, Grails and Seam. Guice would be nice. Any other options?

Thanks for listening.

Update: See here.

Wikis overtook Blogs

When you’ve got nothing better to do, take a look at Google Trends. Although it’s not scientific, you can gain some insights into trends. Today I was astonished: At last wikis overtook blogs on google trends. When I wrote about the future of Radeox one month ago, blog still overtook wiki as a search term with Google searches. This has changed. People now search more often for wikis than they do for blogs. I still believer Gartner is wrong with their Wiki hype cycle. We haven’t even seen the beginning of wikis. The future will be interesting.

Radeox in the Groovy Book

There is a national holiday in Germany and I’ve flipping through the Groovy book in search for regex examples and groovysh usages. When browsing through the index, I found Radeox was mentioned. Heureka. What a nice feeling. Thanks Dierk that made my day.

Documentation for Reposita Messages

I’ve rewrote some documentation for Reposita Messages. Currently I use Apple Pages. This allows for nice design and nice writing. I consider using DocBook as it allows others easier to contribute to the documentation process. But it’s hard to get DocBook working and create a nice Design for DocBook. Guessing that Hibernate and Spring use DocBook, they’ve done a nice design job. DocBook could also be easily created from wiki pages or from fragments in an SVN. Any other ideas? Volunteers? :-) I’ve also set up a user mailing list for Messages.

Messages is a free framework to make internationalization easy for Java applications. It supports locales for threads and bundles can be associated with different packages. This allows the usage of different bundles for different parts of the application like plugins, installer or logging. Bundles can be managed and associated with packages without any changes in the code. This makes bundle management and refactoring much easier than with other solutions.Messages supports global locales for client applications written in Swing or SWT and thread local ones for serverside web applications. Messages currently needs Java 1.5

Thanks for listening.