The unholy legacy of databases

When reading about the status of Qi4j on Rickards blog, I stumbled about

Entities are really cool. We have decided to split the storage from the indexing/querying, sort of like how the internet works with websites vs Google, which makes it possible to implement really simple storages. Not having to deal with queries makes things a whole lot easier.

We made the same experience when we developed the SnipSnap wiki application several years ago. We had a split in storages and search, each part with it’s own Java interface (a component could implement both of course). This way we could have Lucene, database and in-memory search and database and file (XML, plain text) storage. We were very flexible with storage and search this way and people could easily implement different storage backends because developers have been freed from the search implementation. Rickard seems to have made the same experiences:

We have one EntityStore based on JDBM (persistent binary hashmap), one on JGroups (replicated cluster hashmap), one on Amazon S3 (for global storage), and one on iBatis (for RDBMS storage)

So today SnipSnap would easily be able to supply a S3 backend, because of the split, whereas others which rely on the storage/search combination have much more problems to support a storage-only backend. So they have problems to support S3 or WebDav out of the box.

Why don’t more people split the problem of storage into storage and search? After some contemplation on the topic, perhaps it’s the unholy legacy of databases. Databases make it easy to solve the search/storage problem with only one technology. After 30 years of databases the problems have merged in a way that most developers think of them as one problem. By splitting the problem again, projects will be freed for better backends and better search solutions. Open Source projects will emerge which adress each of the problems better than current databases do.

This of course breaks the DAO pattern and the usage of the EntityManager as an DAO replacement and should be replaced by a Storage and Search pattern. Free your mind! Storage and search are two different things, if you split them, you gain flexibility.

Thanks for listening.

Mutable, Immutable and Generics

Immutable objects help with avoiding bugs. Suppose I have two interfaces implementing the Immutable Interface pattern. One interface for Point and one with MutablePoint. The pattern suggests a cast to ImmutablePoint p; ((Point)p).setX( 1.0 ); get the mutable interface. This isn’t safe and can be replaced with a Generics solution.

First we have the mutable point:

public interface MutablePoint {
   public void setX(int x);
   public void setY(int y);
}

which we want to create from an Immutable object without a cast:

  Point point = point(10,10);

  MutablePoint mPoint = point.makeMutable();
  mPoint.setX(20);

The second benefit beside the missing cast is that users of the Point interface know there is a mutable interface which they can get in a defined way.

The makeMutable() method comes from a generic interface called Mutable

public interface Mutable<T> {
   public T makeMutable();
}
 

which is extended by the Point interface.

public interface Point extends Mutable<MutablePoint> {
   public int getX();
   public int getY();
}

Our Point implementation now only needs to implement MutablePoint and Point. Voila.

public class DefaultPoint implements Point, MutablePoint {
   private int x = 0;
   private int y = 0;

   public DefaultPoint(int x, int y) {
      this.x = x;
	  this.y = y;
   }

   public int getX() { return this.x; }
   public void setX(int x) { this.x = x; }
   public int getY() { return this.y; }
   public void setY(int y) { this.y = y; }

   public Point point(int x, int y) {
		return new DefaultPoint(x,y);
   }

   public MutablePoint makeMutable() {
      return this;
   }

}

A negative side effect is that people who have a reference to a Point object can get a mutable version. When the main point is to have more immutable objects. The other way round is often better, have MutablePoint and create an immutable version.

Thanks for listening.

Update: As pointed out, making an mutable object from an immutable one will break the contract, as will making an immutable one from an mutable one. You do need to copy the object to be safe. For example with a BeanCopier or a copy constructor I’ve wrote about in Beautiful Java: Reflection and the BeanCopier. Better name the method asMutable than makeMutable

public MutablePoint asMutable() {
   return new DefaultPoint(this.x, this.y);
}

John Resig on ExtJS, the GPL fiasco and open source community style

It seems as it does not end.

Reading a comment from John Resig, the (or one of the geniuses, sorry if there are more :-) genius behind jQuery, a library which was for some time a basis for ExtJs (beside YUI), irritated me a lot.

We (the jQuery project) worked hard with them to try and fix bugs and add features for an ExtJS integration layer. They turned around and built their own, specialized, library (removing the need for any of our work) and then mutated the licensing into this bizzaro scheme that they have now. We can’t, in good consciousness, even recommend their library anymore due to its very nature. On top of this they ended up hiring our lead evangelist to promote their work. I can’t speak for everyone on the team but I feel quite frustrated and used.

I’m speechless about the style of ExtJS. Without the not correct comments on my blog by Jack I wouldn’t believe someone could act this way.

I think you are missing the point. This has nothing to do with money, we are doing fine financially. It had to do with cleaning up what had until 2.1 been a hodge-podge of licenses that came out of necessity into a clean simple licensing structure that is aligned with what we expected all along.

says Jack. Sure it’s about money, everything could be clear - or to most people is clear - with the LGPL. No need to switch to the GPL at all. When it’s about being open source, switch to the ASL. And I find it highly irritating how Jacks kids are always dragged into this ydiscussion as in “my kids are waking up so I have to cut this short.” and “With my third child due, and savings running low”. I have no kids on my own but most certainly not drag them into a business discussion as an excuse.

Funny thing: “Ext JS is currently asking for input on two FLOSS exceptions to help make open source usage of Ext JS more flexible.” if this plays out like the switch to LGPL or GPL, hooray another episode in this mind boggling soap opera.

Update: “The intention of this exception is to allow for more liberal licensing of extensions, language packs, themes and open source developer toolkits and frameworks for Ext libraries under a variety of open source licenses.” on their blog.

Sure, because the more liberal the licenses of extensions are, the better it would be for the commercial side of ExtJS. Best to have the extensions as Apache licensed code and ExtJS to be GPL. The Ext LLC can reuse and resell the extensions without bothering. “(Note: this exception is not for applications and does not grant any exception for the library itself. A FLOSS exception on the libraries for open source applications will be addressed in the exception discussed in “Next Up” below).” This is so ridiculous and gives Open Source such a bad name. They hurt everyone who tries to make a living in the open source space.

Another ExtJS GPL thought - Should extensions switch to the GPL?

Sorry, this would better go to twitter - but I’m not twittering.

Another thought. And not because I want to bash ExtJS, but because I’ve been interested into the GPL, open source licensing and the implications for over a decade.

IANAL. The best situation for the company behind ExtJS would be if extension developers stay with the LGPL for their extension (or switch to a more liberal Apache license). The people who buy the OEM license from Ext can then use the extension. If someone releases his ExtJS extension as GPL, to be more “in line” with ExtJS, people with the OEM license cannot use the plugin, because it’s GPL (they can use the extension in a way that their customer need to download and install the extension on his own, but this is most often too cumbersome for customers. They are not allowed to distribute their commercial application with the extension or any code which references the extension).

The plugin writers do not gain anything for staying with the LGPL license, but Ext LLC gains a lot. It makes their OEM license much more valuable. If every Plugin writer switches to the GPL version, this could have an impact on the OEM sale. Especially because most enterprise won’t touch GPL software.

The best for a plugin author is to also go to a dual GPL/commercial license.

Very interesting situation.

Update: Very interesting

Book Review of “The ThoughtWorks Anthology: Essays on Software Technology and Innovation”

My very short review of The ThoughtWorks Anthology. The ThoughtWorks Anthology contains 13 essays on software development from different people. The quality of the essays varies widely but in the end: Recommended.

Solving the Business “Last Mile”

Excellent. Worth the book alone. This essay will lead the community to accept that there is a growing last mile problem for companies between the end of development and the going live.

One Lair and Twenty Ruby DSLs

Nothing really new, but an extensive overview over several DSL techniques in Ruby.

The Lush Landscape of Languages

Sub-par. Only interesting if you know nothing about the differences in programming languages. Filler stuff.

Polyglot Programming

Awful. For an explanation see below.

Object Calisthenics

Interesting and thought provoking. Could make you a better developer.

What is an Iteration Manager Anyway?

Good new insights about the general role of Scrum Masters and other Iteration Managers in agile projects.

Project Vital Signs

The team mood chart is genius.

Consumer Driven Contracts: A Service Evolution Pattern

Do your SOA from the consumer side not the provider side, excellent idea.

Domain Annotations

Very good! If your into Domain Driven Design, a must read.

Refactoring Ant Build files

I’ve been developing Ant build files since the beginning of Ant, but still got some insights.

Single-Click Software Release

Some good points for the deployment process, worth reading.

Agile vs. Waterfall Testing for Enterprise Web Apps

Overview of testing methods and how the fit for agile, solid but could have more punch.

Pragmatic Performance Testing

Sorry to the author, I didn’t read this one :-)

Let me illustrate how bad some parts of the book are. One essay with a clear bias against Java wants to establish how awful Java is and how wonderful other JVM languages like Groovy and JRuby are. After a distorted line number example there is an even more distorted isBlank example. For Java the author presents an implementation from Jakarta Commons, a heap of notoriously bad code.

public static boolean isBlank(String str) {
 int strLen;
 if (str == null || (strLen = str.length()) == 0) {
  return true;
 }
 for (int i = 0; i < strLen; i++) {
  if ((Character.isWhitespace(str.charAt(i)) == false)) {
   return false;
  }
 }
 return true;
}

For comparison the JRuby example (which has other performance and memory characteristics) is short

class String
  def blank?
      empty? || strip.empty?
  end
end

One who is moderatly fluent in Java would write the following code though:

  public static boolean isBlank(String string) {
    return isEmpty(string) || (isEmpty(string.trim()));
  }

  public static boolean isEmpty(String string) {
   return null == string || string.length() == 0;
  }

Not as nice and short as the JRuby example, but much better than the ugly Commons example. And with static imports the methods aren’t that ugly to use than before in Java. A isBlank("Hello") works.

But beside some bad essays the book is worth it’s money (Well it is cheap).

Thanks for listening.

Update: Lowell: “If you are using Java 6, there is a new method: String.isEmpty(). You could use that in your Java example and then you wouldn’t need to write your own.” Thanks.

More cluelessness from ExtJS lead developer Jack on the GPL issue

Reading the excellent analysis on A little Madness about the GPL and ExtJS issue, there is more cluelessness in a comment by Jack Slocum the ExtJS lead. He claims that others

“[...] wrap it up and sell it as their own. [...] With no mention of us at all.”

Nope, that would be illegal in most countries. You cannot deprive the author of his ownership and tell people the work was yours.

If someone would repackage ExtJS he would be required to keep the headers and original copyrights intact. So by definition he can’t “sell it as his own”. This just shows how uninformed the ExtJS core developers are.

“Since we started we have been open source and that is the route we will continue on.”

Running a company with a distorted view on the GPL (”backend code must be GPL if it generates HTML with ExtJS in it”) is not “beiing open source”. The open source argument is just a fake front to make more money.

IANAL of course.

The sad thing is though that Java open source projects have learned over the last 10 years that an Apache license leads to much more success. Apache will lead to companies using your library, more developers and in the end more users. More users lead to more support contracts and more license sales. Few users lead to few sales. And with lots and lots and lots of users someone big will buy you (Oracle, Sun, Google, Yahoo, MS). The GPL model only works for standalone products like MySQL, not for libraries. I’ve needed to learn this the hard way too.

The most important point is: Open Source success is build on the trust of your community. Lots of developers are let down by ExtJS. If you destroy the trust - as ExtJS did - your project is history. Go jQuery UI, go!

It will be interesting how they fare against e.g. Tibco GI which is commercial but liberally BSD licensed.

Thanks for listening.

GPL and ExtJS for Intranets

My latest thought about ExtJS going GPL. Although ExtJS is GPLv3 and the developers claim (falsely) that your backend needs to be GPL too when generating code that contains ExtJS, for internal/intranet applications you still can use ExtJS as you’re not distributing ExtJS.

When people don’t understand GPL and LGPL - or - ExtJS is history

The people behind ExtJS are funny. First they have changed their license to the LGPL, without understanding it in any way, now they’ve changed it to the GPL without understanding it in any way. They claim that server side code which creates HTML pages which contain ExtJS must be GPL, wuahahaha.

I’ve been running LGPL and GPL open source projects for several years, had lots and lots and lots of discussions about the very fine points of the LGPL and GPL and let me tell you the ExtJS guys have absolutely no clue. Which is a shame because ExtJS was a nice library. Now they’re history. No sane company will touch a project which changes it’s license frequently and has such an attitude towards the - dangerous - GPL.

Update: My latest thought. Although ExtJS is GPLv3 and the developers claim (falsely) that your backend needs to be GPL too, for internal/intranet applications you still can use ExtJS as you’re not distributing ExtJS.

Update 2: A little madness has a nice analyis on the switch.

All variables in Java must be final

I haven’t been using the final keyword in Java for 10 years, but more and more I think it’s an excellent keyword in Java. All local variables should be declared final. Today one of the developers of my team made every local variable in a method final and I was wondering how strange that looked. But of course it was the right thing to do. Declaring all variables final will lead to less bugs (logic and typos e.g.).

public void doSomething(final input) {
   final result = calc(input);
}

If you reassign a variable you most probably do something wrong. I’ll bet if you reassign a variable your method does at least two different things though every method should only do one thing. As should a class. Final also goes for method parameters, all should be final. Others think so too even for methods.

How do you deal with final attributes? When making most attributes final, just initialize them in the constructor. Immutable objects also play nicely with Tell, don’t ask.

All these final keywords around your code create noise though. It would be best to have all variables automatically be final and have a new transient keyword for reassignable variables. Or at least for my IDE - Intellij IDEA - to hide the final keyword and write the variable in a different color. Hear me IDEA developers!

Update: After some more work with Scala, var and val are a good solution.

REST: Lean JSON and XML from the same code

Generating JSON and XML with the same code is difficult. One can create the semantically richer XML and convert it to JSON, but JSON notations for XML like Badgerfish look quite ugly to JSON advocates.

The problem at the core is that XML is typed whereas JSON is not. Every node in XML needs a type - it’s name - for example <item><id>123</id><item>. JSON doesn’t need such a type, { id: 123 } is fine for an item. { item: {id: 123}} looks too verbose. Especially getting to the data in Javascript: var id = item.item.id. The same goes for accessing arrays with var id = items[0].item.id; instead of var id = items[0].id;. The problem exists with other dynamic languages and data structures too, see Cobra vs. Mongoose for Ruby.

As I currently develop a REST based Jersey application in Java I needed a way to generate lean JSON and XML. Wouldn’t it be best to have one code for both? DRY. My previous solution for generation JSON worked fine. The $(...) method calls create a node tree with nodes and lists. With a JsonRenderer and the Visitor pattern I generate JSON from the node tree. The problem was that this Java code

$(
  $("id", listId),
  $("items",
    ...
   )
);

creates nice JSON like { id: 123, items: [ ... ] }, but was unable to generate XML. As written above, the outer list has no type and a XmlRender therefor cannot render <shoppinglist><id>123</id>...</shoppinglist>.

The solution I thought about is to add type information to nodes which have no names.

$( type("shoppinglist"),
  $("id", listId),
  $("items",
    ...
  )
);

The implementation uses a simple static method and a Type class.

public static Type type(String name) {
    return new Type(name);
}

The type is attached to the node and if the node has no name but a type, the XmlRender uses the type instead of the name. The JsonRender doesn’t use the type information and renders the same JSON as before. The piece of Java code now generates XML

<shoppinglist>
  <id>123</id>
  <items>
    <item><id>234</id><price></price><shop></shop>
      <description>Apple</description></item>
    <item><id>233</id><price></price><shop></shop>
      <description>Banana</description></item>
    </items>
</shoppinglist>

and lean JSON where neither shoppinglist nor item has a type

{ id: "123", items: [ { id: 234, price: "", shop: "", description: "Apple"}, { id: 233, price: "", shop: "", description: "Banana"} ]}

Next thing is to automatically apply the right renderer, toXml and toJson from within Jersey. The content negotiation then choses the accepted format for the client. Attributes (Meta-Information?) are not solved yet and I’m not sure if they are needed, or how to nicely add meta information to the $(...) tree. There is some discussion in the context of markup builders and attributes on James blog.

Probably the code will be released as an open source RESTkit if someone is interested.

Thanks for listening.