Maybe you haven't noticed, but every single piece of information on JDK 1.5 Metadata Annotations is nothing but a simple /Hello World/ tutorial (of course, I exclude the specification). There is no real world implementation. This is especially true for codes processing the annotations. I'll call such an application an annotation reader.

I was publicly cursing about that and Cedric Beust answered in his blog and gave me some hints.

I want to add some more tips and comments about Cedric's list, based on my experience with the development of Hibernate annotations:

• default values, what about null?

• faking a non-existing annotation programmatically

• no extends, what about code duplication?

default values, what about null?

Nothing to declare, except that Cedric is right. The whole point is that there should be something that let's the annotation reader know whether the user set the value or used the default one. However, no such mechanism has been provided by the JSR.

As a workaround I've also build my own BooleanImplied enum type that takes care of values not explicitly set by the user. With Strings, I don't really like using some restricted key-string, such as #implied. For now, my special key-string works like a charm, but one day it will face the what-if-I-want-to-use-that-keyword-as-a-value syndrome. I've no better answer :-(

faking a non existing annotation programmatically

In annotation reader code, there is no way to programmatically build a new annotation /instance/, whether with or without default values. Of course, an annotation is an interface and there is no implementation provided! However, building your own annotation could be useful to keep the code as generic as possible and to fake a user provided annotation. (Especially if most of your annotations have default values).

Let's have a look at an example:

public @interface Column {
    /** 
     * column name
     * default value means the name is guessed.
     */
    String name() default "#implied";

    /**
     * column length
     * Default value 255  
     */
    int length() default 255;

    [...]
}

public @interface Property {

    /**
     * column(s) used
     * Default is guessed programmatically based on the default @Column()
     */
    Column[] columns() default {};

    [...]
}

@Property()
public String getMyProperty() {
    return myProperty;
}

I want to use the default values of @Column() when the user does not set any @Column() array in @property(). Why? Because I want the simplest annotations, I want implicit and default values to be the rule, and I don't want to duplicated my annotation processing code.

Two solutions:

• duplicate the default values somewhere in a value holder class (don't like that one)

• keep a fake method somewhere with the default @Column() inside:

/**
 * Get default value for annotation here
 * (I couldn't find a way to build an annotation programatically)
 */
@Column()
private static void defaultColumnAnnHolder() {}

You'll then be able to read and reuse this method in your annotation reader code. I think some bytecode manipulation framework can easily and elegantly provide such feature without the ugly extra method in your code. (Javassist 3.0 should be able to do that if I understand Bill Burke, but I've not tested it yet)

no extends, what about code duplication?

That one that really sucks. The extends keyword is not allowed for annotation. Without it, we can't build a generic method, handling an annotation hierarchy. Bye-bye to the beauty of OO.

Suppose I have @CoreProperty() and then @ComplexProperty():

public void handleProperties(CoreProperty ann) {
    [...]
}

This can't process @ComplexProperty(), as there is no way to let @ComplexProperty inherit @CoreProperty.

Some solutions:

• force your user to set @ComplexProperty() and @CoreProperty() on its annoted method (I don't consider this as an option)

• use composition

• copy annotation data into a new class hierarchy (or flatten it)

• use reflection on annotation

The second one is pretty simple but it may not be appropriate for every model:

@ComplexProperty(
    @CoreProperty(...)
)

If the CoreProperty is an implementation concept, it's pretty ugly to show that in the annotation.

The third solution kind of sucks, but that is the one I'm using now. The annotation reader has to be implemented with two layers:

• a layer reading and flatteninng the annotation (or putting it in an extra class hierarchy)

• a process method, handling properties in flat (or hierarchical) representation:

public void processPropertyAnn(Ann property) {
    [...]
    String name = property.name();
    processFlatProperty(name, null);
}

/**
 * Process a property
 * @param name property name
 * @param extraValue this extraValue can be set using ComplexProperty
 */
private void processFlatProperty(String name, String extraValue) {
    [...]
}

This leads to some extra metadata structure: the good old DTO are evil issue appears in annotation!

The fourth solution will probably be complex and unreadable code, it seems to me that reflection is overkill in this case (I haven't implemented it, however).

Time to conclude

Well, I stayed focused on JSR 175 practical flaws and gave some tips to make life easier. I still strongly believe annotations are good and will give us a new way of thinking metadata in our apps. I don't care which one is best: XDoclet or JSR 175, I care about a standard metadata facility (this topic is mature enough for standardization). I care about a standard metatada desription to easily plug tools on top of frameworks like Hibernate.

This spec will evolve. But for now, don't curse about it, use it for real!

Like, I suppose, many Java developers, I have so often read about the supposed scalability problems associated with stateful session beans, that I simply accepted that these problems were real, and refused to even consider using stateful beans. I guess this was laziness, but we don't have time to verify everything we read - and I'd never had cause to doubt that what I read was correct.

A few months ago, as Christian and I were nailing down our notion of application transactions in a hotel room in Kohn Kaen, Thailand, it really struck me hard that a stateful bean is the perfect place to keep state associated with the application transaction. (An application transaction is a unit of work /from the point of view of the user/; it spans multiple database/JTA transactions.) For example, you could keep a dedicated Hibernate session for the lifetime of the stateful bean, obviating the need for detaching and reattaching the object graph at every request.

This made me wonder about the cause of the scalability problems that everyone talks about. Once I really started thinking about this, it just didn't add up! A stateful bean without failover is quite equivalent to a HttpSession with server affinity, a design that is commonly understood to scale acceptably well. Similarly, failover for stateful beans should be no more difficult to implement than HttpSession failover.

Indeed, it seemed that the use of stateful beans should actually improve performance, since we would no longer need to wear the cost of serializing state relating to the application transaction to and from either the web tier or database /upon each request/. It seems far more natural to keep the state right there, in the middle tier, along with the business logic, where it belongs.

The conclusion I came to at that time was that the scalability problems must be due to implementation problems in existing appservers.

Well, I was talking about this with a friend who works for one of the other J2EE vendors the other day and she pointed me to this excellent paper:

HTTP Session Object vs Stateful EJB

which really debunks that particular superstition.

Actually, there is a lot of nonsense written about the desirability of so-called stateless architectures. It certainly might be true that a truly stateful design has some nice scalability characteristics. The trouble is that a stateless application can't really /do/ anything of consequence. In a real life application, the state /has to live somewhere/. Serializing it to and from the web tier is just a waste of breath. Serializing it to the database is even worse.

In future, I think I'll find myself using stateful beans all the time.

Lesson: beware J2EE folklore!

If you ever work with relational databases, you should go out and buy O'Reilly's /SQL Tuning/, by Dan Tow. The book is all about how to represent a SQL query in a graphical form and then, using some simple rules of thumb, determine an optimal execution plan for the query. Once you have found the optimal execution plan, you can add indexes, query hints, or use some other tricks to persuade your database to use this execution plan. Fantastic stuff. There is even sufficient introductory material for those of us (especially me) who know less than we should about the actual technical details of full table scans, index scans, nested loops joins, hash joins, etcetera to be able to start feeling confident reading and understanding execution plans. Unlike most database books out there, this book is not very platform-specific, though it does often refer specifically to Oracle, DB2 and SQL Server.

After more than a year of activity, development of the Hibernate2 branch has finally been wound up; Hibernate 2.1.3 will be one of the last releases and represents a rock-solid POJO persistence solution with essentially all the functionality needed by a typical Java application. Any future release of Hibernate 2.1 will contain only bugfixes. The branch that we have been calling 2.2, will actually be released as version 3.

The Hibernate project has previously had quite circumscribed goals - we have limited ourselves to thinking about just the very next release, and just what our users are asking for right now. That approach has finally reached the end of its usefulness. Hibernate3 is concieved in hubris, with the goal of innovating beyond what our users are asking for, or have even thought of. Certainly we will be adding features that go well beyond the functionality of the best commercial ORM solutions such as TopLink. (Some of these features will be most interesting to people working with certain specialized kinds of problems.)

So, in this spirit of hubris, I'll drop my normal policy of not boasting about things we have not yet implemented, and give a sketch of some of the things that are planned for Hibernate3 alpha (as usual, we do not commit to dates for future releases).

Virtualization

A number of important and interesting problems may be solved by presenting the user with a certain, filtered, subset of the data. For example, a user might want to see data valid at a particular point in time, or may have permissions to view data only in a particular region. Rather than forcing application code to specify this filter criteria in tedious query conditions, Hibernate 3 will allow the (parametrized) filters to be applied at the session level.

(This feature has not yet been implemented, though some of the needed refactorings are currently sitting on my laptop.)

More Mapping Flexibility

For the overwhelming majority of cases, Hibernate2 provides all the O/R mapping options you will need. However, there are now a few new features that are powerful, if used judiciously.

• single-class-to-multiple-table mappings using <join>

• table-per-concrete-class-mappings using <union-subclass>

• flexible discriminators using SQL formula mappings

We have long argued favor of extremely fine grained classes, and have viewed multi-table mappings as a huge misfeature (except for the very important special case of table-per-subclass inheritance mappings). Unfortunately, several commercial vendors insist upon trying to spin this misfeature as an actual competitive advantage of their products, causing us regular irritation. So we have provided the <join> mapping mainly to shut them up.

Well, I do conceed that there is one great use-case for <join>. We can now mix together table-per-hierarchy and table-per-subclass mappings in the same hierarchy. For example:

<class name="Document" table="DOCUMENTS">
    <id name="id">...</id>
    <discriminator column="TYPE" type="string"/>
    ...
    <subclass name="Book" discriminator-value="BOOK"/>
    <subclass name="Newspaper" discriminator-value="NEWSPAPER"/>
    <subclass name="XML"  discriminator-value="XML">
        <join table="XML_DOCUMENTS">
            ....
        </join>
    </subclass>
</class>

Hibernate's implicit polymorphism is a nice way to achieve most of the functionality of table-per-concrete-class without placing unnatural restrictions upon column types of different tables. It also nicely allows table-per-concrete-class to be mixed with other inheritance mapping strategies in the same hierarchy. Unfortunately, it has two limitations:

• we cannot have a polymorphic collection of the superclass type

• queries against the superclass resolve to muktiple SQL queries, which could be slower than using an SQL UNION.

The new <union-subclass> construct provides an explicit way to map classes to a table-per-concrete-class model and is implemented using SQL UNIONs.

If your table-per-class-hierarchy mapping does not feature a nice simple discriminator column, where values map one-to-one to the different classes, formula discriminators are for you. You can now discriminate between subclasses using multiple columns, multiple values of the same column, arbitrary SQL expressions or functions, even subqueries! An example:

<class name="Document" table="DOCUMENTS">
    <id name="id">...</id>
    <discriminator type="string"
        formula="case when TYPE='MONOGRAPH' then 'BOOK' when type='TABLOID'\
        or type='BROADSHEET' then 'NEWSPAPER' else TYPE end"/>
    <property name="type" column="TYPE"/>
    ....
    <subclass name="Book" discriminator-value="BOOK"/>
    <subclass name="Newspaper" discriminator-value="NEWSPAPER"/>
</class>

All of these features have been implemented in the Hibernate3 branch.

Representation Independence

Hibernate was conceived as a persistence solution for POJO domain models, and that remains the focus. Occasionally, we run into people who would like to represent their persistent entities in some more dynamic way, as a map, essentially. We used to point them to OFBiz Entity Engine. Well, Hibernate3 lets you represent your domain model as trees of HashMaps, or, with a little bit of user-written code, as just about anything. Here are three immediate applications of this new feature:

• reimplement JBoss CMP 2.1 engine on top of Hibernate

• reimplement OFBiz EntityEngine using Hibernate

• natively persist SDOs

Indeed, the end goal is to allow the application to switch between whichever representation is appropriate to the task at hand; the same entity might be represented as a typesafe POJO, a Map, or an SDO, all with just a single Hibernate mapping.

This feature has been implemented in the Hibernate3 branch.

JDK 1.5 Support

JSR 175 annotations are a perfect fit for Hibernate metadata and we will embrace them aggressively. Emmanuel Bernard is working on this.

We will also need to support Java generics, which basically boils down to allowing typesafe collections (which is very trivial).

Stored Procedure Support

We are not enormous fans of using stored procedures for CRUD operations (of course, there are some other use cases where SPs make wonderful sense) but people working with legacy databases often need Hibernate to call a SP instead of generating its own SQL statement. In Hibernate 2.1, it is possible to achieve this with a custom persister. Hibernate3 will allow arbitrary SQL statements to be specified in the mapping document for create, update and delete. Max Andersen is currently implementing this feature.

Full Event Driven Design

In the early days of the project, I applied YAGNI aggressively. Hibernate was occasionally criticized for supposed architectural deficiencies, due to the absence of a clear upfront design. (In my view, this lack of upfront design was not actually an architectural problem - at least by my understanding of the word architecture - but that debate is for another day...) Anyway, it was my firm belief that an elegant and flexible design would eventually grow naturally, and that our scarce attention and development resources were better spent solving actual user-visible problems. I now feel quite vindicated in this decision; Hibernate has flourished, and the design that has emerged is both powerful and reasonably elegant. Take that, YAGNI-skeptics!

There is one final step of this process, that is now in the hands of Steve Ebersole (and already partially implemented). Hibernate3 will feature an event-oriented design, with event objects representing any interesting thing that occurs, and listener classes that implement standard Hibernate behaviours, or customized user-defined behaviors. This user extension capability has applications ranging for auditing to implementing strange cascade semantics. (Our previous attempt to solve these problems - the Interceptor interface - proved to be insufficient.) Importantly, the redesign simplifies the current monolithic SessionImpl class.

New AST-driven Query Parser

Well, actually /two/ final steps. (No-one expected the Spanish inquisition, did they?)

When I wrote the HQL parser, I knew zip about parsers. Hibernate ended up with a wierd-ass handwritten pure-look-behind parser which, to my great and enduring surprise, has actually served us very well and been the source of very few bugs. (YAGNI, once again.) But it is now well past time we had a proper ANTLR-grammar-based AST, and Joshua Davis is currently writing one. This probably won't mean much in the way of user-visible changes, but it should allow us to more easily support other query languages such as EJBQL.

Declarative Session Management

This is a JBoss-specific feature for now. New users often find session management to be tricky at first. It can take a bit of thought to get your head around the schizophrenic role played by the Hibernate session as both a kind of cache, and a kind of connection. A JDBC Connection is stateless; a Hibernate Session, on the other hand, is a stateful connection! We further confuse the issue by telling people that, actually, the session is really representing a kind of transaction!

Rather than force people to implement their own session handling, messing with threadlocals and exception handling, it would be nice to have some way to specify the session model (session-per-database-transaction, or session-per-application-transaction) declaratively, in the metadata for a session bean. The Spring Framework already provides some support for this kind of approach. Michael Gloegl is working on implementing this functionality using a JBoss Interceptor. It would be wonderful to have this facility in appservers other than JBoss. J2EE needs portable Interceptors for EJBs (they already exist for servlets).

Well, I have some more things on my list, but that will do for now!