Help

I have been fed up with Velocity ís ability to ignore and even hide errors and exceptions occurring in the templates used in Hibernate Tools .

This blog tells about why and how FreeMarker became my new interest. If you just want to see the results then go and fetch the code in the TOOLS_FREEMARKER branch...read on to get the full story.

The problems with Velocity

I started to see more and more forum postings and bug reports about issues that were caused by typoís in users templates or even worse errors in the Hibernate Tools. Many of these issues would be solvable within seconds if Velocity would actually tell where in the templates the error occurred and unit tests would have failed if underlying exceptions were exposed; but Velocity simply does not.

I have added every safety-precaution I have been able to apply to Velocity error handling. I have created my own UberSpect and EventHandler implementation that will not allow you to invoke methods that does not exist and I have tweaked the logging settings to be more informative; but it does not (hardly) solve all the problems that can occur.

Logging is excessive in Velocity even at WARN and INFO level, one good reason for this is most likely that the developers know that Velocity is ignoring situations where it should actually fail, thus since there is no easy other way implemented in Velocity they put it in the log for users to discover by accident!

The choice originally fell on Velocity since it was the biggest player around, and I added it naively thinking that the error and log handling could not be that bad if so many people were using it and if there were an issue it would be fixed soon.

As time went by I learned that it was definitely not the case.

The beauty of FreeMarker

Last week I decided to look around for alternatives, the only real alternative I found were FreeMarker; everything else looked either too simple or way to complex for the Hibernate Tools needs. Now that I have spent just 1,5 day converting the existing Velocity templates to FreeMarker Iím more than happy I did.

Here are some examples of the beauty of FreeMarker:

Assume we have the following bean:

public class Table {
 String getName();
}

The bean is available via table in the following code:

${table.namee}

That typo will just be ignored by default in Velocity, with a custom EventHandler it can be convinced to throw an exception which comes out like this:

Caused by: java.lang.IllegalArgumentException: $table.namee is not a valid reference.
at org.hibernate.tool.hbm2x.HibernateEventHandler.referenceInsert([=>HibernateEventHandler.java:11])
        at org.apache.velocity.app.event.EventCartridge.referenceInsert([=>EventCartridge.java:131])
        ... 19 more

No information about which template nor where in the temmplate it went wrong.

In FreeMarker I get the following with no special configuration and custom code:

Expression table.namee is undefined on line 15, column 14 in doc/tables/table.ftl.
The problematic instruction:
----------
==> ${table.namee} [on line 15, column 12 in doc/tables/table.ftl]
----------

Java backtrace for programmers:
----------
freemarker.core.InvalidReferenceException: 
 Expression table.namee is undefined on line 15, column 14 in doc/tables/table.ftl.
    at freemarker.core.TemplateObject.assertNonNull([=>TemplateObject.java:124])
    at freemarker.core.Expression.getStringValue([=>Expression.java:118])
    at freemarker.core.Expression.getStringValue([=>Expression.java:93])
    ...

Nice! And even better, the on line 15, ... works like a link in e.g. Eclipse Console view. Clicking it brings you to the location of the error in the table.ftl. file.

Similar and precise error messages you get if you refer to non existing methods, Just brilliant! The great thing is that if I really wanted FreeMarker to ignore this I could do so by installing a different Exception handler. But that is my choice, not a hard to change behavior.

The built in primitives in FreeMarker is also great, e.g. <#assign> that allows me to store any generated output in a variable for later usage.

${pojo.getPackageDeclaration()}
// Generated ${date} by Hibernate Tools ${version}
<#assign classbody>
 <#include "PojoTypeDeclaration.ftl"/> {
  ..more template code..
 }
</#assign>

${pojo.generateImports()}
${classbody}

This allows me to remove the need to have a magically second-pass which I did with Velocity. There are more gems like these to be found in the excellent FreeMarker documentation .

Another big plus in FreeMarker's favor is the Configuration API . Let us compare, here is our Velocity setup:

engine = new VelocityEngine();
context = new VelocityContext();

EventCartridge ec = new EventCartridge();
ec.addEventHandler(new HibernateEventHandler()); // stricter evaluation
ec.attachToContext( context );
            
Properties p = new Properties();
p.setProperty(RuntimeConstants.RUNTIME_LOG_LOGSYSTEM_CLASS, 
              "org.apache.velocity.tools.generic.log.CommonsLogLogSystem");
p.setProperty(CommonsLogLogSystem.LOGSYSTEM_COMMONS_LOG_NAME, 
              "org.hibernate.tool.hbm2x.template");

p.setProperty( RuntimeConstants.UBERSPECT_CLASSNAME, HibernateUberspect.class.getName() ); // stricter evaluation

p.setProperty("velocimacro.library",""); // make it shut up about VM_global_library blah
      
p.setProperty("resource.loader", "file, class");
p.setProperty("file.resource.loader.path", directory );
p.setProperty("class.resource.loader.class", ClasspathResourceLoader.class.getName() );

engine.init(p);

Here is the corresponding FreeMarker config:

engine = new Configuration();
context = new SimpleHash(ObjectWrapper.BEANS_WRAPPER);

//Logger.setCategoryPrefix("org.hibernate.tool.hbm2x.template"); 
// Not really needed since the logging is much more sensible.

freeMarkerEngine.setTemplateLoader(new MultiTemplateLoader( 
                                    new FileTemplateLoader(directory), 
                                    new ClassTemplateLoader(this.getClass(),"/"));

Notice the difference? FreeMarker has good practice defaults and actually allows me to use java code to configure it; what a neat concept.

The only two bad things I have found yet with FreeMarker is that itís syntax is based on <#..> which does not compute very well when trying to show it in an XML editor. This has been fixed in the latest release by also allowing [#...] syntax.

Another bigger issue is that ${} and #{} is not escapable. This syntax collides in templates that generates ant build and jsp files.

In Velocity they were just ignored (the only place were it were useful to ignore them). FreeMarker complains because the values are undefined and unfortunately there is no easy-on-the-eyes method to escape these characters. The following show the methods that I found to allow the me to output ${..}:

${r"${build.dir}"}  
${'$í}{build.dir} 
<#noescape>${build.dir}</noescape>

Still the brilliant exception handling, powerful template language and configuration API makes FreeMarker a much better choice for Hibernate Tools.

What now ?

Velocity served me well and is probably serving many projects well; it just did not cut it well for Hibernate Tools. Today I am convinced that I could have saved myself and the Hibernate world from a lot of trouble if I had decided to use FreeMarker from the beginning.

Come and see for your self in the TOOLS_FREEMARKER branch. The code in there will be merged into the main development in the near future unless someone steps up with a very good reason for not doing so ;)

To be fair, I must tell you that Velocity 1.5 is being worked on right now, and it does seem to solve some of these issues, but not completely and Velocity has some external dependencies I would rather not add to the tools project.

23 comments:
 
04. Feb 2006, 01:10 CET | Link
Hani Suleiman
You're absolutely right. Velocity is a great example of why Apache is evil. Back when it first started, the freemarker people got quite upset because Velocity came along, and was a FAR inferior product, yet managed to get all the attention and developer mindshare simply through the Apache name.

It's nice to see that people are finally realising that freemarker is superior, it's such a shame though that in the interim, so many people have hitched their wagons to such an abysmal product.
ReplyQuote
 
04. Feb 2006, 23:00 CET | Link
Will Glass-Husain | wglass(AT)forio.com
Max,

Thanks for sharing your story. Good feedback to the Velocity community on important issues re: error handing and integration.

Velocity is progressing slowing but inexorably towards a new 1.5 release. We've had
a complete turnover in the committer community in the past year. Somehow I've turned from an enthusiastic if
semi-frustrated user into a committer in part due to concern about similar issues concerning integration with my applications.

There's a lots of new features that have piled up. In particular, ones that are relevant to your post:

(1) passing through all RuntimeExceptions to the calling code (rather than catching and logging),
(2) reporting template name, line, and column with all errors,
(3) new event handler that allows configuration via a properties file instead of attaching to a context
(4) a new event handler that catches all invalid references

I'm thinking about the other points you mention, particularly those about easy-to-understand error messages.

Anyway, good luck with your new approach. I'm a big fan of Hibernate, and use the Tools library extensively.
(albeit an earlier version) In the end all that matters is that you can create a good product for your users.

Will Glass-Husain
committer, Jakarta Velocity

P.S. Ouch, Hani - that's a little harsh!


 
05. Feb 2006, 02:25 CET | Link
Anonymous
"There's a lots of new features that have piled up."

For some reason you remineded me of this:
 
05. Feb 2006, 02:25 CET | Link
Anonymous
"There's a lots of new features that have piled up."

For some reason you reminded me of this:

http://tauquil.com/archives/2006/01/06/re-introducing-the-real-windows-vista/
 
05. Feb 2006, 04:31 CET | Link
Will Glass-Husain | wglass(AT)forio.com
I know I know. An updated release is long overdue.

Of course, there's a big difference between Windows Vista and an open source project like Velocity. All the features I mentioned except #4 are already in the source code tree and available. That's what I meant by "piling up". Coded, put in source control, but not labeled "release". (and #4 has a proposed patch in JIRA). Anyone can download the latest code, read the draft docs, and ask questions on the mailing list. Many have done so.

All that's left for the 1.5 release right now is bug fixes, some documentation updates, and a site upgrade.

Again, thanks to Max for the critical and candid comments.

Cheers, WILL

 
05. Feb 2006, 06:05 CET | Link
Eitan
sounds like a step in the right direction. i've been using erb recently and i like it.
 
05. Feb 2006, 16:06 CET | Link

(3) new event handler that allows configuration via a properties file instead of attaching to a context

The part of Velocity's configuration I actually liked we the possiblity of configuring the EventHandler's in Java api instead of typeless properties.





 
06. Feb 2006, 22:24 CET | Link
Jonathan Revusky
Well, the fact is that even with the extra features that Will has outlined for Velocity 1.5, it does not fully catch up with functionality FreeMarker has had for the past 3 years at least.

For example, there is a wiki page describing a Velocity 2.0 (2.0 is pure vapor) wishlist and it seemed to me looking at it that every last point on the 2.0 wishlist is already implemented in the stable production version of FreeMarker!

I re-iterate: the features that Max has discovered in FM in the last week or so, that he is oohing and aahing over, have been part of the stable/production version of FM for well over 3 years! And FM is a fairly well known project. This means that this codebase has been subjected to the stresses of real-world usage in a large number of projects out there. This is a codebase that is very well tested.

Now, I am obviously biased. For example, as regards the line/col error reporting in FreeMarker, I was the one who implemented this in the code. But what I want to know from Will is why on earth anybody looking for this kind of tool should opt for Velocity 1.5 beta as opposed to FreeMarker 2.3 stable? AFAICS, the reason to use beta code is that you need the features. However, Velocity 1.5 offers no features that are not present in the current highly stable, well tested version of FreeMarker. In fact, it's quite the contrary. Velocity 1.5 does not catch up with versions of FM of even several years ago.

As far as I can tell, the only reason that so many projects do opt for Velocity is entirely nontechnical -- the huge visibility advantage of apache.org. But correct me if I'm wrong, Will, but I see no technical reason to use Velocity. Any technical comparison between FM and Vel is devastating. This blog entry from Max is just one of various similar blog entries where people switched to FreeMarker and were extremely happy. Are there any similar blog entries detailing a switch in the other direction?


 
06. Feb 2006, 22:33 CET | Link
I used to use FreeMarker alot a year or so ago and it is great. With regards to the issue with XML editors, there is an eclipse (and others) plugin for FM at http://freemarker.sourceforge.net/editors.html
 
07. Feb 2006, 02:01 CET | Link
Vincent
The reason I dislike Velocity is because it doesn't allow for clean separation of template logic and model objects.

Ever tried getting a correctly formatted Date in a template with Velocity? Not possible, unless you format the Date as a String in your model. But I don't want a String in my model. I want a Date!

FreeMarker supports this much better, and even allows you to register formatters for user-defined types.
 
07. Feb 2006, 03:42 CET | Link
oooh and aaaah ;)

For the record I knew about the goodness of Freemarker when I started the project, but velocity seemed to be the "defacto" of templating and I did not really imagine that velocity could be so non-fixable.

But now after a year of velocity usage I felt that enough was enough - and from that you get a conversion story and a ooh-aah blog ;)
 
07. Feb 2006, 11:07 CET | Link
just another way to escape ${}

${"$" + "{build.dir}"}
 
08. Feb 2006, 15:46 CET | Link
sohaib
Recently I had to choose between Velocity and Freemarker and I chose Freemarker for exactly the same reasons outlined in this blog. The only thing that I don't like about Freemarker is that it is less readable.

While we are discussing template engines I was wondering why StringTemplate and WebMacro haven't come to the spot light.

 
09. Feb 2006, 16:00 CET | Link
max
webmacro feels/works very much like velocity...

stringtemplate has some nice ideas, but didn't feel complete enough.


 
10. Feb 2006, 19:03 CET | Link
Attila Szegedi
Jonathan wrote: "For example, as regards the line/col error reporting in FreeMarker, I was the one who implemented this in the code."

Yep, and I was one sanitized it to only display macro invocations and includes in the stack trace.

Speaking of which, one notable thing about the error reporting is that it won't just pinpoint you the exact location of the error, but also it will pinpoint how execution got there, the template stack trace, in essence. i.e. if you have

a.ftl
----

----

b.ftl
----

${y.foo}



----

Then you'll see this (assuming "bar" is a simple string):

Expected hash. y evaluated instead to freemarker.template.SimpleScalar on line 2, column 3 in b.ftl.
Quoting problematic instruction:
----------
==> ${y.foo} [on line 2, column 1 in b.ftl]
 in user-directive printFoo [on line 5, column 1 in b.ftl]
 in include "b.ftl" [on line 1, column 1 in a.ftl]
----------

So you can see precisely what template execution path led to the error - invaluable to figure out what's going on when you can have macros and includes invoked from multiple locations.

 
10. Feb 2006, 19:05 CET | Link
Attila Szegedi
Reposting the code snippets with alternate syntax, as this blog engine is not friendly with pointy brackets...

a.ftl
----
[#include "b.ftl"/]
----

b.ftl
----
[#macro printFoo y]
${y.foo}
[/#macro]

[@printFoo bar/]
----
 
14. Feb 2006, 14:23 CET | Link
Erik G
We made the jump from Velocity to FreeMarker about 2 years ago for a code-generation tool and we haven't looked back.

One of my favorite features is the super easy way to write macros (including parameterized macros.)

Regarding escaping #{}, we wrote a macro to encapsulate the issue (hb = hash-brace)

$lt;#macro hb&gt;${r"#{"}${r"}"}&lt;/#macro&gt;

which gives you in your templates:

foobar
 
14. Feb 2006, 14:24 CET | Link
Erik G
Whoops, that macro def should be:

&lt;#macro hb&gt;${r"#{"}${r"}"}&lt;/#macro&gt;
 
14. Feb 2006, 14:27 CET | Link
Erik G
hmmm... darn thing ate my #nested tag :-)

&lt;#macro hb&gt;${r"#{"}&lt;#nested&gt;${r"}"}&lt;/#macro&gt;
 
11. Mar 2006, 17:48 CET | Link
Armond
Max, when do you expect to have the next version of HibernateTools using FreeMaker? Going to be in 3.1.0?
 
27. May 2006, 11:12 CET | Link
Bill
Nearly installed Velocity until it failed due to the fact that there is no stable version that supports the current Java JDK.

Went straight to FreeMarker. Seems to work so far, just learning.
 
21. Nov 2007, 20:35 CET | Link

I read ur article n i m too a newbie to freemarker. what if i want to ignore the InvalidReferenceException in my java code? can u suggest me other way?

11. Dec 2013, 18:58 CET | Link

Now that 8 years have passed... I wonder what you think of StringTemplate. Or anyone that has used it along with Velocity or Freemarker. I chose StringTemplate for my project after just a quick evaluation of Freemarker and Velocity. Freemarker seemed fragmented and I wasn't convinced of it's stability. Velocity seemed complex.

Post Comment