Help

I'm sitting in at Michael Nygard's talk at JAOO about handling failure in production systems. He wrote 'Release It!' a pattern book that describes common anti-patterns and illustrates how they can cause massive downtimes and lost business. During his talk he called out ORM Tools to be bad at handling some of these which urged me to make this blog.

Unbounded result set

He mentioned ORM Tools are bad at handling the pattern he calls Unbounded result set which is done by applications assuming that a query will only return a few records but when put into production the query suddenly starts returning million of rows instead of those few that were in the development/test systems.

I asked Michael what he actually meant by this since Hibernate have always had pagniation support allowing you to safely iterate over results in a controlled manner:

Query q = sess.createQuery("from OrderLines lines where lines.order = :id")
           .setProperties(order)
           .setFirstResult(20).setMaxResults(10)
List lines = q.list();

It turns out what Michael was referring to (as I expected) was that developers instead of doing focused queries maps where mapping their complete object model blindly and was doing this instead:

List lines = order.getOrderLines();

which probably works fine in 99% of the cases because there seldom are more than a few records in an Order, but sometimes you got an enterprise making a big order and you can end up with thousands line items which can have a ripple effect on the memory and cpu usage when each orderline and its connected objects starts to be fetched/used.

In my mind that is not really the fault of the ORM Tool, but more that Java's collections does not have the API to provide a transparent iteration over large collections. But saying that does not make the problem go away, some ORM Tools solves this with vendor specific collection's - in Hibernate we have Collection filters:

Collection lines = session.createFilter(
    order.getOrderLines(), "")
    .setFirstResult(20).setMaxResults(10)
    .list();

This gives you the exact same pagniation features on mapped collections as normal queries and helps you avoid the Unbound Result Set anti-pattern.

Timeouts

Another issue Michael urged developers to take care of is to always use timeouts for any remote calls, incl. database queries.

Timeouts are important to use when you are accessing external systems which might become overloaded and start having slow response times or maybe even never respond.

Hibernate have support for Transaction time outs:

try {
    //set transaction timeout to 3 seconds
    sess.getTransaction().setTimeout(3);
    sess.getTransaction().begin();

    // do some work
    ...

    sess.getTransaction().commit()
}
catch (RuntimeException e) {
    sess.getTransaction().rollback();
    throw e; // or display error message
}
finally {
    sess.close();
}

and Query timeouts.

Query q = sess.createQuery("from OrderLines lines where lines.order = :id")
           .setProperties(order)
           .setFirstResult(20).setMaxResults(10)
           .setTimeOut(3);
List lines = q.list();

which can also be used on the Collection filters:

Collection lines = session.createFilter(
    order.getOrderLines(), "")
    .setFirstResult(20).setMaxResults(10)
    .setTimeout(3)
    .list();

Hibernate does not do anything magically here we just delegate the timeout to the underlying systems thus the functionality are dependent on your database or application server vendors implementing the standard timeout functionalities.

I hope Michael will be glad to know (I know he knows ;) that if you by accident repeats some of these anti-patterns Hibernate does have the features to allow you to avoid them.

7 comments:
 
30. Sep 2008, 21:37 CET | Link
Excellent post. I'm happy to see these recipes here. It's great to see that there are such easy ways to get Hibernate to handle these operational concerns.

The main point I try to make is that developers and architects need to consider these issues. It's not so much that the tools are incapable of handling timeouts and limited results, but rather that most developers don't bother to do so.
ReplyQuote
 
01. Oct 2008, 03:48 CET | Link

Yes, excellent post. I was unaware of the setTimeOut() method on transactions and that will be very helpful for me. This was just what I was looking for. I also am stoked to see pagination, this will also greatly improve my applications' performance and is a concept I'm looking to integrate ASAP. Thanks Max!

01. Oct 2008, 20:08 CET | Link
reisi
Now that people have successfully adopted Spring WebMVC to do their web applications with rather ease, how would you suggest to avoid errors like the "Unbounded Result Set" you have discussed above in settings like using separate DAO-layers to abstract the use of Hibernate/IBATIS/etc.?

Using Collection filters right in the web controller code would be big no no, even bigger no for using those in JSPs (views).

Perhaps having a common collection filtering Interfaces/support from the used framework would be nice idea, to enable those being used in controller -level code. I think Controller level would be preferred as in you don't even have to write a complex system to notice your dao-interfaces grow and grow and become more and more vendor specific.

How would one implement the above in a framework? No idea, I guess that too would require some framework dependencies either at POJOs (doesn't sound too comforting idea) or at the DAO-layer.
01. Oct 2008, 21:14 CET | Link

I don't believe in abstracting out such an essential part as the ORM layer, and especially not when the two framework you mention (iBatis and Hibernate) will require very different flow of your application to be used effectively.

Ignoring that part then I never suggested you should use the Collection filters directly in your viewlayer; it would be something your DAO's would do.

I would use the Query level pagination which should exist in any decent persistence framework.

 

--max

 
01. Oct 2008, 23:05 CET | Link
Marcos de Sousa

Definitively I love Hibernate.

I am already using hibernate pagination support at all part of an application.

I really forget about Hibernate have support for Transaction timeouts and Query timeouts.

Since I always use hibernate with spring, I think Query timeouts is easy to acquire, hope Transaction timeouts too.

I am thinking that in transaction timeouts it really means I can // do some work related to hibernate or not of course

Today I will start to use it with more attention.

Thanks

 
03. Oct 2008, 07:53 CET | Link

@Michael, I would go a step further and say that it is ridiculous that we have to tap into a vendor extension to get these core features, a sure oversight of the JPA 1.0 specification. The filters feature is a necessity in the case in question since it isn't always convenient or possible to issue a new query using the Query interface. You could argue that the lack of filters in JPA is understandable, but timeouts certain is not, yet they are absent in the API. I just wish the standards bodies could get more serious about including the features that prevent the downtime in applications that you cite so beautifully in your book. And while I'm on the topic, great job on the book. I smiled many times in concert with your experiences.

 
18. Jun 2014, 04:40 CET | Link

Click HELP for text form Let's face it: that while Hermes replica handbags people's aboriginal best as far as handbags go, the accuracy of the amount is that the appraisement of aboriginal Gucci accoutrements generally drives them actually above the ability of abounding humans who would adulation to accept theirs.Personalized items such as abstract totes and monogrammed tote accoutrements can accomplish a abundant allowance account account for ladies. You may duke it as a appropriate present during birthdays, anniversaries, weddings, graduation ability Replica watches and added celebrations. This is fabricated appropriate because you'll be able to beautify and cipher the cast of your receiver, that of advance can put a smile on her face. This can be bought at your bounded food and online stores. A lot of online food action cheaper rates, and advantageous abundant if they don't allegation added fee for adornment or monogramming. For a lot of of the humans who adulation Fendi Replica Handbags but acquisition them too baby to afford, Replica Gucci accoutrements action an addition - these accepting accoutrements that are marketed as accepting actually like the aboriginal Guccis, but advancing at abundant added affordable prices.atting instructions. Then edit this text and check the preview.

Post Comment