Help

I'm sitting in at Michael Nygard's talk at JAOO about handling failure in production systems. He wrote 'Release It!' a pattern book that describes common anti-patterns and illustrates how they can cause massive downtimes and lost business. During his talk he called out ORM Tools to be bad at handling some of these which urged me to make this blog.

Unbounded result set

He mentioned ORM Tools are bad at handling the pattern he calls Unbounded result set which is done by applications assuming that a query will only return a few records but when put into production the query suddenly starts returning million of rows instead of those few that were in the development/test systems.

I asked Michael what he actually meant by this since Hibernate have always had pagniation support allowing you to safely iterate over results in a controlled manner:

Query q = sess.createQuery("from OrderLines lines where lines.order = :id")
           .setProperties(order)
           .setFirstResult(20).setMaxResults(10)
List lines = q.list();

It turns out what Michael was referring to (as I expected) was that developers instead of doing focused queries maps where mapping their complete object model blindly and was doing this instead:

List lines = order.getOrderLines();

which probably works fine in 99% of the cases because there seldom are more than a few records in an Order, but sometimes you got an enterprise making a big order and you can end up with thousands line items which can have a ripple effect on the memory and cpu usage when each orderline and its connected objects starts to be fetched/used.

In my mind that is not really the fault of the ORM Tool, but more that Java's collections does not have the API to provide a transparent iteration over large collections. But saying that does not make the problem go away, some ORM Tools solves this with vendor specific collection's - in Hibernate we have Collection filters:

Collection lines = session.createFilter(
    order.getOrderLines(), "")
    .setFirstResult(20).setMaxResults(10)
    .list();

This gives you the exact same pagniation features on mapped collections as normal queries and helps you avoid the Unbound Result Set anti-pattern.

Timeouts

Another issue Michael urged developers to take care of is to always use timeouts for any remote calls, incl. database queries.

Timeouts are important to use when you are accessing external systems which might become overloaded and start having slow response times or maybe even never respond.

Hibernate have support for Transaction time outs:

try {
    //set transaction timeout to 3 seconds
    sess.getTransaction().setTimeout(3);
    sess.getTransaction().begin();

    // do some work
    ...

    sess.getTransaction().commit()
}
catch (RuntimeException e) {
    sess.getTransaction().rollback();
    throw e; // or display error message
}
finally {
    sess.close();
}

and Query timeouts.

Query q = sess.createQuery("from OrderLines lines where lines.order = :id")
           .setProperties(order)
           .setFirstResult(20).setMaxResults(10)
           .setTimeOut(3);
List lines = q.list();

which can also be used on the Collection filters:

Collection lines = session.createFilter(
    order.getOrderLines(), "")
    .setFirstResult(20).setMaxResults(10)
    .setTimeout(3)
    .list();

Hibernate does not do anything magically here we just delegate the timeout to the underlying systems thus the functionality are dependent on your database or application server vendors implementing the standard timeout functionalities.

I hope Michael will be glad to know (I know he knows ;) that if you by accident repeats some of these anti-patterns Hibernate does have the features to allow you to avoid them.

11 comments:
 
30. Sep 2008, 21:37 CET | Link
Excellent post. I'm happy to see these recipes here. It's great to see that there are such easy ways to get Hibernate to handle these operational concerns.

The main point I try to make is that developers and architects need to consider these issues. It's not so much that the tools are incapable of handling timeouts and limited results, but rather that most developers don't bother to do so.
ReplyQuote
 
01. Oct 2008, 03:48 CET | Link

Yes, excellent post. I was unaware of the setTimeOut() method on transactions and that will be very helpful for me. This was just what I was looking for. I also am stoked to see pagination, this will also greatly improve my applications' performance and is a concept I'm looking to integrate ASAP. Thanks Max!

01. Oct 2008, 20:08 CET | Link
reisi
Now that people have successfully adopted Spring WebMVC to do their web applications with rather ease, how would you suggest to avoid errors like the "Unbounded Result Set" you have discussed above in settings like using separate DAO-layers to abstract the use of Hibernate/IBATIS/etc.?

Using Collection filters right in the web controller code would be big no no, even bigger no for using those in JSPs (views).

Perhaps having a common collection filtering Interfaces/support from the used framework would be nice idea, to enable those being used in controller -level code. I think Controller level would be preferred as in you don't even have to write a complex system to notice your dao-interfaces grow and grow and become more and more vendor specific.

How would one implement the above in a framework? No idea, I guess that too would require some framework dependencies either at POJOs (doesn't sound too comforting idea) or at the DAO-layer.
01. Oct 2008, 21:14 CET | Link

I don't believe in abstracting out such an essential part as the ORM layer, and especially not when the two framework you mention (iBatis and Hibernate) will require very different flow of your application to be used effectively.

Ignoring that part then I never suggested you should use the Collection filters directly in your viewlayer; it would be something your DAO's would do.

I would use the Query level pagination which should exist in any decent persistence framework.

 

--max

 
01. Oct 2008, 23:05 CET | Link
Marcos de Sousa

Definitively I love Hibernate.

I am already using hibernate pagination support at all part of an application.

I really forget about Hibernate have support for Transaction timeouts and Query timeouts.

Since I always use hibernate with spring, I think Query timeouts is easy to acquire, hope Transaction timeouts too.

I am thinking that in transaction timeouts it really means I can // do some work related to hibernate or not of course

Today I will start to use it with more attention.

Thanks

 
03. Oct 2008, 07:53 CET | Link

@Michael, I would go a step further and say that it is ridiculous that we have to tap into a vendor extension to get these core features, a sure oversight of the JPA 1.0 specification. The filters feature is a necessity in the case in question since it isn't always convenient or possible to issue a new query using the Query interface. You could argue that the lack of filters in JPA is understandable, but timeouts certain is not, yet they are absent in the API. I just wish the standards bodies could get more serious about including the features that prevent the downtime in applications that you cite so beautifully in your book. And while I'm on the topic, great job on the book. I smiled many times in concert with your experiences.

 
18. Jun 2014, 04:40 CET | Link

Click HELP for text form Let's face it: that while Hermes replica handbags people's aboriginal best as far as handbags go, the accuracy of the amount is that the appraisement of aboriginal Gucci accoutrements generally drives them actually above the ability of abounding humans who would adulation to accept theirs.Personalized items such as abstract totes and monogrammed tote accoutrements can accomplish a abundant allowance account account for ladies. You may duke it as a appropriate present during birthdays, anniversaries, weddings, graduation ability Replica watches and added celebrations. This is fabricated appropriate because you'll be able to beautify and cipher the cast of your receiver, that of advance can put a smile on her face. This can be bought at your bounded food and online stores. A lot of online food action cheaper rates, and advantageous abundant if they don't allegation added fee for adornment or monogramming. For a lot of of the humans who adulation Fendi Replica Handbags but acquisition them too baby to afford, Replica Gucci accoutrements action an addition - these accepting accoutrements that are marketed as accepting actually like the aboriginal Guccis, but advancing at abundant added affordable prices.atting instructions. Then edit this text and check the preview.

 
30. Jul 2014, 12:39 CET | Link
Steve Lopez | stev45lop(AT)gmail.com

The unbounded result sets have always created problems for me. This is mainly because this application is too unsafe. I am tired of doing debugging and that sort of stuff. windows support scam I hope you can help me with a permanent solution.

 
05. Oct 2014, 09:32 CET | Link
lkm

Try getting ready to do shows on the home around late Feb, that way you have about 5 good months of possible customers before it decelerates. National Home Buyers

 
04. Nov 2014, 04:18 CET | Link

I asked Michael what he actually meant by this since Hibernate have always had pagniation support allowing you to safely iterate over results in a controlled manner:

 
10. Nov 2014, 14:54 CET | Link
thanks

Filmmaker Ravine Change , Datascrip said Reedlike soul sophistication and hybrid ink method printer, gives author evaluate for its users to play the situation illustrator wild and sensual , if you need the driver printer, you can check at http://driverresetter.blogspot.com They all go together with Epsons built piezo-electric subject proverbial as PrecisionCore that, per the companys spokespersons, doubles photograph grade whereas doubling if you need the driver printer, you can check at http://2downloaddriver.com From the leaked it , mentioned the Samsung Collection Set 2 performed with sib SIM , 4.5 in. WVGA direct ( 480x800 pixels ) and a quad-core processor 1.2 GHz , quicker than their predecessors that the if you need the gadgets info, you can check at http://gadgetsdrunk.blogspot.com Request the exclusive angle , not notable manifestly his sincere write , yet by worldcarfans , weekday ( seventeen / 04 ) , if you need the last info about car automotive, you can check at http://yourautocar.blogspot.com Canon Pixma Mp560 Driver For Windows, Mac OS X, and Linux Download, but let me to review the printer first. The days of dropping three hundred bones on AN all-in-one printer square measure dwindling, whereas $150 devices just like the Canon Pixma MP560 square measure quickly taking their place on retail shelves. The MP560 appearance nice ANd offers helpful options like an auto-document if you need the last info update about technology, you can check at http://echokita.blogspot.com To any raise human publication caliber, PIXMA MG7570 adopts a 6-color mortal ink scheme with the element of achromatic ink tank to spicery up the vesture sound in B W someone printing, understandably sharing the hatched and shining if you need more info about geophysics, you can check at http://rahmatprinterdriverdownload.blogspot.com

Post Comment