I spent my Friday doing quite a bit of performance benchmarking, and some of the observations where quite eye-opening.
The basic test was retrieving 50 different objects with 2 related collections per object (1-to-many mapping with Hibernate) 1000 times. I did four different tests:
- With Hibernate, no caching: test took 192 seconds.
- With Hibernate, EHCache enabled as 2nd level and query cache: 60 seconds.
- Direct access to EHCache, only use Hibernate on a cache miss: 20 seconds.
- Direct access to EHCache, only use Hibernate on cache miss, only open a Hibernate session when required: 1 second.
It has to be noted that my initial results with Hibernate and EHCache as a Hibernate cache was even worse – there are a few gotchas in there, among them being applying the Caching annotations on both class and collection relationship level, and enabling the query cache properly.
But still, the best case is a factor of 60 better than the best case with Hibernate caching, and almost 200 times better than no caching at all!
It’s quite obvious from these numbers that caching can improve performance tremendously, even in the “worst” case with caching.
However there are a few take-aways from this apart from that simple observation – it would seem that Hibernates use of second level and query caching is highly inefficient compared to just hitting a cache explicitly yourself, why this is, I do not know.
Secondly, there seems to be a major overhead in just opening and closing Hibernate Sessions (as per the Open Session in View/transaction per request pattern).
The fact that in-memory operations are quicker than a relational database is hardly a surprise, but even when Hibernate is not supposed to do any queries, it seems to slow things down, which to me is a surprise (indicates to me it’s doing things internally it shouldn’t, or doing things it should very inefficiently).
The Open Session in View pattern is extremely convenient and productive to use for developers – but I think my findings make a pretty good case for someting akin to a “lazy initialized Open Session in View” – where you would have a “transaction context” of sorts around a request-response, but only actually initialize it IF you have cache misses OR need to write to the database.
Spring and Hibernate most certainly do not do anything like this in their implementations of the OSIV pattern, I found they will happily get a db connection for each request regardless of whether it is actually needed or not.
Finally, I should add that the Open Session in View pattern and these findings should not be a problem for 99% of web apps/sites, but at the moment I am working on one of those diverging high traffic, high volume sites where this might become an issue..
March 23, 2009 at 7:47 am
Very interesting findings, thank you! I will take that into account in my current project.
March 23, 2009 at 9:58 am
Hi Wille,
Wow, that is quite a difference!
Maybe we need an ‘Open Session in View – DB connection when needed’ pattern.
Have you tried enabling logging to see what Hibernate does in all those precious extra seconds?
– Daan
March 23, 2009 at 10:15 am
Daan: That is exactly what we need. Hibernate will probably also exhaust the connection pool before the server otherwise has run out of steam under high concurrent load, so Open Session in View as it is implemented by most today will limit concurrency as well.
I have even gotten the statistics out of Hibernate, measuring cache hits and misses. Enabling query caching properly is very important (something I think most probably miss out on, thus missing out on most of the benefit of Hibernate caching).
March 23, 2009 at 9:43 pm
Just verifying: did you use the SQL or HQL as the query cache key? Did you use Hibernate’s standard query cache to retrieve results as Lists of primary keys and then resolved the entities from the 2nd level cache as Hibernate does?
March 25, 2009 at 2:15 am
Hi Wille,
In my work as a developer for Oracle Coherence, I’ve observed my fair share of caching with Hibernate. Without a doubt, performance and scalability are far greater if you run a cache in front of Hibernate instead of relying on L2 caching.
In fact, Coherence can read through to the database upon a cache miss which means client code can deal with the cache directly and not have to worry about opening Hibernate sessions. I imagine this means that use of the Open Session in View would no longer be necessary, although I’ve never tested this myself.
Anyway enough of my shameless plug.
I enjoy your blog, keep the posts coming!
-Patrick
March 25, 2009 at 10:25 am
ophir:
Hibernate Query Cache and Hibernate 2nd level cache. I’m still debugging how things are retrieved, as the code is quite complex (not written by me).
Patrick:
No worries about the plug, good to get third party confirmation that my observations are accurate.
April 30, 2009 at 12:30 am
[...] April 30, 2009 Posted by Wille in Java, Software Development, Wicket RAD. trackback I had a rant a while back about how the Open Session in View pattern is mostly useful, but under some [...]
October 13, 2009 at 12:05 pm
If I recall correctly, opening a Session in Hibernate will also pick up a connection from the connection pool. So if you pool is not big enough, you will have to wait for a released connection .
If you use Spring, you can use the LazyConnectionDataSourceProxy. It will only retrieve an actual connection when you actually want to execute a query.
See: http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/jdbc/datasource/LazyConnectionDataSourceProxy.html