Hibernate fetching optimizations – Batch prefetching

There are many optimizations for data fetching in Hibernate.

One of the not so common method used is batch prefetching.

By default Hibernate utilizes proxy objects as placeholders for associations and collections.

For collections Hibernate uses its own collection wrapper implementations which acts as smart collections but the entities loaded in the collections are by default also proxies.

These proxy objects just have their id set and they only load their all properties by send the query to database only on property access other than their id.

Let’s say we have a Department entity and Employee entity with one to many relations. Each Department has many employees and for sake of simplicity each employee belongs to only one department

We now load all departments like this

List allDeps= session.createQuery("from Department").list();

Imagine if we have 10 departments, then this will result in a list of size 10. Further assume that in each of the department we have 10 employees each and we need to access the salary of each employee in this list.

for (Department dep : allDeps){

printSalaryForEachEmployees(dep);

}

printSalaryForEachEmployees(Department dep){
for (Employee emp : dep.getEmployees()){
System.out.println(emp.getSalary);
}

The print method will iterate through each of the employee in each department and call the getSalary() method. This call will initialize the Employee proxy and each call will send a SQL select to database.

All in all, this one use-case will send 1+ (10 x 10) =101 selects to database which is horrible number.

This is also worst case for infamous problem of n+1 selects. In our use case it becomes  (n x n+1).

There are many ways to optimize this.

Today we are looking into one of the methods given by Hibernate called batch prefetching.

It works like this.

Hibernate can prefetch the employee by initializing its proxies beforehand. This is how it is mentioned in the configuration.

...

This tells Hibernate that if it is using proxies for Employees( which by default Hibernate does), then on the the first initialization of single proxy, automatically initialize upto 10 proxies even before their property access. If there are more than 10 proxies then on the access of 11^th proxy preload another 10 proxies until the there are no proxies left.

Understandably this kind of optimization is referred to as blind-guess optimization by Hibernate as you don’t know beforehand how many proxies are there.

This optimization can also be applied for collections:

...

So for our use case since we have 10 departments in the list, the moment we initialize collection of employees for one of the department object it now initializes 10 more employees’ collections of 10 Departments, all by using a single select something this:

select e.* from Employee e 
where e.DEPARTMENT_ID  in (?, ?, ?,?,?,?,?,?,?,?)

RESTful url - few tips

Many beginners associate REST url’s with url having some identifier to identify the resource even if the identifier is present in the url parameter.

For e.g.,

http://www.myshop.com/shopping/products?productid=1

Here the url seems to be identifying a resource (product) with an identifier (productid) 1

However technically this is not a RESTful url.

REST ways of looking at resources says that the web is a web of information resources on which you can take standard actions.  HTTP is a good example of REST with the resources identified by url’s and the standard actions on those resources are GET, POST, PUT, and DELETE. The most predominant operations done in HTTP is to get the resource representation using GET and make updates to it using POST.

A resource oriented RESTful  url must serve as an unique identifier without the additional parameters, meaning the url itself should have the identifier in itself, something like this

http://www.myshop.com/shopping/products/1

Another important distinction is that the url must not include any verb rather only nouns because the verbs (actions that could be performed) are already defined as standard actions. For e.g., have a look at this url.

http://www.myshop.com/shopping/getproducts/1

Some frameworks might make use of this kind of url to map a method “getproducts” in the controller and pass the identifier 1. However this cannot be taken as a true resource identifier as it includes a part “getproducts” which has nothing to do to identify a resource. So in essence a RESTful url should be a resource oriented url which uniquely identifies a resource.

Spring MVC Framework - SessionAttributes annotation

Spring MVC is a highly configurable web framework. It can be configured using xml or now more popular annotations. Spring MVC is very powerful , however the number of different ways in which a single thing can be done can be sometimes overwhelming.  I have found it extremely useful to have Spring framework source code in my IDE and go through it whenever I need to know the internals. Of course technical blogs and discussions on StackOverFlow site has always been very helpful. Today I am going to discuss about an important annotation called SessionAttributes.

This annotation is declared at type level on the Controller class.

You specify SessionAttributes annotation like this:

@Controller

@RequestMapping("/editPet.do")

@SessionAttributes("pet")

public class EditPetForm {

    // ...

}

As you can see that you mention just the attribute name. This is a signal to the framework that whatever object is stored against the attribute named “pet” in the Model during the processing of the handler, it will be available as long as the methods are being called in the same controller.

Now the question is who creates / stores the object corresponding to this attribute.

You may remember that a controller may contain a method annotated with @ModelAttribute which means that the method will be called before any handler method is called. Also that method stores the return value in the Model against the model attribute name specified by @ModelAttribute.

Also a handler method may contain an argument annotated by @ModelAttribute which means that the framework will inject that attribute from Model if present or create new. So when you add a @ModelAttribute("pet") controller method argument you're telling Spring MVC to look for it in the model or create a new instance if not found. But  if you also add @SessionAttributes, at controller class level framewotk will not attempt to create a new instance and will expect the object to be either in the model or in the session.

So now armed with this knowledge let’s recreate the steps

1.  Our controller class is annotated with @SessionAttribute name “pet”

2.   A method in controller class is annotated with @ModelAttribute named “pet” and it returns a Pet object usually from  database. We do not want to load it again when next time this method is called.

3.   Because of point 1, the model attribute “pet” is available not only in the view but it is also stored session so that when again the handler method annotated with @ModelAttribute(“pet”) is called, it is injected by framework

4.   Also we would like such conversational attributes to be removed from session when the conversation ends. To accomplish this we can declare an argument of type SessionStatus in the handler method and framework will inject it into the handler method. We can then call setComplete() method on SessionStatus to indicate that the attributes need to be cleared.

So then what is the difference between storing an attribute in HttpSession and SessionAttribute.

An object stored in http session will be there till the life time of http session. Now imagine that in the above mapping example for /editPet.do, the “pet ” attribute is a Pet object loaded from database through a method annotated with @ModelAttribute  in the controller class. We probably need the Pet object for the time as long as we are doing some processing logic for the editing. All such methods doing this processing logic should ideally be in the same controller.

So @SessionAttribute along with SessionStatus provides a way to make the “pet” attribute available while the methods of ONLY this controller are being called. That also means that session attributes using @SessionAttributes are not available across controllers while attributes stored in HttpSession are always available. So in nutshell we can say that attributes specified by @SessionAttributes are meant for storing conversation related objects into http session for conversation time period.

Hibernate automatic versioning - Part 1

Hibernate can provide automatic versioning for the entities without you doing ANYTHING. By WITHOUT ANYTHING I really mean WITHOUT  ANYTHING ! You just have your entity classes and you wnat hibernate to provide versioning so that an update on a already changes object would not happen. This sounds exciting but in actual practise its not that very used. We'll see how and why.

If you simply want hibernate to provide automatic versioning (without addition of version column or timestamp in the correspoding table and entity) then hibernate can do this but the caveat is that object should be retrieved and modified in the same session or persistence context. This essentially means that the session must not be closed (it can be flushed though to send the changes to database).

When using this feature, Hibernate checks the current database state of the entity in question and if the earlier state ( when it was retrieved or when the last flush operation was called upon session) is different from the current database state, it concludes that the entity has been modified and throws StaleObjectException.

To enable this feature on an entity you will have to add attribute optimistic-lock="all" in the entity mapping if you are using xml based configuration or Hibernate specific annotation like below if you are using annotation approach

@Entity
@org.hibernate.annotations.Entity(
       .................
        optimisticLock = OptimisticLockType.ALL,
       ........................)

public class TestEntity{ ... }

Hibernate team does not recommend using this automatic versioning without version or timestamp column as this is bit slow and complex. Also another bigger disadvantage is that it does not work for detached objects. It works only as long as the session is not closed. For this reason too, this is not effective in long running conversations as well.

 In the next part I will explain about other better way of adding optimistic version control to your entities which uses version or timestamp column to keep track of the entity state.

Resource loading in JSF 2 - Part 1

UIComponents are central to the functioning and usability of JSF. What you see on the user agent ( for e.g, web browsers) is the markup generated by UIComponent classes and whatever is submitted through browsers in web  request is decoded by the UIComponent instances for that particular page. However for any component having slight complexity, how the markup looks like and behaves is dependent on other resources like images, javascript and stylesheet etc. So if a component author provides a component, the framework should have a mechanism of including these resources (preferably within the jar bundling the component) so that components can look and behave as desired. This way components can be self contained.

Prior to JSF 2 such a mechanism was missing. JSF 2 provides this important missing piece to enable a component to load resources uniformly from class path resources.

Typically in a web application, how static resource requests are served depends on the mapping in web.xml. Most commonly the web.xml does not contain any specific mappings for extensions of static resources like images, scripts ( for e.g., *.jpg,*.css etc). In that case a Servlet provided by container usually serves up the request. For e.g., in Tomcat such requests are served up by

How JSF recognizes that a request is for a resource depends upon the existence of string "/javax.faces.resource" in the request url. If this string is present then the JSF main Servlet "FacesServlet" considers this as a resource request and forwards it to default ResourceHandler.

Hibernate Collections implementations

So your Hibernate entity has a reference to collection of other entities mapped using Set interface (or for that matter any other collection interface). You load an entity from database and in your debugger inspect the type of the collection Hibernate is using ( assuming it is configured as eagerly loaded. Lazy loading is quite another story for another day). You expect it to be usual java implementation HashSet but you see something as PersistentSet. You are bewildered and scratching your head jump to the Hibernate manuals or different forums. Go no further ! Here is what is happening under the hood !

Whenever hibernate is used for querying an object, an entity is returned which is managed by Hibernate as long as the session in which it was queried is open. Behind the scenes Hibernate queries the database and builds the entity object. The entity might contain associations in form of Single object association or Collections. In case of collections, hibernate creates its own implementation of collection. For e.g., for a List type it creates a wrapper list Persistent List. All the java collections like Set, HashMap etc. have their corresponding implementaions in Hibernate.

The base class for all these hibernate implementations is AbstractPersistentCollection as you can see below in the type hierarchy.

You will never have to directly deal with these Hibernate classes however it's good to understand the inner working of collection mapping.

All these implementations are wrapper over the original collection in the entity (decorator pattern you see !). There are many  things for which Hibernate uses these implementations classes. One of the things Hibernate uses these implementations is to mark the collections as dirty. Whenever you add/remove an object in these collections, it marks the collection dirty and Hibernate can queue the update query later when it inspects the collection. AbstractPersistentCollection maintains a Boolean variable “dirty” to indicate this and it is updated whenever there is any change to the collection via its interface.

My experiences with Hibernate - Part 1

Hibernate is arguably the most popular ORM tool which is not only widely used and well documented but also motivator for other ORM standards like JPA. There is one side of camp which hates ORM in general (championed by Ted Neward in early 2006 and propagated by his article "The Vietnam of Computer Science"). I read that article long way back in 2007 and thought in horror"My God ! Why should I bother about ORM in general. I am happy with my JDBC and cluttering my code with mapping code". But as things turned out, ORM tools slowly gained ground and now a days majority of Enterprise applications use some sort of ORM to speed up the development work. So I begin to have a taste of Hibernate during work and personal projects. Hibernate does a great job when you begin to appreciate the impedance mismatch betweeen object model and relational model it's trying to solve ( though that's impossible to completely solve. To know why read the article mentioned above). Also its well documented and an exceptionally well written book "Hibernate in Action" (now in second edition with title "Java Persistence with Hibernate"). The mailing list is helpful and there are lots of sample examples.
However while advocating Hibernate to anyone I had a feeling inside and I could not give word to that  until one day when I stumbled upon a phrase by ever popular Joel. He mentioned about "Leaky Abstractions" and when I read that I was convinced that is what ORM and in general Hibernate is. In summary what "Leaky Abstraction" refers to is an abstraction which promised to act like an abstraction to its users but behind the scenes requires the users to understand about its details ( which is violation of  the principle of abstraction).
I am not taking away anything from Hibernate and maintain that it a well written tool full of features. But just a word of caution. If you think you just need to know what to use in Hibernate for use case, you will shoot yourself in your foot. You have to keep an eye on what hibernate is doing behind the scene and how to use it myriads of configuration settings. If you do not you will loose several hours in debugging it. I would suggest if you foresee yourself using Hibernate or JPA for a considerable time in your projects, then spend some time in soaking in some basic and advanced concepts from the book "Java persistence with Hibernate". You won't regret this.

I coming weeks I would be listing some of the concepts in Hibernate which are bit confusing and at times misleading. These posts would not be necessarily long ones and some may even be bullet points but they will be helpful to understand more about the inner workings of Hibernate.