SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye

Welcome, Guest
Guest Settings
Help

Compass Support Forums » Compass » Compass Developers

Thread: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship

This question is answered.


Permlink Replies: 43 - Pages: 3 [ 1 2 3 | Next ] - Last Post: Dec 3, 2011 4:23 AM Last Post By: longchamp
jsun

Posts: 3
Registered: 10/28/08
CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Oct 28, 2008 11:29 PM
 
  Click to reply to this thread Reply
I'm building an application with Hibernate 3.3.1, Lucene 2.4.0, and Compass 2.1.0-RC. Here is a simple example:

@Entity
@Searchable
public class A {
@OneToMany(mappedBy="a")
@SearchableReference
private Set<B> b;
}

@Entity
@Searchable
public Class B {
//@todo must change to fetch=FetchType.EAGER for CompassGps.index() to work
@ManyToOne(optional=false, fetch=FetchType.LAZY)
@JoinColumn(nullable=false, updatable=false)
@SearchableReference
private A a;
}

everything works great, search indices mirrored properly, search returned correct results etc.. then I decided to rebuild the entire index, here is what i'm doing:

//Spring annotation
@Autowired
public void setCompassGps(CompassGps compassGps) {
this.compassGps = compassGps;
}
...
..
.
this.compassGps.index();

and got the following exception:



Caused by: java.util.concurrent.ExecutionException: org.compass.gps.device.hibernate.HibernateGpsDeviceException: {hibernateDevice}: Failed to index the database; nested exception is org.compass.core.converter.ConversionException: Trying to marshall a null id id for alias [A]
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.compass.gps.device.support.parallel.ConcurrentParallelIndexExecutor.performIndex(ConcurrentParallelIndexExecutor.java:123)
... 64 more
Caused by: org.compass.gps.device.hibernate.HibernateGpsDeviceException: {hibernateDevice}: Failed to index the database; nested exception is org.compass.core.converter.ConversionException: Trying to marshall a null id id for alias [A]
at org.compass.gps.device.hibernate.indexer.ScrollableHibernateIndexEntitiesIndexer.performIndex(ScrollableHibernateIndexEntitiesIndexer.java:172)
at org.compass.gps.device.support.parallel.ConcurrentParallelIndexExecutor$1$1.doInCompassWithoutResult(ConcurrentParallelIndexExecutor.java:105)
at org.compass.core.CompassCallbackWithoutResult.doInCompass(CompassCallbackWithoutResult.java:29)
at org.compass.core.CompassTemplate.execute(CompassTemplate.java:130)
at org.compass.gps.impl.SingleCompassGps.executeForIndex(SingleCompassGps.java:162)
at org.compass.gps.device.support.parallel.ConcurrentParallelIndexExecutor$1.call(ConcurrentParallelIndexExecutor.java:103)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
... 1 more
Caused by: org.compass.core.converter.ConversionException: Trying to marshall a null id id for alias [A]
at org.compass.core.converter.mapping.osem.ClassMappingConverter.convertId(ClassMappingConverter.java:416)
at org.compass.core.converter.mapping.osem.ClassMappingConverter.marshallIds(ClassMappingConverter.java:388)
at org.compass.core.marshall.DefaultMarshallingStrategy.marshallIds(DefaultMarshallingStrategy.java:87)
at org.compass.core.converter.mapping.osem.ReferenceMappingConverter.doMarshall(ReferenceMappingConverter.java:76)
at org.compass.core.converter.mapping.osem.AbstractRefAliasMappingConverter.marshall(AbstractRefAliasMappingConverter.java:48)
at org.compass.core.converter.mapping.osem.ClassMappingConverter.doMarshall(ClassMappingConverter.java:202)
at org.compass.core.converter.mapping.osem.ClassMappingConverter.marshall(ClassMappingConverter.java:88)
at org.compass.core.marshall.DefaultMarshallingStrategy.marshall(DefaultMarshallingStrategy.java:154)
at org.compass.core.impl.DefaultCompassSession.create(DefaultCompassSession.java:361)
at org.compass.core.impl.DefaultCompassSession.create(DefaultCompassSession.java:350)
at org.compass.gps.device.hibernate.indexer.ScrollableHibernateIndexEntitiesIndexer$RowBuffer.flush(ScrollableHibernateIndexEntitiesIndexer.java:212)
at org.compass.gps.device.hibernate.indexer.ScrollableHibernateIndexEntitiesIndexer$RowBuffer.close(ScrollableHibernateIndexEntitiesIndexer.java:206)
at org.compass.gps.device.hibernate.indexer.ScrollableHibernateIndexEntitiesIndexer.performIndex(ScrollableHibernateIndexEntitiesIndexer.java:151)
... 10 more

everything works If I change the fetch type to EAGER in class B.. of course I'd like to use LAZY fetch for performance reasons..

Is this a limitation/bug in Compass? or am I doing something wrong here?

Thanks,

//js
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Oct 30, 2008 6:21 AM   in response to: jsun in response to: jsun
Correct
  Click to reply to this thread Reply
Can you try and put the annotations on a getter instead of a field? Maybe Hibernate fails to load the data since it can't intercept the method call to lazily load it.
jsun

Posts: 3
Registered: 10/28/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Oct 30, 2008 3:16 PM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Thanks Shay.

You are right on. I moved the annotations to the getter, now I'm able to rebuild the index successfully. however, I'm getting the following hibernate exception when fetching entity B:

org.hibernate.QueryException: not an association: a at org.hibernate.loader.criteria.CriteriaQueryTranslator.getPathEntityName(CriteriaQueryTranslator.java:239) at org.hibernate.loader.criteria.CriteriaQueryTranslator.createCriteriaEntityNameMap(CriteriaQueryTranslator.java:214) at org.hibernate.loader.criteria.CriteriaQueryTranslator.<init>(CriteriaQueryTranslator.java:104) at org.hibernate.loader.criteria.CriteriaLoader.<init>(CriteriaLoader.java:82) at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1577) at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:306) at org.hibernate.impl.CriteriaImpl$Subcriteria.list(CriteriaImpl.java:481)

I'll look through the hibernate/jpa annotation reference docs again, thanks for your help.

//js

p.s. upon inspection of the generated search index using Luke, entity B no longer has reference back to A.. I suppose this is related to the error above: hibernate lost the ManyToOne association from entity B to A (even though ManyToOne annotation is specified for the getter)

Edited by: jsun on Oct 30, 2008 3:30 PM
jsun

Posts: 3
Registered: 10/28/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Oct 30, 2008 5:29 PM   in response to: jsun in response to: jsun
 
  Click to reply to this thread Reply
it turns out that I cannot mix field annotations with getter annotations.. it works after moved all jpa/compass/validation/versioning annotations to the getters.. a lot work and introduced many other issues in my project but that's irrelevant for this discussion. will workout the details with Hibernate community.

Thanks,

//js
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Oct 31, 2008 7:28 PM   in response to: jsun in response to: jsun
 
  Click to reply to this thread Reply
I don't know, but maybe, with byte code enhancement, Hibernate can intercept also field access, though I doubt that... . This always struck me as strange with Hibernate, that you can annotate fields, but they won't be lazily loaded. It has to be possible to do that. In any case, I hope that you will manage to solve your issues, good luck!.
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 22, 2008 7:15 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
I have very similar problem as Jsun. I have two entities:

@Entity
@Searchable
public class A {

@Id
@GeneratedValue(generator = "table-hilo", strategy = GenerationType.TABLE)
@Column(name = "a_id")
@SearchableId
private Long id = new Long(-1);

@OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
@JoinColumn(name="b_id", nullable = false)
@SearchableComponent
private B b;
}

@Entity
@Searchable(root = false)
public class B {

@Id
@GeneratedValue(generator = "table-hilo", strategy = GenerationType.TABLE)
@Column(name = "b_id")
private Long id = new Long(-1);

@Column(nullable = false, length = 150)
@SearchableProperty(name = "b_name")
private String name;
}

As in Jsun's case everything works fine... BUT when i want to reindex database the A.b property doesn't get indexed (there is no exception as in Jsun's case) I tried to move annotations from fields to getters but that also didn't work (result was the same as when I was using field access). The only way to get it work is to make the OneToOne relation EAGER instead of LAZY, but for performance reasons that is unwanted.

Is this bug in compass or hibernate or am I doing something wrong here ?
Here is my setup:
Hibernate 3.3.1.GA
Compass 2.1.0
Hibernate bytecode provider: javassist 3.9.0.GA
Entity indexer: org.compass.gps.device.hibernate.indexer.PaginationHibernateIndexEntitiesIndexer

Martin
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 23, 2008 3:29 PM   in response to: Martin Kovacik in response to: Martin Kovacik
 
  Click to reply to this thread Reply
Can you try and set the compass.reflection.type setting in Compass to plain? In this case it will use reflection and not ASM to get the value out of the getter/field? Can you try when the annotations are on the getter?
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 24, 2008 7:21 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
I tried both cases (annotations on fields and getters) using compass.reflection.type set to plain and neither worked. The hibernate OneToOne lazy association was not indexed.

Martin
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 25, 2008 2:53 AM   in response to: Martin Kovacik in response to: Martin Kovacik
 
  Click to reply to this thread Reply
Then it is strange, what Compass will do is just call the getter / field using reflection (or ASM byte code enhancement). I would expect Hibernate to fetch the relationship when the getter is called. Maybe you can debug it into Hibernate?
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 27, 2008 9:41 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
I figured out what was (and still is) the problem. I have two entities:

@Entity
@Searchable
public class A {

@Id
@GeneratedValue(generator = "table-hilo", strategy = GenerationType.TABLE)
@Column(name = "a_id")
@SearchableId
private Long id = new Long(-1);

@OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
@JoinColumn(name="b_id", nullable = false)
@SearchableComponent
private B b;
}

@Entity
@Searchable(root = false)
public class B {

@Id
@GeneratedValue(generator = "table-hilo", strategy = GenerationType.TABLE)
@Column(name = "b_id")
private Long id = new Long(-1);

@Column(nullable = false, length = 150)
@SearchableProperty(name = "b_name")
private String name;
}

The OneToOne LAZY association is not indexed when using hibernate field annotations. To be more specific, compass correctly processes the A.b association, but then compass tries to marshall B.name property which is NULL and therefore it doesn't index it. The B.name property is NULL because the A.b hibernate proxy ( A.b is LAZY) doesn't get initialized when compass gets value for NAME property (Compass is using field access in this case). When I switched from field annotations to getter annotations in B class, indexing started to work correctly. I suspect the problem is in javassist bytecode provider in hibernate 3.3.1 which doesn't initialize proxy using field access. I would like to try cglib instead of javassist but it seems that cglib support in hibernate 3.3.1 is broken (http://opensource.atlassian.com/projects/hibernate/browse/HHH-3504).

There are few workarounds to this problem:
1. Use cglib instead of javassist (but obviously not in 3.3.1) - I haven't tried this
2. Use getter annotations instead of field annotations (which is not what I want)
3. Explicitly inilialize hibernate proxies in HibernateIndexEntitiesIndexer implementations using org.hibernate.Hibernate.initialize(Object proxy) method - I haven't tried this also

PS: In my previous post to this thread I wrote that I tried getter annotations and it didn't work. Well, I was wrong because I only used getter annotations in A class. In B class I was using field annotations, and therefore A.b proxy was not initialized.

Martin
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 29, 2008 4:56 PM   in response to: Martin Kovacik in response to: Martin Kovacik
 
  Click to reply to this thread Reply
First of all, thanks for the effort in investigating this, I am sure many Compass users would find this information very useful. Regarding the field access, yea, that is what I suspected might happen.

I would love to manage to solve this in Compass and not depend on Hibernate. It would be great if you can test if explicit initialization using Hibernate API within the Hibernate Indexer would help. I am not sure about it, since the initialize would be called on A, and I am not sure if Hibernate would initialize B properly in this case. But, if it does, I can easily add this to Compass.
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 29, 2008 7:18 PM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
I tried to use Hibernate API to initialize Hibernate proxies within the Hibernate Indexer. You were right, the initialize method just initializes the proxy itself, not its references and/or collections.

I will somehow manage to switch hibernate bytecode provider to cglib to see whether to blame javassist or not.

Anyway, for now it looks like I have only one option (using EAGER associations is not a solution): use getter annotations, which I really don't want.

Martin
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 30, 2008 12:02 PM   in response to: Martin Kovacik in response to: Martin Kovacik
 
  Click to reply to this thread Reply
I tried cglib instead of javassist and it didnt't work either. It's just not possible (at least using cglib or javassist) to "intercept" direct field access in proxy object and therefore the proxy is not initialized. Because of this it is not possible to use compass with field annotations.

Solution to this would be some mechanism to explicitly initialize proxy object which is being marshalled. For example there could be some Initializer interface with method initialize(). To use compass field annotations together with hibernate, it would be mandatory to supply HibernateInitializer (which implements Initializer interface). HibernateInitializer would be pretty simple just calling Hibernate.initialize(proxy) method.

I know ... it is not exactly pretty.... :(

Martin
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Dec 31, 2008 2:06 PM   in response to: Martin Kovacik in response to: Martin Kovacik
 
  Click to reply to this thread Reply
Actually, its not that ugly :). I already do that when the marhsalling process needs to get the class name of the object. I just added special code that will also initalize an object or a collection if it is a Hibernate proxy ( CMP-805). I have kicked a 2.1 nightly build, you can test it once it is done and tell me if it solves the problem.
Martin Kovacik

Posts: 16
Registered: 12/3/08
Re: CompassGps.index() fails with Hibernate ManyToOne LAZY fetch relationship
Posted: Jan 1, 2009 7:47 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Attachment compass.diff (3.4 KB)
It did not work. The initialization of proxy was successful, but fields of proxy object were still null (even after explicit initialization), therefore were not indexed.

I further investigated this problem and I found this thread: http://forum.hibernate.org/viewtopic.php?t=974275

However I was able to make it work by changing
Hibernate.initialize(proxy)
to
obj = ((HibernateProxy)obj).getHibernateLazyInitializer().getImplementation()
in HibernateProxyExtractor class

The diff file with my changes is attached.

Martin

PS: coherence.jar is missing in compass trunk
Legend
Guru: 2001 + pts
Expert: 501 - 2000 pts
Super-star: 101 - 500 pts
Assistant: 51 - 100 pts
Participant: 0 - 50 pts
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums