SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye

Welcome, Guest
Guest Settings
Help

Compass Support Forums » Compass » Compass Users

Thread: TerracottaProcessor: "Should not happen really..." yet it happens

This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 17 - Pages: 2 [ 1 2 | Next ] - Last Post: Aug 23, 2011 9:26 PM Last Post By: louisvuittonbox
Amir Gur

Posts: 10
Registered: 2/2/09
TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 2, 2009 8:24 AM
 
  Click to reply to this thread Reply
Hello Shay, or anybody else for that matter.

I try to use a stack of compass over terracotta, and let spring manage the transactions.
The code is pleasant and it works most of the time, yet sometimes I get this error:

Feb 2, 2009 2:31:39 PM org.compass.needle.terracotta.transaction.processor.TerracottaTransactionProcessorFactory$TerracottaProcessor run
SEVERE: Another instance is running on the sub index, make sure it does not. Should not happen really...

In some other times, I get this exception:
Feb 2, 2009 3:52:27 PM org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$ScheduledRefreshCacheRunnable$1 doInTransaction
SEVERE: Failed to perform background refresh of cache for for sub-index page
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(Unknown Source)
at java.util.HashMap$EntryIterator.next(Unknown Source)
at java.util.HashMap$EntryIterator.next(Unknown Source)
at java.util.HashMap$AbstractManagedEntriesIterator.nextEntry(HashMap/java:744)
at java.util.HashMap$AbstractManagedEntriesIterator.next(HashMap/java:738)
at java.util.HashMap$AbstractManagedEntriesIterator.nextEntry(HashMap/java:744)
at java.util.HashMap$AbstractManagedEntriesIterator.next(HashMap/java:738)
at java.util.HashMap$KeysIterator.next(HashMap/java:788)
at java.util.AbstractCollection.toArray(Unknown Source)
at org.compass.needle.terracotta.TerracottaDirectory.list(TerracottaDirectory.java:137)
at org.apache.lucene.index.SegmentInfos.getCurrentSegmentGeneration(SegmentInfos.java:127)
at org.apache.lucene.index.IndexReader.indexExists(IndexReader.java:613)
at org.compass.core.lucene.engine.store.DefaultLuceneSearchEngineStore.indexExists(DefaultLuceneSearchEngineStore.java:478)
at org.compass.core.lucene.engine.store.DefaultLuceneSearchEngineStore.indexExists(DefaultLuceneSearchEngineStore.java:467)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$ScheduledRefreshCacheRunnable$1.doInTransaction(DefaultLuceneSearchEngineIndexManager.java:806)
at org.compass.core.impl.DefaultCompass$CompassTransactionContext.execute(DefaultCompass.java:389)
at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$ScheduledRefreshCacheRunnable.run(DefaultLuceneSearchEngineIndexManager.java:802)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask/java:292)
at java.util.concurrent.FutureTask.runAndReset(FutureTask/java:58)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I created a minimal eclipse project that demonstrates the problem.
If anyone is interested, all you have to do is unzip the file I attached, import it to eclipse, make sure you have the terracotta plugin installed, add the following jar files:
• compass-2.2.0-M2.jar
• spring2.5.6.jar
• lucene.2.4.0.jar
• h2.jar
(I removed them because they were too large)
and run the bug.Main class.

The different outputs are not deterministic.

Thanks in advance,

Amir.

Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 2, 2009 12:30 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
Hello all interested readers.

Following my post I noticed two things.

1. My problem happens also when I use FSDirectory, not just terracotta.
2. Changing from dbcp dataSource to c3p0 dataSource doesn't help.

Still thanking in advance,

Amir.
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 2, 2009 1:39 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
I will have a look at it, can you try and disable the managed terracotta: compass.engine.store.tc.managed=true?

Also, why do you use the datasource? Compass does not need one, and if you don't access a database, I am not sure why you need it.

Last, when you say it happens with file based index, do you mean you get the same exceptions?
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 3, 2009 2:21 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Hello Shay,
Thanks for the reply.
I'll try to answer.

• About disabling managed terracotta,

I tried to add the following line to my applicationContext-compass.xml file:

<prop key="compass.engine.store.tc.managed">true</prop>

What I got is:

Feb 3, 2009 9:55:22 AM org.compass.needle.terracotta.transaction.processor.TerracottaTransactionProcessorFactory$TerracottaProcessor run
SEVERE: Failed to process jobs [UPDATEPage#http://example.com/164#],
java.lang.NullPointerException
at org.compass.core.lucene.engine.transaction.support.ResourceEnhancer.applyBoostIfNeeded(ResourceEnhancer.java:87)
at org.compass.core.lucene.engine.transaction.support.ResourceEnhancer.enahanceResource(ResourceEnhancer.java:78)
at org.compass.core.lucene.engine.transaction.support.WriterHelper.processUpdate(WriterHelper.java:47)
at org.compass.core.lucene.engine.transaction.support.WriterHelper.processJob(WriterHelper.java:68)
at org.compass.needle.terracotta.transaction.processor.TerracottaTransactionProcessorFactory$TerracottaProcessor.run(TerracottaTransactionProcessorFactory.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask/java:268)
at java.util.concurrent.FutureTask.run(FutureTask/java:54)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Then I tried <prop key="compass.engine.store.tc.managed">false</prop>

And there was no Exception.

For a moment it seemed to work.
But when I ran it again against the same terracotta server I got the same NullPointerException.

• About the data source:

Compass doesn't need it data source, but it needs transaction.
As far as I know, the easiest way to get and manage transactions is through Spring.
All of Spring's transaction manager require data source, so what can I do?
Is there any better option?
I don't want to encapsulate my business logic in a CompassTemplate + CompassCallback.
I'd rather add @Transactional annotation.

• As for what I wrote about the problem happens also in FSDirectory

I'm sorry, I was probably wrong.
(I disabled eclipse automatic build so the change had no effect)

Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 3, 2009 9:47 AM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
I will test the terracotta processor and get back to you.

Regarding the data source, I think that using CompassTemplate is much simpler than the need to use a database to just do transaction management in Compass, no? It will be slower as well, Compass local transactions are much faster.
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 3, 2009 10:32 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Thanks.

...and as for the data source, I use in-memory h2 database and I believe it has minimal effect on the performance.
I hope I'll find time to benchmark it.
And about the simplicity: I guess it's a matter of taste, but even Spring doesn't recommend using templates any more,
See here:.
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 3, 2009 1:19 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
I have just kicked a nightly build of trunk which I hope will help (which version of Compass are you using)? To get the nightly build, see here: http://wiki.compass-project.org/display/CMPKB/Nightly+Builds.

Regarding the database part, the Hibernate template and the Compass template part is completely different. I don't really get your reasoning, in order to use Spring transaction management, where you only need Compass local transaction management (it has been simplified as well, check out my blog), you want to introduce a database to your application?
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 4, 2009 9:51 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Hello Shay.

Today (4-2-2009) I used the latest version from the SVN (revision 3717).

It declares itself as version 2.2.0.RC1.
The problem didn't go away, though it took more time to reproduce it.

Again, it checked with <prop key="compass.engine.store.tc.managed">true</prop>.
When I changed to <prop key="compass.engine.store.tc.managed">false</prop> the application crashed almost immediately.

Anyway, I did a little benchmark.
Here it is:

Edited by: Amir Gur on Feb 4, 2009 10:03 AM
Artyom Sokolov

Posts: 9
Registered: 1/21/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 4, 2009 2:31 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
Amir,

Hi. Is it okay that you use next line in tc-config.xml:

<module group-id="org.compass-project" name="compass" version="2.2.0.M1" />

while actually using compass 2.2.0-M2 in java application's classpath?

Thanks.
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 4, 2009 5:06 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
Which problem happened now? You specified several of them. Also, what crashed when you used managed set to true?
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 5, 2009 12:45 AM   in response to: Artyom Sokolov in response to: Artyom Sokolov
 
  Click to reply to this thread Reply
No.
It's not O.K.
When you upgrade compass you need to change:
1) your classpath
2) tc-config.xml
3) the jar in the terracotta/modules directory.
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 5, 2009 6:40 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
I'll try to be as specific as possible.

Again, I took the trunk's head. (revision 3723)

I ran this simple test.
All you have to do is unzip it and import it to eclipse.

the first time I ran it (against a newly created index), no document made it into the index.
the second time I ran it, I got this:

Exception in thread "main" org.compass.core.engine.SearchEngineException: Failed to obtain order lock on sub index page; nested exception is org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: TerracottaLock: order.lock
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: TerracottaLock: order.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:85)
at org.compass.needle.terracotta.ManagedTerracottaLockAdapter.obtain(ManagedTerracottaLockAdapter.java:55)
at org.compass.core.lucene.engine.transaction.support.AbstractJobBasedTransactionProcessor.obtainOrderLockIfNeeded(AbstractJobBasedTransactionProcessor.java:168)
at org.compass.core.lucene.engine.transaction.support.AbstractJobBasedTransactionProcessor.update(AbstractJobBasedTransactionProcessor.java:123)
at org.compass.core.lucene.engine.LuceneSearchEngine.createOrUpdate(LuceneSearchEngine.java:274)
at org.compass.core.lucene.engine.LuceneSearchEngine.save(LuceneSearchEngine.java:252)
at org.compass.core.impl.DefaultCompassSession.save(DefaultCompassSession.java:484)
at org.compass.core.impl.DefaultCompassSession.save(DefaultCompassSession.java:468)
at bug.PageSpringDao.save(PageSpringDao.java:12)
at bug.PageSpringDao$$FastClassByCGLIB$$3eb76e5c.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:635)
at bug.PageSpringDao$$EnhancerByCGLIB$$864836b9.save(<generated>)
at bug.Main.main(Main.java:21)

Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 5, 2009 9:17 AM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
I got the code, but have not run it yet. I have a question though from looking at the code, I see that you are not storing the index in terracotta, and you are not using the terracotta transaction processor, is that right? You basically do plain Compass API with local transaction manager?
Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 5, 2009 9:48 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
No.
The zip file I uploaded contains 2 spring configuration files:

applicationContext-spring-transactions.xml that stores the index in terracotta (<prop key="compass.engine.connection"> tc://sample </prop>) and uses TerracottaTransactionProcessorFactory ,

and applicationContext-compass.xml that stores the index in file system (<prop key="compass.engine.connection"> index </prop>) and doesn't use TerracottaTransactionProcessorFactory.

The main application I sent uses the first one.

Amir Gur

Posts: 10
Registered: 2/2/09
Re: TerracottaProcessor: "Should not happen really..." yet it happens
Posted: Feb 14, 2009 12:20 PM   in response to: Amir Gur in response to: Amir Gur
 
  Click to reply to this thread Reply
Hello Shay.

I'm just checking.
Did you run my bug demonstration?
Thanks,

Amir.
Legend
Guru: 2001 + pts
Expert: 501 - 2000 pts
Super-star: 101 - 500 pts
Assistant: 51 - 100 pts
Participant: 0 - 50 pts
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums