SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye

Welcome, Guest
Guest Settings
Help

Compass Support Forums » Compass » Compass Users

Thread: indexed resource searcheable only after server restart

This question is not answered. Helpful answers available: 1. Correct answers available: 1.


Permlink Replies: 11 - Pages: 1 - Last Post: Nov 30, 2011 9:37 PM Last Post By: sweet123
pangea

Posts: 368
Registered: 9/25/07
indexed resource searcheable only after server restart
Posted: Dec 22, 2008 10:34 AM
 
  Click to reply to this thread Reply
I have modified my cfg to disable caching and now the newly created resources are only found in search after I restart my web application. I thought that if the caching is disabled then the index is searched from database everytime...below is my config

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:compass="http://www.opensymphony.com/compass/schema/spring-core-config"
    xsi:schemaLocation="
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.opensymphony.com/compass/schema/spring-core-config http://www.opensymphony.com/compass/schema/spring-compass-core-config.xsd">
    
    <!--
    Compass 2.0.2 does not supprt property place holders while using compass namespace handler. The support was added only from 2.1.x onwards.
    http://forum.compass-project.org/thread.jspa?threadID=215690&tstart=0
    In the next release 4.5 or 5.0, we need to migrate to 2.1.x of compass and then we can start using spring-compass-online-2.0.1.xml
    -->
    <bean id="compass" class="org.compass.spring.LocalCompassBean">
        <property name="resourceLocations">
            <list>
                <value>classpath:/index/persistableEntry.cpm.xml</value>
	     </list>
        </property>
        <property name="compassSettings">
            <props>
                <!-- Jdbc Directory Settings 
                <prop key="compass.engine.connection">jdbc://</prop>
                <prop key="compass.engine.store.jdbc.connection.provider.class">
			org.compass.core.lucene.engine.store.jdbc.ExternalDataSourceProvider
                </prop>
                <prop key="compass.engine.store.jdbc.managed">true</prop>
                
		<!-- 
		http://issues.compass-project.org/browse/CMP-485
		Compass opens <b>new thread</b> for performing certain tasks, such as
		deleting MarkDeleted rows. When used from a webapp (WebSphere), this is
		causing connection leaks due to the fact that WAS doesn't (from IBM PMR)
		allow to commit transactions from the new thread.
		
		The scheduler is disabled by setting the compass.engine.indexManagerScheduleInterval to -1. 
		However, the scheduled tasks still need to be executed. This can be performed by invoking 
		org.compass.core.engine.SearchEngineIndexManager#performScheduledTasks
		-->
		<prop key="compass.engine.indexManagerScheduleInterval">-1</prop>
		
	    <!-- Optimizer settings-->
        	<prop key="compass.engine.optimizer.schedule">false</prop>
                <prop key="compass.engine.optimizer.type">org.compass.core.lucene.engine.optimizer.NullOptimizer</prop>
                <prop key="compass.transaction.isolation">read_committed</prop>
                
                
                <!--
			This is set to false to reduce the time taken to save. When an insert/update changes the index, it automatically
			invalidates the internal cache, that way all the search results are always up to date. But it can have some
			negative impact on the index performance. http://issues.compass-project.org/browse/CMP-218
	 	-->
                <prop key="compass.transaction.clearCacheOnCommit">false</prop>
                          
                <!-- Use non compound index format when using JDBC Directory -->
                <prop key="compass.engine.useCompoundFile">false</prop>
 
                <!-- Customize locking mechanism to handle PhantomRead exceptions: interval is in millis and timeout is in seconds -->
                <prop key="compass.transaction.lockPollInterval">200</prop>
                <prop key="compass.transaction.lockTimeout">30</prop>
 
                <!-- turn it off -->
                <prop key="compass.engine.cacheIntervalInvalidation">-1</prop>
 
                
                <prop key="compass.transaction.factory">org.compass.core.transaction.LocalTransactionFactory</prop>
                
                
                <!-- Mark resources for deleting. Actual deletion will be done by a shceduled program -->
                <prop key="compass.engine.store.jdbc.fe.__default__.type">
                    org.apache.lucene.store.jdbc.handler.ActualDeleteFileEntryHandler
                </prop>
                
                <prop key="compass.engine.store.jdbc.fe.__default__.indexInput.bufferSize">200000</prop>
                <prop key="compass.engine.store.jdbc.fe.__default__.indexOutput.bufferSize">100000</prop>
                
                <!-- Cache the index in RAM. We can also use File System based cache 
                
                Since the cache update and invalidation is happening using a thread pool executor 
                (ScheduledThreadPoolExecutor) and since WebSphere doesnt propogate transactional context to
                a non-container managed thread etc etc...this has to be disabled to avoid connection 
                leaks while using JNDI based DataSource
                -->
                <!--prop key="compass.engine.localCache.__default__.connection">ram://</prop-->
            </props>
        </property>
        
        <property name="dataSource" ref="localTestDataRepo" />
        
    </bean>
    <bean id="compassIndexerDao" class="com.equifax.ic.tdp.framework.search.CompassDaoIndexer">
        <property name="compass">
            <ref local="compass" />
        </property>
    </bean>
 
</beans>
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: indexed resource searcheable only after server restart
Posted: Dec 25, 2008 2:23 AM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply
Do you mean cacheIntervalInvalidation? I understand that you are using 2.0?
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: indexed resource searcheable only after server restart
Posted: Dec 25, 2008 2:37 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
If you set the cache invalidation to -1, it never perform cache invalidation. If you set it to 0, it will always perform cache invalidation. Why do you want to always perform cache invalidation?
pangea

Posts: 368
Registered: 9/25/07
Re: indexed resource searcheable only after server restart
Posted: Dec 25, 2008 9:01 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Hi Shay - yes i am using compass 2.0.2. Below is my use-case..let me know the best way to handle this

User creates a new resource using web-form
App saves it in the index
User goes to the search page and enters search query....here is the problem....the newly saved resource is not found or returned as part of the results...it is found however after restarting the applications

I have to disable cache invalidation because this operation is being performed in async using threads and this is causing connection leaks (as i reported earlier)...u gave me a way to specify external WorkManager but i have few more questions on that...i have posted another thread regarding that
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: indexed resource searcheable only after server restart
Posted: Dec 29, 2008 4:43 PM   in response to: pangea in response to: pangea
Helpful
  Click to reply to this thread Reply
You set the clearCache flag to false, this means that when a transaction commits, it will not clear the cache so new searches will see the up to date changes the transaction performed. When you set the cacheInvalidation to -1, it means that no cache invalidation will happen, and when you set it to 0, it means that every search will check against the database to see if there is a new updated version of the index.

The comment put on the clearCache flag is a bit wrong. It won't slow down inserts, but will cause the first search that will happen again to refresh the cache.

I would suggest that you set the clearCache flag to true, since you want new search operations to see the results. And keep the index invalidation value to its default value (which is 5 seconds).

Also, I would suggest, if you can, move to Compass 2.1 (and Lucene 2.4). Refreshing of the cache is much faster in that version.
pangea

Posts: 368
Registered: 9/25/07
Re: indexed resource searcheable only after server restart
Posted: Mar 20, 2009 5:08 PM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Migrated to compass 2.2.0 RC1 and followed your suggestion on enabling clearCache flag and cacheInvalidation. It is faster now. Does any of these two operations use the commonj work manager I configured...i sometimes see the below in the logs...curious what task is being executed by the commonj wm here...

20 Mar 17:48:04:511 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Beginning ne
w Spring transaction, and a new compass transaction on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] (IID=, TID=)

20 Mar 17:48:04:511 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Beginning ne
w Spring transaction, and a new compass transaction on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] (IID=, TID=)

20 Mar 17:48:04:520 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Committing S
pring transaction controlled by compass on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] (IID=, TID=)

20 Mar 17:48:04:520 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Committing S
pring transaction controlled by compass on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] (IID=, TID=)

20 Mar 17:48:04:520 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Committing c
ompass transaction using Spring synchronization beforeCompletion on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10]
(IID=, TID=)

20 Mar 17:48:04:520 [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10] DEBUG transaction.SpringSyncTransaction - Committing c
ompass transaction using Spring synchronization beforeCompletion on thread [java.lang.ThreadGroupname=DefaultWorkManager: WM Service Group,maxpri=10]
(IID=, TID=)
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: indexed resource searcheable only after server restart
Posted: Mar 21, 2009 5:36 AM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply
The scheduled optimizer and async cache invalidation are using the work manager in your case. The other cases where the ExecutorManager is used (merging segments, concurrent transaction processors) are disabled because you store the index on the database.
louisvuittonbox

Posts: 411
Registered: 8/23/11
Re: indexed resource searcheable only after server restart
Posted: Aug 23, 2011 9:32 PM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply
Welcome to louis vuitton online! Here at our online shop, we enable you to select as many styles louis vuitton bags as you can at greatly discounted prices. Established in 2006, We have already landed so many returning customers from louis vuitton outlet. We have been known for the authentic quality of our items,such as louis vuitton belts and louis vuitton purses unsurpassed customer service, rapid delivery, unique prices and so on and so forth. What we did, are doing and will do is only to make our esteemed customers consume those world-renowned luxury designer merchandise with peace of mind. So we sincerely hope that you could be our next returning customer. We are confident that you will be quite satisfied with everything we offer here. Wish you a very good shopping louis vuitton store!
sweet123

Posts: 211
Registered: 10/31/11
Re: indexed resource searcheable only after server restart
Posted: Oct 31, 2011 3:29 AM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply
As one of the most famous brands Coach Factory Outlet and the representatives of fashion, Coach Outlet Store Online always goes in the top of fashion, never fall behind Coach Outlet Online Store. Look into the fashion world, you will find Coach Outlet Factory that there are so many newest products pushed out by Coach Handbags Outlet. Coach Store, the most support of Coach Factory Outlet Online once appeared had earned a lot of women's heart, even the famous actresses. All of these, they are our Prada Store, and I will cherish them in lifetime. Perhaps you don't know, in my heart, Air Yeezy Shoes are my sisters, in the daily life, I call Gucci Bags sisters, that is because Coach Oulet USA really are my sisters, when I have trouble, Coach Outlet Store Online always help me to deal with it, when I am wronged, Louis Vuitton Outlet Factory will more angry than me to condemn others, Yes, Louis Vuitton Outlet Store Online are my sisters, we are envy Coach Handbags.Because I have Nike Dunk Shoes, my dear sisters!
hd8781187

Posts: 50
Registered: 10/22/11
Re: indexed resource searcheable only after server restart
Posted: Oct 31, 2011 3:54 AM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply

Report from our jingruihd correspondent (reporter Zhang Hong) the United States Yamaxun website general reachs electronic edition at the gucci outlets paper edition that autumn publishs 122 books this year, yamaxun a tall canal weighs headquarters, of publisher closing down finally is hasten of general trends place, can come sooner or later. Recently, li Kaifu of famous IT person sends small gain to say paper book Gucci Backpack can disappear. Li Yan of China press general manager, president of press of social science literature thanks Shou Guang, dolphin press president expresses Yu Xiaoqun, paper book won't disappear.

There is media coverage before this, yamaxun Gucci Clutches begins and direct autograph made an appointment with the author, and preparation opens the data of partial tiring-room to them. The website Gucci Evening will publish Yamaxun the paper edition of 122 books and electronic edition, hopeful replaces traditional publisher. Li Kaifu of famous IT person publishs small gain to say subsequently, paper book can disappear " long-term for this Gucci Hobos is inevitable " .

Li Yan of China press general manager expresses, "The thing may be done not have so simple, for the author that has become famous to great majority, likelihood more be used to counting a publishing house, presswork in Gucci Shoulder Bags order to make sure the publication of its work is mixed quality. On traditional sense, the editor is pleasant is done for them marry the garment, this kind of function won't disappear. Want to make an author complete accept an Gucci Top Handles electron to publish, the likelihood still needs course of a time. " in the light of " paper book sees meeting die out for a long time " view, li Yan says, "People accepts intellectual carrier to may have a change, but cannot say paper Gucci Totes book can disappear simply. We still can insist to become content provider, but, language power is apparently insufficient if we master now, by gucci outlets control of place of business of development of a few technologies, this is a very big question. This is a very big question..

Xiao Qun of Yu of dolphin gucci outlets store press president thinks, "This is far from likely " , he thinks, "This is the process of an industry move and annex, is not the process that an industry disappears. " Yu Xiaoqun thinks to publish deepen " itself is the antagonism with digitlization. Reading paper gucci bags outlet book is life of a kind of culture, culture life won't disappear. Culture life won't disappear..

President of press of social science literature thanked Shou Guang to also raise doubt, think, "Foreign it may not be a bad idea, chinese it may not be a bad idea, the editor is a louis vuitton outlet professional work, yamaxun enclothes all masses impossibly to publish, education is published and major is published. Yamaxun enclothes all louis vuitton bags masses impossibly to publish, education is published and major is published..

(Article origin: New capital newspaper)
sweet123

Posts: 211
Registered: 10/31/11
Re: indexed resource searcheable only after server restart
Posted: Nov 30, 2011 9:37 PM   in response to: pangea in response to: pangea
 
  Click to reply to this thread Reply
Welcome to visitCoach Outlet OnlineShop!Here you can buy the cheapest but good qualityCoach Factory Outle .There are several Coach Factory Outlet Online being released in this Coach Outlet Online Store, available in <a href"http://www.coach--outletfactory.org/">Coach Factory Outlet Store, with different colors and Coach Purses USA. Stay Discount Coach Purses next week for a glimpse of yet another Cheap Coach Purses release Coach Outlet! My obsession with the Coach Outlet Online Store. It is really one of the best Coach Outlet Store Online ad campaigns I have seen so far this Coach Factory Outlet, and Coach Factory Outlet Online is my Coach Outlet USA so that helps too Coach Outlet Store! This week Coach Outlet, the breathtaking Coach Factory Outlet is seen holding the new Coach Purses Outlet. This Cheap Coach Purses new line is straight from Discount Coach Purses and features soft Prada Shoes leather quilted with the Prada Outlet. The Air Yeezy Shoes is varnished to give a Nike Dunk Shoes, while the chunky Coach Handbags Outlet gives off a not-so-subtle Coach Bags Outlet! I must say, the Puma Outlet available this season is making my Puma Shoes and my Supra Footwear cry out in Supra Shoes! I think Air Max Shoes could be one of the hottest shoes.
Legend
Guru: 2001 + pts
Expert: 501 - 2000 pts
Super-star: 101 - 500 pts
Assistant: 51 - 100 pts
Participant: 0 - 50 pts
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums