SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye

Welcome, Guest
Guest Settings
Help

Compass Support Forums » Compass » Compass Users

Thread: Storing index in replicated coherence cache

This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 8 - Pages: 1 - Last Post: Jan 9, 2012 9:25 PM Last Post By: anitas
Andrew Klochkov

Posts: 2
Registered: 3/19/09
Storing index in replicated coherence cache
Posted: Mar 19, 2009 9:05 AM
 
  Click to reply to this thread Reply
Hello

We're trying to use Lucene-Coherence integration from Compass and all I can say now is it works.
Looks like nobody uses this feature in production or at least tested it performance. Is it so?
Also I wonder does it make sense to store small index (128Mb) in a replicated Coherence cache, so every coherence node will have a complete index which could be used by collocated Lucene instance to search, so this way we'll have a cluster of searchers which are connected by Coherence. What do you think Compass experts and Shay in particular :-)

Thanks, Andrew
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: Storing index in replicated coherence cache
Posted: Mar 19, 2009 3:29 PM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply
aklochkov wrote:
Hello

We're trying to use Lucene-Coherence integration from Compass and all I can say now is it works.
Looks like nobody uses this feature in production or at least tested it performance. Is it so?

Why do you say that? Do you see performance problems?

Also I wonder does it make sense to store small index (128Mb) in a replicated Coherence cache, so every coherence node will have a complete index which could be used by collocated Lucene instance to search, so this way we'll have a cluster of searchers which are connected by Coherence. What do you think Compass experts and Shay in particular :-)

Yep, that should work nicely.

Thanks, Andrew
Andrew Klochkov

Posts: 2
Registered: 3/19/09
Re: Storing index in replicated coherence cache
Posted: Mar 20, 2009 6:25 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
kimchy wrote:
aklochkov wrote:
Hello

We're trying to use Lucene-Coherence integration from Compass and all I can say now is it works.
Looks like nobody uses this feature in production or at least tested it performance. Is it so?

Why do you say that? Do you see performance problems?

No, it's just not clear for me how Lucene will perform while using index stored in Coherence. And there are so many configuration options in Coherence which will affect that so that's why I'm interested if someone have done such deployments and what's the results. IMHO the most important of these Coherence options is the topology of cache to use. I'm not sure is it better to store our small index in a replicated or partitioned cache.

It seems that replicated way is good from the search performance point of view but I doubt about updates performance - how will they be propagated, won't they be a bottleneck in the case of large amount of cluster nodes?

In case of partitioned cache for index: do we have to enable near cache to improve search performance? Is it possible to perform "distributed" queries so they run on all cluster nodes and then the results are combined somewhere (i.e. using map-reduce pattern for search)? Should compass be collocated with coherence on every cluster node and so one will get a cluster of compass instances every one of which could be used for searches and updates?

Sorry for that load of questions but I think they should be answered so people begin to use these features of Compass.


Also I wonder does it make sense to store small index (128Mb) in a replicated Coherence cache, so every coherence node will have a complete index which could be used by collocated Lucene instance to search, so this way we'll have a cluster of searchers which are connected by Coherence. What do you think Compass experts and Shay in particular :-)

Yep, that should work nicely.

I'm going to do benchmarking of different coherence cache topologies used for lucene index storage.

Also I'm about to test Solr java replication and I'm curious to see if the cluster of Solr instances could perform better. Could you please share your thoughts on this subject also?

Andrew

Edited by: Andrew Klochkov on Mar 20, 2009 6:26 AM
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: Storing index in replicated coherence cache
Posted: Mar 20, 2009 7:20 AM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply
aklochkov wrote:
kimchy wrote:
aklochkov wrote:
Hello

We're trying to use Lucene-Coherence integration from Compass and all I can say now is it works.
Looks like nobody uses this feature in production or at least tested it performance. Is it so?

Why do you say that? Do you see performance problems?

No, it's just not clear for me how Lucene will perform while using index stored in Coherence. And there are so many configuration options in Coherence which will affect that so that's why I'm interested if someone have done such deployments and what's the results. IMHO the most important of these Coherence options is the topology of cache to use. I'm not sure is it better to store our small index in a replicated or partitioned cache.

It seems that replicated way is good from the search performance point of view but I doubt about updates performance - how will they be propagated, won't they be a bottleneck in the case of large amount of cluster nodes?

I don't know, since I am no expert in Coherene to be honest (though have some basic understanding in distributed computing :) ).


In case of partitioned cache for index: do we have to enable near cache to improve search performance? Is it possible to perform "distributed" queries so they run on all cluster nodes and then the results are combined somewhere (i.e. using map-reduce pattern for search)? Should compass be collocated with coherence on every cluster node and so one will get a cluster of compass instances every one of which could be used for searches and updates?

Possibly, the best and simplest solution would be a partitioned cache with near cache on the clients. Collocation can be the best solution, but I don't know how this can be done in Cohernece. You should register on each node a change listener, and index the changes on the local cohernece node, and then distribute the search operation.


Sorry for that load of questions but I think they should be answered so people begin to use these features of Compass.

Sure, no problem. Compass basically provides the basics. The ability to store Lucene on Coherence, and the ability to index objects (OSEM) which can be derived from data stored on Coherence nodes.



Also I wonder does it make sense to store small index (128Mb) in a replicated Coherence cache, so every coherence node will have a complete index which could be used by collocated Lucene instance to search, so this way we'll have a cluster of searchers which are connected by Coherence. What do you think Compass experts and Shay in particular :-)

I think that a partitioned cache with near cache will perform very well, since the index will probably be mostly on the client side, so search operations will be very fast. Also, I don't think that you will need to map reduce search operations with such a small index.

You can try Solr as well, I don't know how it will perform against it, but, if you have most of the index in a near cache, then it will be very difficult for Solr to perform better than that...


Yep, that should work nicely.

I'm going to do benchmarking of different coherence cache topologies used for lucene index storage.

Also I'm about to test Solr java replication and I'm curious to see if the cluster of Solr instances could perform better. Could you please share your thoughts on this subject also?

Andrew

Edited by: Andrew Klochkov on Mar 20, 2009 6:26 AM
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: Storing index in replicated coherence cache
Posted: Mar 20, 2009 7:20 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
And on more thing, I know of a very large financial institute that use Compass with Coherence, and it seems to work for them very nicely. Don't have more details though...
louisvuittonbox

Posts: 411
Registered: 8/23/11
Re: Storing index in replicated coherence cache
Posted: Aug 23, 2011 9:25 PM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply
Welcome to louis vuitton online! Here at our online shop, we enable you to select as many styles louis vuitton bags as you can at greatly discounted prices. Established in 2006, We have already landed so many returning customers from louis vuitton outlet. We have been known for the authentic quality of our items,such as louis vuitton belts and louis vuitton purses unsurpassed customer service, rapid delivery, unique prices and so on and so forth. What we did, are doing and will do is only to make our esteemed customers consume those world-renowned luxury designer merchandise with peace of mind. So we sincerely hope that you could be our next returning customer. We are confident that you will be quite satisfied with everything we offer here. Wish you a very good shopping louis vuitton store!
factoryoutlet

Posts: 523
Registered: 10/5/11
louis vuitton outlet
Posted: Oct 5, 2011 9:34 PM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply

louis vuitton outlet leads you to bags collection of wide, such as LV bags and LV handbag etc. They are main name in luxury and style. It is excellent quality, fine Italian technology and extraordinary beauty.LV is universally renowned as a luxury brand. louis vuitton online store offers customers the newest fashion collections which can make you different.

In the past, Louis Vuitton only emphasis on classic, but in recent year, Louis Vuitton change its strategy and style to acquire new breakthrough. Especially louis vuitton bags like Louis Vuitton Vernis Wallet.Want to become more charming and faddish? Go to visit louis vuitton outlet webpage and select the most suitable products for yourself.

Want to become more charming and faddish? Go to visit louis vuitton outlet webpage and select the most suitable products for yourself.louis vuitton outlet online may often be the consideration alternative inside of your event to get these kinds of bags.Is without a doubt louis vuitton online seriously is present? Is it possible to offer an outlet designed for Louis Vuitton merchandise speaks out?

The choices are likely to be basically countless seeing that louis vuitton outlet occurs with the help of completely new and also incredible concepts once in a while.Mainly because mentioned before, you could buy in the louis vuitton online, via the producer's the web outlet. Owning worn it in past times. They can supply a appropriate evaluate for the property's simplicity as well as client service description.louis vuitton outlet online is the method to show the confidence of our handbags. The person who show himself in front of other people is a confident one.

LV bags in the louis vuitton outlet are a feminine update on an urban classic. With plenty of room for all of your essentials this covetable carryall will take you through the weekend in style.LV is universally renowned as a luxury brand. louis vuitton online store offers customers the newest fashion collections which can make you different.

This collection focus more on color selection compared with other LV handbags. Cute and sunshine design style increases its charm. louis vuitton outlet online provides you the very famous luxury worldwide brand.We are glad you come to join in our louis vuitton outlet online store, because the winter is upcoming now, Louis Vuitton released many new handbags to feedback our customers.Just come to visit once the louis vuitton online, you will fall in love with its fancy bags, purses, sunglasses, shoes and so on.

factoryoutlet

Posts: 523
Registered: 10/5/11
coach factory outlet
Posted: Oct 5, 2011 9:41 PM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply

coach factory outlet offer excellent, elegant coach products in order to generate room for your new collections.They're purchased more quickly that you may at any time imagine.coach factory store is the greatest destination to buy what type of bags you like. If you need to manage a new Coach carrier and most preferable strategy is to the coach regarding coupon codes.Here's your new way to look sophisticated styles. Carry the hot coach factory outlet store of this summer and be the true trendsetter!in the market you definitely can find various colorways that are designed in as well as the high quality that applied in. For most of you would like to come. So just come to our coach factory outlet online store to choose one.

coach factory outlet is really sizzling kinds of shopping way for you. With the usage of the replica designer coach bags, you can surely be able to change your individual looks in a stunning manner.You know, Coach items are so perfect and fascinating. Now I grow up, and find coach outlet on the Internet offering affordable products with reliable quality.Many fashionable women match with practical Coach Purses which will make the street shopping become relaxed,and make every person can enjoy more diversiform combination in coach factory outlet online.The coach factory outlet store are utilized for formal event in company area that will display your excellent taste.Not only does a briefcase make is easier to carry things.

Almost everyone has identified effectively about this, below, among the best to make sure you will get much more information about coach outlet.In the coach outlet online shop, a collection of high quality and hottest Coach Shoulder Bags is offered to you. They are all at the best prices you will ever find.

You know, Coach items are so perfect and fascinating. Now I grow up, and find coach outlet on the Internet offering affordable products with reliable quality.Once visit, you will be unable to restrain loving it. Warmly welcome all of you to the Summer Sales Promotion on coach factory outlet.coach factory outlet has new coach laptop bags in listing. No matter which style do you want, we can provide it for you.Coach bags enjoy high popularity throughout the world. I would like to share the coach factory outlet online with you. What are you waiting for? Just come to visit.

coach factory outlet supplies the newest and most fashionable brand Coach on sale.The more Coach shoulder bags you order, the more discount you will get.It's wonderful to find out it does not slip in your signature at the bottom of the case and the appearance of brass accents from coach outlet online, Because women who can look stylish this summer.coach outlet has a zippered closure and buckles for extra security. Shiny brass hardware, rounded leather handle, and an interior pocket. It also includes a limited edition Hawaii luggage tag and lock.You can find all kinds of coach bags whatever you chose which one in coach factory outlet online will never be out of fashion. Not only they have many nice and design styles, but also made from the finest leather and fabric.

With the great diversity of styles, patterns and sizes available nowadays, finding the right pieces to highlight your personality is quite important. coach outlet offers exactly what you want.coach factory outlet supplies the newest and most fashionable brand Coach on sale.The more Coach shoulder bags you order, the more discount you will get.Look at this golden Coach bag in the? coach factory outlet online.Those new bags with classic and fashionable design are more and more popular now!After graduating from university, my cousin came back to my hometown from USA last month. She told me that coach outlet store is very popular among her classmate.

coach outlet online own a couple of line, color, or the selected theme, it should be a good choice. It may be the place where offer the bag ladies handbag, you will be able to follow a course of education or training receives for the slim project chest strap wallet.coach outlet will send you a coupon in the post to use in their upon only after you type a achieve. The Coach bags are customarily somewhat many than the ones in the stores.In particular, products from coach factory outlet with leather design are fashionable, handmade,leading the wave of American pop.It with simple,durable and unique style to win consumers.coach factory not only supply with the highest quality crystal at reasonable price but also provide good after-sales service. Companies operating purposes: science and technology in order to survive, credibility and development, strict quality control.

coach outlet handbags's special design and fashion is certainly and so eye-catching, I hope you are able to get much more attention towards the Cheap Coach Crossbody.As long as you open our coach factory outlet webpage, you will view various Coach New Arrivals which are the most popular also the most fashionable in this year.Are you anxious to choose some of the brand bags? Are you searching for reliable sellers? When you come to the coach factory outlet online, these are not problems.

At the coach outlet online you have the largest selection of the day. If you touch the item and like it, keep it in your possession until you make your final decision.Over the years, coach factory online has added a multitude of new handbag shapes, styles and materials to their collection. However, the highest care is taken that every Coach handbags is both aesthetically beautiful and functional.Are you searching for a cool and stylish coach bag? The coach factory outlet online Online is your first priority.

anitas

Posts: 95
Registered: 11/20/11
Re: Storing index in replicated coherence cache
Posted: Jan 9, 2012 9:25 PM   in response to: Andrew Klochkov in response to: Andrew Klochkov
 
  Click to reply to this thread Reply
I love this article, it's very well.oakley sunglasses saleFake Oakley Flak Jacket Sunglassesreplica Oakley Polarized SunglassesOakley Batwolf sunglasses outletoakley sunglasses wholesaleReplica Oakley Flak Jacket SunglassesOakley Restless SunglassesOakley Dispatch sunglassesreplica oakley sunglassesReplica Oakley Fuel Cell Sunglassesoakley scalpel sunglassesOakley Flak Jacket sunglassesreplica Oakley Commit SQ SunglassesFake Oakley GascanOakley Split Jacket SunglassesOakley Frogskins sunglasses outletoakley fives squared sunglassesFake Oakley Polarized Sunglassesoakley straight jacket sunglassesOakley Gascan sunglassesOakley Half Jacket SunglassesReplica Oakley Polarized SunglassesOakley Ten SunglassesFake Oakley Hijinx sunglassesreplica Oakley Half Wire SunglassesFake Oakley RadarOakley Zero SunglassesOakley Jawbone sunglassesreplica Oakley Half X SunglassesReplica Oakley sunglassesOakley Juliet sunglassesOakley Hijinx SunglassesCheap oakleys Commit SQ SunglassesOakley Jupiter sunglassesOakley Holbrook SunglassesReplica Oakley Fives SquaredOakley M Frame sunglasses outletoakley jawbone sunglassesReplica Oakley Flak JacketOakley Minute sunglassesoakley juliet sunglassesReplica Oakley FrogskinsOakley Monster Dog sunglassesreplica Oakley Jupiter SunglassesReplica Oakley Fuel CellOakley Oil Rig sunglassesOakley Jury SunglassesReplica Oakley GascanOakley Pit Boss sunglassesoakley m frame sunglassesCheap oakleys Half Wire SunglassesOakley Polarized sunglasses outletoakley minute sunglassesCheap oakleys Half X SunglassesOakley Radar sunglassesreplica oakley minute sunglassesCheap oakleys Hijinx SunglassesOakley Restless sunglassesoakley monster dog sunglassesCheap oakleys Jawbone SunglassesOakley Scalpel sunglassesOakley Necessity SunglassesCheap oakleys Oil Rig SunglassesOakley Straight Jacket sunglassesoakley oil rig sunglassesCheap oakleys Pit Boss SunglassesOakley Ten sunglassesOakley Pit Bull SunglassesCheap oakleys Radar SunglassesOakley Zero sunglassesOakley Plaintiff SunglassesCheap oakleys Straight Jacket Sunglasses
Legend
Guru: 2001 + pts
Expert: 501 - 2000 pts
Super-star: 101 - 500 pts
Assistant: 51 - 100 pts
Participant: 0 - 50 pts
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums