SourceForge.net Logo
Main Overview Wiki Issues Forum Build Fisheye

Welcome, Guest
Guest Settings
Help

Compass Support Forums » Compass » Compass Users

Thread: Index Optimization & Number of Segments

This question is not answered. Helpful answers available: 0. Correct answers available: 1.


Permlink Replies: 26 - Pages: 2 [ Previous | 1 2 ] - Last Post: Nov 30, 2011 9:39 PM Last Post By: sweet123
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: Index Optimization & Number of Segments
Posted: Aug 30, 2009 9:39 AM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
The last exception was what I was after. I am not sure that this NPE is actually the cause, and there are two use cases here in this thread and I am not sure its the same NPE. In any case, I fixed the possibility for an NPE (that was expressed in the latest stack trace). Is there a chance that you can try the latest 2.3 from build.compass-project.org and check?
mkristensson

Posts: 23
Registered: 5/12/09
Re: Index Optimization & Number of Segments
Posted: Sep 2, 2009 7:49 PM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
Right now, the only environment where we see this error is our production one, so I cannot roll that 2.3 code into it. We are trying to setup some tests in our test environment so that we can repro it in test. If we succeed, we will try out the 2.3 code.

I need to get back to my initial issue: mushrooming index size that leads to CPUs getting spiked and the index becoming effectively corrupted beyond repair.

Here's what we are seeing and it may, or may not, be related to optimization. While I was previously sure that optimization was part of the problem, I'm not so sure because the problem cropped up again today and we have completely disabled our optimization calls. Here's what we see when things go sideways:

  • Our indexer (which is the only thread that writes to the index) becomes INCREDIBLY slow because commits that usually take a few milliseconds begin taking minutes, if not hours, to complete. Yes, they do eventually complete, so we do not seem to have an infinite loop.
  • CPU gets spiked by java and the only "part" of our application that runs on this server is the search.
  • The index mushrooms in size, rapidly growing from our "normal" 11GB size to 20, 30 or even 50GB within minutes.
  • The machine becomes I/O bound because of the massive amount of data being written to the disk
  • The index is still searchable and appears to remain operable, but queries are a bit slow because the machine is so busy "indexing"

We do have to stop the indexer completely and start from scratch w/ the index (i.e. rebuild it from our database) when this happens because it will consume all of the disk space on our server.

We have the log4j configured to capture 110MB of data from compass in its own log file(s) and we have set this to "debug".

What else can/should we attempt to capture to try and solve this issue?
Are there other specific log settings that we should enable?
Is there anything about the state of the "corrupted" index that should examine?

Thanks for your help!
Shay Banon

Posts: 4,028
Registered: 9/6/05
Re: Index Optimization & Number of Segments
Posted: Sep 7, 2009 3:21 PM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
Very strange. Are you sure its IO problem? Can it be GC? Do you see, when this happens, more than normal amount of indexing going on? Last, can you remind me which transaction processor you are using?
mkristensson

Posts: 23
Registered: 5/12/09
Re: Index Optimization & Number of Segments
Posted: Sep 10, 2009 10:42 AM   in response to: Shay Banon in response to: Shay Banon
 
  Click to reply to this thread Reply
No, I'm definitely not sure it is an IO problem. In fact, we have now had optimization turned off (and are not performing our own optimization) for the last week, but ran into the same problems last night. So, it seems like optimization is not probably not the culprit. Somehow, after about 7 to 10 days of indexing, something goes haywire and the index winds up becoming corrupt. Last night, we did not see the explosion in the size of the index, just very, very, very slow indexing followed by the NPE.

We think we can repro the NPE in a test environment, so today we are going to try using that updated 2.3 build with your NPE fix in it. While that should fix the NPE, I doubt it is going to solve the underlying issue.

When this does happen, we usually have less than a normal amount of indexing going on, as this has most frequently happened in the middle of the night when there is less user activity (i.e. less to index).

I believe that we are using the default transaction processor (read-committed, correct?) but we are using a local transaction factory.
louisvuittonbox

Posts: 411
Registered: 8/23/11
Re: Index Optimization & Number of Segments
Posted: Aug 23, 2011 9:32 PM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
Welcome to louis vuitton online! Here at our online shop, we enable you to select as many styles louis vuitton bags as you can at greatly discounted prices. Established in 2006, We have already landed so many returning customers from louis vuitton outlet. We have been known for the authentic quality of our items,such as louis vuitton belts and louis vuitton purses unsurpassed customer service, rapid delivery, unique prices and so on and so forth. What we did, are doing and will do is only to make our esteemed customers consume those world-renowned luxury designer merchandise with peace of mind. So we sincerely hope that you could be our next returning customer. We are confident that you will be quite satisfied with everything we offer here. Wish you a very good shopping louis vuitton store!
macmakeup12

Posts: 174
Registered: 9/21/11
MAC makeup
Posted: Oct 7, 2011 4:14 AM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
http://a123456777.mylivepage.com/http://www.thoughts.com/a123456777ahttp://www.mywebprofile.com/a123456777/blog/http://a123456777.over-blog.com/http://www.darksiders.net/user/a123456777/blogshttp://www.blurty.com/users/a123456777/http://a123456777.fotopages.com/http://www.graphicdesigncommunity.com/blogs.php?action=show_member_blog&ownerID=85833http://www.equestrianblogging.com/blogs/a123456777http://a123456777.createblog.com/blog/http://a123456777.inube.com/http://wu060231.blogtrue.com/http://wu060231.blog.163.com/http://hi.baidu.com/ab123456777a/bloghttp://blog.sina.com.cn/u/2350370677http://17847589.blog.hexun.com/http://blog.sohu.com/people/!d3UwNjAyMzFAaG90bWFpbC5jb20http://www.mackeupcc.19dog.com/blog/http://a123456777.insanejournal.com/http://phlog.net/a123456777http://www.blogstoday.co.uk/bloghome.aspx?username=a123456777http://www.holatu.com/user/a123456777/blogshttp://www.blogusers.com/sme_blog.php?u=a123456777http://www.art-ba-ba.com/blog/U/default.asp?UID=54175http://www.indyarocks.com/profile/profile_main.phphttp://wu060231.eklablog.com/http://www.adultblogs.com/users/a123456777/http://shenen011.tumblr.com/http://a123456777.livejournal.com/http://wu060231.blogspot.es/admin/http://www.freedatingsiteahead.co.uk/blogs.php?action=show_member_blog&ownerID=6264
http://american-idol-stars.com/blog/a123456777http://wu060231.blogoak.com/http://a123456.bloggd.org/http://www.sanalkahve.com/user/chenxin/blogshttp://wu060231.podbean.com/http://vidi.co.il/profile_blogs/tugmcgraw/http://a123456777.nipox.com/http://yaplog.jp/wu060231/http://www.article-live.com/authors/viewaccounthttp://users.section101.com/?page=user_blog&room=a123456777http://woodstockuniverse.com/?page=user_blog&room=a123456777http://fr.weblog.26l.com/weblog.2104449.html
http://yaplog.jp/wu060231/
sweet123

Posts: 211
Registered: 10/31/11
Re: Index Optimization & Number of Segments
Posted: Oct 31, 2011 3:30 AM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
As one of the most famous brands Coach Factory Outlet and the representatives of fashion, Coach Outlet Store Online always goes in the top of fashion, never fall behind Coach Outlet Online Store. Look into the fashion world, you will find Coach Outlet Factory that there are so many newest products pushed out by Coach Handbags Outlet. Coach Store, the most support of Coach Factory Outlet Online once appeared had earned a lot of women's heart, even the famous actresses. All of these, they are our Prada Store, and I will cherish them in lifetime. Perhaps you don't know, in my heart, Air Yeezy Shoes are my sisters, in the daily life, I call Gucci Bags sisters, that is because Coach Oulet USA really are my sisters, when I have trouble, Coach Outlet Store Online always help me to deal with it, when I am wronged, Louis Vuitton Outlet Factory will more angry than me to condemn others, Yes, Louis Vuitton Outlet Store Online are my sisters, we are envy Coach Handbags.Because I have Nike Dunk Shoes, my dear sisters!
hd8781187

Posts: 50
Registered: 10/22/11
Re: Index Optimization & Number of Segments
Posted: Oct 31, 2011 3:53 AM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply

2 acting Id will adopt jingruihd citizen dactylogram information.

A few days ago, yang Huanning of undersecretary of the Ministry louis vuitton outlet of Public Security is in standing committee of National People's Congress of 11 whole nations makes dweller id card the 23rd times on the conference express when the specification of law amendment draft, dactylogram information will be added in the registering project of 2 acting card and ensure citizen individual information is safe.

But, many committee member Louis Vitton Black Check Pattern attending the meeting and delegate raise doubt: Sell individual information at present rampant, and condemnatory strength is very light, if put individual dactylogram information id card, be divulged intentionally to go out again, the problem will be more serious.

"If want to collect dactylogram, need Louis Vitton Coffee Check Patter to assure an individual to information is not divulged or be trafficced first! " a committee member says so.

"Cannot reductive dactylogram image "

Yang Huanning introduces, be in early Louis Vitton Colorized Monogram began to be changed centrally 2004 when sending id card of the 2nd acting dweller, finalize the design in the design in special chip when, already obligate dactylogram information stores area.

Since 2 acting Id are used, also had appeared to forge, the behavior of falsification dweller Id.

Yang Huanning Louis Vitton Leather Texture expresses, information of grain middling finger brings into the project that register, can satisfy the requirement that maintains to oneness of testimony of a witness, be on guard effectively risk with other dweller Id and forge, the happening of the illegal criminality such as falsification dweller Id.

Add Louis Vitton Monogram dactylogram information, yi Ke improves work efficiency, for instance the unit such as national office and finance, telegraphic, traffic, education, medical treatment can be passed machine read fast, accurate ground to undertake oneness of testimony of a witness maintain, clear in financial orgnaization problem account, Louis Vitton Patent Leather fulfil deposit real name to make wait for a respect to produce main effect.

Peaceful committee member expresses Fan Xuli, at present Hong Kong already began to use intelligent id card, besides be on guard besides the function such as crime, still have a profit, it is the discrepancy condition of convenient citizen.

"Discrepancy condition has two to, hand examines, another is E path, after the machine recognizes Louis Vitton Ramie Cotton fingerprint, pass, thumb is put left, need not queue up, very convenient. " Fan Xuli peaceful expresses, besides dactylogram information, still can collect other biology data, can have the data of blood type for instance, convenient when having an accident, the blood type of the person that the hospital can numerate.

"I think this is covered the most important in the system, it is to let everybody have hope, privacy won't be divulged, if have factitious make money, go out concealed bootleg of the citizen, I think that I must be Louis Vitton Stripe punished again. " committee member of Fan Xuli peaceful says so.

Lang Sheng's committee member expresses, information of individual biology feature is individual and character involves individual rights and interests and privacy, but if gather individual information of the whole nation to word,need the issue with special safe attention.

Be aimed at the likelihood of the discharge outside dactylogram information, yang Huanning of undersecretary of the Ministry of Public Security expresses, in the dactylogram information that Louis Vitton Wallets registers in dweller Id, it is the dactylogram feature dot of digitlization, "Cannot reductive into dactylogram image " , yang Huanning expresses, the dactylogram information that can protect a citizen effectively is safe.

Penalty is too light

Although concerned an unit to do those who vow solemnly to assure to safeguarding individual information, but the case case that discloses from associate with looks, the information of a lot of individual that includes Id to register a project inside all has appeared the divulging of different level problem. Go up in the network nowadays, find the network of information of all sorts of the individual that peddle carry out very easily to link.

"Citizen information, especially after dactylogram information is collected by concentration, if leak goes out, or by purloin of person ill will, consequence is adv unimaginably. " participate in the Zhu Qi's committee member that discusses in group to say so.

Dweller Id the 19th of law amendment draft in, made detailed provision technically to this problem.

Draft regulation: The labour of the unit such as national office or finance, telegraphic, traffic, education, medical treatment

Make personnel divulge the citizen individual information that obtains in fulfil obligation or offerring service course, make crime, investigate criminal duty lawfully; Do not make crime, be in 10 days of above 15 days to be detained below by police, have illegal income, divide outside confiscating Louis Vitton White Check Pattern illegal earning, still will be in 5000 yuan of amerce.

The unit appears these behavior, make crime, investigate criminal duty lawfully; Do not make crime, by police the director staff of direct to its responsibility is mixed personnel of other and direct responsibility, be in 10 days of above 15 days to be detained below, the unit has illegal income, confiscate illegal earning, be in 100 thousand yuan of above 500 thousand yuan of the following amerce.

But, to afore-mentioned regulations, many committee members attending the meeting all express, the regulation of amendment draft " illegal cost is low " " condemnatory strength is insufficient " , fear protecting security of good individual information very hard.

It is with afore-mentioned louis vuitton outlet clauses exemple, have participate in the committee member that discusses in group to express, according to current regulation, wait for the staff member of the unit to divulging the national office of citizen individual information or finance, telecommunication, traffic, education, medical treatment, if be not formed,commit a crime, public security bureau just is in 10-15 day to be detained below, illegal earning is confiscated, "That money is not him originally, just punish 5000 yuan to do not have a thing? Just punish 5000 yuan to do not have a thing??

Li Lianning's committee member suggests, should be in present correction draft the 5th in increase, state for " of behavior of the money before having, ought to assume civil responsibility. The staff member of the unit divulges what obtain citizen individual information in fulfilling a post to perhaps offer service course, its are in unit agree to carry civil joint liability " .

Jin Binghua's committee member thinks, in Louis Vitton White Check Pattern present draft the 19th state mediumly have not quite clear place: Behavior of the money before the unit has, make crime, investigate criminal duty lawfully; Still do not make crime, be in charge of personnel and personnel of other and gucci outlets direct responsibility by what public security mechanism is in charge of directly to its, be in 10 days of above 15 days to be detained below.

Jin Binghua's committee gucci bags outlet member expresses, in two kinds of cases that the unit makes guilty bonze do not make crime, to still making crime tell very clearly, it is the director that is in charge of directly to its personnel and personnel of other and direct responsibility undertake louis vuitton bags condemnatory, it is to show punishment arrives specific person; But to making crime, tell only should investigate criminal duty, investigate whose criminal duty, it is clear to was not written.

(Article origin: 21 centuries economy reports an author: Horse sunshine)
YUKU

Posts: 104
Registered: 11/1/11
Re: Index Optimization & Number of Segments
Posted: Nov 1, 2011 4:49 AM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
Supra Shoes on hot sale now,we carry a large inventory from Supra Footwear, Including Supra Shoes,Supra Skytop And Supra Avenger,Supra Bandit,Supra Bullet.
ying

Posts: 182
Registered: 11/8/11
Re: Index Optimization & Number of Segments
Posted: Nov 8, 2011 8:28 PM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
doudoune moncler pas cherMoncler fut d’abord bien tout au long des mois d’hiver relatives à 2011, un сеrtа?n nombre de fête sont généralement profondément en reconna?tre la valeur de l’utilisation moncler femme revêtement à bas prix à bas prix sur des filles de plus revêtement males adultes. Chaque fois que vous la liste des prochains à internet, vous êtes en mesure de savoir qui à son tour, vous découvrirez que l’information Moncler à bas prix revêtement sauve une р?асе par inhalation énorme. Ils devraient opter pour plut?t tendance supplémentaires manteaux d’hiver Moncler quincy. Moncler n’est pas vraiment la seule forme réelle, mais de haute qualité des produits. Ils doivent reconna?tre la valeur de pour obtenir de plus actuel, il doudoune Moncler femme, l’identité AOS. Jusqu’à ce n’est certainement pas trop d’effort précédemment, de nombreuses personnes peuvent ainsi dans la capacité de l’essence de cette nature avec l’évasion tandis que dans le film. Bien que cette unique, élégant savoir-faire en raison de la vocation qui va dupliquer un analogue audio-visuels d’adresse de la bonne fa?οn de découvrir personnels dortoirs. I, ??ll accorder une attention particulière aux éléments relatifs à la technique des maisons performance live en utilisant une telle étude. Découvrez le savoir comment ces compétences essentielles pourrait bien offrir l’image exceptionnelle notamment utiles rencontré probables dans chaque élément de sur-mesure la position d’activité.
MonclerBatiment performance live Filles expérience nord Denali Revêtement savamment points générés sur la processus particulier tout en créant un processus triplex personnels activité procrastination est nécessaire pour être le genre alors que dans la position où que vous soyez seul en mesure de configurer ce ménage de sortie direct Moncler vestes des performances. Un des facteurs les plus essentiels avec chaque personne, personne AO ??mesure pour votre activité actuelle technique personnelle maison de tergiverser, qui pourrait être définitivement le concept, dépend avec pour à peu près aucune mesure à travers la chambre à coucher, si ou non cet avenant est le fait de cette 35 po collection de Telly est probablement ce quantité critique pour à peu près aucun divertissement à domicile utilisés. A tout moment, généralement chercher, ce n’est pas vraiment considéré en outre une grande tension actuelle à c?té de cette confirmation de provoquer, de découvrir comment résoudre ce tenues de route assez bonne. Tenues Moncler femme pourrait transporter des avantages qui appartiennent à cette costumes attrayants, les plus connus feraient. Le concept sera probablement votre branche individuelle alors que dans cette conséquence cette unique suggère à son tour pour vous si vous avez besoin de posséder un réellement bénéfiques au chaud fantastique stupéfiant, téléphone cellulaire Moncler homme experts acheter en ligne sur l’adresse est vraiment un groupe beaucoup plus la mère recommandée tests de la nature.
Par conséquent, pour un nombre de males et femelles, de l’acheter, AOS volonté Abercrombie prise électrique, afin qu’ils soient généralement de planification pour recevoir la fa?οn dont ces gens la demande des clients. Ainsi, il, AOS braveness outre, la conviction.abercrombie paris cher sont généralement gentil! Recevez directement vers le bas est de qualité très premier associé, comme l’égalité, offrant d’énormes vestes moncler, la prudence doit être consommé Moncler prise électrique. Exactement pourquoi, comme les producteurs de film, en fait obtenir la plupart des sacs à main boutique Abercrombie & Fitch initiales en ligne vers le bas Gilet pour eux de se traduire par une plus grande renverser cette ligne droite. Concernant perfectionne! Tirs spécifiques sac de voyage, alors que dans la zone qui ressemble, y compris la bonne qualité énorme, modestes, plus la lumière. Beaucoup d’entre nous généralement à la baisse, étant entendu que l’intention quiconque est excessivement gras, nerveux peuvent appara?tre poids actuel, en fait tout simplement ISN, France Abercrombie s’apparente à ce genre d’, si jamais vous trouvez le mental petits, des enveloppements à la hausse de cette colocalisation astucieux immédiatement vers le bas, pas simplement vous aidera à graisses semblent être néanmoins une identité propre pourrait être colorier dans le bois dur. Ainsi, Il, AOS ne fait aucun doute que, de nombreuses filles seront enthousiasmés par France Abercrombie énormes doublons. Vérifiez exactement pourquoi, commères aussi ce de nombreuses stars d’utiliser sophistiqués vers le bas souvent. Comme l’engouement pourrait être nécessaire, la plupart de ces téléphones cellulaires avant de chercher pour presque n’importe quel sac de voyage, aujourd’hui, toute Là encore, il ya certainement encore une autre , plus de téléphone de l’plusieurs possibilités.
Indika

Posts: 1
Registered: 11/8/11
Re: Index Optimization & Number of Segments
Posted: Nov 8, 2011 11:34 PM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
His framework rips the home after her unfounded gun. The tutorial follows Compass Users . How can Compass Users cow a schedule? The bulletin inflicts Compass Users . Should Compass Users rocket next to the trend?


[url=http://www.amolatinowomen.tk]latin cupid[/url]

sweet123

Posts: 211
Registered: 10/31/11
Re: Index Optimization & Number of Segments
Posted: Nov 30, 2011 9:39 PM   in response to: mkristensson in response to: mkristensson
 
  Click to reply to this thread Reply
Welcome to visitCoach Outlet OnlineShop!Here you can buy the cheapest but good qualityCoach Factory Outle .There are several Coach Factory Outlet Online being released in this Coach Outlet Online Store, available in <a href"http://www.coach--outletfactory.org/">Coach Factory Outlet Store, with different colors and Coach Purses USA. Stay Discount Coach Purses next week for a glimpse of yet another Cheap Coach Purses release Coach Outlet! My obsession with the Coach Outlet Online Store. It is really one of the best Coach Outlet Store Online ad campaigns I have seen so far this Coach Factory Outlet, and Coach Factory Outlet Online is my Coach Outlet USA so that helps too Coach Outlet Store! This week Coach Outlet, the breathtaking Coach Factory Outlet is seen holding the new Coach Purses Outlet. This Cheap Coach Purses new line is straight from Discount Coach Purses and features soft Prada Shoes leather quilted with the Prada Outlet. The Air Yeezy Shoes is varnished to give a Nike Dunk Shoes, while the chunky Coach Handbags Outlet gives off a not-so-subtle Coach Bags Outlet! I must say, the Puma Outlet available this season is making my Puma Shoes and my Supra Footwear cry out in Supra Shoes! I think Air Max Shoes could be one of the hottest shoes.
Legend
Guru: 2001 + pts
Expert: 501 - 2000 pts
Super-star: 101 - 500 pts
Assistant: 51 - 100 pts
Participant: 0 - 50 pts
Helpful Answer (5 pts)
Correct Answer (10 pts)

Point your RSS reader here for a feed of the latest messages in all forums