Lucene indexer not indexing all documents

[Sam Collett, módosítva 9 év-val korábban]

Using Liferay Portal Community Edition 6.2 CE GA2. The document search does not seem to be working - when I perform a re-index, it is failing to re-index the existing documents, though new ones are indexed (probably because it is done after upload). On further investigation, it seems to be a bug caused by orphan data in the wiki, causing the indexer to stop indexing, according to the logged issue Reindexing with orphaned data will throw exceptions (LPS-43849). When I look at the related issue Comments of a renamed wiki are not deleted properly (LPS-43848) and run the SQL, there are a few entries. How do I safely delete them to get indexing to work again? Substituting `SELECT *` with `DELETE` does not work (even after doing a services/service reboot and reindexing after the deletion).

Exception from log:

10:44:53,140 ERROR [http-apr-8081-exec-27][LuceneIndexer:136] Error encountered while reindexing
com.liferay.portal.kernel.search.SearchException: com.liferay.portlet.wiki.NoSuchPageException
	at com.liferay.portal.kernel.search.BaseIndexer.getDocument(BaseIndexer.java:179)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer$3.performAction(MBMessageIndexer.java:443)
	at com.liferay.portal.kernel.dao.orm.BaseActionableDynamicQuery.performActionsInSingleInterval(BaseActionableDynamicQuery.java:309)
	at com.liferay.portal.kernel.dao.orm.BaseActionableDynamicQuery.performActions(BaseActionableDynamicQuery.java:45)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer.reindexMessages(MBMessageIndexer.java:454)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer$2.performAction(MBMessageIndexer.java:403)
	at com.liferay.portal.kernel.dao.orm.BaseActionableDynamicQuery.performActionsInSingleInterval(BaseActionableDynamicQuery.java:309)
	at com.liferay.portal.kernel.dao.orm.BaseActionableDynamicQuery.performActions(BaseActionableDynamicQuery.java:45)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer.reindexDiscussions(MBMessageIndexer.java:412)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer.doReindex(MBMessageIndexer.java:340)
	at com.liferay.portal.kernel.search.BaseIndexer.reindex(BaseIndexer.java:457)
	at com.liferay.portal.search.lucene.LuceneIndexer.reindex(LuceneIndexer.java:159)
	at com.liferay.portal.search.lucene.LuceneIndexer.doReIndex(LuceneIndexer.java:125)
	at com.liferay.portal.search.lucene.LuceneIndexer.reindex(LuceneIndexer.java:64)
	at com.liferay.portal.search.lucene.LuceneIndexer.reindex(LuceneIndexer.java:57)
	at com.liferay.portlet.admin.action.EditServerAction.reindex(EditServerAction.java:384)
	at com.liferay.portlet.admin.action.EditServerAction.processAction(EditServerAction.java:199)
	at com.liferay.portal.struts.PortletRequestProcessor.process(PortletRequestProcessor.java:166)
	at com.liferay.portlet.StrutsPortlet.processAction(StrutsPortlet.java:216)
	at com.liferay.portlet.FilterChainImpl.doFilter(FilterChainImpl.java:71)
	at com.liferay.portal.kernel.portlet.PortletFilterUtil.doFilter(PortletFilterUtil.java:48)
	at com.liferay.portlet.InvokerPortletImpl.invoke(InvokerPortletImpl.java:618)
	at com.liferay.portlet.InvokerPortletImpl.invokeAction(InvokerPortletImpl.java:649)
	at com.liferay.portlet.InvokerPortletImpl.processAction(InvokerPortletImpl.java:312)
	at com.liferay.portlet.PortletContainerImpl._doProcessAction(PortletContainerImpl.java:402)
	at com.liferay.portlet.PortletContainerImpl.processAction(PortletContainerImpl.java:110)
	at com.liferay.portlet.SecurityPortletContainerWrapper.processAction(SecurityPortletContainerWrapper.java:109)
	at com.liferay.portlet.RestrictPortletContainerWrapper.processAction(RestrictPortletContainerWrapper.java:75)
	at com.liferay.portal.kernel.portlet.PortletContainerUtil.processAction(PortletContainerUtil.java:115)
	at com.liferay.portal.action.LayoutAction.processLayout(LayoutAction.java:356)
	at com.liferay.portal.action.LayoutAction.doExecute(LayoutAction.java:178)
	at com.liferay.portal.action.LayoutAction.execute(LayoutAction.java:79)
	at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
	at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
	at com.liferay.portal.struts.PortalRequestProcessor.process(PortalRequestProcessor.java:173)
	at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
	at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
	at com.liferay.portal.servlet.MainServlet.callParentService(MainServlet.java:546)
	at com.liferay.portal.servlet.MainServlet.service(MainServlet.java:523)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.secure.SecureFilter.processFilter(SecureFilter.java:293)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.jsoncontenttype.JSONContentTypeFilter.processFilter(JSONContentTypeFilter.java:42)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:96)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:412)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
	at com.liferay.portal.servlet.FriendlyURLServlet.service(FriendlyURLServlet.java:160)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.strip.StripFilter.processFilter(StripFilter.java:359)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.gzip.GZipFilter.processFilter(GZipFilter.java:123)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.secure.SecureFilter.processFilter(SecureFilter.java:293)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.i18n.I18nFilter.processFilter(I18nFilter.java:243)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.etag.ETagFilter.processFilter(ETagFilter.java:70)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.jsoncontenttype.JSONContentTypeFilter.processFilter(JSONContentTypeFilter.java:42)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.autologin.AutoLoginFilter.processFilter(AutoLoginFilter.java:263)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.sso.ntlm.NtlmPostFilter.processFilter(NtlmPostFilter.java:83)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.sharepoint.SharepointFilter.processFilter(SharepointFilter.java:88)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
	at com.liferay.portal.servlet.filters.virtualhost.VirtualHostFilter.processFilter(VirtualHostFilter.java:226)
	at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:185)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:738)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:165)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:165)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:185)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
	at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:96)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2430)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2419)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: com.liferay.portlet.wiki.NoSuchPageException
	at com.liferay.portlet.wiki.service.impl.WikiPageLocalServiceImpl.getPage(WikiPageLocalServiceImpl.java:985)
	at com.liferay.portlet.wiki.service.impl.WikiPageLocalServiceImpl.getPage(WikiPageLocalServiceImpl.java:950)
	at com.liferay.portlet.wiki.service.impl.WikiPageLocalServiceImpl.getPage(WikiPageLocalServiceImpl.java:940)
	at com.liferay.portlet.wiki.service.WikiPageLocalServiceWrapper.getPage(WikiPageLocalServiceWrapper.java:643)
	at sun.reflect.GeneratedMethodAccessor3921.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
	at $Proxy691.getPage(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor3921.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:115)
	at com.liferay.portal.spring.transaction.DefaultTransactionExecutor.execute(DefaultTransactionExecutor.java:62)
	at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:51)
	at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:111)
	at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:175)
	at $Proxy188.getPage(Unknown Source)
	at com.liferay.portlet.wiki.service.WikiPageLocalServiceUtil.getPage(WikiPageLocalServiceUtil.java:605)
	at com.liferay.portlet.wiki.util.WikiPageIndexer.addRelatedEntryFields(WikiPageIndexer.java:94)
	at com.liferay.portlet.messageboards.util.MBMessageIndexer.doGetDocument(MBMessageIndexer.java:271)
	at com.liferay.portal.kernel.search.BaseIndexer.getDocument(BaseIndexer.java:151)
	... 150 more

[Suresh Nimmakayala, módosítva 9 év-val korábban]

did you observed in lucene folders write.lock ?

[Vishal Patel, módosítva 9 év-val korábban]

Hi,

I was facing exactly same issue in one of my previous project. Later I found that some entry was deleted directly from Database Or some entry was directly created in Database. thats why Lucene was not able to get the entity data and It was throwing error.

So we dig into table data and clear it by removing such details properly and hence It was back to normal.

[Valéry Lambert, módosítva 9 év-val korábban]

Hi, did you fix this bug? I'm facing the same problem