Foros de discusión

Severe memory leaks with JRuby

donino donino, modificado hace 11 años.

Severe memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Hi,

I use Liferay 6.1.1.GA2 with tomcat 7, 1Gb heap memory on a 32 Bits Sun JVM..

I encounter a critical memory leak in my context., after 2 or 3 days i run out of memory. Heap dumps show it is mainly due to JRuby. For example after only one day up:



And the relevant "dominator tree" shows JRuby is using more than 200Mb. This JRuby memory usage is growing and growing until an OutOfMemory :



I found this issue has already been reported in JIRA one year ago as /LPS-22464. It seems Liferay is only using JRuby to parse CSS files. I suspect it happens when pretty large css files are included in a liferay-portlet.xml descriptor. A workaround may be to include CSS in the theme rather in portlets, but in my context it is not so that simple.

Is there any option to disable JRuby in Liferay? Does anyone have further informations about this issue?

Thanks!
donino donino, modificado hace 11 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Hi,

It appears my workaround is not sufficient: JRuby,'s memory usage is rising much slowier, but it is still rising until an OutOfMemory. Please does someone know in which classes JRuby is used, so that i try to fix this? Is it only used to parse SASS files? Does it process css included in a theme, or only css included in a liferay-portlet.xml file? Is there a cache involved in this SASS parsing process, and a Liferay parameter to limit it?

Thanks
thumbnail
Riccardo Ferrari, modificado hace 11 años.

RE: Memory leaks with JRuby

Regular Member Mensajes: 139 Fecha de incorporación: 13/11/10 Mensajes recientes
Hi Donino,

Can you try to add the following option to your JVM arguments:
-Djruby.min.runtimes=1
-Djruby.max.runtimes=1

Hope it helps,
Riccardo
donino donino, modificado hace 11 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Riccardo Ferrari:
Hi Donino,

Can you try to add the following option to your JVM arguments:
-Djruby.min.runtimes=1
-Djruby.max.runtimes=1

Hope it helps,
Riccardo


Riccardo, thank you for your response, i have just restarted my server with the arguments you suggested. I will post if it improves the Jruby memory issue in my context.
thumbnail
Hitoshi Ozawa, modificado hace 11 años.

RE: Memory leaks with JRuby

Liferay Legend Mensajes: 7942 Fecha de incorporación: 24/03/10 Mensajes recientes
Liferay is using sass (http://sass-lang.com/) to process css. Sass is written in Ruby and Liferay is using jruby so it will run.
It appears if you have css in hooks, it'll use sass.

I can't say for sure without looking at your system, but it maybe a problem with your css. Check if you have any custom css and try removing it to see if memory leakage problem remains.
donino donino, modificado hace 11 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Hitoshi Ozawa:
Liferay is using sass (http://sass-lang.com/) to process css. Sass is written in Ruby and Liferay is using jruby so it will run.
It appears if you have css in hooks, it'll use sass.

I can't say for sure without looking at your system, but it maybe a problem with your css. Check if you have any custom css and try removing it to see if memory leakage problem remains.


Thank your for your response Hitoshi,

I removed all my custom CSS (mainly a jQuery themeroller) from liferay-portlet.xml descriptors, and included them in a JSP page instead. Thus they seem to be kept out of reach of Jruby. It is much better: previously it i lost about 100 Mb per day due to Jruby, now with a better frequentation i loose only 5Mb per day. So now i only need to restart Liferay once a month. Before restarting my server to apply the arguments suggested above by Riccardo , i analyzed a heap dump with Eclipse MAT, the leak seems entirely due to jRuby, though it should not process any custom CSS anymore. I even tried to remove all CSS files from the knowledge base portlet, but i still have this issue.

In a common Liferay server install, how much max memory Jruby is supposed to consume permanently? Do you currently use a 6.1+ CE version, without any memory issue?

Thanks
thumbnail
Hitoshi Ozawa, modificado hace 11 años.

RE: Memory leaks with JRuby

Liferay Legend Mensajes: 7942 Fecha de incorporación: 24/03/10 Mensajes recientes
JRuby 1.7.0 release Oct 22, 2012 seems to have fixed several memory leak problems.

Another option that is going to require more work is to replace sass so jruby won't be required.
vedri vedri, modificado hace 10 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 26 Fecha de incorporación: 11/02/06 Mensajes recientes
To remove jruby from version 6.1.1 with tomcat:

- remove jruby.jar from webapps/ROOT/WEB-INF/lib

- set scripting.jruby.compile.mode=off

- comment out in scripting-spring.xml:
<!-- entry key="ruby">
<bean class="com.liferay.portal.scripting.ruby.RubyExecutor" />
</entry -->


- from com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil remove any all ruby import
Replace
parsedContent = _parseSass(
request, themeDisplay, theme, cssRealPath, content);


to

if (cacheCssRealFile.exists()) {
     parsedContent = FileUtil.read(cacheCssRealFile);
} else {
_log.warn("This is implementation without jruby, please provide cache css file " + cacheCssRealFile);
}

Remove method _parseSass.


[indent]Please note: if you remove jruby.jar from liferay web application your theme build will not work if you use web application for sdk build configuration. Also last code replacement expects that there is css cache file already build in theme. After changing scripting-spring.xml there will be not available ruby scripting.[/indent]
donino donino, modificado hace 10 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Thank you for this workaround Vedran, it worked for me! I hope the Liferay team will take this jruby issue into consideration in the next release emoticon
thumbnail
Niall O'Grady, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 15/01/13 Mensajes recientes
Hello All,

We have been experiencing issues which seem closely related to those mentioned in this thread. I would be interested to know if other continue to experience JRuby memory usage issues.

Our environment includes:
Portal v 6.1.2
Tomcat 7
Windows OS
MS SQL back-end

Our issue is:
Severe production server instability. 2 ~ 3 day uptime before the memory usage bumps up against the 4gb heap ceiling.
A heap dump when the server is in this unusable state shows 20% ~ 30% JRuby memory usage

What have we tried:
1. Moved as much CSS to the theme as is possible at this time

2. Added the following to your JVM arguments per recommendation in this thread
-Djruby.min.runtimes=1
-Djruby.max.runtimes=1

3. JRuby upgrade to v1.7.4

4. I attended Berlin DevCon (what a fantastic event!) in recent days and lucky enough to get chatting to some of the Liferay escalations engineers who recommended that I check that the .saas-cache dir is created in the deployed WAR (it is) and that the files within that dir have an identical name and timestamp as those in the css dir (it is).

5. It was confirmed to me at DevCon that the Saas compilation should only run at build time. In our situation it clearly is not.

Any suggestions would be most welcome.

Niall O'Grady
donino donino, modificado hace 10 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 81 Fecha de incorporación: 2/07/11 Mensajes recientes
Hi,

I am not entirely sure yet, but with Liferay 6.2 it seems the problem is now solved. We only have 1.1Gb heap memory, and after 8 days without restart the memory consumption is stable around 53%, with a good frequentation! With Liferay 6.1 and jruby enabled, it was necessary to schedule a restart every 2 days to prevent OutOfMemory emoticon
thumbnail
Niall O'Grady, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 15/01/13 Mensajes recientes
Hi Donino,

Thanks for your comment.

I am pleased report that (with significant help from Liferay Support) we have managed to resolve the issue of JRuby presence in our heap.

The actions we took to resolve this were:
- Ensuring that the timestamp associated with the .sass-cache version of all CSS entries was exactly in sync (the the millisecond) with the non .sass-cache version of the file. Liferay Support provided us with a script to resolve any inconsistencies.

- We had some hooks with CSS overrides in them. We changed that

- The biggest difference we saw was when we deleted the \meta-inf\context.xml file from each plugin. At deploy time this file is generated to include two entries, none of which are necessary, one of which is potentially damaging it would seem. Once we cleared out these files there wasn't a trace of JRuby in the heap.

If anybody needs more information on the specifics on the steps we took, or who within Liferay support we dealt with, please let me know.

Thanks,
Niall O'Grady
Srilalitha Pusuluri, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 24/02/12 Mensajes recientes
Niall ,

I am using 6.2 EE . My custom theme is based on _unstyled theme. I am still getting PermGen Errors from Ruby. Any suggestions ?

java.lang.OutOfMemoryError: PermGen space
16:13:58,779 ERROR [MemoryQuartzSchedulerEngineInstance_Worker-4][ErrorLogger:120] Job (com.liferay.portal.deploy.messaging.RequiredPluginsMessageListener.com.liferay.portal.deploy.messaging.RequiredPluginsMessageListener threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: PermGen space]
at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.OutOfMemoryError: PermGen space
16:13:58,779 INFO [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][AutoDeployDir:204] Processing JHU_Basic_New-theme-6.2.10.1.war
16:13:45,174 ERROR [MemoryQuartzSchedulerEngineInstance_Worker-5][JobRunShell:120] Job com.liferay.calendar.messaging.CheckBookingsMessageListener.com.liferay.calendar.messaging.CheckBookingsMessageListener threw an unhandled Exception:
java.lang.OutOfMemoryError: PermGen space
16:14:12,486 ERROR [MemoryQuartzSchedulerEngineInstance_Worker-5][ErrorLogger:120] Job (com.liferay.calendar.messaging.CheckBookingsMessageListener.com.liferay.calendar.messaging.CheckBookingsMessageListener threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: PermGen space]
at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.OutOfMemoryError: PermGen space
16:14:10,564 ERROR [http-bio-8080-exec-133][DynamicCSSFilter:177] Unable to parse SASS on CSS /css/layout.css
com.liferay.portal.kernel.scripting.ScriptingException: java.util.concurrent.ExecutionException: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:254)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:129)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil._parseSass(DynamicCSSUtil.java:391)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil.parseSass(DynamicCSSUtil.java:161)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.getDynamicContent(DynamicCSSFilter.java:142)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.processFilter(DynamicCSSFilter.java:204)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1427.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.aggregate.AggregateFilter.processFilter(AggregateFilter.java:515)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1427.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.gzip.GZipFilter.processFilter(GZipFilter.java:123)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:165)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.ExecutionException: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:248)
... 68 more
Caused by: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:141)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1231)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1224)
at com.liferay.portal.scripting.ruby.RubyExecutor.doEval(RubyExecutor.java:189)
at com.liferay.portal.scripting.ruby.RubyExecutor$EvalCallable.call(RubyExecutor.java:332)
at com.liferay.portal.scripting.ruby.RubyExecutor$EvalCallable.call(RubyExecutor.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
... 1 more
Caused by: java.lang.OutOfMemoryError: PermGen space
16:14:10,563 ERROR [http-bio-8080-exec-138][DynamicCSSFilter:177] Unable to parse SASS on CSS /css/theme_custom.css
com.liferay.portal.kernel.scripting.ScriptingException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: PermGen space
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:254)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:129)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil._parseSass(DynamicCSSUtil.java:391)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil.parseSass(DynamicCSSUtil.java:161)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.getDynamicContent(DynamicCSSFilter.java:142)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.processFilter(DynamicCSSFilter.java:204)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1428.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.aggregate.AggregateFilter.processFilter(AggregateFilter.java:515)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1428.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.gzip.GZipFilter.processFilter(GZipFilter.java:123)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1428.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:165)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: PermGen space
at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:248)
... 67 more
Caused by: java.lang.OutOfMemoryError: PermGen space
16:13:58,780 ERROR [http-bio-8080-exec-141][DynamicCSSFilter:177] Unable to parse SASS on CSS /css/portlet.css
com.liferay.portal.kernel.scripting.ScriptingException: java.util.concurrent.ExecutionException: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:254)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:129)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil._parseSass(DynamicCSSUtil.java:391)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil.parseSass(DynamicCSSUtil.java:161)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.getDynamicContent(DynamicCSSFilter.java:142)
at com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSFilter.processFilter(DynamicCSSFilter.java:204)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1425.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.aggregate.AggregateFilter.processFilter(AggregateFilter.java:515)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.GeneratedMethodAccessor1425.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.BaseFilter.processFilter(BaseFilter.java:169)
at com.liferay.portal.servlet.filters.gzip.GZipFilter.processFilter(GZipFilter.java:123)
at com.liferay.portal.kernel.servlet.BaseFilter.doFilter(BaseFilter.java:59)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.liferay.portal.kernel.bean.ClassLoaderBeanHandler.invoke(ClassLoaderBeanHandler.java:67)
at $Proxy541.doFilter(Unknown Source)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDirectCallFilter(InvokerFilterChain.java:165)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:96)
at com.liferay.portal.kernel.servlet.PortalClassLoaderFilter.doFilter(PortalClassLoaderFilter.java:74)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.processDoFilter(InvokerFilterChain.java:204)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:109)
at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.ExecutionException: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at com.liferay.portal.scripting.ruby.RubyExecutor.eval(RubyExecutor.java:248)
... 68 more
Caused by: org.jruby.embed.EvalFailedException: java.lang.OutOfMemoryError: PermGen space
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:141)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1231)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1224)
at com.liferay.portal.scripting.ruby.RubyExecutor.doEval(RubyExecutor.java:189)
at com.liferay.portal.scripting.ruby.RubyExecutor$EvalCallable.call(RubyExecutor.java:332)
at com.liferay.portal.scripting.ruby.RubyExecutor$EvalCallable.call(RubyExecutor.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
... 1 more
Caused by: java.lang.OutOfMemoryError: PermGen space
thumbnail
Niall O'Grady, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 15/01/13 Mensajes recientes
Hello Srilalitha,

The issues we experienced with JRuby consuming all available memory may have been different to what you have reported in your stacktrace. I don't recall us having seen specific references to jruby on our server console, rather we identified JRuby as a leak suspect from performing a memory analysis of the heap after a heapdump was generated. As mentioned in my earlier comments, I recall these steps as having contributed to our issue being resolved:

- Ensuring that the timestamp associated with the .sass-cache version of all CSS entries was exactly in sync (the the millisecond) with the non .sass-cache version of the file. Liferay Support provided us with a script to resolve any inconsistencies.

- We had some hooks with CSS overrides in them. We changed that

- The biggest difference we saw was when we deleted the \meta-inf\context.xml file from each plugin. At deploy time this file is generated to include two entries, none of which are necessary, one of which is potentially damaging it would seem. Once we cleared out these files there wasn't a trace of JRuby in the heap.

If you have a heap dump or can generate one when the server is in a distressed state we always found that to be useful. A memory analyzer tool can then point you to what is chewing up all your memory.

Hope this helps.
Niall
Srilalitha Pusuluri, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 24/02/12 Mensajes recientes
Niall,

We did have the heap dump and examined in a memory leak analyser. Found that it is the Ruby engine that is has the memory leak. It was there in 6.1 too. Then we were waiting it to be fixed. Also from our side we did not use any SCSS to avoid ruby engine. But with the new themes I noticed that a lot of SCSS is being used in the basic themes such as _unstyled. when I build my theme these files are being inherited. a few example are :

_aui_custom.scss
_liferay_custom.scss
_aui_variables.scss
/aui/....scss files.

Thanks,
Srilalitha
Srinivas Konakanchi, modificado hace 10 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 33 Fecha de incorporación: 4/12/11 Mensajes recientes
We had this issue , when fast load is turned on and liferay merges css at runtime.
So, i simply moved generated main.css to cdn and updated top-head/?.jsp to refer this main.css . No more jruby errors.
Srilalitha Pusuluri, modificado hace 10 años.

RE: Memory leaks with JRuby

New Member Mensajes: 11 Fecha de incorporación: 24/02/12 Mensajes recientes
Srinivas ,
this might work in the local dev environment. But for production ? every time there is a css change after I deploy I need to manually move the main.css to css dir. this does not work for production environment.
Liferay need to fix the memory leak issue.

To reproduce the issue , create a theme based on one of life ray themes such as _unstyled . keep making minor changes and deploy the changes. do this a several times. you will perm gen errors.
Srinivas Konakanchi, modificado hace 10 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 33 Fecha de incorporación: 4/12/11 Mensajes recientes
If your custom css stays outside theme , this solution works. In this case main.css contains only liferay css.
good luck with the fix.
Virginia Calderón, modificado hace 8 años.

RE: Memory leaks with JRuby

New Member Mensajes: 7 Fecha de incorporación: 24/03/11 Mensajes recientes
Hello Niall,

¿Could you fix this problem in your portal?

We have changed these options but we haven’t found improvements:
  • scripting.jruby.compile.mode=off
  • Djruby.min.runtimes=1 -Djruby.max.runtimes=1
  • Update jruby.jar from 1.5.0_19 to JRuby 1.7.21


Thank you very much,
Kurt Xu, modificado hace 9 años.

RE: Memory leaks with JRuby

Junior Member Mensajes: 57 Fecha de incorporación: 2/02/13 Mensajes recientes
Vedran Pugar:
To remove jruby from version 6.1.1 with tomcat:

- remove jruby.jar from webapps/ROOT/WEB-INF/lib

- set scripting.jruby.compile.mode=off

- comment out in scripting-spring.xml:
<!-- entry key="ruby">
<bean class="com.liferay.portal.scripting.ruby.RubyExecutor" />
</entry -->


- from com.liferay.portal.servlet.filters.dynamiccss.DynamicCSSUtil remove any all ruby import
Replace
parsedContent = _parseSass(
request, themeDisplay, theme, cssRealPath, content);


to

if (cacheCssRealFile.exists()) {
     parsedContent = FileUtil.read(cacheCssRealFile);
} else {
_log.warn("This is implementation without jruby, please provide cache css file " + cacheCssRealFile);
}

Remove method _parseSass.


[indent]Please note: if you remove jruby.jar from liferay web application your theme build will not work if you use web application for sdk build configuration. Also last code replacement expects that there is css cache file already build in theme. After changing scripting-spring.xml there will be not available ruby scripting.[/indent]


Hello Vedran:
I did as you said, but some portlets wouldn't work, saying below error message:

Status
Internal Server Error

An error occurred while accessing the requested resource.

http://localhost:8080/en/web/guest/haier-index


Don't know if -Djruby.min.runtimes=1 and -Djruby.max.runtimes=1 can make the same effect as your method.

Thanks.
Kurt