Vista Combinata Vista Piatta Vista ad Albero
Discussioni [ Precedente | Successivo ]
Jens Meinecke
Caching algorithm causes outofmemory
18 dicembre 2012 23.42
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Hi,

we are using Liferay 5.2 EE with Weblogic 12c and the brand new released Liferay Faces Bridge 3.1.1-ga2 . We investigate the caching algorithm, because our clustered server hangs up with an out of memory error. The memory dump contains large com.liferay.faces.bridge.scope.BridgeRequestScopeCacheImpl objects. The cache implementation extends the ConcurrentHashMap with the intention to limit the capacity. But the size of a ConcurrentHashMap is fixed to 2^30.

 1public class BridgeRequestScopeCacheImpl extends ConcurrentHashMap<String, BridgeRequestScope>
 2    implements BridgeRequestScopeCache {
 3
 4    // serialVersionUID
 5    private static final long serialVersionUID = 4546189667853367660L;
 6
 7    public BridgeRequestScopeCacheImpl(int maxSize) {
 8        super(maxSize, 1.0f);
 9    }
10}


The setting MAX_MANAGED_REQUEST_SCOPES has no effect, because you can only set the initial and not the maximal capacity of a ConcurrentHashMap.

We suggest to use something like a LRUMap, ConcurrentLinkedHashMap (e.g. http://code.google.com/p/concurrentlinkedhashmap). In fact apache myfaces brigde is using an LRUMap to handle this: BridgeImpl of Apache Myfaces Bridge

What do you think?
Atul Patel
RE: Caching algorithm causes outofmemory
19 dicembre 2012 5.53
Risposta

Atul Patel

Punteggio: New Member

Messaggi: 21

Data di Iscrizione: 12 gennaio 2012

Messaggi recenti

Interesting idea. Will discuss with Neil regarding pros/cons.
Neil Griffin
RE: Caching algorithm causes outofmemory
19 dicembre 2012 6.18
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Thanks so much for reporting this issue. I just discussed it with Atul and we have some questions for you:

1. Do you have the "com.liferay.faces.bridge.bridgeRequestScopePreserved" option enabled in web.xml? By default this is set to false, so that BridgeRequestScope instances are removed from the cache.

2. On average, how many simultaneous requests do you have coming in to each node in the cluster?

3. Since the BridgeRequestScopeCache is not based on the session, it is not a feature that will work in a failover situation in a cluster. Is failover something you require?

Neil
Jens Meinecke
RE: Caching algorithm causes outofmemory
20 dicembre 2012 9.25
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

1. No we haven't any such setting in the web.xml. So we are using the defaults.

2. I am sorry i don't know the exact cluster setup. The clustered environment runs about 20 hours until the memory of the machine is full. Meanwhile we can reproduce this error on a single node. Therefor we reduce the memory settings, start 30-40 Threads with 5 sec ramp up and endless loop. We place some jsf portlets on a portal page in our portal installation. After some time (maybe half an hour) you can see the out of memory exception.

3. Failover will be a requirement in the future but now it is not.

Any ideas for further investigations?
Neil Griffin
RE: Caching algorithm causes outofmemory
20 dicembre 2012 9.34
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Real quick -- I forgot that you mentioned Liferay 5.2 EE -- please use Liferay Faces version 3.0.1-legacy-ga2 which is designed for Liferay 5.2.x.

There was a memory leak bug in the GA1 version of Liferay Faces that could be the cause of your problem. (See FACES-1445 for more info).

But the bug is fixed in GA2, so I am very surprised that you are running out of memory. Are you sure that you don't have both GA1 and GA2 jars in the WEB-INF/lib folder of your deployed portlets?
Jens Meinecke
RE: Caching algorithm causes outofmemory
20 dicembre 2012 23.38
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Thank you for your quick reply.

Our team discovered the bug, so we know it well. But we use the latest 3.1.1 ga2 instead of the legacy version 3.0.1. And i am wondering why we missed that. Is it described somewhere to use the legacy jar with liferay 5.2? Anyway, we will change it and give you feedback.
Neil Griffin
RE: Caching algorithm causes outofmemory
21 dicembre 2012 5.28
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Complete descriptions for versions of Liferay Faces can be found here:
http://issues.liferay.com/browse/FACES#selectedTab=com.atlassian.jira.plugin.system.project%3Aversions-panel

We're going to update the download page soon too, as it currently does not mention that version.

Looking forward to getting your feedback on liferay-faces-3.0.1-legacy-ga2
Jens Meinecke
RE: Caching algorithm causes outofmemory
4 gennaio 2013 6.55
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Happy new year to you and all Liferay developer! :-)

We have tested the liferay-faces-3.0.1-legacy-ga2 and i am sorry to say that: The error still exists.

After another session with debugging and heap dumping we recognized that after several requests the BridgeRequestScopeCache caches BridgeRequestScopeLiferayImpl objects. If i have understood you correctly, you said that these objects should be removed at the end of a request.

To check that, i make heap dump (with jvisualvm) after a clean start of the app server und after a single request of a portal page with jsf portlets. The first dump contains no BridgeRequestScopeLiferayImpl objects (expected), but the second one does (not expected). In the second dump i can see the references to the BridgeRequestScopeLiferayImpl object that prevent it from being removed from heap. These references are internal references e.g. BridgeRequestScopeCompatImpl$FacesContextAttribute or BridgeRequestScopeImpl$RequestAttribute

I wondered why so i took a deeper look into the implementation:

In BridgeRequestScopeCompatImpl.saveJSF2FacesContextAttributes() you store all attributes from facesContext into a List. This list contains objects of BridgeRequestScopeCompatImpl$FacesContextAttribute and contain references to attributes of the facesContext. Because these objects are instances of a internal class they also hold a reference to its parent.

In BridgeRequestScopeCompatImpl.restoreJSF2FacesContextAttributes() you restore all attributes but the objects are not cleared explicitly. In my opinion this is the reason why the BridgeRequestScopeLiferayImpl objects cannot be removed from the heap.

An easy fix in BridgeRequestScopeCompatImpl.restoreJSF2FacesContextAttributes() should solve this:

 1
 2[...]
 3if (restoredFacesContextAttibutes) {
 4    savedFacesContextAttributes.clear();
 5    logger.debug("Restored FacesContext attributes");
 6}
 7else {
 8    logger.debug("Did not restore any FacesContext attributes");
 9}
10[...]


We tested this small fix successfully. Now no BridgeRequestScopeLiferayImpl objects stay in the heap after a request anymore. The cache is empty after a request.

So what do you think?
Neil Griffin
RE: Caching algorithm causes outofmemory
4 gennaio 2013 8.20
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Hi Jens,

Happy New Year to you too, and thanks so much for spending so much time debugging this issue! emoticon

When you get a chance, could you send a pull request and then click on the "Contribute Solution" button for the FACES-1463 issue?

Thanks again,

Neil
Jens Meinecke
RE: Caching algorithm causes outofmemory
5 gennaio 2013 7.23
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Done! Looking forward to getting the new version of the faces bridge. Meanwhile we use a patched version.
Jens Meinecke
RE: Caching algorithm causes outofmemory
7 gennaio 2013 5.03
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Atul Patel:
Interesting idea. Will discuss with Neil regarding pros/cons.


Have you discussed this yet? We think that the bridge becomes more robust if it uses a cache with a maximum capacity that is lower than 2^30. And this is one of the best pro we think. Do you plan to refactor the cache?
Atul Patel
RE: Caching algorithm causes outofmemory
7 gennaio 2013 8.02
Risposta

Atul Patel

Punteggio: New Member

Messaggi: 21

Data di Iscrizione: 12 gennaio 2012

Messaggi recenti

I'll bump this to Neil.... I know we talked about it over the holidays but I don't recall the resolution.

Neil?
Neil Griffin
RE: Caching algorithm causes outofmemory
7 gennaio 2013 11.08
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

For updates on how we plan on refactoring control of the cache size, please click on the "watch" link in JIRA for FACES-1464.
Jens Meinecke
RE: Caching algorithm causes outofmemory
8 gennaio 2013 22.42
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Thank you for your answer.
Neil Griffin
RE: Caching algorithm causes outofmemory
10 gennaio 2013 17.01
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Jens,

Regarding the pull request sent to FACES-1463, the code has been committed. When you get a chance, please try a SNAPSHOT release from the snapshot repository.

Regarding the caching algorithm in FACES-1464, we should be able to refactor that in the coming days/weeks. Please click on the "watch" link in JIRA for updates.

Thanks so much for the contribution! emoticon

Neil
Jens Meinecke
RE: Caching algorithm causes outofmemory
11 gennaio 2013 0.56
Risposta

Jens Meinecke

Punteggio: Junior Member

Messaggi: 43

Data di Iscrizione: 1 settembre 2008

Messaggi recenti

Thanks Neil! I enjoy contributing to the open software community. And I am watching these issues.
Do you have any plans regarding the new release?
Neil Griffin
RE: Caching algorithm causes outofmemory
14 gennaio 2013 12.59
Risposta

Neil Griffin

LIFERAY STAFF

Punteggio: Liferay Legend

Messaggi: 2107

Data di Iscrizione: 26 luglio 2005

Messaggi recenti

Hi Jens,

We're in a bugfix sprint right now, but will begin a development sprint for Liferay Faces in February.

Neil