Forums de discussion

Session replication in Tomcat

thumbnail
Ritabrata Mukherjee, modifié il y a 10 années.

Session replication in Tomcat

Junior Member Publications: 26 Date d'inscription: 03/03/09 Publications récentes
Hi All,
I am in the process of setting up a Tomcat server cluster with session replication, which will run Liferay 6.1 GA2. We have been following the below links for the same.

http://www.liferay.com/community/wiki/-/wiki/Main/Clustering
http://www.liferay.com/community/wiki/-/wiki/Main/High+Availability+Guide
https://www.liferay.com/web/ricardo.funke/blog/-/blogs/liferay-6-1-simple-cluster

I have some confusion on the setup, as below.

1. The first wiki link above actually recommends not to implement session replication, which I believe conflicts with the second wiki link.

2. We took a dump of session data in our server (Liferay 6.1) and observed that one of the larger objects, the “PortletSessionTracker” class, contains variables ‘_instance’ which is static and ‘_sessions’ which is transient. Now, if session replication is implemented, and if one server goes down, I believe that values for both of these variables will be lost. What impact does this have to end user? Can the end user seamlessly continue interacting in the next available server or will they have to login again ? (Reference: https://github.com/liferay/liferay-portal/blob/6.1.x/portal-service/src/com/liferay/portal/kernel/servlet/PortletSessionTracker.java)

3. In Liferay 6.2, “PortletSessionTracker” does not implement the Serializable interface and ‘_sessions’ variable is static. Similar to #1, what impact will this have when Tomcat session replication is turned on? (Reference: https://github.com/liferay/liferay-portal/blob/6.2.x/portal-service/src/com/liferay/portal/kernel/servlet/PortletSessionTracker.java)

Can someone please comment on the above, and also share experience with Liferay 6.1/6.2 on clustered app server with session replication ?
thumbnail
David H Nebinger, modifié il y a 10 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 14915 Date d'inscription: 02/09/06 Publications récentes
Replicating sessions in a cluster may or may not be necessary, depends upon your load balancing solution.

For example, you could use a "sticky" load balancer that redirects all traffic from a specific host to a specific instance in the cluster. In this setup, you don't need session replication because a user is always hitting the same node of the cluster. This setup actually works great because you don't have to worry about replicating the sessions around the cluster, but it does suffer if a node goes down the sessions on it go with it.

For a non-stickly load balancer setup, you do have to have session replication because you cannot guarantee that a system will hit the same node in the cluster every time. This ensures your users will not be impacted if a node in the cluster goes down, but it incurs extra overhead (network and cpu) to push the sessions around the cluster.

For #2 and #3, static and transient are normally not replicated anyway, even with replication enabled. That said, PortletSessionTracker is just used for tracking purposes so losing that data in the case of a node crash is not a big deal.

My recommendation - go w/ a sticky load balancer setup and disable the session replication, it will perform a lot better, unless of course you want the high availability aspects in which case the replication will be your only option.
Pritesh Shah, modifié il y a 10 années.

RE: Session replication in Tomcat

Junior Member Publications: 31 Date d'inscription: 05/07/12 Publications récentes
Agree with David's points above.
You can use sticky session on load balancer and load balancer will take care to directing user to the node where its session is created.
But if in case you need high availability of severs in case of crash/down server/node, implement session replication using in your cluster definition in server.xml using session managers as below,
Standard Manager
Persistent Manager
Delta Manager
Backup Manager

Refer below link for session replication in tomcat,
http://www.ramkitech.com/2012/11/tomcat-clustering-series-part-3-session.html

Thanks,
Pritesh
thumbnail
Ritabrata Mukherjee, modifié il y a 10 années.

RE: Session replication in Tomcat

Junior Member Publications: 26 Date d'inscription: 03/03/09 Publications récentes
Thanks David and Pritesh.
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Hey David,

I am, replying to this message because the topic is relevant and even though it was a few months back that you answered it, I figured best to "be green" and recycle emoticon.

I have a situation were we're using a combination of sticky session and session replication. Users will be bound to a server (hence the sticky) but we also want to replication for the HA reasons you mentioned. We have made the necessary "distributable" changes, and I have configured the tomcat instances to be clustered and use the DeltaManager for replication. I created a small test portlet and validated that all of this works. I can bounce between 2,3,4 ... n boxes and the session is replicated (my counter keeps increasing instead of starting over). Now, the problem.

When I do a controlled shutdown of one of the servers in the cluster the session is being expired. My tomcat cluster config instructs TOMCAT not to as you can see here --


<manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true" />


... and I have validated that this setting works in a Non-Liferay Tomcat Cluster environment. However, in a Liferay environment, a message is sent to the other nodes telling them to expire --


Oct 10, 2014 6:12:33 PM org.apache.catalina.ha.session.DeltaManager handleSESSION_EXPIRED
FINE: Manager [/session-replication-portlet]: received session [FA29B5DAAFA0AFFF6748672B71D7848D] expired.


.. that was the message I received on Node B when I shut down Node A. Given my tests, it seems that Liferay is triggering a session destroy action which the DeltManager must be considering "different" and not applicable for the expireSessionsOnShutdown. I tried modifying a couple of portal.properties (shown here) be removing ChannelHubAppShutdownAction and ChannelSessionDestroyAction -- but that hasn't helped.


    #
    # Application shutdown event that runs once for every web site instance of
    # the portal that shuts down.
    #
    application.shutdown.events=com.liferay.portal.events.AppShutdownAction[s],com.liferay.portal.events.ChannelHubAppShutdownAction[/s]


    #
    # Servlet session destroy event
    #
    servlet.session.destroy.events=com.liferay.portal.events.SessionDestroyAction[s],com.liferay.portal.events.ChannelSessionDestroyAction[/s]


I have also noticed that there are several listeners that are configured around Portal destruction, Session destruction, Session listeners etc. I'm a little stumped at this stage. I checked the GlobalShutdownAction and can see the unregistration of plugins, and perhaps they cause the scenario -- but I can't very well STOP that from happening emoticon

Wondering if you have come across this before or know a little more about this part of the system and can shed some light?
thumbnail
David H Nebinger, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 14915 Date d'inscription: 02/09/06 Publications récentes
Sorry, Andrew, but I don't know what's going on here. I did look for any sort of references in the source, but truly nothing jumped out that might be overriding the manager definition...

I know that's not the answer you were looking for, and I'm sorry about that.

Clearly the event is getting sent from the node that is shutting down, though. The question is whether we can identify what piece of code is actually sending the message, and actually disable that code instead.

Sitting here, though, I'm wondering if this might actually be something with Tomcat and not Liferay itself (i.e. perhaps Liferay's not doing anything to override the setting, maybe it's just a Tomcat bug).

Are you using the tomcat from the bundle, or have you tried the latest tomcat 7?
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Hey David,

I know! It's the oddest thing. I am using the Tomcat 7.0.42 Bundle with 6.2 CE (though on my client site we're working with EE SP6). I just turned on full debug for the portal on 3 nodes -- like looking into the matrix! At any rate, I trapped thousands of lines of output in hopes that something would jump out. The strangest thing though -- the line that shows the session kill message being sent is


FINE: Manager [/session-replication-portlet]: create session message [DD76A6DA89738D4D8F70E71608BAFA34] expire.


a. There is no time stamp which means its a console out I think.
b. I think I remember a few months ago when I was working on this (it was back burnered for a while) that I found that line in the Tomcat source while debugging.

... If I recall correctly, the logic in in the tomcat source looked correct. In fact, the default value for that property (if not specified) is actually FALSE -- I did try also letting the default carry through, same result. At the time I ran around placing blame on Tomcat, but when, like I mentioned above, tried a session replication test using the identical settings with a vanilla Tomcat (read: No liferay) then I was able to startup and shutdown nodes at whim without any sessions expiring (adhering to the property value).

Question -- when you initiate a TOMCAT_HOME/bin/shutdown.sh -- does that trigger an event that Liferay has a listener attached to? I think that might be the secret.

I'm tempted to disable all shutdown actions to see if that makes a difference next, but the problem is that they seem to come from several places. The MainServlet has a destroy() for example and there appear to be several listeners defined in the web.xml that I think might also participate. What I am not 100% sure of (just about 99% sure at this point) is that a shutdown signal to Tomcat triggers actions in Liferay.

Do you know if that is true? (bringing me to 100% emoticon )
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
I wonder is this might be my culprit --

In the Portals web.xml file there is a filter --


       <listener>
		<listener-class>com.liferay.portal.servlet.PortalSessionListener</listener-class>
	</listener>
	<listener>
		<listener-class>com.liferay.portal.kernel.servlet.PortletSessionListenerManager</listener-class>
	</listener>



In the latter of the two, if I go to the class I can see this method --


	@Override
	public void sessionDestroyed(HttpSessionEvent httpSessionEvent) {
		httpSessionEvent = getHttpSessionEvent(httpSessionEvent);

		HttpSession session = httpSessionEvent.getSession();

		PortletSessionTracker.invalidate(session.getId());

		for (HttpSessionListener httpSessionListener : _httpSessionListeners) {
			httpSessionListener.sessionDestroyed(httpSessionEvent);
		}
	}


the attribute on the delta manager doesn't say "Don't destroy sessions" it just says don't communicate them. I'm not sure if that is the same thing though -- basically who initiates the kill commnad? Lifreray or Tomcat.
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Hey David,

Just to close the loop on this one, I did find in the end that it is a tomcat bug, but a tricky one to catch. What I found was that when session replication occurs, the server that initiated the change gets promoted to the "primary". So if you have 3 servers, for example, and you modify something on Server A, then in the tomcat cluster your session on Server A becomes the primary and on Server B and C it is there as a non-primary.

If you shut down a server, any session that is not a primary will be retained on the other servers (no communication to purge the session occurs). BUT, if all primary sessions go through some additional code in Tomcat that has a hard coded value of TRUE for communicate to the cluster, rather than reading the cluster configuration setting.

So looks like a bug to me.
thumbnail
David H Nebinger, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 14915 Date d'inscription: 02/09/06 Publications récentes
7.0.56 is available, you might try using that version of tomcat w/ Liferay.
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
I might do that. Firs though I think I'll have a look at the source and make sure that issue is no longer present. I didn't find a bug entry for it in the tomcat bug tracker so it might not yet be reported.
thumbnail
Muhammed Shakir, modifié il y a 9 années.

RE: Session replication in Tomcat

Junior Member Publications: 36 Date d'inscription: 26/02/09 Publications récentes
Hi Andrew, Hi David,

Any updates on the final outcome of your efforts ?
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Hi Muhammed,

I did get this working, but I made a change to the TOMCAT source. I filed a bug with the tomcat group but they told me that I was not configuring it properly and suggested a change. I have not yet tried it, but I am planning to try it tomorrow. I'll update this thread tomorrow some time with the results of both my test, and the details on what I did to make it work (customizing the tomcat source.

Stay tuned.
thumbnail
Andew Jardine, modifié il y a 9 années.

RE: Session replication in Tomcat (Réponse)

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
For Muhammed -- and anyone else that comes across this.

For starters I am most certainly no Tomcat expert -- this is just what worked for me. I will say that I did my due diligence though and spent literally DAYS trying just about every combination of settings I could find from blogs, tutorials, and the tomcat docs itself. I thought I found a bug so I filed it with Tomcat, and shared my configuration. They said it was not a bug and told me I was missing a config in my cluster. Even after adding it though, it still did not work for me. All I can guess is that the tomcat dev wasn't validating with a portal and that perhaps there is something around using a portal that changes things. At any rate -- an explanation of my findings, my settings and what I did.

Updating the server.xml

1. Log into the server and open $TOMCAT_HOME\conf
2. Locate the <Host ...> node, which should be nested inside the <Engine ...> node. Paste the following into it.


<cluster classname="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelsendoptions="6">
<manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true" />
      <channel classname="org.apache.catalina.tribes.group.GroupChannel">
            <membership className="org.apache.catalina.tribes.membership.McastService" address="224.5.0.1" port="45564" frequency="500" dropTime="3000" />
           <receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto" port="4000" selectorTimeout="5000" maxThreads="25" />
           <sender classname="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                 <transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
           </sender>
           <interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
           <interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" />
           <interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor" />
</channel>
<clusterlistener className="org.apache.catalina.ha.session.ClusterSessionListener" />
</cluster>
 


3. Save and close the file.

Patched (Tomcat) Binaries

It is unclear whether or not this is a bug but it seems that there are several scenarios where the "expireSessionsOnShutdown" attribute that is set on the DeltaManager is not honoured. To remedy this situation a patched set of binaries is provided here. You need to use these binaries in order for the session replication to behave as required by Yellow Pages. Detailed explanation of the issue and the patch is provided after deployment instructions below.

Patch Deployment Instructions
This is what you need to do to create the patch -- the code to go into the patch is just below in the next section.
1. Checkout the tomcat source from SVN (http://svn.apache.org/repos/asf/tomcat/)
2. In the branches, pick the version you want to patch. I used 7_0_57 but checked a few releases, they all have the same code for this piece.
3. Open the file org.apache.catalina.ha.session.DeltaSession
4. Find the method with the signature public void expire(boolean notify)
5. Update the contents of the method to use --

/**
* Perform the internal processing required to invalidate this session,
* without triggering an exception if the session has already expired.
*
* @param notify
* Should we notify listeners about the demise of this session?
*/
@Override
public void expire(boolean notify) {
        boolean notifyCluster = true;
        if ( manager instanceof DeltaManager ) 
       {
               notifyCluster = ((DeltaManager)manager).isExpireSessionsOnShutdown();
               if ( log.isDebugEnabled())
              {
                      log.debug("\n");
                      log.debug("----[  Customization ]-------------------------------------------------------"); 
                      log.debug("\tProcessing SessionID: " + getIdInternal());
                      log.debug("\tDELTA MANAGER configured for Cluster.");
                      log.debug("\tChecking flag for expire sessions on shutdown..."); 
                      log.debug("\tCluster Notification is set to: " + notifyCluster); 
                      log.debug("----[  Customization ]-------------------------------------------------------");
                      log.debug("\n");
               }
        }
        expire( notify, notifyCluster );
}

6. Build the branchke a backup of the EXISTING TOMCAT_HOME/libs folder
7. Copy the built libs folder contents into your TOMCAT_HOME
8. Restart the server

Patch Explanation
The main issue with the out of the box Tomcat binaries is that a controlled shutdown of a server in the cluster does not honour the "expireSessionsOnShutdown" setting for any sessions on that node that are marked as primary. A close examination of the code shows that there are three classes of concern.

org.apache.catalina.session.StandardManager
org.apache.catalina.session.StandardSession
org.apache.catalina.ha.session.DeltaSession

The StandardManager class houses the stopInternal method that is invoked when you perform a controlled shutdown of the server. As part of the shutdown process it goes through all the sessions and initiates a call to session.expire() with a single boolean argument. Since we have configured the DeltaSession as our session implementation, the call is passed to that object (bypassing the StandardSession). The implementation details for DeltaSession.expire( boolean notify ) show that it makes a call to an overladed version of the method and that rather than checking for the cluster notification, it automatically passes TRUE as a value. This is what causes PRIMARY flagged sessions on the server to be expired across the cluster during shutdown.

The patch that has been put in place in a forked version of the code includes updating the DeltaSession.expire( boolean notify ) logic to the following:


/**
* Perform the internal processing required to invalidate this session,
* without triggering an exception if the session has already expired.
*
* @param notify
* Should we notify listeners about the demise of this session?
*/
@Override
public void expire(boolean notify) {
        boolean notifyCluster = true;
        if ( manager instanceof DeltaManager ) 
       {
               notifyCluster = ((DeltaManager)manager).isExpireSessionsOnShutdown();
               if ( log.isDebugEnabled())
              {
                      log.debug("\n");
                      log.debug("----[  Customization ]-------------------------------------------------------"); 
                      log.debug("\tProcessing SessionID: " + getIdInternal());
                      log.debug("\tDELTA MANAGER configured for Cluster.");
                      log.debug("\tChecking flag for expire sessions on shutdown..."); 
                      log.debug("\tCluster Notification is set to: " + notifyCluster); 
                      log.debug("----[  Customization ]-------------------------------------------------------");
                      log.debug("\n");
               }
        }
        expire( notify, notifyCluster );
}


This patch was applied to the TOMCAT_7_0_57 tag from the SVN repository that is used to store the tomcat source code. Once the patch was in place the source was built and the output/build/lib folder exported to the destination server.

As mentioned above, at this stage it is unclear whether or not this is a "bug" in Tomcat. Regardless, this method appears to have resolved the issue of replication session expire messages across the cluster in a controlled shutdown scenario.

Updating the context.xml

1. Open $TOMCAT_HOME\conf\context.xml
2. Change the <Context> node to include an attribute so it looks like this: <Context crossContext="true"> – this will allow PortletSession replication
3. Save the file.

Making the apps distributable

1. By Default, Liferay is not configured to be distributable. In addition to that, the plugins you create are also not distributable by default. In order to support the session replication, the web.xml file of each application needs to be updated. For example, to update the Liferay portal to be replication enabled --
a. Open $TOMCAT_HOME/webapps/ROOT/WEB-INF/web.xml
b. Immediately following the <web-app> tag, add the tag <distributable/>

Now, for each of the plugins that use the session and also need to be distributed you would follow the same process. For example, if you have a hello-works-portlet in the webapps directory --
c. Open $TOMCAT_HOME/webapps/hello-world-portlet/WEB-INF/web.xml
d. Immediately following the <web-app> tag, add the tag <distributable/>

2. Start / Restart the servers.

Validation (IPs will vary by configuration)

The easiest way to validate is to monitor the log at startup. First we increase the logging level for the Tribes classes (those responsible for the tomcat clustering and communciation). Open $TOMCAT_HOME/conf/logging.properties and add the following line.

org.apache.catalina.tribes.MESSAGES = FINE

Start the server and tail (tail -f $TOMCAT_HOME/logs/catalina.out) the log file.
If the cluster channels are created you should see this ---

INFO: Cluster is about to start
Jun 4, 2014 1:56:09 PM org.apache.catalina.tribes.transport.ReceiverBase bind
INFO: Receiver Server Socket bound to:/192.168.56.113:4000
Jun 4, 2014 1:56:09 PM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast soTimeout to 500
Jun 4, 2014 1:56:09 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4
Jun 4, 2014 1:56:10 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 56, 111}:4000,{192, 168, 56, 111},4000, alive=472623, securePort=-1, UDP Port=-1, id={-87 85 76 25 107 21 71 17 -92 -55 84 -120 124 16 -59 -40 }, payload={}, command={}, domain={}, ]
Jun 4, 2014 1:56:10 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 56, 112}:4000,{192, 168, 56, 112},4000, alive=233573, securePort=-1, UDP Port=-1, id={-86 72 108 -105 -45 46 70 127 -73 90 -42 18 -113 -66 -87 -43 },
payload={}, command={}, domain={}, ]
Jun 4, 2014 1:56:10 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:4
Jun 4, 2014 1:56:10 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:8
Jun 4, 2014 1:56:10 PM org.apache.catalina.tribes.io.BufferPool getBufferPool
INFO: Created a buffer pool with max size:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl
Jun 4, 2014 1:56:11 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:8
Jun 4, 2014 1:56:12 PM org.apache.catalina.ha.session.JvmRouteBinderValve startInternal
INFO: JvmRouteBinderValve started
4. When a server is already running and a new one joins the pool, the running server should show --
INFO: Created a buffer pool with max size:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl
Jun 4, 2014 1:52:17 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 56, 112}:4000,{192, 168, 56, 112},4000, alive=1017, securePort=-1, UDP Port=-1, id={-86 72 108 -105 -45 46 70 127 -73 90 -42 18 -113 -66 -87 -43 }, payload={}, command={}, domain={}, ]


5. When the session is shared the in both logs you should see something similar to the following --

Jun 4, 2014 1:52:33 PM org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor report
INFO: ThroughputInterceptor Report[
Tx Msg:1 messages
Sent:0.22 MB (total)
Sent:0.22 MB (application)
Time:0.09 seconds
Tx Speed:2.51 MB/sec (total)
TxSpeed:2.51 MB/sec (application)
Error Msg:0
Rx Msg:1 messages
Rx Speed:0.00 MB/sec (since 1st msg)
Received:0.00 MB]


The main difference will be that one will show Tx because it is sending the session information, and the other will show Rx. However, there is likely scenarios (multiple servers already running for example) where the send and receive labels may have values.

6. On occasion I have been seeing this error –

Jun 4, 2014 1:58:54 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberDisappeared
INFO: Received member disappeared:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 56, 113}:4000,{192, 168, 56, 113},4000, alive=165136, securePort=-1, UDP Port=-1, id={96 103 86 -108 6 -103 77 16 -110 34 49 -65 -27 51 -2 118 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]
Jun 4, 2014 1:59:20 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 56, 113}:4000,{192, 168, 56, 113},4000, alive=1026, securePort=-1, UDP Port=-1, id={108 124 -2 4 89 59 77 -40 -85 -86 105 56 67 70 68 -38 }, payload={}, command={}, domain={}, ]


Where it appears that a member has fallen out of the pool, but then is added back in. I have noticed around these events that sometimes the session replication (on the front end) testing isn't working as expected. For now though, no root cause has been found.

Conclusion
I am using this configuration now and have tested it with up to 5 Liferay VMs and it works. As I mentioned in the intro, everything else I tried would cause my sessions to be lost on occasion, and it always appeared to be when I shut down a node that had sessions flagged as "primary". With this in place, it doesn't matter which type of session is being shut down and the method is written such that the way it works without the patch will remain for any session that is not a DeltaSession.

Hope this helps.
thumbnail
Muhammed Shakir, modifié il y a 9 années.

RE: Session replication in Tomcat

Junior Member Publications: 36 Date d'inscription: 26/02/09 Publications récentes
Hi Andrew,

Thanks so much for spending time and effort to provide the aforementioned solution. I will implement what you have suggested. Thanks a tonne.
Ashish Goyal, modifié il y a 7 années.

RE: Session replication in Tomcat

New Member Envoyer: 1 Date d'inscription: 30/03/15 Publications récentes
Hi Andrew,

Much thanks for your post.
I have setup Liferay tomcat cluster and facing same issue in replicating the session.
In tomcat logs, it is showing that cluster has been setup but session is not getting replicated.

It is working intermittently. Sometimes session gets replicated and user is still logged in in case of controlled shutdown, sometimes not.

Could you please share your catalina-ha.jar after patch application ?
It would help a lot.

Thanks in advance!!
Harsh Kanakhara, modifié il y a 6 années.

RE: Session replication in Tomcat

Junior Member Publications: 74 Date d'inscription: 06/04/17 Publications récentes
Hi Ashish Goyal,

I am facing the same issue. Did you found the solution ?
Harsh Kanakhara, modifié il y a 6 années.

RE: Session replication in Tomcat

Junior Member Publications: 74 Date d'inscription: 06/04/17 Publications récentes
Hi Andrew Jardine,

Great explanation.

As I am new to Clustering and Load Balancing environment I have not much idea how it's working internally.

Can we implement sticky session on all the nodes and perform session replication only and only during node fail-over or controller shutdown ?

Regards,
Harsh Kanakhara.
thumbnail
David H Nebinger, modifié il y a 6 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 14915 Date d'inscription: 02/09/06 Publications récentes
Harsh Kanakhara:
Can we implement sticky session on all the nodes and perform session replication only and only during node fail-over or controller shutdown ?


No. Think of the worst case scenario - server is crashing (hardware fault, someone kill -9'd your tomcat, ...).

There is simply no opportunity there to say "please wait on crashing until I can replicate my session data to another node"...








Come meet me at the 2017 LSNA!
Harsh Kanakhara, modifié il y a 6 années.

RE: Session replication in Tomcat

Junior Member Publications: 74 Date d'inscription: 06/04/17 Publications récentes
Thanks David H Nebinger,

Thanks for your quick response. I got it that we cannot Replicate Session on sudden failure emoticon

But still, is there any chance to replicate session during manual shutdown by using shutdown hook or by firing linux scripts ? emoticon

Sorry for my stupid question as I have just started to exploring this area.

Regards,
Harsh Kanakhara.
thumbnail
Andrew Jardine, modifié il y a 6 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Nope -- and it wouldn't make sense. The whole point of clustering / replication, whether it is the session, Liferay cache, a database, a search server or whatever else you have i your infrastructure, the whole point is to maintain all nodes of your cluster in sync. Imagine you tried to trigger the sync during shut down, but then something goes wrong during the shutdown process. In that case your replication will fail.

Honestly, the best solution, specifically for session replication, is to avoid USING THE SESSION TO STORE DATA when you can. 95% of the time you don't need to use the session to store information. The problem is, storing details in the session is easy, and we developers are inherently lazy. For my own project work, the session for storage is always a last resort for me. I'll store things in the database, cache or user preferences before ever considering the session. Never say never but use the session as a last resort and you won't have to worry about replication in the first place.

Another alternative is to use an external server to store user sessions (like a Terracotta server) and then all your Liferay nodes will reference the same session object. In this case there is no need to do any replication at all (which can be an expensive and volatile process) since all servers will reference the same object. THe session not being stored locally on the server itself also means that you can shut down Tomcat (or whatever you are using) without the session being lost.
Harsh Kanakhara, modifié il y a 6 années.

RE: Session replication in Tomcat

Junior Member Publications: 74 Date d'inscription: 06/04/17 Publications récentes
Thanks Andrew Jardine for clearing my doubt. emoticon

Regards,
Harsh Kanakhara.
thumbnail
Giles Westwood, modifié il y a 6 années.

RE: Session replication in Tomcat

New Member Publications: 14 Date d'inscription: 25/07/16 Publications récentes
For what it's worth I've tested with tomcat 7.0.85 (no patching) and the sessions aren't expired.
thumbnail
Andrew Jardine, modifié il y a 6 années.

RE: Session replication in Tomcat

Liferay Legend Publications: 2416 Date d'inscription: 22/12/10 Publications récentes
Hi Giles,

Thanks for letting us know. I haven't tried it out yet with Liferay 7 (Tomcat 8) but I would assume then that it would work without patches there as well.