« Back

Speeding up LDAP import process and Liferay authentication

General Blogs December 13, 2010 By Jonas Yuan

Liferay 5.2 EE and 6 improved the capabilities of LDAP integration in many areas (refer to blogs posts LDAP Enhancements and Keeping user password secure with LDAP integration):

  • synchronize user custom attributes between Liferay and LDAP
  • support LDAP chains and LDAP pagination
  • create a role for each LDAP group
  • override LDAP import and export processes via Spring
  • secure LDAP users' password

As you know, base DN is used as a base to search users and groups. When the number of users and groups is small, you would not meet any performance issue when searching users and groups. But if the number of users and groups is huge (like 500K users and 50K groups in LDAP), you would meet any performance issue when searching users and groups, since each user may be part of 50 groups.

Use case A (as shown in following screenshot): 500K users and 50K groups. Each user may be part of 50 groups. When logged in as a user from LDAP, it took 20-30 seconds by default. It should take less than 1 second.

The portal does introduce following property to search groups.

ldap.import.group.search.filter.enabled=true

As shown in above code, if set above property to true, the group filter will be applied, but only to groups in the specified base DN. If set to false, the filter will not be applied and all groups that are associated with the imported users will be imported regardless of the base DN.

The above workaround proposed would not be an option. Each user may belong to over 50 groups and it would just clutter the system with useless data if you were to import all groups for every user.

This article will address how to speed up LDAP import process and Liferay authentication.

Solution Overview

In two steps, you should be able to speed up LDAP import process and Liferay authentication.

1) Set up custom group base DN in portal-ext.properties

# Set this to true to enable custom group based DN settings.
# Set this to false to disable custom group based DN settings.
ldap.import.group.base.dn.enabled=true

# set up group base DN when the property ldap.import.group.basedn.enabled is set to true
# You can add group base DN against your LDAP here
ldap.import.group.base.dn.default=ou=groups,ou=system

2) Use custom group base DN in LDAP import process

Results

LDAP import process and Liferay authentication just took less than 1 second.

1) Set up base DN as users base DN like

2) Test LDAP users

3) Test LDAP groups

Is this feature useful? your comments / suggestions?

Summary

As you can see, LDAP import process and Liferay authentication can be improved a lot by using base DN as users base DN for users search, and custom group base DN as groups base DN for groups search. Ideally, we should divide base DN ldap.base.dn into ldap.users.base.dn and ldap.groups.base.dn for users search and group searchs, respectively. Refer to LPS-14322.

Last but not least, I'd like to send special thanks to Eduardo Carneiro and Jenny Chen who did a great job to narrow down the issue and to identify solutions.

Threaded Replies Author Date
Very nice feature, looking forward to test the... Jan Gregor December 14, 2010 1:29 AM
good,I like it! l b December 17, 2010 1:17 AM
@jan and @lb, thanks. Here I want to share test... Jonas Yuan December 20, 2010 2:32 PM
I'm trying ! l b December 21, 2010 7:03 AM
@lb, thanks. This feature is not available in... Jonas Yuan December 21, 2010 7:11 AM
useful info, thanks and I will look at this so... Jack Bakker January 1, 2011 12:39 PM
Hi Jack, Thanks. Here are fix patch testing: ... Jonas Yuan January 2, 2011 8:03 AM
Thanks Jonas Jack Bakker January 3, 2011 7:34 AM
Hi Jack, the following is testing result... Jonas Yuan January 5, 2011 7:40 AM
Hi, Is there some avaiable patch for 6.0.5... Henrique Simoes de Andrade February 7, 2011 11:56 AM
Hi Henrigue, thanks. Yes, it is available for... Jonas Yuan February 7, 2011 1:12 PM
Hi.. I am using Liferay 6.I am not able to... ankit yakkundi March 1, 2011 12:49 AM
HI Ankit, which version are you using? EE or CE? Jonas Yuan April 29, 2011 11:05 AM
Hi Jonas, I'm new with LDAP. I just... Danny Francisco June 15, 2011 3:39 PM
HI Danny, sorry to miss your post. Which... Jonas Yuan June 21, 2011 7:32 AM
Hi Jonas, Nice post,We are using Liferay... Vamsi Tekuru July 10, 2011 9:04 PM
Hi Vamsi, thanks. You may drop an email to... Jonas Yuan July 11, 2011 7:30 AM
Hi i Jonas, I have identified and fixed this... Vamsi Tekuru July 21, 2011 11:26 AM
Hi Jonas, I'm using 6.0.5 CE. Best regards, ... Danny Francisco September 8, 2011 2:28 AM

Very nice feature, looking forward to test the performance increase ! emoticon
Posted on 12/14/10 1:29 AM.
good,I like it!
Posted on 12/17/10 1:17 AM.
@jan and @lb, thanks. Here I want to share test results as follows.

"That’s great news! The patch also worked successfully. In this test, I was able to log-in very quickly and I see that only one group that matched the filter was imported. "
Posted on 12/20/10 2:32 PM in reply to l b.
I'm trying !
Posted on 12/21/10 7:03 AM in reply to Jonas X. Yuan.
@lb, thanks. This feature is not available in 6.0.5 CE and 6.0 EE and current alpha version 6.1 so far for now. But a fix patch is available for 6.0 EE already, and, especially, 6.0.5 CE on demand.
Posted on 12/21/10 7:11 AM in reply to l b.
useful info, thanks and I will look at this

so far for me for 6.0.5 for large LDAP imports where multiple subsystems authenticate against LDAP (LR only one such subsystem) ; I have been using "ldap.import.method=group", and involve group naming to delineate by subsystem ( I use LR_* group prefixes in LDAP ) where I only reference groups needed in group import filter

moving to AD from OpenLDAP for me has introduced new challenges ; in OpenLDAP, all was quite flat , now in AD ou hierarchies are more intense
Posted on 1/1/11 12:39 PM.
Hi Jack, Thanks.

Here are fix patch testing:

Following features got fixed.
a. LDAP user temporal password should be auto-generated.
b. Make ‘Import Enabled’ and ‘Import on Startup Enable’ working.
c. Super fast sign-in
d. No password imported

How to apply the patch?

You can apply the updated patch in following steps:

1) Shutdown the portal
2) add JAR lpe9000-ldap-ee6010-portal-impl-jdk5.jar.
3) restart the portal

make sure adding following lines at the end of portal-ext.properties

ldap.import.method=group

ldap.import.create.role.per.group=t­rue

ldap.import.user.password.enabled=false

ldap.import.user.password.autogenerate­d=true

ldap.import.user.password.default=test

ldap.import.group.base.dn.enabled=tr­ue

ldap.import.group.base.dn.default=ou=groups,ou=system

Note that you need to use your own LDAP group base DN.

Last but not least, check ‘Import Enabled’ and ‘Import on Startup Enable’ in LDAP UI.

Before testing, it would be better to start from scratch or remove exiting users, user groups, and roles in Liferay portal.
Posted on 1/2/11 8:03 AM in reply to Jack Bakker.
Thanks Jonas
Posted on 1/3/11 7:34 AM in reply to Jonas X. Yuan.
Hi Jack, the following is testing result (tested AD has more than 500K users and 50K groups) based on above settings plus patch:

"The new patch seems to work really well. Great Job! It imported the groups on startup and loaded my test users. The login performance is great and the password is being auto generated.

I’m still running a few tests but things are looking good."
Posted on 1/5/11 7:40 AM in reply to Jack Bakker.
Hi,
Is there some avaiable patch for 6.0.5 CE?

tks
Posted on 2/7/11 11:56 AM in reply to Jonas X. Yuan.
Hi Henrigue, thanks. Yes, it is available for 6.0.5 CE. You can drop me email: jonasliferay@gmail.com.
Posted on 2/7/11 1:12 PM in reply to Henrique Simoes de Andrade.
Hi..
I am using Liferay 6.I am not able to import the user.While testing LDAP groups,it shows me the groups,but when i try to see in usergroup its empty.

please help.
Posted on 3/1/11 12:49 AM in reply to Jonas X. Yuan.
HI Ankit, which version are you using? EE or CE?
Posted on 4/29/11 11:05 AM in reply to ankit yakkundi.
Hi Jonas,

I'm new with LDAP. I just successfully installed OpenLDAP in our server. I'm still trying to figure out the structure of my LDAP tree and this article really helped. where can I get lpe9000-ldap-ee6010-portal-impl-jdk5.jar patch? I think I will need it.
Posted on 6/15/11 3:39 PM in reply to Jonas Yuan.
HI Danny, sorry to miss your post.

Which version are you using? CE or EE?

Thanks

Jonas
Posted on 6/21/11 7:32 AM in reply to Danny Francisco.
Hi Jonas,

Nice post,We are using Liferay Portal Enterprise Edition 6.0 EE SP1 and we are facing a strange issue, LDAP import is failing for the usergroup having users greater tha 17k.import process is being aborted from then.No errors being written to the logs.
Posted on 7/10/11 9:04 PM in reply to Jonas Yuan.
Hi Vamsi, thanks.

You may drop an email to jonasliferay@gmail.com, where we could discuss more details.

Best regards,

Jonas
Posted on 7/11/11 7:30 AM in reply to Vamsi Tekuru.
Hi i Jonas,

I have identified and fixed this issue.Thanks
Posted on 7/21/11 11:26 AM in reply to Jonas Yuan.
Hi Jonas,

I'm using 6.0.5 CE.

Best regards,

Danny
Posted on 9/8/11 2:28 AM in reply to Vamsi Tekuru.