Vista combinada Visión Plana Vista de árbol
Discusiones [ Anterior | Siguiente ]
toggle
Liferay 6.0.4 Document Library Slow Performance Ivan Fong 3 de agosto de 2010 0:32
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 3 de agosto de 2010 17:05
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 3 de agosto de 2010 22:47
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 4 de agosto de 2010 0:43
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 5 de agosto de 2010 0:00
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 5 de agosto de 2010 12:03
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 6 de agosto de 2010 0:06
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 6 de agosto de 2010 9:57
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 6 de agosto de 2010 10:19
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 6 de agosto de 2010 12:20
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 8 de agosto de 2010 9:12
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 8 de agosto de 2010 14:30
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 8 de agosto de 2010 20:50
RE: Liferay 6.0.4 Document Library Slow Performance Samuel Liu 9 de agosto de 2010 11:06
RE: Liferay 6.0.4 Document Library Slow Performance Ivan Fong 16 de agosto de 2010 23:06
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance Alan Schrock 16 de septiembre de 2010 7:23
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance Jack Bakker 19 de septiembre de 2010 9:14
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance Jack Bakker 19 de septiembre de 2010 9:48
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance Alan Schrock 19 de septiembre de 2010 11:16
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance Jack Bakker 20 de septiembre de 2010 11:12
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 11 de octubre de 2010 7:21
RE: Liferay 6.0.4 Document Library Slow Performance Matti Lindell 14 de octubre de 2010 2:12
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 26 de octubre de 2010 12:53
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 26 de octubre de 2010 12:58
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 27 de octubre de 2010 5:50
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 27 de octubre de 2010 8:51
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 28 de octubre de 2010 17:31
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 2 de noviembre de 2010 8:50
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 5 de noviembre de 2010 7:54
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 5 de noviembre de 2010 8:14
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 8 de noviembre de 2010 5:45
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 9 de noviembre de 2010 7:16
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 9 de noviembre de 2010 7:40
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 12 de enero de 2011 12:27
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 19 de enero de 2011 11:43
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 19 de enero de 2011 12:32
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 19 de enero de 2011 12:39
RE: Liferay 6.0.4 Document Library Slow Performance Alexey Kakunin 17 de febrero de 2011 4:52
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 17 de febrero de 2011 8:27
RE: Liferay 6.0.4 Document Library Slow Performance Alexey Kakunin 17 de febrero de 2011 9:09
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 17 de febrero de 2011 9:29
RE: Liferay 6.0.4 Document Library Slow Performance Alexander Chow 17 de febrero de 2011 9:31
RE: Liferay 6.0.4 Document Library Slow Performance Rafael Vega 5 de mayo de 2011 4:00
RE: Liferay 6.0.4 Document Library Slow Performance ilke Muhtaroglu 6 de junio de 2011 0:21
RE: Liferay 6.0.4 Document Library Slow Performance Srikanth Reddy Sanivarapu 6 de junio de 2011 0:24
RE: Liferay 6.0.4 Document Library Slow Performance ilke Muhtaroglu 6 de junio de 2011 23:18
RE: Liferay 6.0.4 Document Library Slow Performance Zenobia L 1 de julio de 2011 9:54
RE: Liferay 6.0.4 Document Library Slow Performance Teet Raid 26 de diciembre de 2011 12:48
RE: Liferay 6.0.4 Document Library Slow Performance Carlos Vicente 19 de enero de 2012 8:06
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 3 de noviembre de 2010 8:46
RE: Liferay 6.0.4 Document Library Slow Performance Alan Schrock 5 de noviembre de 2010 8:30
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 5 de noviembre de 2010 11:01
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 1 de noviembre de 2010 11:03
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 2 de noviembre de 2010 6:03
RE: Liferay 6.0.4 Document Library Slow Performance Ray Augé 2 de noviembre de 2010 8:43
RE: Liferay 6.0.4 Document Library Slow Performance JohnnyBrew . 3 de noviembre de 2010 6:53
Ivan Fong
Liferay 6.0.4 Document Library Slow Performance
3 de agosto de 2010 0:32
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

I have installed LR 6.0.4 on our server and found the following strange behavior in Document Library Portlet.

I have imported 1000 folders and 6600 files into the LR through WebDAV yesterday.

I have then re-index all the search index on the server.

It takes about 15 seconds to load when Organization Administrator click the Document Tab to load the Document Library Portlet.

But it takes almost 3 minutes to load when Organization Member do the same action.

During the page load, the mySQL consumes all the CPU power on the server.

Then, I enable the query log on mySQL server to log down all SQL statement and found that the number of SQL statements the Organization Administrator will do is A LOT less than around 30 compare to Organization Member of 1500 SQL statements.

I wonder is there any way to improve this performance issue??

Thank you in advance.

Ivan Fong
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
3 de agosto de 2010 17:05
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hi Ivan,

This certainly does sound like a curious issue. Do you know how previous Liferay versions compare at all, or are you only using Liferay 6.0.4?

If you need a doc library right now and this bug doesnt seem like it will be fixed soon, may I recommend that Liferay hooks into Documentum, Alfresco, and many other document Libraries emoticon

I'm going to look into this one some more. Thanks for the tip!

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
3 de agosto de 2010 22:47
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Dear Samuel:

Actually, I have not tried with such a large dataset in LR 5 before.

The truth is the total documents we target to put on the LR server will be around 160GB and more than 20000 files.

I have attached the mysql query log of the Organization Administrator and Member for your kind review.

I guess I will try to use LR5 and see if the same thing happens again before try to use Alfresco. It is because I am worry about whether the search portlet will search the Alfresco documents.

Please kindly update me if there is any new findings.

Thank you for your kind help.

Ivan Fong
archivos adjuntos: Admin-SQL-Queries.txt (790,5k), User-SQL-Queries.txt (1.412,2k)
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
4 de agosto de 2010 0:43
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Thanks Ivan!

Yeah I'm going to look into this tomorrow when I get into work. It would be terrible if document library wasn't capable of handling a large number of files. I'll load up a fairly large number and see if anything goes wrong. If nothing happens I'll keep adding till I get the same error.

I don't know too much about Alfresco, Documentum or the alternatives either -- I just know that we are built for integration with them. I'll ask about the searching bit and get back to you on that tomorrow.

Thanks for finding these bugs btw, they're really helpful!

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
5 de agosto de 2010 0:00
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Hi Samuel:

I have tried setup the LR 5.2.3 with similar setting of 6.0.4 including the users, organization, roles...etc.

I have upload similar files to it through WebDAV and their document library folder information are shown below respectively.

LR 5.2.3
Path: /opt/liferay-portal-5.2.3/data/document_library
Size: 6.52 GB
Files: 8926
Folders: 8847

LR 6.0.4
Path: /opt/liferay-portal-6.0.4/data/document_library
Size: 11.98 GB
Files: 7740
Folders: 7724

I have put the document library portlet on an individual page in the Organization's Private page.

I found that time it takes to load the page for both Organization Administrator and Organization User are REALLY fast, it takes less than 3 seconds and 4 seconds for the 5.2.3 to load the portlet respectively.

I have also upload the SQL log for you to do some analyze.
archivos adjuntos: AdminSQLQueries.log (115,7k), UserSQLQueries.log (1.018,3k)
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
5 de agosto de 2010 12:03
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hi Ivan,

Thank you for this detail. It makes my job so much easier! I'll run similar tests locally and file a bug report to get this fixed ASAP. Seems like a regression bug, and document library bugs are all serious.

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
6 de agosto de 2010 0:06
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Hi Samuel:

Base on your experience, is it possible for you to give me a time frame on when it will be fix? In terms of weeks or in terms of months does affect how and when we launch our intranet portal.

It is because I need to decide whether we should launch LR 5 or LR 6 to our intranet and update our board of directors next Monday.

Thank you for kind help.

Ivan Fong
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de agosto de 2010 9:57
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hi Ivan,

Sorry I do not know when this will be fixed. However, I can tell you that I am in the middle of reproducing it right now, so that I can tell people to fix it asap. I'm not sure who will work on it, how long it will take, or which nightly build it will come out in though.

If you launch LR5 on your intranet but decide to upgrade when the bug is fixed though, that might be optimal for you. I am hoping this is something that will come out in a nightly build in a matter of weeks, not months. Document Library is very important to Liferay so I will do my best to get this out the door for you fast.

Sorry for the inconvenience! emoticon

Warm regards,
Sam
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de agosto de 2010 10:19
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hi Ivan,

Internally the Document Library is being worked on actively as we speak. Turns out our most recent nightly build has a broken webDAV somehow emoticon

I'm going to contact the guy working on it to make him aware of this issue you've discovered.

Hopefully we can have this resolved quickly.

Thanks for your help!
Sam
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de agosto de 2010 12:20
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hi Ivan,

I might have a temporary workaround for you in 6:

So the developer working on WebDAV told me that the bug seems to have to do with document library's permissioning. Your fix would be to set Liferay to use permission algorithm 5. You can set this in portal-ext.properties {../tomcat-6.0.26/webapps/ROOT/WEB-INF/classes/portal-ext.properties}

1permissions.user.check.algorithm=5


Let me know how it goes if you try emoticon

Webdav is working again in the latest nightly build and hopefully 6.0.5 will get released today or within a few days.

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
8 de agosto de 2010 9:12
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Hi Samuel:

Thank you very much for your replay.

I have tried it and now it takes about 15 seconds to open the document library.

It is a GREAT improvements but I think LR 6 can do a lot better than this when I compare to 5.2.3.

I will upload more files and folder to test its performance tomorrow.

Thank you once again and I will keep you posted.

Ivan Fong
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
8 de agosto de 2010 14:30
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Thanks so much Ivan for your help emoticon Our guys are working on it now, keep us posted!

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
8 de agosto de 2010 20:50
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Hi Samuel:

I have found that after I implemented the workaround, the load time of the document library portlet is a lot faster but it introduce a new problem as well.

After I use the workaround, I found that all organization member cannot access the folder in document library except the organization administrator.

So I have reverted the workaround now.

For the time being, I think I will propose our team to use 5.2.3 until 6 is more stable and usable in our case.

Thank you for your kind help.

Ivan fong
Samuel Liu
RE: Liferay 6.0.4 Document Library Slow Performance
9 de agosto de 2010 11:06
Respuesta

Samuel Liu

LIFERAY STAFF

Ranking: Expert

Mensajes: 301

Fecha de incorporación: 27 de mayo de 2010

Mensajes recientes

Hey Ivan,

We will work hard to get 6.0's doclib working right emoticon

Thanks for all your help and support.

Warm regards,
Sam
Ivan Fong
RE: Liferay 6.0.4 Document Library Slow Performance
16 de agosto de 2010 23:06
Respuesta

Ivan Fong

Ranking: Junior Member

Mensajes: 35

Fecha de incorporación: 11 de junio de 2010

Mensajes recientes

Hi Samuel:

May I know whether the document library is working properly in 6.0.5?

Thank you.

Ivan Fong
Alan Schrock
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance
16 de septiembre de 2010 7:23
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

Are there any updates to this? I have a similar situation from what Ivan is talking about. I have been using Liferay 5.2.3 successfully with around 1300 files, 800 folders, 180 MB. It works fairly quickly in Liferay 5.2.3 but is very slow on initial load for non-admins still on 6.0.5 with the nightly build from September 3rd. Using mysql as well.

We utilize this also fairly extensively and switched over to Liferay 6 already for production.

NOTE: it also doesn't matter whether it is the full document library on the page or the document library display. Any page with a large document library repository/document library display of that repository takes at least 20-40 seconds to initially load.

Please give an update.

Thanks!

Alan
Jack Bakker
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance
19 de septiembre de 2010 9:14
Respuesta

Jack Bakker

Ranking: Expert

Mensajes: 457

Fecha de incorporación: 3 de enero de 2010

Mensajes recientes

webdav is painfully slow in 6.0.5 compared to 5.2.3

I also noticed dlfileversion title is only 75 characters wide ; so many files I try to transfer over to new version don't get there till I widen that field (using postgresql)
Jack Bakker
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance
19 de septiembre de 2010 9:48
Respuesta

Jack Bakker

Ranking: Expert

Mensajes: 457

Fecha de incorporación: 3 de enero de 2010

Mensajes recientes

here is why: http://issues.liferay.com/browse/LPS-12238
Alan Schrock
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance
19 de septiembre de 2010 11:16
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

That would do it!

Also, the nightly build from the 16th of September seems to have sped up the access significantly so they must have made this change.

Not sure that will help your webdav issue, but I also find webdav slow with the image gallery so that may be an overall issue with webdav and liferay still...
Jack Bakker
RE: Liferay 6.0.4/6.0.5 Document Library Slow Performance
20 de septiembre de 2010 11:12
Respuesta

Jack Bakker

Ranking: Expert

Mensajes: 457

Fecha de incorporación: 3 de enero de 2010

Mensajes recientes

thanks for the note on the nightly option

my postgresql cpu allocation is hitting the ceiling on webdav imports
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
11 de octubre de 2010 7:21
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

What is the current status of the document library for 6.0.x? I have just applied the nightly build from 10/10/10 and though, as my other post reports, the speed of the access is better, it is still really too slow for use with larger numbers of files. My overall set of files is large, but each folder isn't large.

I see that the issue refered to has been closed so I am not sure if there is anyone working on this anymore or not.

So, if a fix isn't coming soon, is there a way to backrev to the 5.2.3 version of the document library? If so, can this be done without destroying my existing documents or will I have to do a full copy in again?

Thanks

Alan
Matti Lindell
RE: Liferay 6.0.4 Document Library Slow Performance
14 de octubre de 2010 2:12
Respuesta

Matti Lindell

Ranking: New Member

Mensajes: 1

Fecha de incorporación: 14 de octubre de 2010

Mensajes recientes

I'd also like to know the status of this. Even if the DocumentLibrary UI is fixed, WebDAV suffers from same/similar permission checks which start from DLFolderServiceImpl.getFolders

When the scenario is the following:

-PERMISSIONS_USER_CHECK_ALGORITHM=6
-Large DocumentLibrary content (> 1000 folders and with files)
-WebDAV -> Document library import

importing a file to DL, inline permission checks create very massive DB queries (posgresql) and cause big performance hit.

Can someone verify is this is caused by the new permission check algorithm?
Has the new algorithm been extensively tested with large (as in production) contens?


I took a thread dump when the server was under heavy load and the culprit is the permission checking done in DLFolderPersistenceImpl.filterFindByG_P:

java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:129)
org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:135)
org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:104)
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)
org.postgresql.core.PGStream.ReceiveChar(PGStream.java:259)
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1620)
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)
org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
org.hibernate.loader.Loader.getResultSet(Loader.java:1849)
org.hibernate.loader.Loader.doQuery(Loader.java:718)
org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
org.hibernate.loader.Loader.doList(Loader.java:2294)
org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2172)
org.hibernate.loader.Loader.list(Loader.java:2167)
org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:316)
org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1832)
org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165)
org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:179)
com.liferay.portal.dao.orm.hibernate.SQLQueryImpl.list(SQLQueryImpl.java:86)
com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:60)
com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:52)
com.liferay.portlet.documentlibrary.service.persistence.DLFolderPersistenceImpl.filterFindByG_P(DLFolderPersistenceImpl.java:2317)
com.liferay.portlet.documentlibrary.service.persistence.DLFolderPersistenceImpl.filterFindByG_P(DLFolderPersistenceImpl.java:2227)
com.liferay.portlet.documentlibrary.service.impl.DLFolderServiceImpl.getFolders(DLFolderServiceImpl.java:208)
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
26 de octubre de 2010 12:53
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

You guys feel like testing the following patch?

It's against trunk, but it shouldn't be hard to adapt for any version (at least for 6.0+).

Basically this uses the index for listing files instead the DB.

Note that I do think that while trying this I think I uncovered a bug with the re-indexing logic. When you use this, the top level files disappear after a re-index. But, adding more files or re-adding any missing files results in showing all the files again. Might be something with a cache somewhere even.

I tried solving the slowness of the import, but I haven't spotted specific issue yet. I did notice a small speedup when using the NIO file transfer setting.

-Dcom.liferay.portal.kernel.util.StreamUtil.use.nio=true ## default=false

There is also

-Dcom.liferay.portal.kernel.util.StreamUtil.buffer.size=8192 ## default=8192

As for permissions during import, I'm not sure how this could affect the import but I'll try looking into it.
archivos adjuntos: WEBDAV-TEST.patch (3,2k)
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
26 de octubre de 2010 12:58
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

One more thing, my test was to set:

dl.file.max.size=30720000

dl.file.extensions=*

Then take workspace/portal-trunk (eclipse working copy) and drop that on the Guest webdav root. That's 80k+ files, totaling ~1.6 GB.
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
27 de octubre de 2010 5:50
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

I will let the other guys who are dealing with the WEBDAV issues more consistently try this out.

You mentioned the indexing issue. Another issue I have found is file/folder counts. I believe these counts started showing incorrectly once the nightly build came out to help speed up the library some. The files are there, but at the top level it shows only a few files in the count. There are more than 2000 files in there and as you drill down, they are actually there, just not counted correctly. I did add most of these files initially via WEBDAV, so it may be something with that and the indexing you are talking about. Doing a reindex of the database doesn't matter. I am also using the file system, not database for storage of the files (can't remember if either is an option for doc library or if that is only an image gallary thing).

Library is still very slow for general users. It seems every role but administrator it is slow for (even community owners). Doesn't matter if I give portal permissions to Doclibrary either - all rights, it is still slow. I haven't tried the latest nightly builds for a week or so to see if anything has changed. It is to the point where I am thinking about going back to 5.2.3 or to another platform.

Thanks for your work on this.

Alan
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
27 de octubre de 2010 8:51
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

The solution to solving the slowdown in later builds was to only show the count for files and folders one level bellow a give folder.

The query to calculate the viewable total counts for each folder in a page was deemed too costly (since it was cascading down the entire tree, which was resulting in the huge queries mentioned earlier).

So, you have a tradeoff:

- enabled permission checking so that you hide things the user can't see and show less information (gonna be a little slower since you can't do as much caching)
- disable permission checking which will revert back to where you show everything to everyone, and only give errors on access (gonna be fast since you're caching straight up queries since everyone is the same)

Since we had so many complains about the old way of doing it (no permission checking/give error on access) that we went with the permission + less detail option.

The current implementation shouldn't be to slow (only slightly slower than the old way, but the benefits should outweigh the slowdown.
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
28 de octubre de 2010 17:31
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

Thanks. I appreciate the difficulty of trying to address many competing desires and appreciate all of your work.

With the latest build, I am still getting between 20 and 40 seconds for a subfolder to appear. My one folder has 787 files, 657 folder, when I go down to one of the folders within this (that has 185 files/137 folders) it takes 40 seconds between the time I click on the link and the library displays. This is similar to the other folders. All together, I have 1401 files, 823 subfolders. This is logged in as a general user, not admin. Much faster as admin. I am running last night's build (10/28/10) with mysql, tomcat 5.

Also, to add to the computation issues and wish lists, it would really be nice to have folder inheritance as well. Right now, the permission's tag goes on the file and it gets permissions from the folder it is put in. If that folder changes, though, it does affect it's permissions. It is really nice to be able to add roles to an upper file and have the lower files accessible. Even if there would be a routine to propogate the persmissions down, that would be a start. Just a thought.

Thanks for all you do.

Alan
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
1 de noviembre de 2010 11:03
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Any Update on how to address this issue without having to use the LR5 permission? Thanks in advance.
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
2 de noviembre de 2010 6:03
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Ray Augé:
...
- enabled permission checking so that you hide things the user can't see and show less information (gonna be a little slower since you can't do as much caching)
- disable permission checking which will revert back to where you show everything to everyone, and only give errors on access (gonna be fast since you're caching straight up queries since everyone is the same)


Ray - where do we disable permission checking? Is this something we can manage in the portal ext properties file? Any help would be greatly appreciated. Thanks!
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
2 de noviembre de 2010 8:43
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

Sigh!!! My vague explanations strike again!

I didn't mean "disable" as in "a configuration option". I should not have used the word "disable" at all. In it's place I should have written "not program it to use inline permission checking".

My apologies for sending you down a road that don't exist.
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
2 de noviembre de 2010 8:50
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

Ok Alan,

Let's try something slightly different.

Open up /html/portlet/document_library/folder_columns.jspf

Comment out line 20:

//DLFolderServiceUtil.getSubfolderIds(subfolderIds, scopeGroupId, curFolder.getFolderId(), false);

And one line 27 set the value to 0:

int curFileEntriesCount = 0;//DLFolderServiceUtil.getFileEntriesAndFileShortcutsCount(scopeGroupId, subfolderIds, status);

And then see how the performance changes.

Maybe we still need to look at some alternatives here for the statistics.
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
3 de noviembre de 2010 6:53
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Ray Augé:
In it's place I should have written "not program it to use inline permission checking".


Ray - thank you for the clarification. I am on the business side and am not very technical. So, I don't exactly understand what you mean by "not program it to use inline permission checking". I shared your response with my developer and he also doesn't understand. Is your suggestion something that can be done at the portlet level? I ask because we have tried changing the permission structure for the Portal (e.g., use the permissions for LR5) and while it fixed the responsiveness of the Doc Library, it had huge negative effects on other aspects of our implementation. Thus we rolled back to the original permission structure. If we could remove the permission checking for just the Doc Library as a work-around, it would suffice for the time being.

Thanks for all of your help. We eagerly look forward to your response.

Cheers, Johnny Z.
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
3 de noviembre de 2010 8:46
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Alan Schrock:
Thanks. I appreciate the difficulty of trying to address many competing desires and appreciate all of your work.

With the latest build, I am still getting between 20 and 40 seconds for a subfolder to appear. My one folder has 787 files, 657 folder, when I go down to one of the folders within this (that has 185 files/137 folders) it takes 40 seconds between the time I click on the link and the library displays. This is similar to the other folders. All together, I have 1401 files, 823 subfolders. This is logged in as a general user, not admin. Much faster as admin. I am running last night's build (10/28/10) with mysql, tomcat 5.

Also, to add to the computation issues and wish lists, it would really be nice to have folder inheritance as well. Right now, the permission's tag goes on the file and it gets permissions from the folder it is put in. If that folder changes, though, it does affect it's permissions. It is really nice to be able to add roles to an upper file and have the lower files accessible. Even if there would be a routine to propogate the persmissions down, that would be a start. Just a thought.

Thanks for all you do.

Alan


Hi, Alan,

I thought I would check with you to see if you had any success in implementing any of the suggestions or if you had success with finding a work-around/resolution on your own?

Any help/suggestions would be greatly appreciated.

Regards,

JZ
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
5 de noviembre de 2010 7:54
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

I implemented this the other night, but had too many other processes going on my machine yesterday to give it a good test. I just completed this.

For smaller folders, this seems to be doing better. We are down to maybe 10-15 seconds to display. Still slow, overall, but better.

For my larger, more top-level folders, I am still seeing between 20 and 30 seconds before the folder displays. That is still a bit long in web response.

Thanks

Alan
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
5 de noviembre de 2010 8:14
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

I have on last ditch suggestion for you to try. (this is just for evaluation purposes, not a final solution).

Inside /html/portlet/document_library, find all references to DLFolderServiceUtil (in the jsps and jspfs only, there should be ~19) and rename them to DLFolderLocalServiceUtil (except the import in init.jsp, leave that one).

This will effectively eliminate all inline permission checking.

If you test that and see significant improvement, then we may have some work to do yet on inline permissions on large data sets.

Please let us know.
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
5 de noviembre de 2010 8:30
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

Johnny,

I just responded to the one test so you can see the results from that. Still slow, but a little bit better. For the webdav one, I haven't done any testing with that it.

I was curious on my response times with 5.2.3 so I fired up my old version. Nearly an identical repository yieled approx. 3 seconds between click and folder display. Obviously, changes have happened which were needed to get some better permission checking and this is the pain of the process of change. I do like, though the 3 better than 20! :-)

And, yes, as you mentioned, disabling permission checking does it portal-wide and that isn't an option.

Just got an email with another test from Ray, I'll try that and post a reply.

Alan
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
5 de noviembre de 2010 11:01
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Thanks, Ray. And thanks Alan too! We will try and let you know what we find. Cheers, Johnny
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
8 de noviembre de 2010 5:45
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

Ray,

I just tried this on my test system and it makes the document library work fast like in 5.2.3.

You say this isn't a permanent solution, just a test. What would be the negatives if I put this on my production server?

Thanks!

Alan
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
9 de noviembre de 2010 7:16
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

The only drawback is that you loose the features added in 6.0+ which was to have filtered content appear to users.

What you'll see now is what we had in <=5.2 which was to show everything, and just throw an error when someone tried to access something they didn't have permission to see.

There are no other drawbacks.
Alan Schrock
RE: Liferay 6.0.4 Document Library Slow Performance
9 de noviembre de 2010 7:40
Respuesta

Alan Schrock

Ranking: New Member

Mensajes: 9

Fecha de incorporación: 16 de septiembre de 2010

Mensajes recientes

That is clear.

Thanks Ray.

Alan
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
12 de enero de 2011 12:27
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Ray Augé:
I have on last ditch suggestion for you to try. (this is just for evaluation purposes, not a final solution).

Inside /html/portlet/document_library, find all references to DLFolderServiceUtil (in the jsps and jspfs only, there should be ~19) and rename them to DLFolderLocalServiceUtil (except the import in init.jsp, leave that one).

This will effectively eliminate all inline permission checking.

If you test that and see significant improvement, then we may have some work to do yet on inline permissions on large data sets.

Please let us know.


Hi, Ray,

Checking in to see if a permanent solution has been found or released? Please let me know. Thanks in advance and Happy New Year.

Regards,

John
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
19 de enero de 2011 11:43
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Hi, Ray - any help on a permanent solution? We would really like to know. Thanks.




Ray Augé:
I have on last ditch suggestion for you to try. (this is just for evaluation purposes, not a final solution).

Inside /html/portlet/document_library, find all references to DLFolderServiceUtil (in the jsps and jspfs only, there should be ~19) and rename them to DLFolderLocalServiceUtil (except the import in init.jsp, leave that one).

This will effectively eliminate all inline permission checking.

If you test that and see significant improvement, then we may have some work to do yet on inline permissions on large data sets.

Please let us know.
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
19 de enero de 2011 12:32
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

There is currently a patch that is being evaluated.

I'll try to get that done as soon as possible.

It does follow the general principle that the statistics could be off slightly whereby it would not account for permissions, that way it could use a cached query (as it did in 5.2). Thus the counts would be off. But for the actual listing of paged entries, that would always take permissions into account.

So this is what is being looked at.

Sorry for the delay.
JohnnyBrew .
RE: Liferay 6.0.4 Document Library Slow Performance
19 de enero de 2011 12:39
Respuesta

JohnnyBrew .

Ranking: New Member

Mensajes: 19

Fecha de incorporación: 10 de julio de 2009

Mensajes recientes

Thank you for the update, Ray. Please post a note here when the patch has passed QA and is ready for release.

Thanks again,

Johnny Z.
Alexey Kakunin
RE: Liferay 6.0.4 Document Library Slow Performance
17 de febrero de 2011 4:52
Respuesta

Alexey Kakunin

Ranking: Expert

Mensajes: 370

Fecha de incorporación: 7 de julio de 2008

Mensajes recientes

I've tested with Liferay 6.0.5 - disabling counting of subfolders and subfiles count helped to improve performance.

But it is better simple check folderColumns in folder_columns.jspf - does it need to calculate counts or not (since folderColumns may be configured):

 1
 2int curFoldersCount = 0;
 3int curFileEntriesCount = 0;
 4
 5if (ArrayUtil.contains(folderColumns, "num-of-folders") || ArrayUtil.contains(folderColumns, "num-of-documents")) {
 6    List subfolderIds = new ArrayList();
 7   
 8    subfolderIds.add(new Long(curFolder.getFolderId()));
 9   
10    DLFolderServiceUtil.getSubfolderIds(subfolderIds, scopeGroupId, curFolder.getFolderId());
11   
12    curFoldersCount = subfolderIds.size() - 1;
13    curFileEntriesCount = DLFolderServiceUtil.getFileEntriesAndFileShortcutsCount(scopeGroupId, subfolderIds, status);
14}



==
Alexey Kakunin
EmForge: Liferay Based Project Hosting Service
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
17 de febrero de 2011 8:27
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

I've pretty much resolved myself to the fact that statistics should be using the cached non-permissioned result and therefore should be as fast as it was in previous versions. (i.e. only the actual paged query itself should be permissioned)

All you need to do is replace the ServiceUtil with the LocalServiceUtil calls where the use is for collection of stats (num-of-folders, num-of-documents).

You should also change the call getSubfolderIds on search to the LocalService. I've actually found that having it as it is, is in fact duplicating some permission checking to begin with so changing to the LocalService is both faster due to cache, as well eliminating duplication of a check which is done later anyway.

I'm going to commit these changes soon (as I get the chance) and I'm sure that the outcry is worthy for this to reach back to next 6.0 and 5.2 EE SPs.
Alexey Kakunin
RE: Liferay 6.0.4 Document Library Slow Performance
17 de febrero de 2011 9:09
Respuesta

Alexey Kakunin

Ranking: Expert

Mensajes: 370

Fecha de incorporación: 7 de julio de 2008

Mensajes recientes

Not sure - since it may confuse users: someone will see that folder has subfolders and files (in statistics) - but then will click into folder - will not see them (because has no permission to see them).
So, next question will be: "THere are the files????"

So, I'd like the fact, what Liferay used remote-services to get this statistic and it returned values taking into account current user permissions.
Of course - performance is a problem - we met problem even with 50 files and folders!

So, disabling displaying the statitics was a solution for us - it is better to do not display any statistic even displayb incorrect. Especially in case it is possible to avoid statistics columns from the portlet configuration - only problem was (in Liferay 6.0.5) - even statistics columns was avoided - statistics calculation was called - and my fix is only for this.
Ray Augé
RE: Liferay 6.0.4 Document Library Slow Performance
17 de febrero de 2011 9:29
Respuesta

Ray Augé

LIFERAY STAFF

Ranking: Liferay Legend

Mensajes: 1233

Fecha de incorporación: 7 de febrero de 2005

Mensajes recientes

That sounds very reasonable.

Perhaps what we could do is:

1) make sure that disabling the columns disables the calculation of stats
2) offer 2 columns that can be used in the settings
a) stats with permission checks
b) stats without permission checks

Thoughts?
Alexander Chow
RE: Liferay 6.0.4 Document Library Slow Performance
17 de febrero de 2011 9:31
Respuesta

Alexander Chow

LIFERAY STAFF

Ranking: Liferay Master

Mensajes: 519

Fecha de incorporación: 19 de julio de 2005

Mensajes recientes

Or add what many operating systems do -- asynchronously load the stats.
Rafael Vega
RE: Liferay 6.0.4 Document Library Slow Performance
5 de mayo de 2011 4:00
Respuesta

Rafael Vega

Ranking: New Member

Mensajes: 3

Fecha de incorporación: 5 de mayo de 2011

Mensajes recientes

Ray Augé:
That sounds very reasonable.

Perhaps what we could do is:

1) make sure that disabling the columns disables the calculation of stats
2) offer 2 columns that can be used in the settings
a) stats with permission checks
b) stats without permission checks

Thoughts?


Hi,
I have Installed LifeRay 6.0.5 CE + Tomcat + MySQL.
I have 300 roles, 200 folders/subfolders and 300 files in use.
I have poor performance when browse through the document libraries. I've reported this issue days ago (LPS-16599).
I've been reading this thread and I don't have clear idea about the steps to follow.
Initially, I changed the configuration to use algorithm ver 5, but disappear aprox. 90% of permissions in existing document libraries, folders and files emoticon

Ray Auge propose edit folder_columns.jspf in lines 20, 27 and it seems almost a right workaround but after this, he proposed a better workaround ( rename DLFolderServiceUtil by DLFolderLocalServiceUtil)

I have no clear what to do to in this moment:
What is the best workaround I can use?
Can I use the last nightly build? It works?
I know LR V6.1.X has resolved the LPS-12238. Do I have to wait for the 6.1.X release?

Any help would be greatly appreciated.
Thanks,
ilke Muhtaroglu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de junio de 2011 0:21
Respuesta

ilke Muhtaroglu

Ranking: Regular Member

Mensajes: 223

Fecha de incorporación: 12 de mayo de 2009

Mensajes recientes

Hi,

I have applied what have been told at Forum post that I have started because of performance failure at document library:

http://www.liferay.com/community/forums/-/message_boards/view_message/8986666#_19_message_8986666

The solution proposed was to disable the inline permission check via returning false in the method

InlineSQLHelperImpl.java -> public boolean isEnabled(long groupId)

I have done what has been proposed but the solution yield to unaccessible document ?

Is there any solution to this issue till now ?

We are in production and we distributed the documents in several folders to balance this performance issue and the reuslt is 25 seconds per page. Can you please assist to a solution ?

We are really stucked.


Kind Regards.

ilke



ilke









******************************

Sir,

thanks I applied what you said.

It improved the performance that is right but It the documents unaccessible!!!
------

It says,

" the requested resource was not found ! "
http://xx.xx.xx.xx/documents/323/323/aaa.pdf

-----

But it works fine for test (administrator) user.

Any idea, or help please ?

thanks in advance.

ilke



*******************************
Srikanth Reddy Sanivarapu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de junio de 2011 0:24
Respuesta

Srikanth Reddy Sanivarapu

Ranking: Regular Member

Mensajes: 190

Fecha de incorporación: 14 de noviembre de 2008

Mensajes recientes

Did you try to bounce the DB Server once? Thanks.

Regards,
Srik
ilke Muhtaroglu
RE: Liferay 6.0.4 Document Library Slow Performance
6 de junio de 2011 23:18
Respuesta

ilke Muhtaroglu

Ranking: Regular Member

Mensajes: 223

Fecha de incorporación: 12 de mayo de 2009

Mensajes recientes

Sir,

what is bouncing the DB server, I couldnt get the term ? Can you explain the operation supposed to do ?

Regards.

ilke
Zenobia L
RE: Liferay 6.0.4 Document Library Slow Performance
1 de julio de 2011 9:54
Respuesta

Zenobia L

Ranking: Junior Member

Mensajes: 27

Fecha de incorporación: 8 de abril de 2011

Mensajes recientes

Hello,

is this issue resolved in 6.0.6? I see JIRA has an issue reporting a fix in 6.1, but didn't find information regarding 6.0.6

thank you,
Z
Teet Raid
RE: Liferay 6.0.4 Document Library Slow Performance
26 de diciembre de 2011 12:48
Respuesta

Teet Raid

Ranking: New Member

Mensajes: 2

Fecha de incorporación: 26 de diciembre de 2011

Mensajes recientes

Hi,

Ray Augé:
That sounds very reasonable.

...

Thoughts?


Just a sidenote. There is no performance degradation when logged in as a user who has administrator privileges (role).

t.
Carlos Vicente
RE: Liferay 6.0.4 Document Library Slow Performance
19 de enero de 2012 8:06
Respuesta

Carlos Vicente

Ranking: New Member

Mensajes: 24

Fecha de incorporación: 8 de junio de 2010

Mensajes recientes

I Have the same problem. But i think i found the real cause.

When y use Document Library (or forums), Liferay throws a query to the database with a join to the resourcemission table
The problem is that for some reason, at least in mysql, this query doesn't use any of the indexes defined in resourcepermission and do a full scan. In our installation resourcepermission has 6 million rows, and this query has a really poor performance creating temporary tables in disk.
I'm triyn to force in the query to use a index, but without success at the moment.