Showing 55 Comments
Jonas Yuan
2/25/11 7:14 AM
cool! Thanks a lot, Alex.
James Min
2/25/11 8:26 AM
awesome!
Ahmed Hasan
2/25/11 11:59 AM
Good Post Alex
Peter Monks
2/25/11 3:43 PM
Great news! It would be great to hear how the demand for the CMISHook looks after this new functionality is released. Would you be willing to make a prediction, perhaps? ;-)
Also, is anyone from Liferay engaged with the CMIS OASIS technical committee? Most of the limitations of CMIS 1.0 raised in the post are well known to the TC and being considered for inclusion in future versions, but it would be great to also have client implementers (such as Liferay) engaged in the process to ensure the spec meets their needs.
Sandeep Nair
2/25/11 9:06 PM
Very nicely explained and I have to say its very innovative. One doubt i have is about permissions. Will the permissions be at liferay's end and the repository will just act to its name thats is storing documents or is it necessary to have permission in the repository to access the documents?
Alexander Chow
2/26/11 1:59 AM
@Peter The concerns I have raised in this post are well known in OASIS. Actually, if you look at the links I posted, they all link to OASIS public documents.
@Sandeep The CMIS protocol specifies a set of permissions that, within Liferay, is translated over. All the permissions are managed from the third-party repository's end.
Sandeep Nair
2/26/11 3:49 AM
Oh I see. Thanks for the response.
Peter Monks
2/26/11 12:25 PM
Thanks Alexander - does that mean Liferay is indeed directly involved in the ongoing work of the OASIS CMIS TC?
Florian Müller
2/27/11 2:06 AM
Hi Alexander,
The CMIS performance can actually be pretty good.
I think you are using OpenCMIS under the hood. Reuse the Session object as much as you can. It does a lot of caching for you and tries to avoid unnecessary calls to the repository. Especially, the AtomPub binding benefits from it. OpenCMIS is thread-safe. It's no problem to share the Session or other CMIS objects across threads.
Also, only fetch the properties you really need. Some repositories send lot of properties by default, which bloat the messages and reduce the performance. Some CMIS properties have to be calculated on the repository side, which can take time. If you filter them out with an OpenCMIS OperationContext, you gain performance.
Have a look at the CMIS Workbench that is part of OpenCMIS. There is no noticeable delay when you navigate through the folder hierarchy.
Alexey Kakunin
2/28/11 12:23 AM
Great! Really killer feature of Liferay 6.1.
Is it already in svn trunk? Is it possible to play with it?
Alexander Chow
2/28/11 1:37 AM
@Peter I was part of it last year but there were some changes on our status that I'm not too clear on.
@Alexey Definitely. Give her a try and let me know what you think. You can email me directly if you like: alexander.chow { at } liferay.com.
@Florian Good to hear from you and your thoughts on performance. I am very much aware that running things via the workbench can be quite snappy. If you like, we can have more of a technical discussion offline (not sure if this is the best forum for clarifying technical problems) -- feel free to email me.
Briefly, for the OperationContext, though this is true, the bulk of the properties are needed. I mean, fundamentally, to the end user, we are trying to portray the third-party repository to look and feel like a Liferay repository that has many properties stored in the database and other things in the filesystem or something like that. So, there is not much filtering that can occur.
In regards to the reuse of sessions, the difficulty comes when we have a multi-user environment (as compared to the workbench). Each session must be configured different for each user. So, if you have 100 logged in users, you need to keep that many sessions opened, with their respective caches. When things are managed all within Liferay's own repository, we have the ability to cache based on the roles of the users which, obviously, shrinks the number of individual caches stored. And when we load a page, we load not only the contents of a folder, but also statistics for its subfolders and sub-subfolder listing and other such things -- none of which are retrieved when we are using the workbench. Instead, we now require multiple queries to address this across the wire.
There are undoubtedly areas which we can improve on our end. Maybe we can configure it so a lot of these extra statistics and random information can be removed. But, at present, that is the current experience our users are used to without the same speed hit.
Anyhow, as I said, if you would like to discuss this further, drop me an email or maybe we can arrange a call.
Ivano Carrara
2/28/11 10:19 AM
Hi Alexander !
It is not clear for me if we can configure multiple DL portlets instance with each instance pointing (for example) to a separate Alfresco servers...
In fact, in a possible scenario, we can aggregate in a single portal the contents contained in various Alfresco servers, of course instantiating the same quantity of DL portlets, each one in a different community...
Thank you for your carification !
Ivano C.
Alexander Chow
2/28/11 10:55 AM
Hi Ivano,
I am not quite sure I understand your question. But ultimately, in one DL portlet, you can point to multiple servers -- each mounted as a separate repository. In my example above I simulate having two Alfresco servers (one via CMIS AtomPub and another via CMIS Web Services) mounted to the same DL portlet.
Hope that clarifies things!
Alex
Jakub Liska
2/28/11 2:49 PM
Hi Alex,
I wanted to give it a try, but it is impossible to even connect from liferay via alfresco share http://issues.liferay.com/browse/LPS-15543 ... I spent half a day on this issue. I simply can't figure that out.... whatever I tried I ended up with this error...
I'd really appreciate if you could take a look at it... I feel hopeless
http://issues.alfresco.com/jira/browse/ALF-7423
Jakub Liska
2/28/11 5:51 PM
I see, only LR 5.2.3 is supposed to work with Alfresco share ... I could spare half a day then :-)
Jakub Liska
2/28/11 7:26 PM
Could you please guys take a look at the last comments ? http://www.liferay.com/community/forums/-/message_boards/message/7703791
It is a relevant issue to this one.
Jonas Yuan
2/28/11 8:54 PM
Hi Alex, got error when adding repository (at revision 74180, created account admin/admin with role Administrator, and log in as admin/admin).
04:46:03,851 ERROR [RepositoryServiceImpl:305] com.liferay.portal.security.auth.PrincipalException: org.apache.chemistry
.opencmis.commons.exceptions.CmisRuntimeException: Unauthorized
com.liferay.portal.security.auth.PrincipalException: org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeExcepti
on: Unauthorized
at com.liferay.portal.repository.cmis.CMISRepository.processException(CMISRepository.java:1408)
at com.liferay.portal.repository.cmis.CMISRepository.initRepository(CMISRepository.java:619)
at com.liferay.portal.service.impl.RepositoryServiceImpl.createRepositoryImpl(RepositoryServiceImpl.java:415)
at com.liferay.portal.service.impl.RepositoryServiceImpl.mountRepository(RepositoryServiceImpl.java:302)
Jay Brown
2/28/11 9:21 PM
Hi Alex. I'm the development lead for IBM's CMIS servers. Would you be interested in doing some interoperability testing if I could give you access to a test IBM CMIS P8 cloud system. I'm just looking for you to point your client at my public server and see if there are any compatibility issues. We do this with a lot of CMIS client vendors (e.g. Apache Chemistry ) and we have found that is makes both clients and implementers better. Sometimes we even end up tightening up the spec as a result of gray areas that we uncover.
Looking forward to hearing from you. You will find my correct email address in your database under this account.
Thanks in advance,
Jay Brown
IBM
Alexander Chow
3/1/11 2:52 AM
@Jakub In general, Liferay has provided a CMISHook since 6.0 and, in the upcoming 6.1 (not yet released), we provide mounting of multiple repositories -- CMIS being one of them via CMISRepository. I'll respond more in your message board post.
@Jonas From the looks of your exception, it looks like you have an authentication exception on the Alfresco end. Did you check their logs? Also, make sure you can hook into it using the CMIS workbench (http://incubator.apache.org/chemistry/cmis-workbench.html). If you can't there, you definitely won't be able to via Liferay.
@Jay That would be brilliant! P8 is what of the repositories I am tasked with testing against so that is exactly what I would need. Drop me an email -- alexander.chow { at } liferay.com.
Jignesh Vachhani
3/1/11 5:01 AM
Lots of thanks Alex.
Really its a nice enhancement !!!
Jignesh Vachhani
3/1/11 5:02 AM
Lots of thanks Alex.
Really its a nice enhancement !!!
Jonas Yuan
3/1/11 6:08 AM
Hi Jay, Thanks. I want to test IBM CMIS P8, too. In LR 6.1, OpenCMIS Apache Chemistry should get supported 100%.
Eventually it would be nice that the CMIS of Day Software, Dennis Hamilton, EMC, FatWire, Microsoft, Open Text,
Oracle, and SAP could be tested in details in Liferay 6.1.
Jamie L Sammons
3/1/11 8:52 AM
Hi Alex,
Awesome feature! Its nice to be able to give end users the consistent look and feel of Liferay with a powerful backend repository like Alfresco.
However I am noticing the performance hit that you mentioned quite a bit. I have Alfresco running on a pretty hefty dedicated system and it still takes 5 to 10 seconds for files and folders to show up. The Alfresco process is pretty pegged whenever Liferay is pulling up files and folders. Hopefully some of the things Florian mentioned can help with this.
Thanks,
Jamie
Alexander Chow
3/3/11 8:03 AM
@Jamie Florian and I have had quite a few emails about improving the performance. I have taken a few steps at boosting the performance which you should be able to see as of rev 74447. No matter what, the first time a logged in user accesses a given folder, it will take a little bit to load for the data to be cached.
I can't comment on how well Alfresco performs because I don't use Alfresco on a regular basis. But now, I have minimised the number of calls I am making over the wire so it may improve for you.
Let me know how things go.
Alex
Peter Monks
3/3/11 9:10 AM
Yeah minimising the number of round trips is the #1 priority from a performance perspective. If a single call is found to be slow then the usual Alfresco performance tuning and analysis process can be followed (a service Alfresco provides, fwiw).
Jamie L Sammons
3/3/11 2:13 PM
@Alex It does seem faster. I'm also going to try upgrading the Alfresco side to Alfresco Enterprise 3.5 since its still at 3.3.
@Peter I was thinking about opening a ticket with Alfresco to see if I could get some basic tuning parameters as well.
Hajo Kliemeck
5/12/11 3:31 PM
does anyone have experience with the combination of Liferay 6.1, Alfresco, CMIS and CAS? i think it is not possible because the password has to be saved... any ideas?
Drew Blessing
5/12/11 5:08 PM
How will authentication to the external repository work if we use SSO, like CAS, with the portal? In this case the portal never gets to see the password, so cannot store it to send on to the repository.
The only thing I can think of is that somehow the external repository will forward the request to the CAS server and receive an auth ticket behind the scenes. This is a clear shot in the dark!
Other than this, I am very excited about this support and cannot wait to try it out!
Alexander Chow
5/13/11 9:13 AM
@Hajo @Drew Yeah the question of CAS (and other SSO systems) is something that is in the back of my mind. The question really is whether we can either (1) use the ticket used to log into the portal or (2) create a proxy ticket to log into the third-party repository, probably the latter. Or, to put it another way, it is a question of whether the third-party repository supports CAS (proxy)tickets via CMIS... and I can throw in the hook for that (should be relatively simple, if it does).
I have tried exploring this with Alfresco, but haven't gotten too far... mainly because I don't know too much about how to get CAS running on Alfresco (not an Alfresco expert here) and my contact in Alfresco wasn't too clear on how CAS works with CMIS either. I also have been a bit strapped with some other work so haven't really been able to research this too much.
@Peter @Florian Any thoughts on this one?
Drew Blessing
5/16/11 7:21 AM
@Alexander I completely understand the struggle with Alfresco CAS. The documentation are not very helpful, and neither are their engineers on the forums. I have been able to get Alfresco CASified, though. I would be willing to provide my documentation if you're interested. I am fairly certain that CASifying Alfresco also puts all web service URLs behind CAS as well. Although it's wishful thinking I am hoping that if both Alfresco and Liferay are CASified and the user is already authenticated, attempting to access Alfresco via CMIS will automatically trigger a CAS ticket request and push through on it's own. I haven't had a chance to test this yet, but plan to soon. Thanks for your work. It's really going to be slick!
Hajo Kliemeck
5/18/11 10:10 AM
@alex, look at this: http://issues.alfresco.com/jira/browse/ALF-7074
if you are willing to modify your cmishook, you should create a seperate alfresco-cmishook!
Alexander Chow
5/21/11 8:17 PM
@drew why don't you drop me an email. alexander.chow { at } liferay.com
@hajo thanks for the link. Will need to explore this further. Not sure if that is a CAS ticket or an Alfresco ticket. But I think we can probably create a generic authentication utility that Liferay can use to login to not only Alfresco but any other system.
Jakub Liska
6/15/11 8:28 AM
@Alex, regarding managing metadata, custom types / JCR mixins, it is called "Secondary types" and it seems to be in development
http://tools.oasis-open.org/issues/browse/CMIS-713
Alexander Chow
6/15/11 9:16 AM
@Jakub, yes I'm very much aware of that. Thanks!
[...] LOS ANGELES, CA – Liferay, provider of the world’s leading enterprise-class open source portal, today announced its plans to further improve ease of integration with third-party ECM document... [...] Read More
Jakub Liska
6/26/11 2:19 PM
@Alex, 3 months ago, I was about to implement BaseRepositoryImpl so that I would mount a custom JCR repository to DL, via this implementation. I said to myself that I better wait until it stabilizes or some documentation shows up.
In regard to BaseRepositoryImpl / BaseLocalRepositoryImpl :
Is it so, that CMISRepository doesn't implement the local interface, because ACL is handled on the repository side ?
So that if the third party repository hadn't its own authentication / authorization mechanism, it would implement both interfaces and the remote one would be a permission checking wrapper for the local one, right ?
In regard to sync of a 3rd party repository with RepositoryEntry, the idea is (in CMISRepository impl) having all documents synchronized, right ? It can be seen in CMISRepository.cacheFoldersAndFileEntries(); You mentioned that in this blog post. But shouldn't CMIS be only a mounting point ? I don't see a reason for keeping state of remote repository in RepositoryEntry table. Whatever CRUD I do, I want it to be reflected only in the remote repository, imho. What is the idea behind the synchronization ?
Not only CMIS repository, but even a custom JCR repository should be only "mounted" to DL. Why would one need to keep track of documentIDs ?
Jakub Liska
6/26/11 3:52 PM
I suppose that the reasons have "integration nature", because DL uses DLAppService and it operates with FileEntries, so that addFileEntry() method of a third part repository must return FileEntry and there must be some tracking between FileEntry table and third party repository. For instance, right after file is added via DL, it is added to AssetPublisher.
I believe that a lot of LR developers would actually need to employ DL only as UI for their custom repositories. If you google around, you see that in most cases, people think about using their document store in LR. They've got a document store and they are searching for a portal to wrap it with.
Considering that hooks are meant only for scaling purposes. When used with LiferayRepository, the store must be empty, to be in sync with FileEntries. They can be used directly though (not necessarily via LiferayRepository).
It isn't clear to me, what third party repositories that implement BaseRepositoryImpl are meant for. Because it is very hard to think about the conflicts that may arise when you make changes into the third party repository not only via BaseRepositoryImpl implementation but even by some other means. It practically excludes the possibility to use DL as a user interface for custom document stores, which I think is what people would like to see. Though it is hard to imagine something like this if the state in the document store and FileEntry table is to be synchronized, no matter how perfect would the implementation be.
Alexander Chow
6/27/11 8:14 AM
Hi Jakub,
The only things that are really synchronized are IDs. The cache you refer to is just for a quick lookup that gets reset for ever HttpRequest. This is because each HttpRequest may access the same files or folder multiple times and we want to minimize excessive calls within a few milliseconds. Hence, there should not really be too much of a conflict between CRUD operations in Liferay vs. the third party repository.
Hope that clarifies things.
Alex
Jakub Liska
7/1/11 4:03 AM
I'm referring to the synchronized IDs. I'm just saying that for the sake of LR , DL might offer the possibility of being UI for repositories... So that creation of FileEntry could theoretically be part of Repository implementation. That way, DL could be used as UI for third party repositories without any possible sync issues when creating new files. I know it is practically Utopia, because DL is built on FileEntries, it's not built for JCR stores... But I can't even figure what happens if a file is removed from repository (not via DL), but DL already got it's ID :-)
The tight integration can be also seen for instance here : EditFileEntryAction.updateFileEntry() method has :
AssetPublisherUtil.addAndStoreSelection(
actionRequest, DLFileEntry.class.getName(),
fileEntry.getFileEntryId(), -1);
Shouldn't this be part of the Repository implementation ? If I use my custom repository, it should decide itself about AssetPublishing right ?
I'm just trying to point this out, because for past years, as I said, I've seen many people asking about how to utilize Liferay for their existing document stores. Imho DL now is not enough transparent for doing this.
[...] Liferay Improves Access to ECM Document Repositories with CMIS 1.0 Compatibility LOS ANGELES--(BUSINESS WIRE)--Liferay, provider of the world’s leading enterprise-class open source portal, today... [...] Read More
[...] Hi, I am new to Liferay. I downloaded and deployed Liferay 6.1 from trunk. Then I tested the new integration features with Alfresco CMIS. In the Document Library I have added the repository via... [...] Read More
[...] After banging my head against this problem for a few days I turning to the Liferay community for some suggestions. I'm trying to configure an instance of Sharepoint 2010 to serve as a CMIS repository... [...] Read More
[...] Capability to manage different document types: basic document, image (Image Gallery is merged into Document Library), video, audio, etc. Providing Dynamic Data List (DDL) and Dynamic Data Mapping... [...] Read More
[...] Capability to manage different document types: basic document, image (Image Gallery is merged into Document Library), video, audio, etc. Providing Dynamic Data List (DDL) and Dynamic Data Mapping... [...] Read More
Navin Agarwal
12/5/11 9:26 PM
Hi Alex,
I have tried to do the same But facing some issues with the Visibility of the file. I have raised the issue in the http://issues.liferay.com/browse/LPS-23408.
Please help me to fix the issue.
Thanks in advance.
Alexander Chow
12/6/11 3:21 AM
Thanks Navin. I'll track the ticket there.
Navin Agarwal
12/6/11 3:39 AM
Hi Alex,
I have one more issue , which also I have raised . http://issues.liferay.com/browse/LPS-23407
[...] Montar múltiples repositorios – En la nueva Biblioteca de documentos y multimedia, podrás montar múltiples repositorios (mediante el protocolo CMIS) y desarrollar conectores a medida que enlacen con... [...] Read More
[...] Hi, I want to mount the Alfresco repository to Liferay repository using Liferay's latest feature of mounting multiple CMIS repositories. I am following this blog. But I am not sure how to get the... [...] Read More
Michael Kroh
3/27/12 2:39 PM
Hi Alex,
are there any news about testing against IBM's CMIS servers?
We have here inhouse a IBM CMIS Server and I'm not able to connect against that Server
AtomPub:
01:16:15,669 DEBUG [FullNameValidatorFactory:46] Return com.liferay.portal.security.auth.DefaultFullNameGenerator
01:16:15,671 WARN [DLFileEntryPersistenceImpl:5544] No DLFileEntry exists with the key {groupId=19, folderId=0, title=IBM}
01:16:15,751 WARN [RepositoryLocalServiceImpl:95] Unable to initialize CMIS session for repository with {repositoryId=10569}
01:16:15,752 DEBUG [InvokerPortletImpl:370] processAction for 20 takes 89 ms
01:16:15,753 DEBUG [InvokerFilterChain:113] Skip disabled filter class com.liferay.portal.servlet.filters.servletcontextinclude.ServletContextIncludeFilter
WebServices
01:22:42,188 DEBUG [FullNameValidatorFactory:46] Return com.liferay.portal.security.auth.DefaultFullNameGenerator
01:22:42,189 WARN [DLFileEntryPersistenceImpl:5544] No DLFileEntry exists with the key {groupId=19, folderId=0, title=ibm}
01:22:42,200 WARN [RepositoryLocalServiceImpl:95] org.apache.chemistry.opencmis.commons.exceptions.CmisConnectionException: Cannot initalize Web Services service object [org.apache.chemistry.opencmis.binding.webservices.RepositoryService]: 2 counts of InaccessibleWSDLException.
01:22:42,202 DEBUG [InvokerPortletImpl:370] processAction for 20 takes 22 ms
It seem's to me that I'm doing some principal error ?
Thanks
Michael
Alexander Chow
3/27/12 4:46 PM
Hi Michael,
I've worked with Jay Brown on IBM's P8 server in the past and have been able to get it to work. However, IBM has quite a few servers that have various levels of CMIS support. You may want to try using the CMIS workbench (http://chemistry.apache.org/java/developing/tools/dev-tools-workbench.html) against your server and perhaps even running the built-in TCK to make sure it works first. If it is a principal error, Liferay should be returning that.
Also, the forums are perhaps a better forum to discuss problems you may be having than on my blog. It widens the discussion to input from others.
Alex
[...] Multiple Repository Mounting. In the new Documents and Media app, you can now mount multiple repositories (e.g. through CMIS) and develop custom connectors to link to existing CMS repositories. ... [...] Read More
Natasa Bulatovic
4/11/12 2:53 AM
Dear all,
interested if there is any possibility/plans to add a SVN repository of choice via LR document library?
Alexander Chow
4/11/12 3:55 AM
Hi Natasha,
That's a great question. I don't think that is part of our use case but it is surely a very conceivable one. The framework, however, was built so one can fairly easily build a plugin. Somebody in the community has already done one for te file system.
Alex
[...] Liferay 6.1 brings with it the ability to mount multiple content repositories (e.g. CMIS, Documentum, Sharepoint, etc). And now Tomas brings the ability to mount OS filesystems! Yes, this means you... [...] Read More