LPS 6.1 GA1 - DuplicateFileException during upgrade

[Stan X, módosítva 12 év-val korábban]

LPS 6.0.5 to 6.1 GA1, MySQL

I have been trying to upgrade to v6.1 GA1, but running into issues with ImageGallery and DocumentLibrary. The images and docs are stored in the db. Since I kept on getting duplicate errors, I have deleted all our images and docs from the organizations and communities. The only remaining images are from the users' pages and their profile pictures. With the reduced db, I started the portal again after cleaning the document_library. The catalina.out displayed the following after the first run:

04:06:42,928 INFO  [UpgradeProcess:205] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery
04:07:05,010 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
        at com.liferay.portal.events.StartupAction.run(StartupAction.java:58)
        at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1267)
        at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:195)
        at javax.servlet.GenericServlet.init(GenericServlet.java:160)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5001)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:866)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:842)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
        at com.liferay.portal.upgrade.UpgradeProcessUtil._upgradeProcess(UpgradeProcessUtil.java:93)
        at com.liferay.portal.upgrade.UpgradeProcessUtil.upgradeProcess(UpgradeProcessUtil.java:46)
        at com.liferay.portal.events.StartupHelper.upgradeProcess(StartupHelper.java:81)
        at com.liferay.portal.events.StartupHelperUtil.upgradeProcess(StartupHelperUtil.java:40)
        at com.liferay.portal.tools.DBUpgrader.upgrade(DBUpgrader.java:119)
        at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:144)
        at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
        ... 20 more
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
        at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
        ... 27 more
Caused by: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:211)
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:225)
        at com.liferay.portal.upgrade.UpgradeProcess_6_1_0.doUpgrade(UpgradeProcess_6_1_0.java:67)
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
        ... 27 more
Caused by: com.liferay.portlet.documentlibrary.DuplicateFileException: /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0
        at com.liferay.portlet.documentlibrary.store.FileSystemStore.addFile(FileSystemStore.java:76)
        at com.liferay.portlet.documentlibrary.store.BaseStore.addFile(BaseStore.java:105)
        at com.liferay.portlet.documentlibrary.store.StoreProxyImpl.addFile(StoreProxyImpl.java:52)
        at com.liferay.portlet.documentlibrary.store.SafeFileNameStoreWrapper.addFile(SafeFileNameStoreWrapper.java:70)
        at com.liferay.portlet.documentlibrary.store.DLStoreImpl.addFile(DLStoreImpl.java:98)
        at com.liferay.portlet.documentlibrary.store.DLStoreImpl.addFile(DLStoreImpl.java:171)
        at com.liferay.portlet.documentlibrary.store.DLStoreUtil.addFile(DLStoreUtil.java:186)
        at com.liferay.portlet.documentlibrary.util.DLPreviewableProcessor.addFileToStore(DLPreviewableProcessor.java:184)
        at com.liferay.portlet.documentlibrary.util.ImageProcessorImpl._storeThumbnail(ImageProcessorImpl.java:269)
        at com.liferay.portlet.documentlibrary.util.ImageProcessorImpl.storeThumbnail(ImageProcessorImpl.java:130)
        at com.liferay.portlet.documentlibrary.util.ImageProcessorUtil.storeThumbnail(ImageProcessorUtil.java:87)
        at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImage(UpgradeImageGallery.java:359)
        at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.migrateImageFiles(UpgradeImageGallery.java:398)
        at com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery.doUpgrade(UpgradeImageGallery.java:248)
        at com.liferay.portal.kernel.upgrade.UpgradeProcess.upgrade(UpgradeProcess.java:208)
        ... 30 more
Stopping the server due to unexpected startup errors

---

I have viewed /opt/liferay/liferay-portal-6.1.0-ce-ga1.prod/data/document_library/10110/0/document_thumbnail/274737/91/49/274779/279448-2.jpg/1.0 and it is a profile picture. There are no other user using the same picture.

Any ideas?

[Stan X, módosítva 12 év-val korábban]

I am also able to replicate the issue with the stock installations, so I have created LPS-25041 .

[Artur Linhart, módosítva 12 év-val korábban]

Hello, I have same exception, but not during the upgrade of image gallery, but during the upgrade of document library...

12:21:58,825 INFO [UpgradeProcess:205] Upgrading com.liferay.portal.upgrade.v6_0_0.UpgradeDocumentLibrary
12:21:58,872 ERROR [MainServlet:198] com.liferay.portal.kernel.events.ActionException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portal.kernel.upgrade.UpgradeException: com.liferay.portlet.documentlibrary.DuplicateFileException: 307

I have created an issue for it:
http://issues.liferay.com/browse/LPS-25082
- there are described further details about stacktrace,e tc.

Do You have any idea, what problem this could be? Currently this is for us a blocker for upgrade. Similar like the errors
http://issues.liferay.com/browse/LPS-25081
(can be related to the above error)
and error in the permission upgrade:
http://issues.liferay.com/browse/LPS-25086

The cause of the errors is not very simple to identify, I looked into the source code and data and found no direct cause of the problems...

Will anybody take care of this errors? because for the upgrade the solution of them is essential...

[Alexander Chow, módosítva 12 év-val korábban]

Hi Artur,

In all three of those tickets (i.e., LPS-25081, LPS-25082, and LPS-25086), it has already been commented that you should upgrade first to 6.0.6 before going to 6.1. From the looks of your errors, it seems as though you are moving from 5.2.x to 6.1 directly. This process is also described in the user guide.

Hope that helps,
Alex

[Artur Linhart, módosítva 12 év-val korábban]

Hello, Alex,

You are right, I came already to this conclusion too - I cannot understand how is it possible I have forgotten we go over 2 major versions and not only over 1 :-). Maybe because before we have planned the upgrade to 6.0.6 and... :-)))

I will check it out and hope, I can then close the bugs with conclusion "no bug", this would be the best solution of our problems :-)...

Thanx for your fast response :-)

[Alexander Chow, módosítva 12 év-val korábban]

Sure thing. Hopefully, that does it for you.

Alex

[Stan X, módosítva 12 év-val korábban]

Hi Alex,

I have upgraded to trunk rev 98688, and put the codes you have suggested. I was able o compile, but LP did not start properly. It is a different issue though.

As for the out of memory message, have you gotten a chance to reproduce the problem with the same number of row counts from the Images table?

[Alexander Chow, módosítva 12 év-val korábban]

Stan X:

Hi Alex,

I have upgraded to trunk rev 98688, and put the codes you have suggested. I was able o compile, but LP did not start properly. It is a different issue though.

As for the out of memory message, have you gotten a chance to reproduce the problem with the same number of row counts from the Images table?

As I mentioned earlier, I made a test framework with double the number of Images you have and was not able to reproduce your problem.

[Stan X, módosítva 12 év-val korábban]

Thanks for the reply. I have missed the second page of the thread.

I will try again to put the patch when I find a trunk build that I am able to start properly.

[Stan X, módosítva 12 év-val korábban]

I am able to run trunk 98895, so I have added the code changes in both files, but I received the same error.

It appears to be in BaseDB.java. I say that because I have also added "--- BaseDB.java ---" to see which file is being accessed. (that is the level of my debugging skills).

07:43:01,070 INFO  [UpgradeProcess:207] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery
--- BaseDB.java ---
--- BaseDB.java ---
--- BaseDB.java ---
--- BaseDB.java ---
--- BaseDB.java ---
Jan 27, 2012 7:46:24 AM org.apache.catalina.startup.HostConfig deployDescriptors
SEVERE: Error waiting for multi-thread deployment of context descriptors to complete
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space

I am currently searching the Wiki on how to setup a debugger.

[Artur Linhart, módosítva 12 év-val korábban]

So, no :-(
Unfortunatelly, the LPS-25081 occurs too if upgrading from 5.2.3 to pure 6.0.6 portal. The only solution is to remove the DL tables content and the DL files, what causes the errors LPS-25081 and LPS-25082 do not occur anymore.
This could be possibly done in this critical situation, we could try to copy the data from the old version through webdav and copy them into 6.1.0 through webdav again... But then the error LPS-25086 is really a killer of us (also occuring again even if upgrading from 5.2.3 to 6.0.6 only). We cannot perform the upgrade before this problem is solved.
Other option would be start with blank portal and take over all sites over LAR files... Is it possible to export the LAR-Files from 5.2.3 and import them to 6.1.0? Will this work or not?

[j latour, módosítva 11 év-val korábban]

Hello Alex,

We try to spend pathologically migration step 6.1.0 to 6.0.12 (with db upgrade 6.1 EE tool), but we fail on a out of memory exception at line 398 of the class UpgradeImageGallery.

Image image = ImageLocalServiceUtil.getImage(imageId);

we run the ant task with these settings

-Xmx700m-Xms700m-XX: MaxPermSize = 256m

We use MySql and the last jdbc drivers. Note that our table image is 4 gigabytes ...

here's the stacktrace :

Thread [main] (Suspended (exception OutOfMemoryError))	
	Arrays.copyOfRange(char[], int, int) line: 2694	
	String.<init>(char[], int, int) line: 234	
	HeapCharBuffer.toString(int, int) line: 561	
	HeapCharBuffer(CharBuffer).toString() line: 1201	
	StringUtils.toString(byte[], int, int, String) line: 1868	
	BufferRow(ResultSetRow).getString(String, MySQLConnection, byte[], int, int) line: 795	
	BufferRow.getString(int, String, MySQLConnection) line: 542	
	JDBC4ResultSet(ResultSetImpl).getStringInternal(int, boolean) line: 5692	
	JDBC4ResultSet(ResultSetImpl).getString(int) line: 5569	
	JDBC4ResultSet(ResultSetImpl).getString(String) line: 5609	
	NewProxyResultSet.getString(String) line: 3342	
	VarcharTypeDescriptor$2.doExtract(ResultSet, String, WrapperOptions) line: 61	
	VarcharTypeDescriptor$2(BasicExtractor<j>).extract(ResultSet, String, WrapperOptions) line: 64	
	StringType(AbstractStandardBasicType<t>).nullSafeGet(ResultSet, String, WrapperOptions) line: 253	
	StringType(AbstractStandardBasicType<t>).nullSafeGet(ResultSet, String, SessionImplementor) line: 249	
	StringType(AbstractStandardBasicType<t>).nullSafeGet(ResultSet, String[], SessionImplementor, Object) line: 229	
	StringType.nullSafeGet(ResultSet, String[], SessionImplementor, Object) line: 89	
	CompositeCustomType.nullSafeGet(ResultSet, String[], SessionImplementor, Object) line: 213	
	CompositeCustomType(AbstractType).hydrate(ResultSet, String[], SessionImplementor, Object) line: 104	
	SingleTableEntityPersister(AbstractEntityPersister).hydrate(ResultSet, Serializable, Object, Loadable, String[][], boolean, SessionImplementor) line: 2283	
	EntityLoader(Loader).loadFromResultSet(ResultSet, int, Object, String, EntityKey, String, LockMode, Loadable, SessionImplementor) line: 1527	
	EntityLoader(Loader).instanceNotYetLoaded(ResultSet, int, Loadable, String, EntityKey, LockMode, EntityKey, Object, List, SessionImplementor) line: 1455	
	EntityLoader(Loader).getRow(ResultSet, Loadable[], EntityKey[], Object, EntityKey, LockMode[], List, SessionImplementor) line: 1355	
	EntityLoader(Loader).getRowFromResultSet(ResultSet, SessionImplementor, QueryParameters, LockMode[], EntityKey, List, EntityKey[], boolean) line: 611	
	EntityLoader(Loader).doQuery(SessionImplementor, QueryParameters, boolean) line: 829	
	EntityLoader(Loader).doQueryAndInitializeNonLazyCollections(SessionImplementor, QueryParameters, boolean) line: 274	
	EntityLoader(Loader).loadEntity(SessionImplementor, Object, Type, Object, String, Serializable, EntityPersister, LockOptions) line: 2037	
	EntityLoader(AbstractEntityLoader).load(SessionImplementor, Object, Object, Serializable, LockOptions) line: 86	
	EntityLoader(AbstractEntityLoader).load(Serializable, Object, SessionImplementor, LockOptions) line: 76	
	SingleTableEntityPersister(AbstractEntityPersister).load(Serializable, Object, LockOptions, SessionImplementor) line: 3293	
	DefaultLoadEventListener.loadFromDatasource(LoadEvent, EntityPersister, EntityKey, LoadEventListener$LoadType) line: 496	
	DefaultLoadEventListener.doLoad(LoadEvent, EntityPersister, EntityKey, LoadEventListener$LoadType) line: 477	
	DefaultLoadEventListener.load(LoadEvent, EntityPersister, EntityKey, LoadEventListener$LoadType) line: 227	
	DefaultLoadEventListener.proxyOrLoad(LoadEvent, EntityPersister, EntityKey, LoadEventListener$LoadType) line: 269	
	DefaultLoadEventListener.onLoad(LoadEvent, LoadEventListener$LoadType) line: 152	
	SessionImpl.fireLoad(LoadEvent, LoadEventListener$LoadType) line: 1090	
	SessionImpl.get(String, Serializable) line: 1005	
	SessionImpl.get(Class, Serializable) line: 998	
	SessionImpl.get(Class<!--?-->, Serializable) line: 128	
	ClassLoaderSession.get(Class<!--?-->, Serializable) line: 237	
	ImagePersistenceImpl.fetchByPrimaryKey(long) line: 422	
	ImagePersistenceImpl.findByPrimaryKey(long) line: 372	
	ImageLocalServiceImpl.getImage(long) line: 230	
	GeneratedMethodAccessor72.invoke(Object, Object[]) line: not available	
	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
	Method.invoke(Object, Object...) line: 601	
	ServiceBeanMethodInvocation.proceed() line: 112	
	TransactionInterceptor.invoke(MethodInvocation) line: 71	
	ServiceBeanMethodInvocation.proceed() line: 108	
	ServiceBeanAopProxy.invoke(Object, Method, Object[]) line: 211	
	$Proxy25.getImage(long) line: not available	
	ImageLocalServiceUtil.getImage(long) line: 175	
	UpgradeImageGallery.migrateImage(long) line: 350	
	UpgradeImageGallery.migrateImageFiles() line: 434	
	UpgradeImageGallery.doUpgrade() line: 251	
	UpgradeImageGallery(UpgradeProcess).upgrade() line: 210	
	UpgradeProcess_6_0_12_to_6_1_0(UpgradeProcess).upgrade(Class<!--?-->) line: 227	
	UpgradeProcess_6_0_12_to_6_1_0.doUpgrade() line: 56	
	UpgradeProcess_6_0_12_to_6_1_0(UpgradeProcess).upgrade() line: 210	
	UpgradeProcessUtil._upgradeProcess(int, String, ClassLoader) line: 103	
	UpgradeProcessUtil.upgradeProcess(int, String[], ClassLoader, boolean) line: 56	
	UpgradeProcessUtil.upgradeProcess(int, String[], ClassLoader) line: 36	
	StartupHelper.upgradeProcess(int) line: 81	
	StartupHelperUtil.upgradeProcess(int) line: 48	
	DBUpgrader.upgrade() line: 119	
	DBUpgrader.main(String[]) line: 54	
</t></t></t></j></init>

thanks

[Alexander Chow, módosítva 12 év-val korábban]

Hi Stan,

This has already been fixed with LPS-24803. Sorry about that.

Alex

[Stan X, módosítva 12 év-val korábban]

Thanks for the response Alexander.

What revision is the fix? I have tried trunk revision 98310, and it appeared that I have the same issue.

[Alexander Chow, módosítva 12 év-val korábban]

Hi Stan,

After looking at it again, it turns out that this has not yet been fixed. I presume you are following the ticket you created but I have just sent a GIT pull request of the fix to be reviewed. It can be found on Github.

Alex

[Trey Howard, módosítva 12 év-val korábban]

@Alexander

I see that LPS-24803 is marked as a fix for 6.2.x.
Is this due to be released before Dec 2012?
I'm currently trying to upgrade to 6.1 now and this is blocking me.

-Trey

[Alexander Chow, módosítva 12 év-val korábban]

Hi Trey,

It is marked for 6.2.x because that is the default next major release. When 6.1 GA 2 comes out is a different question… one which I have no answer to (we are trying to get the EE release out right now).

However, if you look at the fix, it actually fairly simple. I've attached the patch which is really just adding six lines of code to UpgradeImageGallery.java.

Alex

[Stan X, módosítva 12 év-val korábban]

Hi Alexander,

I have added the lines to trunk 98427 and built it. I did not get the duplication error, but OutOfMemory.

SEVERE: Error waiting for multi-thread deployment of context descriptors to complete
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:573)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:469)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1412)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:312)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
        at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:401)
        at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:346)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1117)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:782)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1525)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1515)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2882)
        at java.lang.StringValue.from(StringValue.java:24)
        at java.lang.String.<init>(String.java:178)
        at com.mysql.jdbc.SingleByteCharsetConverter.toString(SingleByteCharsetConverter.java:286)
        at com.mysql.jdbc.SingleByteCharsetConverter.toString(SingleByteCharsetConverter.java:262)
        at com.mysql.jdbc.ResultSet.getStringInternal(ResultSet.java:5671)
        at com.mysql.jdbc.ResultSet.getString(ResultSet.java:5544)
        at com.mysql.jdbc.ResultSet.getString(ResultSet.java:5584)
        at com.mchange.v2.c3p0.impl.NewProxyResultSet.getString(NewProxyResultSet.java:3342)
        at org.hibernate.type.descriptor.sql.VarcharTypeDescriptor$2.doExtract(VarcharTypeDescriptor.java:61)
        at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:64)
        at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:253)
        at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:249)
        at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:229)
        at com.liferay.portal.dao.orm.hibernate.StringType.nullSafeGet(StringType.java:89)
        at org.hibernate.type.CompositeCustomType.nullSafeGet(CompositeCustomType.java:213)
        at org.hibernate.type.AbstractType.hydrate(AbstractType.java:104)
        at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2283)
        at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1527)
        at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1455)
        at org.hibernate.loader.Loader.getRow(Loader.java:1355)
        at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:611)
        at org.hibernate.loader.Loader.doQuery(Loader.java:829)
        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
        at org.hibernate.loader.Loader.loadEntity(Loader.java:2037)
        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:86)
        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:76)
        at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3293)
        at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:496)
        at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:477)
        at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:227)
        at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:269)
</init>

It appears that some images were converted, but the process did not complete. The last settings in the setenv.sh that I tried:

JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF8 -Dorg.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES=false -Duser.timezone=GMT -Xmx2048m -XX:MaxPermSize=512m"

Regards,
Stan

[Alexander Chow, módosítva 12 év-val korábban]

Unfortunately, that's a bit hard to debug because there is only one line in your stack trace that has Liferay code.… and it's not that useful a line. Is there more to your stack trace by chance?

[Stan X, módosítva 12 év-val korábban]

Hi Alex,

That was it. Are there any debug settings that I need to enable?

[Alexander Chow, módosítva 12 év-val korábban]

It looks like the memory problem is occurring in the MySQL driver because the entire result set is being read into memory (see MySQL documentation about this). This is peculiar to me because I always thought those queries were supposed to be paginated. But, I guess not!

What I would do is this… First, track down which SQL query that is causing the problem. You can do this by turning on the portal-ext.properties entry:

    hibernate.show_sql=true

To warn you, this will give you a lot of data dumped out to your console. However, when the java.lang.OutOfMemoryError occurs, you can tell which SQL query is causing that. Then, you can try applying the code as described in the MySQL documentation:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
              java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);

This will not load everything into memory but fetch the result set, line-by-line. Hopefully, that will solve your problem.

[Stan X, módosítva 12 év-val korábban]

Thanks for the response.

The line prior the error (above) was :

<timestamp> INFO [UpgradeProcess:207] Upgrading com.liferay.portal.upgrade.v6_1_0.UpgradeImageGallery

I am sharing it just in case it will help.

I will try the property after the db is reloaded.

[Stan X, módosítva 12 év-val korábban]

Hi Alex,

There were bunches of these:

Hibernate: select imageimpl0_.imageId as imageId13_0_, imageimpl0_.modifiedDate as modified2_13_0_, imageimpl0_.text_ as text3_13_0_, imageimpl0_.type_ as type4_13_0_, imageimpl0_.height as height13_0_, imageimpl0_.width as width13_0_, imageimpl0_.size_ as size7_13_0_ from Image imageimpl0_ where imageimpl0_.imageId=?

and then followed by :

Jan 24, 2012 8:19:49 AM org.apache.catalina.startup.HostConfig deployDescriptors
SEVERE: Error waiting for multi-thread deployment of context descriptors to complete
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space

I do not know yet how to apply the additional code mentioned above.

[Alexander Chow, módosítva 12 év-val korábban]

You aren't running a debugger right? If you are, you could just step through the code and find the exact line within our upgrade code that is causing this problem. Your logs show the hibernate SQL for any call to ImageLocalServiceUtil.getImage(imageId). While that is useful, I forgot that the rest of the upgrade code uses raw SQL calls which is not displayed in the hibernate logs.

However, my guess is that the out of memory error is happening either within the method UpgradeDocumentLibrary.updateSyncs() (which is called from UpgradeImageGallery) or UpgradeImageGallery.migrateImageFiles(), perhaps in the big block to delete from Image. If you add a few lines of logging, we can know for certain.

If it is the first, you can make a change like the following in UpgradeDocumentLibrary.updateSyncs():

	protected void updateSyncs() throws Exception {
		Connection con = null;
		PreparedStatement ps = null;
		ResultSet rs = null;

		try {
			con = DataAccess.getConnection();

...
			String sql = sb.toString();

//			ps = con.prepareStatement(sql);
// Begin
			ps = con.prepareStatement(
				sql, java.sql.ResultSet.TYPE_FORWARD_ONLY,
				java.sql.ResultSet.CONCUR_READ_ONLY);

			ps.setFetchSize(Integer.MIN_VALUE);
// End
			rs = ps.executeQuery();

If it is the second, the problem is in the runSQL() method which, in turn, calls BaseDB.runSQL(). So you can make the change at roughly line 202 to:

	public void runSQL(Connection con, String[] sqls)
		throws IOException, SQLException {

		Statement s = null;

		try {
//			s = con.createStatement();
// Start change
			s = con.createStatement(
				java.sql.ResultSet.TYPE_FORWARD_ONLY,
				java.sql.ResultSet.CONCUR_READ_ONLY);

			s.setFetchSize(Integer.MIN_VALUE);
// End change
			for (int i = 0; i < sqls.length; i++) {

In any case, this probably requires you to debug a bit more to find exactly where in the code you are having problems and then add those changes as necessary. I would be very interested to know where the problem is coming from considering others may run into the same problem.

BTW, do you think you can give me a count of how many entries you have in the Image table? Just do a "select count(*) from Image". It seems as though you have quite a few images and I wonder if I could build a test framework to see if I can run into the same problem.

[Alexander Chow, módosítva 12 év-val korábban]

Also, if you can give a "select count(*) from IGImage" along with the "select count(*) from Image", that would be helpful too.

[Stan X, módosítva 12 év-val korábban]

Set1:

Image = 20215
IGImage = 5769

Set2/Reduced:

Image = 5194
IGImage = 765

Size = ~3MB/image.

The Set2/Reduced is after I have deleted all images from organizations and communities via the portal. The remaining images are from users' image gallery and profile pictures. It is also the same set where the logs came from. Set1 is the full set that I plan to upgrade when the problems are fixed.

[Alexander Chow, módosítva 12 év-val korábban]

Stan, I have been testing now with 10,000 IGImage entries and 30,000 Image entries and have no OutOfMemoryError. I even turned down my max heap to 512 and it still had no problems. In other words, I can't reproduce your bug.

Have you tried patching the code yourself and seeing if you can reproduce it?