Streaming a file ressource from a portlet

[Thierry Dagnino, geändert vor 11 Jahren.]

Hi,

we have the requirement to return a large file from a portlet. This is a zip file which is built by the portlet and we want
to return it as it is built to the user. That is, we want to stream it back to the user.

We tried to use the following code but it seems the buffer gets flushed only after the zip is complete and no streaming takes place.
We tried the same thing with a regular servlet and the streaming works fine... Are we missing something ... ?


@ResourceMapping("resourcePDF")
public void resourcePDF(@ModelAttribute(FORM_NAME) final CockpitForm form,
@RequestParam(required = true) String idPrintDocument, final ResourceResponse response,
PortletRequest request) throws OurException {

ThemeDisplay themeDisplay = (ThemeDisplay) request.getAttribute(WebKeys.THEME_DISPLAY);
OutputStream outputStream = null;
try {
outputStream = response.getPortletOutputStream();
ZipOutputStream zip = new ZipOutputStream(outputStream);
ZipEntry entry = null;

response.setProperty("Cache-Control", "public");
response.setProperty("Pragma", "public");

response.setProperty("Content-disposition", "inline; filename=\"test.zip\"");
response.setContentType("application/zip");

byte[] b = new byte[1024];
for (int i = 0; i < 10; i++) {

byte[] pdf = getPdfFile();
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(pdf );

for (int j = 0; j < 10; j++) {

entry = new ZipEntry("pdf" + i + "_" + j + ".pdf");
zip.putNextEntry(entry);

byteArrayInputStream.reset();
while (true) {
int bytes = byteArrayInputStream.read(b);
if (bytes <= 0) {
break;
}
zip.write(b, 0, bytes);
}

zip.closeEntry();
zip.flush();
outputStream.flush();

}

}
zip.close();


Any suggestions would be appreciated.

Thank you.

[Brittney Berkenpas, geändert vor 8 Jahren.]

Hello - I know your post is old but I am facing this same issue. Were you ever able to resolve your problem and actually stream your file as it was being written to the output stream? Thanks in advance

[David H Nebinger, geändert vor 8 Jahren.]

Are you trying to use the serveResource() method to return the stream?

[Brittney Berkenpas, geändert vor 8 Jahren.]

I'm using a struts2 action that gets the HttpResponse object via PortalUtil and directly writes bytes to the output stream. The action is invoked via a resourceUrl and I see the exact same behavior no matter how / when I flush the response - the entire response body is not actually sent until the output stream is closed. Is Liferay doing something special to buffer the entire response entity until it sees the output stream is closed?

[David H Nebinger, geändert vor 8 Jahren.]

Brittney Berkenpas:

I'm using a struts2 action that gets the HttpResponse object via PortalUtil and directly writes bytes to the output stream.

You should not be trying to use the HttpResponse object directly (it is likely wrapped some number of times anyway).

You should be using the ResourceResponse instance directly to access the output stream.

[Brittney Berkenpas, geändert vor 8 Jahren.]

Thanks for the response - I actually just started looking at how to obtain the ResourseResponse object specifically - assuming I can get that, I'll try using its portletOutputStream to write to

[David H Nebinger, geändert vor 8 Jahren.]

And once you get the output stream you can invoke flush() on it to flush the buffer.

[Brittney Berkenpas, geändert vor 8 Jahren.]

I've got the outputStream from the resourceResponse but I am still unable to actually stream the file to the browser. I would expect that at the point I call .flush() - the browser would pop a Save dialog for me.

Regardless of calling .flush, the browser still seems to get the data/save-dialog only after the action completes (and it just returns a null). I also tried setting the buffer size on the resourceResponse via .setBufferSize before I start writing to the outputStream. When I check the buffer size after the .write is complete - the buffer size is set to the size of the output data.

[David H Nebinger, geändert vor 8 Jahren.]

Are you setting the content length before you start streaming? Sometimes using content type application/octet-stream will get an immediate save as dialog, but nothing is guaranteed. Each browser will handle things differently, you just need to find the combo that works best for your browser support requirements.

[Brittney Berkenpas, geändert vor 8 Jahren.]

I have tried setting the content-length prior to starting streaming and I've also been setting the contentType to application/octet-stream. I'm wondering if it could be something in the way the struts2 portlet bridge is handling the resource request.

[David H Nebinger, geändert vor 8 Jahren.]

No, it's the browser. The browser decides when to show the save dialog, not the server.

[Brittney Berkenpas, geändert vor 8 Jahren.]

Could something still be preventing the response from being written to the browser until the full response is in a buffer? It seems strange the buffer size ends up being set to the size of the response data once things are written. I even stuck a Thread.sleep in after the response is written and the stream is flushed and closed, and I'm seeing the save dialog does not popup until after the Thread.sleep finishes/struts action completes (maybe that's not a very valid test though)

I also tried a couple different browsers (FF & Chrome on Linux and Windows) with the same result. (Thanks very much for the continued help)

[Tomas Polesovsky, geändert vor 8 Jahren.]

Hi,

there are some servlet filters in portal that could potentially prevent that.

First I would try to find out where the problem lies - browser or portal.

Can you try call the URL using cURL? For example Chrome Developer Tools has "Copy as cURL" command inside Network tab, which you can use from cmd line (on Windows you need to download cURL.exe)

This way you can see if the response arrives with flush(), ie. before calling Thread.sleep() or after.

[Brittney Berkenpas, geändert vor 8 Jahren.]

I grabbed the cUrl and pasted into terminal and I definitely don't get any data back until after the Thread.sleep call when the action completes. What servlet filters can I look at?

[Tomas Polesovsky, geändert vor 8 Jahren.]

Hi,

So the response seem to be related to the content encoding.

I'm testing with:

	public void serveResource(ResourceRequest resourceRequest, ResourceResponse resourceResponse) throws IOException, PortletException {

		OutputStream out = resourceResponse.getPortletOutputStream();

		Random random = new Random();

		for (int i = 0; i < 20; i++) {
			for (int j = 0; j < 1024; j++) {
				out.write(48 + random.nextInt(10));
				out.flush();
			}

			try {
				Thread.sleep(1000);
			} catch (InterruptedException e) {
				e.printStackTrace();
			}
		}

		out.close();
	}

When I use Accept-Encoding: identity (means no changes), I get the response immediatelly:

curl -v 'http://localhost:8080/web/guest/home?p_p_id=test_WAR_testportlet&p_p_lifecycle=2' -H 'Accept-Encoding: identity'

When I use Accept-Encoding: gzip (means compression), I get only full response.

I'm trying on Tomcat, so maybe there is away to make stream it during compression.

[Brittney Berkenpas, geändert vor 8 Jahren.]

Thanks for the help Tomas
This is junky but just so you can see it - the cUrl I'm working with is below. I can work with setting the Accept-Encoding header to identity and turning off compression - I'm also running tomcat. (TC 7, LR 6.1.0 CE GA1)

curl 'http://localhost/search?p_p_id=searchtabs_WAR_searchportlet_INSTANCE_1soVeAY6cpIy&p_p_lifecycle=2&p_p_state=exclusive&p_p_mode=view&p_p_cacheability=cacheLevelPage&_searchtabs_WAR_searchportlet_INSTANCE_1soVeAY6cpIy_struts.portlet.action=%2Fview%2FcartDownload' -H 'Cookie: COOKIE_SUPPORT=true; JSESSIONID=013BD021A933F81FE9A342A1E79BE8FA.testmachine; COMPANY_ID=949469; ID=716e676250463258586b4d3d; USER_UUID=ivgYfBBK7na6GMYDQHsWzbeMh6EGYoKd; GUEST_LANGUAGE_ID=en_US; LFR_SESSION_STATE_949510=1440603310439' -H 'Origin: http://localhost' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8,es;q=0.6,fr;q=0.4' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.125 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'Referer: http://localhost/search?p_p_id=searchtabs_WAR_searchportlet_INSTANCE_1soVeAY6cpIy&p_p_lifecycle=0&p_p_state=pop_up&p_p_mode=view&_searchtabs_WAR_searchportlet_INSTANCE_1soVeAY6cpIy_struts.portlet.action=%2Fview%2FprocessInvoicedCheckout' -H 'Connection: keep-alive' --data 'allPurchaseDetailIds%5B0%5D=3514900&allPurchaseDetailIds%5B1%5D=3514901&allPurchaseDetailIds%5B2%5D=3514902' --compressed

[David H Nebinger, geändert vor 8 Jahren.]

Makes sense. In order to do gzip compression you need access to some amount of data in the stream to start compressing.

[Brittney Berkenpas, geändert vor 8 Jahren.]

I'm now more suspect of struts2 or some other configuration or something I'm missing. I've tried setting the Accept-Encoding header to identity and even disabling the GZip filter and I still do not start getting response data back until my action completes (currently this happens after my 10 second Thread.sleep for testing) I flush and close response output streams prior to this so I would expect the data to be returned at that point.

[Brittney Berkenpas, geändert vor 8 Jahren.]

I took Tomas's example and wrote a little MVCPortlet that writes out a random stream of integers so that I could use a straight up serveResource method w/the ResourceResponse object and mimic what he's done.

Regardless of how I set the Accept-Encoding header I still don't get an immediate response - I only get response after the outstream is closed. I'm testing this w/curl in a terminal. This leads me back to something in Liferay or Tomcat. I'd appreciate any suggestions as to what I can look at that may be different in terms of servlet filters or other config? I've turned off the GZipFilter explicitly in my portal-ext.properties file but that's about the only nonstandard thing I'm running right now.

[Brittney Berkenpas, geändert vor 8 Jahren.]

Hi again Tomas,

Can you think of anything else I can try to help determine where my problem is? Could you let me know what base portlet class you used in your test? As well as what LR version? I'm still unable to get things to stream immediately. Do you have any thoughts as to servletFilters I should try tuning?

Thanks in advance for any further insight