unencoded ampersand and taglibs

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hi everybody,

I am working to sanitize errors related to unencoded ampersand (xhtml validation) and i am facing an ambiguous situation : where i am supposed to encode ampersands when it concerns tags?

In the example of the portlet:actionURL tag; i am going to encode url in ActionURLTagUtil class.

Is it the good way?

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hi Jorge, hi everyone,

I suggest a more global solution to solve ampersand error instead of treating it in the level of jsps or taglibs :

• An output filter which allows encoding ampersand in URL by distinguishing two cases xhtml encoding

  &

  and javascript encoding

  %26

  .
• An input filter which transform (in the case of javascript url request)

  %26

  in a valid variables separator

  &

  .
• Properties to configure these new filters.

Are there any elements relating to the portal to take into account for the implementation of this solution?

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

I explore the way to use jtidy to validate the xhtml code and to correct errors in the output flow. There remains a problem to be regulated, jtidy do not offer functions to clean Javascript's code. Someone knows another lirbrary able to clean Javascript's code?

[Jorge Ferrer, módosítva 15 év-val korábban]

Hi Boubker,

I like the idea of doing the encoding in a global way for all situations. That way if someone decides to do an HTML theme they could disable the conversion and use unencoded ampersands.

A filter might be a good way to do this although we would need to do performance tests to make sure it doesn't introduce any bottleneck or significant latency.

I've tried using jTidy before for this approach but all my attempts failed because there were many secondary effects. I think the reason is that jTidy was also prettying up the HTML and that made some Javascript and CSS to fail because the selectors used where not valid any more.

Were you able to find a way to prevent that from happening?

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hi Jorge,

I'll explore the various ways to use jtidy...and i'll come back to expose my choices.

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hello Jorge,
I made some attempts with Jtidy which were unfruitful (problems of performances and bad taking into account of the Javascript)… I thus take again the way of a traditional method (more obstrusive).

Concerning the <portlet:actionurl> tag i put the property escapexml to true in the portal-ext.properties but the resulting URL is not with the awaited format (i.e unencoded ampersands).

Have you any idea about that?

Regards,

Boubker

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hi everybody,

For the setting in conformity of the portal, in particular the problem involved with ampersands, I am working on a solution which implies a code refactoring of most of the jsps.
Indeed, this solution consists of adding an additional parameter "isJS" for renderURL and actionURL tags.
That will make it possible to distinguish the following cases:
- XML-compliant URL (attribute escapeXML=true): the & is encoded as & amp;
- XML-compliant URL within javascript (attribute escapeXML=true and isJS=true): the & is encoded into %26 then escaped for a browser interpretation.

unescape(<%=urljsencoded%>)

What do you think about that? Is this relevant solution?

[Jorge Ferrer, módosítva 15 év-val korábban]

Hi Boubker,

It seems like a lot of work, but you may be right that it's the only solution. In any case let me ask Nate to take a look at the idea before working on it.

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Ok Jorge, I am waiting for Nate's opinion...

[Jorge Ferrer, módosítva 15 év-val korábban]

Hi Boubker,

I've had some conversations with him but we didn't reach any conclusion. I'm trying to clear some time to look at it in more depth but so far I think what you suggested is probably the best solution.

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

i also think that it is the best solution.
Maybe Nate will be able to evaluate the estimated time to accomplish this task (portal and portlets)?

Regards,

Boubker

[Nate Cavanaugh, módosítva 15 év-val korábban]

Hi Boubker,
Sorry for the delay, but I wanted to try and take a look at this.

I think I may be having some trouble understanding a few things.

From what I understand is that when a URL is included inside of an XHTML document, the ampersands must be encoded, so this url:

/web/guest/home?t=1234&s=5678

would need to be parsed as this:

/web/guest/home?t=1234&amp;s=5678

However, when you try to use that URL via javascript, it is ignorant of specific encoding, so it won't handle the &amp; So the conundrum comes when you have a URL in a block of javascript that is embedded inside of an XHTML page (which we often do).

The way we try to handle this is by wrapping the block of Javascript in a CDATA comment.
This will tell the validator to ignore everything inside of it, and won't parse the URL. So if you have something like:
<script>
/*<![CDATA[*/
var url = '/web/guest/home?t=1234&s=5678';
/*]]>*/
</script>
The invalid ampersand will be ignored as part of the XML.

I guess the part I am confused about is why you're creating a flag of isJS when we have the native URL already.
My worry is that escaping the URL for Javascript, and then doing an unescape() on the client side slows things down, and should not be needed.

However, I will fully admit that there might be some part of the problem I'm not seeing.

Thanks Boubker,

[Boubker TAGNAOUTI, módosítva 15 év-val korábban]

Hi Nate,

As a matter of fact implementing the encoding at JSP (or velocity) level forwards the responsibility to the view. It should be treated by the model -- all the more since the case has partially been taken into account when implementing the model. This results in URL double encoding when the "escapeXml" property is set to true -- model-level encoding -- and encoding is also performed at view-level.
Example with a velocity template:

#set ($my_account_url = $htmlUtil.escape($theme_display.getURLMyAccount().toString()))

The URL is encoded twice: first when getURLMyAccount() is called (remember excapeXML is set to true), then when $htmlUtil.escape() is used.

The most efficient way to deal with the issue may be to keep on with the isJS property. This would allow to solve the case in both environments:

1. The escapeXML property is set to true in portal-ext.properties or in portlet-custom.xml.
The generated URL is encoded in the xHTML code. For JS code the isJS property is set to true in order to generate an unencoded ampersand and we add the CDATA comments to prevent the block to be interpreted -- that's your solution. The CDATA comments must be added to each JS code block in JSPs for coherence's sake.

2. The escapeXML is set to false.
The xHTML and JS generated URLs do not make use of character encoding. ExcapeXML and isJS properties are still available for being set as required in each URL generation.

Regards,

Boubker

[Vishal Bhanderi, módosítva 13 év-val korábban]

Sorry to bring this up again...

Any response on this? I still get Validation errors to do with ampersands. of 29 errors on the page. 19 are to do with ampersands (Others im going to work on now).

Im using liferay 6.0.5. I don't really have much javascript running on the site.

Any chance on a fix for this?

Also script tags don't seem to be commented out correctly according to http://www.w3schools.com/TAGS/tag_script.asp

[Julio Camarero, módosítva 13 év-val korábban]

Hi Vishal,

can you help us identify those ampersands in order to fix them? If you can help us fix them providing patches it would be great too.

Can you identify which portlets and which elements in those portlets contain the ampersands?

Thanks a lot

[alex wom, módosítva 12 év-val korábban]

Hi I've posted in the wrong forum, sorry, but my problem is here (my post about w3c validation): can you help me?
Moreover I tried to validate also css and I had a lot of errors with aui css...

[alex wom, módosítva 12 év-val korábban]

In taglib/ui/page_iterator/start.jsp there is a method _getHREF and is sufficient add a line code:
href = url + curParam + "=" + cur + urlAnchor;
HtmlUtil.escape(href); // add this line

[Sergio González, módosítva 12 év-val korábban]

Thanks for finding it and share it! I have just created a ticket and committed a fix for that
Let us know if you find anything else!

Sergio

[alex wom, módosítva 12 év-val korábban]

Hi Sergio the last W3C XHTML+RDFa validator error is (I suppose) in /html/common/portal/layout/view/common.jspf. W3C validator doesn't like
<form action="" id="hrefFm" method="post" name="hrefFm"></form>
it is something like a container filled by javascript in 23 places in portal (I searched hrefFm the code). W3C doesn't like it because it wants some envelope inside form tag like fieldset or span. In this case I have some doubt to put another tag inside because I don't know if the javascript code works again. This error is blocking for W3C validation. We are deploying the Privacy Authority portal and in this case it is necessary W3C validation.
Do you have any suggestion to solve this?
thanks
Alex

[Julio Camarero, módosítva 12 év-val korábban]

Hi Alex,

I just tried to validate (using validator.nu and http://validator.w3.org/) and the problem I had with that form was the following:

Error: Bad value  for attribute action on element form: Must be non-empty.
From line 1, column 13494; to line 1, column 13549
v&gt;  <form action="" id="hrefFm" method="post" name="hrefFm"></form>

<br><br>I understand that the action attribute should not be empty. However, it doesn't say anything about the body being empty.... don't you get the same error?

[alex wom, módosítva 12 év-val korábban]

Hi Julio
I solved in html/portal/layout/view/common.jspf with this:

<form action="" id="hrefFm" method="post" name="hrefFm"><div></div></form>

in this way I validated all portal pages with XHTML strict + RDFa W3C validator. The portal is for italian Data Protection Authority and in this month they are going to publish it.

[Julio Camarero, módosítva 12 év-val korábban]

Ok, I was using HTML5 validation....

Would it be the same if we use a <span> instead of a <div> ?

thank you!

[alex wom, módosítva 12 év-val korábban]

Sorry for the delay, anyway I don't know. I tried with div and it works.

[Marco Calamaro, módosítva 12 év-val korábban]

Sorry I have the same problem.
This code is not validated by W3C as XHTML 1.0 Strict
<form action="" id="hrefFm" method="post" name="hrefFm"></form>

But the problem is not solved by adding "<div></div>" inside form.
The problem is "there is no attribute name".
How could you validate your code as XHTML 1.0 Strict?

Marco

[alex wom, módosítva 12 év-val korábban]

Marco,
it works, I validated with XHTML 1.0 strict + RDFa. Is it possibile that error is in another place?