You might be already aware of that, but for the others - Liferay 6.1 will have WYSIWYG editors for Wiki Creole and BBCode.

You may use the first one in order to create Wiki pages and the second one - to write messages in Message boards. Actually, you can try Wiki editor right now - it was backported to 6.0 and it is already available on Liferay site (edit or create a new Wiki page in order to check it).

WYSIWYG editor for Message boards will be available in Liferay 6.1.

The current approach to write Wiki articles or BBCode postings and its issues

The user opens an editor (which is often simple textarea), writes the text and then applies some formatting on it, using special syntax. This might be Wiki Creole 1.0 or BBCode. Secondly, a Wiki or BBCode engine parses the input and generates (X)HTML code.

For example, the following Wiki Creole code:

**bold**

should generate this XHTML code:

<strong>bold</strong>

Here is the same code, but created following BBCode syntax:

[b]bolded text[/b]

This is very simple example and the syntax is easy to remember. The things get more complicated, when you have complex Wiki page or message. It is not so easy to remember the syntax for tables, ordered and unordered lists (especially when they are deeply nested), links, images and so on. Moreover, the users are often unaware of some important rules - for example, according to Wiki Creole 1.0 specification bold and italics can not cross paragraphs. Also, it is often hard for people to check code for validity. That's why most sites provide a 'Preview' button. Clicking on it shows how the page will look after publishing. If you have syntax error, you should go back, find and correct it manually, then preview again and so on. This is time consuming and usually very annoying.

In order to resolve these issues, some sites provide some kind of limited editors. The idea is that user selects some text, then clicks on a button, say “bold” and the editor surrounds the selected text with corresponding tags. Better than nothing, but the user is still unaware of how the page or post will look after publishing, so he/she should preview it in order to avoid some errors.

How we solved these issues in Liferay Portal

We decided to provide a WYSIWYG editor which transparently outputs Wiki Creole or BBCode. The user uses this text editor as any other word processor - MS Word, LibreOffice Wrtiter, etc. In the same time, she is able to switch to 'Source' mode and edit the code directly. Once she switches back to WYSIWYG mode, the editor will apply her changes.

As you know, there are a few popular Web text editors. One of them is CKEditor and this is the default editor in Liferay Portal. We have not created something specific to CKEditor, neither we modified its source code. The code we wrote can be easily adopted for other editors too.

CKEditor outputs HTML code, but it has a nice feature, which is called Data Processor. Basically, an data processor transforms the input data to HTML and vice versa. This is very important, because the data might be anything - in our case that was Wiki Creole and BBCode.

So, we created two data processors. Both of them take the HTML, which CKEditor generates, parse it and create the corresponding code. They also ignore any special formatting, which cannot be translated to Wiki Creole code or BBCode. The final result is well formatted code, which we store in the database.

And what about the opposite process? As we mentioned, the user is able to click on ”Source” button. How should we convert Creole/BBCode back to HTML?

Well, it would be too expensive to send request to server to parse the text and return HTML code back, so we decided to create two JavaScript parsers – Wiki Creole 1.0 parser and BBCode parser. Luckily, we found a very good Wiki Creole parser and we decided to use it (despite we applied some minor modifications).

Unfortunately, with BBCode parser we had no such luck. There were some implementations, but they were very limited, badly written and XSS vulnerable. So, we had no other choice except to write our own from scratch. This parser is available here.

Translating JavaScript BBCode parser to Java

Once we finished with JavaScript parser, we found some differences in the way our server-side implementation processed the BBCode in comparison to JavaScript parser. In addition, the server-side parser had some other issues too, so we decided to replace it entirely. You may check the final result here.

The translating process was very fast, most of the code translated 1:1 to Java. We had only a few differences, mostly of them related to Regular Expressions. Also, the code had to be modified a bit in order to work in multi-threaded environment (credits to Brian Chan here).

Feedback

Liferay 6.1 is in beta now, so we would be extremely happy to receive some feedback from the excellent community of Liferay.

Go ahead, test it and say what you think about it!

Enjoy!

Liferay 6.1 will have one more significant improvement because we just added History support to trunk. It will allow JavaScript developers to restore the normal browser behavior and the user will be able to navigate through pages as usual - by pressing Back/Forward buttons.

The issues

For those, who are not aware why is that important, let me explain it with a few words. Normally, the Web is driven by request/response. The browser sends a request to the server and it responds with an answer. The user may request a few pages and she can always return to the previous one by pressing Back button of her browser. Correspondingly, she can press Forward button and navigate to the last page again. This is the normal browser's behavior and people feel comfortable with it.

http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

Now, if JavaScript is enabled, everything on client side will be fine - the application can read these parameters after the # and restore the state accordingly. The problem is that server is unaware of these parameters, because fragment identifiers (the parts after the #) are not sent to the server with http requests. For that reason, the server is not able to provide the required content and that led to more issues. Developers usually managed that situation by sending an Ajax request as soon as the page was loaded in order to retrieve the required content. As you may have guessed, that leads to a bad user experience - the page initially loads with some content (or without any content at all) and almost immediately it is being replaced because the server just returned the response to the Ajax request.

Browsers tried to help developers and as result a new event has been introduced - 'hashchange' (now defined in HTML5 specification). But what about browsers that don't support this event? We can achieve that by using an iframe. In order to simplify the whole picture, I will skip the details, but you can just remember that this is not a trivial task. And the worst part is that this approach works only if JavaScript is enabled in the browser.

How HTML5 resolves these issues

In the browser environment (you know that the browser is not the only environment where JavaScript may run, don't you?) we can retrieve the implementation of this interface from window.history. Then we can invoke 'back()' or 'forward()' functions and navigate the user through the history. With pushState we can add history entries. Currently, the second argument (title) is ignored by some browsers - for example in Firefox 4/5 and Opera 11.50. If the developer provides the third parameter (please note that it is optional), the browser will fully replace the URL with the provided one. That is huge, because we can easily turn this request:

http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

into:

http://liferay.com/manage?p_p_id=123&param_one=1&param_two=2&param_three=3

See the difference? Now we have 'normal' parameters and server may provide the required content on page load without any further requests! And that will always work, regardless of JavaScript enabled/disabled status in the browser.

Imagine this scenario: let's say that your users are navigating your site, and taking advantage of the faster response of Ajax, but that they want to share a link with their friends over Twitter. Then let's say that Google sees that link and decides to crawl it. Because that URL is a full link to the content of your site, Google can index your content without problem. This also means that people without JavaScript enabled will have the same access to content as people with JavaScript enabled.

Now compare that with the old approach – by using hashes. If you have link like this:

http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

you are in trouble. Why? Because, as you know, Google crawlers don't understand JavaScript and fragment identifiers (the parts after the #) are not sent to the server with http requests. Google cannot just follow this link. That won't work properly.

http://liferay.com/manage?p_p_id=123&param_one=1#!param_two=2&param_three=3

then Google will send the following request to your server:

http://liferay.com/manage?p_p_id=123&param_one=1&_escaped_fragment_=param_two=2%26param_three=3

Now, you must return HTML snapshot of the corresponding #! URL and it must contain the same content as the dynamically created page. In some cases that might be not so easy task and you may even duplicate your work.

Liferay History module

Enough theory, back to the real world. How have we implemented this in Liferay and how can portlet developers reap the benefits?

First of all, as you know, we are using Alloy, great UI meta-framework, built on top on YUI3. In YUI3 we already have the History module, which does the most of the hard work for us. Let me explain what we have added on top on it.

There were some situations we had to resolve. For example - what should we do if the user loads URL, which contains hashes, in a browser that supports the new history interface? Should we leave these parameters as they are? Or we should convert them? Also, what should we do when the next browser version introduces HTML5 History support? Opera did that in version 11.50, released recently. Well, if you paste a URL that contains parameters after the #, Liferay's History module will convert them automatically to 'normal' parameters, so you can update your bookmarks after that. Also, it will add them to its initial state, so later, when a developer requests a value, it will be provided successfully!

In general, the Liferay History module will try its best to provide you with the required values. This means that if developer tries to get a value from History and it is not found, the module will check for it in the 'normal' parameters too. For example, let's suppose you have AlloyUI Paginator on your page and on load you would like to set its 'page' and 'rowsPerPage' configuration properties. In this case you can ask History module for their current values and it will transparently provide them from History - either from the parameters after the # or from the 'normal' parameters!

How Liferay History module deals with old browsers

Now the second question - what should we do with old browsers? What will happen if a user pastes a URL, generated in HTML5 browser in an HTML4 one and starts working? In this case we will add these parameters after the # and once you get back to HTML5 browser, History module will convert them back to 'normal' parameters, taking in consideration the importance. For example, let's suppose that the user pastes this URL in IE7:

http://liferay.com/manage?p_p_id=123&param_one=1

She starts working and over time she gets the following URL:

http://liferay.com/manage?p_p_id=123&param_one=1#param_one=2

In this case the parameter has been overwritten. Now, if she were to switch back to an HTML5 browser and loads the same URL there, the Liferay History module will convert it to:

http://liferay.com/manage?p_p_id=123&param_one=2

Usage and further improvements

We just added this module to trunk, so only two portlets are currently using it - the TagsAdmin portlet and the new Document Library. Before our 6.1 release, we are working to optimize it as much as possible.

Feedback

This module is experimental, so we would be extremely happy to get your feedback!

Credits

Nate Cavanaugh, who helped me during the implementation of the module and edited this article too.