Making Liferay Portal RTL friendly - the story behind.

Company Blogs January 27, 2014 By Iliyan Peychev Staff

 

The previous versions of Liferay Portal already supported RTL (right-to-left text) in limited way, but since we started the development cycle of Liferay 7.0 we really wanted to improve it. In front of us there were two main goals:

- to convert automatically as much as possible of the UI when language direction is changed

- to free the developer as much as possible from the task of supporting these two modes

 

As most of you probably know, in Liferay 6.2 we did one of the most important changes in the UI we ever made - we get rid of our home brewed CSS framework in favor of Bootstrap and Font Awesome. Among many other good things these changes provided, they allowed us to have stable base for further improvements. One of these was that we were finally able to add support for RTL in a proper and smart way.

 

How did we make it?

 

The whole idea is very simple - when we build the CSS files, once we execute the Sass2CSS task, we run one more. Its purpose is to convert automatically properties like “margin-left”, “left”, “padding-left” to “margin-right”, “right”, “padding-right” and so on. In order to achieve this, we involved in the game an excellent Open Source project - R2. Our initial results looked incredible good, even more than we expected. It turned that applying R2 to Portal converted most of the UI properly. However, as usually happens, the devil is in the details. “Most of the UI” does not mean all of the UI. And we wanted to do it properly.

 

The problems we met

 

I can categorize these in two main categories - glitches in our UI and issues with the third party tools and frameworks we use.

 

Every engineer knows that if you have issues in your own code, you are lucky - you have full control on it and you can make whatever changes you need. However, when there are some issues with third party libraries, it is not so easy. It turned we have to deal with cases, unsupported by R2 and with some uncovered cases by YUI, the JavaScript framework AlloyUI is built on top of.

 

It turned we didn’t have many issues in our own UI. We had a few - glitches with Dockbar, with some input text fields and so on, which we solved successfully and relatively fast.

 

About the libraries and frameworks, one of the issues we had with R2 was how exactly to execute it as part of our build process? For those which are not aware of R2, this is a NodeJS project but for different reasons, we cannot use NodeJS in Portal. The only option we had now was to use Rhino. When we started to use R2, they weren’t using any external NodeJS projects so it was very easy for us to make it working with Rhino.

However, we discovered very soon an issue (they weren’t parsing some comments properly), which was fortunately fixed in their new version. The point was that in this new version they not only fixed that, but also started to use another Open Source project - css-parse. This is another NodeJS project which has its own dependencies and we were unable anymore just to get R2 script and execute it with Rhino. The solution turned to be simple - we used Browserify to extract all dependencies and create a single file and (with only one very small patch) we were able to execute R2 in Rhino again.

 

Creating our own fork of R2 and css-parse

 

We had to fork both R2 and css-parse in order to resolve some different issues. For example, css-parse throws exception when an at-rule is being placed in some qualified rule’s block. Because we prefix with .aui class the whole Bootstrap, Sass parser generates invalid CSS, which the browsers simple ignore, but css-parse dislikes so much, that it throws exception. We reported that issue and it seems they will consider to fix it. Until then, we patched css-parse to simple ignore such rules and to exclude them from the AST.

 

The fork of R2 was needed not only because we had to include our patched version of css-parse, but also in order to add support for cases which were not yet supported. We also introduced in our fork the conception of plugins. One of these plugins is about swapping icons in Font Awesome, another one is to fix the resize handlers CSS in YUI3 Resize module and so on.

 

Fixing an issue in YUI3

 

During our testing process, we discovered an issue in YUI3. Basically, updating the position of an element via setXY in Y.DOM does not work properly with right positioned elements. We filled an issue and also proposed a fix, so let’s hope we will have it fixed there (otherwise we will have to fix our fork of YUI3). Apart from that, the process of tuning up both AlloyUI and YUI was flawless.

 

Conclusion

 

Supporting RTL turned to be an interesting task from engineering point of view. It wasn’t quite simple and required dealing with issues in both our UI and our core frameworks.

 

Ok, perfect, so what’s next?

 

Well, from engineering point of view, the task is finished. We will backport the solution to 6.2 very soon and voilà, we are good to go!

 

At the end, I would like to say thanks to the people who also get involved in this task and made great job - namely Chema, Eduardo Garcia, Robert Frampton and Nate.

 

[Edit] As requested by James Falkner, here are some screenshots.

 

Edit page in RTL mode:

 

 

 

 

Document Library in RTL mode:

 

 

Happy hacking!

UI Developers, make your life easier

General Blogs October 24, 2013 By Iliyan Peychev Staff

Last week, during Liferay Spain Symposium 2013, Olaf Kock recorded a new episode for Liferay Radio with your truly. I talked about my life as developer in Liferay, about the development process of Liferay Portal 6.2, about AlloyUI and so on. The episode will be published soon.

 
As you probably know, I'm working on everything, which is somehow related to UI - Alloy, Portal, Plugins. During our conversation, we talked about the way I'm developing software and one of the things I mentioned was that I try to minimize as much as possible the time I spend after every change in JSP, JS or CSS. Let's face it - deploying manually to Tomcat and refreshing the browser page after that is boring. And everything which is boring, should be fixed.
 
Keeping that in mind, some time ago I tried to resolve this issue. For that purpose, I get a program, called LiveReload, which tracks the changes on the file system and applies those in CSS and HTML immediately, without refreshing browser's page, or if the change was in JS, JSP or JSPF - refreshes the page.
 
I applied this to AlloyUI relatively easy, but the tricky part was to apply the same pattern to Portal, so initially I left it. However, after my conversation with Olaf, I felt it is time to achieve it. Surprisingly, I turned to be very easy:
 
The first step should be to apply the changes to Tomcat immediately, without redeploying manually. I know two ways to achieve this, and here I will talk about the second one - using JRebel. For those of you, who missed the blog post by Miguel Pastor about JRebel, here it is.
 
The second pass is to apply LiveReload to Portal. This is easy and works very well.
 
What you gain? Changes in CSS and HTML in Alloy and Portal are being applied immediately, without even reloading the page. In case of changes in JS, JSP, JSPF, etc. - LiveReload reloads the page.
 
Hope that will help someone to save some time and spend it for real development!

Contributing to Alloy UI - New Rules and Policies

Company Blogs August 15, 2013 By Iliyan Peychev Staff

AlloyUI is the engine which drives the UI of all Liferay products. Since it was created in 2009, it has grown really fast and we are very close to the point where we will release ver. 2.0. We will appreciate if you give us some feedback!

 

Over the years, tests and documentation have sometimes been sacrificed in order to speed up the development process. As result, not all of the modules had tests or documentation. When there were only two or three developers, the lack of tests wasn't such a big issue. They knew the code very well and the risk of regressions was relatively low. However, when more and more contributors were involved, this stopped working and made it harder and harder to keep the library in good condition.

 

As a result, we have instituted new policies related to the tests and documentation which have recently been added to AlloyUI. We would love to share them with you:

 

1. No single commit should go to AlloyUI source tree without tests. Exceptions are allowed for some source formatting (like renaming variables or converting spaces to tabs, etc.), but for each bugfix or for each feature added, tests must be present.  

 

Tests in Alloy are done using YUI tests, so if you want to contribute to the library, but you are not familiar with YUI tests, you may take a look here. It is quite easy, interesting, and it does not take much time. In fact, every single minute spent on making tests will repay you and our community double. In addition to YUI tests, Yeti can be used to execute the tests on multiple browsers simultaneously.

 

2. All code should be properly documented. Documentation should be considered as part of the code. The guidelines for creating documentation could be found here:

http://yui.github.io/yuidoc/syntax/index.html and AlloyUI API Docs Guidelines

 

3. When tests are added (especially as part of a bug fix), in the beginning of the function there must be an annotation, pointing to the corresponding ticket, published here. For example:

 

/**

* Ensure all event listeners are destroyed properly.

*

* @tests AUI-1234

*/

 

4. The changes should be described in HISTORY.md file which every module has, so it is easy to track which changes have been added between two versions. An example:

 

2.0

------

 

* #AUI-938 Fixed issue with TogglerDelegate collapsing direct ancestors on structures with nested togglers

* #AUI-939 Addressed missing destructors in Toggler and TogglerDelegate

 

1.7

------

 

In short, that is it! We would highly appreciate if you help us in the process of creating tests and documentation. There is no better way to get involved in this great project!

 

Happy hacking!

 

WYSIWYG editors for Wiki Creole and BBCode in Liferay 6.1

Company Blogs November 4, 2011 By Iliyan Peychev Staff

You might be already aware of that, but for the others - Liferay 6.1 will have WYSIWYG editors for Wiki Creole and BBCode.

You may use the first one in order to create Wiki pages and the second one - to write messages in Message boards. Actually, you can try Wiki editor right now - it was backported to 6.0 and it is already available on Liferay site (edit or create a new Wiki page in order to check it).

WYSIWYG editor for Message boards will be available in Liferay 6.1.

 

The current approach to write Wiki articles or BBCode postings and its issues

The user opens an editor (which is often simple textarea), writes the text and then applies some formatting on it, using special syntax. This might be Wiki Creole 1.0 or BBCode. Secondly, a Wiki or BBCode engine parses the input and generates (X)HTML code.

For example, the following Wiki Creole code:

**bold**

should generate this XHTML code:

<strong>bold</strong>

Here is the same code, but created following BBCode syntax:

[b]bolded text[/b]

This is very simple example and the syntax is easy to remember. The things get more complicated, when you have complex Wiki page or message. It is not so easy to remember the syntax for tables, ordered and unordered lists (especially when they are deeply nested), links, images and so on. Moreover, the users are often unaware of some important rules - for example, according to Wiki Creole 1.0 specification bold and italics can not cross paragraphs. Also, it is often hard for people to check code for validity. That's why most sites provide a 'Preview' button. Clicking on it shows how the page will look after publishing. If you have syntax error, you should go back, find and correct it manually, then preview again and so on. This is time consuming and usually very annoying.

In order to resolve these issues, some sites provide some kind of limited editors. The idea is that user selects some text, then clicks on a button, say “bold” and the editor surrounds the selected text with corresponding tags. Better than nothing, but the user is still unaware of how the page or post will look after publishing, so he/she should preview it in order to avoid some errors.

 

How we solved these issues in Liferay Portal

We decided to provide a WYSIWYG editor which transparently outputs Wiki Creole or BBCode. The user uses this text editor as any other word processor - MS Word, LibreOffice Wrtiter, etc. In the same time, she is able to switch to 'Source' mode and edit the code directly. Once she switches back to WYSIWYG mode, the editor will apply her changes.

As you know, there are a few popular Web text editors. One of them is CKEditor and this is the default editor in Liferay Portal. We have not created something specific to CKEditor, neither we modified its source code. The code we wrote can be easily adopted for other editors too.

CKEditor outputs HTML code, but it has a nice feature, which is called Data Processor. Basically, an data processor transforms the input data to HTML and vice versa. This is very important, because the data might be anything - in our case that was Wiki Creole and BBCode.

So, we created two data processors. Both of them take the HTML, which CKEditor generates, parse it and create the corresponding code. They also ignore any special formatting, which cannot be translated to Wiki Creole code or BBCode. The final result is well formatted code, which we store in the database.

And what about the opposite process? As we mentioned, the user is able to click on ”Source” button. How should we convert Creole/BBCode back to HTML?

Well, it would be too expensive to send request to server to parse the text and return HTML code back, so we decided to create two JavaScript parsers – Wiki Creole 1.0 parser and BBCode parser. Luckily, we found a very good Wiki Creole parser and we decided to use it (despite we applied some minor modifications).

Unfortunately, with BBCode parser we had no such luck. There were some implementations, but they were very limited, badly written and XSS vulnerable. So, we had no other choice except to write our own from scratch. This parser is available here.

 

Translating JavaScript BBCode parser to Java

Once we finished with JavaScript parser, we found some differences in the way our server-side implementation processed the BBCode in comparison to JavaScript parser. In addition, the server-side parser had some other issues too, so we decided to replace it entirely. You may check the final result here.

The translating process was very fast, most of the code translated 1:1 to Java. We had only a few differences, mostly of them related to Regular Expressions. Also, the code had to be modified a bit in order to work in multi-threaded environment (credits to Brian Chan here).

 

Feedback

Liferay 6.1 is in beta now, so we would be extremely happy to receive some feedback from the excellent community of Liferay.

Go ahead, test it and say what you think about it!

Enjoy!

Browser History support in Liferay 6.1

Company Blogs July 4, 2011 By Iliyan Peychev Staff

Liferay 6.1 will have one more significant improvement because we just added History support to trunk. It will allow JavaScript developers to restore the normal browser behavior and the user will be able to navigate through pages as usual - by pressing Back/Forward buttons.

The issues

For those, who are not aware why is that important, let me explain it with a few words. Normally, the Web is driven by request/response. The browser sends a request to the server and it responds with an answer. The user may request a few pages and she can always return to the previous one by pressing Back button of her browser. Correspondingly, she can press Forward button and navigate to the last page again. This is the normal browser's behavior and people feel comfortable with it.

When Ajax became popular in 2005, more and more applications started to use JavaScript and they no longer reloaded the full page, but only parts of it. This was an awesome improvement, but one of its downsides was that it broke the built in History support in the browser. In fact, people were no longer able to navigate through the history states as they normally would. For that reason the developers tried to restore the normal behavior by adding state entries to the history by using hashes. Let's suppose the following request:
http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

Now, if JavaScript is enabled, everything on client side will be fine - the application can read these parameters after the # and restore the state accordingly. The problem is that server is unaware of these parameters, because fragment identifiers (the parts after the #) are not sent to the server with http requests. For that reason, the server is not able to provide the required content and that led to more issues. Developers usually managed that situation by sending an Ajax request as soon as the page was loaded in order to retrieve the required content. As you may have guessed, that leads to a bad user experience - the page initially loads with some content (or without any content at all) and almost immediately it is being replaced because the server just returned the response to the Ajax request.

Browsers tried to help developers and as result a new event has been introduced - 'hashchange' (now defined in HTML5 specification). But what about browsers that don't support this event? We can achieve that by using an iframe. In order to simplify the whole picture, I will skip the details, but you can just remember that this is not a trivial task. And the worst part is that this approach works only if JavaScript is enabled in the browser.

How HTML5 resolves these issues

With HTML5 everything gets much better. It added some significant improvements, one of the most important being that when adding history state, the developer may change the URL in the address bar too.
The HTML5 specification defined the following interface:

 

interface History {

    readonly attribute long length;

    readonly attribute any state;

    void go(in optional long delta);

    void back();

    void forward();

    void pushState(in any data, in DOMString title, in optional DOMString url);

    void replaceState(in any data, in DOMString title, in optional DOMString url);

};

 

In the browser environment (you know that the browser is not the only environment where JavaScript may run, don't you?) we can retrieve the implementation of this interface from window.history. Then we can invoke 'back()' or 'forward()' functions and navigate the user through the history. With pushState we can add history entries. Currently, the second argument (title) is ignored by some browsers - for example in Firefox 4/5 and Opera 11.50. If the developer provides the third parameter (please note that it is optional), the browser will fully replace the URL with the provided one. That is huge, because we can easily turn this request:

http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

into:

http://liferay.com/manage?p_p_id=123&param_one=1&param_two=2&param_three=3

See the difference? Now we have 'normal' parameters and server may provide the required content on page load without any further requests! And that will always work, regardless of JavaScript enabled/disabled status in the browser.

Imagine this scenario: let's say that your users are navigating your site, and taking advantage of the faster response of Ajax, but that they want to share a link with their friends over Twitter. Then let's say that Google sees that link and decides to crawl it. Because that URL is a full link to the content of your site, Google can index your content without problem. This also means that people without JavaScript enabled will have the same access to content as people with JavaScript enabled.

Now compare that with the old approach – by using hashes. If you have link like this:

http://liferay.com/manage?p_p_id=123&param_one=1#param_two=2&param_three=3

you are in trouble. Why? Because, as you know, Google crawlers don't understand JavaScript and fragment identifiers (the parts after the #) are not sent to the server with http requests. Google cannot just follow this link. That won't work properly.

In order to resolve this, Google suggested a specification - “an agreement between web servers and search engine crawlers that allows for dynamically created content to be visible to crawlers”. Basically, if you replace “#” with “#!” and make your links to look like this:
http://liferay.com/manage?p_p_id=123&param_one=1#!param_two=2&param_three=3

then Google will send the following request to your server:

http://liferay.com/manage?p_p_id=123&param_one=1&_escaped_fragment_=param_two=2%26param_three=3

Now, you must return HTML snapshot of the corresponding #! URL and it must contain the same content as the dynamically created page. In some cases that might be not so easy task and you may even duplicate your work.

Liferay History module

Enough theory, back to the real world. How have we implemented this in Liferay and how can portlet developers reap the benefits?

First of all, as you know, we are using Alloy, great UI meta-framework, built on top on YUI3. In YUI3 we already have the History module, which does the most of the hard work for us. Let me explain what we have added on top on it.

There were some situations we had to resolve. For example - what should we do if the user loads URL, which contains hashes, in a browser that supports the new history interface? Should we leave these parameters as they are? Or we should convert them? Also, what should we do when the next browser version introduces HTML5 History support? Opera did that in version 11.50, released recently. Well, if you paste a URL that contains parameters after the #, Liferay's History module will convert them automatically to 'normal' parameters, so you can update your bookmarks after that. Also, it will add them to its initial state, so later, when a developer requests a value, it will be provided successfully!

In general, the Liferay History module will try its best to provide you with the required values. This means that if developer tries to get a value from History and it is not found, the module will check for it in the 'normal' parameters too. For example, let's suppose you have AlloyUI Paginator on your page and on load you would like to set its 'page' and 'rowsPerPage' configuration properties. In this case you can ask History module for their current values and it will transparently provide them from History - either from the parameters after the # or from the 'normal' parameters!

How Liferay History module deals with old browsers

Now the second question - what should we do with old browsers? What will happen if a user pastes a URL, generated in HTML5 browser in an HTML4 one and starts working? In this case we will add these parameters after the # and once you get back to HTML5 browser, History module will convert them back to 'normal' parameters, taking in consideration the importance. For example, let's suppose that the user pastes this URL in IE7:

http://liferay.com/manage?p_p_id=123&param_one=1

She starts working and over time she gets the following URL:

http://liferay.com/manage?p_p_id=123&param_one=1#param_one=2

In this case the parameter has been overwritten. Now, if she were to switch back to an HTML5 browser and loads the same URL there, the Liferay History module will convert it to:

http://liferay.com/manage?p_p_id=123&param_one=2

Usage and further improvements

We just added this module to trunk, so only two portlets are currently using it - the TagsAdmin portlet and the new Document Library. Before our 6.1 release, we are working to optimize it as much as possible.

Feedback

This module is experimental, so we would be extremely happy to get your feedback!

Credits

Nate Cavanaugh, who helped me during the implementation of the module and edited this article too.

Showing 5 results.