Determining Cause of Slow Page Generation

[Mark Fitzgerald, geändert vor 9 Jahren.]

I am trying to track down the cause of a performance problem on our production intranet system. We consistently get 6.5 - 9 seconds TTFB (time-to-first-byte) when accessing web pages on our live site. In other words, the system is taking 6.5 - 9 seconds to generate the web page to send to the browser. On our development system, we consistently get 1 - 2 seconds TTFB. System specs listed below.

I'm looking for ideas as to why there is such a difference between the two systems. I would think that the lower powered system would be slower, but it turns out to be the opposite. I used a Java profiling tool, but couldn't find any "smoking gun".

Common Settings:
[indent]Liferay v6.2 EE SP1 - Tomcat bundle
Theme: Based upon Classic theme from Liferay
NTLM auto login on every page, not just /c/portal/login
portal-ext.properties match between the two systems (other than database and clustering settings)
MS SQL Server database[/indent]

Production Specs (per server in a 2 node clustered environment):
[indent]RAM: 8GB
CPUs: 4
TTFB: 6.5 - 9 seconds[/indent]

DEV Specs:
[indent]RAM: 4GB
CPUs: 2
TTFB: 1 - 2 seconds[/indent]

[Rashmi Rajappa, geändert vor 9 Jahren.]

Did you check performance by accessing nodes individually in the cluster?
I would start debugging from there. If performance is 1-2 second on individual nodes, then you need to check your web server/load balancer etc.

Hope this helps.

[Mark Fitzgerald, geändert vor 9 Jahren.]

Rashmi Rajappa:

Did you check performance by accessing nodes individually in the cluster?
I would start debugging from there. If performance is 1-2 second on individual nodes, then you need to check your web server/load balancer etc.

Hope this helps.

Yes. I am conducting my testing by accessing the Liferay applications directly (http://<server-name>:8080/web/guest/home).

[Milen Dyankov, geändert vor 9 Jahren.]

For the time taken to generate particular portlets' contents you may find https://www.liferay.com/web/olaf.kock/blog/-/blogs/displaying-portlet-monitoring-information useful!

That said, you need to collect data on all levels of your infrastructure to find the bottleneck. It could be your network, OS, HTTP server, Liferay, Database, external services, ... and many other things. There is no single tool that you'll simply run and it will result with something like "look ma, here it is ->" ;) However tools like NewRelic, AppDynamics, JVisualVM, profilers, thread dump analyzers ... can help a lot (particularly for the JVM part).

[Mark Fitzgerald, geändert vor 9 Jahren.]

In my haste to get to the root of my question, I didn't include other background information about my setup. I've already got a handle on our network, database, and other infrastructure components that are involved in the web request. The last remaining part is the application. I can trace the actions in our portlets, theme, and hooks, but I am blind when it comes to assessing what Liferay is doing. This has been especially confusing when accessing two systems in the same data center, but getting significantly different results.

The link that you provided to Olaf's blog entry looks promising. Thank you. I'll see if that can provide some additional information to help in our optimization efforts.