Polarion Performance & Scalability
Introduction
This document will describe the critical performance factors of the Polarion platform, scalability pitfalls and limitations, and recommendations related to capacity-planning your production environment. It will do so based on a few scenarios that are representative of our install base.
System Configuration Landscape
Polarion is a web-based application. Clients interface with it through a standard web browser. As a result of this, Polarion can be accessed via LAN, WAN (e.g. company-internal inter-office networks), Internet or VPN.
Application Architecture
Polarion is built on top of a growing number of open source frameworks and APIs. All of these components have their own performance and scalability characteristics, only some of which are relevant to Polarion, because of how these components are used by the Polarion platform. Some factors may not even come into play except in very large, or very heavily stressed environments.
Platform Components
Apache HTTP Server
Apache is the de facto web server solution, rivaled only by IIS on Windows platforms. It is extremely robust, and powers some of the largest websites on the Internet.
Scalability and Performance
There are no known issues related to Apache as a performance bottleneck for Polarion.
Recommendations
The Polarion installation procedure installs and configures Apache for you. It is recommended that you don’t deviate from this configuration unless you have very specific reason to, and only after consulting with your Polarion Support Team, to avoid unnecessary performance degradation or system downtime.
Subversion
Since its initial launch in 2004, Subversion[i] has quickly become the de facto solution for version control in organizations of all sizes around the world. Polarion uses Subversion to store almost all of its configuration and data[ii].
Scalability and Performance
One of Subversion’s key strengths is also its primary performance bottleneck. Subversion uses a strict transactional commit model, which means only one commit transaction can be processed by a repository at one time. This means that if the repository is experiencing heavy write traffic, write requests are queued and processed one at a time.
That being said, Polarion change sets are typically in the range of kilobytes, which is far from heavy lifting for Subversion.
Subversion has no proven upper limit as far as repository size is concerned. The Apache Foundation uses a single repository for all of its projects (including Subversion itself, as well as the Apache Web Server project), which consists of well over 900k revisions[iii].
Recommendations
The key factor is to limit the amount of processing that is done in the various commit hooks on the Subversion repository itself to boost the operations’ performance. Any non-trivial automation should be implemented to run out-of-process of the commit itself whenever and wherever possible.
Subversion requires ultra-fast access to its file system. Anything other than physically attached storage is strongly discouraged.
Lucene
Polarion stores all of its (XML) data (and configuration) in Subversion. This means we have to bridge the gap between not having a database backend, and having to be able to query and quickly retrieve and filter data. Lucene[iv] is a powerful indexing framework, and Polarion used it to implement what we affectionately refer to as “The Index”, which is what the Polarion platform uses for all of its read operations.
Scalability and Performance
Lucene has built-in mechanisms to balance fast in-memory storage with on-disk overflow to limit the memory footprint of larger data sets. Beyond that, there is no known upper limit to what Lucene can handle. It scales in an almost perfectly linear fashion when increasing the number of concurrent requests.
The critical performance factor for Lucene is the number of returned results. To mitigate this, Polarion makes heavy use of lazy loading whenever possible.
Recommendations
Users are encouraged to narrow the scope of their queries to extract more relevant results.
Microsoft Word/Excel
The one exception to the storage model (XML in Subversion) is the LiveDoc feature, where work items are stored in Word and/or Excel documents, which are in turn stored in Subversion.
Scalability and Performance
Because to Polarion, the document is a single entity that may contain many work items, every time the document is updated, all of its work items need to be refreshed in the index. As a result, there is a scalability concern related to the number of work items contained in a single LiveDoc Word or Excel document, as well as the overall size of each LiveDoc document file.
Recommendations
Polarion recommends that you limit the number of work items in a single LiveDoc to 1000 for Excel, and 500 for Word. The size of any LiveDoc should not exceed 20MB.
Polarion Modules
Scalability and Performance
Work items contained in a module are stored separately from other work items in a project, and are treated differently because of the added constraints of the module concept. As a result, modules with more than 1,000 work items are only supported when auto-suspect links are disabled (using the “speedup” property, see manual for details).
Recommendations
No single module should contain more than 5,000 work items.
Overall Performance and Scalability
This topic needs to be looked at from two separate yet related perspectives: load and volume of data.
Load
When looking at Polarion as a whole, you will notice it scales almost perfectly linearly on Dual Core-CPU platforms. When moving to 8 CPU cores, the application essentially scales infinitely, for all practical intents and purposes.
The graph points that 80% of save operations with 100 concurrent users actively working with the server will be served in less than 4 secs on one Intel i7 CPU platform. * If there are 100 users changing work items every 5 mins, statistically computed there will be 80%probability that less than 5 save operations go in parallel.
Volume
As long as Polarion’s data remains within the parameters set out earlier in this document, scalability of volume is limited to memory (RAM) consumption, and manifests itself as a largely linear relationship between number of projects in one repository and Polarion’s overall memory footprint.
Reference Customer Installations
The following are examples showing a range of customer installations. Examples are for the purpose of illustrating what kinds of numbers you may need to be thinking of when planning your own installation’s scalability.
Customer “A”
- 15,000 work items
- 12 projects
- 60 users
Customer “B”
- 25,000 work items
- 145 projects
- 1030 users
Customer “C”
- 14,000 work items
- 19 projects
- 600 users
External Factors and Recommendations
Client
Like most other rich web-based applications, Polarion caches a lot of dynamic content in the browser. As a result, memory consumption of the browser process can balloon over time. Polarion recommends that you close your browser after using Polarion, to keep this from becoming a problem.
File System
Polarion can be heavily dependent on disk operations, especially as the server scales to where a growing portion of the index is serialized to the file system.
Performance of the index can be sensitive to disk fragmentation. No operating system is immune to this. Most Linux distribution do give the option of using the ext3 file system, which has features to prevent meaningful fragmentation altogether. In all other cases, regularly scheduled defragmentation is highly recommended.
Subversion requires local or as-fast-as-local file system access to the repository. We strongly recommend either an internal drive, or attached storage (fiber-optic connection). If network-attached storage (NAS) must be used, the length, speed and stability of the network path between server and storage is absolutely critical.
Virus Scanners
Real-time virus scanning can cripple file system performance like nothing else. We recommend you exclude the Subversion repository file structure and all on-disk index (Lucene) data from being scanned, and schedule any scans you feel are needed overnight.
Operating System
Windows is generally slower than Linux in all relevant areas[i]. Beyond ease of installation, there is no recommendation as to a particular flavor of Linux.
Selecting a 64-bit rather than a 32-bit operating system allows for more memory to be assigned to the server process. Even if this is not an immediate concern, it makes for much easier scalability, as replacing a 32-bit with a 64-bit operating system down the road is going to be an intrusive exercise.
For production environments, we recommend a 64-bit Linux with at least 4GB of RAM.
Network
The Polarion client interface relies heavily on quick, short bursts of communication with the server. Network latency is a major factor in client performance degradation. For this reason, network roundtrip (ping) between client and server should ideally be no worse than 150ms.
Virtualization
Virtual environments come at a performance cost, since hardware components such as memory, network, graphics and even storage, are emulated by software. As a result, any application that runs in a virtual machine (VM) will perform worse compared with running the same application on dedicated actual hardware with the same specification.
Polarion recommends that you only run Polarion on virtual machines running Linux. This is largely due to Windows being a slower, larger-footprint operating system to begin with, a fact that only gets amplified by adding virtualization to the mix.
Example Hardware Configurations
| Small | Medium | Large | |
| Operating System | 32 or 64-bit | 64-bit | 64-bit |
| CPU Cores | 2 | 4 | 8 |
| GB RAM (dedicated to Polarion) | 4 (1-2+) | 6 (4+) | 8 (6+) |
| Storage | 250GB | 500GB | 1TB |
| # Projects | < 100 | > 100 | > 300 |
[i] http://subversion.apache.org
[ii] Exceptions include trend and reporting data, as well as log files.
[iii] As of April 2010; see: http://svn.apache.org/repos/asf/
[iv] http://lucene.apache.org/java/docs/index.html
[i] Up to 5% slower in file system operations.
