Polarion Quality Camp: Managing your Simple QA project

Let’s face it: In software development, quality gains are among the hardest points to score and testing your product is as critical as developing it. Practice reveals that with the surging demand for continuous ongoing testing the costs of developing modern applications increase exponentially.

A sound approach and an effective framework at hand for managing your test process is all you need to get things right — that is, maximize your testing efficiency, promote reusability of test artifacts, enable collaboration across your testing organization, increase end-to-end visibility of the entire process, and ultimately reduce the application lifecycle costs.  But how can you make that work? This blog series will cover Polarion’s best practices in test and quality management. You will learn about the Polarion QA concept and capabilities, as well as how to better adjust th em to your own needs and goals.

In the first article we demonstrate steps for installing and working with Simple QA, one of numerous Polarion extensions available on the Polarion POP site. We also discuss the concept of comment-based testing introduced by Simple QA, as well as gauge its advantages and potential for improvement. Subsequent articles will take you into more advanced QA topics.

Managing your Simple QA project

Concept and uses of Simple QA

Built on Polarion platform, Simple QA can greatly assist in managing your teamwork across test planning and execution processes. In fact, it can be used as your entry point in building a fully fledged collaborative web-based QA solution to fulfill your quality goals.

The idea behind Simple QA is to extend Polarion’s core functionality with the Test Lab component, which empowers workflow control, automatic test counting and traceability of test results. The following features are worth-mentioning:

Installing Simple QA Simple QA, just like other Polarion extensions, comes with online instructions on how to install and use it. You can also click here to follow the required steps for installation. Note: Examples described in this article have been performed using Polarion ALM 2010 running on Windows with Firefox 3.6 or Internet Explorer 8.

• Built-in workflow for activating, starting and finishing the execution of test cases


• Customizable template for initiating and iterating test runs


• Additional trigger for activating post-test events, such as creation of linked defects in case of test failure


• Ability to save test results with comments, as well as search and filter them by different criteria


• Reporting functionality for measuring and displaying test execution metrics


• Greater transparency with real-time test data sharing across the project team

As a fully customizable light-weight quality management tool, Simple QA not only automates your test execution, but also enables traceability of test results to corresponding test cases, requirements and defects, thus increasing your team awareness and readiness.

Working with Simple QA

With Simple QA installed to our Polarion system, let us follow step-by-step the entire test process, i.e. initiating test runs, executing test cases, submitting defects, and generating online test reports. The steps are described below:

1. Open an existing project or create a new sandbox project.


2. Go to Test Lab home page in your wiki. By default, the page contains a welcome message from the project owner, a list of test runs, a workspace for creating new test runs, a search field and a TRTemplate link. You can also create new test cases directly from the Test Lab home page.
   

   Figure1. Test Lab wiki page in your project


3. Create a new test run by entering a test run ID and clicking the Create button. The test run ID is normally the number of your iteration, build, or release that will be used later for tracking your test case results. Make sure not to use dots in your ID, since that will be interpreted as a wiki space name.


4. Examine the new wiki page that appears in your browser. It is equipped “out-of-the-box” with a Score Board as well as critical information pertaining to your test cases categorized by Not Executed, Failures with associated defects, Blocked and Passed. You can also drill down to get further detail on the respective test cases or defects.


5. Perform test execution workflow.
   
   
   
   • Open a test case from the Not Executed list or create a new test case. Check if your test case status is Active, if not activate it.
   
   
   • Perform action Start and Save. With this step you set your test case status to In Progress, thus informing your project team that you are currently working on this test. You can also perform a bulk operation to start several test cases at once.
     

     Figure2. Start your test case
   
   
   • Execute your test in accordance with your test case description and requirements.
   
   
   • Upon completion, go to your test case Work Item and in the Status field, select Perform action Finish.
     

     Figure3. Finish your test case
   
   
   • In the Custom Fields area, enter your Test Run ID and select the relevant Test Result: Passed, Failed or Blocked. Optionally you can add your comments in the Test Comment input box. Upon clicking Save, the test case will be set back to the Active state.
   
   
   • In case of test failure, a Defect is automatically created with the title containing by default the test case title and the test run ID. Furthermore, the defect is linked to the current revision of your test case. This is an important feature, since it allows you to refer to the original version of your test case, even if it has been affected by subsequent changes.
     

     Figure4. Defect automatically created and linked to your test case revision
   
   
   
   


6. Notice the information stored in the Comments area of your test case. You might have realized by now that comment-based testing provides for automatic registration of test results, thus alleviating the need for more complex test data structure and workflow configuration.
   

   Figure5. Comments area of your test case with test result


7. Go to your test run ID wiki page and observe the changes. Pay attention to your Score Board for test results on a particular test run. To see the result for all test runs, check the count columns of your test case lists.


8. Iterate the above workflow as needed.

Areas for improving Simple QA

You have just seen your first Test Lab in action and how it can support your activities throughout the testing process. Project stakeholders are intensely interested in the status and results of tests at any given time. Simple QA fulfills this purpose straightforwardly and with minimal effort. As a prototype, it has been designed primarily to demonstrate how simple it is to extend Polarion and implement tailored test execution functionality. Some changes in Simple QA would be desirable. For example, on the usability level it would be nice to have a test run creation date, read-only counts in the custom field of the test case, drop-down list for setting test run targets, etc.

As the name suggests, you can go further along the way and use Simple QA as your reference in creating an enterprise ready solution that satisfies your processes and rules on a larger scale. In the second part of this blog series we will suggest ideas for extending the Simple QA plug-in. You will find useful hints on how to improve testing workflow and data structuring, leverage test planning and capacity management, integrate tighter requirements coverage by test cases and test runs, create and use more complex reporting templates, as well as support a variety of user roles.

Summary

This article described the core features of the Simple QA extension, provided an overview of the test execution workflow and states, and demonstrated built-in test metrics and reporting capacities that will make your testing process smoother and more powerful. Hopefully you are by now ready to use the plug-in to your benefit and with no extra effort. We will be glad to receive your comments.

On a higher level, the demonstrated extension is just one of many possibilities that Polarion opens for you in order to reduce test cycle costs while increasing quality of your software. You come to the right place when looking for a professional platform to tackle such challenges as:

• Process visibility and effective collaboration among all project stakeholders


• End-to-end traceability from requirements to test cases to defects


• Coherent and consistent approach to your testing process


• Flexible data schema and workflow that fit your software development methodology


• Reusable test artifacts and fully customized test reports


• Integration with third-party data sources and test automation tools

Resources

• Check out Polarion POP for new extensions and templates


• Follow and take part in technical discussions on Polarion Forums


• Learn from experts and hone your skills during Polarion workshops and industry events


• Get involved in community and partner gatherings


• Download trial versions of Polarion software


• For more information on Polarion product line visit Polarion Solutions