The main Test Run entity window can be seen below. You should always specify a Name and Description of the Test Run and select the Environment from the Environment drop down against which you are testing.
You must also select all the Test Cases you want to execute as part of this Test Run. This is done by using the 'Select Test Cases for this Test Run' pane on the right of the 'Test Run' window.
If you are setting up a Test Run with Webscript Test Cases you must select one or more web browsers from the 'Select Web Browser(s)' pane. You will need to specify the location of the executable for the chosen browser. This is a one time configuration for your system. Once you have specified the location it is stored within the Vienna database. Vienna will then automatically launch that browser during test execution. If you select more than one web browser this means that you want to run all the tests associated with your test run again in each browser specified. To do this you must set the select 'Multi-Browser Web Scripts' in the 'Runner Settings'. Similarly if you are testing a windows application, and you want Vienna to open that app during test execution, you will need to select 'Automatically Launch Applications', also within the 'Runner Settings'.
Regardless of the Test Case type, you may want Vienna to automatically raise defects for you. This can be achieved by selecting the 'Automatically Raise Defects' option, also listed in the 'Runner Settings'.
When you click on 'Start' to begin your Test Run you will see the application or webpage you entered into your first Test Script appear with the Test Runner bar always on top (see screenshot below). You can now enter actual results and click on pass (tick), fail (exclamation mark) or bugs (Vienna defect icon).
If you selected 'Automatically Raise Defects', whenever you click on the 'bugs' button in the runner it will automatically raise a defect for you capturing the Test Case title, the Test Steps, the browser under test, the operating system, and many more details to help the developer re-produce your issue. You may still edit the details in this automatically created defect - it just does a lot of the work for you.
Finally (if you scroll back up to the Test Run screenshot), you will see the Test Run history at the base of the window. This details the current and all previous Test Runs in terms of dates executed, number of tests run, how many passed, failed and how many contained defects. If you double click on a record you will see a list of all the Test Cases associated with the run, and if there is an associated defect you will be able to click through to it from this window.
A completed Test Run can have one of the following three Status:
- New (when it has first been created)
- Paused (when you exit the run without completing all the test cases assigned)
- Completed (when all test cases have been run).
Other Usage Guide Pages: