15.10 Execution Result Reports
Overview
Execution Report contains crucial information about the test execution session. Here Opkey enables you to export execution reports of your choice. Once test execution has completed, you can export a summary report, detailed report, integrated data report, gherkin report & post-execution validation report by clicking on the Export Report button showing in Execution Result. While exporting a report, you can provide a report name, select report format, add the logo of your choice and export the desired report type.
There are 5 types of Execution Result Reports in Opkey Web – Summary, Detailed, Outline, Integrated Data Report, Gherkin Report & Validation Report.
Let us get familiar with the execution result reports…
Summary Report
The Summary Report displays the name of the Test Case(s) with their status i.e whether the Test Case is Pass or Fail in the Execution. However, the Detailed Report displays information like the name of the Test Case(s), their Test Step(s) along with their status (Pass or Fail during Execution).
- Click on the Export Report button.
- Enter Report Name, select Report Type as Summary and Format as PDF.
- You can now add a logo on the exported reports. Select the Default Project Logo option, if you want to add the default project logo to your report.
- Have a look at the exported Summary report with the default logo.
- You can now add a custom logo on the exported reports. Select the Custom Report Logo option, if you want to add the custom report logo to your report.
- The resolution of the Custom logo should be 150*50 and its size should be up to 35 KB.
- Have a look at the exported Summary report with the custom logo.
Detailed Report & Snapshots
You can view the execution result of the Detailed report along with the snapshots for all the Test scripts. If the execution result has the snapshots then the exported detailed report will have a column for Snapshots, where a link for the snapshot is displayed, clicking on the Snapshot link will open the snapshot. Snapshots older than one week get deleted automatically from the server so snapshots older than a week cannot be viewed.
- Navigate to the Execution Result view & select a Session.
- Click on the Export Report.
- Fill in all the required fields and select Detailed report.
- Check With Snapshots checkbox (if you want a report with snapshot).
- Check Export report either in PDF or in Excel format.
- Click on Export.
- Have a look at the Detailed PDF report exported, which contains a column named as Captured Screen, as shown below:
- When you click on the View Snapshot link of the desired step, the corresponding snapshot opens as follows:
Export Report in PDF
A popup window will get open as follows.
- Enter Report Name and select Report Type to be exported in PDF format.
- Click Export. The execution report gets downloaded into the Download folder of your machine.
- If the selected Report Type is Summary, the following report will be displayed.
OR
- If the selected Report Type is Detailed, the following report will be displayed. This Report includes detailed information about the Execution Result.
Export Report in Excel
Now, users can export their execution report also in Excel format. Once execution has been completed successfully, go to the execution result session and follow the below-given instructions to export reports in Excel format.
- Click on the Export Report button, and a popup window will get open.
- Enter Report Name and select Report Type to be exported in Excel format.
- Click Export. The execution report gets downloaded into the Download folder of your machine.
- The exported detailed Excel report is shown as follows:
Outline Report
Opkey Web users can now send Outline reports to the configured Email Id (under SMTP Settings). It is a high-level report which contains some crucial information about the executed test script. Make sure that you have configured SMTP Settings successfully.
Steps to be followed:
- Click on Run Now to execute your Test Case.
- Check Show Advanced Settings checkbox and click on Next.
- Drag scrollbar to the desired Report Result under the Send Report Result settings.
- If you have not configured SMTP Settings yet then click on SMTP Settings option and configure it.
Email received for the Outline Report. It contains few brief information about the executed test like Suite Name, session, Total Flows, No. of Flows(s) Passed, No. of Flows(s) Failed & No. of Flows(s) Not Executed.
There is a PDF file of Outline Report attached with the mail. This Outline Report of test execution includes Suite Name, No. of Test Cases, Session Start Time, Session End Time, Execution Time, Execution Status under Execution Information section. It also contains an execution outline graph and Keyword Pass/Fail information under Iteration section.
Integrated Data Report
Now, Opkey Web users can export Integrated Data Report for Test Execution of a Suite or Test Case having added Function Libraries. Integrated Report specifically contains information about the input parameters added to the Function Library. There will be detailed information about the input parameters along with their values.
- If there is an input parameter with default value in FL and mapped with the Test Case then Integrated Report will contain the input parameter with its value.
- If there is an input parameter with default value in FL but those input parameters are not mapped with any test Step in FL and you have mapped this FL with the Test Case then Integrated Report will contain the input parameter name with no value.
- If there is an input parameter added to FL and its value are given from TC/FL then the Integrated Report will contain the input parameter with its value.
Steps to be followed to export Integrated Data Report:
- Open your Test Execution session from Execution Result.
- Click on the Export Report button.
- Select Integrated Data Report from the Report Type dropdown.
Select Report format as PDF and click on Export.
Once, the report is exported to PDF format, you can have a look as shown below:
Gherkin Report
Gherkin Report is one of the most awaited features of Opkey which is now available in this Opkey release. You can now export your Gherkin report of the execution having Gherkin & Sparkin artifacts. On export report wizard, only Gherkin & Sparkin files (added in Suite) will be visible from which you can select the desired files from them. Here, you can view that there are 2 Gherkin and one Test Case file added to a Suite.
- The suite has executed successfully and now let us export Gherkin Report.
- Click on the Export Report button as shown below:
- Export Report wizard opens.
- Enter Report Name, select Gherkin Report from Report Type dropdown. You can check Include “Skipped Over” steps checkbox if you want to include skipped test steps.
- PDF Format is selected by default. Click on Export.
- Gherkin Report filter wizard opens which has a filter with few fields like Name, Custom Keyword, Review and Approved By, Show Remarks, Select to add Template, etc. List of all the Gherkin & Sparkin artifact(s) are displayed in the Gherkin Report export wizard; you can select desired Gherkin & Sparkin artifacts and export Gherkin report.
- Select to add Template option allows you to add a template page on the last page of the exported Gherkin Report file which contains some important information about the execution like Test Environment, Windows Edition, Database Version, System Type, etc.
- Here, you can have a look at the exported Gherkin Report.
Snapshots of Failed Steps in Gherkin Report
Now, Opkey users can add snapshots of the failed steps to the exported Gherkin Report in PDF file. In this way, the exported Gherkin Report contains the failed steps along with the snapshots. In order to add failed step snapshots to the report, you need to check “Failed Step’s Snapshots” checkbox while exporting the Gherkin Report of executed Gherkin file and then export the Gherkin Report. Once the Gherkin Report is exported, you can have a look at the Failed Step along with the link (View Snapshot). When you click on the View Snapshot link to view the snapshot of the respective failed step, it opens the snapshot located on the local or cloud server. You should be aware of the snapshots located on the AWS server, as they exist there only for 7 days and get deleted automatically after that.
Pre & Post Execution Report
Overview
Opkey is now available with an all-new advanced feature, Pre(Generate Document) & Post Test Execution Report(Validation Report). This feature allows you to export Pre as well as Post-execution report of the Test Case and Suite. You can upload your sample report(s) template of the desired format (Word form) to the selected project from the Upload Templates tab in Reports under the Project Management of Opkey Admin. Later, you can update, export and delete the uploaded report as required. You can download Sample Template for your reference by clicking on the Download sample template button. A pre-execution report feature allows you to generate a report of your test without doing the actual test execution of the flow whereas you can export the Post-Execution report after completing test execution.
Need of Pre & Post Execution Report?
Execution Reports play an important role in the validation of test execution and its analysis. Now, Opkey is implemented with such advanced feature known as Pre & Post Execution Report (Pre & Post Validation Report). Opkey users were looking for such a reporting feature. This reporting feature helps you to create Pre & Post execution report of your own sample uploaded in word format. Here, you can create your own format so that you can customize your format and add the desired fields that you need.
Getting started with the Pre & Post Execution Report
Now, Opkey allows you to generate Pre as well as Post execution report for the Test Case and Suite. You don’t need to execute the Test Case or Suite in order to generate the Pre-Execution (Validation) Report.
Before creating Pre or Post Execution report, make sure that you have uploaded sample report in Opkey from Reports (Beta) tab under Project Management in Opkey Admin Panel.
Uploading Sample Reports
- Login to Opkey Web and switch to the Opkey Admin Panel.
- Click on Upload Template icon to upload a new report template.
- Select template and click on Open. Only a single template can be uploaded in one go.
- Here, two template samples have been uploaded successfully. You can download a sample Validation report by clicking on the Download Sample Template icon.
- You can update, export and delete uploaded template, as required.
- Next tab, System Fields contains a list of the unique fields that can be added in the sample report to be uploaded which results filling of respective test data in the mapped fields in the Pre and Post-execution reports.
Note: System Fields (E.g. Test Script ID, Test Script Name etc.) in which test data will be filled in exported report, must be mapped under the Test Procedures table in the uploaded sample template. Go through the System Fields table for details about those fields, as showing above…
- You can add your custom field tags from the Custom Fields tab in the selected project. Once you saved the newly added a new tag in Tag Name field, Field Name gets automatically added as per the set standard. While exporting Pre or Post-execution report, you can add Custom Field(s) and get added respective test data in the exported report.
- Click on Add Tags icon to add new tag.
Creating Pre-Execution Report
- Let us generate Pre execution report of a Test Case.
- Open your Test Case, click on Run Now and then select Pre-Report option.
- Fill required fields and click Next. Make sure that your Opkey Agent is online.
- You can choose Step Hierarchy (One Level or Two Level).
Note: Step Hierarchy is the order level of Parent and child artifacts. If you choose One Level then report will contain only one level (Parents) whereas if you select two level then your report will contain Parent as well as child artifacts also.
- From here, you can add Session Tags (Key & Value).
- Click Finish to start creating Pre Execution Report.
- Navigate to the Pre-Reports tab under dockable panel to view the status of report being created.
- Pre-Execution Report has now been generated and you can download and view it.
- Click on Download Report icon and then open it.
- You can have a look at the Pre-Execution/Validation Report created successfully.
Creating Post-Execution Report
- Let us generate Pre execution report of a Test Case.
- Open your Test Case, click on the Execute under the Run Now button.
- Fill all required fields of Local Execution Wizard.
- Check Show Advanced Settings checkbox and click Next.
- From here, you can apply Plugin Settings. Click Next to proceed.
- Check Validation Report checkbox and click Next to proceed.
- Post Report Settings (Beta) page which allows you to select Template, enter Report name, select Snapshot, Steps Hierarchy, Report Logo and then click on Next
- From here, you can add Session Tags. Click Next to proceed.
- From this step, you can choose a Global Variable and apply it.
- Once the test execution gets completed successfully, click on the Export Report, select Validation Report and export it.
- Now, here you can view your exported Post Execution Report in Word format.