Other Case Studies
Executive Summary:
Starchive, a client of Loop Software Services and Testing, recently selected AscentialTest for its web-based application testing. Starchive made the switch for a variety of reasons and through AscentialTest, Loop was able to provide Starchive with the ability to:
Client Objective:
When Starchive first made the switch from manual to automated testing, it was not without challenges, both general and technical. When it came to the performance data of their application, there was no historical data. At that point in time it was not cost-effective to gather data on Starchive’s application performance, therefore it was not always known how a new feature or change affected the application performance. When creating new features and changes in the backend, regressions were run to ensure the changes did not create any breakage, but other than apparent positive or negative performance changes, no performance metrics could be easily tracked.
Another challenge that arose was related to Starchive’s React Framework architecture which creates code generated classes for delivering asynchronous experiences. This caused the application’s object ID’s to be dynamic, making open-source automation testing solutions extremely difficult to maintain. Automated tests consistently broke, leaving Starchive concerned about their automated product testing and the costs to maintain it.
Creation and maintenance of the regression suites presented a unique challenge, as Starchive’s highly effective development team was able to rapidly create new features and backend updates. The application’s mission, storing massive file sets and handling extremely large uploads/downloads has led to issues with verification of uploads/downloads during testing, with the manual approach unable to verify such massive file sets in a cost effective manner.
Our Approach:
Starchive chose to implement AscentialTest based on the recommendation of Matt Osiecki, Head of Automation at LSTS as he saw the large benefits that Starchive would see by choosing this solution. Because of his recommendation, Starchive began utilizing AscentialTest. The initial automation build was focused on performance testing.
AscentialTest is now used to track performance times when uploading, downloading, as well as general page load times within Starchive’s application. The tests are run daily which document performance times in a client facing dashboard and the results are compared to daily, weekly, and monthly results. Any test results that are abnormal are automatically reported to the client within an “Automation” Slack channel. The performance tests are now able to run after any change in the application on both frontend and backend which allows Starchive to see the immediate performance impact of changes.
Another position in which AscentialTest is bolstering Starchive is in its ability to identify objects based on innumerable attributes such as color, position, webtext, parent objects, and others. Dynamic IDs which would normally break open sourced framework automation can be defensively defined within AscentialTest, giving it the definite advantage.
The ability for rapid development was another significant advantage of using AscentialTest. The test case creation and maintenance within AscentialTest proved to be much more efficient than when compared to manual or traditional automated frameworks.
Regarding maintenance, the tests built in AscentialTest are composed of steps and even sub-steps. These building blocks are then used across a variety of different tests. When a change occurs on the frontend, the step that is impacted can be updated, which updates all references and tests containing the step. Because Starchive consistently rolls out new features, a feature change that affects 1000+ test cases would take days to update within a manual suite, but can be done in minutes with AscentialTest’s prowess.
With reference to the creation side of things, Step, the building blocks of tests, can be reused within a test or across multiple tests. Once the steps are created within AscentialTest, it is a simple process to drag and drop them into a new test, instead of building an entire new test from scratch. This means that reusing previously created steps can significantly reduce the hours spent on test creation.
Starchive also shows huge improvements in the verification process via AscentialTest. Before, verifying upload/download/management of thousands of files would be cost-prohibitive and nearly impossible, with the likelihood of human error if done manually. Now, a verification step can be created to verify a file’s image, title, created by date, along with 30+ other properties. This step can be rerun on every file uploaded/download/edited, verifying each file in seconds, while not consuming person hours.
Results Delivered:
The effects of AscentialTest were seen straight away. There was immediate validation of new features and backend change impact on performance which was directly reported to the Starchive team within Slack. Improved user experience and adoption also rose and performance pain points are now able to be identified (slow uploading time, long page load time) via automation and remedied quickly by the development team.
When compared to open-source solutions, AscentialTest’s automation suite proved that it is resilient to dynamic ID changes, saving time and money on maintenance. Less maintenance also means that Starchive is able to obtain the test results more quickly.
The Starchive team and Loop found that the regression suite can be built and maintained much faster when compared to manual testing, saving Starchive considerable monetary investment. The automated suite is faster than manual regressions, completes more test cases with each of them covering more scenarios.
The file set verification system ensures Starchive that any files set, file type, file set size, can be uploaded, edited, and downloaded within Starchive while retaining all original data. Any issues found during file set verification can be remedied before the push to production. AscentialTest has saved Starchive time and money while opening up new testing possibilities.
Automated testing, manual testing and test management tools that are easy to learn, easy to use, and easy to maintain.
Hopkinton, MA 01748
Click here to download the Host ID generator. The download package includes full instructions. Please send us your details below to speed the process along.
Check our new Azure extension for executing AscentialTest command lines in the Azure Marketplace.
We look forward to offering additional integration with Microsoft’s Team Foundation Server and Azure through defect tracking and version control soon.
Automated testing support for OMNIS Studio is coming soon. All OMNIS elements will be supported including complex grids, list boxes, tables and tree views.
Add automated testing support easily for your custom application elements by responding to simple test API messages with json strings. This feature will open up the possibility of testing any GUI element and will be offered free of charge.
Set breakpoints in your automated tests, view global and local variables and execute statements to make it easier to debug tests running on any of our supported platforms.
We will get back to you ASAP. We aim to respond to all inquiries within 24 hours.