Per Wikipedia, “Quality assurance (QA) is a way of preventing mistakes and defects in manufactured products and avoiding problems when delivering products or services to customers.” According to the Oxford dictionary, QA is “the maintenance of a desired level of quality in a service or product, especially by means of attention to every stage of the process of delivery or production.”
At the Information Technology Office (ITO) within the Ohio Department of Public Safety (ODPS), the QA team exemplifies the meaning of QA by adding quality to almost every aspect of IT operations on a daily basis. The QA team was originally formed five years ago and has grown into a major contributor within the ITO. Below are some of the ways in which the QA team adds value within the ITO:
The QA team begins work from day one of the project, much before any development work begins to ensure that the application is testable, beginning with a test strategy document outlining processes and plans to test the application. As the project progresses, the team develops test plans and further defines test cases going into intricate test steps that can be used to verify that the application is working as designed. These test cases end up being an asset to the organization because they provide the providing business areas and information technology staff with documentation that can be used for future reference. They can also be used for testing the application after every enhancement.
Improvements in the Software Development Process
In the last seven years, QA has become an integral part of the development process. There was push back in the early stages of introducing QA into the development process, as development leads preferred to have the QA resource reporting to them. At ODPS, we allow the QA team to have autonomy by allowing them to define processes for defect reporting, tracking, and resolution. The QA resource on each team has the authority to determine when a defect can be considered resolved; this pays high dividends as it allows the QA team member to uphold high standards for each system. Over time,all of our development teams have recognized the value provided by the QA team and have incorporated testing into every stage of development.
Automated Tests for Verifying Integration Functionality within Sophisticated Systems
Automated tests have been a benefit when it comes to verifying complex integration functionality between our systems. These tests generally comprise of thousands of test steps that are dependent on appropriate test data being available on the various systems. For a person to manually execute these steps, it would mean working on them for more than one day. Manual testing lends itself to many missed steps and incorrect results. Automated tests, on the other hand,can complete thousands of tests in a few minutes and provide a detailed log and recording of the test.This is invaluable for regression testing.
Automated Testsfor the Release Pipeline
At ODPS, one of the goals of the QA team is to make bugs visible to the developer closer to the point of development. Reporting a bug is of less use to a developer a few weeks after the development is complete. The sooner the developer learns of the issue, the easier it is for them to fix it. Towards that end, automated testing in the release pipeline works really well. With automated tests in the release pipeline, a series of actions are triggered every time the developer kicks off a build, including automated deployment to a test or production environment and automated tests that are then run against the specific environment. All these tests are run in a matter of minutes, and the developer discovers almost immediately if the most recent deployment caused any existing functionality to break.
"The QA team exemplifies the meaning of QA by adding quality to almost every aspect of IT operations on a daily basis."
Automated tests have been added to the release pipeline for every environment; the tests are extensive in the development and test environments and are scaled down for the production environment. The only tests that run in the production environment involve query and verification tests, allowing us to make sure that the deployment to the production environment was successful and that the queries between the various tiers to the database are working as designed.
Automated Tests to Monitor Application Performance in Response to Changes in Infrastructure
As part of our efforts to keep our infrastructure current, the ITO is constantly making upgrades on a daily basis, including network equipment upgrades, firmware updates, upgrades to operating systems, service pack upgrades, database upgrades, etc. Automated tests are used to monitor application performance in response to continuous changes in our infrastructure. These changes are first made in our test environments, and automated tests are used to monitor any performance impact on the application before the changes are moved to production.
Automated Tests to Monitor Application Performance in Production
Automated tests are also used to monitor application performance in the production environment. By running tests on the application in production, we are able to verify that the application is functional and performing to expected standards. If there is a break to any of our critical applications, we are alerted to it before any of our customers find the issue. By monitoring the performance metrics, we are able to ensure that changes to critical infrastructure do not impact the performance of the application.
Load Testsfor Applications to Verify Scalability of Application and Stress Test of Infrastructure
Another advantage of having automated tests is the ability to apply load to the automated test, allowing us to stress test the application to verify that the infrastructure is capable of handling high loads of traffic. Additionally, the ability of the application to scale is validated by running these tests under stress and monitoring response times for the application.
Testing and automated testing are essential for modern-day IT operations. Given the complexities and expectations in terms of performance and uptime for today’s applications, it is imperative that ITOs incorporate a strong QA team presence. A well run QA function is cost-effective and productive for the organization, improves the user experience and satisfaction level, and saves money by uncovering software defects and infrastructure issues early in the process.