North Highland’s test automation solution provided a state’s health insurance marketplace with clear, easily validated testing outcomes. Our engagement equipped our client to increase its testing capacity and accelerate programmatic outcomes through automation. Specifically, we drove outcomes by documenting platform requirements; developing test plans; writing, executing, and prioritizing automated test scripts; automated dashboard reporting; coordinating user acceptance testing (UAT) efforts; and conducting defect review, prioritization, and coordination with retesting.
Client Situation
In 2019, our client’s state governor signed legislation to move the state from the federal health insurance marketplace to a state-based health insurance exchange marketplace. Our client selected a platform vendor and set down the path to implementing the state marketplace in time to meet a 2020 deadline.
Health insurance marketplace systems are complex and include multiple integration points across commercial and publicly funded health insurance programs (e.g., Medicaid). The client needed a way to quickly, but thoroughly, test and validate that the platform was ready for deployment in time for the Open Enrollment period. The many system integration points created thousands of probabilities and scenarios to be tested and validated. However, the sheer number of possibilities made it impossible to conduct sufficient testing using manual testing strategies and existing resources. The state needed a way to ensure that its marketplace would be deployed on time—and, more importantly, that it would work.
Our Approach
North Highland partnered with the client to develop and implement a test strategy to ensure that the highly visible platform operated correctly and at the scale required. We used our health insurance marketplace, enterprise system implementation, and testing expertise to develop and implement a testing strategy focused on using test automation to increase testing capacity with manual testing to fill gaps. With an extensive history of helping clients successfully plan, manage, and verify the quality of large IT projects, we understand how to take the hassle out of automated testing and help clients—including the business users within an organization—deeply understand the status of testing and overall system health. Our approach also meets the CMS Testing Guidance Framework recommendation for the use of automated testing tools, “Automated testing should be implemented in a cost-effective manner, and both states and their vendor(s) should be participants in this effort. Iterative development and regression testing benefit from the speed and consistency provided by automation tools.”
Because our automated testing approach is technology-agnostic and can be adapted to a client’s specific requirements, such as an existing technology stack, we were able to work with the client to determine a suite of automated testing tools best suited to its specific IT requirements and able to perform continuous and comprehensive regression testing as new development releases are deployed.
Thanks to its partnership with North Highland, our client confidently launched its state health insurance marketplace before the state deadline. It then selected North Highland to continue providing quality assurance (QA) and testing services after the rollout to support the 2022 and 2023 Open Enrollment periods as well as ongoing enhancements.
Value Delivered
North Highland helped the client increase its testing capacity. Within the first year, we were able to scale the state’s testing capacity to run over 260,000 automated test minutes—a volume that would otherwise have required the client to increase its testing resources nine-fold (which would also mean a significant increase in project costs). With an automated testing strategy in place, the client’s testing resources are now able to focus manual testing efforts on high-priority testing scenarios and on ensuring sufficient testing coverage for system requirements.
Our testing approach and team also helped the client proactively identify defects that could otherwise have been released into the live production environment, impacting the state’s constituents. We accomplished this by continuously testing throughout the Development, Design, and Implementation (DDI) and refining and adding to our baseline suite of automated test cases.
North Highland’s automated testing solution provided our state client with methodical and repeatable testing capabilities that are essential and on demand. We also helped the client reduce the overall QA and development cycle. For example, if the system vendor’s engineering team makes a change, the client has testing results within hours rather than waiting weeks—as it previously did—for the change to be tested. In addition, the solution records a video for each completed test case. These videos make it quick and easy to see where defects are occurring and what specific areas of the system need to be examined.
Automated testing also reduced the time and effort previously spent on manual testing, which helps our client drive business outcomes. As a result, the state can spend more time on incremental system improvements that continue to enhance the customer experience.
