Ace your Enterprise Test Management Strategy with Seven Practical Tips

In brief: 

• With improving ways of working as the top transformation objective among health and human services (HHS) organizations this year, enterprise test management is a vital tool in the IT strategic planning process by embedding quality—from strategy-setting to daily working norms.
• Organizations with a top-down approach to Enterprise Test Management Strategy (ETMS) are prepared to plan and address today’s challenges in a multi-vendor, multi-technology, complex business and technology integration environment.
• As you’re setting your ETMS and developing its playbook, consider seven key guardrails.

When designing a comprehensive IT strategy, health and human services (HHS) leaders typically focus first on their operating model, platform selection, and resourcing. Yet, there’s one piece that’s often overlooked in setting strategy: enterprise test management. While leaders may be tempted to relegate enterprise test management to a downstream execution activity, we believe it’s a strategic cornerstone. Enterprise test management embeds quality and value into your operational blueprint when woven into IT strategy.

In this blog, we’ll explore why a formal Enterprise Test Management Strategy (ETMS) should be a foundational part of your IT strategy—connecting your vision for quality and value to the work that gets done. From there, we’ll offer our take on the most critical elements of your ETMS and IT strategy playbook. Whether you aspire to stand up a Testing Center of Excellence (CoE) or execute on a strategic commitment to high-quality IT deliverables, this blog serves as a practical guide for your ETMS.

Where we are today

As technology sets the competitive pace, HHS enterprises are turning to digital capabilities that promise innovation and market responsiveness. With this investment comes increased pressure on IT to generate early and consistent ROI supporting top and bottom-line growth priorities. In response, many adopt agile ways of working to help fuel the cadence of value delivery.

With improving ways of working (via agility and cross-functional collaboration) as the top transformation objective among HHS enterprises this year,^[1] ETMS is a vital tool in the IT strategic planning process by embedding quality—from strategy-setting to daily working norms. It’s a scalable, enterprise-level quality assurance plan that strengthens the IT function’s position as a value-driven, trusted business partner. Unfortunately, many IT leaders don’t look at ETMS strategically, pushing test management to the project implementation phase. When not top-led, test management becomes an impediment to quality, value, and business impact. Conversely, we’ve found that organizations with a top-down approach to ETMS are prepared to plan and address today’s challenges in a multi-vendor, multi-technology, complex business and technology integration environment.

The critical considerations for acing enterprise test management

ETMS is a requisite for strategies that prove early and consistent business return on investment (ROI). As such, they’re part of a comprehensive approach to planning, organizing, controlling, and monitoring the enterprise-wide testing process to deliver high-quality software. As you’re setting your ETMS and developing its playbook, consider the following guardrails.

1. Define your organizational blueprint: Your team’s capability and caliber are paramount in strategy delivery; when enabled by an effective ETMS, they’ll be best positioned to drive toward your strategy’s intended value. To unlock the full potential of your people, you’ll need a thoughtful organizational blueprint that supports employees, sets the structural guardrails for your ETMS, and defines how the testing function will work within the broader matrixed organization. If you’re in a large organization, consider a centralized Testing CoE. We’ve observed this approach pays substantial dividends in the form of quality and employee adoption rates. It also ensures testing standardization and efficiency through re-use of best-in-class practices across the organization. Conversely, a decentralized approach—guided by a steering committee that standardizes testing processes—tends to work well in smaller enterprises. As you’re defining the organizational strategy, consider:

• The test organization's structure.
• How your testing function aligns with the rest of the enterprise.
• Roles and accountabilities within the testing organization.

2. Develop an adaptive testing strategy: With software development practices constantly evolving, keep a pulse on the latest, industry-standard techniques and testing approaches. Within your ETMS, document the approved test approaches and provide guidelines that help teams tailor them to best meet project needs. For example, consider behavior-driven development (BDD), test-driven development (TDD), or acceptance test-driven development (ATDD).

Create guidelines that help project testing teams choose and tailor the optimal testing approach to fit the project needs and timelines, accounting for functional and non-functional testing scenarios. For instance, anticipate forms of testing such as unit testing, integration testing, system testing, acceptance testing, regression testing, performance testing, security testing, accessibility testing, and usability testing.

3. Set your test coverage metric: This metric quantifies testing effectiveness by representing the percentage of the total application code being tested within a given test case. By defining this metric for your enterprise, you can standardize test coverage and introduce guidelines for adapting based on the needs of a specific project.

4. Embrace a continuous delivery mindset: Test automation supports continuous testing—an imperative for quality and value delivery. Continuous testing promotes repeatability, consistency, and reliability in test results, optimizes the testing cycle, and cuts down on excess time and expense. Ultimately, it frees employees’ time for more complex, higher value work. Set your standards for test automation, enterprise-approved test automation frameworks, and minimum test automation script requirements.

5. Manage your test environment, test data, and test cases: Complex, often multi-vendor, environments can hinder project progress, especially in ongoing test management. Fortunately, there is an opportunity to create efficiency through the following ETMS components:

• Establish an optimal test environment. Define approved options for your test environment, accounting for cloud and hybrid environments, budgeting, and cost structures. Plan for different testing scenarios before engaging vendors. After all, the last thing you need to worry about when practicing Agile delivery is standing up the test environments to support it.
• Manage your data with intention. A thoughtful approach to test data is vital in a sound testing environment; after all, you’re likely working with ample volumes of test data and need to consider always-changing data security and compliance standards. Define your approach for creating, reusing, and managing the test data across phases of testing. Consider including synthetic data generation, data-masking needs and practices, automated test data creation, and even recycling test data.
• Determine your approach to test case management. A thoughtful approach to test case management gives your teams the guardrails they need to tailor test case management practices to project needs. Define your test case structure, test case granularity, manual and automation test suite management, test result management, test case management tools, each test’s degree of independence, relationships between test cases, the alignment of test cases to the requirements traceability matrix (RTM), and your approach to integrating different test levels.

6. Keep a pulse on quality, issues, and defects: Determine the process and infrastructure to manage defects and problems. A sound process strengthens the feedback loop and ensures your test management techniques are improving continuously.

In addition to defects, you'll also need to actively manage deliverable quality throughout the stages of a project. Pre-define a set of Quality Gates that include entry and exit acceptance criteria for deliverables, standard SLAs, and quality metrics. In a multi-vendor environment, a formal approach to quality maintains cohesion and consistency across vendor deliverables.

7. Enable people to activate your ETMS: Define and develop the tools, process standards, best practices, estimation guidelines, accelerators, and templates that will help teams get up and running with your ETMS. Then, develop a detailed coordination and communication plan to integrate ETMS processes and protocols within your development and DevOps teams.

A strong enterprise test management approach is a strategic mandate for HHS leaders—a requisite when instilling ways of working that drive ROI from new digital solutions. With an adaptive, scalable, and comprehensive ETMS, you’ll ensure IT teams deliver quality, value, and outcomes, benefitting customers and your workforce alike.