From Quality Assurance to Total Quality Management: the Future of Automated Test Standardization

By Moshe Moskovich,
Co-founder and Chief Technology Officer,
QualiSystems
Design verification and quality assurance processes are the backbone of successful product development. Whatever the product, the ultimate goals are the same: to reduce development costs and accelerate time to market without affecting product quality.

Companies invest considerable time, resources and money in testing. Costs can reach up to 300% of the total product development budget. In order to reduce test development time and improve test coverage and efficiency, many companies have created automated testing systems using skilled, in-house human resources, or have invested in third-party test automation solutions. Although automated tests developed in-house are tailored to an organization’s specific requirements, this solution suffers from a number of disadvantages.
Hard-coded test scenarios are extremely difficult to maintain, modify and re-use, especially as products are constantly changing. In order to modify tests, engineers must update the code — often an extensive process, particularly as different engineers use different programming languages and methods. The result is a lack of standardization throughout the testing process – from test design and execution through documentation, data collection and storage, data retrieval and analysis.
Lack of standardization significantly impacts data management efficiency, test development time and quality management process as a whole – on an enterprise-wide basis.
Furthermore, developing automated tests with complete test coverage is a major challenge for manufacturers of complex products that incorporate both software and hardware. Full test coverage requires the use of many different types of test equipment from a variety of vendors. Incorporating these into automated tests requires engineers to program complex drivers. Often, if an item of test equipment is replaced, the test needs to be re-coded. This is not only expensive in terms of time and resources, but also impacts on standardization and test environment maintenance.
One way of addressing these issues is to view product testing as an enterprise-wide total quality management system similar in concept to ERP (Enterprise Resource Planning). This combines a top-down system that considers quality control across the entire organization with a bottom-up approach. This approach commences with discrete tests and uses them as building blocks to create automated processes for full test coverage. The result is a total automated quality management system, which underpins the product lifecycle from design through post-manufacture.

Schematic depicting the flow of a total automated quality management system

Test Procedures
A user-oriented tool for test design and development ensures uniformity of UI, test flow, data collection, documentation, test results, and coverage. It removes the need for complex programming of test processes and also saves the cost of skilled resources.
Unified Central Database
This acts as the foundation for the quality management infrastructure. All test data is collected in a standardized manner through the testing processes and automatically stored for easy data aggregation (retrieval, reuse, analysis etc).
Process Control
To ensure the quality of the product, it is not enough to test the product in different stages of its lifecycle (QC), we also need to ensure quality process execution (QA). The system will provide tools to construct the test process logic and flow, and to aid data collection in a standard format when test processes are executed.
Decision Support and BI Systems
Managers and engineers can access real-time and historic test data stored in the central database and use it to monitor quality. In multi-site organizations, where testing is conducted in different locations, managers can track test status remotely, receive real-time updates, and generate aggregation reports.

How would such a system impact on the various roles within a company? The examples below show some typical roles in a large, multi-site organization:

Role	Benefits of Automated Quality Management system
R&D Engineer	» Self-sufficiency in testing tasks » Drastically speed up testing set-ups in unstable environments » Storage,reuse and sharing of knowledge
Process / Production manager	» Online production flow monitoring » Real-time event-handling and alerts » Online quality-related reports management
Corporate Manager	» Quality policy planning » Improved cost control » Continuously improved process auditing
Product Manager	» Online monitoring of product quality » Automated product fault data collection and aggregation » Investigation tools for quality related issues

The paradigm shift from in-house development of semi-automated tests to total quality management is already underway as solutions that address these problems start to enter the market. However, only a comprehensive system that covers all test requirements over the complete product lifecycle can take quality management to this next necessary step.This will also improve and simplify quality management in terms of data collection and analysis, increasing efficiency and further a meliorating quality management.
A natural development of such a system will be to incorporate quality standards libraries as templates into product testing, which help ensure compliance with standards and best practices.
A major advance towards total quality management has been recently made by QualiSystems. QualiSystems’ TestShell suite of solutions addresses the issues of total quality management from both a top-down and bottom-up approach. TestShell covers quality management from an enterprise-level perspective, by offering manufacturers real-time business intelligence in addition to code-free test design, and 24/7 automated test run capabilities.
As product complexity increases, remote and outsourced manufacturing become the norm, and customers continue to demand high quality at low cost. In the near future, not only test automation but total quality management systems will become increasingly essential – particularly in large, global organizations.