A Quality Advisor White Paper by Richard E. Biehl.
Copyright 1991, Data-Oriented Quality Solutions. All rights reserved.


Quality Assurance is a very nebulous topic when discussed as an aspect of the data processing industry. One must discuss what it is, and more importantly, what it is not; both as it is perceived by those in the industry who practice it, and those who provide funding and support for its practice. These different perspectives can be diametrically opposed, and as a result the field of Quality Assurance within the data processing community can appear, on the surface, to be very confused and disorganized.

The conclusion must be that the confusion will only be reduced through an alignment of the theory and practice of Quality Assurance. What QA is expected to do must be better coordinated with what it is typically asked to do. Until this is done even the best QA organizations will struggle with inefficiencies and lack of true direction.


Quality Assurance is defined by its practitioners as a function that defines and implements the processes necessary to produce quality products and services, reviews operational activities within the information systems organization to assure compliance to those quality processes, monitors costs associated with the failure to meet quality objectives, and promotes general quality concepts through training and education. One can break up this very complicated definition into its component parts in order to talk about just what it is that the QA function does, both long-term and day-to-day.

A common theme will run through the entire discussion; namely, that sound processes produce quality products. Quality Assurance professionals are more interested in the processes involved in the application of information technology than in the actual products created by those processes. That is not to say that the products themselves don't deserve a great deal of attention; but QA will focus in on the processes. Other quality management specialties will focus in on other aspects, including the product itself.


In defining and implementing the sound processes that will ultimately be used to produce quality information systems, the Quality Assurance function focuses in on three major areas of concern, namely, the organization's System Development Life Cycle (SDLC), its project planning and control mechanisms, and the technical standards and guidelines against which the quality of those systems will be assessed.

The System Development Life Cycle, in principle, is a set of procedures intended to guide the creation and use of information systems from the moment they appear as a simple idea in someone's mind, until they are permanently retired from service because they are replaced or become unnecessary due to changes in the business environment. In practice, however, potential information systems fail to be recognized by the information systems organization, and QA in particular, until they become an explicit request for service from one or more non-information systems organizations. In practice, an organization's SDLC begins at this point with the definition of the requirements to be served by the proposed system. The procedures in the SDLC are then used, more or less serially, to create the information system using a succession of project phases, each intended to define further detail on the proposed system until it can actually be built and tested during a late phase of the SDLC. Traditionally, once an information system is completed and in use by the target community, the SDLC ends. Many SDLC's contain processes for monitoring and supporting systems after implementation, but are actually rarely used in this capacity. As for the principle that the SDLC stays relevant until the ultimate retirement of the information system, this rarely occurs in practice. For this reason, the SDLC is often referred to as the 'development methodology' for the information systems organization; a distinction that only highlights the difference between the SDLC in principle and the SDLC in practice.

Regardless of the level of comprehension of an organization's SDLC, there is always a requirement to adjust its procedures and organization to the requirements of each individual project. This is the role of the management of the project. Project management is not a role that can be assumed by the Quality Assurance function, but there are areas of project management that are well suited to QA support because of their close ties to the SDLC, namely; project planning and project control. QA, because of it cross-project and cross- organization perspectives, is in an excellent position to offer a project manager support and guidance in the planning of projects. The selection of which tasks and deliverables within a methodology are appropriate to a particular project depends upon a thorough knowledge of the purpose and intent of each task and deliverable. The Quality Assurance analyst, as a specialist, is well suited to assist in this planning. In fact, if one were offered only one chance to get involved in a project from a Quality Assurance perspective, always choose to get involved in the planning of the project. No better opportunity exists to prevent quality problems and defects before they occur.

Project planning also depends, however, upon a thorough understanding of the purpose and intent of the project, including any underlying hidden issues, and the project manager, with a single focused view, is best suited to this need, which is why project management, as a function, could never be assumed by QA directly. The very qualities that make QA such a valuable resource to the project manager, are the qualities that would prevent QA from exercising effective project management.

As the project manager executes the project plan, QA is well situated, again through its thorough knowledge of the intent and purpose of the methodology, to assist in the control of project phase transitions, namely, determining that the requirements of one SDLC phase have been achieved, and agreed to by all involved, before a subsequent phase is initiated.

QA's cross-organization and cross-function perspectives make it invaluable in another area of process definition, this time in an area where its practitioners aren't particularly knowledgeable and experienced, namely, technical standards and guidelines. The vast majority of the staff of a large data processing organization are better situated to write some technical standard or guideline than is the typical Quality Assurance analyst to write any. However, what Quality Assurance offers to the picture is a non-biased view that is most likely to take the needs of the entire organization into account. As facilitator, guide, and explainer of the SDLC, the Quality Assurance function is well qualified to assist the organization in developing and adopting standards and guidelines against which an information system can be measured for conformance to those characteristics that the effected people in the organization value.


It is in the measurement of conformance that Quality Assurance adds additional value to the organization. Just as conformance to user requirements is a standard against which a product can be measured, conformance to the SDLC's requirements is a standard against which a project can be measured. Through facilitating and reviewing project plans, and then monitoring the ongoing progress of any project, Quality Assurance is well positioned to identify potential problems in the project process that might lead to product quality problems later.

Quality Assurance continually monitors projects in this way, always working to assess the value and impact of the requirements of the SDLC, with an eye toward enhancing the methodology wherever and whenever shortfalls are spotted. QA always attempts to allow future projects to benefit from the experiences gained during existing and historical projects.

Through monitoring and reviewing project SDLC deliverables, shortfalls in the process can be identified and corrected before major problems result. As the environment changes, QA must be constantly aware of potential problems in the methodology that would effect projects. An SDLC can never be a static thing. QA must constantly be changing it to reflect lessons learned, changed organizational directions, new technologies and the business opportunities created by them, new approaches to old problems, as well as cultural changes that effect the organization and flow of a project.

A static SDLC quickly loses its appeal to the organization, and projects begin to invent their own means of accomplishing their goals. These individual, relatively uncontrolled, approaches tend to increase costs in the long-term, and directly impact the quality of the product produced.


Efforts to measure costs often focus on costs associated with a lack of quality in some product or service. Having to rewrite an information system because its requirements were incorrectly understood is a serious cost to the organization, and one that can be directly perceived as a quality related cost. However, most costs incurred by the information systems organization relate directly or indirectly to the cost of the products and services being produced.

Quality Assurance concerns itself with measuring and evaluating all of these costs within the information systems organization. The focus tends to be on identifying the relationships between the inputs to the processes and the outputs from the processes. This relates to the quality of the systems, as well as the productivity of the project teams that create them. QA attempts to track the quantity of functions that are produced for the users of the information systems being built. There is a growing trend away from measuring based upon the amount of technology used in the creation of the system; a number with no particular meaning when trying to assess value, and a number that is difficult to benchmark over time as the technologies being used change dramatically.

This minor controversy in measuring the outputs of a process is far overshadowed by the problems associated with the measurements of inputs to the process, the majority of which appear as labor time. In organizational cultures, that tend today to emphasize costs and schedules over quality, there exist disincentives to measure input labor time accurately. Project workers tend to be on fixed salaries organized against a typical 40 hour work week. Since compensation isn't tied to work time, and since an implicit objective is to look as productive as possible, the incentive exists to report no more than 40 hours per week, even if a typical week would actually exceed 50 or 60 hours.

When this cultural factor is coupled with the fact that this type of worker overtime is more likely to occur late in the project life cycle when work tasks are largely individualized, such as with programming and testing, as opposed to early in the life cycle when tasks are more group oriented, such as interviewing users and conducting analysis, it places increasing pressure on project management to reduce the time being spent in requirements definition and analysis in order to get on to the implementation tasks that are perceived to offer greater payoff per unit time.

With all of these measurement biases built into the environment and culture it becomes increasingly difficult to measure and evaluate the costs of implementing information systems. This makes it increasingly difficult to assess where QA should be focusing its effort to improve the quality of the overall process.

Along with increased difficulty, though, it becomes increasingly important that these costs be measured more accurately so that reasonable relationships can be identified between costs and quality so that the latter can be emphasized without completely disrupting the former.


Which actually leads to the paramount issue in data processing Quality Assurance today, namely, the cost versus quality issue. Information services management, both upper and project, are measured and evaluated today based upon their ability to contain and control costs. In theory, Quality Assurance principles state that the lowest cost approach to any effort will always be one that concentrates on quality first. Most resources today are expended on solving problems created by a lack of quality in some interim product or service. In practice, however, management tends not to see such a connection between cost and quality and so concentrates on the cost side of the equation, often with disastrous results.

This is the core problem faced by Quality Assurance professionals: If management really believed that the least cost approach was through the high quality processes, QA could act as guide and facilitator through the process. But because management doesn't really believe, QA is constantly placed in an adversary position of trying to coerce projects into concentrating on quality over costs. The fact that most management professes a belief in quality only makes matters more difficult since their actions clearly don't correspond to their statements.

For this reason Quality Assurance must constantly be promoting quality concepts within the organization. It involves an intensive effort to change the culture of the organization, often from the bottom up. As long as senior management continues to stress cost over quality, QA will need to continue to be active in keeping the quality discussion alive within the organization. QA will be successful when management sets organizational goals that include quality as a high priority.


Every information systems organization operates under one or more objectives at any given time, and the list usually includes something related to quality. The problem, though, is that the quality objective is often written by the Quality Assurance function after someone in management has said "We had better set a quality objective for this year; get QA to write one." As a result, it's never an objective that management can identify with and believe in, and the rest of the organization fails to take the objective seriously because they will rarely be evaluated on the basis of their quality related performance. QA then, as its highest priority on-going challenge, must get senior management involved in the quality process. Management needs to be the creator and owner of the organization's quality objectives. This becomes QA's most important, most difficult, and most frustrating job.


Management should be demanding that the organization do everything it can to meet the customer's true quality requirements. The biggest problem here isn't that we can't meet the requirements, it's that we don't know what the requirements are. Management's concentration on costs and schedules creates too many incentives to advance a project beyond requirements definition as quickly as possible in order to get to the more "cost effective" implementation phases. As a result we often build information systems to meet poorly defined requirements, and then continually enhance and maintain them as the real requirements are learned. We've managed to convince ourselves that the problem is with the business. "Business requirements change so quickly" we say to ourselves as we plan and re-plan the rewriting and upgrading of systems already "completed". We've managed to convince ourselves that the business is at fault, rather than blaming ourselves as an industry for failing to identify and understand requirements before building systems. We define quality as conformance to requirements, and then claim to produce quality systems without even knowing what most of the requirements are. The result is defects in new products that can't be recognized because the requirements aren't yet known.


This creates quite a problem for Quality Assurance professionals in the data processing industry. An objective of the organization should be to produce defect-free information systems, and yet the known defects, those problems that slow every project down in its day-to-day operation, are just the tip of the quality iceberg, the great majority of which lies hidden below the water's surface. Actually eliminating all defects, doing everything right the first time including the identification of requirements, presents a massive technical and cultural challenge to the information systems organization.


Expecting such challenges to be accepted by the organization without the full commitment of everyone involved would be foolish. Everyone in the organization must want quality, and since workers will deliver best against those criteria they are actually measured and rewarded against, quality must become the basis for evaluation and reward in the organization. This means that the perception and belief of management must enthusiastically move toward quality. Quality can't be something to talk about because everyone seems to be talking about it, management must really want, they must demand, quality in the workplace.


With a firm commitment toward quality, working toward quality could become much more exciting. Project teams would much rather produce quality systems than defective systems. The problem has never been that quality wasn't desired. The problem has always been that quality wasn't rewarded. What has been rewarded historically has been cost containment, and schedule acceleration; always at the expense of quality.

Shifting the emphasis toward quality would allow quality work to be rewarded, and so quality work would be done. With management commitment, quality can become self-fulfilling, the more we have, the more we'll want.


In the long-run such a shift would bring about the very characteristics that management is looking for today, lower costs and faster schedules. In the long-run, we would hope to eliminate the need for Quality Control (QC), which is an after- the-fact evaluation of the products produced in order to identify and isolate any defects inherent in the product. QC is only needed because we don't always do things right the first time, and errors need to be spotted. With the success of QA, meaning that we increasingly do things right the first time, we would reduce the need for QC with the ultimate goal of eliminating it completely. Realistically there will always be a need for QC because as people we'll never get to the point that we never introduce defects into our work, but we should be looking for improvements of several orders of magnitude from that that we find acceptable today.

This is what is so frustrating to QA specialists today, the fact that quality is the direct route to the most desired result and yet is perceived as one of the greatest obstacles. It represents a vicious cycle that, when broken by changing attitudes among organizational management, will eventually make quality the leading attribute of every successful organization.


One begins to see that Quality Assurance is a strategic function within the information services organization. Its benefits are long-term and high impact, and are brought about through the introduction of significant changes fairly high up in the organization. Difficulties arise because Quality Assurance organizations are often called upon to perform very tactical activities that interfere with their potential strategic focus.

QA organizations are often expected to perform QC activities. These activities, while necessary in the short-run, get in the way of the long-term activities of QA by consuming an inordinate amount of the allocated resources. And, because the QA effort is reduced, the need for QC fails to reduce over time, creating another vicious cycle. Unless QA can get out of the QC business to address the real problem, long-term management commitment to quality, the problems of quality will never go away.

This translates into a need for the QA organization to be placed into the larger information systems organization in a way that gives it the ability to resist the onslaught of tactical activity. QA benefits from reporting into the highest level management possible, for the higher the level of management, the less tactical the day-to-day perspective is likely to be. Lower level management will have too many short-term incentives to use the QA resources toward immediate tactical needs, as driven by cost and schedule perceptions. Functions that take years to provide payback are too easy to sidetrack for a few months for some shorter-term gain.


An existing distinction must be recognized between Quality Assurance in theory, and Quality Assurance in practice. In theory, QA must promote processes in an organization that sees the value of doing a quality job in a quality way. QA should be a leader among committed followers.

In practice, QA fights a constant battle against those who don't really believe that quality processes lead to quality products. QA must often coerce project staff to perform actions to which they don't feel committed, and that they would rather avoid.

It's a tough situation to be in. By the very existence of the QA organization, management can claim to be doing something about quality. By management's very actions, the role of QA is made ambiguous and difficult. It boils down to a problem that is rather simple to state, yet extremely difficult to solve: Management's actions need to be aligned with their words in such a way that workers will listen to the words as clearly as they react to their actions.

It's a tough problem, yet one that offers significant rewards to the individuals that take it on, and the organizations that see it through. And it takes a particular type of individual to take on such challenges. Individuals who believe strongly in quality, are extremely self-motivated, and can find rewards in indirect successes.

Once management comes to really believe that the quality way is the fastest and cheapest way, we will have entered a new era in data processing. Once practice is brought in line with theory, there's no telling what we'll accomplish.