This Briefing shows that prevailing ideas about academic quality are outmoded and inadequate when applied to colleges and universities. It shows that these shortcomings are both logical and empirical in nature and that they do not exist in comparable sectors of the economy. Executive understanding of quality matters because it affects how we write policy, set goals, compensate individuals, present ourselves to the public, and how we measure inputs, processes, outcomes, and impact to convey to others, and with which to manage and improve practices.
Uneasy Tension Between Academics & Quality
More than a decade ago, I addressed the nation’s chief academic officers with a simple message. As used in academic circles, the term “quality” has become a nearly meaningless idea, rooted in the past, and used to defend the received view of academic structure and process.
I was not the first person in higher education to observe this. Forty years ago, Ted Manning, the accomplished and visionary Executive Director of HLC, was fond of pointing out that higher education's claims about quality amounted to little more than hot air. Dr. Manning’s observations – astute when he made them – are no less so today.
Notions of quality are at the heart of the most intractable problems confronting higher education. Among other things, they provide the foundation for federal, state, and institutional policy and practice. Yet, leaders are confused about the meaning of quality and those representing different sectors of higher education lack common ground for definitions, standards, and outcomes. In the end, discussions between sectors amount to arguments in which “quality” is used as a shibboleth, employed to defend a point of view in the absence of good evidence or even a decent vision of the goal.
It is something of an understatement to observe that colleges and universities do not understand, define, measure, manage, or communicate product quality to themselves or to the public in a way that would be considered minimally adequate to those who understand the meaning of quality in other sectors of the economy.
From Then to Now
Forgive me for packing a few centuries of progress in philosophy into a paragraph. It will become clear that doing so is useful to this discussion. Several centuries ago, philosophers attempted to identify the meaning of various properties of our value systems, beauty being one example among many. These scholars reasoned that because people find a flower, sunset, face, or painting “beautiful,” there must be a property which, if understood correctly, can be found in all things we deem "beautiful." This property, whatever it turned out to be, must be "beauty" in the abstract. Twentieth century philosophers abandoned that classic thinking for good reasons. Instead of constructing meaning through philosophical debate, they examined how we actually use such terms. In doing so, we saw - against expectations - that many terms do not share a single criterion that applies to all legitimate uses. For example, the only property all uses (or cases or instances) of "beauty" had in common with each other was the word itself. These insights changed the way we looked at many terms and, especially, value terms. Wittgenstein's "Don't ask for the meaning, ask for the use!" became the useful analytic construct. Indeed, many words in our language derive meaning solely from their context and have no single criterion in common across all uses of the term. [1]
Pretty much everything I just said about “beauty” can be said about “quality” as it applies to higher education. Each context of education can be described along a dozen or more dimensions each of which has meaning that we would want to call "quality." However, the mix of each of those dimensions, and their relative importance within the specific mix, is unique to that instance of education.
Suitability to Purpose – Highest Meaning of Quality
A longer version of this article unpacks a list of progressively more adequate notions of quality. The lower end of this list reflects uses most common in today's colleges and universities. (Click for PDF Table). The list begins with Apodictic Quality, the most ancient among uses of the term (quality is self-evident because of who we are), progresses to a mid-point in which quality appeals to continuous improvement [2], and ends with quality referring to a dimension of suitability to purpose. That is "high quality" means perfectly suitable to the intended purpose (or nearly so).
Empirically and logically, suitability to purpose subsumes and rationalizes other uses of the term "quality."
Example
Automobile quality provides an uncomplicated illustration of how suitability to purpose informs useful definitions of quality and the metrics that follow those definitions. Most of us can list common criteria for automobile product quality. Your list might be slightly different than mine but our lists would be similar. My short list would include the following:
Price
Safety ratings
Fuel economy
Human & luggage capacity
Driver comfort
Passenger comfort
Technical features
Handling
Projected maintenance costs
Style & appearance
Performance
Projected resale value
The Logic of Quality
This list of quality dimensions for automobiles should make it clear that there is no single “highest quality” automobile. Defining quality involves identifying the consumer’s purposes (or uses) and applying them to the list of features and specifications. For one consumer, the highest quality automobile is the one that delivers the best mileage at the lowest cost with above-average safety, a red color, a moon roof, and little or no attention paid to other quality metrics. For another consumer, the highest quality automobile is the highest performing, best looking, full featured car that has a superior maintenance record. Fuel economy is near the bottom of this consumer’s list.
Assuming that these 12 dimensions of quality were equally scrutable and were all deemed important and ranked in order of importance by everyone who purchased an automobile, we would end up with more than 479 million permutations of a high quality automobile for as many consumers. This logic applies to choices in higher education as well.
Why There is No Such Thing as the Highest Quality
It might seem like my argument is grounded in semantics. After all, one or perhaps only a few automobiles will be found at the top of all criteria on the list? In a way, this approach mirrors the errors of the 17th century philosophers. It is not possible to create such a list for the following reasons. Some quality attributes for automobiles form polar dimensions, either empirically or logically. For example, it is not empirically possible that an automobile can have the best acceleration and top speed and also the best fuel economy. The limitations here are grounded in contemporary physics. It is also impossible for a car to be both compact and Spartan, and large and roomy. or to have the best data on road feel and isolation from road feel. These are logical constraints.
There is a less rigorous but no less compelling way to look at the meaning of quality. If there were such a thing as an automobile that somehow, against logic and science, optimized all of the dimensions of quality that apply to automobiles, the market for this machine would be limited to billionaires.
This might be a good time to step back and contrast these points with the rhetoric of quality we commonly hear within the community of higher education. Dr. Manning's words are haunting.
Takeaways for Higher Education
Four important considerations emerge from this example of automobile quality:
Consumers are generally in agreement about what quality can mean when applied to a product or a service that is transparent in the marketplace.
On balance higher education provides a service but our badges acquire some properties of products . . . and we are far from transparent.
As applied to their individual purposes, consumers differ as to which criteria are meaningful to them and what weight should be assigned to each criterion. Each consumer’s purpose will define a unique mix of criteria and weights in determining the meaning of quality. This is as true in higher education as it is anywhere else.
When we think of quality as suitability to purpose, we also introduce questions about whose purpose and, in the case of multiple stakeholders as we see in higher education, how these purposes, all of them potentially conflicting, will be adjudicated for precedence.
Unlike older notions of quality that rest on fuzzy, illogical, or zero sum criteria, it is possible for every institution to deliver high quality. It is done by meeting the purposes of a defined group of consumers.
Higher Education Versus Other Notions of Quality
Some colleagues would have us believe that the notion of quality gets much more complicated, inscrutably mystical even, when applied to higher education. Some even believe that “quality” is what Wittgenstein would have called a private event that can be intuited only by them, and not subject to objective scrutiny. I have even been told by some instructors that no one can measure the quality of what they teach. The conversation deteriorates when I ask, given that, how they manage to assign grades.
Is what there is to mean by “quality” in higher education more complicated than we find in the rest of society? Perhaps only slightly because of its opacity. The meaning of quality in higher education ("the use") is arrived at in same way as the meaning of automobile quality.
Perhaps as with automobiles, although with less certainty, most of us can list common criteria for quality in a specific instance of higher education. Your list might be different than mine but, even in this controversial area, we would recognize each other's lists. Here is my first draft:
Fidelity of content to applicable standards
Alignment of program to need in the marketplace
All-in cost (tuition, fees, hidden fees, discounts)
Mean time to degree (probability of graduating on time)
Opportunity cost of delays in delivering degree [3]
Judgment of merit by potential employers
Convenience (location, blending, pedagogy)
Focus of the degree (applied/practical, theoretical/academic)
Faculty experience in relation to focus (academic cf. professional)
Alignment with learning sciences (goes to benefits and ROI)
The Role of Student Goals in Defining Quality
Questions about goals are questions about purposes and questions about purposes appeal to quality. Because higher education typically involves many stakeholders whose goals are seldom aligned, it becomes important to make decisions about whose goals should be considered and with what precedence in relation to the goals of other stakeholders. Most colleges and universities pay little attention to the specific goals of their students. If they did, students' goals would be assessed and rationalized to a family of metrics at the time of matriculation, and these metrics would be measured and actively managed throughout each student's course of study. They would define success as how well the goals were met.
Which quality metrics will be important to a working adult student who already has a degree in administration and wants to earn a certificate in accounting for the sole purpose of being better informed in her weekly management meetings? Which metrics will be irrelevant? Which criteria will define quality for a student who wants to become a licensed psychologist? How about someone who has no interest in a job and is returning to school purely for personal development? What does quality mean for this person?
Opacity & Optimum Quality
Is it possible for an institution, or department, or program to achieve the highest ratings on all of the above dimensions of quality? The answer is “no” and, as with automobiles, the constraints are both empirical and logical. This fact holds implications for the definitions of quality sometimes invoked by elite institutions.
The automobile consumer will be able to achieve an optimum definition of quality by taking a test drive or two, and by studying the federally mandated window sticker along with a review from Consumer Reports. No such path exists for consumers of higher education. The historically self-serving structures and practices of higher education providers have prevented the kind of transparency that would facilitate rational consumer decisions.
This opacity exists between not only the institution and its consumers; it exists internally at all levels as well. A college president and her cabinet are in a fortunate minority if they possess real-time objective data on two of the above 10 quality dimensions. One would think that the regulatory and oversight bodies would take affirmative action to require not only gathering these metrics but for using them to improve and for communicating quality to consumers. They do not. For some measures, they support the institution in resisting implementation; for other measures, they are silent.
Roles of Other Stakeholders in Defining Quality
Some will object to the fact that I have centered this discussion on the quality criteria of the consumer while ignoring the definitions of other stakeholders. This is intended. The definitions of other stakeholders are important but they have over-determined higher education to the virtual exclusion of the consumer. It is bad reasoning on several levels to continue to push consumers' definitions to the periphery. .
Making the Switch
Managing quality as suitability to purpose is not difficult. Chances are that it is already being done in some areas of your institution, perhaps in materials acquisition. The steps involved in managing quality as suitability to purpose is the topic of another Executive Briefing. Let me know if you are working on these issues. I will be happy to send additional documents that may be helpful. I'll close with an elaboration of a point made above.
Unlike older notions of quality that posit or allow for a single exemplar of highest quality, it is possible for every institution to deliver high quality. It is done by identifying, managing to, and ultimately by meeting the purposes of a defined group of consumers. This is why it is important that we begin to measure, manage, and assess progress toward students' goals.
Endnotes
[1] Philosophers call this “family resemblance” after the work of Wittgenstein. Think of members of a family in which a few relatives share a nose but not cheeks, others share cheeks but not a nose or chin, and so on. All of these people look related to us yet there is no single visual resemblance that unites them all.
[2] CQI was the organizing principle behind HLC’s AQIP, introduced in the late 1990’s. Unfortunately, AQIP’s requirements for continuous quality improvement have been honored in the breach. Schools that choose AQIP as their accreditation method are approved year-upon-year based on “progress reports” showing that they are getting ready to get ready or having meetings about meeting structure. AQIP has become a refuge for institutions that wish to escape real accountability – the very outcome AQIP was designed to avoid.
[3] The opportunity cost of delays for a nursing degree might begin at $59,000 per year plus and minus other lesser factors. Opportunity cost is often overlooked in determining the returns of a degree yet they can quickly add up to make a degree earned on time at one institution a better buy than a less expensive institution that has a history of not having needed classes on time.