Excerpts from Chapter 2: Study Design

ELIGIBILITY CRITERIA

As part of its study design, the committee reviewed and approved criteria for determining which programs to include in the study. The first step was to identify those universities eligible for inclusion in the study. The next step was to invite those institutions to add and delete programs from that list. The number of doctoral degrees awarded by a given institution played a crucial role in determining eligibility.

Level of Degree Production

Once "field coverage" was determined, the committee then decided to invite to participate in the study any institution within a field that produced at least three Ph.D.s between 1988 and 1990 and one Ph.D. in 1991 or that had a rating of 2.0 or better in that field in the 1982 study (in the event the institution did not produce one Ph.D. in 1991). Under these criteria, institutions were eligible to participate in the study even if they had only one program in only one of the fields included in the study. Three hundred universities were identified as having at least one doctoral program that met those criteria.

Invitation to Participate in the Study

The committee next wrote to presidents of the eligible universities inviting them to participate in the study. (See Appendix D.) Sixteen universities did not respond to the invitation or declined to participate.³

The committee also asked each university president to identify an individual at the university to serve as the Institutional Coordinator for the study-someone with whom staff could work at succeeding stages of program selection. The Institutional Coordinator (IC) most often was the Graduate Dean at the university. These individuals made a major contribution to the success of this study through their diligence and care in responding to subsequent requests for information.

Program Selection by Participating Institutions

Institutional Coordinators at 284 universities were sent a list of programs at their institutions eligible for inclusion in the study based on the criterion of Ph.D. production described above. A form also was sent to each IC for each eligible program to collect information as outlined in Appendix D. Ten institutions failed to provide information within the timeframe established by the committee.⁴

The committee also invited ICs to nominate programs in one of the fields included in the study in the event that the committee's criteria had overlooked especially strong programs at their institutions.

The committee acknowledges that this procedure may have resulted in the omission of a number of meritorious programs whose representatives have subsequently expressed interest in having been included in the study.⁵ When individuals, early in the process, indicated to the committee that the study did not include a program that they considered eligible, the committee adopted a specific guideline for staff: to correct any errors that may have been introduced in the handling of program information, such as overlooking a program listed by an IC. However, the committee concluded that it was not feasible to correct (or even anticipate) errors of omission or commission that might have occurred at the campus level, and thus directed staff to refrain from modifying lists provided by the ICs.

Perhaps the most frequent question raised by faculty members who corresponded with the committee about eligibility criteria was the issue of including "new" programs whose faculty were clearly strong scholars but which had not yet produced a Ph.D. or fell below the criteria outlined in the earlier section. Again, the committee decided that unless an IC specifically nominated a new program for review, it would not be included in the 1993 study. We would like to point out, however, that future studies of research doctorate programs will undoubtedly include some of the new programs deemed ineligible for the present study. Omission from the list does not signal that a program is "poor" or "not distinguished." It simply means that the program may not have been included because patterns of degree production as recorded by the Doctorate Records File did not identify it as eligible and/or the Institutional Coordinator did not include it in the list of programs to be rated at that institution.

Faculty Lists

Institutional Coordinators were asked to provide a limited amount of information about each program included in the study. (See Appendix D.) A key piece of information was a list of faculty members associated with each doctoral program included in the study. These faculty lists were included in the National Survey of Graduate Faculty an were also used to generate statistics about faculty research and publication activities.⁸

Most ICs compiled faculty lists that reflected the mix of faculty involved in doctoral studies-including staff from other programs in the same department or from other departments on the same campus. Owing to the increasingly multidisciplinary nature of doctoral studies, the way in which ICs approached the task occasionally had the effect of overlooking some faculty members who might otherwise have been included in a program listing. The committee became aware of this problem during the course of conducting a limited number of focus group discussions of sample questionnaires in anticipation of the National Survey of Graduate Faculty, described in the next section. The committee asked staff to check carefully that faculty lists provided by ICs were handled correctly at each stage of data processing and were satisfied that they had done so.⁹ The committee concluded, however, that it was infeasible to introduce changes into faculty lists once they had been processed by the NRC staff.

The interdisciplinary nature of doctoral studies is evident especially in the faculty lists submitted in the Biological Sciences. The committee is aware that many programs are "large" because of the multiple listing of the same faculty in related fields. It is important to understand how differences occurred in the formation of faculty lists in order to guide the interpretation and use of data presented in this report.

Overall Results of Eligibility Determination

At the conclusion of this entire process, a total of 3,634 programs in 41 fields at 274 universities were included in the entire study. This represents about 35 percent more programs than the number included in the 1982 study. About 78,000 faculty members provided training through these programs, and trained about 90 percent of the total number of Ph.D.s produced in these fields between 1986 and 1992, although this ranged from a low of 79 percent in Religion to a high of 98 percent in Electrical Engineering. (See Table 2-2.)

CHARACTERISTICS OF PARTICIPATING INSTITUTIONS

Appendix E lists the institutions participating in the study and compares them with those participating in the 1982 study. Of the 228 institutions in the 1982 study, 214 participated once again in the 1993 update. Another 60 institutions participated in the present analysis for the first time.¹⁰ Most institutions involved in the 1982 study increased the number of programs being reviewed, a reflection in part of the expansion of field coverage from 32 disciplines in 1982 to 41 in 1993.

Of the 274 universities in the 1993 study, 105 were private and 169 were public universities. The Doctorate Records File provides a useful source of information about the year in which an institution awarded its first Ph.D.-or, more precisely, when the first Ph.D. was recorded by the DRF. In keeping with the degree patterns discussed in Chapter 1, about half of the doctoral programs included in the study are located at universities awarding the first Ph.D. before 1930. It is interesting to note, however, that in the Biological Sciences a significant share of research-doctorate programs may be found at institutions awarding the Ph.D. for the first time in 1950 or thereafter. (See Table 2-2.)

The Carnegie Classification system is widely used for categorizing institutions according to the range and number of programs they offer, the number and types of degrees they award, and the amount of federal research funding they receive. The broadest, most research-intensive institutions fall into the following categories: Research Universities I and 11, and Doctoral Universities I and II, with Research Universities I including the largest, most research-intensive institutions.

It is possible to array the 274 institutions participating in this study by that system, and for institutions within the five "broad fields" comprising this study. As Table 2-3 reveals, the widest dispersion by Carnegie category occurs in the Biological Sciences, which have programs in 34 out of 60 institutions classified as "other." For each of the broad fields, a large share of the programs are at institutions in the Research University I and II category.

When considered by Carnegie Classification (Research University I, and so on), a considerable range in research resources is evident among participating institutions, as indicated in Appendix E. Total Federal research and development (R&D) expenditures in fiscal 1992 ranged from a low of $4 million to a high of $215 million among Research I institutions. Fiscal 1992 Federal R&D expenditures seldom exceeded $10 million at the remaining institutions, with a few exceptions.

Another important feature of the doctoral education environment involves access to resources for conducting research. The committee had hoped initially to gather information about specialized collections, museums, nondegree-granting research institutes, and other campus resources, but was unable to do so.¹¹ Instead it has reported basic information about campus libraries. (See Appendix E.) This information, it is hoped, can be used by interested analysts to calculate changes in those measures as reported by the 1982 study committee.¹²

FOOTNOTES

3. California Institute of Integral Studies, Graduate Theological Union, The Juilliard School, Indiana State University-Terre Haute, Long Island University-Brooklyn, Manhattan College, Marquette University, Middlebury College, Midwestern Baptist Theological Seminary, New School for Social Research, Nova University, Peabody Institute-Johns Hopkins, Southwestern Baptist Theological Seminary, U.S. International University, Villanova University, Wright Institute.

4. Caribbean Center for Advanced Studies, Cornell University Medical School, Cleveland State University, University of Dallas, Depaul University, Louisiana Technical, Memphis State University, Oregon Health Sciences, South Dakota State University, Wright State University.

5. The committee and their staff received expressions of concern from representatives in a few fields. For example, faculty members in Astronomy and Astrophysics at one institution were particularly concerned that their program had not been included in the list of programs reviewed. The committee acknowledges that the smaller size of the doctoral programs in this area could have resulted in the omission of otherwise vigorous research-training sites. However, upon discussion, the committee concluded that it was not feasible in the context of the present study to modify the eligibility criteria for one discipline. The committee urges professional societies or other organizations-to extend the work of this committee to include a review of programs not included in this list.

8. Details about the use of these lists are provided later in this chapter.

9. An erroneous questionnaire printing involving two programs at the same institution was corrected by the staff during the course of the survey.

10. This includes Peabody College and the Mayo Graduate School, which previously had been reported as part of their parent institutes in 1982.

11. Owing to the lack of readily available information about institutional resources in this area, a campus inventory would be needed to generate this type of information. The committee considered conducting such an inventory but restrictions of time and resources prevented such an undertaking.

12. The committee recognizes that the 1982 committee utilized a "composite" measure. However, it is possible to access the component statistics and compare them to those reported in Appendix E of this report.