NRC releases data on CS program rankings

By Juan Antonio Osuna
CRA Staff

The Computing Research Association has extracted data about computer science Ph.D. programs from a recent National Research Council (NRC) study and assembled an interactive database on its Web server. From this database, users can rank up to 108 Ph.D. programs based on any of 20 characteristics, such as female or minority representation, that were reported in the study. Background information and links to the database can be found at http://www.cra.org/NRCStudy/.

Released Sept. 12, the NRC report offers the most comprehensive assessment of research-doctorate programs ever, examining the quality of scholars, teaching effectiveness and other characteristics of more than 3,600 doctoral programs in 41 fields at 27 universities nationwide.

In 1982, the NRC conducted a similar study. However, the latest one incorporates new fields and programs, uses refined techniques and provides a benchmark for future studies.

Ratings of CS programs extracted from the September report were based on data collected in 1993 from 221 computer science researchers considered to be knowledgeable about other programs in their field. Ratings on scholarly quality and teaching effectiveness were based solely on reputation as measured by the 1993 survey.

The report also tabulates other data--from 1986 to 1992--on faculty and Ph.D. production, with specific information on women, minorities, publications and financial support. Unlike the CRA Taulbee Survey, the study focuses on comparative characteristics among the 108 CS programs rather than on growth trends. Although the ratings data were juxtaposed with other information, the various data were gathered independently from different sources at various federal agencies.

CRA traditionally has used the NRC's 1982 rankings of Ph.D. programs as a basis for tabulating various statistics for the annual CRA Taulbee Survey. CRA plans to incorporate the recent rankings into this year's Taulbee analysis.

Because the NRC study may have methodological weaknesses that could render unfair comparisons among some programs, CRA urges its Web readers to read the background information carefully. One notable weakness has been documented by CRA Board Member Ed Lazowska of the University of Washington. This information also is on CRA's Web server.

Lazowksa found pitfalls with a data field labeled "Median Years to Doctorate." According to the NRC's hard-copy report, one might think this number represents the number of years it takes to earn a Ph.D. However, it actually represents a time period that begins when a person enters any educational institution after getting a B.A. or B.S. degree. This means it includes the time it takes to get a master's degree--even if in an unrelated field at another school--as well as any interim time spent working between earning a master's degree and earning a Ph.D.

Using this definition, the average student takes 8.68 years to obtain a Ph.D. in computer science, far more than most researchers would expect. These numbers have drawn numerous inquiries from the community and serve to remind us of the pitfalls of misinterpreting survey results.

Despite these weaknesses, CRA believes the NRC data to be useful and unique enough to warrant attention. We, with the help of the community, will continue to serve as a gateway for such information while remaining aware of potential pitfalls.

The study was sponsored by the Conference Board of Associated Councils and funded by the Ford, Andrew W. Mellon, Alfred P. Sloan and the William and Flora Hewlett foundations and the National Academy of Sciences. The co-chairs of the Committee for the Study of Research-Doctorate Programs in the United States are Marvin L. Goldberger of the University of California at San Diego and Brendan Maher of Harvard University.

Copies of Research-Doctorate Programs in the United States: Continuity and Change are available from the National Academy Press at 2101 Constitution Ave. NW, Washington, DC 20418; tel. 202-334-3313 or 800-624-6242. The cost of the report is $59.95 (prepaid) plus shipping charges of $4 for the first copy and 50 cents for each additional copy.