The National Research Council is once again undertaking a study of Ph.D.
programs at research universities in the United States. The last study, commonly
referred to as the "NRC Rankings," was released in 1995 (http://bob.nap.edu/html/researchdoc).
According to the NRC, the purpose of the current study is primarily to provide
university administrators and faculty with a set of common measures, both
quantitative and reputational. Additional information about the assessment is
available on the Web at: http://www7.nationalacademies.org/bhew/ResDocs_about.html.
The Computing Research Association was invited by the NRC's Committee to
Examine the Methodology for the Assessment of Research-Doctorate Programs (http://www7.nationalacademies.org/bhew/ResDocs_cmte.html)
to provide comments and suggestions on the proposed assessment prior to the
committee's meeting on September 30. Reprinted below is the text of the letter
submitted to the chair of the NRC committee, Dr. Jeremiah P. Ostriker, by CRA
board chair, Jim Foley.
The Computing Research Association (CRA) welcomes the opportunity to
provide input to the National Research Council's assessment of doctoral
programs at research universities. CRA is an association of more than 200
North American academic departments, six computing societies (American
Association for Artificial Intelligence, Association for Computing Machinery,
Canadian Association of Computer Science, IEEE Computer Society, Society for
Industrial and Applied Mathematics, and USENIX Association), and twenty-three
industrial laboratories and institutes all engaging in computing research.
CRA's mission focuses on representing and strengthening research and graduate
education in the computing fields. CRA's members are very aware of the NRC
rankings and the importance of the results.
Your letter asks "…how might the next [NRC] study incorporate
interdisciplinary programs [and] identify emerging areas that were not
included in the 1995 study?" Since 1995, the year the last NRC rankings
were released, computing has experienced great change. The emergence of new
research areas, sub-disciplines, and interdisciplinary fields within the
computer science and engineering fields has reached a critical mass and now
requires a new categorization and new methods of assessment. For example, a
brief survey of CRA member departments finds computing programs housed in nine
different college units from Arts and Sciences to Engineering to Information
Technology and Engineering (attached). Our IT Deans' group includes Deans from
about 40 computing units that report directly to a Provost or similar
official. The names of these units [http://www.cra.org/Activities/itdeans/participants.html]
reflect the breadth of computing. Also, the attached article "Computing
> Computer Science" [see p. 6 at http://www.cra.org/CRN/issues/0204.pdf]
is my own take on the intellectual breadth of computing.
CRA suggests that the committee establish the high-level category
"Computer and Information Science and Engineering" under which
doctoral programs in computer science, computer engineering, information
science, information technology, human-computer interaction, computational
science, etc., would be assessed and ranked. In addition, some
multidisciplinary programs such as bio-informatics and other
"x"-informatics programs may be appropriately housed under this new
Our suggestion mirrors the policy of the National Science Foundation which,
in 1986, reorganized to create CISE-the Directorate for Computer and
Information Science and Engineering-in recognition of computing as an
important cluster of disciplines on a par with the traditional disciplines
housed in the other Directorates of Engineering, Math and the Physical
Sciences, the Biological Sciences, and the Social, Behavioral and Economic
In response to the question of how the next NRC study should assess the
scholarly reputation of doctoral programs, CRA endorses a rigorous evaluation
process that takes into account the differences between computing and other
research disciplines in terms of publication modalities. Attached is the CRA
white paper "Best Practices in Evaluating Computer Scientists and
Engineers for Promotion and Tenure" [http://www.cra.org/reports/tenure_review.html].
As noted in this paper, although "…standard publication is one
indicator of academic achievement, other forms of publication, specifically
conference publication, and the dissemination of artifacts [such as computer
programs and systems] also transmit ideas. Conference publication is both
rigorous and prestigious. Assessing artifacts requires evaluation from
knowledgeable peers. Quantitative measures of impact are possible, but they
may not tell the entire story." CRA strongly recommends that the
committee take these differences in academic culture and procedure into
consideration when judging a program's "scholarly reputation."
In conclusion, CRA recommends that the committee:
- create the new category "Computer and Information Science and
Engineering" to properly capture and classify Ph.D. programs in the
widening areas of computing education and research, and
- specify a rigorous evaluation of computing programs that takes into
account the assessment practices of computer science and engineering.
[end of letter text]
According to the NRC, the first phase of the NRC study will investigate new
measures and methodologies so that the second phase, conducted in 2003-05, may
accurately reflect changes in scholarship and graduate education that have
occurred over the past 20 years.