CRA Information Technology Worker Study Released

By Peter Freeman
and William Aspray

As announced in the November 1998 issue of Computing Research News, CRA has undertaken a study to improve the understanding of the supply of and demand for information technology (IT) workers in the United States, as well as the surrounding contextual issues. The report, "The Supply of Information Technology Workers in the United States," was written by Peter Freeman and William Aspray on behalf of the Intersociety Study Group on Information Technology Workers, with financial support from the National Science Foundation. An intensive eight-month effort led to a completed report in April. Individual copies of the report are available upon request from CRA. You can also download a copy from www.cra.org.

There are four major contributions in this study:

1. Evaluation of data. The report identifies and evaluates all the major sources of statistical information relevant to this subject. The study group found that federal data are by far the most important and reliable, but that they have some serious shortcomings related to untimely reporting, occupational descriptions that are out of date and based on ambiguous job titles, and incompatibilities between supply and demand data collected by different agencies.

There are other data sources. However, it is questionable whether data from geographically restricted studies can be generalized to the national level; and many of the national studies done by private organizations have methodological weaknesses.

2. Definition of 'IT Worker.' This report outlines a way of distinguishing IT workers from a much larger class of workers whose jobs are enabled by information technology. One of the problems with the national debate is that IT workers have been treated as a single, undifferentiated mass. However, the phrase "information technology worker" encompasses many different occupations that require a wide array of skills and knowledge. It would be helpful in future discussions to segment the class of IT workers into classes of occupations that have similar levels of knowledge and skill.

3. Description of the Supply System. A detailed description of the supply system is presented, which includes not only majors in twenty different IT-related disciplines at the associate, bachelor's, master's, and doctoral levels, but also many people majoring in science, engineering, business, and even non-technical disciplines who often take some course work in IT subjects. The supply system also includes an increasingly important and rapidly growing continuing education element.

4. Analysis of shortage claims. The report evaluates the question of whether there is a shortage of IT workers in the United States. The study group determined that the data are inadequate to ascertain what mismatch there is, if any, between national supply and demand. Therefore the report makes use of a variety of other quantitative and qualitative kinds of evidence. These include: secondary indicators such as wage growth and labor certificates awarded, based on federal data; quantitative studies specific to geography and private studies on the national level as mentioned previously; anecdotal evidence about how employers have acted in their search to recruit or retain workers, or take alternative solutions such as refusing work or replacing workers by machines; and other kinds of qualitative evidence.

The preponderance of evidence suggests that there is a shortage of IT workers, or at least a tight labor market. None of this evidence has the certainty of a direct count of supply and demand, and without this kind of direct count it is impossible to distinguish an actual shortage from a mere tightness in the labor market.

There are also a number of contextual issues revealed in the report that need to be considered in order to gain a full understanding of the supply of and demand for IT workers.

5. Political context. The study group evaluated the reports by the Information Technology Association of America (ITAA) and the Department of Commerce, as well as the criticism of these reports by the U.S. General Accounting Office (GAO). We agree with GAO that the low response rates are a serious weakness in the ITAA and Commerce reports, but this speaks against the quality of the evidence, not necessarily against the conclusion that there is a shortage. The ITAA and Commerce reports can also be faulted for their narrow focus on recipients of computer science bachelor's degrees when discussing the supply of IT workers.

The legislation providing a temporary increase in the number of temporary visas permitted annually under the H-1B visa program was also reviewed. CRA and the other professional societies participating in this study did not take a position on the H-1B increase when it was being debated in 1998, and it is not our intention to second-guess the program now.

6. Types of demand. This report differentiates two kinds of demand. There is episodic demand, such as this country is experiencing currently as it struggles with the Y2K problem. There is also long-term demand, created by fundamental changes in technology and society.

7. Limitations on action. Even when organizations recognize a mismatch between supply and demand that they would like to overcome, there are sometimes limitations on their ability to act.

A government organization cannot regulate supply and demand; it can only provide incentives, such as fellowships, to encourage students to study for an area of expertise that appears to be in short supply. But it is difficult for a government to stimulate labor supply by just the right amount since the market is constantly changing, knowledge about supply and demand is imperfect and difficult to obtain in a timely fashion, and there are often unforeseen consequences of any government action.

Industry has its own constraints. Companies are forced by short product life and short product development cycles to hire new employees or reassign existing workers in ways that do not require a lot of break-in training before they can be productive.

The traditional higher education system is constrained by its inability to change directions quickly. This results from its limited ability to allocate resources to new or growing disciplines toward a long-term commitment.

8. International considerations. There is a rising international demand for information technology. There is increasing global competition to supply IT products. The United States will have to assure an adequate supply of IT workers if it wants either to retain its world lead in the IT sector, or remain competitive in other industry sectors that rely on information technology.

Other topics. There are several topics that are important to understand adequately the IT workforce issues, but which we could only touch on in this study. A number of groups are underrepresented in the IT workforce and in the educational programs that prepare people for careers as IT workers. These groups include women, Hispanics, African-Americans, and Native Americans. If these groups were represented in the IT workforce in proportion to their representation in the U.S. population, this country would have more than an adequate supply of workers to fill even the most dire estimates of shortage. This study group chose to focus its efforts on other issues that have been less thoroughly investigated. However, some basic information and statistics have been collected here about the issues concerning the participation of women and minorities in the IT field.

It may also be true that older workers are underrepresented in the IT workforce. There is certainly a widespread perception that programming is an activity for the young, and that IT workers tend to get "burned out" and leave the field by the age of 40. The absence of almost any data precluded this from being a major topic of study in this report. The study group looks forward to the examination of this important issue in a forthcoming study by the National Research Council.

Some people are concerned about a seed-corn problem: that the high industrial demand for IT workers is siphoning off too many graduate students and faculty from the universities, leaving an insufficient number to educate the next generation of IT workers. This study detected preliminary signs of a seed-corn problem. The coordinated efforts by government, industry, and academia to solve a seed-corn problem in computer science that occurred around 1980 are recounted in the report.

The authors struggled with the decision whether to include recommendations in the report. The mandate for this study was to provide an understanding of the issues surrounding the supply of and demand for IT workers, not to provide a call for action. In most policy reports the recommendations have primacy and the analysis is included merely in a supporting capacity. The study group did not want the presence of recommendations to undermine the attention paid to the analysis. Also, as a study group, we do not have any particular standing within the government, industrial, or academic sectors from which to recommend actions.

On the other hand, a number of important issues were raised and actions suggested by the study group during the course of the study. Given the wide range of knowledge and experience represented by the study group, we decided it would be useful to put these suggestions forward in the hope that they will stimulate further discussion and action. Mostly, the recommendations identify a problem and a general course of action without trying to be specific about implementation mechanisms. The thirty-seven recommendations are grouped around a small number of issues: data-collection practices, industry-academic cooperation, industry hiring and training practices, certification of educational and training programs, broadening the supply pipeline, improving the research and teaching environment to retain and recruit faculty, and curriculum development.