CRA-DMP Evaluation Report #1

The Students' Point of View: Issues Involved in Participating in the DMP

II. Immersion in an academic environment: Students' transition to a more sophisticated understanding of academic life

In this section we will first discuss the students' transition into a more sophisticated understanding of the academic world of CS&E through interacting and observing faculty and graduate students and through doing research in the DMP. We will then discuss how the students' experience helped to define their interests and shape their career decisions.

II.A. Learning about the academic environment

As discussed in Section I, many students entered into the program expecting to learn more about graduate school, research, and academic life in order to make decisions about their future career plans. Our interviews indicated that many students felt that they knew little about these academic pursuits prior to their participation in the DMP. We came to understand that through their immersion into the culture of the academic world of CS&E, students made the transition to a more sophisticated understanding of the field. This transition occurred through the following processes: observing their mentor and her interactions with other faculty and graduate students, observing and interacting with other graduate students on a daily basis, and actively participating in a research project.

II.A.1. Learning about graduate school: observing and interacting with graduate students and faculty

II.A.1.a. Being an "honorary graduate student" allowed students to develop an understanding of graduate school life

Most students interacted with faculty and/or graduate students on a daily basis while participating in the DMP. Through these interactions, almost all students stated that they were able to develop a more complete view of graduate school that they would not have had access to through more traditional venues of acquiring information about graduate school. In fact, most students felt that the opportunity to "live the life of a graduate student" was the most positive aspect of the program.

I: What were the most positive aspects of the program for you?

R: Just having the program - being able to go and be sort of an honorary grad student for the summer and see what it was like. And see by watching, instead of just by asking questions. But by being there and sort of watching people do their thing. Figuring out what grad students do. What professors do, when they're not "professoring." That was really helpful because it gave me a much clearer idea of what academia was like.

I: What were the most positive aspects of the program for you?

R: I think the most positive was just being in a research environment just to see what graduate school is like and hearing about what they did in class, what a typical graduate student does I guess.

I: What were the most positive aspects of the program for you?

R: We were essentially graduate students for a summer -- being put in that environment so I knew what I was getting into. So now, since I am going on to graduate school, I know what to expect. I'm not like a scared little freshman as I was when I went into undergrad. I know what to expect. And making the contacts was also very helpful, getting other opinions on what graduate schools are good to go to, getting extra recommendations from other people that are in another school I'm sure really helped me a lot.

Well, for one our mentor is just - she's wonderful. What she did is she put us with some other graduate students, and we've been working with them. We haven't really even been necessarily working with her directly, but we've been working within the lab, which I think is much different than just working on our own. And it really exposed us to a lot of other people. What was really great about it was we had an active and supportive group of students. They were very, very, very helpful, and we have gone out with them and...What's been great about that is being able to go out and meet other people that they hang out with. And we learned a lot about degrees. And the different interests people are in. And how the different labs work. There are different ways that they take their exams. And advisors offer different kinds of support...You know, you're never just working on just what you're doing, you're learning about what everyone else is doing.

Interacting with and observing graduate students and faculty was particularly useful for students from non-Ph.D. granting institutions. Many students from these types of institutions expressed that since they had no contact with graduate students, their experience in the DMP was extremely useful in their formation of a more complete picture of graduate school life.

Many students who did not have the opportunity to interact with graduate students commented that they didn't have a clear idea of what graduate school was about after they completed the program. Since most students entered into the program with the intention of learning about graduate school, these students tended to be disappointed by the lack of interaction with graduate students.

I think the one thing I didn't have was some interaction with grad students. It was not any fault of my mentor. It was just [that] it was over the summer, and [not many] students are usually taking classes over the summer. They had hoped to get another grant through that would bring in a research group, but that one didn't go through. So it was hoped that I would be doing an independent project but also be interacting with students my own age who were also contemplating graduate school, and that kind of didn't pan out.

II.A.1.b. Students learned "strategic" information about applying and choosing graduate schools

Some student participants commented that their experience in the DMP gave them a unique insight to the admissions process and the selection of a graduate school that fit their needs and interests.

And again, that's sort of that whole eye-opening experience about graduate schools that I sort of got. I sort of feel like I'm living in the real world here. And my advisor here and the students here are people who really know what my chances are for graduate school, and can give me up-to-date information [as opposed to the information that I get from] my profs at my home institution, who didn't go to big-name schools, or went to school so long ago, and you know aren't in the mainstream of high-caliber grad schools, or research, or anything. So that's definitely affected my plans in terms of where I'm going to apply and all that sort of thing.

What helped me the most was being able to get some advice on what to do about graduate school. Because I'd been kind of floundering on my own, calling schools and getting information, and I wasn't getting very far! (laugh) So now I know a little bit more about what to look for, and how the program works. I always thought when you applied for an undergraduate degree you go to admission. So, it wasn't real clear to me where you apply to a Masters degree [program] in graduate school. I didn't understand how it works within the department.

II.A.1.c. Students realized they had misconceptions about graduate school

Some students realized that they had misconceptions about graduate school and through this immersion into academic life they were able to acquire a more accurate understanding of graduate school life.

I: What's your impression of graduate school after this summer?

R: You work a lot, but I knew that already. It's not as solitary as I thought that it might be, which is something that I have been worried about.

I: Meaning you work by yourself, in isolation?

R: Exactly.

I: How did you come to that conclusion?

R: Well, working with the other two students in the office and I did see some other grad students around and they didn't seem like they were isolated.

Prior to the DMP, a few students weren't considering going to graduate school because they felt that it would be too expensive. The following student envisioned graduate school in CS&E to be similar to her undergraduate experience, where there was little financial support.

I: So you did talk to [your mentor] about her experiences in graduate school?

R: Yeah.

I: How did that go? Was she helpful?

R: Yeah, a little bit. One of the reasons I'm a little apprehensive about going to graduate school is that I'm planning to get married when I got out of school and I don't see how you can pay for graduate school if you're both going to graduate school because my boyfriend wants to go to graduate school, definitely. But she told me that she and her husband were both going to graduate school when they first got married. She was telling me that in Computer Science, at least at this school, you get paid to go to graduate school. It's not like undergraduate where you're in debt.

After learning that graduate students could be supported throughout their study, this student began to consider graduate school as an option, whereas she had previously not considered it at all.

II.A.1.d. Through interacting with and observing graduate students, students were able to identify with the graduate student experience

Some student participants commented that their interactions with and observations of graduate students gave them an opportunity to see graduate students' motivations for attending graduate school.

Well, it was nice to hear people talking about exactly why they had gone to grad school. And it was nice to hear that not everybody had started out knowing exactly what they wanted to do. And about where they had gone to undergrad school. And just stuff in general.

Through these interactions and observations, some students were able to see whether they "fit in" with the graduate students. Some students who did identify with the graduate students felt more confident that they could succeed in graduate school because they saw other people succeeding who were similar to them.

I: Has participating in the program changed your view of grad school?

R: I guess it's really made me sure that I want to get my Master's because I was still kind of unsure.

I: How did it affect that?

R: I guess I was just kind of scared, like I didn't know what to expect. But just meeting some of the Master's students, that they weren't like, I don't know, they seemed like people I could see myself being in a couple of years or next year.

R: And I got down there, and there was this one girl who's cubby was like right next to mine and we got along great. And she's going for her PhD down there, so it was just great, you know? It's a good influence of watching, you just get to see how people act.

I: Right. How did they act? Did you like things about their experiences, or dislike things?

R: I liked it. I could definitely see myself doing it.

Evaluator Questions Because students who were able to interact with other graduate students on a daily basis developed a more sophisticated understanding of graduate school, should the DMP require the mentor to have some graduate students to be available as a resource for the student?

II.A.2. Learning about the research process: Shifting from a classroom learning style to the self-directed research process

Through their experience in the DMP, many students came to a more complete understanding of the research process. Upon entering the DMP, most students had little experience with research and were accustomed to learning information from their instructors' lectures and structured classroom assignments. Through "doing" research students expressed that they experienced a completely different mode of learning in which they were actively utilizing and applying their knowledge to solve new problems. In this section we will discuss the factors involved in the students making this shift from a structured classroom-style learning to a more self-directed and open-ended research process.

II.A.2.a. Students initially felt overwhelmed when faced with learning how to "do" research coupled with learning a large amount of new information

In addition to having no experience with a self-directed research process, most of the student participants entered into the DMP with little exposure to their mentor's area of research. As a result, many students initially felt overwhelmed by the amount of information they needed to learn to prepare for their project and by having to learn how to "do" research.

I would have liked to have read things on my own. Because in the beginning, everything was new and I felt so overwhelmed. She threw all these big stacks of paper and stuff on my desk and said, "Here. Read this." (both laugh) And I was like, "Oh, my God!" But it was so fun.

I: How was [research] frustrating? Just starting out and working on a new project?

R: Yeah, because we were just thrown into this lab, and they were all doing all this work, and I hardly knew anything about it. I had to learn all this weird stuff about [my mentor's area of research] and this new language, and things... There was a lot to learn when we got in there. But you kind of begin to think, and you start figuring things out.

I liked [research] eventually. At first it was kind of overwhelming because, I was reading things that I really didn't understand because I basically just jumped right into things. Although she made up an itinerary for me to kind of introduce me to all aspects of her field of research. But eventually after I fiddled around and [did] some computer coding on my own, I understood. I gradually began to understand more of what I had read and what I was still reading. So, I mean eventually it was very gratifying to be able to read something and understand what I was reading and not having it completely mystify me.

II.A.2.b. Students came to recognize ways to adapt and work within an open-ended and self-directed research process

Many students came to recognize the process of research as a self-directed inquiry process and commented that in the DMP they could no longer rely on an instructor to teach them or to provide the detailed structure for their learning; rather, they expressed that they were "teaching" themselves.

It was really a change I think for both of us because usually when you're in a class or something they say, "Write a program that multiplies x, y, z." They don't have a set way that you should do it, but they're trying to teach you something. Or, this program has been written before, but what we were writing had never been written before and that was a really new experience for both of us.

Here what I'm doing is just like pretty much my own thing. You come here, they say, "Okay, this is what you have to do here, it's a bunch of reading material -- just read it and do it." And it's just for you, and you have to solve problems yourself pretty much. When you're in school, they take you by your hand, and they tell you, "Okay, read the lesson dah-dah-dah-dah-dah, and do this thing," and they teach you everything. Here you're on your own. I had to learn a new system on my own. Nobody taught me. I had to figure out how to program in it and stuff... and that's very good practice.

R: Being in a research environment really is a lot different from going to classes every day and doing homework.

I: And how is that different?

R: You don't have a set assignment. You have work that you're doing, and you're kind of on your own, and you're trying to figure things out. And your homework is just kind of like, "You have to read this." Or, "Finish this assignment." So it's a lot different way of studying, I think.

I: Did you like that?

R: Yeah, that was fun. (laugh)

I: What about it did you like? Being able to do different things?

R: I think freedom to do your own thing. I mean, you were on your own to do research, in general. Whatever you wanted to do was what you were able to do. You're like your own teacher, kind of. Everything you learned you did on your own.

I expected it to feel like I had a bunch of homework. Especially to feel like I was sort of in school, just not going to class, and I had a bunch of homework to turn in. But it wasn't like that at all. I was much more of an equal with [my mentor]. Instead of going and having her tell me to do something and I'd go home and do it, I'd go, and she'd tell me about a problem - sort of an open problem or a problem that is very hard. So she wanted to generate some test cases to look at them to see if there's any trend that she might be able to use to figure out a conjecture about a problem or something. I thought that was really neat. I like to program, so I'd just go program these things and do them, and it was fun. I'd come back and say, "This is what I did. This is how I did it, and everything."

Through their experience in the DMP, many students learned how to work within the open-ended framework of research. The following student commented that she was learning how to structure her own time in order to achieve a goal, rather than depending upon an external source to provide structure.

I guess having to discipline myself without having a real timetable to stick to. And knowing that I have to get this done, but no one is going to tell me what to do by the end of this day. Because basically I just have this general goal, and so I've had to basically make up my own schedule, and tell myself what to do this day, and what I need to learn, or what questions I need to ask.

Other students came to realize the research benefits of working collaboratively.

I: So then did you and [another DMP student] work specifically together on this project?

R: Yeah, we worked together the entire time.

I: OK. How was that?

R: It was good. It was really fun. We got along really well, so we were lucky. And it was nice -- originally [I] thought we would split off and we would work on different things, but it turned out that we really needed to work at it together. And I think we made a lot fewer errors because we worked on it together, because if one person didn't see it the other person did. So it was good.

I: You said you really needed to work together. Why?

R: ... I think it was helpful just because it was two people looking at the code at the same time. I mean we both had weaknesses and we could reinforce those with the other person's strengths and make better code.

I: Okay. Did you learn anything about the process of doing research?

R: Oh, definitely! A lot! Not only just from doing research myself. And seeing the roadblocks and the different ways people do research. Like I know that I need a supportive group, that I be forced to check in with somebody at a certain point in time and go, "Okay, let's take a look at where we're going from here." Not only in terms of how I personally react to this whole big project idea, and doing research.

Some students realized that they needed to collaborate in order to gain support in coping with the frustrations of adapting to the research process.

It's nice to have [other DMP student participants] there, because it turns out that all of us are frustrated by how slow the research is going. So, I guess that's one of the things that I've learned this summer, which is that research can be slow. So it's nice to discover that you're not the only one who feels that things are moving at a slow pace.

II.A.2.c. Students' confidence increased through "doing" research

By adapting to and being able to work within the open-ended and self-directed research process, many students gained a new sense of confidence. In the following interview excerpt a student described this process.

It was actually an interesting process. That is, doing research. Because at first I just kind of dove in to this research, and I was kind of overwhelmed by everything. But then I just kept on plodding through and reading stuff, and then trying to apply it and eventually I understood what I was reading. I just didn't understand the surface of it. So eventually I started applying what I was reading and doing my own thing instead of just rote regurgitating what I just read. So that was interesting. It was a nice sense of accomplishment, or something like that...At first, I didn't [like research] because there were technical papers that I was reading. And so I didn't really understand what I was reading. It was really difficult for me to go through these papers and try to assimilate what I was reading when I didn't really understand it. But eventually I understood it. So it was nice, I could read these professionally published papers about some high level topic and actually understand it. And know that I've done something, maybe not equivalent but similar.

For many students, this shift from "being taught" to teaching themselves resulted in increased confidence about their understanding of and abilities to succeed in CS&E.

I: Do you feel like you can succeed in graduate school?

R: Yeah. I feel more confident now than I did when I started this project. So yeah. I feel confident that I can do it.

I: Did you have doubts before that you would be able to do it?

R: Although I have been doing well [in school], there was all this stuff, you know in the back of my mind all the time. "Well, I was just being lucky. I made this, because I just got lucky. I had an easy professor. Or, whatever." And those things. But, somehow, maybe it's more than just luck. And this summer, my mentor had said that I did a superb job and everything. So it cannot be luck.

I: So you were able to do this project and realize that some of the work you've been doing has been skill?

R: Yeah, I worked on my own, so whatever I came up with, was with my own doing. I had advice of course. But it was not like another team member rescuing me and not like my project is working out because somebody else had a great idea or whatever. I mean it was all mine really. And I don't think it was just luck.

I accomplished [my project]. I did what I was supposed to do and I think it gave me a lot more confidence in my ability to do something that wasn't an assignment in classes and it seemed like it was supposed to be harder. And I did it. And I think it really boosted my confidence that summer. Which was great.

R: It seemed to me that [the graduate students] were really self-directed and really dedicated to what they were doing. That was a little bit foreign to me.

I: You're talking about being an undergraduate where you do what you have to do in class, is that what you're thinking?

R: Yeah, so I knew that grad students sort of were more self-directed. But just to see people sort of playing with a problem because they really like it and then going after a thesis with it. It's sort of scary to think that if I did that I would need to be that dedicated to a problem and do it. But then I was [as dedicated] to the problem that I worked on last summer, and I had a really good time and also got something out of it. So that made me feel like I could get through grad school and a PhD if I could find another problem that I liked as much.

II.A.3. Learning about the role of faculty: Getting beyond stereotypes

Prior to the DMP, many students' interactions with faculty were limited to the classroom and office hours, and thus many had little or no conception of the faculty role within the university. When students interacted with or even just observed the mentor and other faculty on a daily basis, they were better able to understand the faculty role in the academic setting as well as their lifestyle. The following student commented that the DMP allowed her to acquire a more multi-dimensional view of the role of the faculty in a university.

R: [The DMP] was really helpful because - I mean, looking at it from an undergraduate student's perspective, you have no idea how the professors spend most of their time 'cause all you see them is the three hours a week that they lecture. I mean you know they do lecture and they do office hours, and that's it. I mean, I had no idea, how you have to be on committees and stuff to help improve the curricula and you get little groups, they'll be a couple professors in little groups working on a project. They'll have grad students, and each professor will be in a couple groups, or I don't know how many, working on projects or trying to do the research that will lead to a paper that will lead to hopefully publication or something, and that....

I: You were mainly exposed to this last summer?

R: Yeah.

I: How did that work? Were you invited to these meetings, or it was just through osmosis?

R: Well, I watched.

What stands out about the program for me? Working closely with a professor was a major part of the program. I got to see what sort of things she was doing. What it takes to be a professor. All sorts of projects that [professors] were involved in. And that pretty much gave me a good idea of what would be ahead for me, if I had decided to either go to graduate school or perhaps just for a Master's or for a Ph.D. and maybe regarding becoming a professor.

Previous to the DMP, many students imagined professors' lives to be completely centered around their academic life. Through their daily interactions with their mentor and other faculty, many students were surprised to find out that faculty had interests and activities outside of the academic setting.

I had this stereotype in my mind that most computer science people just did computer science and they had no social life, no families, all they did was computer science all the time. And I found out that almost everybody here has a family. All of the professors. And so I was kind of surprised.

I always thought that professors would be more like, umm...ahh, I don't know. Always involved in school, and being heroes in the classroom. And all that stuff. But they're just like, you know regular people. They have lives. They have their own world outside and a life.

Some students broadened their understanding of the role of a professor through being at a different institution. The following student did not aspire to emulate the professors at her home institution, but upon working closely with her mentor, was able to see a different side of the faculty role.

R: It just looked like a cool thing to do. I don't know, really I was just looking at all of them and they're a lot different than the professors at [my home institution -- a small liberal arts college]. And, I don't know, I came back and I thought about it a lot, and I could see myself doing it.

I: Different meaning they're younger and they might have different ideas?

R: Well, just different in general. I had my one impression and I guess you need another just to confirm whatever opinions you have. Because you only get one impression at one school, and you need to go to another place to see something different.

Many students reacted positively to their new understanding of what faculty do at a university. The following student commented that this new understanding allowed her to recognize professors as real people and gave her the confidence to talk to professors after class.

R: And it's also that after last summer and stuff I just felt more comfortable talking to my professors, going up after class and saying, "I'm confused about something, can you explain it?"

I: And that was partially because of your experience with the mentoring program?

R: Yeah.

I: Because it gave you more confidence to discuss with professors these kind of things?

R: Yeah, sort of, the "faculty are people too" attitude. ... They're not necessarily awe inspiring, but they're people. You can talk to them. It's not like, they're up there, and you can't talk to them even though - I don't know. When you go to classes of 200 people, or whatever, it's like they're sort of moving video tapes or something up there. No, you can go up and talk to them. They actually will respond, and stuff, instead of this weird thing.

In addition to developing the confidence to interact more frequently with faculty about CS&E, many students used their new understanding of the faculty role to define their interests in choosing whether to attend graduate school or to pursue a career in academia.

II.B. DMP career outcomes: Trying graduate school in CS&E on for size helped students to define their interests in graduate school and academia

The experience and knowledge that students developed through their immersion in an academic environment allowed them to define the nature of their career interests. Many came closer to deciding whether they were interested in pursuing graduate studies in CS&E and also whether they were interested in a career in academia. In this section, we will present first student realizations about their interest in pursuing graduate studies and then move to a discussion of student career interests in light of their newly acquired understanding about academic computer science.

II.B.1. Defining interests in graduate school

Through their experience in the DMP, most students were able to discern whether they were interested in attending graduate school. By being immersed in an academic environment, students were able to identify features of graduate life that appealed or did not appeal to them. The following quote reflects student realizations about their new understanding.

Before [the DMP], the decision I was making was not very informed, because I really didn't know. It was just like, I had this idea, "Oh yeah. Graduate school is so cool!" (laughs) But now I know. I know why I want to be there. I know what to expect. I know what it's going to be like.

In this section we will discuss student reactions to their immersive experience in an academic environment and their realization about their interest in graduate school.

II.B.1.a. Graduate school in CS&E is "right" for me

Through doing research and interacting with graduate students and faculty on a daily basis, many students came to realize their interests in graduate school. Some students experienced a change in attitude about the importance of graduate studies

A few students had not considered pursuing graduate studies before being in the DMP and developed a new attitude toward learning and came to realize that they would enjoy continuing their education in CS&E.

When I started my sophomore year [in the DMP] I had no clue what I was going to do when I graduated. I had never even thought about graduate school and continuing my education. It just opened my eyes to a whole, another area where I could just continue my education and learn more things and I think it put me in a different mode when I came back to school too. I mean so many times people in my classes, I don't know why they're at school. I mean they don't put in the effort and they wonder why they don't do well, and then I just think that learning is what it's all about.

Some students came to realize the importance of a graduate school education for specializing in an area of interest, as demonstrated by this 1994 student participant.

I: Has the program affected you today? Is there something about the program that has influenced your choices, career choices?

R: (pause) I'm more likely to go back to graduate school now. I'd definitely say that program emphasized the importance of going to grad school.

I: What is important about going to graduate school?

R: (pause) I think the one really is you need to have a little deeper knowledge. If you're interested in becoming focused on a particular aspect of something, you need to have more education.

With their new knowledge, students had confidence that graduate school "fit" them

Through doing research and interacting with graduate students and faculty, many students realized that they could successfully complete a research project and that graduate school "fit" them. This experience helped these students to define an interest in attending graduate school with the confidence that they could succeed in it.

I: Did this program affect your decision in going or not going?

R: I just finished my sophomore year when I did the program, so I hadn't had to deal with the realities of what I was going to do in two years. So it was more like graduate school is kind of this nebulous idea that I was kicking around. And it definitely showed me that I could really enjoy graduate school. I think I could succeed there. It was something that fit me. So in my case [the DMP] showed me that graduate school was a real possibility.

I: What made you change your mind [about attending graduate school]?

R: Well it wasn't really that I changed my mind. I wasn't really sure what graduate school was all about or what it involved and this showed me what it was about and it showed me that it was something for me.

I: So what is your impression of graduate school, then?

R: Oh, gee, I don't know! It's a place for me. It's where I want to go. And, choosing the right graduate school is going to be important, but now I know better how to do that. And I know that I can go to grad school and make it through and enjoy the time, and get something very, very important out of it.

One 1994 student participant directly related her experience in the DMP to her decision to pursue her Master's degree. This student had not intended to attend graduate school, and worked in a private firm following graduation. However, she did not find her position as challenging as her experience with research in the DMP and after discussion with her DMP mentor, decided to join her mentor's research team as a Master's student.

I: What were the most positive aspects of the program for you?

R: The exposure to people in the computer science program. The work I did definitely gave me a lot of valuable knowledge, and the insight into how research works. I wouldn't be here if I hadn't had that. I wouldn't be in the Master's program if I hadn't had an idea how great research was.

II.B.1.b. Some students came to realize that they were not interested in or ready to commit to pursuing graduate study in CS&E

Through their experience in the DMP, some students came to realize that they were either not interested in pursuing graduate studies or were not prepared to enter into a graduate program. Many of the students who were not interested in pursuing graduate studies in CS&E cited lack of interest in working on an extensive research program and that they needed more direction and structure than research affords. The following student came to this realization after observing a graduate student at her mentor's university.

R: [Doing research] kind of made me sure that I don't want to get my doctorate...

I: Why is that?

R: Because it's just a really huge project, and [the graduate student I worked with] just works away on it, and ... like the third version, and it just seems like from my perspective of this, like of his doctoral project or whatever, it's just neverending. I feel like I would go nuts after four years of working on generally the same thing. I don't think I could deal with it.

Many of the students who expressed that they were not prepared for graduate school indicated that they had not developed an area of interest or specialization to direct their study. These students stated that it was important to have a "passion" for some area of CS&E in order to make graduate school worthwhile and did not see any benefit to pursuing graduate studies without any clear goal.

R: It showed me how graduate school is different from undergraduate [school]. It's a lot less structured. The idea of doing research. The idea of being more on your own. I guess that's the best way I can explain it, it's different.

I: How did you feel about it?

R: I enjoyed it, I thought it was a real good experience.

I: Is that something you're looking for, that kind of an atmosphere?

R: Honestly -- right now strangely enough -- I know this probably wasn't the intent of this whole program but I think it showed me that I am not ready for that just yet. Maybe in a few years I will be, but not just yet. I think that for one thing, to do research and to go to graduate school you really sort of have [to have a] passion for something, maybe some aspect of computer science. And frankly, I haven't found what that is yet. And I think maybe being out there a few years working or reading on my own will help me to find my passion, if there is one.

II.B.2. Pursuing a Career in Academia

Most students felt that their observations and interactions with faculty and their experience with research were accurate indicators of a career in academia. As a result, through their experience in the DMP, many students came to understand whether they would like to work in academia.

II.B.2.a. A career in academia is "right" for me

Many students perceived the role of the faculty in the university to be flexible, fairly autonomous and full of variety. Some students commented that these aspects of the role appealed to them and as a result, they were planning to attend graduate school in order to become a professor.

R: I'm trying to find something that really excites me. Before I ever had the mentor project job, nobody ever paid me to do something that I thought was really neat. I never could be working and then all of a sudden stop and think, "Hey, I don't have to go somewhere and make some money so that I can eat. This is cool, and I really enjoy this, and somebody is paying me to do this!" I don't know why, they're kind of crazy, but I can sit here and play with these partitions, and I don't have to go earn some money to keep a roof over my head because I'm earning money doing something I really think is neat. When I was growing up, the thing I always saw was, the people around me, they had jobs that they hated. I thought that's what you had to have when you grew up, that you had to do - like your work was just something you had to do to eat, and you couldn't have a great time doing it, it just wasn't possible. Then last year when I had that job [at the DMP] I was like, "Wow, somebody really will pay you to do something you like to do!" And that was really exciting. I felt that was really neat.

I: Did it give you any perspective on work and what you can do?

R: Yeah, it made me really excited about going to grad school and computer science and going into teaching.

I: Is that something that helped you decide to go into academia?

R: Yeah, I like the idea of doing a variety of things and also the idea of academia that you teach, you help the students, you work on academic curricula, you're on committees and stuff, you have your office hours, you have your research, you're writing -- and research itself is you're programming or you're writing a paper or you're thinking up ideas or you're working on some proof, or whatever. It's like, you won't be doing the exact same thing. On any given day you spend a couple hours doing research, go to do a lecture, come back, have office hours, where people may or may not show up, do some more research, go to a committee meeting, go talk to somebody from industry about getting money for getting a grant to do something else.

R: I really like the atmosphere of a university too, compared to that out in the industry. I mean I haven't had that much experience out in industry but I really enjoyed being at [my host DMP university] the last couple summers.

I: What's the atmosphere [like]?

R: I was doing my work because I wanted to and it was my work and I worked in an engineering department my freshman summer and it was just, "Here's this task, do it." And it wasn't so much, "Do research and find out. Investigate a problem."

I: So it was more limited?

R: Yeah. I've loved being at [my mentor's university] the last couple of summers and I'd really like to continue my education. And I think I want to be a professor but I don't know if I've given myself the chance to see what the industry is like, and I'll have to figure that out along the way.

R: [My mentor is] this shining model of success, you know? Not a success in terms of like you think of business school success, but success in terms of like, how one would want to live their lives, and things like that.

I: What do you mean by that?

R: Well, she's got two dogs, crazy kids who are- I mean, they're great! ... And, you know, she...had time for vacation, and she's like, "I can't meet at two o'clock today, I've got to take [my daughter] to ballet." All of the reinforcements of the other real reason I want to teach in the college atmosphere is to avoid the 8 to 5, 8 to 7, whatever a typical workday thing. More flexibility in my schedule is very important to me. And I just don't think I could do that sort of punch-in-punch-out kind of physical job stuff. And academic life...while [it may be] more rigorous in other aspects, and [it] has its downfalls, is the kind of thing that I want to do for that reason. It's more flexible when you need it to be.

Some students were able to further define the direction they wanted to take within the field through their experience with research. The following student discussed how her experience conducting research in the DMP inspired her and caused her to re-evaluate an aspect of her career decision.

I was like, "Yeah, I'll do research while I'm in college, and while I'm getting my Ph.D., and that's important and all that sort of stuff, but really I'm interested in getting a job at a small teaching institution." And... I'm not so sure that's actually 100% true anymore. Research here has been really neat. It's been a lot of fun, it's really interesting, especially if you're in a topic that's really interesting. It's really nice seeing how [my mentor] has four or five grad students working with her, full time, and they're each working on a different aspect of her whole big picture problem that she's looking at here. And it's really neat to see how these things all come together. And she's just so excited about it, that you can't help but be excited! It's just like, "This is so great, don't you see?! It's doing this and this is doing that!" And you're like, "Oh my gosh, she's right, you know, this is fairly interesting! This is going to make a difference!" And things like that. And, it is really exciting, and it's a lot of fun, with very big contrast to the research I had done at my home institution. Sort of by myself for my undergraduate honors thesis, and it was a very, very different experience.

II.B.2.b. Some students were ambivalent about a career in academia after completing the program

Some students enjoyed their work in an academic environment during the DMP summer, but were still ambivalent about choosing academic CS&E as a career. Some students had not worked in industry and had no basis of comparison and so were hesitant to decided upon a career in academia without more information. Some students commented that although they enjoyed the freedom and flexibility of an academic schedule, they also had reservations about working in such an unstructured environment.

I liked the fact that you work on your own [in research]. I mean it's your project. You have help, but it's ultimately your project. And also you come in anytime and you can get up at noon if you like and go back to bed at one a.m. And those sorts of things. And then you go to a conference, present papers, and then you get a lot of exposure. And that's nice. I like that. But on the other hand, for the same reason actually, I don't like it because it's open-ended. You don't have a definite assignment, a definite job. And it might not work out. When you have something structured, your boss tells you, "Okay. This is what I expect. This is what I want you to do." So you go and work at it. You go in and you know when you're coming and know when [you're] getting out of work. And that's nice too. And that's something that research doesn't have. So I am really undecided. I mean it has worked out for this summer but I'm not really sure that the same thing would happen if I were to do research for a living.

II.B.2.c. A career in academia is not "right" for me

Some students came to realize that an academic career did not appeal to them. Many of these students expressed that they needed more external direction and structure than a career in research would offer them. These students expressed that they wanted more external structure to frame their work and, in some cases, to motivate them.

I: Did you learn anything new about the process of doing research?

R: I don't know. I guess the only thing that I learned is that it's a lot less structured and a lot less methodical than I expected. And I don't know if it's always like that, maybe it's just like that at [my mentor's university] but that surprised me. I expected to sort of have a schedule of deliverables or whatever and sort of something that you had to stick by, that you had to be at this certain point, you had to progress like to a certain point by a certain date and then at the next date I expect you to be at this point, and it was just really nothing like that. There was no scheduling. There was no itinerary or whatever.

I: How did you feel about that?

R: I guess I work better in a structured environment. I have to admit I like a little bit of bureaucracy.

I: So are there benefits to working for a Master's degree in computer science?

R: I guess I had a lot of problems seeing what I would do with that degree when I was done with it. I mean, if I wanted to teach at the college level I think I would have had a lot of opportunities, but I wasn't totally convinced I would be happy where I was doing research half my day and teaching the other half. I wasn't very convinced. I did another thesis besides the CRA project across junior and senior year at the University. It was a challenging decision for a lot of us who went through the thesis process. Do we want to do research and teach, or do we want to do something else? I'm still not totally positive, but I think right now I'm happy with what I'm doing [at my job].

I: Was that something you learned through the program or through your senior thesis?

R: I think to be in a research environment you have to have a lot of internal motivation, and I'm not sure I had that. I'm one that needs a lot of, I don't want to say approval outside, but I need some kind of measure of how I'm doing, and when you're doing research it's very difficult. Unless you're getting a lot of things published there's not a lot of immediate feedback. In a job you have a little more of that.

Evaluator Point of View We came to realize that by immersing students in an academic environment, the DMP is preparing these students to make knowledgeable career decisions. By giving students an opportunity to experience aspects of graduate school life prior to making their actual career decisions, the DMP is enabling students to make informed decisions about a career and about graduate school. Those students who identify with and enjoy the people and experiences they encounter may realize that graduate school is "right" for them and also be more likely to succeed in graduate school because they are prepared for an academic environment. Others who did not enjoy their research experience may be less inclined to attend graduate school and, in fact, may have been prevented from making a potentially inappropriate career choice. Evaluator Question If students encounter serious problems in the DMP, do they allow this negative experience to be the defining factor in whether they attend graduate school? We are defining "serious problems" as students feeling that their mentor excluded them from professional and personal interactions with her and as a result, encountered problems with the program itself.

Due to the small number of students that had negative experiences in the program, we cannot draw definitive conclusions about this question. Despite this small sample, their experiences raise important questions and issues about the implementation of the DMP and its effects on the students. In subsequent years of this evaluation as more data is gathered we will be able to more completely explore the relationship between the program implementation and its effects.