Article Critique

Assignment 3, by Zach Tomaszewski

for CIS 703, Fall 2003, taught by Dr. Elizabeth Davidson

Galegher and Kraut have put forth a solid study concerning the effect of computer-mediated communication on group work.

They layout their theoretical constructs quite clearly in the first few pages. They note that they see success in a small-group project as a multi-dimensional variable. Not only must one consider successful completion of the task itself, but also participants' satisfaction and their attachment to other group members.

Galegher and Kraut also admit that there are many types of collaborative projects, but choose to focus here on collaborative writing. This sort of activity occurs frequently in business and professional contexts. They cite past studies that have shown that social presense is important in tasks when uncertainty or conflict is high or when negotiation and clarification about goals is needed. In these cases where high levels of communication are needed, expressive and interactional channels are preferred.

Here is a look at Galegher and Kraut's variables. The mapping from construct to operational definition seems valid.

Galegher and Kraut's study used first-year MBA students from their courses. This raises some small question of external validity--whether such students adequately represent the population of professional people engaged in computer-mediated group work. As most other research in the field assumes, the two groups are probably similar enough. There is also the question of whether students who select to take Galegher's courses are different from other MBA students. Again, this seems unlikely to be a major source of error, and the ease of using Galegher's students outweighed this objection.

The three experimental groups were based on the mode of communication available to them: Face-to-Face, Computer and Phone, and Computer Only. Students were randomly assigned both to groups and to conditions, which means this is an experimental design. Galegher and Kraut did not evenly divide groups--50% of participants were in a Face-to-Face group at any one time--but there seem to be enough participants (117) that this should not be detrimental. It is interesting to note that all students completed a second project with a different group under a different condition. There is the possibility of history effects between the two projects (different levels of other course work later in the term, different subject matter, etc). However, Galegher and Kraut address this, pointing out that results were not significantly different between the two projects. (Indeed, they had even planned that they would be.)

Another concern is that there might be a learning curve to using the computer-mediated technology. After all, the study was done in 1994, when the Web was only beginning to gain popularity. Also, the students used ICoSy, which is not a widely known system. Random assignment helps ensure that computer-savvy students don't all end up in one group. Galegher and Kraut try to limit the effect of learning a new mode of communication by providing a tutorial before the assignment and providing help throughout the experiment. Whether this is sufficient is questionable--students would likely have 20-some more years of experience with face-to-face communication than with ICoSy. However, there seems to be no other viable alternative to studying computer-mediated communication, and so this too is minor objection.

Galegher and Kraut collect time series data (students completed a self-report form each day), overall retrospective data (a survey at the end of the project, and project performance (a grade given by 3 graders blind to the experimental condition). Such data collection methods do rely on subjects accurately reporting. By the end of the day, do students accurately remember how much time they spent and exactly what topics they covered during that day? (Galegher and Kraut also admit that not all students complied with the constraints of their experimental condition, and so this may have affected their reports even more than normal.) Some of this information could have been collected in a more objective format--electronic log files and transcripts from recordings of the face-to-face meetings. However, in the interest of ease of use and keeping in mind that participant satisfaction is largely a subjective experience anyway, I agree that participant questionnaires are acceptable.

Galegher and Kraut's unit of analysis is the group, as this accounted for about 50% of the variance in the individual's responses. I agree with this since we are interested in how the groups perform over different media, and do not want this confounded with how particular individuals like group work or a particular medium. [Yet I am not sure whether this could be construed as an ecological fallacy, since the group scores are derived from individual responses.]

Galegher and Kraut find that planning and revising take more time and involve more group members than drafting. Most of these findings are statistically significant. Also, planning was more efficient in the face-to-face groups, as evidenced by less involvement in planning and a quicker decline in time spent planning.

Computer-mediated participants spent more time communicating overall. As the authors admit, this effect could be due to greater enjoyment of or commitment to the different medium. However, self-reported satisfaction does not support this alternate interpretation.

Performance was the same between the different groups, but satisfaction with the work and with the communication quality as less in the computer-mediated groups. There was also less social interaction, less "liking of others", and less perceived fairness in labor distribution.

Galegher and Kraut use well-known and acceptable statistical methods to prove the significance of their findings.

Overall, I felt this was a well-done study. Had they other goals--such as determining whether a particular company should promote more computer-mediated group work or exploring the group dynamics of using computer-mediated communication channels while doing group work--other setups such as a participatory study or a focus group may have been more apt. However, in determining the effect of a communication medium (in terms of its social presense) on successful group work, this was an apt study.

Galegher, J. and Kraut, R. "Computer-mediated Communication for Intellectual Teamwork: An experiment in group Writing." Information Systems Research. 5:2. (1994): 110-138.