Myung-Hwa recently posted a short blurb to introduce her BlogTracks blog:
I am currently taking a comps. I have read my books and articels about the small grouop collaborative learning. Today, I read instructors’ role and tools. I am thinking about providinge a tool to my research participants in my dissertation. but I do not know if they can use the tool. So, I almost decide a courseware system that an instructor provides. Do you think WebCT or blackboard is a collaborative tool?
I thought her question at the end was interesting. Is WebCT and Blackboard a collaborative tool? Has anyone really studied this issue? This is something that I have been interested in lately, which is studying more closely the impact from using CMS tools in higher education. Richard Clark disciples argue that technology does not impact learning, but I think most educational researchers would now say that this argument is behind us. Pedagogy will always be more important than technology, but technology does impact what kinds of pedagogy you can, or are more likely, to employ. So technologies like CMS tools are very important, and it is critical that institutions don’t employ CMS technologies without giving serious thought to what kinds of pedagogies they will promote.
My role in this Blog Tracks is mostly to discuss the historical development of the literature and to look at new patterns in the publication of current research. An interesting pattern that I have found is that there is surprisingly very, very little research done about the impact of using a course management tool in higher education, and the research that has been completed is very weak and not very comprehensive and more along the lines of “did this class like using WebCT? We found that they did like it” or something else that really doesn’t tell us much.
In a review of the literature that I did recently for an article, I found several small research studies reporting that using a CMS can be helpful for improving communication and collaboration in a course (Hutchins, 2001; Pollack, 2003); increasing student preparation for class and improving the quality of in-class time (Massimo, 2003); enhancing class lectures and feedback to students about grades (Morgan, 2003); giving students greater access to materials (Yip, 2004); and improving learning in other ways (Klecker, 2002). However, other studies have found no significant difference between the grades of students using a CMS and students who did not (Vessell, 2001), and that the benefits of using a CMS can be counter-balanced by many flaws in the software, causing slowness or instability (Dutton, Cheong, & Park, 2004).
In my search of several of the major databases in the fall of 2005 (ERIC, Education Full Text, WebSPIRS, PsychInfo and Ingenta) I found 164 published articles that mentioned course management systems, Blackboard, WebCT, Moodle, CMS, or other similar terms in the abstracts. But of these, only 74 appeared to be data-driven articles, and most of these were quick evaluations of how a CMS impacted a particular class or context. Less than 10 studies seemed to attempt a more general evaluation of the impact from using a CMS over multiple contexts, such as multiple university departments.
Two of the more extensive evaluations of CMS technologies have been completed by the Educause Center for Applied Research (Morgan, 2003; ECAR, 2005). In these reports, the authors have used survey research to conclude there are many positive effects from using course management systems, and that the majority of instructors and students are satisfied with these technologies. However I’m a little suspicious of these reports because I don’t know if they had completely objective purposes.
So I argue that we haven’t really done a very good job of studying what happens when a CMS tool is implemented university-wide, as has happened to thousands of institutions across the country. It is becoming more and more imperative that we DO study the effect of using CMS tools because now over 95% of colleges and universities use some form of e-learning system (Pollack, 2003), usually an expensive course management system. I’m still quite surprised that schools will easily fork over hundreds of thousands of dollars for a CMS without extensive research into whether having that CMS will actually improve learning. In fact I’m more than surprised, as a taxpayer I’m very upset about it!
To close, I pose these questions for discussion, and will share my thoughts on them in a later post, if I remember :-).
1. Why isn’t more research conducted about CMS technologies?
2. What kind of research should we be conducting about CMSs? What outcomes? What measures?
3. Do you agree with me that we’re making a mistake by ignoring CMSs in our educational research?
4. Have we missed the opportunity to really see what happens from implementing a CMS because the tools are now almost as ubiquitous as word processing at some universities?
Note: Some of this material was taken, in a couple of places verbatim, from an article I am preparing for publication. I’d post the whole article here if it wasn’t the case that some journals would consider that to be a “prior publishing” of the article and the article would then be disqualified from further publication. That’s ridiculous if you ask me, but I don’t make the rules!
References
Dutton, W. H., Cheong, P. H., & Park, N. (2004). The social shaping of a virtual learning
environment: The case of a university-wide course management system. Electronic
Journal of e-Learning, 2(1), 69-80.
Educause Center for Applied Research (ECAR). (2005). ECAR Study of Students and
Information Technology, 2005: Convenience, Connection, Control, and Learning.
Accessed February 24, 2006, from http://www.educause.edu/ers0506.
Hutchins, H. M. (2001). Enhancing the business communication course through WebCT.
Business Communications Quarterly, 64(3), 87-95.
Klecker, B. M. (2002). Evaluation of electronic Blackboard enhancement of a graduate course in
school counseling. Paper presented at the conference for the Mid-South Educational
Research Association, held at Chattanooga, TN.
Massimo, V. S. (2003). Integrating the WebCT discussion feature into social work courses: An
assessment focused on pedagogy and practicality. Journal of Technology in Human
Services. 22(1), 49-64.
Morgan, G. (2003). Faculty use of course management systems. Denver: Educause Center for
Applied Research.
Pollack, T. A. (2003). Using a course management system to improve instruction. Paper
presented at the annual conference of the Association of Small Computer Users in Education, held at Myrtle Beach, South Carolina.
Yip, M. C. W. (2004). Using WebCT to teach courses online. British Journal of Educational
Technology, 35(4), 497-501.
technorati tags:AECT2006, AECT, Blogtrack, CMS, distance, education, research
Blogged with Flock