Grading online participation

[Lisa]

Greetings everyone!
I teach an online graduate course and although I've completed several semesters, I'm still struggling with how to evaluate students' online participation.

The general structure of my course (which I run on Ning) is that each week there are 1-2 assignments that require students to participate in an online discussion - either responding to a discussion question in our forum, posting a response to a video, or participating in a small group discussion about a case study. The small group discussions are easiest to evaluate because there is a specific product expected. The students respond to a series of questions and then generate a list of recommendations.

Some of my "discussion" questions are quite specific - but what I find is that then I receive the same answer from all students and there isn't much back and forth, or after a while people start posting "I agree with So-and-So". Rather than posting a repetition of someone else's work, I'd rather them respond, even if that means taking the conversation in a different direction. I encourage students to post from different hypothetical points of view, and that helps a bit. But I'm still left with the question of how to evaluate students' participation and the fuzzier the boundaries are, the more concerned I am that students are unclear about my expectations or may feel that grading is unfair.

I've thought about developing a flexible rubric for online discussion. Does anyone have one that they've used and found to be successful?

Thanks.
Lisa

[Bill Guinee]

Great question. I have had these problems every time that I tried to use a discussion forum for classes (though mine are undergraduate): 1) encouraging active participation; 2) grading. So far, I have been unable to come up with a solution. In my case, I often that the comments devolve into "yeah, me too." I think this may have to do with the types of postings that students are used to seeing on Facebook discussions and elsewhere. As a consequence, I am considering requiring students to each do individual blogs rather than a discussion forum. I am also going to ask each of them to leave comments on their classmates blogs (though this could have the same problem. I look forward to reading the comments on your post as I would love to hear solutions to the problem.

[Nathan Ketsdever]

I watched a YouTube of how a french teacher at Columbia University was doing assessment of web 2.0 and social media in the classroom type assignments.

She said the first assignment she just did a homework grade. As she moved along she developed a rubric and linked to it on her class wiki.

I know that from personal experience of grading in class discussion over a semester that it often comes from the gut. On the flip side, why not use those old rubrics of in-classroom discussion (changing the words to suit the online world). It seems a lot easier than starting from scratch.

Also, there is the option of including it as a mere component of an overall online portfolio grade.

[Sue]

It sounds like you want students to get away from trying to come up with the 'right answer'; you're more interested in them showing some intellectual curiosity.

In thinking about your encouragement to consider other points of view, can you turn that into a game of sorts? For example, the challenge to the class on a particular discussion question is to generate responses from10 different points of view, but any one student is only permitted to offer one point of view. The class may brainstorm what those points of view may be and then divvy up the task of generating the post to meet your rubric's criteria. If a particular post is lacking, other students may respond with additional information that would flesh it out so that, collectively, that point of view does meet the criteria. Individual grades could be given for the original post, and then a collective class grade could be given for coming up with 10 points of view that meet the expectations laid out in the rubric.

Alternatively, what do you think about using a rubric that addresses critical thinking? Here are some examples.

[waggz]

I agree with the other comments that having a solid rubric is probably the most important thing. I've also notice what seems to work is to model the behavior you are expecting from students. You could give an example of what you think would constitute "A" work. An argument that is unique and references the reading or other sources, is well written, etc.

[Michelle Everson]

Hi Lisa,

I teach online statistics courses, and in all the courses I teach, group discussion is a big component of the course. I break students into small discussion groups (roughly four to six students per group), and the groups work together on up to seven or eight assignments (although recently, I've started to change the groups one time during the middle of the semester so students can have the opportunity to work with other students in class).

I grade on effort, not on whether the students are right or wrong in their responses. I try to come up with assignments where there is not necessarily one "right" answer (e.g., for one assignment, I give students a question and ask them to think about how they could design an experiment to answer the question; for another assignment, I want students to talk about their understanding of variability and I ask them to come up with examples of variables we'd expect to have a little, a lot, or no variability). I want students to feel comfortable taking risks in discussion because I know for many students the material is challenging (and, in some cases, not that interesting or motivating to them). I try to provide very clear guidelines about what I expect in each assignment. To earn full credit, students must not only post their own thoughts (or "answers") to the discussion assignment, but they need to come back to the discussion and respond in a meaningful way to at least one peer (or to a question I have posed of the group). I try to give examples of what a "meaningful" response is, and if a student does come back and say something like "I agree with so and so," they will not get full credit for the assignment because in my view, this is not a meaningful response. A meaningful response is one that shows the student has taken the time to read, reflect on, and evaluate what his/her peers have said, and it's something that might challenge others in the group and extend discussion.

I also try to be visible during the discussions. I will cheer the group on if they are doing well, and I will sometimes offer some of my own reflections on what students have posted. I also will ask questions and try to challenge students. I hope that by doing this, I am modeling for students the kinds of things I'd like to see in their own posts.

I don't know if this will be helpful, but I'm uploading a sample of the discussion guidelines I post in one of my courses--my undergraduate introductory course.

Michelle

[Sue]

Attached is a Faculty Focus report titled Asynchronous and Synchronous Learning Tools: 15 Strategies for Engaging Online Students Using Real-time Chat, Threaded Discussions and Blogs. I confess to not having read the report yet, but there may be something of interest in there.

[John Thompson]

Here is an excerpt from my online courses. As stated, I inform my students that such "I agree" type statements are not "acceptable" and therefore won't earn a point and furthermore will be deleted.

PARTICIPATION:

Involvement in collegial exchanges among students and between students and the instructor is essential. Points toward the course grade are earned for participation in online conversations. Besides the earning of points, active participation is in your own best interest...and in the best interests of your classmates. You should keep up with the assigned readings and be prepared to contribute to the discussions. Class discussions offer a positive way to hold you accountable for course readings and to provide a forum for the exchange of ideas. Typically, there will be 3-4 discussions in each week.

You earn one point just for submitting an "acceptable" entry to one of the weekly discussions. "Acceptable" does not mean replying, "That's a good idea," "I agree," "ditto" or "I dunno." A posting needs to be thoughtful and insightful and pertinent to the topic to earn a point for your entry. The "I agree" type of posting will most likely be deleted so everyone does not waste time clicking on them to read. Instead, add something to the conversation - a new idea, an illustration of the concept, an experience that relates to the topic. It is appropriate to send agreements and thanks directly, off-list, to the person via e-mail you wish to acknowledge.

Use the discussions to
. add new, informative and resourceful information or insights to conversations or posts.
. encourage and support one another whenever possible.
. grow and learn from one another.

You can earn a maximum of five participation points in any given week. So you can earn as much as 75 points (5 points per week x 15 weeks) toward your course grade by participating in the discussions throughout the course. (Submitting 75 postings to discussions in the last two days of the course is NOT the expectation nor will you earn more than the five possible points for that last week.)

During the course, the expectation is for you to post a minimum of 75 discussion entries, including a minimum of at least one posting in each discussion - either an original posting or in response to someone else's posting.


You need to submit at least your first acceptable entries to each discussion by Sunday... unless otherwise directed. That way, you're encouraged to participate in the discussions in the beginning of the week. If you were to wait until the last day or two of the week (typically Monday or Tuesday), your course colleagues would not have ample opportunity to read and reply to your postings. You will lose one point (a penalty "ding") for each discussion in which you do not submit at least one acceptable posting by Sunday.

Discussion entries must be submitted before the next week starts (typically on Wednesdays) in order to receive credit.

If you ask a question in your discussion posting, insert a ? at the beginning of your subject line for that posting. The ? will alert others that you have asked a question in your posting.


The expectation is that you will read each discussion posting. There may be an occasional embedded "test" posting that may earn you extra points for replying and/or penalty points if you don't. Read each posting but you don't have to reply to every posting.

If someone asks a question about one of your postings, you're expected to reply. So you need to check back on the discussions after you've contributed postings.

It would be a good idea to check the discussions at least every other day, if not each day. Skipping multiple days will give you a shock when you do log in as your course colleagues and Dr. Thompson will have submitted a considerable number of postings by then and it will take you a lot of time to catch up. Dr. Thompson usually checks the discussions at least once a day.

Dr. Thompson will be an active participant in these discussions, often asking questions that need to be asked, putting a different spin on what someone has said, or acting as the devil's advocate. He will be replying to your entries. And although Dr. Thompson will not reply to each of your discussion postings, rest assured that he reads each discussion posting. In fact, Dr. Thompson has read every single discussion posting in every online course he's ever taught.

You need to be cognizant of using proper English (e.g., spelling, grammar, punctuation) in the course discussions. Be sure to spell key technology terms (e.g., e-mail, PowerPoint, Web site) correctly. But feel free to use e-mail/e-chat abbreviations (e.g., BTW for "by the way"). You can use (actually are encouraged) to use informal language (e.g., contractions) in the discussions. However, inappropriate language will not be tolerated, no matter how passionate you feel about a topic. You can be informal, but still professional.

[Leslie]

I use a rubric that details how points are distributed for an online weekly discussion. Let's say the discussion is worth 35 points each week. Here's the breakdown of how they earn points. BTW quality increases significantly if they follow these requirements - they have to read and understand the text in order to discuss (not "answer") the DQs. I always have open-ended questions and they discuss the issues all week long. I also require a minimum of three days participation.

Here's what I give them (see two attached files). I use this for undergraduate freshman classes and with graduate courses.

[Eduardo Peirano]

Leslie, thanks for sharing. Isn't too much work for professors and students. You say it can take as much as 3 times more work for online students and professors

[Leslie]

The reason it's more work in online classes is that writing takes longer than talking and reading takes longer than speaking. That's the only difference in the time commitment for online and f2f classes. A lot depends on one's reading and typing speed.

It doesn't take me longer to teach online v. f2f in terms of hours per week. I speed read and type over 100 wpm with 95% accuracy. I can get through 25 Eng comp papers in about 4-6 hours. And that's counting writing comments throughout papers and a summary paragraph at the end (I'm a big believer in timely and meaningful feedback on students' work).

[Lisa M Lane]

I have a number of colleagues using Wimba and other methods to give individual feedback to student through voice instead of typing to them. They are very happy with the time-saving and personalized feel doing it this way.

[Eduardo Peirano]

Wimba Voice for Higher Education