Day 8: Mr. Data

9 09 2009

dataDay 8 was our first round of data collection and the results were eye-opening. You can read the details of this first survey here (link later), but first things first: how did we get informed consent? As you may know, informed consent is a formal statement which tells the student important things like, “You can choose to participate or not participate in this research,” and “If you drop out, we won’t lower your grade,” and “We promise not to inject you with an eerie luminescent substance which might turn you into the Incredible Hulk.” Actually you only have to include that last one if you’re doing medical research.

Informed consent is a really good idea; it came about back in the 70’s when a fair number of psychology researchers were abusing undergrads for research purposes. So now all our research has to be approved before we can do it. Smart for the students, smart for the university.  The traditional way to do it involves paper (see my discussion here about “paper vs plastic”), so you wind up with a pile of papers signed by students.

For this class we handled informed consent electronically using Google Docs. These handy forms were suggested to me by our CIO Kevin Roberts. In short, I was able to send out an electronic form to all my students, and once completed, their information was captured to a spreadsheet. Not only did it capture their responses, but also their log-in information, the time, etc., so I have a very good record of who responded and when. After two email requests, I had to individually harass fifteen of them to finish it up, but I now have all their permissions in a single file. NOTE: I required all students to complete a form either opting in or opting out. Participation in the study was optional, but telling me whether they would participate or not was required.

Now for the data. In this first questionnaire, I was exploring a very simple question: how do electronic quizzes compare to traditional paper quizzes? In particular, I wanted to know which one the students preferred, and why. I’ll highlight a couple of the findings here, and you can read some more details (at a link I hope to post later); hopefully you will read the full story in some top-flight academic journal in a few months.

In short, here’s what we found: an overwhelming majority of the students prefer taking the quizzes on their phones. When we asked them to list advantages of the electronic quiz over paper, there were two major themes: immediate feedback and economy (which includes stuff like not killing trees). These results surprised me, since, as we’ve discussed, our interface in Blackboard is pretty sloppy for the small screen; of course that is my impression, and it’s possible that the students are so used to mousing around on a screen that they didn’t really notice.

They did note some shortcomings, specifically slow downloads/latency and the inability to move backward to previous items. Because our network traffic study says we seem to have plenty of bandwidth, I suspect the bottleneck may be in our Blackboard server; the ability to move backward can be programmed into some quiz packages as an option. My plan is to address these in future quizzes by changing software. I also have to follow up on this study to be sure that the results are really what they seem to be.

At the end of the first quarter, connected quizzing 1, paper & pen 0.




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: