An Assessment of a MOOC

I'm a member of Indiana University's "teaching academy" (the Faculty Colloquium on Excellence in Teaching), and I was asked to do a presentation on MOOCs at this year's gathering in May.  As part of the background for that, I enrolled in a MOOC—“Generating the Wealth of Nations”—through Coursera, taught by an economist at the University of Melbourne.  (My intention had been to avoid economics courses, as I did not want to be constantly critiquing the choice of material and approach; however, the availability of courses in the time frame I needed led me to this one.)  The course has finished, and here are some data from it, from a end-of-course email from the instructor:

“As we reach the end of the course, I thought you might be interested to know some statistics on participation and outcomes.  At the end of last week, out of the 28,922 enrollments in Generating the Wealth of Nations, 12,197 people had actually participated in the course at some stage, and in the last week 1,935 had been active.  There had been 164,946 viewings of individual lecture videos.  On the Discussion Forum there were 423 threads, 2370 posts and 1413 comments.  About 700 of you completed the first assessment (average grade was 2.86), and about 500 the second assessment (average grade 3.09).”

NOTE:  To complete the course, a participant had to complete all three assessments with a “satisfactory” rating (2 or 3 on a 1—low—to 3—high—scale).  Assuming that all of the people who completed the second assessment also completed the first one, and that everyone who completed the first two assessments also completed the third, we have about 500 completions out of (take your pick) 28,922 initial registrants of 12,197 actual participants.  That’s somewhere between 1.7% and 4.1% (and about 25% of those active during the last week). 

Beyond that.  The course was basically a talking-head-over-Power-Point-slides.  Of its type, the lectures were well-prepared, well-sourced and documented, and the slides were generally good.  The course had an extensive set of suggested readings (I downloaded all of the suggested readings that were available on-line—108 of them—and there were many more that were not available, and actually read about half of them).  The assessment protocol was to complete three written assignments, which were peer-assessed and "graded" on a 1 (low) to 3 (high) scale.  To receive a certificate of completion, you had to complete all three assessments with a score of 2 or 3.  While I was (very) skeptical of the peer assessment component of the course, my experience with is was at least satisfactory.  The (anonymous) participants who scored my (anonymous to them) work seemed to have read them carefully, applied the rubrics provided reasonably well, and even made some cogent comments.*

I should say something about the discussion forums.  They were not moderated, although they were monitored (occasionally either the instructor or one of the course assistants would make an appearance).  I did not read all 2,370 posts, but I did participate fairly actively.  In general, although a high percentage of the posts (about half, maybe more) were not based on any evidence presented by the poster, but rather on unsupported assertion and opinion, the tone was generally civil, and several of the posters were well-informed and making an effort to provide information and interpretation.  I thought that, under the circumstances, this was a creditable performance by the participants.

Overall, for a free, non-credit course, I would judge my experience to have been generally positive.

 

*An example of one of the assessments I wrote:

 

1. a. Data collected and ratio calculated                 3
b. Description of whether/how it changed         3

2. a. Data on 3 indicators
b. Reasonable explanation of why these                       3
Reasonable explanation of which is most valuable      3

3. a. Is the description of the change in GDP relative to US
accurate                                                                                                3
b. Is it interpreted using catch-up/convergence/structural change?     3
Is the information to make this assessment correctly described?        3

c. If there is not an increase, is there an understanding of
convergence and structural change?                                                    3
Are barriers plausible and is evidence of one of these
barriers presented?                                                                               3

4. a. Is there an understanding and application of institutions/
culture/geography/contact with success (only 1 needed)?                  3
b. Evidence to support the argument                                                   3

Comments.

Nice presentation of the relative real GDP per capita data.

I’m not necessarily sure I’d describe Italy’s income distribution as “more just;” clearly, “more equal” is an accurate, and less value-loaded term.  Before we can evaluate the meaning of this, we need to have a little more on why (and to what extent) a more equal distribution is better.  The life expectancy data is important and nicely interpreted.  Also, the data on expenditure on innovation seems quite important.  It might be useful to provide a little more information about what that measure means, though.

Is Graph A.1 Italian RGDP pc relative to the US or something else?  Could part of the immediate post-WW2 growth be catch-up?  Good mention of the Marshall Plan.  The shift out of agriculture is also important, so it would be nice to have seen the extent of it.

The importance of the EEC is nice to see mentioned.  How about the Euro, though?  One way to think about the impact of the EEC would be to look at how Italian trade with the other members of the EEC changed in the post-WW2 period.  Were exports to and  imports from the EEC nations growing?  In what other ways did the EEC lead to increased integration between Italy and the rest of Europe?

Overall: 3