Tuesday, April 21, 2015

Deciphering the MOOC ecosystem: what makes successful MOOCs

Deciphering the MOOC ecosystem: what makes successful MOOCs



Transcript

  • 1. DECIPHERING THE MASSIVE OPEN ONLINE COURSES (MOOCS) ECOSYSTEM: WHAT MAKES SUCCESSFUL MOOCS? Ido Roll, Will Engle, Simon Bates! @hummus_monster @infology @simonpbates @CTLT_UBC! Centre for Teaching, Learning, and Technology! University of British Columbia 1
  • 2. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 Please raise your hand if you have pre-teen child(ren) at home.! Please keep it up if you know where the following sentence is taken from:! “The snow glows white on the mountain tonight” 2
  • 3. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 LEARNING FROM VIDEO ! IS POSSIBLE 3 If someone wants to learn, you cannot stop them.! @eric_mazur #STLHE2014
  • 4. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WE KNOW A LOT ABOUT UNIVERSITY TEACHING What works?! Small classrooms! Interactivity! With material! With peers! With instructors! Feedback! … 4 MOOCs
  • 5. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 5 Who are ! our learners? What defines ! success? What course design supports that?
  • 6. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 CONTEXT Four UBC courses (Spring/Summer 2013)! Climate Literacy; Useful Genetics; Program Design; Game Theory! 330,000 learners registered. ! Five Edinburgh courses (Winter/Spring 2013)! Introduction to Philosophy; Artificial Intelligence; Astrobiology; Nutrition; Critical Thinking.! 309,000 learners registered.! https://www.era.lib.ed.ac.uk/handle/1842/6683! 6
  • 7. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 VIDEO PRODUCTION 7 Averagelengthpervideo(min) 0 6 12 18 24 Average # of videos per week 0 4 8 12 16 UBC 1 UBC 2 UBC 3 UBC 4
  • 8. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 8 Who are our learners? What defines ! success? What course design supports that?
  • 9. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 9 Education level %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera High School Diploma Professional School Some College Bachelor’s Degree Master’s Degree Doctorate Degree
  • 10. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHO ARE OUR LEARNERS? General demographics consistent across courses and with Coursera. 10 Ethnicity %responses 0 25 50 75 100 UBC 1 UBC 2 UBC 3 Coursera White or Caucasian South Asian East Asian Other Asian Black or African American Decline to State
  • 11. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 11 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 11 3.6 3.9 United States India Canada United Kingdom Spain Brazil Germany
  • 12. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Does location matter? Are MOOCs “local”? 12 Country of Origin %responses 0 15 30 45 60 UBC overall Coursera Edinburgh 43.9 5.2 United States India Canada United Kingdom Spain Brazil Germany
  • 13. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ROLE OF GEOGRAPHY Break down by course. weighting by overall registration, population of country:! How many more students are local, per 10,000 registrants, per capita?! In Canada, ! ~150 additional local students! ~500 Canadians overall 13 Extralocallearnersper10,000 registrants,1,000,000residents 0 2.5 5 7.5 10 UBC Edinburgh 2.0 1.0 55.7 3.3 5.5 4.8
  • 14. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 14 Who are ! our learners? What defines success? What course design support that?
  • 15. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT DEFINES SUCCESS? Definitions:! Visitors: log in at least once. Our denominator. 15 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  • 16. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2. 16 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  • 17. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. 17 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  • 18. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 18 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  • 19. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Definitions:! Visitors: log in at least once. Our denominator.! Tasters: watch a video on Week 2.! Learners: watch videos all the way through. ! Active learners: problem sets, discussion forums. 19 Activities Videos Website Week 0 1 2 3 4 5 6 7 8
  • 20. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS Metrics:! Attractiveness: convertVisitors to Tasters.! Retention: convert Tasters to Learners! Interactivity: convert Learners to Active Learners. 20 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
  • 21. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 METRICS OF SUCCESS 21 0 0.15 0.3 0.45 0.6 Attractiveness Retention Interactivity 48% 54% 35% 50% 31%29% 52% 58% 36% 35% 57% 24% Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners 17k 13k 49k 85k 2k 3k 4k 16k
  • 22. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 THREE KEY QUESTIONS 22 Who are ! our learners? What defines ! success? What course design support that?
  • 23. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 WHAT AFFECTS CONVERSION RATES? Correlate with:! Average length per video! Average length per week 23 Activities Videos Website Week 0 1 2 3 4 5 6 7 8 Visitors Tasters Learners Active Learners
  • 24. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per week: r = -0.72*, r² = 0.52! 24 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
  • 25. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 ATTRACTIVENESS Visitors -> Tasters: How many of those who visited the website watched Week 2 videos?! By length per video: r = -0.33! 25 %ofVisitorswho arealsoTasters 0% 20% 40% 60% Average length per video 5 8 11 14 17 UBC Edinburgh
  • 26. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per week: r = -0.29! 26 %ofTasterswho arealsoLearners 0% 26.667% 53.333% 80% Overall legth of videos per week (min) 0 45 90 135 180 UBC Edinburgh
  • 27. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 RETENTION Tasters -> Learners: How many of those who watched @ Week 2 also watched @ Week 5?! By length per video: r = 0.55, r² = 0.30 27 %ofTasterswho arealsoLearners 0% 20% 40% 60% 80% Average length per video 5 8 11 14 17 UBC Edinburgh
  • 28. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 INTERACTIVITY How many of the students who watch videos also submit problem sets? 28 %ofvideowatchers whosubmitted problemsets 20% 30% 40% 50% 60% 70% Week 0 2 4 6 8 UBC 1 UBC 2 UBC 3 UBC 4
  • 29. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 SUMMARY Local matters! Courses have unique “signatures”! Structure, e.g., # and length of videos! Engagement, e.g., % active learning.! Overall length of videos is negatively correlated with retention! But length of individual videos does not show such relationship. ! True for both universities 29
  • 30. @hummus_monster @simonpbates @UBC_CTLT #STLHE2014 DESIGN RECOMMENDATIONS Long videos may be okay! Especially later in the course! Keep overall load reasonable.! Active learning is not affected by week-by-week variations! Two-stage courses: attractiveness and retention. 30

Posted by at