16-T1 Adaptivity in learning and instruction

This course focuses on research and design of adaptive support in learning and instruction. Adaptivity promotes learning, but in order to effectively adapt instructional materials to learners’ needs, insight in a learner’s behavior is necessary. Instructional designers have long since relied on ‘traditional’ measures to gain this insight; examples include interviews, questionnaires, or observations. As most of these measures are taken ‘after the fact’, they cannot be used to adapt the learning and instruction process in real time. Another concern is that these data sources may not be valid. Measures like think-aloud protocols, eye movement registration, or neurocognitive indicators may be superior in this respect, but are equally difficult to implement in real time. This ‘real time’ problem can be solved by using log files which contain computer generated data of learners’ actions in an e-learning environment. Based on these data, a system can immediately determine and implement a next step. This can be selecting a suitable task, presenting useful information, and so forth.

This course revolves around two key issues in the design of adaptive instruction. The first concerns the type of behavior or learner actions that forms the basis for adaptive instruction. So, what is adapted to what? The second issue is how such an adaptive learning process can be designed using educational data mining / learning analytics techniques. Said differently, how can patterns be identified from log file data, and used to realize effective adaptive e-learning?

Course coordinators:
1. Liesbeth Kester (l.kester@uu.nl)
2. Ard Lazonder (a.w.lazonder@utwente.nl
3. Gijsbert Erkens (G.Erkens@uu.nl)

Course Objectives:

Participants will:
- acquire knowledge about the theoretical background of adaptive e-learning;
- gain insight into state-of-the-art empirical works on adaptive e-learning, educational data mining and learning analytics;
- acquire skills in using (large) log files for adaptive e-learning.

Entry level:
Some expertise in the area of e-learning is highly recommended. Participants who lack this experience should familiarize themselves with some state-of-the-art learning environments prior to the course. Examples include Moodle (http://tom.gw.utwente.nl/moodle/course/view.php?id=34), Go-Lab (www.golabz.eu), and WISE (wise.berkeley.edu)

Online preparation
This course starts online with an orientation on the course content. Participants need to study some foundational literature and react in discussion fora on statements in the online environment. Based on the literature review and online discussion, every participant writes an individual blog post about adaptive learning. For the first three f2f meetings, participants study more advanced literature about adaptivity in e-learning and instruction. After these meetings the participants work in pairs on a TED talk about adaptivity. In the last f2f meeting the TED talks are viewed and discussed with the entire group.

F2F meeting1
- Introduction and discussion on the role of adaptivity in participants’ PhD projects.
- Lecture (Liesbeth Kester) on history of research concerning ATI and adaptive systems. What conclusions can be drawn about relevant student characteristics? Why do adaptive systems still not work as well as we would like them to?
- Hands-on experience with an adaptive system
- TED talk preparation

F2F meeting 2
Lecture (Ard Lazonder) on log file analysis. Explanation of log files (what are they, how are they generated, advantages and disadvantages of using log files
- for research) and 4 basic methods for analysis (frequency, duration, transition, sequence) as well as combinations of those.
- Hands-on assignment with real log files. In several steps, participants go from research question to analysis to results. They work in small groups and in between steps there is room for plenary discussion.
- Lecture (Ard Lazonder, Gijsbert Erkens) on learning analytics: the analysis you do as a researcher after the data is generated has to be done beforehand when you design an adaptive system. What is its relationship with adaptive systems? How does its methodology differ? (creating student models real time versus prediction from large data sets) What is the way forward?
- Collaborative assignment during which a log file is created of their actions. These logfiles serve as input for the assignment in meeting 3.
- TED talk preparation

F2F meeting 3
- Lecture (Gijsbert Erkens) focus on how learning analytics can not only help students, but also teachers. By informing the teacher, the teacher can give adapted support. The ideal combination of the computer’s power and the teacher’s human touch?
- Assignment with the self-generated log files from meeting 2. Log files are shuffled throughout the group, so everyone receives an anonymous log file. Participants are asked to think of the question: what variables can we derive from these files? What are indicators of learning? At which moments did it seem that the student needed help? After initial exploration of the data, participants have to find patterns/student profiles. Next, they compare their findings to the experiences of the participant who generated the log file. Do the moments they thought the learner needed help correspond to the learner’s experience? The idea is that participants can compare their predictions to the actual events, because the data is about their own activities.
- TED talk preparation

F2F meeting 4
- Viewing of the TED talks. Short explanation by the makers followed by plenary discussion
- Wrapup of the TED talks and reflection on the blogs written during the preparation phase.
- Course closure (with drinks, of course).

Workload:
- Orientation on the course content in an online learning environment: 10 hours
- Writing a blog post: 8 hours
- Studying the literature for three f2f meetings: 12 hours
- Participation in the f2f meetings: 24 hours
- Preparing and recording a TED talk of 15 minutes in pairs: 30 hours

Dates:
- Online preparation: Wednesday October 5, 2016 – Wednesday October 19, 2016
- F2F meeting 1: Wednesday October 19, 2016
- F2F meeting 2: Thursday October 20, 2016
- F2F meeting 3: Friday October 21, 2016
- F2F meeting 4: Wednesday November 23, 2016

The F2F meetings start at 10h and end at 16h.

Location: Zaal 1, Landelijk Kennisinstituut Cultuureducatie en Amateurkunst (LKCA)
Netherlands Centre of Expertise for Cultural Education and Amateur Arts
Kromme Nieuwegracht 66, Utrecht
Route: http://www.lkca.nl/lkca-english/contact 


Maximum number of participants: 24 participants

Assessment: 
Successful course completion depends on:

- Active participation during the online preparation (discussion posts)
- Quality of the blog
- Attendance in and active participation during the hands-on assignments
- Delivery and quality of the TED talk

Feedback
- Written comments/suggestions during the online discussions in the preparation phase; generally delivered through posts by the lecturers
- Short written feedback on the blog (after meeting 2)
- Immediate oral feedback on performance during the hands-on assignments
- Oral feedback on the TED talk during the fourth f2f meeting

Literature
Below is a list of required and recommended literature. As a minimum requirement, you have to study all required readings and three recommended readings before the start of the course, and use the information as input to the discussions and blog. Which recommended readings you choose is entirely free so you can follow your own interests. (Please note that some recommended readings are required for a particular lecture; studying them thus is a matter of timing).

Required reading (before the course)

Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5/6), 318–331. doi:10.1504/IJTEL.2012.051815

Vandewaetere, M., Desmet, P., & Clarebout, G. (2011). The contribution of learner characteristics in the development of computer-based adaptive learning environments. Computers in Human Behavior, 27, 118-130.

 Recommended reading (before the course)

Asterhan, C. S. C., & Schwarz, B. B. (2010). Assisting the facilitator: Striking a balance between intelligent and human support of computer-mediated discussions. Proceedings of the 2010 Intelligent Tutoring Systems (ITS) conference. Pittsburgh, PA.

Corbalan, G., Kester, L., & Van Merriënboer, J. J. G. (2006). Towards a personalized task selection model with shared instructional control. Instructional Science, 34, 399-422.

Duval, E. (2011). Attention please! Learning analytics for visualization and recommendation. Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 9–17). ACM New York, NY, USA. doi:10.1145/2090116.2090118

Gobert, J. D., Sao Pedro , M., Raziuddin, J., & Baker, R. S. (2013) From log files to assessment metrics: Measuring students' science inquiry skills using educational data mining. Journal of the Learning Sciences, 22, 521-563, doi: 10.1080/10508406.2013.837391

Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education, 79, 28–39. doi:10.1016/j.compedu.2014.07.007

Snow, R. E. (1992). Aptitude theory: Yesterday, today and tomorrow. Educational Psychologist, 27, 5-32.

 

Required reading (during the course)

Lecture 1

Shute, V., & Towle, B. (2003). Adaptive e-learning. Educational Psychologist, 38(2), 105-114.

VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197-221.

Lecture 2

Gobert, J. D., Sao Pedro , M., Raziuddin, J., & Baker, R. S. (2013) From log files to assessment metrics: Measuring students' science inquiry skills using educational data mining. Journal of the Learning Sciences, 22, 521-563, doi: 10.1080/10508406.2013.837391

Lazonder, A. (2014). Log file analysis. In A. Lazonder, J. ter Vrugte, H. Gijlers, S. McKenney & T. de Jong (Eds.), Skills lab syllabus (pp. 18-28). Enschede: University of Twente.

Lecture 3

Asterhan, C. S. C., & Schwarz, B. B. (2010). Assisting the facilitator: Striking a balance between intelligent and human support of computer-mediated discussions. In Proceedings of the 2010 Intelligent Tutoring Systems (ITS) conference. Pittsburgh, PA.

Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education, 79, 28–39. doi:10.1016/j.compedu.2014.07.007