Navigating Between Lessons I have been reading the discussion and trying the prototypes for Python content delivery with great excitement. On seeing how well you were doing on the content delivery, my imagination took off and I wrote down a whole bunch of brainstorming ideas below on an important independent aspect of an interactive learning environment. Just tonight the "Only Python" thread touched on the area I have been thinking about with the idea of choosing what comes next after evaluating code in the Crunchy Frog. I am interested in the learning model and engine for choosing the next lesson to present. I imagine a nonlinear web of potential next steps and a flexible engine to determine the best next step. I would like to consider carrying on this with like-minded folks in parallel with people working on the delivery environment choices and lesson/example/test content. I see the lesson choosing engine as being essentially independent of the type of lessons being presented, so this work should fit into the larger Shuttleworth vision for computer based learning in general. I have written a rather long document here, with an overview, some more nitty-gritty aspects, and ideas for moving forward. I refer to a web page at the end with more specific implementation ideas. One model is just having a lot of tutorials, and let learners choose which one to do and which one to do after that. When we (hopefully) get an enormous number of independent contributions from our user/learner community dynamically added, completely manual choosing of a next tutorial will be onerous. At a minimum we would like an engine to limit the obvious choices of where to go next, while still letting the learner choose. While some learners may be self aware enough to know when they know one thing well enough and are ready to look for something else, others would like to avoid worrying about that and concentrate on the material, and let an intelligent engine determine when and where they should go next to advance toward their goal most efficiently. The goals tie into a separate issue of motivation and reward addressed in recent threads: figuring what *is* the goal and how to make it exciting. Consider a learner starting from a place that immediately describes a bunch of different application areas that might be interesting. Let learners choose goals by saying "I want to be able to do something like that myself!" -- whether it be as simple as Kirby's neat example of a Mad Lib generator, drawing shapes with a Turtle, or as complicated as a dynamic web site connected to a database, or elaborate gaming. Then let the skill set needed for the selected type of application determine the inventory of skills that the engine guides the learner through. Alternately goals could be specified by some formal body like the Virginia Public Schools. The idea of letting learners choose an interesting application to head for could still apply. Applications would be chosen that include the required competencies, with some extra additions depending on the target application area chosen by the learner. The complete target skill set chosen would then guide the path the engine provides through the lessons. Letting the learner *choose* to take on the elaborated goal should help keep the learner on task and motivated. To keep the rewards from being too remote, a sequence of simpler sample application area targets could be chosen incorporating more and more of the final desired skill sets. Again these chosen goals might add a bit to what the learner needs to absorb and affect the path through the lessons, but the extra motivation should be worth it. Anyway, to allow lessons to fit into a larger, at least partly automated environment, we would need some agreed upon interface for messages passed back and forth between lessons and the lesson choosing engine, or other components. As I started thinking about this, I realized that there could be other useful modules integrating multiple lessons, for instance summarizing and reviewing exposition. I thought of Zelle's text, where the main chapters have a lot of exposition with the actual syntax rules interspersed. The book also has an appendix that just has the new syntax from each chapter in the order of the chapters. If the learner forgets some syntax, and wants to check back, it would be nice to be able to have a window pop up with all the syntax introduced so far, in the order that the learner encountered it, dynamically generated from the particular learners' lesson history. Or for a more general review see summary statements from exposition in their lesson history. Or maybe see a summary of things introduced so far by grouped by some topic tags, ... There would need to be a further consistent interface to the lessons to do this. (I would assume a lesson could omit implementation of one of these extra features, and return None to a message with a request, and have the system behave gracefully.) I'm sure I have not thought about everything in the integration of lessons that could be useful and would need an agreed interface to be able to implement it. The most basic data structures and interfaces would revolve around finely divided skill sets: those that are prerequisites and those that should be learned by the end of a lesson. I would hope we could get all lessons to interact on that level. Low level skills are easiest to categorize, set dependencies for, and automatically test. It will be more of a challenge for higher level learning outcomes, but the complication is likely to be in the lessons, while the basic lesson choosing engine would be similar. The idea of skill sets is of course very general. I would plan to set it up so the model interfaces with the certification system Jeff is working on for the State of Virginia. You could have a very basic engine choosing lessons with just this information. You would have to depend on the learners to determine when they understood things well enough to go on to the next lesson. The next elaboration would be to send testing results in some form from the end of a lesson session to the central engine, indicating if the learner did get the desired competencies, or which subset of them, or that more drill is needed, or this was way too hard for the learner. Possibilities for the engine would be to: go on to the next lesson in the author's prepared sequence if one learning outcome was incompletely attained, go to a drill lesson on that learning outcome jump back to an alternate lesson that goes slower with more examples and smaller steps if the current module only provides exposition/exposure, go to a lesson that includes testing, .... Anyhow, agreeing on at least optional interfaces from lessons to a larger system would be useful. I am interested in joining with others to think about that and report back. - - - - - - - - - - - - - - Nitty Gritty It will of course be important to label skills that are prerequisites and skills demonstrated as an outcome of an exercise. It will be harder to deal with the fuzzier ideas of how many creative steps are needed without explicit experience for a specific challenge vs. measuring how many steps together a learner can confidently handle while feeling challenged. Hopefully as we make the engine more sophisticated, we can measure more individual pedagogical parameters for learners and incorporate them in the engine decisions. Ideally we would not only dynamically measure the needs of a learner as s/he progresses, but also measure and modify the overall response of the system. We first keep track of session histories locally and, with the user's permission, upload them to a central site to hone the engine choosing the best place to go next. For instance: we marked module N as closely following module M, but almost nobody got N on the first try after M -- now mark it as being further off, so when the engine looks for the best module to follow M for a learner challenged appropriately by a certain sized jump, it finds module M1 instead ... or FLAG: we really need to write an intermediate module between M and N! We could measure not only the distance between lessons but the effectiveness of a lesson. We could either flag a lesson as not working or dynamically offer up an alternate lesson that is provided, that leads students to the same goals faster than another. Learning styles are very important and diverse, as I have found with my children and my learners. These should be the basis of user options guiding the choice of modules. For example: introduce examples first vs. rules first; supply little bites vs. a major overview before looking for feedback; written vs. verbal or video presentation, module requires cognitive (grade) level N ... A more sophisticated later approach would be to explicitly test the learner's learning style and then set some of these parameters automatically. There are also many other user chosen options. For example: I like modules written by X, modules approved by Y, modules approved by a learned professor, modules using analogies to Java or C or music or football.... Have an easy interface to add your own lesson and label it in the general taxonomy so it works with the choice engine. It would be neat to say to learners: if you struggled through this, and finally got it, how would you redo it to make it easier for you? Ok, do it for everyone who comes after you! Upload it and label the differences in your presentation from the previous one (i.e. smaller step -- comes before the last one, or more examples provided, or just 'written by X'). A learner may find s/he can relate particularly well to what X writes. On categories: the learned professors can specify a bunch of categories and tags, but the deli.ci.us model shows the power of organic growth: what is a useful tag in practice will stick. If we do not allow this initially, at least we should design a system where this would be a modular generalization to add. If we are looking for user contributions and large organic growth, I think flexible preference/descriptor tags could be very useful. Allow a record of learning via secure external certification and also locally without any special security that would enforce avoidance of funny business. Along the way, the learner should be able to see a display of what skills s/he has mastered and which remain for the current goal (either based on informal feedback and testing or by official certification). Way out in front: For video lessons, which fit some people's learning style, there are very effective commercial systems to automatically store in real time and later compact a movie of what appears on the computer screen + synchronous voice. Modern commercial systems are not large resource hogs. If we are to allow anyone to add video lessons, a free alternative would be good to develop! -------------------- Moving Forward I would like to explore a more general engine since people start in different places, head to different personal goals, and have different speeds and styles of learning and different computer capacities. Also we have discussed an environment that makes user contributions easy. We want to dynamically integrate these contributions. The contributions may not fit into preplanned niches. All this suggest a nonlinear web of potential next steps and a flexible engine based on inputs including those listed above to determine the best next step. If people agree that a "choose the next module" engine like this is useful, I would like to develop and reach agreement on the external interface tied into data structures for lesson dependencies, user preferences, and outcomes from modules. Hopefully I will find others to develop it with me, moving to a more specialized list for implementation details, and report progress back to the main list. Some concepts for this engine might come under well-developed interactive learning theory that I have not seen. At least in the early brainstorming stages I am more of an "imaginer from a clean slate" than "searcher for what is out there to avoid reinventing the wheel". References to the existing "wheels" that you have seen are appreciated. I think there should still be a lot of room for new ideas around open source and a dynamic response to a network community of learners and contributors and having flexible relationships that do not depend on a static, predetermined, limited vision. I have thought about some more detailed design ideas for classes that are likely to be involved. See http://www.edupython.org/tutorialEngine/brainstormingEngineObjects.html The first concrete project I would be interested in interfacing with is the Crunchy Frog. -- Andrew N. Harrington Computer Science Department Undergraduate Program Director Loyola University Chicago http://www.cs.luc.edu/~anh 512B Lewis Towers (office) Office Phone: 312-915-7982 Snail mail to Lewis Towers 416 Dept. Fax: 312-915-7998 820 North Michigan Avenue aharrin@luc.edu Chicago, Illinois 60611