Mailman 3 December 2005 - Edu-sig

I've just started reading this paper
by Scott David Daniels Dec. 13, 2005

Dec. 13, 2005

I've just started reading this paper (wish it covered Python): Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers ACM Computing Surveys (CSUR) Volume 37 , Issue 2 (June 2005) Caitlin Kelleher, Randy Pausch http://portal.acm.org/tipsvc.cfm?id=1089734&sess=%27%2BL%2F%2DP%5CK%2B30O%2… If anyone else wants to grab a copy and chat about it after reading, let me know. --Scott David Daniels Scott.Daniels(a)Acm.Org

3 4

Re: [Edu-sig] Python Programming: Procedural Online Test
by Laura Creighton Dec. 7, 2005

Dec. 7, 2005

In a message of Sun, 04 Dec 2005 11:32:27 PST, Scott David Daniels writes: >I wrote: > >> ... keeping people at 80% correct is great rule-of-thumb goal ... > >To elaborate on the statement above a bit, we did drill-and practice >teaching (and had students loving it). The value of the 80% is for >maximal learning. Something like 50% is the best for measurement theory >(but discourages the student drastically). In graduate school I had >one instructor who tried to target his tests to get 50% as the average >mark. It was incredibly discouraging for most of the students (I >eventually came to be OK with it, but it took half the course). <snip> 'Discouraging' misses the mark. The University of Toronto has professors who like to test to 50% as well. And it causes suicides among undergraduates who are first exposed to this, unless there is adequate preparation. This is incredibly _dangerous_ stuff. Laura >--Scott David Daniels >Scott.Daniels(a)Acm.Org > >_______________________________________________ >Edu-sig mailing list >Edu-sig(a)python.org >http://mail.python.org/mailman/listinfo/edu-sig

6 9

Re: [Edu-sig] Python Programming: Procedural Online Test
by damon bryant Dec. 7, 2005

Dec. 7, 2005

I tweaked it now where all other browsers and OS combinations can access the computer adaptive tests. Performance may be unpredictable though. Damon >From: "Kirby Urner" <urnerk(a)qwest.net> >To: "'Vern Ceder'" <vceder(a)canterburyschool.org>, "'damon bryant'" ><damonbryant(a)msn.com> >CC: edu-sig(a)python.org >Subject: RE: [Edu-sig] Python Programming: Procedural Online Test >Date: Fri, 2 Dec 2005 19:50:36 -0800 > >Similar comment. I'm on Windows but don't want to be tested by a service >that won't let me use FireFox. I have tests too. > >Kirby > > > > -----Original Message----- > > From: edu-sig-bounces(a)python.org [mailto:edu-sig-bounces@python.org] On > > Behalf Of Vern Ceder > > Sent: Friday, December 02, 2005 2:38 PM > > To: damon bryant > > Cc: edu-sig(a)python.org > > Subject: Re: [Edu-sig] Python Programming: Procedural Online Test > > > > In my opinion, you would get more responses if the testing system > > accepted a browser/OS combination other than IE/Windows.... > > > > Cheers, > > Vern Ceder (using Firefox and Ubuntu Linux) > >

4 5

Re: [Edu-sig] Python Programming: Procedural Online Test
by Laura Creighton Dec. 6, 2005

Dec. 6, 2005

Interesting. When the law society of Upper Canada was working on CAI for teaching tax law to law students, they found that the percentage correct they set has a definite, measurable effect on the amount of time that the students were willing to drill. At above 85%, the students believed they knew everything, and did not drill much. Some of these students were correct in their estimations. But if you went below 75% correct, the students believed that the computer system was at fault, and that doing the drill was a waste of time. This perfectly matched with the student's own assessment of their own abilities: We asked them ahead of time, how well they thought that they ought to be doing, and they all thought between 70% and 90% was about how well that 'good students should do on tests'. And LSUC does not _have_ any poor students. When they were given things to 'learn things for the first time' they were able to tolerate significantly worse performance by themselves, before they concluded that the effort was a waste of time. Thus one of the things that we tried to measure was the effect of 'what percentage correct' had on actual amount of tax law learned. And the result was unequivocal. It is better to give each student a percent correct that was slightly, but not substantially higher than they estimated their own abilities, and always better than 70% in any case. The poorer students will drill for 16 or 18 hours, and the best ones would only take 4, but by the end of the exercise, everybody was scoring pretty well on tax questions. Laura In a message of Mon, 05 Dec 2005 20:19:20 CST, "damon bryant" writes: > >Hi Rodrigo! > >>If I understood correctly the proposal is to give a "hard"-A for some >>and an "easy"-A >>for others, so everybody have A's (A=='good score'). Is that it? > >No, students are not receiving a hard A or an easy A. I make no >classifications such as those you propose. My point is that questions are > >placed on the same scale as the ability being measured (called a theta >scale). Grades may be mapped to the scale though, but a hard A or easy A >will not be assigned under aforementioned conditions described. > >Because all questions in the item bank have been linked, two students can > >take the same computer adaptive test but have no items in common between >the >two administrations. However, scores are on the same scale. Research has >shown that even low ability students, despite their performance, prefer >computer adaptive tests over static fixed-length tests. It has also been >shown to lower test anxiety while serving the same purpose as fixed-lengt >h >linear tests in that educators are able to extract the same level of >information about student achievement or aptitude without banging a >student's head up against questions that he/she may have a very low >probability of getting correct. The high ability students, instead of bei >ng >bored, are receiving questions on the higher end of the theta scale that >are >appropriately matched to their ability to challenge them. > >>That sounds like >>sweeping the dirt under the carpet. Students will know. We have to >>prepare them to >>tackle failure as well as success. > >In fact computer adaptive tests are designed to administer items to a per >son >of a SPECIFIC ability that will yield a 50/50 chance of correctly >responding. For example, there are two examinees: Examinee A has a true >theta of -1.5, and Examinee B has a true theta of 1.5. The theta scale ha >s a >typical range of -3 to 3. There is a question that has been mapped to the > >theta scale and it has a difficulty value of 1.5, how we estimate this is > >beyond our discussion but is relatively easy to do with Python. The item >is >appropriately match for Examinee B because s/he has approximately a 50% >chance of getting this one right - not a very high chance or a very low >chance of getting it correct but a equi-probable opportunity of either a >success or a failure. > >According to sampling theory, with multiple administrations of this item >to >a population of persons with a theta of 1.5, there will be an approximate >ly >equal number of successes and failures on this item, because the odds of >getting it correct vs. incorrect are equal. However, with multiple >administrations of this same item to a population of examinees with a the >ta >of -1.5, which is substantially lower than 1.5, there will be exceedingly > >more failures than successes. Adaptive test algorithms seek to maximize >information about examinees by estimating their ability and searching for > >questions in the item bank that match their ability levels, thus providin >g a >50/50 chance of getting it right. > >This is very different than administering a test where the professor seek >s >to have an average score is 50% because low ability students will get the > >vast majority of questions wrong, which could potentially increase anxiet >y, >decrease self-efficacy, and lower the chance of acquiring information in >subsequent teaching sessions (Bandura, self regulation). Adaptive testing > is >able to mitigate the psychological influences of testing on examinees by >seeking to provide equal opportunities for both high and low ability >students to experience success and failure to the same degree by getting >items that are appropriately matched to their skill level. This is the >aspect of adaptive testing that is attractive to me. It may not solve the > >problem, but it is a way of using technology to move in the right directi >on. >I hope this is a better explanation than what I provided earlier. > > > >>From: Rodrigo Senra <rsenra(a)acm.org> >>To: edu-sig(a)python.org >>Subject: Re: [Edu-sig] Python Programming: Procedural Online Test >>Date: Mon, 5 Dec 2005 19:53:00 -0200 >> >> >>On 5Dec 2005, at 7:50 AM, damon bryant wrote: >> >> > One of the main reasons I decided to use an Item Response Theory (IRT >) >> > framework was that the testing platform, once fully operational, >> > will not >> > give students questions that are either too easy or too difficult >> > for them, >> > thus reducing anxiety and boredom for low and high ability students, >> > respectively. In other words, high ability students will be >> > challenged with >> > more difficult questions and low ability students will receive >> > questions >> > that are challenging but matched to their ability. >> >>So far so good... >> >> > Each score is on the same scale, although some students will not >> > receive the same questions. This is the beautiful thing! >> >>I'd like to respectfully disagree. I'm afraid that would cause more >>harm than good. >>One side of student evaluation is to give feedback *for* the >>students. That is a >>relative measure, his/her performance against his/her peers. >> >>If I understood correctly the proposal is to give a "hard"-A for some >>and an "easy"-A >>for others, so everybody have A's (A=='good score'). Is that it ? >>That sounds like >>sweeping the dirt under the carpet. Students will know. We have to >>prepare them to >>tackle failure as well as success. >> >>I do not mean such efforts are not worthy, quite the reverse. But I >>strongly disagree >>with an adaptive scale. There should be a single scale fro the whole >>spectre of tests. >>If some students excel their results must show this, as well as if >>some students perform >>poorly that should not be hidden from them. Give them a goal and the >>means to pursue >>their goal. >> >>If I got your proposal all wrong, I apologize ;o) >> >>best regards, >>Senra >> >> >>Rodrigo Senra >>______________ >>rsenra @ acm.org >>http://rodrigo.senra.nom.br >> >> >> >> >>_______________________________________________ >>Edu-sig mailing list >>Edu-sig(a)python.org >>http://mail.python.org/mailman/listinfo/edu-sig > > >_______________________________________________ >Edu-sig mailing list >Edu-sig(a)python.org >http://mail.python.org/mailman/listinfo/edu-sig

1 0

Re: [Edu-sig] I've just started reading this paper
by John Maxwell Dec. 5, 2005

Dec. 5, 2005

That is, Re: Kelleher and Pausch's Lowering the Barriers to Programming, ACM Computing Surveys 37(2) Here's a comment off the top, and apart from the more obvious issue for the edu-sig group (namely, Python's near-complete absence): the "taxonomy" is completely ahistorical; they seem to have set up 60+ programming environments next to one another and considered them all at face value, with no particular consideration for the time and context from which each emerged (we're talking a 45-year span here). The two exceptions to this are the repeated reference to a god-given, eternal "logo turtle", and a chart which tracks the "influences" of various systems on one another, but only by simple bibliometrics. I'm working on a comprehensive history of the Smalltalk/Squeak 'tradition' so this facet pops out at me immediately, as there are a half-dozen or more Smalltalk-derived systems listed here, though you'd never know it from the article. My 2 cents. What do you folk think of this? - John Maxwell jmax(a)sfu.ca

1 0

rest2web created PyGeo docs
by Arthur Dec. 5, 2005

Dec. 5, 2005

My comments have gotten even more particularly PyGeo centric, as I have been on a bit of a PyGeo binge. After some intensive refactoring and testing, I have declared the code "good enough", and am moving on to working on decent docs, and then a decently managed release. Its been some time. Among the problems I feel I have recently solved is turning the individual reStructured text document pages into a coherent set of accessible documentation. The solution came by way of the Fuzzyman's (Michael Foord) rest2web - The Site Builder http://www.voidspace.org.uk/python/rest2web/ a tool others might interested in knowing about. The results of my rest2web efforts on a portion of the PyGeo docs are up at : http://pygeo.sourceforge.net/docs_html/index.html feedback/comments welcome. Art

1 0

Re: [Edu-sig] Python Programming: Procedural Online Test
by damon bryant Dec. 4, 2005

Dec. 4, 2005

Kirby: Thank you for your feedback! You completed the Declarative measure. I am also interested in your feedback on the Procedural test in which Python application or procedural questions are administered. Questions on this part are coded and displayed as they appear in the IDLE with highlighted key words and indentation for nested code. I think the longest code problem is about 20 lines. I appreciate your comment on the ability to specify font type/size because I'm currently working to accommodate persons with disabilities and others who may have difficulty viewing the text. Linda Grandel and I have an experimental site that we use for research and educational purposes; we are going to trial some Python questions for a class of her colleagues but are having some difficultly translating the testing in a short period. This is a long-term project. We have the goal of developing a worldwide database of Python test norms in an effort to track progress on the spread and proficiency of the language in different countries. Although it is a great idea, it is too large for a dissertation research project. If you are interested in trialing it with your class, perhaps we can collaborate. You did notice that towards the end, questions got easier for you. The test algorithm is adaptive but the question bank from which the items are pulled is not that large. In other words, the test presented items that were most appropriate for you when you began the test. As you got more items correct you got harder questions. In contrast, if you initially got questions incorrect, you would have received easier questions. Because the bank is so small (I do have plans of expanding it when I get some more time on my hands), you exhausted the bank of difficult questions and began to receive easier items. The opposite would have happened to an examinee of very low ability. My goal is to administer a computer adaptive Python test where examinees will only receive questions that are most appropriate for them. In other words, different examinees will be tested according to their ability. This goes back to Binet's idea of tailored testing where the psychologist administering the intelligence test would give items to an examinee based on previous responses. In the present case, it's done by computer using an artificially intelligent algorithm based on my dissertation. By expanding the question bank, I'll be able to reach that goal. >From: "Kirby Urner" <urnerk(a)qwest.net> >To: "'damon bryant'" <damonbryant(a)msn.com>, vceder(a)canterburyschool.org >CC: edu-sig(a)python.org >Subject: RE: [Edu-sig] Python Programming: Procedural Online Test >Date: Sat, 3 Dec 2005 07:44:32 -0800 > > > I tweaked it now where all other browsers and OS combinations can access > > the computer adaptive tests. Performance may be unpredictable though. > > > > Damon > >OK, thanks. Worked with no problems. > >As an administrator, I'd be curious to get the actual text of missed >problems (maybe via URL), not just a raw percentage (I got 90% i.e. 2 wrong >-- probably the one about getting current working directory, not sure which >other). > >The problems seemed to get much easier in the last 5 or so (very basic >syntax questions). The one about "James"=="james" returning -1 is no >longer >true on some Pythons (as now we have boolean True). > >The font used to pose the questions was a little distracting. I vastly >prefer fixed width fonts when programming. I know that's a personal >preference (some actually like variable pitch -- blech). Perhaps as a >future enhancement, you could let the user customize the font? > >Anyway, a useful service. I could see teachers like me wanting to use this >with our classes. > >Thank you for giving me this opportunity. > >Kirby > > >-- >No virus found in this outgoing message. >Checked by AVG Free Edition. >Version: 7.1.362 / Virus Database: 267.13.11/191 - Release Date: 12/2/2005 > >

2 3

Python Programming: Procedural Online Test
by damon bryant Dec. 3, 2005

Dec. 3, 2005

Hey folks! Lindel Grandel and I have been working on some Python questions for potential use in high schools, college, and employment. If you are interested in taking one of the online tests go to http://www.adaptiveassessmentservices.com and self-register to take one of two Python tests: one is Declarative (knowledge of built-in data types and functions) and the other is Procedural (application of loops, import, functions,...,etc.). I would like to know what you think. Thanks in advance! Damon

3 2

Recap of 2 Python classes
by Kirby Urner Dec. 1, 2005

Dec. 1, 2005

Quiet around here. So I did two classes for 8th graders, 7 to go. One was pre-Thanksgiving and included some play with Google Earth. Did you know you can share Places using XML files (.kml)? Example: <?xml version="1.0" encoding="UTF-8"?> <kml xmlns="http://earth.google.com/kml/2.0"> <Placemark> <name>Climatron</name> <LookAt> <longitude>-90.2588032444306</longitude> <latitude>38.61412902659804</latitude> <range>191.4235683380654</range> <tilt>-8.190895111017013e-011</tilt> <heading>0.01079095337023216</heading> </LookAt> <styleUrl>root://styleMaps#default+nicon=0x307+hicon=0x317</styleUrl> <Point> <coordinates>-90.2588032444306,38.61412902659804,0</coordinates> </Point> </Placemark> </kml> Just go File | Open and bring it in (e.g. test.kml). Anyway, every kid has IDLE going, and the first thing I had them do is ask for the type of a lot of primitive objects e.g. >>> type(10) <type 'int'> >>> type('cat') <type 'str'> Then I dove into collection types (lists, dictionaries and strings). Shall we call strings a collection type? They're a lot like lists, aren't they? I use the collection types to introduce dot notation. Kids seem to like the ability to manipulate strings more than anything. Numbers are maybe too ordinary or less cool in some way. Next week, we'll be looking at functions for the first time. Anyway, pretty hum drum. I'm posting more about math/cs at the Math Forum than in any Python list, since the concepts are pretty familiar around here. It's the math teachers who're still somewhat innumerate. Kirby

1 0