[Edu-sig] How to evaluate programming skills, are there any international benchmarks?

[Linda Grandell]

Hello everybody,

I have been going through the edu-sig archive for a couple of months 
now; there is really a lot to read and learn in there.

I work at the Department of Computer Science at Åbo Akademi University 
in Turku, Finland. For the last two years I have been active in a 
project, whose aim is to acquaint high school students with computer 
science, since no true computer science education is available to them 
otherwise. While working with these high school students I have realized 
how important it would be for them to get a true and correct conception 
of what computer science is all about. I think the idea of CP4E, 
proposed by Guido van Rossum, is very good. I believe that everybody 
(not only future computer scientist) would gain from knowing how to 
program, due to the meta skills learned in the process. When one learns 
to program one also acquires knowledge in how to solve problems, think 
logically, methodically and algorithmically.

I think the basic education in computer science has not been given 
enough attention; in my opinion the basic computer science courses 
should have high priority, since they act as starting points into the 
field. I feel that programming should be taught in high schools, since 
these should provide all-round learning and ensure that the students 
receive the skills and knowledge needed for further study and in life in 
general.

The reason I am telling you all this is to set some kind of background 
to what I want to ask you about. I am about to engage myself in research 
as a PhD-student, focusing on how programming and algorithmic thinking 
can be taught to high school students. I have been very fascinated by 
Python, and have decided to use that as the first thing to try out in 
order to get some results on how high school students should (or 
shouldn't :) be taught.

I will start by developing a course in Python that I will give to 
newbies in high schools. I have not had any problem in finding material 
for the course, but what I do see as a problem is how I should go about 
evaluating the course. Since I am aiming at a PhD, I would, naturally, 
like to be able to extract some information about whether the course 
promoted the problem solving, programming and algorithmic thinking 
skills of the students. Since there are a lot of different persons on 
this mailing list, I thought it would be a good idea to write down my 
ponderings here.

* Have anybody else done anything like what I am about to do? I would 
really like to hear about your experiences.

* Are there any international benchmarks one can use to measure the 
results of this kind of an "experiment"? What should be measured and how?

* What kind of research methods can and should be used?

* The course will contain about 28-30 hours; I have a feeling that we 
will be able to cover much more programming theory using Python than, 
for instance, Java, but does anybody have an idea about how far we could 
get in that time?

I am very enthusiastic about this project and I am really looking 
forward to giving the course. I am grateful for any answers, thoughts 
and advice you can give me.

Best regards,

Linda Grandell
Åbo Akademi University
Turku, Finland