Hump-Jumping: How the Education of Computer Science can be Saved, err, maybe.

In a paper called 'The Camel has two humps' the authors described a test which can be applied to students before they've begun a computer science course, and can fairly accurately predict those that will do well and those that will not.

I've never agreed with their summary of the results.

They say they are dividing:

"programming sheep from non-programming goats"

...which implies the differences between pass and fail are as pronounced as speciation. That's a big claim and well outside the scope of their research.

And I think they misrepresent their study when they say:

"This test predicts ability to program with very high accuracy before the subjects have ever seen a program or a programming language."

...as, crucially, their study failed to check whether the students "had ever seen a program or programming language."

(Unless I'm misinterpreting this sentence from the paper, taken verbatim:

We believe (without any interview evidence at all) that they had had no contact with programming but had received basic school mathematics teaching.

... I've written to one of the authors to seek clarity.)

I asked Alan Kay for his opinion, when he commented here on a different topic -- he was very kind in providing a lengthy and thoughful answer.

His opening phrase really sticks in my mind:

"They could be right, but there is nothing in the paper that substantiates it."

Then, this morning, I saw a fresh comment at that article, from David Smith

"If there were a definitive test of ultimate programming capability I could apply on the first day of class, what would I say to those who 'failed'?"

Which presents a very human response to the topic from an educator directly affected by it. And I don't have a sufficient answer to that.

But a different approach to the whole problem has occurred to me since:

Let's suppose that this test is indeed an accurate test of those that will and won't succeed in computer science 101.

We put aside all worries about what biases or inconsistencies the study might have. Just accept that the test is effective. Stick with me here.

So we give the test to all students before they start Computer Science 101. At the end of the subject, we see that, as predicated, those that do poorly in the subject tend to do poorly on the pre-test. But instead of looking for correlation, what if we looked for outliers?

Which students did poorly on the pre-test, but did well in computer science? Those are the students with the most to teach us. Why did they buck the trend?

Okay, so maybe some of them cheated. (I remember a high incidence of cheating in early computer subjects I took; particularly amongst those who didn't continue in the field.) And maybe some of people deliberately blew the pre-test, even though they did well at the subject.

But once we find the genuine hump-jumpers, we focus on what it was that helped computer science 'click' for them. Did they find there were specific misconceptions they had to overcome? Did they have extra-tuition? Were there specific problems that helped them get their thinking in order? Was it just hard work? And, regardless of the answer, would they like to become tutors next semester, specifically working with those who perform poorly on the pre-test?

It might be necessary to look at a lot of such hump-jumpers before useful lessons emerge. It might be that every one of them has a different story to tell, there's no common pattern. (As Tolstoy said in his Turing Award speech: "Happy programmers are all alike; but every unhappy programmer is unhappy in her own way.")

So that's my answer for David. I don't know what you could say to those who fail the pre-test. But I think that over time a good pre-test could be used to develop new and better teaching methods, and maybe that's the best we can do.

It's also possible that the entire idea is overly influenced by a generally Western notion that you either have ability or your don't, and there's not much to be done about it. This seems remarkably widespread in our attitudes about math[s], for example, where we accept without comment (or dissent) when people say "Oh, I'm just not a math person."

As has been reported about math education in Asian countries, and about music education in the UK, whatever "talent" a student might or might not have is only a weak determiner in how successful they are in those subjects. A far better predictor turns out to be an old-fashioned kind: how hard they work at it. Malcolm Gladwell (who, admittedly, is the epitome of a popularizer) notes that what might distinguish successful music students is not musical talent as such, but what he refers to as "a talent for practice" -- ie, motivation to practice and an instinct for (or instruction in) doing so effectively.

IOW, motivation and hard work are probably ultimately far more important than native ability. Of course, it doesn't hurt to be a "genius," but one generally discovers that geniuses have somewhere, at some time, put in a whole lotta work into their chosen field of endeavo[u]r.

Maybe I've always been in the "geek" side of the classroom, but I do not remember any of the people I was involved with that had not hacked at least a bit with some form of basic.

I gave this idea a lot of bandwidth after reading your original post on the topic. My sense is that it really boils down to the idea of nurture versus nature - the idea that individuals carry along an inherent set of preferences or perhaps can evolve along the way.

Functionally, Denadi and Bornat found that their first set of tests were not predictive for people who had already been exposed to programming. So, they went back to the drawing board and have refined the test to capture patterns / modes of response that they believe are more predictive.

Intuitively, this seems "truthy" to me (who hasn't worked with someone with a decent academic background that was hopeless as a developer?), but at the same time, it seems to me that the predictive aspect results in too many "programmer positives" - which makes me think that the pattern matching aspect is probably more inate (see Baron-Cohen's E-S Theory and search engine the words "autism quotient" for a self assessment) in some, but teachable for most.

Someone would probably have to test a bunch of 5 year olds and then keep track of them through their academic careers to really sort this out.

Expanding on the hypothesis of Gladwell's (via mike). Maybe an initial test of programming should be "Are you the type of person who is going to stay up late programming".

eg

"Explain the inherent plot flaw of 'The Terminator', ignoring the implausability of time travel and sentient computers."

Any lame answer like "I haven't seen it" disqualifies you from programming.

Maybe the best tests could be built by emulating the military aptitude tests. When I took such a test in 1975, I had the vast computer experience of having once watched a teletype print out a christmas tree of asterisks, and the significant electronics experience of having figured out that I could get distant FM stations from our family cable TV feed. (Oh yeah, lest I forget, I once plugged an 8-ohm speaker into a wall outlet because I wanted to hear what electricity sounded like.) The Air Force put me into electronics, and I am now an accomplished telecommunications engineer, having recently crosstrained after a long career as a proficient software engineer. I think they know something.

Hump-Jumping: How the Education of Computer Science can be Saved, err, maybe.

Articles

Twitter, baby

Recommended Reading

Recommended blogs

Aggregated Links

Human Link Machines