Tuesday, January 11, 2005

How to test programming capability?

[I wrote this on Jan 3rd, 2005, but never had a chance to make it online. So here it is.]

Well, I started my academic gig this new year with an exam. I guess, at 7 AM, our class is one of the first to do exams in this university. My class is "Foundation of (object-oriented) computer programming".

This class is not really a true-strict object-oriented programming. It is taken by second-year students in EE. The first year teh students take what I think basically a review of high-school topics; like physics, math, chemistry, and biology. All students, regardless of their departement, take these classes. So, my programming class is their first formal course on programming.

Unlike CS students, we don't have plenty of time to teach programming. There are other topics to teach in a limitted time. We don't want to make students stay longer than what they should. So, we have no option but to pack the programming material into one semester course. We have to teach them algorithm, logic, data structure, and the syntax of a language.

We picked C++ as the language to teach. I know that Java is hot. But, unfortunately the majority of EE-related topics still need C or C-like programming ability. [It's no coincident that JoelOnSoftware has an article that support this: Advice for Computer Science College Students. Although this article is focused on CS students, it's still valid for EE students. I found Joel's article after I wrote this piece.]

As I expected, the quality of the students varies. Although we have "the cream of the crop" of Indonesian students, they come from different background. You'll get different results; from excellent to just barely pass the exam. (Judging from previous class.)

On top of that, how do you test programming capability? How do I grade students in my class?

I always believe that programming is not only a science, but an art as well. I don't want to dwell on this issue. It's a topic on its own. I'll write on this later on. Back to the original question, should I just test the science aspect of programming? Meaning, if they can answer the question or write a program that gives the correct output then they pass. Or should I also consider the style? But then, it won't be objective.

Tell me what you think!

8 comments:

andika said...

I believe programmers must (always) rely on documentations/manuals to do his/her job properly. Open book test will give them this kind of "normal" situation.

Programmers also need compiler/interpreter to do tedious job of syntax checking, so let them use computers to do the exam.

Grading can be done by giving them enough problems, so some who are fluent can solve more.

Just my $.05 :D

Sirait said...

C++ is quite flexible compared to Java. Perhaps, there will be more than one solution for one problem. Let them explain solution they took, why they chose that solution, so you can see if they understand OO. Or you may give solution of one problem, and ask them to find what's wrong with the solution, ask them to give another solution.

Harry said...

If I ever teach programming to students (unlikely, since I'm not very good at teaching others), the test would be :

# open book
# have well-defined scoring criteria, eg:
* write a code snippet that solves this and this problem
* must give correct result
* an answer giving solution in less than 10 seconds will be given extra score
* etc

But you mentioned that for many students, this will be their first exposure to computer programming. If I may, could I suggest that you be lenient to them.

I remember when I was in the first programming class. I fared better because I've coded since I was in high school, but even some of the bright ones suffers. This is because they've had so much work on them already, so they were unable to spend enough time on the subject.
If you suspect that'd be the same case to your students, I hope you can be understanding and have it easy on them.

Anonymous said...

Don't forget to also test debugging skills, as in
the real world, maintenance costs is usually the same
or even higher to the software development cost.
Another way to really test the student is practical
exam (computer based), with automatic grading system
(simple ICPC-style output comparison test) and automatic plagiarism detection program too. Btw considering ITB is considered to be cream of the crop of engineering students, do they actively participate in projects/competitions such as ICPC, Robocup, or solar car racing?

budi said...

Don't forget to also test debugging skills, as in the real world, maintenance costs is usually the same or even higher to the software development cost.Unfortunately, I didn't have enough time devoted to this. Debugging is only mentioned once in the class. I am assuming that they did debugging on the assignments they got. (They have several programming assignments, ranging from easy ones to a more difficult ones.)

Practical exam is not an option, at least right now, since I have around 60 students in my class. (The whole class is around 180 students divided into three classes.) There's not enough computing facilities to do this.

As for participating in competition, I have to say that they don't. I think they are too arogant to participate in competition, or afraid of losing. But this is another story.

Anonymous said...

From: Dermeval Aires Jr
To: Budi Rahardjo
Subject: Finding Diana Budi Rahardjo

------

Rio de Janeiro, Jan 17, 2005

Dear Budi Rahardjo,

Greetings from Brazil. My name is Dermeval Aires Jr and I am a student of philosophy in Rio de Janeiro. I am sorry for having written this message, which to me is very important, as a comment to one of your blog postings without concerning it. I have surfed your pages looking for a contact link or an e-mail, but this was nowhere to be found.

I have been looking for a most dear friend on the internet during these past years but never succeeded. Now I have found your webpage and, since you have the same family name, I was wondering whether you could give me some light on her. I pray you are a relative of her, and if you could forward my e-mail to her it would be most wonderful.

Here is our story: in 1994-1995, I lived in New Jersey in the United States as a high school exchange student. There, one of my best friends was a girl from Indonesia named Diana Budi Rahardjo, or Renzy, as closer friends would call her. When the exchange year was finished, she went back to her country and left with me an address from the city of Jawah Tenga, unless I am mistaken. She was a great friend but it has been impossible to find her. A few times I attempted to call, but the phone number does not exist anymore. Letters remained unanswered, so I presume she has moved somewhere else.

This year, 2005, there is a 10-year reunion being organized for all the students who were in the US in 1995. If you would like to check my reference, here is the address of the website: http://home.s04.itscom.net/ogawa/rotary1995/. In it you can also verify my personal reference, and if you go to the missing people link you will see her name there among a few other Indonesian friends.

I hope you can help me with this, and look forward to an answer. If you do not know anything about Renzy, could you give me any reference that may be useful in finding her? My e-mail is airesjr@usa.net.

Best Regards,

Dermeval

w said...

Mas Budi, not being a CS/EE alum. I wanted to give this opinion on programming, i have experienced programming being taught by Industrial Engineering Prof., Physics Prof., an Astronomy Prof., a Mech Eng. Prof and finally a CS Professor.
They all have different styles and nuances. One just cares about getting the numerical results correct, doesnt care about style, commenting or even getting multiple instructions all on one line. Another cares about style, commenting, efficiency, logic, then finally the right numerical answer. Sorry, being an astronautics student the primary focus is getting the right numerical answer first, programming comes second.
After graduating and programming at work I feel some of the most important things are:
1. style. you have to have programming style.
2. clarity. your style doesnt mean anything if no one else can read your code. your code will be read by others trying to modify or improve your code. it doesnt help if its not clear how your logic goes.
3. logic. what may be obvious to you and your IQ peers may not be clear and logical to those above or below it. i guess this helps point number 2. and vice versa.

if you have 1,2,3, the correct answer will follow.

I suggest grading on the "science" portion and extra credit for the "art" portion. I think you have to give credit to those people who make the effort to make their code "beautiful" (not just indented and commented, you know what i mean).

Lastly I wanted to comment on Mr. Anonymous who said "cream of the crop? but doesnt attend various events?"
I think a should reply sound like "well, be it as it may, but the SIPENMARU tests still show it".
Inwardly I want to say to the adminsitrators of the uni. to start thinking about how they want to maintain their position as one of the better uni's. If they continue this indifferent attitude and dont put a strong effort they will no longer be the cream of the crop but cheapest of the crop.

Power to you mr. boing. salam.

Roohi said...

Hi, i have been a programming student, and even though your checking should follow the basic criteria that the output should be correct and all, but the style of programming is definitely important aswell. I believe being a little subjective is ok, since eventually it'll even challenge the other good programmers to figure out better ways to code.