Fog Creek Software
Discussion Board




Your Opinion of Test Question Value

In as university CS class, 40% of the grade is performance on test questions which ask the student to write programs longhand without documentation.  If you call an event handler 'DragOver' instead of 'MouseDrag', you lose points.  If you forget that it's '.size' instead of '.count' in the Java collection interface, more points taken off. 

My question:  Do you think this an effective testing strategy?  Why, or why not?

anon
Wednesday, June 30, 2004

The point of a test is not that you remember language specific names for events or names that were presented to you in the class; it is instead that you be able to effectively know what an event is and how to use it.  Thus you should be given the task of first defining what an event is in the generic sense of the word and then writing an event handler for a given language.  Learning or memorizing the names and elements of specific structures or classes based on platforms is useful however and occurs over time, but it is probably not the best use of test questions.

Dave B.
Wednesday, June 30, 2004

What's the course? That test might well be appropriate if the course is "Java syntax".

Gord Schmidt
Wednesday, June 30, 2004

>> "What's the course? That test might well be appropriate if the course is 'Java syntax'."

It a course on Web Programming.  I took it because I'm a senior, and it sounded like a senior blow-off after getting 'A's in Advanced system software & Compilers, etc.  Looks like I'm flirting with a 'C' now because it took too long trying to write code with a pencil and paper and without documentation.


>> "Learning or memorizing the names and elements of specific structures or classes based on platforms is useful however and occurs over time..."

That's interesting that you should say that.  I'm also am a developer as well as a student, and every day, in my group, we use C++ (MFC, ATL, COM, etc.), Visual Basic, C#, Java.  Applets, Servlets, XML, XSLT, SOAP, SQL (SQL Server & Oracle flavors), HTML, DHTML, ASP, JSP, JDBC, etc., etc.  I would be absolutely lost without documentation and intelli-sense.  I just can't remember it all, and as time goes on, I get more fuzzy because I'm exposed to more stuff that just blends together in my mind.

anon
Wednesday, June 30, 2004

>> "I would be absolutely lost without documentation and intelli-sense."

I agree documentation and intelli-sense are indispensible tools.  It seems like you use a heck of a lot of different technologies thus I would make the assumption (please correct me if I'm wrong) that you are a consultant or work for an IT department and not a software company.  Working for a software company on a specific product has the advantage of giving a person the opportunity to work with the same technology enough that they may be able to remember certain things.  This is how it has been in my experience and perhaps I should have clarified this earlier.

(Please don't take that to mean that all software companies are the same or that software companies don't use enough new technology or whatever else one could imply or read into it.)

Dave B.
Wednesday, June 30, 2004

Also, remembering the syntax of a language is generally expected of programmers(SQL, C, BASIC), but remembering the names of language specific elements like events and platform dependant library elements is not expected (or should not be IMO) unless of course you are given time to review the specific library etc.. that you'll be dealing with.

Dave B.
Wednesday, June 30, 2004

I don't think it is effective because it primarily tests memory.
Some of us don't have great memories so suck at this
sort of test. Part of becoming a good programmer
is creating the environment in which you can best
succeed.

son of parnas
Wednesday, June 30, 2004

This test is only as good as the concreteness of the language.  In other words, if there were no other way to do it, then yes, it's ok to test those parameters. However, if I put .size and .length is the right answer, but .size also does it, I shouldn't be wrong.

Using the English language as an example, they could say, "spell the name of that critter that has four legs and eats cheese".  You could write mouse, rat, guy on four legs eating cheese, etc.  Any of these could be the answer.  And that's why questions are dumb.  Any questions?

Devin
Wednesday, June 30, 2004

Effective? Not especially. When I was a TA in a well-respected university's CS dept., we definitely focused on whether students got the semantics of the problem right, not so much on syntactical details -- though I wouldn't be surprised if repeatedly blowing basic stuff or having internal inconsistencies would lose you a small number of points. (This was back in the days when the introductory language of choice was Pascal, or Scheme for the more ambitious students; the world of syntactical details you might have to know was a lot smaller than today.)

John C.
Wednesday, June 30, 2004

Unless you know the function inside and out, you should be referring to the documentation to check for any gotchas. So in this way the test doesn't give an accurate depiction of the real world (or it gives a depiction of a real world with a lot of bugs in the software).

Other deficiencies in normal programming courses include error handling, debugging, testing, and the development lifecycle. If they are taught at all, it would be informal or in electives, when in fact they should be taught in parallel with the programming language (IMHO).

Derek
Wednesday, June 30, 2004

It's a pretty poor question. However it does prepare you for the stupid type of things you'll encounter later on in your career as a programmer.

Jorel on Software
Wednesday, June 30, 2004

And the arbitrary tyranny of PHB's.  Must be the "what you don't learn in college class" class .

hoser
Wednesday, June 30, 2004

anon,

Are you actually saying thta these are real-life examples from a course you are taking?

Please humor us and tell us what fine institution you attend.

Dennis Atkins
Wednesday, June 30, 2004

Nevermind, you have chosen a very poor school. It's too bad people can't get heir money back on overpriced useless educational scams like the one you are being subjected to.

First of all, they should not be taking points off for BS like that.

Second of all, for a test in programming you should be provided with all the documentation you need, without restriction. It's not like you have the dates of the peloponessian wars written on your cufflinks. There is no way to cheat on a programming exam by relying on crib notes. Let the students bring in as much documentation as they are able to carry, or just give them a net connection and log it to make sure they didn't spend the hour typiing test questions into JoS and asking for help there.

Dennis Atkins
Wednesday, June 30, 2004

You might be happy to know that an "authority" lends weight to the fact these memorization tests are completely, astoundingly foolish.
http://download.microsoft.com/download/a/c/f/acfc8c6f-e449-488d-bb29-95ccf0340000/20040624CSHARPAH.exe

Here, Heijlsberg was asked whether intellisense should be turned off during tests. He opined that such tools are important to coding, and the algorithms should simply be harder or something, if this makes tests trivial.

Further back in time, the Interlisp guys knew in the 70s that an IDE was a vital part of the coding environment. They experimented with things we don't even have know, like Programmer's Apprentices. I'm sure you'll find more in the Smalltalk community. Quotes no doubt abound.

Syntax is overrated, garbage tests like this serve only to damage the enjoyment of programming even further. Hopefully you can drop that class, or perhaps take the bad grade with pride and show it off.

Tayssir John Gabbour
Wednesday, June 30, 2004

> Also, remembering the syntax of a language is generally expected of programmers

Hell, I bounce between C, C++, Javascript, Python, sh, make, and a dribble of Perl on a weekly basis.  It takes me a bit of ramp up time to switch my brain from one syntax to another.  Mind you, they're all in my grey matter somewhere.  It just takes a while to page swap the syntax du jour from my tailbone to my frontal lobe.

In other words, if I've just spent 2 days on Python and you ask me for Javascript syntax, I'm hosed.  And vice versa.

Snotnose
Wednesday, June 30, 2004

Incidentally, if you didn't want an odd rant like I gave... I have programmed in notepad to get away from IDEs. Bad IDEs can be stifling, and even little flaws can show up greatly. It was to get a rest from Borland JBuilder, which was then just awful. Being alone with code isn't bad. You can feel the characters between your toes, and it gave appreciation for a good IDE. (One big sin is insulating people from how code is stored on disk. Rarely done well.)

So that's one case in favor of their tests. I wouldn't personally mind writing code longhand, I kinda like it... but memorizing tiny API details?

Tayssir John Gabbour
Wednesday, June 30, 2004

That's a stupid lecturer and or school, anon. Give us a hint. Where is it?

.
Wednesday, June 30, 2004

>> "That's a stupid lecturer and or school, anon. Give us a hint. Where is it?"

Too afraid.  There's only one web programming class here this semester.

anon
Wednesday, June 30, 2004

Dennis:

"It's too bad people can't get heir money back on overpriced useless educational scams like the one you are being subjected to."

You would have to wait for the school to die first.

Steamrolla
Wednesday, June 30, 2004

Is the school in the U.S.?  If not, in which country?  If so, in which state?

Brent
Wednesday, June 30, 2004

The name of the school is unimportant.  I'd be amazed if someone here attended a university and never had at least one prof impose some stupid requirement or graded with some twisted subjective reasoning.  If you made it through 4 years with only one class like this, consider yourself blessed.

yet another anon
Wednesday, June 30, 2004

In my seven years as an undergraduate, I never had an examination that stupid.

Brent
Wednesday, June 30, 2004

7 years as an undergrad? Part time?

Ian H.
Thursday, July 01, 2004

Spoil my punch line, will you?  Yes, I transferred once and changed majors at the new school and ended up working and finishing my degree part-time, mostly due to family problems beyond my control. 

Brent
Thursday, July 01, 2004

Hmmm...  I got my Math BS first and then became a developer.  I started my BSCS shorly thereafter, taking maybe one course per semester - maybe not.  So it's taken me ten years to get the stupid thing! 

anon
Thursday, July 01, 2004

*  Recent Topics

*  Fog Creek Home