Fog Creek Software
Discussion Board




What we should teach programmers


I taught SW engineering a few years back.  I saw several areas where students were lacking, and had some useful methods for addressing those needs.

Here's what I saw:


1.  RUSHING TO CODE

A tendency to want to rush to the coding phase, before requirements are gathered and spec is done. BTW, this was true in the telecom consultancy I worked for (as an EE Engineer as well. So, perhaps just human nature).

Many reasons for this, including:
a.  We have better coding tools (IDEs) than we do tools for design.
b.  Coding FEELS productive, earlier stages do NOT feel that way.  (How it feels is contrary to the reality, of course).

SOLUTION: I'd give a test with 10 questions. #1 thru 9 were worth a point (listed at the end of question). #10 worth 91 points.  #1 thru 9 were fairly hard (10 minute problems) #10 was moderately hard but the ANSWER was in #1 thru #9.

I gave them 10 minutes to take the quiz.  Only one student ever got #10 right.

He'd forgotten his book.

He couldnt' answer #1-9 so he kept going til he got to #10.

Interesting responses DURING the test. A lot of bitching and moaning. (I never used the test score. It was just a mental exercise for them).  I'm not sure how much it helped them, but I think it got the point across. (Whether they were ready to LEARN that point is another matter).

2. NO TROUBLESHOOTING SKILLS

This just is NOT taught in colleges (in my experience). I learned the basics in the Navy and then went on to study it.

My students would do fine in class and then, if they encountered an error, they were LOST and couldnt' recover.

SOLUTION:  I would teach troubleshooting methodology.  Then the first student who came in with an error would be used as a case study. I did this without embarassing the student.  The CLASS would solve the problem as a group. Worked great.


ANYONE ELSE HAVE SIMILAR INSIGHTS?

Entrepreneur
Thursday, July 03, 2003

"SOLUTION: I'd give a test with 10 questions. #1 thru 9 were worth a point (listed at the end of question). #10 worth 91 points.  #1 thru 9 were fairly hard (10 minute problems) #10 was moderately hard but the ANSWER was in #1 thru #9.

I gave them 10 minutes to take the quiz.  Only one student ever got #10 right.

He'd forgotten his book.

He couldnt' answer #1-9 so he kept going til he got to #10."

I had a *really* hard time trying to evaluate the lesson here - it seems like the guy who came to class unprepared then blew through the test as quickly as he could got rewarded?

I understand that the lesson *should* be "review the entire project before starting", but recognize that in 99% of college exams there is no reward in doing that.

How *I* would teach that lesson - give a coding exercise. The exercise consists of requirements gathered from ten people, the design should meet their requirements as closely as possible. Same kind of drill - nine fairly dense (and often contradictory) sets of "I'd like it to do [x]"

The tenth paragraph is the CEO who says "do what 2, 4, and 8 wanted. Ignore the rest, we'll get them later." (and of course 2, 4, and 8 create a well-formed design that's fairly straightforward to design.)

Philo

Philo
Thursday, July 03, 2003

I'm not sure I understand the quiz.  Students are given 10 minutes to do 100 minutes worth of design/coding, the one who made the most points rushed to the coding phase while everyone else failed, and it was supposed to *discourage* rushing to coding?

Anon
Thursday, July 03, 2003

Ah, Philo just cleared up my confusion...

Anon
Thursday, July 03, 2003

Semi off-topic, but I've sometimes wanted to teach solely because I'd love to mess with student's heads a little bit.

I want to give people a multiple-guess test where all of the answers are C.  Except for the last one, which is A.

To make sure that they have confidence that they really know what they are doing, but aren't so encouraged by previous correct guesses to blow the last one. ;)

Flamebait Sr.
Thursday, July 03, 2003

as long as we're off-topic and messing with students' heads...

A roommate of mine had a stats professor who gave all-multiple-choice tests, except that along with your choice, you had to give a confidence level (0-100%).  If you were 100% sure and got it right, you got all the points for that question.  If you were 100% sure and got it wrong, you failed the entire test (-infinity points for the question).  I don't know the details, but it was scored in such a way that if you were 0% sure the answer was A (that is, 100% sure it *wasn't* A), you still got some points even though you didn't actually get the answer.  This leaves lots of interesting ways to mess with students' heads, based on the multiple-choice answers...

schmoe
Thursday, July 03, 2003

I think a major problem is that few are able to READ code.

That would be a neat thing to teach at University; now they also have a huge corpus of open source software to practice reading/understanding code / decomposition of existing programs.

Michael Moser
Thursday, July 03, 2003

What I would teach is what I've dubbed The Tale of The Den and the Dining Room. If I'm sitting in the den in front of my PC, I don't design. The temptation is too great to just try things out. Cowboy coding is fun until there's a problem.

I have to move to the dining table, with a pencil, paper, and any needed reference books to plan out the design if I want to get things done quickly. Moving away from the PC forces me to think about the problem before trying to come up with a solution.

More than once I have hacked away for an hour or more, tweaking the code, trying to get it to work with no success.  Then I moved to a table, really thought about what I was trying to do and what my code was actually doing and came up with a working solution in less than 10 minutes.

[This could be called The Desk and the Conference Room, but that doesn't have as good as alliteration.]

Nick
Thursday, July 03, 2003

The program that I am presently taking (Engineering Science at the University of Toronto) has found a good solution to this program - a year long project building an autonomous robot, which ended up consuming between 20-100 hours/week, depending on the week.  We worked in teams of three - I did code, a friend did mechanical, and a third friend did circuits.  I learned more from that project than I have in all of my other classes combined - including things like the importance of preplanning code (but ALSO the importance of producing something that your collegues can test against).  I learned about things taking twice as long as they should, tradeoffs of using different programming languages, teamwork, and of course, debugging (and more debugging, and then more debugging).  Seems like this type of thing would be a good solution to these kinds of educational problems

Daniel Schwartz-Narbonne
Thursday, July 03, 2003

Make it the "Desk and the Drawing Board" ;)

-Rich

Rich
Thursday, July 03, 2003

While I generally agree with not rushing to code, I think there are exceptions, sort of.

For my most recent app, I was not sure I could do certain parts of it (2 areas to be exact, let's call them X and Y). There is no book, saying here is how you do X and Y.  X and Y are probably < 1% of the app in terms of LOC, but they are essential to the entire purpose of the project.

I therefore rushed to code X and Y (well to be fair, I also mixed this with trying to figure out how to do them hundreds of times on paper).  When I had them, I knew the app was feasible.

It was only then, I began to think about the other 99% of the app, which is mostly standard GUI stuff, and obviously do-able backend stuff.  I always knew I could do the other 99%, so I figured it wasn't important or worthwhile planning this part at all (I know I can make dialog boxes, save into a file, etc.) until I had the critical 1% proven.

If I had planned out the entire app, before doing the critical 1%, and found I couldn't solve the 1%, the planning would be entirely wasted.


Another situation where you might have to rush to code, at least part of something, is where simply not sure how so interface or API works.  Let's face it, many modern interfaces and APIs are only possible to truly figure out, by doing some coding experiments.  If you haven't done the code experiments, you don't know the best/right/only way to design your app.

S. Tanna
Thursday, July 03, 2003

1. That time == money

2. That giving their source code away is all part of a scenario working to undermine their chance of getting a job

.
Friday, July 04, 2003

There are two valid approaches to programming: up-front design (before coding) and evolutionary design (while coding). I find evolutionary design combined with continual refactoring to be the most productive approach. The end result is better, and it can handle spec changes more readily. Many excellent programmers prefer up-front design, but it's not the only way to do things.

Regarding the larger question, nobody gets good at anything until they do it. Classes can provide advice and assign self-contained projects. Still, everyone has a lot to learn when they begin their first software job, working on messy long-term collaborative projects.

Julian
Friday, July 04, 2003

I" understand that the lesson *should* be "review the entire project before starting", but recognize that in 99% of college exams there is no reward in doing that. "


Not true in engineering classes that I took.  We could answer any 5 of 8 questions. So, you learn really quickly to find the ones you can solve the easiest.  (There weren't necessarily EASY problems, just ones that YOU could sove the easiest).

That was one of the most instructive things I learned.

Entrepreneur
Friday, July 04, 2003

And of course, the clear surperiority of the design first model over the evolutionary model is why we are all using gnu hurd instead of linux.

Wait a sec ...

Daniel Schwartz-Narbonne
Friday, July 04, 2003

REGARDING ITERATIVE DESIGNS
I agree, those work best.

However, it's important to have a good understanding of the problem.  Coding is ONE WAY to better understand the problem. But a much faster way is paper-prototyping or "doing it manually" as I told my students.  The programs I've written are all about FORMALIZING some process as a program. It's VITAL that you undestand the process and be able to do it manually.

I realized this one day when I was trying to analyze "how do I come up with algorithms" for, say, manipulating strings.  I sit down and draw out the string and and say "oh, I'll want to take the first 3 characters, character #1 to #3 and then do a Mid$ from Character #4..." blah blah.

Philo wrote:

"The tenth paragraph is the CEO who says "do what 2, 4, and 8 wanted. Ignore the rest, we'll get them later." (and of course 2, 4, and 8 create a well-formed design that's fairly straightforward to design.)"

PHILO- I like that a lot. It's more realistic.

Were you thinking this would be done by them in 10 minutes, etc. ?

Grading a design is really really difficult. It's very very subjective.  (And many students --many people-- write like crap.  They can't put a coherent sentence together.)  That's thier problem.  Just don't make me read it. It's painful. 

A variation would be to have them list the features rather than create a design. (Or perhaps that's what you meant)


DIFFICULT TO LEARN FROM LONG PROJECTS?

I think one problem with learning lessons from projects is that the PROJECT IS SO LONG that people aren't able to learn from them. Perhaps that's just because TIME is not set aside for debriefing afterwards. But, also, the key is to get the point across as quickly as possible. 

Entrepreneur
Friday, July 04, 2003

S. Tanna:

I think you bring up a well-illustrated point, which is that design-first often works best with low-risk, high-knowledge products (in more derogatory terms, commodity software).

Anyway, I digress:

If you want to teach people software engineering and its merits, for God's sake let's stop making it so fucking BORING.  Noone wants to design when it entails a morass of paperwork and metrics and bureaucracy.  And that, by the way, is what so much of academic software engineering teaches.

Design should be more like art, and less like CAD, and it should be broken up with some kind of active creation.  People are not machines, and therefore the approach to creating a program should be fairly organic.  Let's just acknowledge that, psychologically, creators want to create and see something come around sooner rather than later.

In this sense, the interests of businessmen and creators intersects, and we should therefore encourage the _most_ evolutionary model.  IMO, design processes that work more towards strict formalizing miss the whole point of "programming as a craft."  So too do tests where one answers #1-9, then 10 (is the Waterfall methodology in vogue in your particular sector of the economy?)

c++_conventioner
Friday, July 04, 2003

Maybe not ten minutes, but not much longer. Or heck, even give them an hour - that should still be sufficient for those who start working before they read the whole thing to really shoot themselves in the foot.

As for grading design, yes it's subjective and there can be many answers, but so long as they can explain their work (which should be part of the answer), then a functional design should still be an "A".

For example - you have to process invoices and purchase orders. Do you make one documents table or an invoices table and a purchase orders table? Both are "correct" answers, they each have their issues.

Philo

Philo
Friday, July 04, 2003

The variation I had on this was my maths class.  There would be a huge sum with a single times zero embedded in it that invalidated most of the sum.

Those that attempted to step through the whole sum piece by piece go nowhere.  If you took the time to get an overview then you found that and saved yourself a lot of time.

Ged Byrne
Friday, July 04, 2003

These are both aspects of the one failing.

Most people don't have the mind set to read documents/code carefully and understand what is being built before beginning to impose their own will on things.

Anyone who does have the mind set soon finds out that large numbers of documents and programs are ramshackle constructs working by luck, if at all.

When doing maintenance work I find myself having to balance what the spec/design intended (if this can be deciphered), what the developer thought they were doing (however deranged), and what is actually required (occasionally related to one of the other two).

However, this doesn't stop me using sketched-out code to assess my theories and the problems likely to be introduced by alternative implementations.  It never does to forget that there is an order of problems which arises from the different natures of spec and implementation (all those leaky abstractions).

If you want people who can read code and interpret specs you will probably be better off looking for good History, Literature, or Archaeology graduates IMHO.  It's not about literacy or maths, it's about accommodating someone else's world in your own head and reviewing it critically.

Find some who can also write decent code and you will be well away.  But if they have the right thinking skills they can be trained to write code ...

Mathematical Dunce
Friday, July 04, 2003

A similar test is the one that has a hundred questions to answer in an hour and starts of with the rubric ---Please read ALL questions carefully before you start to write anything."

The last question says "Ignore everything you have read before. Sigh loudly, hand in the paper unanswered and leave the examination room quietly."

Stephen Jones
Sunday, July 06, 2003

Since the penalty for incorrectly stating 100% confidence is so severe, wouldn't you just makr 99%? Or would that give you too few points?

pb
Monday, July 07, 2003

Stephen F. Austin State University solved this problem by having a design lab in one building and a coding lab in another building.

Students were required to spend X number of hours in the design lab before they received an account and password for the coding lab.  In the coding lab, you only had X number of hours to completely code the assignment.  Once it was coded, you submitted your project.  If you exceeded the time limit, points were severely deducted.

In the design lab, students were encouraged to collaborate, but not copy each other.  There were NO computers allowed in the room.

By using this method, students were forced into learning how to design something before they coded it.

Bryan Shaw
Monday, July 07, 2003

"A similar test is the one that has a hundred questions to answer in an hour and starts of with the rubric ---Please read ALL questions carefully before you start to write anything."

Those types of tests separate a group of students in two types:  those who have heard about such tests and those who haven't.  Those tests are like 'A-ha' interview questions and they say nothing of the intelligence of the people involved. 

Mister Fancypants
Monday, July 07, 2003

Stephen F. Austin State University's solution with set hours is just as bad as the problem it is trying to solve, IMO.

Student A may come up with a rather clever design in 1 hour whereas student B may need 10 hours to come up with a design half as good. 

Mister Fancypants
Monday, July 07, 2003

Mr. Fancy Pants,

I think the point of "encouraging" students to spend more time on development (vs. coding) is to build up that skill.  The more you use the design skill, the stronger it gets, the easier it is to apply and the more comfortable the coder is with it.

It's desirable to spend effort on design, because in the real world:

1.  Once programmers start coding, they usually don't go back to the design stage. They now have an emotional investment in the code they've written.  Poorly designed code locks them into MORE poorly designed code.

NO amount of coding will correct a bad design.

2.  The design stage is where you can make mistakes cheaper.  As one of my students said "erasers are cheap".

3. The stages that come after design take a LOT longer than students (or beginners) THINK they will. So if you think the project will take an hour, you spend 10 minutes on design. Then you find that it's a 20 hour project.

When I started out as an engineer I paid attention to how long I spent on the design phase.  It started at about 1% of project time. It's now at about 30% and my software project are developed much more quickly.  (To give you an idea of that: I've developed 18 commerical applications in about 2 years, while simultaneously marketing and selling them internationally.  Programming took up about 30 to 40% of that time).

CONVERSELY, you don't want to end up in analysis paralysis where you're too AFRAID to start coding because you don't think you know enough.

My rule is that I need to have an idea of how it would be DONE MANUALLY.  If you can't do the process manually you won't be able to code it.

Naturally, there are times that you need to iterate. That's not excluded by spending lots of time on design.

Entrepreneur
Monday, July 07, 2003

Uck. All this up-front design is making me a little sick.

I've NEVER had a project where doing complete up front design was anything other than a waste of time. Designs need to evolve, and the only way to get that evolution is to get something simple working.

Is that "rush to code?"

Chris Tavares
Monday, July 07, 2003

Of course the idea of a bad design being preventable by lots of up-front work is one of those things that's nice in theory but not always so in practice.  It takes a good deal of experience and/or luck to design completely up-front.

More often than not you can definitely expose a bad design through some up-front proof-of-concept coding because what you are dealing with is unknowns and the only way to deal with unknowns is to run into them.

I also would not say that no amount of coding will correct a bad design.  In my opinion these are not separate activities.  If in the process of coding you expose design flaws then the obvious thing to do is redesign using the knowledge you now have and recode.

Richard Kuo
Monday, July 07, 2003

My favorite test variations

1) From _Space Cadet_: a machine of gates and levers, with some elaborate rules that, when evaluated, determine that you cannot reach any of the scoring states from the initial conditions.

2) A highschool physics exam, in a class where the instructor made a big fuss over "significant digits".  The exam author got a bit careless and wrote one of the parameters of the problem with only a single digit of precision. 

Danil
Monday, July 07, 2003

Chris and Richard,

Let me get specific about "up front design".

We may be disagreeing about different things.  I think that there are certain up front questions that need to be answered. A lot of PRODUCTS (not just software) do NOT address these questions, contributing to thier failure.


I'll give an example from my own programs.

Here are the steps we now go through on our educational programs.

1. What is the goal of the program?
2. What are the benefits the program will provide to the user?
(I try to state this from a *good* marketing perspective . E.g., "Will increase reading ability from single sentences to paragraphs". "This will expand vocabulary by 500 words", etc.)

3.  How would a teacher teach this lesson?  (We do sort of a paper-prototyping here.  I say "if you could write anything down and have automated responses to the student, what would you show them?".  (This is sort of like Alan Cooper's "If it were magic" (from About Face).

4.  Then we paper prototype it just so we know how the user would LIKE it to work.

5. We do a quick mock up to show the user some of the things that we're will be effective.  (e.g., if we're doing speech synthesis we might present some sample lessons and see how effective that is).

This is the sort of Design that I'm talking about.  It's very effective in letting us know what we want the program to do.

Naturally, there are things y ou can NOT simulate easily on paper. Likewise, it'll seem different to the user when it's on the computer. But we're talking to teachers who understand this gap b/t paper and computer and they can "visualize" what it would look like on paper.

We used to do the above steps IMPLICITILY (USUALLY). I.e., we'd sorta already done them subsconsciously. However, we've found that making this exlicit is very helpful.

The chief benefit is that it reveals when you have NOT gone through these steps  (subsconscously or otherwise)

My underlyng philosphy is that you should know WHAT you want your program to do.  You should know that by doing it MANUALLY first.  So, if it's a billing system, do a few iterations of the whole process MANUALLY on PAPER.

A lot of people implement systems on the computer that they really don't fully understand.  Then they finish it and wonder why no one will buy it.



QUESTION:
Does the above seem like to much up front design work?

Entrepreneur
Monday, July 07, 2003

Ahh, ok, I see what you're getting at.

You're talking about requirements gathering, and what Alan Cooper calls "Interaction Design." The user-visible stuff.

When you said design, I immediately thought about software design; what classes should be used, the inheritance hierarchy, etc.

I can agree that a paper prototype of the UI is very useful. I do that myself on occasion.

What I was worried about was up-front design of the system internals.

Chris Tavares
Monday, July 07, 2003

Chris,

I've only recently delved into visual inheritance.  But, it *seems* like you'd need to plan that out rather carefully. Certainly the examples I've read about inheritance in the Designed Patterns Explained book (by Shalloway et al) illustrates this point.

1. Have you used Visual Inheritance much?
(i.e., having one form inherit from another,e tc.)

2.  Has it saved on development time?

3.  Did you have to do a lot of up front planning to get decent benefit from it?

Entrepreneur
Monday, July 07, 2003

Entrepreneur,

Yes, that makes quite a bit more sense.  The high level focus of what really needs to be done is crucial.  We might do things a little differently here by focusing more on getting a prototype up and running that a user can at least sit in front of.  A lot of times the requirements are changing anyway (aka the user doesn't even know what he wants) and what we're looking to do is find the closest point that we can move to that will take us forward and let us reevaluate where to head next.

Richard Kuo
Monday, July 07, 2003

Visual inheritance? No, I've never really had a need for it.

Chris Tavares
Wednesday, July 09, 2003

How about teaching programmers how to test?  They seem to skip that part in college.  Programmers who value Quality Assurance and actually take a part in it are far and few inbetween.  I see a lot of 'Wow that's a great idea' on Monday turn into a production nightmare on Friday.  No requirements. No documentation, No design, No review, No Quality Assurance, No testing  - Just cowboy code running rampant.  Often, unit test plans are bulletted lists of psuedocode. No test data, no condition, no expected result - no traceability to a requirement, sometimes no requirement. No formal test methodologies applied, etc.  I think programmers should have to troubleshoot, test and analyze existing code before learning 'how to code'.  The perspective and learnings may take them much farther. 

Jen
Wednesday, July 16, 2003

*  Recent Topics

*  Fog Creek Home