Fog Creek Software
Discussion Board




QA process for games

I was wondering if they do anything differently when testing games. Is it more people, more time, a different process altogether, or what? Especially console games. I've never seen a console game hang and I think that's remarkable.

TJ Haeser
Tuesday, June 15, 2004

"I've never seen a console game hang"

They don't run on Windows.

sgf
Tuesday, June 15, 2004

Consoles are tightly regulated environments, so it's a lot easier writing software for them than for Windows.


Tuesday, June 15, 2004

Console games have to be approved by the console maker's QA (as part of the license for the API you have to use), and they don't allow any kind of crashes.

The process is just internal QA for a month or 6 weeks and fix everything you can during 90 hour weeks, then submit it to Sony/Nintendo/MS and hope for the best. If they come back with 'A' bugs (crashes), you keep fixing and repeating until it's approved.

Console Programmer Ted
Tuesday, June 15, 2004

Interesting. Thanks.

TJ Haeser
Wednesday, June 16, 2004

"I've never seen a console game hang..."

I hadn't either till ten or fifteen minutes into Metroid Prime.  Twice in a row.  And two different types of hang, to boot; one was a plain old screen freeze, while in the other the graphics went all screwy and it became totally unplayable.

(Fortunately it wasn't my game or console so I didn't have to care that much.)

Kyralessa
Wednesday, June 16, 2004

First, the Dreamcast ran Windows CE, so the "not Windows" explanation doesn't work.

Second, the problems with Metroid Prime might have been caused by overheating.  It's pretty easy to accidentally block the air intake and exhaust vents on the GameCube, and that'll lead to overheating problems if you play for a while.

Emperor Norton
Wednesday, June 16, 2004

+"First, the Dreamcast ran Windows CE"

Only the first few batch of games, mostly quick and dirty PC conversions. After the initial releases, WinCE-based games were relatively rare.

TJ Haeser
Wednesday, June 16, 2004

Whether they're running CE or not, the environment is much more stable. Your application (game) is not going to face situations where it's only got a tenth the memory that was available on the test machine.


Wednesday, June 16, 2004

AFAIK, using Windows CE in Dreamcast was optional.

YF
Wednesday, June 16, 2004

Console games not crashing (as much) has nothing to do with Windows and everything to do with the fact that consoles are a fixed system.  The hardware doesn't change, the drivers don't change, etc.  Much easier to QA in this sort of enviornment.  But if you want to keep beating the "Windows is unstable" drum, go ahead.  It must be nice to still be living in 1999, buddy...

Anyway, as console games get ever more complex, the amount of subtle bugs in them has been climbing as even with a fixed spec system, the more complex the software becomes the harder it is to QA everything...

Metroid Prime DOES have at least one confirmed software bug that causes the game to lockup once in a while when playing the elevator lift animation after you board an elevator.  The recent Thief 3 has a confirmed bug (on both the PC and the Xbox version) that causes it to reset the difficulty level of the AI characters back to normal any time you save or load a game, etc...

MrFancypants
Wednesday, June 16, 2004

Also..

It is true that very few developers used the Windows CE enviornment for Dreamcast programming.  But they *all* use a modified version of the Win2K OS on the Xbox, and Xbox games are no more or less prone to crashes than any other current console, in my experience.

MrFancypants
Wednesday, June 16, 2004

There are two main reasons:

1. Consoles are a known platform, everything is the same on all of them. (not quite true, but close enough)

2. In order to release your game you have to pass Sony (or MS, or Nintendo)'s TRCs - if your game crashes at all then it will fail. This means not crashing is a very high priority for developers.

Mr Jack
Wednesday, June 16, 2004

*  Recent Topics

*  Fog Creek Home