Fog Creek Software
Discussion Board




Tracking use of options to improve software

Does anyone build any type of logging options into their software to track usage of options?

I am actually of the belief that I “know” what options are being used the most. However, that is not always the case. Here is a copy a log file from my tour software. The only real large surprise is the first option. I had no idea that this option was used that much (this is why I write logging software in my applications that keep track of options used). I will now endeavor to make sure that the users use the features in the software to *reduce* the number of steps to accomplish this most common task.

Option                Count
-------                ------
Print Invoice (Single)        508
Print Invoice (all)            176
Create Booking            139
Change Bus            119
Name Removed - payments kept    73
Swap room names            66
Remove Single            53
Add Single            53
Change room            37
Pull Name from existing invoice    14
Book To tour            11
Un Book From tour            10
Remove from Bus            5

Any other good ideas for using the software to actually tell me what options are being used? (in place of actually asking the users?).

Albert D. Kallal
Edmonton, Alberta Canada
Kallal@msn.com

Albert D. Kallal
Friday, May 17, 2002

Asking users: bad. Their memories will be faulty; and asking them to keep a log is a pain.

Automated logging like you did: brilliant! Thanks for the great example!

Martin L. Shoemaker
Friday, May 17, 2002

This is one of the benefits of on-line software.  Every mouse click and keypress is seen by the server which can then track it and produce reports on most used features, most retries, etc.

Of course for any softare, desktop or on-line, there are huge privacy concerns here.  I wouldn't want my spreadsheet logging and e-mailing all my activity to some company...

IanRae
Friday, May 17, 2002

Hmm, but this could be done for trusted users who beta-test.  It could even pretty-format the information sent to the mothership, so once a week the app could ask the user if she wants to send it out.

Greg
Friday, May 17, 2002

I like the idea, and I think it would be much easier to pull off for in-house developers.

Really cunning in-house developers would also capture the user ID, and analyze the logs to find out which features your boss, company executives, etc. used the most.  The revised, management-friendly version would then be released, say, a month before review time. ;-)

Nick Hebb
Friday, May 17, 2002

Yes it is done, though it's not common.  (I wrote about some results from such tracking in the "enough already Joel" discussion.)

Software that already includes code for scripting purposes or for profiling QA testing coverage is usually fairly easy to modify.  For the tooled wordprocessor I mentioned, we tracked every menu item and dialog box plus meta-information on data entry.  I don't recall the level of detail on the meta-information - it was along the lines of the number of charaters type in, any time the cusor was repositioned by the mouse, etc.  Basically, information on how they created and modified documents that wouldn't be captured by other means.

Of course, we were careful to keep the overhead down so that the performance hit was not noticeable and we did not collect any personal information.  In addition to the log, we also collected the file containing the customizeable options for the software to compare against the defaults.

Ron Zeno
Friday, May 17, 2002

I heard the Microsoft did this with an Excel beta many years ago.

Zwarm Monkey
Friday, May 17, 2002

I've gotta agree with whoever recommended that it be a beta-tester thing only. The possible privacy concerns of compiling this kind of thing into a release version are just a bit much for me to stomach. From a business standpoint, I wouldn't want to be on the wrong end of the lawsuit from cautious/paranoid users that feel their privacy is/could be violated.

Alex Russell
Friday, May 17, 2002

A few notes should be made here.

One thing that was suggested that this log idea should be only for beta. Actually, not at all! In fact, what I do is expose the log system to the users. This makes for a great built in audit system. I find that my users love this feature. Thus, if a user creates a hotel booking, but some other staff in the office changes that booking to a different room.....then who did it? The users can now easily check who did what. In other words, if Susan down the hall changed the room type, then most likely she knows what is going on here. My users love this, since if there is some type of screw up, then the correct person can be contacted. (wrong person cannot get blamed!).  It is not viewed by my users as a tracking system, but in fact just helps the person on the phone, since they all know who last dealt with the customer. Hence, I get double use out of this log system.

Hence, I don’t just “print out a log”, but I have a menu option called “booking History”, and this option simply is one of the menu options in the menu bar. It of course only displays the current information for the current booking (in a grid). This idea can be used for just about any type of applications. For me, I have some queries that “summarize the data”.

Here is a example of what the log looks like (I doubt this will be readable -- can we imbedded HTML in this discussion group?)


  Time And Date  Initials network  Work        Action
                                        Logon    station

3/19/02 10:08:50  TD  reception  RECEPTION  Change Bus                                       
3/21/02 16:39:49  TD  groups    GROUPS      Print Invoice (all)                             
3/21/02 16:41:39  TD  groups    GROUPS      Print Invoice (all)                             
3/27/02 15:20:26  TD  tammy      TAMMYBOOK  Change room                                     
3/27/02 15:20:28  TD  tammy      TAMMYBOOK  Add Single                                       
3/27/02 15:20:29  TD  tammy      TAMMYBOOK  Add Single                                       
3/27/02 15:20:48  TD  tammy      TAMMYBOOK  Pull Name from exisitng invoice                 
4/02/02 16:44:36    SS  reception  RECEPTION  Print Invoice (all)

A few more notes:

The “Action” text is actually another table, and thus the size of the log is not that large (in other words, I don’t store the action text for each booking, but only a id number). Same thing with the time, and date field...it is simply a date field (which allows for both date and time). Hence, the actual size of a log entry is quite small, only about 30 bytes.

So, don't just view a log as a big brother thing. Expose it to your users...and they in general will love it, and not view it as a evil.

Albert D. Kallal
Edmonton, Alberta Canada
Kallal@msn.com

Albert D. Kallal
Saturday, May 18, 2002

*  Recent Topics

*  Fog Creek Home