Fog Creek Software
Discussion Board




Do you know of a free data-analysis tool?

Anyone knows of a free data-analysis tool that aids market researchers enter and analyse data collected from surveys etc.?

I've searched Google and have also used Excel. Anything else?

Sathyaish Chakravarthy
Friday, November 21, 2003

Freshmeat is your friend

http://freshmeat.net/search/?q=data+analysis&section=projects

So is sourcefoge. Use google to search this one though... their search sucks.

data analysis site:sourceforge.net

Looks like there are quite a few tools that might suit your requirements.

Tapiwa
Friday, November 21, 2003

"When we believe that we are analysing data we are really only trying out our stock of existing ideas to see which one might fit. It is true that if our stock of ideas is rich then our analysis will be adequate.

But the analysis of data will not by itself produce new ideas. This is a rather important point, because the whole basis of science and progress is based on the belief that the analysis of data will produce all the ideas we need in order to move forward. In fact, the creator of new ideas must do a lot of `idea work' in his or her mind and then check our these ideas against the data. Just analysing the data is not enough. "

-- Edward de Bono

Tapiwa
Friday, November 21, 2003

"This is a rather important point, because the whole basis of science and progress is based on the belief that the analysis of data will produce all the ideas we need in order to move forward"

All respect to Mr de Bono but this looks like a very weak straw man.

Dominic Fitzpatrick
Friday, November 21, 2003

Indeed it leaves out things like theoretical physics, doesn't it? "We have observed that X happens. We don't really know why, because current data doesn't allow for that (e.g. whatever causes it cannot be measured by todays instruments)."


Friday, November 21, 2003

I'm with Edward on this one.

The problem with these theoretical sciences, and sciences by observation is that they can lead to false conclusions

Look at the data below.


1 is a prime number
3 ditto
31 ditto
331 ditto
3331 ditto
33331 ditto
333331 ditto
3333331 ditto
33333331 ditto
333333331 is *NOT* a prime number

There are many similar cases where there are exceptions to apprant observations.

Read a book called "Does God play dice?" for more on 'science' by observation.

http://www.amazon.com/exec/obidos/tg/detail/-/0631232516/002-9287697-9666423?v=glance

Tapiwa
Friday, November 21, 2003

I don't dispute that observation produces errors and false claims.

However, I have never come across the suggestion in science that ideas are generated by data analysis or even observation since reading about Francis Bacon.

In the prime number example, there is the idea that "all numbers of the pattern 3...1 (err, can't embed TeX can I?) are prime. This is easily disproven, as you did. If someone comes to the conclusion that all numbers in that pattern are prime, they would be incompetent, not guilty of allowing data to produce ideas.

I am not even sure what it would mean to just analyse data without any preconcieved ideas on what you are looking for.

Dominic Fitzpatrick
Friday, November 21, 2003

If you are thinking of statistical data analysis, R is a free version of (or inspired by) Splus. You can enter the data in Excel and analyse it using R (which is available for many operating systems). Have a look at http://www.r-project.org

There is also this page ( http://members.aol.com/johnp71/javasta2.html ), which has hundreds of links to free statistical software.

Concerning data analysis, it does not necessarily create new ideas, but it helps you to challenge concepts and reviewing your hypothesis. This is quite important in the process of modifying or creating ideas. O course, you can be taking a shower and have great ideas without any piece of data to support them ;-)

uncronopio
Friday, November 21, 2003

There is simply so much data to analyze that a scientist normally first makes the hypothesis and then looks at the data to see if it fits.

Indeed certain intellectual climates lend themselves to certain discoveries. Darwinism was based on Ricardo and Adam Smith, and Marx wished to dedicate 'Das Kapital' to Darwin but Darwin declined.

Stephen Jones
Saturday, November 22, 2003

At CERN in Geneva, Switzerland they develop ROOT.

http://root.cern.ch/

"The ROOT system provides a set of OO frameworks with all the functionality needed to handle and analyse large amounts of data in a very efficient way. Having the data defined as a set of objects, specialised storage methods are used to get direct access to the separate attributes of the selected objects, without having to touch the bulk of the data. Included are histograming methods in 1, 2 and 3 dimensions, curve fitting, function evaluation, minimisation, graphics and visualization classes to allow the easy setup of an analysis system that can query and process the data interactively or in batch mode."

Fredrik Svensson
Saturday, November 22, 2003

Back before Crystal acquired Seagate Software Seagate had a tool call Seagate analysis, there was a free version they passed out for a while, but no longer.

Li-fan Chen
Saturday, November 22, 2003

MS Office or Open Office both have analytics capable spreadsheet applications. Open is free, but Office is standard on most office desktops. You'll be able to do some pretty nice analysis on these tools. Either one can connect to remote data sources. On another note, SQL Server's data warehousing comes free with the RDBMS.

Li-fan Chen
Saturday, November 22, 2003

Free?

There's no such a thing as free lunch dude. Not in America!

timothy not in MA
Monday, November 24, 2003

*  Recent Topics

*  Fog Creek Home