Exploratory data analysis

Encyclopedia : E : EX : EXP : Exploratory data analysis

Exploratory data analysis (EDA) is that part of statistical practice concerned with reviewing, communicating and using data where there is a low level of knowledge about its cause system. It was so named by John Tukey. Many EDA techniques have been adopted into data mining and are being taught to young students as a way to introduce them to statistical thinking.

Tukey held that too much emphasis in statistics was placed on evaluating and testing given hypotheses (confirmatory data analysis) and that the balance was in need of redressing in favour of using data to suggest hypotheses to test. In particular, confusion of the two types of analysis and employing them on the same set of data can lead to bias owing to the issues endemic in testing hypotheses suggested by the data.

The objectives of EDA are to:

Suggest hypotheses about the causes of observed phenomena
Assess assumptions on which statistical inference will be based
Support the selection of appropriate statistical tools and techniques
Provide a basis for further data collection through surveys or experiments

The principal graphical tools used in EDA are:

Box plot
Histogram
MultiVari chart
Run chart
Pareto chart
Scatter plot
Stem-and-leaf plot

The principal quantitative tools are:

Median polish
Letter values
Resistant line
Resistant smooth
Rootogram

Software

[XLisp-Stat] (free software and Lisp based EDA development framework for Mac, PC and X-Windows)
[DataDesk] (free-to-try commercial EDA software for Mac and PC)
[Orange] (free component-based software for interactive EDA and machine learning)
[GGobi] (free interactive multivariate visualization software linked to R)
[MANET] (free Mac-only interactive EDA software)
[Mondrian] (free interactive software for EDA)
Fathom (for high-school and intro college courses)
TinkerPlots (for upper elementary and middle school students)

Bibliography

Hoaglin, D C; Mosteller, F & Tukey, J W (Eds) (1985) Exploring Data Tables, Trends and Shapes ISBN 0471097764
Hoaglin, D C; Mosteller, F & Tukey, J W (Eds) (1983) Understanding Robust and Exploratory Data Analysis ISBN 0471097772
Tukey, J W (1977) Exploratory Data Analysis ISBN 0201076160
Velleman, P F & Hoaglin, D C (1981) Applications, Basics and Computing of Exploratory Data Analysis ISBN 087150409X

From Wikipedia, the Free Encyclopedia. Original article here. Support Wikipedia by contributing or donating.
All text is available under the terms of the GNU Free Documentation License See Wikipedia Copyrights for details.