Advantaged points of data analysis
As stated above, data collection and presentation is an important and necessary step in all psychological researches. Without data presentation, the researches lack of the most convincible tool. Therefore, data presentation is prevailing in all aspects of scientific studies. “Industrial companies spend billions of dollars each year implementing enterprise applications or integrating customer or product data, and industry estimates show these projects fail or go over budget 65-75 percent of the time.” (http://www.dataflux.com)
Likewise, nowadays, psychologists want to create a new era for the complicated descriptions on a social issue or a psychological research. Instead of just going on a long and bored essay, psychologists add some charts with numerical orders on them allowing readers to understand the issue faster and simpler. Mean, median, and mode, all descriptive statistics that are discussed above, are present in most of psychological researches. Nominal data, normal distribution (data following a bell-shape curve), ordinal data, or outlier... are also having much significance or values on many psychological surveys. Statistics are considered as “reliability” or “validility”. Reader usually feel confident when seeing much data, many social statistical charts, and statistical figures in a research. Consequently, “data analysis is the process of looking at and summarizing data with the intent to extract useful information and develop conclusions.” (http://en.wikipedia.org). With the support of data analysis, scientists or not, people believed that numbers are unquesionable correct. People trust numbers more than words. They accept a short study with much data and many graphs easier than an elaborated essay.
There seems to be a pervasive notion that “you can prove anything with statistics.” (http://my.execpc.com/~helberg/pitfalls/). Indeed, there are many problems that data analysis can’t solve: bias selections and preparations, methodological errors, result interpretations.
First of all, bias selections or preparation. To have groups of true and reliable samples, “the sample must be similar to the target population in all relevant aspects, and certain aspects of the measured variables must conform to assumptions which underline the statistical procedures to be applied.” (http://my.execpc.com/~helberg/pitfalls/). Usually, the researchers randomly select a group of samples whom they believe can represent all other groups nationwide. For example, two groups of students at two different junior high schools, ages from 12 to 15, are randomly selected to be the samples for a survey. The students answer perform the test seriously. The results are considered accurate. However, if looking at the survey from a bigger view, there are many factors that severely influence the survey: both locations of the schools are at rural sides, the students mostly come from working families, the teachers are not professional, and the survey take place at a down time of real estates business. Therefore, the students in the survey are neither representatives of the whole generation, nor the students in general. Because the students learn from the same teacher, they may convey the same notions about the same thing. They don’t have their own opinions. The selecting process is a bias.
Secondly, methodology error. A researcher may have both Type I Error and Type II Error. An adult patient has Attention Deficit Disorder. He can’t focus in aything for 15 minutes. He also had nightmares that wake him up at night several times. A psychologist rejects the null hypothesis (the patient is born that way) and said that the client had nightmares from an accident that happened to him years ago. The psychologist had Type I Error because in fact, the client had a tumor in his brain since his birth. Type II occures when the psychologist does not reject the second null hypothesis (the nightmares are caused by the accident. Indeed, the accident happened two years ago but the patient had ADD four years before that). The psychologist had both Type I and II Error. Besides, there may be some unecessary comparisons. Some psychologist create many charts that confuse the readers. Last but not least, the measurements of noise, of happiness, of knowledge.. sometimes lead the reader to a confusion.
Thirdly, result interpretation. Many errors happen when psychologists try to interpret the significance. “There is still a strong tendency for people to equate stars in tables with importance of results. "Oh, the p-value was less than .001--that's a really big effect," we hear our clients say. Well, as I pointed out earlier, significance (in the statistical sense) is really as much a function of sample size and experimental design as it is a function of strength of relationship. With low power, you may be overlooking a really useful relationship; with excessive power, you may be finding microscopic effects with no real practical value.” (http://my.execpc.com/~helberg/pitfalls/). Then, presision and Accuracy is another factor that influence the result of a survey. For instance, 3.0224 is more 3.0 but people usually take 3.0 in short. And, when a researcher says that the mean of this issue is 1.2, he is wrong because, in fact, the mean is 1.297. It is almost 1.3, not 1.2.
Many times, the interpretations of researches are not appropriate due to lack of experiments performed nationwide. In case a researcher really wants to have an accurate and simple data analysis, he needs to travel a lot, doing a lot more researches everywhere,
take many samples regardless he must spend most of his life for that study. Practically, nobody can do that, except only big companies that controls many continents.