http://baw8.wordpress.com/2012/03/25/popps/#comment-68
http://itsstats.wordpress.com/2012/03/11/why-is-the-file-drawer-problem-a-problem/#comment-76
The File Drawer problem
‘The extreme view of the “file drawer problem” is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show non significant results.’
http://psycnet.apa.org/index.cfm?fa=fulltext.journal&jcode=bul&vol=86&issue=3&page=638&format=PDF
This was a statement by Robert Rosenthal in 1979 bringing to attention a publication bias called the file drawer problem. It is when research is completed but never gets published as the findings stand as ‘not statistically significant’. It is called the file drawer problem because it is assumed that these papers will be filed away and never published.
It is becoming increasingly concerning that there can be problems with only publishing significant results. It can mean that the majority of published studies are biased to indicate an effect when in fact there could be a large weight of papers that do not support the effect. For example there could be 50 studies which show that a certain type of medication does not help reduce the symptoms of schizophrenia, however there may be only 5 studies which show a significant result from the same drug. It is the 5 studies which are published, however, is this representative of the truth?
There are also less scientific and more practical problems such as the possibility for one researcher to carry out the same experiment as another, unknowingly, only to find that the results are non significant, which they did not know as the previous study was not published. This problem may be overcome by the use of peer reviews to determine which papers get published, however Mayo et al. (2006) states that there can be quite a difference between the views of the panel and therefore the decision has not always been thoroughly thought through. http://www.sciencedirect.com/science/article/pii/S0895435606000059
The following article ‘Unpublished results hide the decline effect’ talks about the decline effect and how unpublished research hid this from the public eye. The decline effect is when scientific claims receive decreasing support over time and it was found that ‘some effects diminish when tests are repeated’. These declining effects, however, have not been published and therefore the decline effect has been hidden.
http://www.nature.com/news/2011/110223/full/470437a.html
Overall, studies can be monitored as there are always processes such as funding which will draw attention to the flow of the study. Larger funded studies will be harder to forget and therefore harder to leave unpublished, therefore it is the smaller studies which need to be focussed on more (Light & Pillemer, 1984).
Homework for Naomi, sorry Naomi
Is there anything that can’t be measured by psychologists?
All psychologists are encouraged to take a scientific approach to their methodology. This means they have to study objectively. Psychologists, therefore, cannot measure things that have to be studied subjectively. For example, it would be difficult to measure emotions such as sadness because they cannot be measured by someone else objectively and it would be easier to measure objective variables such as reaction time.
There is also another factor that has to be taken into consideration when psychologists are studying. This is whether the variable they are studying is overt or covert. An overt variable may be a reaction time score or a memory recall score. These are overt behaviours as they are observable and they can be objectively measured by the psychologists. A covert variable would be sadness, for example, something which cannot be objectively measured by the psychologist.
These problems, however, can be solved using different methods. Psychologists can measure anything as long as it can be operationalised. This is when a researcher removes the ambiguity and defines the variable being measured it so that it can be measured and expressed quantitatively or qualitatively. Using the example of emotion, if a psychologist were to measure stress, they could use physiological measures such as heart rate or blood pressure (Cohen, 1983). The quantitative data collected could be used as a measure of stress. Problems of covert and subjective behaviors can also be measured using self report studies given by the psychologist (Bradley & Lang, 1994). This makes it possible for psychologists to measure variables such as emotions.
Intelligence is also difficult to measure objectively. The Binet-Simon intelligence scale, developed by French psychologists Alfred Binet and Theodore Simon, was administered to children to evaluate their intelligence/performance at a given age. They are given test item, puzzles etc in which they can receive scores on. The total score /IQ score rates the subject’s cognitive ability as compared to the general population.
To conclude, it is difficult for psychologists to measure covert and subjective variables, such as emotion, however it can be made possible by operationalising variables and by using self report studies.
Blog 2 – homework for my TA
Blog 2 – The methodology of the most recent SONA study that you have been a participant for
I know the suggestion was to write about the methodology of your last SONA experiment however I was not keen on the last SONA I did. I have decided to write about one I took part in last year which really interested me.
The experiment was a ‘3D object recognition task’ in which I had to wear 3D glasses for a computer task. The room was blacked out apart from the computer screen I was viewing. There were two computer tasks:
TASK 1 – I had to view 12 shapes/objects on the screen whilst wearing 3D glasses. I had to use any method possible to remember the shapes and also what order the shapes came in.
TASK 2 – This task involved recalling the shapes for example I was asked to judge whether the object/shape appearing on the screen was an object I has just learned or whether it was a new object. I was also asked to recall what number the shape was.
Requirements for this experiment were normal or corrected-to-normal vision and participants could not be colour-blind. This was to ensure that there were limited individual differences in the sample and that the results of the experiment could not be explained by any other factors.
I found this experiment particularly interesting due to the theory behind it. Once I had spoken to the experimenter I had learnt that they expected to see data showing that viewing 3D objects had no effect on memory or recall. She explained that viewing 3D in a cinema was not beneficial to our brains as it did not help us remember events more accurately or promptly, however, our brain has to work harder to process the 3D information. Due to the long term health issues with 3D imaging that they were slowly discovering, there should really be a health warning before a 3D showing in a cinema. This will not happen, unfortunately for another 10 years or so as more evidence needs to be produced to support these claims.
Take a look at this hub which presents the effects of 3D imaging
http://anjili.hubpages.com/hub/12-NEGATIVE-EFFECTS-OF-3D-TVS-ON-VIEWERS
Blog 1 – homework for TA
Blog 1 – The difference between a case study and single case designs
My aim of this blog is to explain to you the difference between a case study and single case designs. I am hoping to do this without the use of text book jargon as there is nothing worse than having to constantly refer to a thesaurus or Wikipedia when trying to read and comment on blogs. Here goes….
A case study is a research design that involves in depth description of a single individual or sometimes one very small group.Case studies are often used to provide information about rare and unusual behaviours, also to demonstrate exceptions to the rules.
An advantage of case studies is that they can bring to attention any new variables that might account for a particular outcome. Case studies however are often weak in external and internal validity as the theory and observations can only be generalised to that particular individual/case.
A very rare example of a case study was the girl Oxana Malaya who is now 28. She was found at the age of 8, in the Ukraine, living as a feral child. Having spent most of her life raised by dogs, she picked up many of their habits and norms. This was an extraordinary case that gave psychologists the opportunity to explore development and social issues. To learn more, you can check out this youtube video…
A single case design is also known as a single subject design. It is a research design that, like many others, aims to establish a cause and effect relationship. A single case design, however, uses the results from a single participant to do this. A series of observations are made over the participant. These are called the baseline phase and the treatment phase. The baseline phase is the stage at which observations are made in the absence of a treatment and the treatment phase is the stage at which observations are made during the treatment. The collection of these observations should show a pattern that describes the behaviour and the important part is the consistency or stability of this pattern. A researcher may change phases by applying or withdrawing a treatment and this is done to show that by adding or removing a treatment, there is an obvious change in behaviour. The researcher interprets the results not using SPSS or traditional tests, they must use graphs to interpret the results. This means that the graph must show a change in the behaviour more than once so it is clear that the first change was not due to chance.
You may be thinking that this is a very subjective method to interpret the results…it is. There are, therefore, four specific characteristics to help researchers determine whether changes are meaningful.
- Does the graph show change in the average level of behaviour?
- Does the graph show immediate change in level of behaviour?
- Does the graph show change in trend of behaviour?
- Does the graph show latency of change in behaviour?
The primary advantage of using a single case design is that they can use flexible methods to allow cause and effect relationships to be shown with one participant. The main disadvantage is that the results can only be generalised to this one individual.
Viewing both separately, it is easy to distinguish the differences. The main being that with single case designs, the researcher is in full control of what is going on in the experiment and they can manipulate it to whatever extent needed. In case studies, however, it is common that psychologists are not able to explore results until after changes have occurred. The cannot manipulate the process/development of the case.
I’m sorry this blog may seem a little rushed, it’s because I’m getting the train in an hour to go home for my 21st 😀
Week 9 – homework for my TA
Week 9 – Qualitative research isn’t as scientific as quantitative research….
Before we start on the debate, I think it is necessary to establish the definitions for qualitative research, quantitative research and scientific research. Qualitative research is research aims to produce information that is rich and in-depth through observations, case studies, interviews and questionnaires. Quantitative research is research that uses a ‘systematical and empirical investigation’ (Wikipedia) to measure variables in order to obtain numerical values that are then analysed for interpretation. The main difference between the two is the type of data that is produced. An example to demonstrate this would be a test of aggression. A qualitative researcher may investigate this using an unstructured interview in which the participants would talk about what made them aggressive or what they did when they feel aggressive. A quantitative researcher, however, may measure aggression on a scale for each participant and then produce an average score to express the amount of aggression for different groups.
Scientific methodology, according to The Oxford English Dictionary, is “systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.” In lamens terms, it is a method used to acquire empirical/factual and measurable knowledge.
Scientists would argue that qualitative research is not as scientific as quantitative. In Miles and Huberman’s book (1994) Qualitative Data Analysis, Fred Kerlinger states that “There’s no such thing as qualitative data. Everything is either 1 or 0.”They argue that because qualitative research does not need a hypothesis to begin research, it removes it from the category of ‘scientific’. Another aspect of qualitative research that poses a question as to whether it is scientific is the role of the researcher. A criterion for scientific methodology is that the researcher is testing objectively as they do not participate and they do not influence the research. How can this be concrete in qualitative research? Surely when observing or interviewing, there will be some degree of involvement and therefore subjectivity from the researcher.
Other scientists would argue that that qualitative research is as scientific as quantitative. For it to be scientific it needs to gain empirical and measurable knowledge and the qualitative methods that are used such as questionnaires and interviews do this. In response to the objectivity of the researcher, it is thought that the researcher can learn the most by participating or being immersed in a research situation. Information gained solely by quantitative methods does not take into consideration the environment, upbringing or situation of an individual and these factors are surely some of the most important when analysing data. With this extra information, provided by qualitative research, we are gaining more valuable knowledge about the world.
It may be thought that the easiest way to ensure scientific validity of your research and findings is to combine the two methodologies, however most scientists prefer to choose just one. In my opinion, both research methods are equally valid and equally scientific as they both gain knowledge about a phenomenon. Whether it was a qualitative or quantitative study, surely any gaining knowledge about the world is beneficial? I will leave that to you to think about….
To read up on the debate and see where most of this debate came from, please visit http://writing.colostate.edu/guides/research/observe/com2d3.cfm
psucb9 