Written Critique And Research Protocol Program

Written Critique And Research Protocol Program

The essay has 2 critical sections:
1) A critical analysis of the article’s methods
2) A proposed research protocol with rationale & methods of a new study.

1) A critical analysis of the article’s methods
This section of the assignment will need to be a critical analysis of the research methods used in
the article chosen. It will test your ability to assess what methods were accurate, valid, or reliable
and their shortcomings.
What should this section contain?
• Start with what is this paper about: give a concise summary of what was done in the
research (research question(s), rationale, research techniques used, etc.). DO NOT exceed
½ page – save your words for the critique. If your assignment includes too much a
summary of the paper analysed, the mark will stay down.
• Make sure you provide evidence for any claim you make. Your points of critique need to
be valid, logical deductions or refer to supporting evidence (studies that already addressed
these points, if you found any). This piece is an academic critical analysis and involves a
scientific argument – so ‘just giving your opinion’, without that opinion being supported by
logic or evidence, is not enough.
• The critique could develop the following points (it is your choice what you focus on):
– Are the research questions clear?
– Are the hypotheses quantifiable?
– Were the experimental design, manipulations & measures appropriate for the research
questions asked?
– Were appropriate controls / comparison groups employed?
– Have the researchers overlooked any possible confounds or extraneous variables which
could affect interpretations of the findings?
– How do the methods employed compare with the methods used in other investigations
of the same topic/phenomenon?
– Do you think the authors conducted the experiments correctly? If no, why?
– Are the results reported and analysed correctly? -> The manner in which data are
reported and analysed could dramatically affect interpretation. For example, different
types or formats of graphs can emphasize or de-emphasize the sizes of effects, and
different inferential tests can yield different results.
– Have the appropriate interpretations of the results been made?
2) A proposed research protocol with rationale & methods of a new study
This section of the assignment will need to be a proposed new study. This new study can be a
logical extension of that paper’s findings, or a study designed to address the paper’s limitations or
ideas for further research. To help you think of a new study, think of the following questions:
Can you think of a better way to address the research questions then what the authors did?
OR
If you were to ask the authors to do a different experiment, what aspects do you think they
should modify/improve? 4
What should this section contain?
• The research question(s) or why are you conducting this research
– List each research question included in this study;
– For each research question listed previously, provide one or multiple specific and
testable hypotheses;
– State if hypotheses are directional or non-directional. If directional, state the direction.
• The rationale that justifies the research question
– In maximum ¾ page explain why you think that is it important to conduct this study:
What – from the article you just critically analyzed – justifies you planning this study?
Why is this study necessary? (in other words: Does it make up for a shortcoming in the
article’s studies? Will it address something they missed? Etc.)
• The methods that will be used to attempt answering the research question.
Provide precise information on each of the following sections:
Sampling Plan
Participants:
– What is the population from which you will recruit participants?
– How many participants will take part in your study?
– How did you decide the number of participants in your study? Did you conduct a
power analysis? (if yes, what were the parameters and where did you take the
effect size from?) If you did not conduct a power analysis, what other factors
have constraint your sample (e.g., time, money, personnel)?
– How will you recruit the participants? (e.g., using the Department’s Subject Pool,
directly contacting people in public places, on online data collection platforms
such as Prolific Academic, Amazon Mechanical Turk, Testable Minds, by
disseminating ads for your study on social media, etc);
– Is there a payment for participation and how much?
– What are the inclusion and exclusion rules?
– What’s your study timeline (proposed start and end of data collection)?
Stopping rule: If your data collection procedures do not give you full control over your exact
sample size, specify how you will decide when to terminate your data collection.
Variables
Manipulated variables:
– Describe all variables you plan to manipulate and the levels of each variable.
Measured variables:
– Describe each variable that you will measure, including outcome measures and
any predictors or covariates. Include any variables that you plan on collecting if
they are not going to be included in the confirmatory analyses of this study.
– Provide complete information about the measures you use: the construct
measured, the authors and year (reference), basic reliability info (e.g., alpha
Cronbach), what are participants asked to do and what scale they answer on
(i.e., how is it anchored – from what to what?)
Indices:
– Are any measures going to be combined into an index (or even a mean)? Which
ones and how? Include either a formula or a precise description of your method.
If you are using a more complicated statistical method to combine measures
(e.g., a factor analysis), you can note that here. 5
Design Plan
Study type:
– Is your study observational (i.e., data is collected from participants who are not
randomly assigned to a treatment)? This includes surveys, “natural
experiments,” and regression discontinuity designs.
– Or is your study experimental? If yes, how many cases does your design have?
– Blinding: Who is aware of the experimental manipulations within a study: Only
the participants, only the experimenter, or both? If no blinding is involved in the
study, specify that.
– Study design: What is your study design? If your study is observational, typical
designs include cohort, cross sectional, and case-control studies. If your study
is experimental, typical designs include two-group, factorial, randomized block,
and repeated measures. Is it a between (unpaired), within-subject (paired), or
mixed design? Describe any counterbalancing required.
– Randomization: If you are doing a randomized study, how will you randomize,
and at what level?
Analysis Plan
– How you expect to analyse your data? Depending on your design and the
variables involved, what would be the appropriate tests? Be specific: just saying
‘I am going to conduct an ANOVA’ is insufficient: you need to show that the test
you are planning to use is appropriate for the variables and design that you
have.
Statistical models:
– What statistical model will you use to test each hypothesis? Include the type of
model (e.g. ANOVA, multiple regression, SEM, etc) and the specification of the
model (this includes each variable that will be included as predictors, outcomes,
or covariates). Specify any interactions that will be tested.
Transformations:
– If you plan on transforming, centering, recoding the data, or will require a coding
scheme for categorical variables, please describe that process.
Follow-up analyses:
– Will you be conducting any follow up analyses on effects in your statistical
model? (e.g., If you plan to use a simple slope test to decompose an interaction,
you need to specify that)
Inference criteria:
– What criteria will you use to make inferences? (e.g. specify the p-values, Bayes
factors, specific model fit indices). Will you be using one or two tailed tests for
each of your analyses? If you are comparing multiple conditions or testing
multiple hypotheses, will you account for this?
Data exclusion:
– How will you determine which data points or samples (if any) to exclude from
your analyses? How will outliers be handled?
Exploratory analysis:
– If you plan to explore your data set to look for unexpected differences or
relationships, you may describe those tests here. An exploratory test is any test
where a prediction is not made up front, or there are multiple possible tests that
you are going to use.