Conclusion - Reasoning About the Design and Execution of Research - MCAT Physics and Math Review

MCAT Physics and Math Review

Chapter 11: Reasoning About the Design and Execution of Research


In this chapter, we focused on one of the four Strategic Inquiry and Reasoning Skills that will be tested on the MCAT: reasoning about the design and execution of research. We began by reviewing the scientific method and the value of historical data in the formulation of a research question. We then compared the methodology for both basic sciences research and human subjects research, especially with regard to error. We finished our investigation by examining the ethical and practical concerns in research design. The questions for this chapter are designed to allow you to practice this new skill, rather than to test your memorization of this content. In the next chapter, we’ll specifically work with data and graphical analysis, another of the Strategic Inquiry and Reasoning Skills that will be essential on Test Day.

Concept Summary

The Scientific Method

· The scientific method is a series of eight steps for the generation of new knowledge.

o The initial steps (generate a testable question, gather data and resources, form a hypothesis) focus on generating a hypothesis.

o The intermediate steps (collect new data, analyze the data, interpret the data and existing hypothesis) focus on testing that hypothesis.

o The final steps (publish and verify results) relate to providing the results for further testing of the hypothesis.

· The FINER method assesses the value of a research question on the basis of whether or not it is feasible, interesting, novel, ethical, and relevant.

Basic Science Research

· Basic science research uses chemicals, cell cultures, or animal subjects and is experiment-based.

· During research, we manipulate independent variables and observe changes in the dependent variable.

· Controls are used to correct for any influences of an intervention that are not part of the mdel. Controls may be positive or negative.

o Positive controls ensure that a change in the dependent variable occurs when expected.

o Negative controls ensure that no change in the dependent variable occurs when none is expected.

· Basic science research is often the best type for demonstrating causality because the experimenter has the highest degree of control over the experimental conditions.

· Error in basic science research most often results from errors in measurement.

o Accuracy (validity) is the quality of approximating the true value.

o Precision (reliability) is the quality of being consistent in approximations.

Human Subjects Research

· Human subjects research is subject to ethical constraints that are generally absent in basic science research.

· Experiments may still be performed, but causal conclusions are harder to determine because circumstances are harder to control.

· Much of human subjects research is observational.

o Cohort studies record exposures throughout time and then assess the rate of a certain outcome.

o Cross-sectional studies assess both exposure and outcome at the same point in time.

o Case–control studies assess outcome status and then assess for exposure history.

o Causality in observational studies is supported by Hill’s criteria, which include temporality, strength, dose–response relationships, consistency, plausibility, consideration of alternate explanations, experiments, specificity, and coherence.

· Error may be in the form of bias, confounding, or random error.

· Bias is systematic and results from a problem during data collection.

o Selection bias, in which the sample differs from the population, is most common in human subjects research.

o Detection bias arises from educated professionals using their knowledge in an inconsistent way by searching for an outcome disproportionately in certain populations.

o The Hawthorne effect results from changes in behavior—by the subject, experimenter, or both—that occur as a result of the knowledge that the subject is being observed.

· Confounding is an error in data analysis that results from a common connection of both the dependent and independent variable to a third variable.


· Medical ethics generally refers to the four principles of beneficence, nonmaleficence, respect for patient autonomy, and justice.

· Research ethics were established by the Belmont Report.

o Respect for persons includes autonomy, informed consent, and confidentiality.

o Justice dictates which study questions are worth pursuing and which subjects to use.

o Beneficence requires us to do the most good with the least harm. We cannot perform an intervention without equipoise—a lack of knowledge about which arm of the research study is better for the subject.

Research in the Real World

· Populations are all of the individuals who share a set of characteristics. Population data are called parameters.

· Samples are a subset of a population that are used to estimate population data. Sample data are called statistics.

· Internal validity refers to the identification of causality in a study between the independent and dependent variables. External validity refers to the ability of a study to be generalized to the population that it describes.

· In order to be supported, an intervention must display both statistical and clinical significance.

o Statistical significance refers to the low likelihood of the experimental findings being due to chance.

o Clinical significance refers to the usefulness or importance of experimental findings to patient care or patient outcomes.

Answers to Concept Checks

· 11.1


1. How do medical errors relate to sleep deprivation of medical residents? This is a current topic of investigation and a consensus in the scientific community is still being reached. Medical residents are available for interview, and the research has very relevant outcomes.

2. What is the average lifespan of bacteria in Martian rocks? While it is not very feasible to acquire the Martian rocks, the results would be both novel and interesting.

3. How long does it take the Earth to complete one revolution around the Sun? This question has been asked and answered to the satisfaction of the scientific community. It is neither novel nor interesting (in terms of further research).

2. Errors during publication of current studies adversely affect the quality of future experimentation by providing an incomplete or flawed research base. Without accurate resources, subsequent hypotheses are likely to be flawed.

3. False. While the statement may or may not be true, this is not an easily testable hypothesis. While not required, if–then formatting of a hypothesis necessarily implies a testable relationship between ideas.

· 11.2

1. This experiment would likely have inaccuracy error but not imprecision error. In other words, the scale would reliably read the same mass or weight, but the mass or weight it reads is not correct. This would lead to bias in the results.


3. Controls in experiments help to establish causality by demonstrating that the outcome does not occur in the absence of an intervention. Controls are used to keep the manipulations of different systems as similar as possible, or as a known standard against which to judge an experimental manipulation. Without controls, it is far more difficult to establish causality.

· 11.3

1. False. Temporality is the only necessary criterion from Hill’s criteria. If temporality is not satisfied, the relationship cannot be said to be causal. The addition of other criteria increases the probability of a causal relationship, assuming that temporality has not been invalidated.

2. Observational research does not involve manipulation of the subjects’ environment. It is generally less conclusive and more subjective than experimental research, which does involve manipulation of the subject or environment.

3. Bias is a systematic (unidirectional) error that occurs during the selection of subjects or the measurement and collection of data. Confounding is an error that occurs during data analysis, in which an association is erroneously drawn between two variables because of a shared connection to a third variable.

· 11.4

1. Autonomy is simply the right of an individual to make decisions on his or her own behalf and to have those decisions be respected. Respect for persons also requires honesty, confidentiality, informed consent, and freedom from coercion.

2. The company is violating the principle of justice by choosing participants that are not part of the target population. The company is also introducing selection bias.

3. The line between a coercive influence and a compensatory influence is often debated. In general, a compensatory influence is one that does not impact the decision to participate, while a coercive influence is one in which the subject loses autonomy to make the decision to participate.

4. Children, pregnant women, and prisoners are considered especially at risk for coercion and thus are granted special protections.

· 11.5

1. Internal validity is the tendency of the same experiment to produce the same results when repeated, and provides support for causality. External validity is the ability to take the information generated during research and apply it to a larger group. External validity is also called generalizability.

2. Small samples are subject to more random variation than large samples. If only one person is selected, he or she may be an outlier, but if a much larger sample is selected, an outlier will have less of an effect on the results.

3. A study must have both statistical significance and clinical significance to provide justification for an intervention. A study without statistical significance may be the result of random chance, whereas one without clinical significance will not impact patients.

Shared Concepts

· Behavioral Sciences Chapter 4

o Cognition, Consciousness, and Language

· Behavioral Sciences Chapter 8

o Social Processes, Attitudes, and Behavior

· Behavioral Sciences Chapter 11

o Social Structure and Demographics

· Behavioral Sciences Chapter 12

o Social Stratification

· Critical Analysis and Reasoning Skills Chapter 6

o Formal Logic

· Physics and Math Chapter 12

o Data-Based and Statistical Reasoning