Critical appraisal of the clinical evidence
The cart before the horse
I'm getting ready to add an Education section to the website, I decided to go back to first principles. Bloom's Taxonomy had a pervasive impact on my philosophy of learning, teaching, and cognitive complexity. I used it back in February of this year for an evidence-based medicine (EBM) presentation at work. Bloom's Taxonomy* stipulated six levels of "knowing" or cognitive complexity. The six levels in increasing order were knowledge, comprehension, application, analysis, synthesis, and evaluation.
Here is the conundrum that Bloom's Taxonomy exacts upon applied EBM practice:
There are five steps to EBM: Asking, acquiring, appraising, applying, and assessing.
With asking, the EBM literature posits that clinicians experience "cognitive dissonance" when they have a knowledge gap in their clinical practice. In order to deter the dissonance, the clinician decides to ask a clinical question to fill that gap.
With acquiring, the clinician uses the PICO (population, intervention, comparator, outcome) mnemonic to acquire the best clinical evidence, given the resources and time available.
Now we get to critical appraisal of the literature. When looking at the nomenclature of the word "appraisal," it is reflective of the highest level of "knowing" or cognitive complexity in Bloom's Taxonomy, evaluation. EBM stipulates that clinicians must be able to critically appraise the methods and statistical analyses of published studies. This means that clinicians have to have functioning at a very high cognitive level to do this correctly.
However, past literature has shown that researchers feel anxious and intimidated by statistics due to a lack of experience and competency.** Also, undergraduate and graduate medical training rarely equips clinicians with the necessary competencies to conduct and effectively interpret clinical research evidence.***
So, how can your everyday clinician with limited empirical/statistical training who feels "cognitive dissonance" a second time in the five steps of EBM critically appraise the literature? Therein lies the conundrum, in my opinion.
I'm positing that we need to refocus our efforts on the lower echelons of Bloom's Taxonomy by educating physicians, residents, fellows, faculty, pharmacists, nurses, and staff to better understand (knowledge), recognize (comprehension), choose (application), examine (analysis), and design (synthesis) research studies before we can expect them to critically appraise (evaluation) the literature.
Here is the conundrum that Bloom's Taxonomy exacts upon applied EBM practice:
There are five steps to EBM: Asking, acquiring, appraising, applying, and assessing.
With asking, the EBM literature posits that clinicians experience "cognitive dissonance" when they have a knowledge gap in their clinical practice. In order to deter the dissonance, the clinician decides to ask a clinical question to fill that gap.
With acquiring, the clinician uses the PICO (population, intervention, comparator, outcome) mnemonic to acquire the best clinical evidence, given the resources and time available.
Now we get to critical appraisal of the literature. When looking at the nomenclature of the word "appraisal," it is reflective of the highest level of "knowing" or cognitive complexity in Bloom's Taxonomy, evaluation. EBM stipulates that clinicians must be able to critically appraise the methods and statistical analyses of published studies. This means that clinicians have to have functioning at a very high cognitive level to do this correctly.
However, past literature has shown that researchers feel anxious and intimidated by statistics due to a lack of experience and competency.** Also, undergraduate and graduate medical training rarely equips clinicians with the necessary competencies to conduct and effectively interpret clinical research evidence.***
So, how can your everyday clinician with limited empirical/statistical training who feels "cognitive dissonance" a second time in the five steps of EBM critically appraise the literature? Therein lies the conundrum, in my opinion.
I'm positing that we need to refocus our efforts on the lower echelons of Bloom's Taxonomy by educating physicians, residents, fellows, faculty, pharmacists, nurses, and staff to better understand (knowledge), recognize (comprehension), choose (application), examine (analysis), and design (synthesis) research studies before we can expect them to critically appraise (evaluation) the literature.
*Bloom, B. S.; Engelhart, M. D.; Furst, E. J.; Hill, W. H.;Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.
**Marquardt, DW. Criteria for evaluating the performance of statistical consultants in industry. The American Statistician 1981; 35; 216-219.
***Wegwarth O. Statistical illiteracy in residents: What they do not learn today will hurt their patients tomorrow. Journal of Graduate Medical Education 2013; 5; 340-341.
**Marquardt, DW. Criteria for evaluating the performance of statistical consultants in industry. The American Statistician 1981; 35; 216-219.
***Wegwarth O. Statistical illiteracy in residents: What they do not learn today will hurt their patients tomorrow. Journal of Graduate Medical Education 2013; 5; 340-341.