Personalized MD Curriculum in Personalized NSCLC Treatment Produces High, “Clinically Significant” Educational Effect Size

In non-small cell lung cancer (NSCLC), evidence points to the benefits of tumor biopsy for biomarker analysis, which in turn may allow individually targeted therapy [e.g., 1-3]. In the last five years of this age of pharmacogenomics and prognostic markers, the clinical excitement for individualized medicine has produced a robust count of 256 review articles indexed in PubMed found with a search on “non small cell lung cancer treatment biomarker review,” even with additional filtering to “Abstract [available], English, [and] Humans.” But diagnostics in surgery and pathology, as well as personalized treatment for cancer are expensive, so the societal context of the Affordable Care Act enacted five years ago (March 23rd, 2010, with its emphases on quality measures, patient-centered care, and accountability in care decisions) cannot be ignored.

Individualized intervention is not just important to cancer biology and treatment: it is important to clinical education, as well. Not only do clinicians caring for patients with cancer have their own knowledge and competence gaps—mainly because of the discovery of new evidence in this rapidly changing therapeutic area—they have the healthcare system context to work within, from local to national levels. The newly published, featured articleby Hermann et al focuses on NSCLCeducation in the quality-driven system environment of the ACA, titled, “EducationalOutcomes in the Era of the Affordable Care Act: Impact of PersonalizedEducation About Non-Small Cell Lung Cancer.” The authors argue for timely opportunities for immediate, practical, and translatable education for individual clinicians, as follows: “Quality medical education must be available when the health care provider is ready to learn, provide feedback, and maximize translation of knowledge from desk to clinic” [4].

The educational methods and instructional design are of greatest interest. Oncologists completed a pre-intervention self-assessment of knowledge, skills, and attitudes. This was used to develop an individualized learning plan and a personalized curriculum, which included up to 5 distinct activities selected to address identified knowledge and practice gaps. The activities were distributed online, and learners received feedback at the completion of each activity. Learners were tested on 5 knowledge and decision-making areas relevant to NSCLC treatment.  

The results of education were dramatic: “Completion of the learning plan was associated with a high effect size (d = .70),” a Cohen’s d that indicates that the educational intervention was much more meaningful than the statistically significant differences between learners’ pre- and post-intervention testing would suggest. (Remember that p values tell the statistician only how likely it is that the hypothesis could be accepted or rejected in error.) If one reviews the Effect Size (ES) lecturenotes provided by Dr. Lee Becker on his University of Colorado webpages, this translates to what Cohen himself (reluctantly) defined as a medium-to-large effect but which has become standard usage where historical data from research teams are not published with current results. This effect size surpasses even what Wolf (1986) identified as the lowest benchmark for change results that are “clinically significant,” not just educationally meaningful, at d = .50.

Looking at this educational study’s effect size more simply at Becker’s site, Cohen’s d = .70 means that 43.0% of participating learners (oncologists) had posttest scores that did not overlap with pretest scores, indicating learning that facilitates change. This is a big percentage when one considers that even an effect size of .20 (small) is difficult to achieve in one initiative. In other words, personalized education on NSCLC affected quality care. Kudos to the researchers.

P.S. For additional reading on Cohen's d and effect sizes in CEhp, check out the AssessCME blog written by my outcomes colleague, Jason Oliveri: assesscme.wordpress.com/category/effect-size.

References cited: 

1. Remark R, Becker C, Gomez JE, et al. The non-small cell lung cancer immune contexture. A major determinant of tumor characteristics and patient outcome. Am J Respir Crit Care Med. 2015;191(4):377-90.
2. Cagle PT, Allen TC, Olsen RJ. Lung cancer biomarkers: present status and future developments. Arch Pathol Lab Med. 2013 Sep;137(9):1191-8.
3. Raparia K, Villa C, DeCamp MM, Patel JD, Mehta MP. Molecular profiling in non-small cell lung cancer: a step toward personalized medicine. Arch Pathol Lab Med. 2013;137(4):481-91.
4. Herrmann T, Peters P, Williamson C, Rhodes E. Educational outcomes in the era of the Affordable Care Act: impact of personalized education about non-small cell lung cancer. J Contin Educ Health Prof. 2015;35(Suppl 1):S5-S12. [Featured Article]
5. Becker L. Effect size (ES). University of Colorado—Colorado Springs Website. http://www.uccs.edu/lbecker/effect-size.html. Accessed September 16, 2015.
MeSH *Major* terms: This study [4] is so new, NLM librarians have not yet assigned Medical Subject Headings. Check for updates at http://www.ncbi.nlm.nih.gov/pubmed/?term=26115247.