Personalized MD Curriculum in Personalized NSCLC Treatment Produces High, “Clinically Significant” Educational Effect Size

In non-small cell lung cancer (NSCLC), evidence points to the benefits of tumor biopsy for biomarker analysis, which in turn may allow individually targeted therapy [e.g., 1-3]. In the last five years of this age of pharmacogenomics and prognostic markers, the clinical excitement for individualized medicine has produced a robust count of 256 review articles indexed in PubMed found with a search on “non small cell lung cancer treatment biomarker review,” even with additional filtering to “Abstract [available], English, [and] Humans.” But diagnostics in surgery and pathology, as well as personalized treatment for cancer are expensive, so the societal context of the Affordable Care Act enacted five years ago (March 23rd, 2010, with its emphases on quality measures, patient-centered care, and accountability in care decisions) cannot be ignored.

Individualized intervention is not just important to cancer biology and treatment: it is important to clinical education, as well. Not only do clinicians caring for patients with cancer have their own knowledge and competence gaps—mainly because of the discovery of new evidence in this rapidly changing therapeutic area—they have the healthcare system context to work within, from local to national levels. The newly published, featured articleby Hermann et al focuses on NSCLCeducation in the quality-driven system environment of the ACA, titled, “EducationalOutcomes in the Era of the Affordable Care Act: Impact of PersonalizedEducation About Non-Small Cell Lung Cancer.” The authors argue for timely opportunities for immediate, practical, and translatable education for individual clinicians, as follows: “Quality medical education must be available when the health care provider is ready to learn, provide feedback, and maximize translation of knowledge from desk to clinic” [4].

The educational methods and instructional design are of greatest interest. Oncologists completed a pre-intervention self-assessment of knowledge, skills, and attitudes. This was used to develop an individualized learning plan and a personalized curriculum, which included up to 5 distinct activities selected to address identified knowledge and practice gaps. The activities were distributed online, and learners received feedback at the completion of each activity. Learners were tested on 5 knowledge and decision-making areas relevant to NSCLC treatment.  

The results of education were dramatic: “Completion of the learning plan was associated with a high effect size (d = .70),” a Cohen’s d that indicates that the educational intervention was much more meaningful than the statistically significant differences between learners’ pre- and post-intervention testing would suggest. (Remember that p values tell the statistician only how likely it is that the hypothesis could be accepted or rejected in error.) If one reviews the Effect Size (ES) lecturenotes provided by Dr. Lee Becker on his University of Colorado webpages, this translates to what Cohen himself (reluctantly) defined as a medium-to-large effect but which has become standard usage where historical data from research teams are not published with current results. This effect size surpasses even what Wolf (1986) identified as the lowest benchmark for change results that are “clinically significant,” not just educationally meaningful, at d = .50.

Looking at this educational study’s effect size more simply at Becker’s site, Cohen’s d = .70 means that 43.0% of participating learners (oncologists) had posttest scores that did not overlap with pretest scores, indicating learning that facilitates change. This is a big percentage when one considers that even an effect size of .20 (small) is difficult to achieve in one initiative. In other words, personalized education on NSCLC affected quality care. Kudos to the researchers.

P.S. For additional reading on Cohen's d and effect sizes in CEhp, check out the AssessCME blog written by my outcomes colleague, Jason Oliveri: assesscme.wordpress.com/category/effect-size.

References cited: 

1. Remark R, Becker C, Gomez JE, et al. The non-small cell lung cancer immune contexture. A major determinant of tumor characteristics and patient outcome. Am J Respir Crit Care Med. 2015;191(4):377-90.
2. Cagle PT, Allen TC, Olsen RJ. Lung cancer biomarkers: present status and future developments. Arch Pathol Lab Med. 2013 Sep;137(9):1191-8.
3. Raparia K, Villa C, DeCamp MM, Patel JD, Mehta MP. Molecular profiling in non-small cell lung cancer: a step toward personalized medicine. Arch Pathol Lab Med. 2013;137(4):481-91.
4. Herrmann T, Peters P, Williamson C, Rhodes E. Educational outcomes in the era of the Affordable Care Act: impact of personalized education about non-small cell lung cancer. J Contin Educ Health Prof. 2015;35(Suppl 1):S5-S12. [Featured Article]
5. Becker L. Effect size (ES). University of Colorado—Colorado Springs Website. http://www.uccs.edu/lbecker/effect-size.html. Accessed September 16, 2015.
MeSH *Major* terms: This study [4] is so new, NLM librarians have not yet assigned Medical Subject Headings. Check for updates at http://www.ncbi.nlm.nih.gov/pubmed/?term=26115247. 

Eight-year Canadian study on opioid prescribing among regulator- and self-referred physicians to intensive course

This educational study in a clinical journal by Kahan et al at the University of Toronto examined “the effects of an intensive 2-day course on physicians' prescribing of opioids” [1]. The most impressive feature of this study is its eight-year-plus data-gathering period of opioid-prescribing levels among participating physicians, most of whom were family physicians. Other interesting features are worth mentioning, in both instructional design and study design.

The study design grouped participants into self-referred physicians vs. physicians who were referred by medical regulators, and added a control (nonparticipant) group. Undertaking a challenging matching procedure, researchers matched nonparticipants according to specific variables, including quarterly rates of opioid-prescribing, expressed as milligrams of morphine equivalent. Subgroups of participant groups with very high opioid-prescribing patterns were also identified; unfortunately, nonparticipants to match these participants were difficult to find. Yet this targeted approach to matching is appropriate and represents a significant investment of the researchers’ time, allowing the comparative group findings shown below. Nonparticipants were added to the study concurrently with their matched participants, per an “index date” defined as “the date of course completion for participating physicians. Control physicians were assigned the same index date as their matched pair.” In one deviation from the primary outcome measure, matching was done by number of opioid prescriptions rather than milligrams of morphine equivalent. Another study design feature is the specific comparison of opioid-prescribing rates for 2 years before vs. 2 years after the educational intervention, again by group and subgroup vs. nonparticipants; participants who could not be matched were analyzed separately from participants with matched pairs.

The instructional design of the 2-day course incorporated several educational settings and modalities. Planners used didactic presentations but added problem-based case discussions and mock-interview learning interactions with standardized patients who offered feedback. Pros and cons of changing prescribing patterns were discussed in a session at the end of the course, featuring a faculty interview with a patient. The course also provided a detailed syllabus with notes and references before the course, as well as office materials. It should be noted that benzodiazepine-prescribing was also addressed in course content. Finally, each 2-day course enrolled up to 12 participants, a limit that would confer an individualized learning environment and some professional privacy in what might be a sensitive concern among participating physicians.

The authors noted in the introduction, “Medical education has been suggested as one strategy to improve opioid prescribing among physicians” [2,3] and “Educational interventions focused on opioid prescribing lead to positive improvement in physicians’ knowledge and self-reported practices” [4]. Let's look at results by reported subgroup.

Among physicians referred by medical regulators, “the rate of opioid prescribing decreased dramatically in the year before course participation compared with matched control physicians,” and “the course had no added effect on the rate of physicians' opioid prescribing in the subsequent 2 years.” It seems that these physicians might have changed their behavior by arbitrarily reducing prescribing rates because of the regulatory investigation, even without an educational intervention to inform their clinical decision-making. In fact, the authors acknowledge this, noting, “We measured only the quantity of opioids prescribed, not the quality of opioid prescribing.” The regulatory concerns may have created a false baseline for the educational study that measured only quantity of opioid prescribed rather than patient-selection or other measure of competence.

Among the self-referred physicians who were matched to nonparticipants, “there was no statistically significant effect on the rate of opioid prescribing observed” from baseline to 2-year follow-up, although there had been a temporary decrease, particularly in prescribing for patients aged 15 – 64 (here’s a nice graph with patient ages: http://www.cfp.ca/content/59/5/e231/F4.expansion.html). On the other hand, “the rate of opioid prescribing decreased by 43.9% in the year following course completion” among self-referred physicians with high prescribing rates who could not be matched, suggesting that these physicians “might have responded to what was taught in the course.”  

References cited:
1. Kahan M, Gomes T, Juurlink DN, et al. Effect of a course-based intervention and effect of medical regulation on physicians’ opioid prescribing. Can Fam Physician. 2013;59(5):e231-e239. http://www.cfp.ca/content/59/5/e231.full.pdf+html.
[Featured Article]
2. College of Physicians and Surgeons of Ontario. Avoiding Abuse, Achieving a Balance: Tackling the Opioid Public Health Crisis. Toronto, ON: College of Physicians and Surgeons of Ontario; 2010.
3. National Opioid Use Guideline Group. Canadian Guideline for Safe and Effective Use of Opioids for Chronic Non-Cancer Pain. Hamilton, ON: National Opioid Use Guideline Group; 2010.
4. Midmer D, Kahan M, Marlow B. Effects of a distance learning program on physicians’ opioid- and benzodiazepine-prescribing skills. J Contin Educ Health Prof. 2006;26(4):294-301.
Free full text PDF: http://www.cfp.ca/content/59/5/e231.full.pdf.
MeSH *Major* terms: Analgesics, Opioid/therapeutic use*; Drug Prescriptions/standards*; Education, Medical, Continuing*; Physician's Practice Patterns/standards*