Dr Teirstein Comments

The authors include highly paid employees of the ABIM. The study was funded by the ABIM.

As part of recertification, the American Board of Internal Medicine requires completion of at least 1 practice improvement module (PIM). The authors assessed whether completing an asthma-specific PIM resulted in improved patient outcomes.

The primary outcome was the dispensing of an inhaled corticosteroid (ICS) after an office visit for asthma. Secondary outcomes included patient reported processes of care, asthma-related heath care use, and asthma severity.

For the primary outcome, patients seen by intervention group physicians were not more likely to fill an ICS prescription in the post intervention period than patients seen by control group (adjusted odds ratio = 1.0). Patients seen for asthma by intervention group physicians were less likely to receive a written action plan than patients seen by control group physicians (adjusted odds ration 0.67). However, patients seen by the intervention group were more likely to discuss potential asthma triggers and had lower self-reported asthma severity measures (not the primary endpoint). The authors conclude that a “PIM designed to improve asthma care did not improve filling of ICS prescriptions but may have lessened asthma severity through an increased discussion of asthma triggers.”

This is an odd report in that it is used by ABIM in their marketing and also quoted in their editorials as supporting MOC, yet the study was undeniably negative. The only data point supporting the intervention was that patients had “lower self-reported asthma severity measures (unadjusted P=.03).” But this was not a pre-specified end point and the difference in asthma severity is never provided. All we are told in the results section is: “In the unadjusted analysis, patient-reported asthma severity (ie Asthma Symptom Utility Index score) was significantly lower in patients seen by physicians in the intervention group (P=.03) but was of borderline significance after adjustment (P=.09) (Table 5).” I am surprised ABMS continues to promote such a negative trial in their marketing materials as being supportive of MOC.

Dr Cohen Comments

The main strength of this particular study is that it is one of the few randomized trials in the field to assess the relationship between recertification (in this case, a specific practice improvement module [PIM]) and processes of care. The design of the study was a cluster randomized trial conducted at the practice level to assess whether performing a practice improvement module led to increased fill rates for inhaled steroids in asthma patients (an established quality measure). And as noted above, for the primary endpoint, the study was unequivocally negative (albeit somewhat underpowered). The authors did find a relationship between assignment to PIM completion and follow-up asthma severity, but this was a secondary (or possibly tertiary) endpoint. As such, the findings can only be considered hypothesis- generating despite the fact that they occurred in the setting of a randomized trial. Despite the authors’ arguments to the contrary, it is very likely that this positive finding (among a large number of endpoints tested) is a false positive. As such, I agree with Dr. Teirstein’s review that this study provides fairly weak evidence of the benefit of PIM in improving asthma care or outcomes. One thing to note about this study is that the rate of completion of the PIM intervention was quite modest, and this may have biased the study toward the null hypothesis.

Dr Ajay Kirtane Comments

This is the report of a cluster randomized trial of 16 practices that had physicians undergo a practice improvement model for asthma care. The primary outcome was fill rates for inhaled corticosteroids, and was no different among the intervention group vs. the control group. Remarkably, only 5 (26%) of physicians actually completed the intervention (despite 10/19 other physicians who initiated it), clearly biasing towards the null, but perhaps emphasizing the challenges of implementation of such a module (in a randomized trial that was designed to study it)! A written action plan was less likely to be received by patients treated in practices randomized to intervention, although these patients were more likely to discuss asthma triggers. Of note, while the unadjusted asthma acuity score was improved in the intervention group, this was not significant in adjusted analyses (only the unadjusted p value is reported in the abstract). Overall, this trial failed its primary endpoint, and among multiple secondary comparisons, only significant ones are reported, which is not appropriate.