Disease severity (DI score) at RR activation rose slightly post-intervention with special cause variation

Description #

Contrary to the authors' expectation that earlier detection would lower acuity at activation, the average monthly Deterioration Index (DI) score at the time of RR system activation rose slightly from 41.65 pre-intervention to 42.97 post-intervention, and this rise showed special cause variation by Nelson rule 5 (two of three consecutive points more than 2 standard deviations from the center line) on the X bar chart, with a stable R chart (Fig. 4). The authors note the increase in DI score at activation accompanied by the decrease in mortality "suggests the need for further investigation."

"Average disease severity (DI scores) at the time of RR system activation showed special cause variation by Nelson rule 5 (two out of three consecutive points more than 2 standard deviations from the center line in the same direction) on the X bar chart, with a slight rise in the average DI scores per month for the post-intervention period (41.65 pre- to 42.97 post-intervention, Fig. 4)." (Raff, 2024, p. 1108)

"The increase in the DI score at the start of a RR system activation accompanied by a decrease in mortality afterward suggests the need for further investigation of the clinical effects of the intervention." (Raff, 2024, p. 1110)

Methods Context #

What? #ⓘ

The observable: monthly average Deterioration Index (DI) score — an Epic-embedded early warning score (0–100) representing the probability of an adverse event within 12 h or mortality within 36 h — recorded at the time of each RR system activation.

"The DI is a value from 0 to 100 generated every 15 min and represents the probability of an adverse event within the next 12 h or mortality within the next 36 h (see Appendix 2)." (Raff, 2024, p. 1105)

How? #ⓘ

Quasi-experimental pre-post QI design; monthly average DI-at-activation plotted on X bar and R control charts and tested for special cause variation with the Nelson Rules. Post-intervention, RR nurses monitored DI scores via the LEP dashboard at least every 12 h and used a DI threshold (≥60 or sustained ΔDI ≥10 over 4 h) to trigger evaluation.

"As shown in Figure 3, if a patient's DI score achieved a value ≥ 60 or exhibited a sustained change (≥ 4 h) of ≥ 10 since last review, a RR nurse examined the patient's medical record and consulted with the patient's primary bedside nurse." (Raff, 2024, p. 1106)

Who? #ⓘ

All adult non-ICU hospitalized LEP patients experiencing an RR system activation at a 950-bed quaternary academic medical center (UNCMC), May 2021–March 2023; monthly cross-sectional samples across 222 patients / 302 activations.

"Charts represent measures derived from monthly cross-sectional samples of all hospitalized general medical patients with LEP and occurrence of a RR system activation." (Raff, 2024, p. 1107)

Other Notes #

The authors had hypothesized the intervention would trigger RR activation earlier and at lower DI scores; the observed higher acuity at activation runs opposite to that expectation, complicating a simple "earlier-detection" mechanism story.

Caveats #

• Single-center pre-post QI pilot with a small sample and bundled intervention limits causal inference about the mortality effect This is a single-center, quasi-experimental pre-post pilot at one quaternary academic medical center, so findings may not generalize. The population was modest (222 patients; mortality counts of 8 pre vs 7 post) and the project period was constrained. The design supports only limited causal inference: the multi-component "bundle" (EWS dashboard, RR-nurse monitoring, EWS-triggered activation, and interpreter integration) was implemented together, so the mortality decrease cannot be attributed to interpreter integration specifically, and secular trends cannot be ruled out. The authors also could not comprehensively assess implementation fidelity or user perceptions of the intervention.