LEP and English-primary patients engaged equally with the multilingual chatbot (12.3 vs 12.2 responses, P=.959)
Source
Joshua P Rainey (2023). A Multilingual Chatbot Can Effectively Engage Arthroplasty Patients Who Have Limited English Proficiency. Journal of Arthroplasty.
Description #
Among TJA patients enrolled in the multilingual SMS chatbot, LEP patients responded to a mean of 12.3 ± 16.6 (range 0-80) chatbot messages versus 12.2 ± 12.1 (range 0-106) responses by patients with English as their primary language (EPL) — a difference that was not statistically significant (P = .959) (Fig. 3). The figure further shows no significant engagement difference between the two groups either before or after surgery. Engagement was thus effectively equal across the language groups.
"On average, LEP patients responded to 12.3 ± 16.6 (range 0-80) SMS chatbot messages in comparison to 12.2 ± 12.1 (range 0-106) message responses by EPL patients, which was not significantly different (P = .959), Figure 3." (Rainey, 2023, p. S79)
"Number of patient responses to chatbot messages before and after surgery in those patients enrolled in the chatbot service. (No significant difference seen between English speakers and non-English speakers before or after surgery.)" (Rainey, 2023, p. S81, Fig. 3)
Methods Context #
What? #ⓘ
The observable: patient engagement, operationalized as the total count of patient-generated text messages sent to the chatbot across the perioperative period.
"Patient engagement was measured using the total number of patient-generated text messages sent to the chatbot during both the preoperative and postoperative period." (Rainey, 2023, p. S79)
How? #ⓘ
Retrospective review comparing message-response counts between LEP and EPL patients enrolled in the A-0004ArtifactA-0004Initial AI draftStreaMD multilingual orthopaedic SMS chatbotTo coach total joint arthroplasty (TJA) patients through the perioperative period in their own language via automated text messages, delivering perioperative education and remote monitoring to patients — including those…, analyzed with independent t-tests.
"The multilingual SMS chatbot service used in this study was provided by StreaMD (Chicago, IL)." (Rainey, 2023, p. S79)
Who? #ⓘ
The 1,282 TJA patients (705 knees, 577 hips) enrolled in the chatbot at a single institution (University of Utah) from 2020-2022; of these, 47 received their messages in a language other than English (LEP) and the remainder were EPL.
"1,282 TJA patient were identified (705 knees, 577 hips), and of these patients, 47 were enrolled to receive their chatbot messages in a language other than English." (Rainey, 2023, p. S79)
Other Notes #
This is a null (no-difference) result and is reported here as such; it supports an equivalence claim (LEP engage the chatbot as much as EPL patients), not a superiority claim.