S-MINDS concept-based speech translation system
Purpose #
Why might someone use this thing?
S-MINDS (Speaking Multilingual Interactive Natural Dialog System), developed by Fluential Inc. and evaluated with the UCSF School of Pharmacy, is a bidirectional English–Spanish speech translation device intended to let a clinician deliver spoken medication-counseling recommendations to a limited-English-proficiency (LEP) patient — and receive the patient's spoken response back — when an in-person human interpreter is not available.
"Fluential Inc. and the University of California, San Francisco (UCSF) School of Pharmacy Center for Self-Care collaborated to demonstrate the usefulness of Fluential's English-to-Spanish speech translation system (S-MINDS©—Speaking Multilingual Interactive Natural Dialog System) for Spanish-speaking LEP patients with diabetes." (Soller, 2012, p. 928)
Mechanism #
How does this thing fulfill its purpose?
Rather than free-text machine translation, S-MINDS uses a concept-based approach: the clinician's spoken utterance is matched to a pre-built domain concept (here, medication action plan recommendations) which is then rendered as a validated audio translation, with optional on-screen text choices for confirmation at each step. The concept-based design is what lets it accept longer utterances (up to 20 words) and resist degradation from ambient noise and speech disfluencies that defeat general-purpose speech recognition.
"S-MINDS is a concept-based speech translation approach that can be applied to bridge technologic gaps limiting the complexity and length of utterances that can be recognized and translated by devices." (Soller, 2012, p. 928)
"The communication steps of S-MINDS are (1) speech initiation of a medication-related recommendation by the English-speaking practitioner, (2) visual choice by the English-speaking practitioner of the correct corresponding English sentence from three options on the smart phone screen, (3) audio Spanish-language translation of the visual choice by S-MINDS, (4) verbal restatement of the audio sentence by the Spanish-speaking LEP patient, (5) choice by the patient from three Spanish-language written options on the smart phone screen, and (6) audio English translation by S-MINDS" (Soller, 2012, p. 928)
Example(s) of usage #
Describe at least one concrete example of this artifact in use.
In the evaluation, a pharmacist delivered single-concept diabetes medication action plan (MAP) recommendations (e.g., "Test your blood sugar twice a day, when you wake up and 1 to 2 h after lunch") to Spanish-speaking LEP patients through S-MINDS running as an iPhone app, with the patient verbalizing the Spanish version back in a teach-back step.
"Twenty-one single-concept MAP recommendations were selected from 25 LEP patient visits with two MTM-trained pharmacists and over 1000 visits from our other clinical services." (Soller, 2012, p. 929)
Other Notes #
Deployed and tested as an Apple iOS iPhone application ("Fluential UCSF iPhone App v0.1", Table 1). In the comparative laboratory assessment, Fluential's concept-and-translate component was also layered onto the automatic speech recognition of Dragon, Jibbigo, and Google to test hybrid configurations.