Machine-translation postediting was 2 to 10 times faster than human-translation postediting (11.76 vs 3.30 WPM)

Description #

Human postediting of the machine-translated (Google Translate) Spanish documents took between 28 and 85 minutes per document, at 10.32 to 13.40 words per minute (WPM), averaging 11.76 WPM. By contrast, estimated postediting of the human-translated versions of the same documents ran 1 to 12 hours at 1.58 to 5.88 WPM, averaging 3.30 WPM. The calculated MT postediting was therefore 2 to 10 times faster than HT postediting (Table 3).

"The calculated postediting times following MT were 2 to 10 times faster than the estimates of the postediting times following HT (Table 3)." (Turner, 2014, p. 527)

"Postediting of MT documents took between 28 and 85 minutes and WPM ranged from 10.32 to 13.40, with an average time of 11.76 WPM." (Turner, 2014, p. 527)

"The estimated times for postediting the human-translated documents ranged from 1 to 12 hours and WPM ranged from 1.58 to 5.88, with an average time of 3.30 WPM." (Turner, 2014, p. 527)

Methods Context #

What? #ⓘ

The observable: postediting speed — words per minute (and total minutes) needed for a fluent reviewer to correct the initial translation for grammar and accuracy.

"Words per minute (WPM) are calculated using the total time and number of English words." (Turner, 2014, p. 526)

How? #ⓘ

Three documents (<1500 words) were machine-translated into Spanish with A-0016ArtifactA-0016Initial AI draftGoogle TranslateA free, general-purpose machine translation app increasingly used as an ad-hoc communication tool in healthcare settings to bridge language barriers with LEP patients, including via voice-to-voice translation. "One such…, and postditors' correction time was recorded in a timed laboratory session (correcting only grammar/accuracy, not style); HT postediting times were estimated from Phase-2 self-reports of the same documents.

"We chose 3 of the 11 documents from the time and task analysis, based on the type of document, language, and length (<1500 words) to translate into Spanish using MT. Postditors conducted timed postediting of the translations using the same process as described in Phase 1." (Turner, 2014, p. 526)

Who? #ⓘ

Three Spanish public health promotion documents, with postediting performed by native Spanish-speaking public health professionals recruited for the study.

"We recruited native Spanish-speaking public health professionals (n = 12)" (Turner, 2014, p. 524)

Other Notes #

MT postediting was measured directly (observed times) whereas HT postediting time was estimated from self-report, and MT postediting occurred in a controlled laboratory session rather than in normal field workflow — both limitations are captured as caveats on this EVD.

Caveats #

• Machine-translation postediting times were measured in a controlled lab setting not normal field workflow The machine-translation postediting times were obtained in a controlled laboratory setting, whereas in real public health practice such documents would be postedited as part of a normal, interruption-prone workflow. Lab conditions may yield faster and more consistent postediting than field conditions, so the observed MT postediting speed (and hence the measured MT-versus-HT speed gap) may not transfer directly to routine operational use.
• Human-translation postediting times were self-reported estimates rather than directly observed The postediting speed comparison is not a like-for-like measurement: the human-translation (HT) postediting times were reconstructed from staff self-reports of how much time they had spent, whereas the machine-translation postediting times were directly observed in timed sessions. Self-reported time estimates are subject to recall and estimation bias, so the HT side of the 2-to-10-times-faster comparison may be imprecise, weakening confidence in the exact size of the speed advantage.