.png){kind=logo}
AI & NLP
AQuA
Augmented Quality Assessment
AQuA seeks to develop capacity and increase the pace of translation quality assurance by harnessing the latest artificial intelligence (AI) techniques to assist human reviewers in objectively assessing multiple facets of translation quality. AQuA is able to produce an increasingly detailed suite of quality-related diagnostics, with at least five augmented quality assessment methods in the areas of accuracy, clarity, and naturalness.
Current Functionality
The AQuA team has been working in a multidisciplinary and cross-organizational manner to validate the following methods for efficiently probing the “big three” qualities of Bible translations:
Accuracy
Semantic similarity, agreement similarity, and word alignment
Clarity
Machine question answering
Naturalness
Consistency, reading level
The figure plots the output of these 5 methods to create a quality “fingerprint” of two different translations. In the example above, the plot indicates that the translation represented in blue is similar to the translation represented in green with respect to most qualities. The notable exception is clarity, where the blue translation shows noticeably more clarity.
Use in the Real World
Over the past year, the AQuA team has created a series of private beta prototypes and reports integrating its AI-driven quality metrics. These prototypes have been applied to analyze real world translations in 5 translations from Middle East/North Africa (MENA), Mexico and Southeast Asia.
A figure from one of these reports is included below. This report confirms (with AQuA) the state of a draft prior to and after review, where one can see AQuA-generated quality warnings addressed during the review process. This also confirms that the review process is improving quality (with respect to accuracy in this case) and that AQuA is generating warnings aligned with the human quality assessment process.
