Quality Dashboard

Quality Dashboard

Measuring and Benchmarking Translation Quality

Reduced Costs

More Flexibility

Higher Efficiency

The Quality Dashboard is an industry-shared platform which visualizes translation quality and productivity data in a flexible reporting environment. Both internal and external benchmarking is supported, offering the possibility to monitor your own development and compare your results to industry averages.

Dynamic Quality Framework

Translation quality is a pressing theme in the translation industry. The diversification in content types and rapid adoption of translation technologies (including machine translation) drives the need for more dynamic and reliable methods of quality evaluation.In order to meet this demand, TAUS developed the Dynamic Quality Framework (DQF), a comprehensive set of tools, best practices, metrics, reports, and data to help the industry set benchmarking standards. The vision behind DQF is to standardize the methods and tools for quality evaluation, aggregate the scores and measurements and make these available as industry-shared metrics.

DQF-MQM Error Typology

The Multidimensional Quality Metrics (MQM) is an error typology metric that was developed as part of the (EU-funded) QTLaunchPad project based on careful examination and extension of existing quality models. In a series of meetings, the developers of MQM and DQF agreed to make substantive changes to both DQF and MQMto bring them into harmony. The DQF-MQM metric can be used ‘stand-alone’ but is also available through the DQF open API.

Quality Dashboard

The Quality Dashboard delivers on the vision of DQF as an industry collaborative platform for the global translation services sector, helping all stakeholders – translation buyers and providers, technology developers, and translators – to get deeper insights into the processes and the technologies. Through an open API that connects their translation tools and workflow systems with DQF, translators and project, vendor and quality managers can track and benchmark the quality, productivity, and efficiency of translation.

DQF Data Connector

If you’re using your own dashboard tool, you can make use of the DQF Data Connector. The DQF Data Connector lets you generate a JSON string which allows you to visualize your data on your own, custom dashboard.

Integrations

A DQF integration allows seamless communication between your translation tool and the DQF platform so that you can keep working in your into the environment while the reports are generated in real time. Reports can be visualized on the TAUS Quality Dashboard or you can use your own.

In 2014 a survey was carried out, aimed at collecting user feedback on the TAUS Dynamic Quality Framework (DQF) tools and resources, and on translation Quality Evaluation (QE) in general. This report contains some use cases, results of the survey and recommendations on how to apply DQF to real life scenarios in the translation and localization business.

DQF-MQM Error Typology

The TAUS Dynamic Quality Framework (DQF) was developed in consultation with TAUS members. DQF includes various tools for the evaluation of translation quality, the error typology being one them. The Multidimensional Quality Metrics (MQM) is an error typology metric that was developed as part of the (EU-funded) QTLaunchPad project based on careful examination and extension of existing quality models. Despite the variety of approaches taken in industry and research, the two models turned out to be broadly similar, but they were also different in important ways due to their history.

In a series of meetings, the developers of MQM and DQF agreed to make substantive changes to both frameworks to bring them into harmony. The newly harmonized metric offers translation professionals a standard and dynamic model that can be used in every context. It can be used ‘stand-alone’ but is also available through the DQF open API .

The DQF-MQM Error Typology Template allows you to manually track your translation quality using the DQF-MQM Error Types. If you're interested in integrating DQF-MQM into your workflow, please check out the available DQF integrations contact our team via dqf@taus.net.

DQF-MQM Error Categories

Harmonized DQF-MQM Error Typology

ID

High-level error type

Granular error type

Definition

Example

1

Accuracy

The target text does not accurately reflect the source text, allowing for any differences authorized by specifications.

Translating the Italian word 'canali' into English as 'canals' instead of 'channels'.

11

Addition

The target text includes text not present in the source.

A translation includes portions of another translation that were inadvertently pasted into the document.

12

Omission

Content is missing from the translation that is present in the source.

A paragraph present in the source is missing in the translation.

13

Mistranslation

The target content does not accurately represent the source content.

A source text states that a medicine should not be administered in doses greater than 200 mg, but the translation states that it should be administered in doses greater than 200 mg (i.e., negation has been omitted).

14

Over-translation

The target text is more specific than the source text.

The source text refers to a boy but is translated with a word that applies only to young boys rather than the more general term.

15

Under-translation

The target text is less specific than the source text.

The source text uses words that refer to a specific type of military officer but the target text refers to military officers in general.

16

Untranslated

Content that should have been translated has been left untranslated.

A sentence in a Japanese document translated into English is left in Japanese.

17

Improper exact TM match

An translation is provided as an exact match from a translation memory (TM) system but is actually incorrect.

A TM system returns Press the Start button as an exact (100%) match when the proper translation should be Press the Begin button.

2

Fluency

Issues related to the form or content of a text, irrespective as to whether it is a translation or not.

A text has errors in it that prevent it from being understood.

21

Punctuation

is used incorrectly (for the locale or style).

An English text uses a semicolon where a comma should be used.

22

Spelling

Issues related to spelling of words.

The German word Zustellung is spelled Zustetlugn.

23

Grammar

Issues related to the grammar or syntax of the text, other than spelling and orthography.

An English text reads The man was seeing the his wife.

24

Grammatical register

The content uses the wrong grammatical register, such as using informal pronouns or verb forms when their formal counterparts are required.

A text used for a highly formal announcement uses the Norwegian du form instead of the expected De.

25

Inconsistency

The text shows internal inconsistency.

A text uses both app. and approx. for approximately.

26

Link/cross-reference

Links are inconsistent in the text.

An HTML file contains numerous links to other HTML files; some have been updated to reflect the appropriate language version while some point to the source language version.

27

Character encoding

Characters are garbled due to incorrect application of an encoding.

A text document in UTF-8 encoding is opened as ISO Latin-1, resulting in all upper ASCII characters being garbled.

3

Terminology

A term (domain-specific word) is translated with a term other than the one expected for the domain or otherwise specified.

A French text translates English e-mail as e-mail but terminology guidelines mandated that courriel be used.The English musicological term dog is translated (literally) into German as Hund instead of as Schnarre, as specified in a terminology database.

31

Inconsistent with termbase

A term is used inconsistently with a specified termbase.

A termbase specifies that the term USB memory stick should be used, but the text uses USB flash drive.

The translation of a light-hearted and humorous advertising campaign is in a serious and “heavy” style even though specifications said it should match the style of the source text.

41

Awkward

A text is written with an awkward style.

A text is written with many embedded clauses and an excessively wordy style. While the meaning can be understood, the text is very awkward and difficult to follow.

42

Company style

The text violates company/organization-specific style guidelines.

Company style states that passive sentences may not be used but the text uses passive sentences.

43

Inconsistent style

Style is inconsistent within a text.

One part of a text is written in a light and terse style while other sections are written in a more wordy style.

44

Third-party style

The text violates a third-party style guide.

Specifications stated that English text was to be formatted according to the Chicago Manual of Style, but the text delivered followed the American Psychological Association style guide.

45

Unidiomatic

The content is grammatical, but not idiomatic.

The following text appears in an English translation of a German letter: “We thanked him with heart” where “with heart” is an understandable, but non-idiomatic rendering, better stated as “heartily”.

5

Design

There is a problem relating to design aspects (vs. linguistic aspects) of the content.

A document is formatted incorrectly.

51

Length

There is a significant discrepancy between the source and the target text lengths.

An English sentence is 253 characters long but its German translation is 51 characters long.

52

Local formatting

Issues related to local formatting (rather than to overall layout concerns).

A portion of the text displays a (non-systematic) formatting problem (e.g., a single heading is formatted incorrectly, even though other headings appear properly).

53

Markup

Issues related to markup (codes used to represent structure or formatting of text, also known as tags).

Markup is used incorrectly, resulting in incorrect formatting.

54

Missing text

Existing text is missing in the final laid-out version.

A translation is complete, but during DTP a text box was inadvertently moved off the page and so the translated text does not appear in a rendered PDF version.

55

Truncation/text expansion

Truncation-text-expansion.

The German translation of an English string in a user interface runs off the edge of a dialogue box and cannot be read.

6

Locale convention

The text does not adhere to locale-specific mechanical conventions and violates requirements for the presentation of content in the target locale.

An incorrect format for currency is used for a German text, with a period (.) instead of a comma (,) as a thousands separator.A text translated into Japanese uses Western quote marks to indicate titles rather than the appropriate Japanese quote marks (「 and 」).

61

Address format

Content uses the wrong format for addresses.

An online form translated from English to Hindi requires a house number even though many addresses in India do not include a house number.

62

Date format

A text uses a date format inappropriate for its locale.

An English text has 2012-06-07 instead of the expected 06/07/2012.

63

Currency format

Content uses the wrong format for currency.

A text dealing with business transactions from English into Hindi assumes that all currencies will be expressed in simple units, while the convention in India is to give such prices in lakh rupees (100,000 rupees).

64

Measurement format

A text uses a measurement format inappropriate for its locale.

A text in France uses feet and inches and Fahrenheit temperatures.

65

Shortcut key

A translated software product uses shortcuts that do not conform to locale expectations or that make no sense for the locale.

A software product uses CTRL-S to save a file in Hungarian, rather than the appropriate CTRL-M (for menteni).

66

Telephone format

Content uses the wrong form for telephone numbers.

A German text presents a telephone number in the format (xxx) xxx - xxxx instead of the expected 0xx followed by a group of digits separated into groups by spaces.

7

Verity

The text makes statements that contradict the world of the text.

The text states that a feature is present on a certain model of automobile when in fact it is not available.

71

Culture-specific reference

Content inappropriately uses a culture-specific reference that will not be understandable to the intended audience.

An English text refers to steps in a process as First base, Second base, and Third base, and to successful completion as a Home run and uses other metaphors from baseball. These prove difficult to translate and confuse the target audience in Germany.

8

Other

Any other issues.

Severity levels for error categories

#

Definition

Description

1

Critical

Errors that may carry health, safety, legal or financial implications, violate geopolitical usage guidelines, damage the organization's reputation, cause the application to crash or negatively modify/misrepresent the functionality of a product or service, or which could be seen as offensive.

2

Major

Errors that may confuse or mislead the user or hinder proper use of the product/service due to significant change in meaning or because errors appear in a visible or important part of the content.

3

Minor

Errors that don't lead to loss of meaning and wouldn't confuse or mislead the user but would be noticed, would decrease stylistic quality, fluency or clarity, or would make the content less appealing.

4

Neutral

Used to log additional information, problems or changes to be made that don´t count as errors, e.g. they reflect a reviewer’s choice or preferred style, they are repeated errors or instruction/glossary changes not yet implemented, a change to be made that the translator is not aware of.

This white paper describes how the TAUS Dynamic Quality Framework (DQF) generates a Quality Dashboard serving the interests of all stakeholders in the global translation industry. Through easy-to-use plug-ins translators and managers share data and reports that give them valuable statistics, benchmarking.

Upcoming features 2017

Trend reports

Improving and extending project-based reports

Reviewing bugs and calculations in the current translation, correction and error annotation reports

Review on the data format for the charts

Extending functionality of the charts on the current translation, correction and error annotation reports with dropdowns for customer and vendor filters

Internal benchmarking by project

DQF Data Connector

If you’re using your own dashboard tool, you can make use of the DQF Data Connector. The DQF Data Connector lets you generate a JSON string which allows you to visualize your data on your own, custom dashboard.

Integrations

In order to generate quality and productivity reports on the Quality Dashboard, as well as help you track and benchmark your performance for your projects, you need to enable DQF in your CAT tool. A DQF integration allows seamless communication between your translation tool and the DQF platform so that you can keep working in your usual environment while the reports are generated in real time.

Select your CAT tool to get started...

DQF Plugin for SDL Trados Studio

About the TAUS DQF Plugin for SDL Trados Studio

With the DQF Plugin for SDL Trados Studio you can track translation quality and productivity from within the live SDL Trados Studio environment. The TAUS Quality Dashboard consequently allows you to visualize your data in customized reports. Learn how to get started with DQF for SDL Trados Studio, here on this web page.

Support Materials

Support materials

About SDL

SDL

SDL plc is a multinational software and professional services company headquartered in Maidenhead, UK. SDL specializes in; digital marketing software and services, structured content management and language translation software and services (including interpretation services). SDL are listed on the London Stock Exchange. SDL is an abbreviation for "Software and Documentation Localization".

For more information about the DQF Plugin for SDL Trados Studio 2015, please contact dqf@taus.net.

Make sure that before you start, you have created a TAUS account.

DQF for SDL TMS

DQF for SDL TMS should be available soon. For more information about DQF for TMS, please contact dqf@taus.net.

DQF for SDL WorldServer

About TAUS DQF for SDL WorldServer

The TAUS DQF for WorldServer connector enables project managers to add projects to TAUS DQF and track productivity and quality on the TAUS Quality Dashboard. The connector works in combination with the TAUS DQF for SDL Trados Studio plugin. The plugin can be downloaded from our website or the SDL AppStore free of charge. Please contact us at dqf@taus.net if you would like to purchase the WorldServer connector.

Support Materials

Support Materials

Demo Video

About

SDL plc

SDL plc is a multinational software and professional services company headquartered in Maidenhead, UK. SDL specializes in; digital marketing software and services, structured content management and language translation software and services (including interpretation services). SDL are listed on the London Stock Exchange. SDL is an abbreviation for "Software and Documentation Localization"

For more information about the DQF Plugin for SDL WorldServer, please contact dqf@taus.net.

Make sure that before you start, you have created a TAUS account.

DQF for XTM Cloud

About TAUS DQF for XTM

Thanks to the XTM LQA and TAUS DQF integration, enterprises can now benefit from enhanced reliability from translation service providers. The combined solution tracks translation quality and productivity giving you a better insight into performance, as well as reducing the time and cost of quality evaluation.

XTM and TAUS DQF Tutorial

About XTM International

XTM International

XTM is the world’s leading cloud based enterprise translation management solution with an integrated CAT tool. It is designed to be scalable, flexible and agile, with industry open standards at its core. For more information or a free trial go to www.xtm-intl.com

Make sure that before you start, you have created a TAUS account.

DQF for Lingotek

A DQF Plugin for Lingotek is not available yet. If you're interested in using DQF in Lingotek, please let us know via dqf@taus.net.

DQF for Memsource

A DQF Plugin for Memsource is not available yet. If you're interested in using DQF in Memsource, please let us know via dqf@taus.net.

DQF & MateCat

A DQF Plugin for MateCat is not available yet. If you're interested in using DQF in MateCat, please let us know via dqf@taus.net.

DQF for GlobalLink

About the TAUS DQF Plugin for GlobalLink

The GlobalLink suite from Translations.com allows clients to effortlessly transmit data to TAUS DQF as part of their localization workflows. It’s just a matter of inserting the “Submit to DQF” step into a workflow using GlobalLink’s visual modeler – from that point onward GlobalLink will send translation data and translator metrics to DQF automatically for each job, allowing clients to generate large amounts of data in DQF in a relatively short amount of time.

DQF for Kaleidoscope

DQF for Kaleidoscope should be available soon. For more information about DQF for Kaleidoscope, please contact dqf@taus.net.

DQF for Smartling

DQF for Smartling should be available soon. For more information about DQF for Smartling, please contact dqf@taus.net.

Request DQF API key

DQF Tools

DQF provides a vendor independent environment for evaluating translated content. Users can post-edit machine-translated segments to track productivity, evaluate adequacy and fluency of target sentences, compare translations and count errors based on an error-typology. The tools help establish return-on-investment and benchmark performance enabling users to take informed decisions.To ensure evaluation results are reliable it is vital that best practices are applied. The DQF Tools help users apply best practices whether they are selecting their preferred engine, measuring productivity or evaluating the quality of translations.

Please note that as of January 1st, 2017 DQF Tools is no longer supported by TAUS. DQF Tools will remain online for educational purposes. If you're interested in tracking and benchmarking translation quality and productivity in a professional setting, we advise you to use the Quality Dashboard or contact dqf@taus.net.

Content Profiling

The DQF Content Profiling feature is used to help select the most appropriate quality evaluation model for specific requirements. This leads to the knowledge base where you find best practices, metrics, step-by-step guides, reference templates, and use cases.Your underlying process, technology and resources will guide the choice of the quality evaluation model.