With growing content, shorter release dates and many target languages, it’s important for global companies to have a process in place to track and measure translation effectiveness. Learn how big companies like Microsoft, Snap-on Diagnostics and Symantec manage translation quality.

1. Language Quality Management:
Models, Measures, Methodologies
How do others do it?

2. Introductions
Gráinne Maycock
Vice President of Sales
Sajan, Inc.
• 18 years in localization industry
• Degree in applied languages
• Passion for language and technology
• Advisory board member Localization
World
• Advisory board member Brand2Global
• Build global business solutions for:
• Global translation management
• Language quality management
• TMS – CMS integration
• Machine translation and post-editing
• Global marketing content

3. Quality Expectations: We All Have
Them

4. Language Quality Matters
• Translation quality is critical to
your brand and company’s
international success.
• Brand, message and product
consistency need to be
maintained across all
languages as well as accurate
terminology and local nuance
and regulatory differences.
Your style also needs to
“translate” across markets.
Predictive Quality
Segment Level Revision Analysis

5. Scoring and Comparisons

6. Active Real-Time Metric Management
“Early in Process”

7. Quality Matters
• But does one size fit all?
• Is there one overarching definition?
o One industry standard model?
o One way to approach resourcing?
• What role does technology play?
• What are the organizational challenges to
optimizing language quality?
• And the challenges?

8. Three Companies Tell Their Story

9. Language Quality Management
for Microsoft Office
• Over 60 different applications and services.
• Moving to multiple platforms
• Continuous releases
• Over 100 languages supported
• Over 3,000 people involved
• Over 70 million User Interface resources
managed in our TMS.
• Localisation as a service model
partnering with our localisation vendors on:
•Linguistic service
•Translation service
•Linguistic testing and support

10. How do we approach language quality?
Measures
Linguistic
(monthly)
Define a reference!
For each language, focusing
on Style and terminology.
• Feedback received
• Regular style update
• Language reforms
• Terminology update
Translatio
n
(daily)
Check the translation!
Translation errors against
both the English source and
approved terminology/style.
• Proofing editing
• Self certification
• LQA cross checks
• Automated functional
testing
Visual
(monthly)
Verify fit and finish.
Check the translation in the
context of the actual UI.
• Screenshot reviews
• Scenario based reviews
User
experience
(Quarterly)
Collect user experience.
Check that the translation
matches the way a feature
works.
• Scenario based reviews
• Internal feedback
• External feedback
• Subject matter experts
Very classic approach to software quality, our challenge is to keep it agile!

11. Major quality challenges
Agile
•Can we break the 24 hours turn around barrier?
•Difficult to predict upcoming volume
•Continuous localisation + continuous release
MT + Post editing
•How efficient is MTPE for a User interface localisation?
Context and disambiguation

12. Addressing scale and agility: Context example
Problem: Lack of context is the #1 cause of mistranslation for us.
Today:
• Leaf environment supports already various types of context
• Key terminology
• English screenshots (when available)
• Translation suggestions
• Context query plug-in:
• Translators should receive an answer within 4 days max
• Leverage questions and answers across language teams
Future:
• Improve the relevance of the context provided
• Provide terminology disambiguation
• Reduce the context lag
• Anticipate the most common context queries
• Fast track English screenshots in Leaf
Measuring success in that space
QTO: Quality of Translation Output
% of resources changed 1-month
after sign off.
This metric can be drilled further
down by root cause of the change
and should help us drive future
investments in our translation
environment.
Measuring success in that space
QTO: Quality of Translation Output
% of resources changed 1-month
after sign off.
This metric can be drilled further
down by root cause of the change
and should help us drive future
investments in our translation
environment.

13. 13
Linguistic Quality for
the entire Symantec
Customer Experience
Linguistic Quality for
the entire Symantec
Customer Experience
Our Mandate
• Global Team of Language Quality Process
Managers
• Terminology Mgt, Linguistic Review, Machine
Translation, TM Management, Language Coordination
• ~ 30 Languages supported
• ~ 1000 signoffs on products per year
• ~ 25,000 signoffs on quality for product-related items per
year
• Main Goals:
• Improve upstream quality
• Provide Linguistic Standards and Manage Linguistic
Assets (Terminology Database and TM)
• Quality Control

14. 14
What Does Quality Mean To Us?
*True to Function
*Unambiguous
*Right for Purpose
*Fluent
*On Message
*Right for Audience
*Speed
*Correct Product references
Terminology, Content Standards
Consistent Rating Scale

15. 15
How?
Process, Tools & Automation
Upstream Work and
Standard Setting
*Content Analysis
*Terminology Harvesting,
Translation, SEO
*Translation Brief
Translation Enablement
*Terminology Portal
*Searchable Context UI Tool
*Translator-Reviewer Network
Quality Control
*Linguistic Review Tool
*Consistent Rating for data
collection
*Translator-Reviewer Feedback
Loops
Machine Translation:
*Users enabled to machine-
translate Support content
*Small amount of human
translation

16. 16
Challenges
• Subject matter expertise, creativity
• Updating context in agile development process
• Agile cross-vendor collaboration on terminology and translation questions
• Timing of Linguistic QA in agile process
• Improving MT quality for Support

17. Outlook – What We Need Next
People:
•Collaboration across our vendors on shared platforms
•Partnership and expertise: vendors provide us on best practice
and expertise
Technology:
•Translators enabled in context environment
•Environment enables fast-cycle translation by dedicated experts
•Enabling user-based MT output improvement
17

18. Conor Clune
Product Development Manager
Localization Business Unit

19. Leading global innovator, manufacturer and marketer of tools, equipment,
diagnostics, repair information and systems solutions for professional users
performing critical tasks.
•1920 - Founded
•1931 - Entered the International arena
•Today – Products and services can be found in more than 130 countries
Products and services include diagnostics tools (hardware and software),
information and management systems and other solutions for automotive
dealerships and repair centers.
The content solutions is highly technical and specialized, which means it
can be difficult for people outside the industry to understand what is being
said.

20. Small team… 4 internal, 3 external and great business partners
Various content types…
•Vehicle Communications Software… 66 million word count a day processed
•Component Test Meter… 10 million word count a year
•System Software
•Software delivery systems
•Tech Publishing's… 1+ million word count a year
Tools Set… all developed in house and with business partners
Localization Business Unit

21. Scenario…
Translator is processing large volume of strings (60,000 word
count)
Quality is key to our business

22. Scenario…
High fuzzy matches presented to the translator who has been
working on the task for 6-7 hours.
Quality is key to our business

23. Quality is key to our business
Scenario…
Herr Hertz (Mechanic) in the repair shop with Scanner tool is
instructed to replace the Rear Right ABS system instead of the
Rear Left…
Message:
English: RR Wheel ABS Ctrl Status: Fault
German: Status Rad ABS-Steuerung hinten links: Fehler
Potential Result…
The customers ABS malfunctions on the
Autobahn

24. • When a translator/linguist is reviewing
thousands of strings over hours, mistakes will
happen… snow blindness.
• How do you build in fail safes?
– Education
– Critical Term Glossary
– Automation
Quality is key to our business

25. • Education
– Work with the business partner to educate the
global translation team on the risks and impact
– Identify processes for risk avoidance
• Critical Term Glossary
– Build a list of know terms that could have negative
consequences
– Match a source term with translated terms and
its synonyms
Quality is key to our business

26. • Automation
– Develop tools for different stages of the process
– Source: Identify critical misleading terms (full
form and abbreviated)
– Translation: All content is checked against CTG
• Translators run automation
• Loc Engineers run automation
– Internal:
• Snap-on team review the automation reports from
Business Partner… sanity check and offer feedback
• Further automation (in house) is run and issues feed
back to business partner
Quality is key to our business

27. Other Linguistic Issues
•Automotive have terms that have a multiple
meaning in a different context.
•Education & Assistance
– Keep the translator focus on Automotive
– Give the the translators access to source expects…
automated query system
– Term mining
– Language Champions
– Crowd sourcing
Quality is key to our business

28. Common Threads
• Terminology and Style Management key
• Context has and continues to be King
– Lack of context causes rework
• Speed and agility accelerating
• Technology enablement is a must
• Defining what will be measured and how upfront
• Solid investment in client-Supply partnerships
• Continuous investment in partnership
– Education, optimization, rapid continuous improvement

29. Panel Questions
• In the run up to this event, we reached out to a
number of industry clients and vendors on this topic
and asked what questions they would like to pose.
The following slides contain a handful of those
questions, which we will pose to the panel on your
behalf. We will then open it up for any audience
questions.

30. Question 1: Continuous Improvement
• How do you trend and measure improvement? And
what do you do if your language partners are
consistently missing the mark? Do you provide
training? If so, who performs it and how?

31. Question 2: Content Types
• Do you set different measures for different content
types and, if so, can you describe examples?

32. Question 3: Trends
• Are there any trends in speed, content type or tools
that have helped or hindered your approach to
measuring and delivering language quality?

33. Question 4: Challenges
• What has been the biggest challenge you have faced
when adding a new language or a new product?

34. Question 5: Resources
• What skills do you believe resources need to have—
both inside your organizations and when it comes to
your language partners?

35. Question 6: Tools and Technology
• What tools, technology and automation processes
do you employ to manage language quality
(consistency, adherence to terminology, etc.)?

36. Question 7: Models
• What models or methodologies (LISA QA, TAUS DQF,
internal standards, vendor-defined standards or
other) do you use in your company to define and
measure quality?

37. Question 8: Evolution
• Have you seen an evolution in the past 5+ years in
models, tools and challenges?

38. Question 9: The Future
• What do you hope to see that will help you achieve
language quality excellence moving forward?

39. Questions from the Audience