Insights how Intuit, working with Welocalize, architectures a machine translation (MT) program meeting an aggressive launch schedule that now supports the entire enterprise. Presentation given at Localization World 2013 in Silicon Valley http://www.welocalize.com/welocalize-intuit-machine-translation-locworld/

1. Silver Linings Playbook:
Intuit's MT Journey
Fri Oct 11 9am
Render Chiu, Intuit
Group Manager, Global Content & Localization
Tuyen Ho, Welocalize
Senior Director
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serve
informational and educational purposes only.

2. MT Journey Outcome?

3. MT in 3 Months?
Silver Linings Playbook is a 2012 American romantic comedy-drama film written and directed by David O. Russell, adapted from the novel The
Silver Linings Playbook by Matthew Quick. Reference is for informational purposes only.

4. • $4.15 billion rev in 2012
• Flagship products: QuickBooks,
TurboTax and Quicken
• New: Mint.com, Intuit Money Manager
• Markets: North America, Europe,
Singapore, Australia, India

5. Globalization Business Driver:
Opportunity to Serve a Global Ecosystem

6. Business & Technical Landscape
• Focus: QuickBooks Online Software
• Localization Readiness
• Limited i18n of the codebase
• In-house team for French Canadian
only
• Architecture
• WorldServer SaaS
• Mix of various DBs, authoring tools
and CMS
• GCL Platform
• Go-to-Market Goals
• Aggressive goal to SimShip 10 to 20
languages as fast as possible

7. Team & Products - Today
Tax
2 FTE
Payroll
2 FTE
Mobile
1 FTE
QBO
3 FTE
SCM process – QBO/QBO-P Build Process -- 1 FTE
Simplified Tools
FTE
Platform &English––21FTE
Internal Translation (CA French) – 4 FTE
External Translation – 2 FTE
9 Writers, 4 Translators, 2 ENG – 13 Products

8. Why MT?
SPEED
SCALE
COST
✔
✔
✔
QUALITY ?
(sure hope so)

9. Business Case for MT+Post-Editing
Benefits
Considerations
• Efficiencies
• Is our UI and UA content
suitable
• How much do we need to
invest in engine training
• What efficiency is needed to
justify the investment
• What about language pairs &
productivity, e.g. FIGS higher
than CJK?
• What tradeoffs do we need to
be prepared to make in terms
of quality vs cost
– 5-100% productivity increase
• Target cost savings
– 30% lower translation rates
• Faster time to market
– Needed to launch in less than
4 months
• Quality
– No compromising on UI
content

10. Challenges (or Reality Check)
How do you go global ASAP when you start from ground zero?
Requirement
Bilingual translations
In-house MT expertise
MT engine/technology
TMS + MT connector
Structured Content
Status
None, except for FR-CA
None
None
None
One Major Plus We Had
Going for Us: STE

11. Why Simplified Technical English
(STE)?
• It’s the international standard
• Widespread adoption; started in the aerospace
industry, but not limited to that any more
• Actively maintained and enhanced
• Several checker tools that support it
• More precision, less ambiguity
• Easier to understand (esp. by non-native English
speakers
• Easier and cheaper to translate due to clear,
unambiguous glossary and sentence structure
11

12. What Were Our Options Then?
Extreme Options
We Chose
Collaboration
• Lower cost by spreading the risk
• Speed w/ immediate expertise
• Scalability via deep supply chain

13. Comprehensive MT Approach Drives Quality Output
Welocalize has a multi-tiered approach to machine translation
(MT) implementation:
1. Evaluate content for MT readiness
– source content audit
– pre-translation editing
– style and glossary verification
2. Assist in selection and integration of one or multiple MT
engines into the localization technology ecosystem
3. Perform MT post-editing services
– evaluation of MT output quality via workbench
– human assessment and automated scoring
– engine training feedback / engine improvement
4. Support transition from SaaS/hosted “black box” model to
hosted glass box or in-house model

14. Ensuring Quality with MT+PE
Req.
gathering
Solution
Architecture
Engine
Training
Feedback
Loop(s)
PE Metrics
“Go Live”
Intuit – Welocalize – MT Engine Coordination:
1) Client formulates the program requirements
2) MT provider, LSP and client define the solution architecture
3) MT or LSP provider trains the engine
•
•
•
•
•
linguistic training
metadata analysis
workflow architecture
feedback loops with automated scores
human PE measurement and assessment
4) LSP calculates PE metrics
5) MT-PE projects go “live”

15. Engine Strategy: SaaS, Trained
Use Microsoft Translation Hub engine to achieve immediate
cost savings and productivity gains
• Automated engine training process, with minimal human involvement
• No additional investment required
Pros
• Cost-effective
• Rapid deployment
Cons
• Less control over engine training and tuning
• Potentially lower productivity gains due to engine customization limitations

16. Engine Integration into L10N Ecosystem
Source
Source
Source
Files
Files
Translationn
TMS
1
Segmentation
& TM
propagation
3
Translation
Translation
Project (XLIFF
Project (XLIFF
file w/TM
file w/TM
propagated
propagated
for X%
for X%
matches and
matches and
higher
higher
TM
TM
TM
Translation
Translation
complete
complete
(TM + MT)
(TM + MT)
2
5
7
Terminology
Target
Files
8
4
MT engine
MT engine
invoked for
invoked for
non-TM
non-TM
segments
segments
5
5
MT
server
6
2
Translated
files
uploaded;
project
complete
MT with Post-Editing
7
Postediting
7
Linguistic
settings

17. Post-Editing Philosophy
• Language teams familiarized with MT environments
• Talent selection and testing is the key
• Human quality assessment is performed in a structured
non-subjective environment
• Post-editing throughput figures are captured by iOmegaT
and subsequently analyzed
• Translators realize the other benefits of the MT-based
process: terminology consistency, predictability of errors,
higher degree of control over the integrity of translation

18. Initial Results with 1 Engine Training
BLEU
GTM
70
70
60
60
50
50
40
40
30
Bing
30
Hub
Hub
20
20
10
10
-
Bing
-

19. Bootstrap Approach
Fast
Cheap
Let’s Give it a Try
• Adopted SaaS MT
ready-to-go
engines with prepopulated
financial domainspecific data
• Created minimum
training data with
3K glossary
entries and 4.5K
TU for first
training
• Leveraged pre-built
MT connector
• Applied automatic &
human scoring to only
a subset of translated
data
• Experimented with
different free
engines for
branded and
support site to
gather feedback
from customers,
test markets, and
identify quality
gaps

20. MT Journey Recap
10 Engines & Post
Editors Ready for Any
Content
Requirements or
Scope Change
Deployed MT
Connector, Workflows,
Engines + 1 Training
2.5 – 3 months
Created Training
Data
3 Months
Confirmed Target Languages
4.5 months
RFP
Process
2 months
May
2012
July
2012
Sep
2012
Nov
2012
Jan
2013
March
2013

21. Lessons Learned
• Good wine comes from
great grapes
• You can hire a
professional tennis player
to play for you
• You need a great team
and a great partner

22. Looking Forward
• Continue investment on MT
quality
• Evaluate maintenance &
sustainability, e.g. re-training
existing engines for improved
performance
• Expand beyond 10 languages
• It’s not all about text

23. Questions?
Contact:
Tuyen Ho
www.welocalize.com
tuyen.ho@welocalize.com