This document provides an overview and summary of JD Edwards archiving and upgrades using ARCTOOLS. It summarizes a case study from DBG, a manufacturing company, that used ARCTOOLS to successfully archive data and complete an upgrade from JDE 8.10 to 9.1 on time over a weekend while virtualizing servers. The purge and archive using ARCTOOLS significantly reduced database sizes and improved system performance, backup times, and disaster recovery capabilities for DBG.
3. 3
WELCOME!
Housekeeping:
- Visit us
- Pass the hat drawing – biz card or fill out a slip
- Show of hands:
- JDE? E-Biz? Demantra? DIY?
- AS400? Oracle? SQL?
4. Agenda
• Introductions to DBG and ARCTOOLS
• Why purge?
• Overview of Archiving in JDE
• Why is it soooooooo difficult?
• Case studies
• Upgrade Case Study - DBG Canada
• ARCTOOLS®
5. Definitions According to Webster’s
• Purge
• transitive verb
• “to make free of something unwanted”
• Archive
• noun
• “a place where records or historical documents are preserved”
• transitive verb
• “to file or collect in or as if in an archive”
According to ARCTOOLS:
Purging is removing records from the production environment.
Archiving is saving those records in an archive environment.
Purging and archiving together is removing records from production and adding them to the archive.
Destruction means permanently removing data from the archive (i.e. “purging from the archive”)
6. DBG
6
DBG is a privately owned supplier to the
auto and defense industries,
manufacturing metal components at 2
plants in Canada and 2 more in Mexico.
DBG has been running JDE since 1998.
7. DBG Canada
• Privately owned manufacturing company
• 35 years in business
• 2 facilities in Canada, 2 in Mexico, running 6x24
• Customers: truck, auto, military, consumer goods manufacturers
• Products:
• metal stampings
• stainless steel and carbon steel assemblies
• design and prototyping
8. ARCTOOLS Company History
• DCSoftware founded 1996, created ARCTOOLS/400
• David Shea, President
• Over 25 years of ERP experience
• Six years with a ‘Big Eight’ consultant
• Created JDE purge and archive methodology in collaboration with JDE and
several customers
• World – 2003. E1 on AS400 – 2004
• Created ARCTOOLS/EM (Oracle, SQL Server) 2005
• Barry Yelverton, Chief Product Architect, Oracle/SQL
• Locations: Boston, Denver, Paris, Limerick
• Over 400 JDE customers on AS400, Oracle, SQL Server
8
9. ARCTOOLS and JD Edwards
• Early 2001 – JDE customer makes introductions to Denver after a successful custom purge effort
• June 2001 – meetings with JDE at Quest Global / Focus
• July thru… developed (with JDE) ‘proof of concept’ purges
• June 2002 – SRO crowd at Quest Global presentation – over 300 customers
• Sept 2002 – development effort in Denver to work with JDE and several large customers
• Oct 2002 – beta site – Cargill
• Jan 2003 – first “JDE Module” released for World
• 2004 – JDE Module for XE on AS/400
• 2005 – Oracle / SQL
• OVI for World and EnterpriseOne
9
10. ARCTOOLS
• Mature, dependable product
• Over 400 Oracle JD Edwards customers world-wide
• A ‘true database solution’
• Not a UBE. Not ODBC. Processes run on the server, in the native
language of the box
• Multiple “accelerators” available for out-of-the-box archiving:
• JDE, Demantra, Oracle E-Business, “DIY”
• Easily tweaked for customized criteria, custom tables
• Use “DIY” capabilities for anything else on your server
10
11. Why the Purge & Archive Project?
• Swamped by data
• Ongoing savings and performance enhancement
• Upgrade on the horizon – good time to clean house
• Legal requirements
• Disaster Recovery
12. Some Benefits of Archiving
• DB Maintenance Savings
• Data Storage Savings
• Batch processing improvements
• Index build times
• Quicker Upgrades
• Faster backups
• DISASTER RECOVERY
BENEFITS MULTIPLIED THROUGHOUT OTHER
COPIES OF PRODUCTION
13. Archiving in JDE
• Pre-ARCTOOLS:
• JDE built-in purges are few and far between
• Cause significant issues – integrity reports, orphaned records
• Customer development efforts proved difficult, risky, painful
A fairly common issue with most packaged applications.
13
14. Why is Purging so Difficult? Data Integrity…
• Textbook:
• You would think it’s just a matter of headers and details, but you would be wrong…
• JDE specific issues:
• Cross functional area data relationships (PO, inventory, AP, GL…)
• “Batches” in AP, AR, GL
• Programs expecting data in other areas (ex: AP inquiry looking for F0911)
• JDE integrity reports (ex: AP by Offset)
• Complex criteria
• Complex scenarios
• “Unfortunate realities” in JDE database
15. Top-Down
Header
Child 1 Child 2
Child 2A
Child 3
Simple scenario – date, status, etc, criteria from the header.
This is what you would think it looks like…. But in reality it’s rarely this easy.
16. Bottom-Up
Header
Child 1 Child 2
Child 2A
Child 3
Other Table(s)
Complex scenario – multiple criteria across multiple files, cross references with other files… This is far more
common, and often far more complex than this.
17. Many to Many
Very complex scenario – multiple criteria across possibly all files, cross references with other files…
many to many relationships…
Voucher 1
Payment 1
Payment 2
Voucher 3
Voucher 4
Voucher 6
Voucher 5
Voucher 7
Payment 4
Payment 5
Payment 3
Voided after
horizon
Voucher 1
F0911
18. And Don’t Forget…
• All those ‘unfortunate realities’ in the JDE database:
• Bad data (converted… self-inflicted… JDE errors)
• Duplicate keys (World more so than E1)
• Orphan records
• Inconsistent field definitions – numeric versus alpha
• Inconsistent key sequences – kco/doc/dct?? doc/dct/kco???
• F00165 keys (pipes… truncated leading zeroes… inconsistent sequence, even within the same parent
document…..)
• Right justified text columns???
• Hooks into F0911…
• Doc type? What doc type? Why would you need a doc type?
• Moving targets (ex: doc type buried in a random text column… in different positions… for the same document type…)
• … etc… etc… etc…
19. Case Study – French Opticians Guild
• Reduced database maintenance time from 22 to 8 hours per weekend, leading to an average annual saving of > $100k / year
• Archived 430 GB of data from production alone
• Reduced the production database by 55%
• Savings multiplied across environments
ARCTOOLS Project Kick Off
Regular Scheduling of
Purge & Archive Jobs
20. Case Study – George P. Johnson
During the upgrade to 9.0 data conversion times improved over prior upgrade:
• 8.10 Data Conversion: 36 Hours
• Purge and archive using ARCTOOLS
• 9.0 Data Conversion: Less than 20 Hours
“The benefits can be seen not just during the upgrade process, but also on a daily basis by end users and the IT department –
with less data in the production environment, end-user queries run more efficiently and database maintenance is simplified.”
Brian Bartkowiak
Vice President, Information Technology
Worldwide at George P. Johnson
21. DBG Canada
• Privately owned manufacturing company
• 35 years in business
• 2 facilities in Canada, 2 in Mexico, running 6x24
• Customers: truck, auto, military, consumer goods manufacturers
• Products:
• metal stampings
• stainless steel and carbon steel assemblies
• design and prototyping
22. JDE at DBG
• On JDE since 1998
• Painful tables:
• F0911 - > 70 million
• F4111
• F3111
• F42199
• … others…
23. DBG – Long Running Processes
• MRP
• Canada: 4 hours
• Mexico: 8 hours
• Backups
• “daily” backup took more than 24 hours
• Concerns about restore time and disaster recovery
• Restore takes as long (or worse??)
• Any backup was at least more than a day old
• Long lead time from lightning strike to up and running
• What about data updates since backup?
24. DBG – 9.1 Upgrade
• 8.10 to 9.1
• Also implementing virtualization at the same time
• VMWare, server farm, etc.
• Virtualized other things, too, not just JDE (domain controllers, etc)
• Timeline:
• March – new hardware
• November – go live
• But in July / August, the realization…
25. DBG Mid-upgrade Reality Check
Data upgrade run time would
shut down plants!!!
(Four plants, 700 workers…)
26. DBG Canada – Upgrade and ARCTOOLS
• Without purging, DBG’s upgrade from 8.10 to 9.1 would have shut down plants
and idled hundreds of workers
• DBG looked at their options and decided to go with archiving. With 15 years of
data, they were ready…
27. DBG - ARCTOOLS
• Archiving decision made in AUGUST
• ARCTOOLS installed August 16th.
• “Dropped in”, vanilla, with only 10 weeks to go-live upgrade
• Decided to just “throw ARCTOOLS at it…”. “followed the script”
• Implemented in parallel with on-going upgrade project
• Accounts payable, accounts receivable, sales orders, work orders, purchase orders,
inventory, general ledger
• Significant data reductions
• Purge / archive a success
• Upgrade successful, on time
• Realized performance improvements, DR improvements
28. DBG – ARCTOOLS Project Team
• Senior analyst – “go figure it out”
• Light on the consulting required
• More on the project management side of parallel upgrade project
• Set a horizon date – ended up at ‘current plus three’
29. DBG – Stats…
• Driver run times – “minutes” (5…. 30…)
• Archive run times – “minutes” (15… 40…)
• Throughputs ranging from 2… 10… 40 million rows an hour
30. DBG – Benefits of the Purge/Archive
• Speed and Run Times – MRP runs each dropped to under an hour (from 4 and 8
hours each previously)
• Backup time dropped from more than a day to under two hours
• DR improvements
• Multiple copies of PD (PY, CRP, other sandboxes) all benefitted
• Increased enthusiasm for new version of JDE
31. DBG – Upgrade Weekend
• Four plants – 700 employees
• Shut down JDE on Friday 6pm
• Took a backup, restored to new system, ran table conversions
• Up and running Monday morning.
“We did it all in a weekend, which would have been impossible without
ARCTOOLS.”
- Julian Wainewright, DBG Canada
32. DBG – Going Forward
• Nothing formalized yet, but…
• Planning to purge every year or two
• Have some data issues that were made visible by the purge
33. Why Archive Before the Upgrade?
“When you’re doing an upgrade project and you unveil something new to people,
before they get to ‘hey this is new, this is cool’, they are going to try what they
know already. If it’s slower than what they had before, then in the first couple
minutes it will kill your project. They haven’t even gotten to the ‘whiz-bang’ stuff.
If they try the new system and it’s not faster it can really kill adoption.”
- Julian Wainewright, IT Manager, DBG Canada
34. Access to the Archive Data
• What you do with the archived data is up to you… you can
• Destroy it
• Back it off to tape
• Move it to another platform
• Leave it on the system but not configure JDE access to it
• Leave it on the system and configure JDE access to “archive only”
• Leave it on the system and configure JDE access to “live plus archive”
35. JDE Access to Live – E1
Access to Archived data
using the
Production ‘role’
Live
Data
PRODUCTION
36. JDE Access to Archive – E1
Access to Archived data
using the
Archive ‘role’
Live
Data
Archive
Data
ARCHIVE
37. JDE Access to Live + Archive – E1
Access to Archived data
using the
Union ‘role’
Live
Data
Archive
Data
41. A Wise Choice!!
• JDE Market leader - over ten years, more than 400 JDE customers on six continents
• Supports E1 (XE to current), World (a6.2 to current) and co-existence
• Runs on IBM i (a.k.a. AS/400), Oracle, SQL
• Quick implementation
• No expensive additional hardware required, no expensive middleware, no requirement to extract all
related master data
Advantages of ARCTOOLS®
42. Performance!!
• A true database solution
• NOT a UBE (UBEs could be suh-lowwwwwww….)
• Not ODBC
• Not a fat client
• No JDE services required
• Not a proprietary hardware / middleware / compression software solution
• ARCTOOLS code generator creates purge/archive programs at run-time in the native language of the
platform, and then executes the process on the server, close to the data
• Throughputs typically measured in millions of rows per hour
Advantages of ARCTOOLS®
43. Simplicity!!
• No JDE services required
• No ASU – we won’t add hundreds of objects to your E1 environment
• No ESUs – no hassles with every JDE upgrade
• No illogical archive table names – data is archived to original JDE table name – i.e. F4301 is archived to
F4301, not FYK57901
• Archive data is not co-mingled with production
Advantages of ARCTOOLS®
44. Flexibility!!!
• Not hard-coded. Completely flexible – add custom tables and custom criteria quickly and easily.
• Add F55 tables directly to the related purge process, not as a ‘custom’, ‘one-off’ or ‘orphan’ purge.
• Delivered pre-configured for JDE, but can purge any tables on your server inside or outside of JDE.
Advantages of ARCTOOLS®
45. Easy Access to Archived JDE Data!!!
• Data archived to proper table name (i.e. F4301 F4301), not some random custom table name (i.e.
F4301 FYK97xxx)
• Easy to set up with your existing JDE objects (screens and reports) – not mocked up, look-alike, cloned
UBEs (which would require even more mods, ESUs, etc…)
Advantages of ARCTOOLS®
46. Summarization!!!
• Can use JDE summarization for GL Detail F0911 (R09811 / P09811) for the archived data, or;
• ARCTOOLS provides custom summarization for F0911
• ARCTOOLS summarization routines are easily customized (versus E1 BSFN, UBE or RPG)
• JDE provides no summarization for Cardex F4111… but ARCTOOLS DOES
Advantages of ARCTOOLS®
47. • Gather control totals, run ‘before’ reports (aging, etc)
• Run driver process from JDE Module
• Run sanity checks on drivers
• Run purge/archive process from ARCTOOLS
• Gather control totals, run ‘after’ reports for comparison
How to Run a Purge/Archive
48. Not Just for JDE… Pre-configured Modules for:
• JDE World
• JDE E1
• Oracle E-Biz Financials
• BPCS
• JDA
• Demantra
• Agile
• Maximo
• Others in development…
ARCTOOLS®
50. For More Information
• www.arctools.com
• Case Studies
• White Papers
• Webinars, demo requests, speaking schedules, user groups
• www.jdealliance.com
• ARCTOOLS and other “best of breed” bolt-ons for JDE
• Joe DiPietro
• (+1) 508.435.8243 x201
• joe@arctools.com
50
51. Go to ARCTOOLS.com for more information
@jdearctools
facebook.com/ARCTOOLS
ARCTOOLS Channel
ARCTOOLS Group
Joe DiPietro
• 508.435.8243 x201
• joe@arctools.com
52. Please complete the session evaluation
We appreciate your feedback and insight
53. Join more than 500 other
JD Edwards users at
April 12-16, 2015
Mandalay Bay- Las Vegas, NV
• Keynote from Lyle Ekdahl,
Vice President of JD Edwards
• JD Edwards’ Roadmap and Strategy
(Including Big Announcements)
• Education around JD Edwards with Edge products, macro-level
content and high-level strategy
• More than 230 Solution Providers across several Oracle Products
register now!
QuestDirect.org/COLLABORATE