Presentation ran @ASUG new jersey chapter around mitigating risk around Business Objects upgrade while automating tests. 360bind is THE Business Objects test automation product!

1. Track Calculation Engine Changes &
Regressions in
SAP BusinessObjects
Bruno Masek, VP, GB&SMITH
Amy Betten, Director of Sales, GB&SMITH

2. • How do you track calculation engine changes?
 Alt+Tab ?
 Print Reports and put on top of each other
 2 screens
 Asking Business
 Or no checking
STATE OF THE ART?

3. REGRESSION TESTING
WHEN?
• Migration
• Service Pack / Patch
• Univers Update (UNV —> UNX)
• ETL / Data-warehouse changes
• Etc

4. 3.1 Webi 4.2 Webi

5. 3.1 Deski 4.2 Webi

6. 4.1 SP74.1 SP5

7. ROI
• In house checking => Safety
• No more manual checking
• Automated and Scheduled process
• Can be used by Business, QA & IT
ROI

8. USE CASE
While POC, a service pack was implemented 4.1 SP4 to 4.1 SP6. SP6 bugs +
regressions were found due to Sybase DB upgrade.
Upgrade was stopped, until Bind checked all reports for calculation engine
changes/regressions & all issues fixed.
Risk management was included in the loop/process and permanent
automated regression testing became mandatory.
BI4.1 SP6 and Sybase upgrade
CENTRAL BANK

9. USE CASE
While POC, administrator ran the automated regression tool live on a few
financial reports that had been checked by consultants. Admin found on
some reports with data coming from same database : $1Billion margin in
3.1 and $1.2Billion in 4.1. Customer estimated 40+hours were needed to
check certain reports each quarter due to service packs.
BI4.1 Quality check
REAL-ESTATE COMPANY

10. USE CASE
During a migration with very tight budget, City of Chicago implemented
automated calculation engine/regression testing after report clean-up and
conversion. As a result $200,000 were saved and regressions were found
even on reports 100% converted.
Migration 3.1 to 4.2
CITY OF CHICAGO

11. USE CASE
500 reports (Webi-Deski-Crytal) complex reports needed to be checked. 250
man days were needed with the Alt+tab method. Automated process was
implemented taking 15 days and finding 5% Calculation engine
changes/regression.
Quality check of 500 reports
INSURANCE COMPANY

12. USE CASE
One of the largest company in the world. 10% Regressions found.
4.1 to 4.2 Migration
LARGE OIL & GAS COMPANY

13. USE CASE
4.1 had been implemented for a while. Automated calculation engine
changes was implemented. They found out that for years they were
regressions in 3.1 that were fixed in 4.1.
3.1 to 4.2
HUGE TELECOM COMPANY

14. USE CASE
Organization checks regressions randomly, part of QA control. Regressions
found between Dev / Prod due to a table change.
DEV and PROD
HEALTHCARE ORGANIZATION

15. 12,000 Documents
730 Users
70 Universes
DETAILED REGRESSION TESTING USE CASE
SAP BusinessObjects XI3.1 SP7
to
SAP Business Intelligence 4.2 SP2
Federal Agency

16. 1. RUN THE 360BIND EXPORTS & COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM DOCUMENTS

17. 1. Run Exports and Comparisons: 1.5 hours

18. EXPORT 1

19. Run Exports and Comparisons 1.5 hours
Create an Export Task

20. Select Documents
Run the 360Bind Exports and Comparisons
1.5 hours

21. Add Documents
Run the 360Bind Exports and Comparisons
1.5 hours

22. Select Frequency
Run the 360Bind Exports and Comparisons
1.5 hours

23. Select an Export Category
or Create a New one
Run the 360Bind Exports and Comparisons
1.5 hours

24. Populate the Prompts
and Schedule!
Run the 360Bind Exports and Comparisons
1.5 hours

25. EXPORT 2

26. Export Task
XI 3
Run the 360Bind Exports and Comparisons
1.5 hours
Select the first export task
Export it into a Zip file

27. Import Task
BI 4
Run the 360Bind Exports and Comparisons
1.5 hours
Import the task saved
and Run it

28. COMPARISON

29. Schedule Comparison Job
Run the 360Bind Exports and Comparisons
1.5 hours

30. Select Export Category
Run the 360Bind Exports and Comparisons
1.5 hours
Working folder you exported the reports

31. Select Export Results
Run the 360Bind Exports and Comparisons
1.5 hours

32. Select Comparison Options
Run the 360Bind Exports and Comparisons
1.5 hours

33. 1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM
DOCUMENTS

34. 2. Identify all Kinds of Regressions: 1 hour

35. Data - Structure - Images - Style
28/28 documents with regressions
Identify all Kinds of Regressions
1 hour

36. 1. Vertical Alignment & Font Families
Identify all Kinds of Regressions
1 hour
XI 3.1 BI 4.2

37. Data - Structure - Images - Style
24/28 documents with regressions
Identify all Kinds of Regressions
1 hour

38. 2. Image: Label font, Scales and Layout
Identify all Kinds of Regressions
1 hour
XI 3.1 BI 4.2

39. 2. Image: Label font, Scales and Layout
Identify all Kinds of Regressions
1 hour
XI 3.1 BI 4.2

40. Data - Structure - Images - Style
17/28 documents with regressions
Identify all Kinds of Regressions
1 hour

41. 3. Data: Missing Decimals (48.32  48.00)
Business Expenses Details (in %)
Percentage Total
< 100%!!!
Identify all Kinds of Regressions
1 hour
XI 3.1 BI 4.2

42. 3. Data: Error DIV/0
Business Expenses Details (in %)
Identify all Kinds of Regressions
1 hour
XI 3.1 BI 4.2

43. 1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL PLATFORM
DOCUMENTS

44. 3. Analyze root cause of Data Regressions: 1 hour
• Document: “State A14 - Expenses Details”
• Variable Analyzed: “C Fch Ttc”

45. • Variable is using “Round()” function included in “FormatNumber()”
function: FormatNumber(Round(…))
Analyze root cause of Data Regressions
1 hour

46. Analyze root cause of Data Regressions
1 hour
Working fine in XIR3.1 SP7
but no longer in BI 4.2 SP2 due to
Calculation Engine Modification

47. 1. RUN THE 360BIND EXPORTS AND COMPARISONS
2. IDENTIFY ALL KINDS OF REGRESSIONS
3. ANALYZE ROOT CAUSE OF REGRESSIONS
4. RUN IMPACT ANALYSIS ON ALL
PLATFORM DOCUMENTS

48. 4. Run impact analysis on all the Documents: 0.5 hour
Identify all documents using “FormatNumber(Round(…))” formulas in
variables or cells
 596 Documents probably impacted by the Regressions
IMPACT ANALYSIS

49. Run impact analysis on all the Documents
0.5 hour
• Identify which one are really used:
 262 Documents to Fix for the Migration

50. 28 documents troubleshot between versions:
• Service proposal from a BI consulting company
= 3 days to test the 28 documents with manual comparison
• 360Bind and 360Eyes:
1. Run the 360Bind exports and comparisons : 1.5 hours
2. Identify all kinds of regressions : 1 hour
3. Analyze root cause of regressions : 1 hour
4. Run impact analysis on all platform documents : 0.5 hour
= 4 hours to identify all the 28 document regressions
and the impact on the 12,000 platform documents
FIND CALCULATION ENGINE CHANGES

51. EASIER, FASTER, CHEAPER & SAFER MIGRATION

52. Backup, promote & version BO
content
Schedule & burst your BO reports
Explore & analyze BO metadata
and licenses
Automate report qualification
Non-regression testing
XIR2
XI3
BI4
Secure your BO deployment
Streamline and Extend
SAP BusinessObjects
STREAMLINE SAP BUSINESSOBJECTS

53. ABOUT GB&SMITH
• 1,000,000 administrated use
• Won EY entrepreneur of the
year and other famous prices
• Continuously evolving
company and tools
• More than 500 customers
• In more than 30 countries
• Small to big deployment
(100-92000 users)
• 70 employees
• Very reactive support
team (98% renewal rate)
• 50% in R&D

54. THEY TRUST US

56. Take a look at our solutions on:
www.youtube.com/360suite
Let’s work together
contact@gbandsmith.com
www.gbandsmith.com
QUESTIONS?