3. CASE INTRODUCTION
Title： 7 Times Evolution of Test Automation Framework under JS
Product for Efficiency Improvement
Target：
❶ Build a robust test automation framework
❷ Manage test cases efficiently
❸ Reduce test case running time and maintenance cost
❹ Shared framework under scrum development
4. WHO AND WHAT?
Data Analysis Visualization
• reporting, online analytical processing
• analytics
• data mining
• process mining
• event processing
• business performance management
• benchmarking
• text mining
• predictive analytics
5. WHO AND WHAT?
Data Analysis Visualization
• reporting, online analytical processing Programming language:
Javascript/Java
• analytics
• data mining Provide service:
• process mining provides various type of visualization objects that can be used in
The Visualization Service is a software module which
• event processing applications on different platforms (Web, Mobile and Desktop).
different
• business performance management
• A basic
• visual component like chart (Bar, Column Pie, Line, etc ), Geo Map, Visualizationor Grid.
benchmarking Gauge, Table
types:
• text mining• An advanced visual component like (Tree map, Candlestick, Tagcloud)
• predictive analytics • Above visual component with interactions and animations
• Combination of multiple visual components, it could be composed by same type of visual components or different type
of component.
7. What’s more…
2012.12
Developer environment
2012.09 2012.10
Full Automation
Test Link
2011.12 2012.03 2012.06
Jenkins
Data Driven
Selenium
2011.6
We have it!
9. VERSION-1
1.
+
First automation framework setup
- writing test case
2. Overview functional test case
coverage
3. Reduce testing overlap for shared
component
HT
Test
Test ML
page
page
resource
SilkTest maintenance
the relation between TC and record
10. What’s more…
2012.12
Developer environment
2012.09 2012.10
Full Automation
Test Link
2011.12 2012.03 2012.06
Jenkins
Data Driven
Selenium
2011.6
We have it!
11. VERSION-2
Scheduler: ASTEC
TC Management: Excel/XML (test manager)
TC: test page
Test tool: Selenium Test Driver
Other tools
• Auto generate test plan from Excel to xml
DIV DIV
• Test Manager Test
DIV
page
DIV
Selenium
12. VERSION-2
+
1. Reduce test pages
- test driver
2. Reduce resource running cost
3. Test steps turns to be keyword driven in xml case unit
4. Write test case in .xls, auto generate .xml
DIV
DIV DIV
DIV
Test
Test
DIV
DIV
page
page
DIV
DIV
Selenium
test data redundancy
test case sync
13. What’s more…
2012.12
Developer environment
2012.09 2012.10
Full Automation
Test Link
2011.12 2012.03 2012.06
Jenkins
Data Driven
Selenium
2011.6
We have it!
14. VERSION-3
Unified Test Driver
Abstract Test Data
And How?
Wrapper API
Eliminate Test Page
15. VERSION-3
backgroud
color
visible
legend
position
verticalOffset
horizontalOffset
visible
labelFont
labelColor
formatString initChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
layout
padding
tooltip
enable
customization
text
mainValue
subValue
title
main sub
Selectability
mode
deSelectale
defaultSelectedItems
xAxisMember
yAxisMember
bar
barMarkerSize
measureMember
Wrapper API
16. VERSION-3
backgroud
color
visible
legend
position
verticalOffset
horizontalOffset
visible
labelFont
labelColor
formatString initChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
layout
padding
tooltip
enable updateChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
customization
text
mainValue
subValue
title
main sub
Selectability
mode
deSelectale
defaultSelectedItems
xAxisMember
yAxisMember
bar
barMarkerSize
measureMember
Wrapper API
17. VERSION-3
backgroud
color
visible
legend
position
verticalOffset
horizontalOffset testUtil.generateChart(viz/line; TestData.a1a1m1m1)
visible
labelFont
labelColor
formatString initChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
layout
padding
tooltip
enable updateChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
customization
text
mainValue
subValue
title
main sub
Selectability
mode
deSelectale
defaultSelectedItems
xAxisMember
yAxisMember
bar
barMarkerSize
measureMember
Wrapper API
18. VERSION-3
backgroud
color
visible
legend
position
verticalOffset
horizontalOffset
visible
labelFont
labelColor
formatString testUtil.generateChart(viz/line; TestData.a1a1m1m1; ['legend.visible',false])
initChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
layout
padding
tooltip
enable
customization
text
mainValue
subValue
title
main sub
Selectability
mode
deSelectale
defaultSelectedItems
xAxisMember
yAxisMember
bar
barMarkerSize
measureMember
Wrapper API
19. VERSION-3
testUtil.generateChart(viz/line; TestData.a1a1m1m1; ['legend.visible',false])
public
mySQL
private
testUtil.generateChart(viz/line; TestData.a1a1m1m1; legendVisibleTrue)
Abstract Test Data
22. VERSION-3
selenium1
{StartBrowser}
Safari (selenium1)
selenium2 SeleniumServer seleniumServer = new SeleniumServer();
seleniumServer.boot();
selenium = new DefaultSelenium("localhost", 4444, "*safariproxy "+
safariLocation,urlRoot);
ImageComparer CommandExecutor executor = new SeleneseCommandExecutor(selenium);
DesiredCapabilities dc = new DesiredCapabilities();
Robot()
webDriver = new RemoteWebDriver(executor, dc);
Unified Test Driver
23. VERSION-3
selenium1
selenium2 {TakeSnapshot}
Point p = this.getElementOffset(targetElement);
Dimension d = this.getElementSize(targetElement);
BufferedImage bi = new Robot().createScreenCapture(p.x, p.y, d.width, d.height);
ImageComparer
Robot()
Unified Test Driver
25. What’s more…
2012.12
Developer environment
2012.09 2012.10
Headless
Test Link
2011.12 2012.03 2012.06
Full Automation
Data Driven
Selenium
2011.6
We have it!
26. VERSION-4
Table Text
Tooltip Selection Animation
Button Scroll CSS
Checkbox Hover Theme
Link Gesture
Gesture Upload
Dropdown Download
30. What’s more…
2012.12
Developer environment
2012.09 2012.10
Headless
Test Link, Jenkins
2011.12 2012.03 2012.06
Full Automation
Data Driven
Selenium
2011.6
We have it!
34. VERSION-5
.xml TC mngmt
.xls
Test Case Pool
Manual Auto
3. Build info 5. result write back
4. get test plan
2.TP/Platform/build info Test Driver
CI Server Test Executor
5. result log Selenium2 Java/C++
1. Trigger
4. get assets
libs resources
Build Machine Web Server Code repository
Where we are.
35. VERSION-5
.xml TC mngmt
.xls
Test Link Test Case Pool
Manual Auto
3. Build info 5. result write back
4. get test plan
2.TP/Platform/build info Test Driver
CI Server
5. result log
Java Tool
Test Executor
Selenium2 Java/C++
Jenkins
1. Trigger
4. get assets
libs resources
Build Machine Web Server Code repository
Perforce
Where we are.
36. What’s more…
2012.12
Developer environment
2012.09 2012.10
Headless
Test Link, Jenkins
2011.12 2012.03 2012.06
Full Automation
Data Driven
Selenium
2011.6
We have it!
39. VERSION-6
config
Browser?
CasperJS IE/Safari/FF/Chrome
TP Read JS list Open browser
PhantomJS: RunJS (test step)
1. open webkit
2. runJS
….
3. Take snapshot
4. close Take Snapshot
Result validation
not complete not complete
Save result log
Reporting
Test driver inner logic
41. What’s more…
2012.12
Developer environment
2012.09 2012.10
Headless
Test Link, Jenkins
2011.12 2012.03 2012.06
Full Automation
Data Driven
Selenium
2011.6
We have it!
42. VERSION-7
.java TC Management
Test Case Pool
local
CI Server Test Executor
Test Driver
Selenium2 Java/C++
per check-in
trigger
result log
sync up Casper/PhantomJS
Build Machine
get resources P4 server
Test Executor
43. VERSION-7
.java TC Management
Test Case Pool
CI Server
Eclipse Plug-in
local
Test Executor
Test Driver
Selenium2 Java/C++
per check-in
trigger
result log
Jenkins
Build Machine
sync up Casper/PhantomJS
get resources P4 server
Test Executor
Perforce
45. CASE ROI ANALYSIS
Functional Test: BAT (nightly), FA
1) Test machine
2) Assets (util, BM…)
1. Set up environment 3) TP
4) Libs
2. Test case execution 1) Browser compatibility
4.5h
2) Run in headless
1) Email sending
3. Test result reporting 2) Sonar integrated with CI
1) BM update
4. Maintenance 2) TC management
1000+ cases
46. CASE ROI ANALYSIS
Functional Test: BAT (nightly), FA
1) Test machine
2) Assets (util, BM…)
1. Set up environment 3) TP
4) Libs
2. Test case execution 1) Browser compatibility
47min
2) Run in headless
1) Email sending
3. Test result reporting 2) Sonar integrated with CI
1) BM update
4. Maintenance 2) TC management
1000+ cases
54. CONTRIBUTOR
Xiaolu, Ye
Christina, Geng
Helen, Jiang
Chen, Zhou
Peipei, Jiang
Coco, Dai
Grace, Zhao
Minghao, Lu
Kent, Wang
Izzie, Wang
Phil, Xu
NO
THE END
SUPPORTED FROM
Daoyu, Wei
Li, Zhao
Jimmy, Yang
Alex, Feng
Alex, Su
Jiwei, Li
... …
Email: Christina.Geng@sap.com
Sina Weibo: 耿晓儿de测试
