SlideShare une entreprise Scribd logo
@hughleo01 Liberty Information Technology
Liberty IT, Belfast and Dublin
Test trend analysis:Towards robust,
reliable and timely tests
Hugh McCamphill
@hughleo01 Liberty Information Technology
2Liberty IT @hughleo01
Indianapolis Dover Belfast GS LIU DCs Europe RDC Russia DRDC
Redmond DC
Global DRDC
& Utilities
Kansas City DC
Latin America RDC
GS LIU DRDC
Portsmouth DC
GS LIU DC
Turkey DCs India DCs Asia RDC China DCs
Wausau
Fairfield
Boston
Seattle Blanchardstown
11 major IT offices
7 country data centers (DCs)
DRDC = Disaster Recovery DC
3 US domestic data centers (DCs)
4 international regional data centers
LIU = Liberty International Underwriters
More than 5,000+ IT
employees around the globe
Australia
About Us
3Liberty IT
Hugh McCamphill
Principal Software Engineer in Test
Writing Automation since 2004
@hughleo01
@hughleo01 Liberty Information Technology
4Liberty IT @hughleo01
Which tests are slow
Which steps are slow
By Martin Lewison from Forest Hills, NY, U.S.A. - Cedar Point and Oberlin's Commencement Uploaded by
Astros4477, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=27339553
@hughleo01 Liberty Information Technology
5Liberty IT @hughleo01
Tumbling Dice : Decorated Anti-tank Blocks near Harkess Rocks, Bamburgh, Northumberland
Image Copyright Richard West. This work is licensed under the Creative Commons Attribution-Share Alike
2.0 Generic Licence
Which tests are intermittently passing
What are common failures across tests
@hughleo01 Liberty Information Technology
6Liberty IT @hughleo01
“Create commons cute robot mural, Toronto"
by https://www.flickr.com/photos/margonaut/ is licensed under CC BY 2.0
Where are the issues?
@hughleo01 Liberty Information Technology
7Liberty IT @hughleo01
@hughleo01 Liberty Information Technology
8Liberty IT @hughleo01
@hughleo01 Liberty Information Technology
9Liberty IT @hughleo01
~ Henry David Thoreau
@hughleo01 Liberty Information Technology
10Liberty IT @hughleo01
@hughleo01 Liberty Information Technology
11Liberty IT @hughleo01
Example dashboard
@hughleo01 Liberty Information Technology
12Liberty IT @hughleo01
RunTests Results
Post
Results
A Simple Process Flow
@hughleo01 Liberty Information Technology
13Liberty IT @hughleo01
A time series database is a
software system that is optimized
for handling time series data,
arrays of numbers indexed
by time
Elasticsearch is a tool for storing, searching, and
analyzing structured and unstructured data
@hughleo01 Liberty Information Technology
14Liberty IT @hughleo01
Setting up a mapping
{"mappings":{"result":{
"properties":{
"project":{"type":"string"},
"environment":{"type":"string"},
"duration":{"type":"double"},
"name":{"type":"string","index":"not_analyzed"},
"classname":{"type":"string","index":"not_analyzed"},
"message":{"type":"string","index":"not_analyzed"},
"status":{"type":"string"},
"created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
@hughleo01 Liberty Information Technology
15Liberty IT @hughleo01
Setting up a mapping
{"mappings":{"result":{
"properties":{
"project":{"type":"string"},
"environment":{"type":"string"},
"duration":{"type":"double"},
"name":{"type":"string","index":"not_analyzed"},
"classname":{"type":"string","index":"not_analyzed"},
"message":{"type":"string","index":"not_analyzed"},
"status":{"type":"string"},
"created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
@hughleo01 Liberty Information Technology
16Liberty IT @hughleo01
Setting up a mapping
{"mappings":{"result":{
"properties":{
"project":{"type":"string"},
"environment":{"type":"string"},
"duration":{"type":"double"},
"name":{"type":"string","index":"not_analyzed"},
"classname":{"type":"string","index":"not_analyzed"},
"message":{"type":"string","index":"not_analyzed"},
"status":{"type":"string"},
"created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
@hughleo01 Liberty Information Technology
17Liberty IT @hughleo01
Setting up a mapping
{"mappings":{"result":{
"properties":{
"project":{"type":"string"},
"environment":{"type":"string"},
"duration":{"type":"double"},
"name":{"type":"string","index":"not_analyzed"},
"classname":{"type":"string","index":"not_analyzed"},
"message":{"type":"string","index":"not_analyzed"},
"status":{"type":"string"},
"created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
curl -XPUT “http://localhost:9200/test-result” –
d @mapping_schema_framework.json
@hughleo01 Liberty Information Technology
18Liberty IT @hughleo01
{"index":{"_index":"test-result","_type":"functional" }}
{"project":“TestProject",
"environment":"QA",
"classname":“TestProject.Foo.Bar",
"name":"ShouldBeAbleToAddItem",
"duration":"47.12",
"created":"2016-09-14 21:35:53",
"status":"Passed"}
curl -XPOST “http://localhost:9200/test-
result/_bulk?pretty” --data-binary @testResults.json
Posting of results
@hughleo01 Liberty Information Technology
19Liberty IT @hughleo01
A Simple Process Flow
RunTests Results
Post
Results Visualize
@hughleo01 Liberty Information Technology
20Liberty IT @hughleo01
Kibana is an open source data visualization plugin for ElasticSearch
@hughleo01 Liberty Information Technology
21Liberty IT @hughleo01
@hughleo01 Liberty Information Technology
22Liberty IT @hughleo01
New visualization results in a single bucket
@hughleo01 Liberty Information Technology
23Liberty IT @hughleo01
Splitting error messages into buckets of top occurring error messages
@hughleo01 Liberty Information Technology
24Liberty IT @hughleo01
Showing top ten most occurring error messages
@hughleo01 Liberty Information Technology
25Liberty IT @hughleo01
Splitting error messages into buckets of tests affected by that error
@hughleo01 Liberty Information Technology
26Liberty IT @hughleo01
Error Messages
Individual Tests
Splitting error messages into buckets of tests affected by that error
@hughleo01 Liberty Information Technology
27Liberty IT @hughleo01
Top Failure Reasons By Test
@hughleo01 Liberty Information Technology
28Liberty IT @hughleo01
Top Slowest Tests
@hughleo01 Liberty Information Technology
29Liberty IT @hughleo01
Top Failing Tests
@hughleo01 Liberty Information Technology
30Liberty IT @hughleo01
Step Time Versus Number Of Times Executed
@hughleo01 Liberty Information Technology
31Liberty IT @hughleo01
Step Time Versus Number Of Times Executed
@hughleo01 Liberty Information Technology
32Liberty IT @hughleo01
Step Time Versus Number Of Times Executed
@hughleo01 Liberty Information Technology
33Liberty IT @hughleo01
Example Dashboard With Time Filter
@hughleo01 Liberty Information Technology
34Liberty IT @hughleo01
References
http://martinfowler.com/articles/nonDeterminism.htm
https://watirmelon.blog/2015/11/11/your-tests-arent-flaky/
@hughleo01 Liberty Information Technology
35Liberty IT @hughleo01
Thank you
Hugh McCamphill
@hughleo01
Run Tests Results
Post
Results Visualize
In Summary

Contenu connexe

Tendances

Knowledge Graph Introduction
Knowledge Graph IntroductionKnowledge Graph Introduction
Knowledge Graph Introduction
Sören Auer
 
The Bounties of Semantic Data Integration for the Enterprise
The Bounties of Semantic Data Integration for the Enterprise The Bounties of Semantic Data Integration for the Enterprise
The Bounties of Semantic Data Integration for the Enterprise
Ontotext
 
Applying large scale text analytics with graph databases
Applying large scale text analytics with graph databasesApplying large scale text analytics with graph databases
Applying large scale text analytics with graph databases
Marissa Kobylenski
 
Enterprise knowledge graphs
Enterprise knowledge graphsEnterprise knowledge graphs
Enterprise knowledge graphs
Sören Auer
 
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
Ontotext
 
Tools and techniques for data science
Tools and techniques for data scienceTools and techniques for data science
Tools and techniques for data science
Ajay Ohri
 
Pandas, Data Wrangling & Data Science
Pandas, Data Wrangling & Data SciencePandas, Data Wrangling & Data Science
Pandas, Data Wrangling & Data Science
Krishna Sankar
 
Integration of data ninja services with oracle spatial and graph
Integration of data ninja services with oracle spatial and graphIntegration of data ninja services with oracle spatial and graph
Integration of data ninja services with oracle spatial and graph
Data Ninja API
 
Do it on your own - From 3 to 5 Star Linked Open Data with RMLio
Do it on your own - From 3 to 5 Star Linked Open Data with RMLioDo it on your own - From 3 to 5 Star Linked Open Data with RMLio
Do it on your own - From 3 to 5 Star Linked Open Data with RMLio
Open Knowledge Belgium
 
Towards Knowledge Graph based Representation, Augmentation and Exploration of...
Towards Knowledge Graph based Representation, Augmentation and Exploration of...Towards Knowledge Graph based Representation, Augmentation and Exploration of...
Towards Knowledge Graph based Representation, Augmentation and Exploration of...
Sören Auer
 
Linking Open, Big Data Using Semantic Web Technologies - An Introduction
Linking Open, Big Data Using Semantic Web Technologies - An IntroductionLinking Open, Big Data Using Semantic Web Technologies - An Introduction
Linking Open, Big Data Using Semantic Web Technologies - An Introduction
Ronald Ashri
 
[Conference] Cognitive Graph Analytics on Company Data and News
[Conference] Cognitive Graph Analytics on Company Data and News[Conference] Cognitive Graph Analytics on Company Data and News
[Conference] Cognitive Graph Analytics on Company Data and News
Ontotext
 
Knowledge graphs on the Web
Knowledge graphs on the WebKnowledge graphs on the Web
Knowledge graphs on the Web
Armin Haller
 
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Connected Data World
 
A New Year in Data Science: ML Unpaused
A New Year in Data Science: ML UnpausedA New Year in Data Science: ML Unpaused
A New Year in Data Science: ML Unpaused
Paco Nathan
 
iRODS UGM 2018 Fair data management and DISQOVERability
iRODS UGM 2018 Fair data management and DISQOVERabilityiRODS UGM 2018 Fair data management and DISQOVERability
iRODS UGM 2018 Fair data management and DISQOVERability
Maarten Coonen
 
Towards an Open Research Knowledge Graph
Towards an Open Research Knowledge GraphTowards an Open Research Knowledge Graph
Towards an Open Research Knowledge Graph
Sören Auer
 
Python for Data Science
Python for Data SciencePython for Data Science
Python for Data Science
Gabriel Moreira
 
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the CloudFirst Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
Ontotext
 
Property graph vs. RDF Triplestore comparison in 2020
Property graph vs. RDF Triplestore comparison in 2020Property graph vs. RDF Triplestore comparison in 2020
Property graph vs. RDF Triplestore comparison in 2020
Ontotext
 

Tendances (20)

Knowledge Graph Introduction
Knowledge Graph IntroductionKnowledge Graph Introduction
Knowledge Graph Introduction
 
The Bounties of Semantic Data Integration for the Enterprise
The Bounties of Semantic Data Integration for the Enterprise The Bounties of Semantic Data Integration for the Enterprise
The Bounties of Semantic Data Integration for the Enterprise
 
Applying large scale text analytics with graph databases
Applying large scale text analytics with graph databasesApplying large scale text analytics with graph databases
Applying large scale text analytics with graph databases
 
Enterprise knowledge graphs
Enterprise knowledge graphsEnterprise knowledge graphs
Enterprise knowledge graphs
 
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
Transforming Your Data with GraphDB: GraphDB Fundamentals, Jan 2018
 
Tools and techniques for data science
Tools and techniques for data scienceTools and techniques for data science
Tools and techniques for data science
 
Pandas, Data Wrangling & Data Science
Pandas, Data Wrangling & Data SciencePandas, Data Wrangling & Data Science
Pandas, Data Wrangling & Data Science
 
Integration of data ninja services with oracle spatial and graph
Integration of data ninja services with oracle spatial and graphIntegration of data ninja services with oracle spatial and graph
Integration of data ninja services with oracle spatial and graph
 
Do it on your own - From 3 to 5 Star Linked Open Data with RMLio
Do it on your own - From 3 to 5 Star Linked Open Data with RMLioDo it on your own - From 3 to 5 Star Linked Open Data with RMLio
Do it on your own - From 3 to 5 Star Linked Open Data with RMLio
 
Towards Knowledge Graph based Representation, Augmentation and Exploration of...
Towards Knowledge Graph based Representation, Augmentation and Exploration of...Towards Knowledge Graph based Representation, Augmentation and Exploration of...
Towards Knowledge Graph based Representation, Augmentation and Exploration of...
 
Linking Open, Big Data Using Semantic Web Technologies - An Introduction
Linking Open, Big Data Using Semantic Web Technologies - An IntroductionLinking Open, Big Data Using Semantic Web Technologies - An Introduction
Linking Open, Big Data Using Semantic Web Technologies - An Introduction
 
[Conference] Cognitive Graph Analytics on Company Data and News
[Conference] Cognitive Graph Analytics on Company Data and News[Conference] Cognitive Graph Analytics on Company Data and News
[Conference] Cognitive Graph Analytics on Company Data and News
 
Knowledge graphs on the Web
Knowledge graphs on the WebKnowledge graphs on the Web
Knowledge graphs on the Web
 
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
 
A New Year in Data Science: ML Unpaused
A New Year in Data Science: ML UnpausedA New Year in Data Science: ML Unpaused
A New Year in Data Science: ML Unpaused
 
iRODS UGM 2018 Fair data management and DISQOVERability
iRODS UGM 2018 Fair data management and DISQOVERabilityiRODS UGM 2018 Fair data management and DISQOVERability
iRODS UGM 2018 Fair data management and DISQOVERability
 
Towards an Open Research Knowledge Graph
Towards an Open Research Knowledge GraphTowards an Open Research Knowledge Graph
Towards an Open Research Knowledge Graph
 
Python for Data Science
Python for Data SciencePython for Data Science
Python for Data Science
 
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the CloudFirst Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
 
Property graph vs. RDF Triplestore comparison in 2020
Property graph vs. RDF Triplestore comparison in 2020Property graph vs. RDF Triplestore comparison in 2020
Property graph vs. RDF Triplestore comparison in 2020
 

En vedette

Colorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latestColorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latest
Onur Baskirt
 
End-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical ScaleEnd-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical Scale
Erdem YILDIRIM
 
Protractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applicationsProtractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applications
Ludmila Nesvitiy
 
Better Bullshit Driven Development [SeleniumCamp 2017]
Better Bullshit Driven Development [SeleniumCamp 2017]Better Bullshit Driven Development [SeleniumCamp 2017]
Better Bullshit Driven Development [SeleniumCamp 2017]
automician
 
Testing Metrics - Making your tests visible
Testing Metrics - Making your tests visibleTesting Metrics - Making your tests visible
Testing Metrics - Making your tests visible
Alper Mermer
 
Grading the Quality of Selenium Tests
Grading the Quality of Selenium TestsGrading the Quality of Selenium Tests
Grading the Quality of Selenium Tests
Marcus Merrell
 
Design patterns in test automation
Design patterns in test automationDesign patterns in test automation
Design patterns in test automation
Mikalai Alimenkou
 
Excuse me, sir, do you have a moment to talk about tests in Kotlin
Excuse me, sir, do you have a moment to talk about tests in KotlinExcuse me, sir, do you have a moment to talk about tests in Kotlin
Excuse me, sir, do you have a moment to talk about tests in Kotlin
leonsabr
 
SoapUI one key to all doors
SoapUI one key to all doorsSoapUI one key to all doors
SoapUI one key to all doors
Yegor Maksymchuk
 
How does Java 8 exert hidden power on Test Automation?
How does Java 8 exert hidden power on Test Automation?How does Java 8 exert hidden power on Test Automation?
How does Java 8 exert hidden power on Test Automation?
Sergey Korol
 
WixAutomation - Test State Pattern - Selenium Camp 2017
WixAutomation - Test State Pattern - Selenium Camp 2017WixAutomation - Test State Pattern - Selenium Camp 2017
WixAutomation - Test State Pattern - Selenium Camp 2017
Roi Ashkenazi
 
Test Automation Architecture in Microservices
Test Automation Architecture in MicroservicesTest Automation Architecture in Microservices
Test Automation Architecture in Microservices
Alper Mermer
 
An easy way to automate complex UI
An easy way to automate complex UIAn easy way to automate complex UI
An easy way to automate complex UI
Ivan Pashko
 
Fabulous Tests on Spock and Groovy
Fabulous Tests on Spock and GroovyFabulous Tests on Spock and Groovy
Fabulous Tests on Spock and Groovy
Yaroslav Pernerovsky
 
Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014
Oren Rubin
 
Unit Testing TypeScript
Unit Testing TypeScriptUnit Testing TypeScript
Unit Testing TypeScript
Daniel Jimenez Garcia
 
Combining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
Combining Heritrix and PhantomJS for Better Crawling of Pages with JavascriptCombining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
Combining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
Michael Nelson
 
Testing NodeJS, REST APIs and MongoDB with UFT
Testing NodeJS, REST APIs and MongoDB with UFTTesting NodeJS, REST APIs and MongoDB with UFT
Testing NodeJS, REST APIs and MongoDB with UFT
Ori Bendet
 
Angular js automation using protractor
Angular js automation using protractorAngular js automation using protractor
Angular js automation using protractor
Ravi Gupta
 
API Testing with Frisby and Mocha
API Testing with Frisby and MochaAPI Testing with Frisby and Mocha
API Testing with Frisby and Mocha
Lyudmila Anisimova
 

En vedette (20)

Colorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latestColorful world-of-visual-automation-testing-latest
Colorful world-of-visual-automation-testing-latest
 
End-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical ScaleEnd-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical Scale
 
Protractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applicationsProtractor framework – how to make stable e2e tests for Angular applications
Protractor framework – how to make stable e2e tests for Angular applications
 
Better Bullshit Driven Development [SeleniumCamp 2017]
Better Bullshit Driven Development [SeleniumCamp 2017]Better Bullshit Driven Development [SeleniumCamp 2017]
Better Bullshit Driven Development [SeleniumCamp 2017]
 
Testing Metrics - Making your tests visible
Testing Metrics - Making your tests visibleTesting Metrics - Making your tests visible
Testing Metrics - Making your tests visible
 
Grading the Quality of Selenium Tests
Grading the Quality of Selenium TestsGrading the Quality of Selenium Tests
Grading the Quality of Selenium Tests
 
Design patterns in test automation
Design patterns in test automationDesign patterns in test automation
Design patterns in test automation
 
Excuse me, sir, do you have a moment to talk about tests in Kotlin
Excuse me, sir, do you have a moment to talk about tests in KotlinExcuse me, sir, do you have a moment to talk about tests in Kotlin
Excuse me, sir, do you have a moment to talk about tests in Kotlin
 
SoapUI one key to all doors
SoapUI one key to all doorsSoapUI one key to all doors
SoapUI one key to all doors
 
How does Java 8 exert hidden power on Test Automation?
How does Java 8 exert hidden power on Test Automation?How does Java 8 exert hidden power on Test Automation?
How does Java 8 exert hidden power on Test Automation?
 
WixAutomation - Test State Pattern - Selenium Camp 2017
WixAutomation - Test State Pattern - Selenium Camp 2017WixAutomation - Test State Pattern - Selenium Camp 2017
WixAutomation - Test State Pattern - Selenium Camp 2017
 
Test Automation Architecture in Microservices
Test Automation Architecture in MicroservicesTest Automation Architecture in Microservices
Test Automation Architecture in Microservices
 
An easy way to automate complex UI
An easy way to automate complex UIAn easy way to automate complex UI
An easy way to automate complex UI
 
Fabulous Tests on Spock and Groovy
Fabulous Tests on Spock and GroovyFabulous Tests on Spock and Groovy
Fabulous Tests on Spock and Groovy
 
Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014Page Objects Done Right - selenium conference 2014
Page Objects Done Right - selenium conference 2014
 
Unit Testing TypeScript
Unit Testing TypeScriptUnit Testing TypeScript
Unit Testing TypeScript
 
Combining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
Combining Heritrix and PhantomJS for Better Crawling of Pages with JavascriptCombining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
Combining Heritrix and PhantomJS for Better Crawling of Pages with Javascript
 
Testing NodeJS, REST APIs and MongoDB with UFT
Testing NodeJS, REST APIs and MongoDB with UFTTesting NodeJS, REST APIs and MongoDB with UFT
Testing NodeJS, REST APIs and MongoDB with UFT
 
Angular js automation using protractor
Angular js automation using protractorAngular js automation using protractor
Angular js automation using protractor
 
API Testing with Frisby and Mocha
API Testing with Frisby and MochaAPI Testing with Frisby and Mocha
API Testing with Frisby and Mocha
 

Similaire à Test trend analysis: Towards robust reliable and timely tests

Searching for Meaning
Searching for MeaningSearching for Meaning
Searching for Meaning
Trey Grainger
 
The Relevance of the Apache Solr Semantic Knowledge Graph
The Relevance of the Apache Solr Semantic Knowledge GraphThe Relevance of the Apache Solr Semantic Knowledge Graph
The Relevance of the Apache Solr Semantic Knowledge Graph
Trey Grainger
 
How to Build a Semantic Search System
How to Build a Semantic Search SystemHow to Build a Semantic Search System
How to Build a Semantic Search System
Trey Grainger
 
The Semantic Knowledge Graph
The Semantic Knowledge GraphThe Semantic Knowledge Graph
The Semantic Knowledge Graph
Trey Grainger
 
Docker Summit MongoDB - Data Democratization
Docker Summit MongoDB - Data Democratization Docker Summit MongoDB - Data Democratization
Docker Summit MongoDB - Data Democratization
Chris Grabosky
 
Building a Real-Time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-Time Data Pipeline: Apache Kafka at LinkedInBuilding a Real-Time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-Time Data Pipeline: Apache Kafka at LinkedIn
Amy W. Tang
 
Koshy june27 140pm_room210_c_v4
Koshy june27 140pm_room210_c_v4Koshy june27 140pm_room210_c_v4
Koshy june27 140pm_room210_c_v4
DataWorks Summit
 
Building a Real-time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-time Data Pipeline: Apache Kafka at LinkedInBuilding a Real-time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-time Data Pipeline: Apache Kafka at LinkedInDataWorks Summit
 
From keyword-based search to language-agnostic semantic search
From keyword-based search to language-agnostic semantic searchFrom keyword-based search to language-agnostic semantic search
From keyword-based search to language-agnostic semantic search
CareerBuilder.com
 
President Election of Korea in 2017
President Election of Korea in 2017President Election of Korea in 2017
President Election of Korea in 2017
Jongwook Woo
 
Thinkful DC - Intro to Data Science
Thinkful DC - Intro to Data Science Thinkful DC - Intro to Data Science
Thinkful DC - Intro to Data Science
TJ Stalcup
 
Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...
EDB
 
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Neo4j
 
Getting to Know Your Data with R
Getting to Know Your Data with RGetting to Know Your Data with R
Getting to Know Your Data with R
Stephen Withington
 
Agile Data Science 2.0
Agile Data Science 2.0Agile Data Science 2.0
Agile Data Science 2.0
Russell Jurney
 
Bigdatacooltools
BigdatacooltoolsBigdatacooltools
Bigdatacooltools
suresh sood
 
Paper presentation
Paper presentationPaper presentation
Paper presentation
K.K. Tripathi
 
Agile Data Science 2.0
Agile Data Science 2.0Agile Data Science 2.0
Agile Data Science 2.0
Russell Jurney
 
MongoDB Schema Design: Practical Applications and Implications
MongoDB Schema Design: Practical Applications and ImplicationsMongoDB Schema Design: Practical Applications and Implications
MongoDB Schema Design: Practical Applications and Implications
MongoDB
 
Introduction to PySpark
Introduction to PySparkIntroduction to PySpark
Introduction to PySpark
Russell Jurney
 

Similaire à Test trend analysis: Towards robust reliable and timely tests (20)

Searching for Meaning
Searching for MeaningSearching for Meaning
Searching for Meaning
 
The Relevance of the Apache Solr Semantic Knowledge Graph
The Relevance of the Apache Solr Semantic Knowledge GraphThe Relevance of the Apache Solr Semantic Knowledge Graph
The Relevance of the Apache Solr Semantic Knowledge Graph
 
How to Build a Semantic Search System
How to Build a Semantic Search SystemHow to Build a Semantic Search System
How to Build a Semantic Search System
 
The Semantic Knowledge Graph
The Semantic Knowledge GraphThe Semantic Knowledge Graph
The Semantic Knowledge Graph
 
Docker Summit MongoDB - Data Democratization
Docker Summit MongoDB - Data Democratization Docker Summit MongoDB - Data Democratization
Docker Summit MongoDB - Data Democratization
 
Building a Real-Time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-Time Data Pipeline: Apache Kafka at LinkedInBuilding a Real-Time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-Time Data Pipeline: Apache Kafka at LinkedIn
 
Koshy june27 140pm_room210_c_v4
Koshy june27 140pm_room210_c_v4Koshy june27 140pm_room210_c_v4
Koshy june27 140pm_room210_c_v4
 
Building a Real-time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-time Data Pipeline: Apache Kafka at LinkedInBuilding a Real-time Data Pipeline: Apache Kafka at LinkedIn
Building a Real-time Data Pipeline: Apache Kafka at LinkedIn
 
From keyword-based search to language-agnostic semantic search
From keyword-based search to language-agnostic semantic searchFrom keyword-based search to language-agnostic semantic search
From keyword-based search to language-agnostic semantic search
 
President Election of Korea in 2017
President Election of Korea in 2017President Election of Korea in 2017
President Election of Korea in 2017
 
Thinkful DC - Intro to Data Science
Thinkful DC - Intro to Data Science Thinkful DC - Intro to Data Science
Thinkful DC - Intro to Data Science
 
Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...Application Development & Database Choices: Postgres Support for non Relation...
Application Development & Database Choices: Postgres Support for non Relation...
 
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
Graphs & Big Data - Philip Rathle and Andreas Kollegger @ Big Data Science Me...
 
Getting to Know Your Data with R
Getting to Know Your Data with RGetting to Know Your Data with R
Getting to Know Your Data with R
 
Agile Data Science 2.0
Agile Data Science 2.0Agile Data Science 2.0
Agile Data Science 2.0
 
Bigdatacooltools
BigdatacooltoolsBigdatacooltools
Bigdatacooltools
 
Paper presentation
Paper presentationPaper presentation
Paper presentation
 
Agile Data Science 2.0
Agile Data Science 2.0Agile Data Science 2.0
Agile Data Science 2.0
 
MongoDB Schema Design: Practical Applications and Implications
MongoDB Schema Design: Practical Applications and ImplicationsMongoDB Schema Design: Practical Applications and Implications
MongoDB Schema Design: Practical Applications and Implications
 
Introduction to PySpark
Introduction to PySparkIntroduction to PySpark
Introduction to PySpark
 

Dernier

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
Juraj Vysvader
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, BetterWebinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
XfilesPro
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
Neo4j
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Globus
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 
Game Development with Unity3D (Game Development lecture 3)
Game Development  with Unity3D (Game Development lecture 3)Game Development  with Unity3D (Game Development lecture 3)
Game Development with Unity3D (Game Development lecture 3)
abdulrafaychaudhry
 
AI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website CreatorAI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website Creator
Google
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 

Dernier (20)

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, BetterWebinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Game Development with Unity3D (Game Development lecture 3)
Game Development  with Unity3D (Game Development lecture 3)Game Development  with Unity3D (Game Development lecture 3)
Game Development with Unity3D (Game Development lecture 3)
 
AI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website CreatorAI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website Creator
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 

Test trend analysis: Towards robust reliable and timely tests

  • 1. @hughleo01 Liberty Information Technology Liberty IT, Belfast and Dublin Test trend analysis:Towards robust, reliable and timely tests Hugh McCamphill
  • 2. @hughleo01 Liberty Information Technology 2Liberty IT @hughleo01 Indianapolis Dover Belfast GS LIU DCs Europe RDC Russia DRDC Redmond DC Global DRDC & Utilities Kansas City DC Latin America RDC GS LIU DRDC Portsmouth DC GS LIU DC Turkey DCs India DCs Asia RDC China DCs Wausau Fairfield Boston Seattle Blanchardstown 11 major IT offices 7 country data centers (DCs) DRDC = Disaster Recovery DC 3 US domestic data centers (DCs) 4 international regional data centers LIU = Liberty International Underwriters More than 5,000+ IT employees around the globe Australia About Us
  • 3. 3Liberty IT Hugh McCamphill Principal Software Engineer in Test Writing Automation since 2004 @hughleo01
  • 4. @hughleo01 Liberty Information Technology 4Liberty IT @hughleo01 Which tests are slow Which steps are slow By Martin Lewison from Forest Hills, NY, U.S.A. - Cedar Point and Oberlin's Commencement Uploaded by Astros4477, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=27339553
  • 5. @hughleo01 Liberty Information Technology 5Liberty IT @hughleo01 Tumbling Dice : Decorated Anti-tank Blocks near Harkess Rocks, Bamburgh, Northumberland Image Copyright Richard West. This work is licensed under the Creative Commons Attribution-Share Alike 2.0 Generic Licence Which tests are intermittently passing What are common failures across tests
  • 6. @hughleo01 Liberty Information Technology 6Liberty IT @hughleo01 “Create commons cute robot mural, Toronto" by https://www.flickr.com/photos/margonaut/ is licensed under CC BY 2.0 Where are the issues?
  • 7. @hughleo01 Liberty Information Technology 7Liberty IT @hughleo01
  • 8. @hughleo01 Liberty Information Technology 8Liberty IT @hughleo01
  • 9. @hughleo01 Liberty Information Technology 9Liberty IT @hughleo01 ~ Henry David Thoreau
  • 10. @hughleo01 Liberty Information Technology 10Liberty IT @hughleo01
  • 11. @hughleo01 Liberty Information Technology 11Liberty IT @hughleo01 Example dashboard
  • 12. @hughleo01 Liberty Information Technology 12Liberty IT @hughleo01 RunTests Results Post Results A Simple Process Flow
  • 13. @hughleo01 Liberty Information Technology 13Liberty IT @hughleo01 A time series database is a software system that is optimized for handling time series data, arrays of numbers indexed by time Elasticsearch is a tool for storing, searching, and analyzing structured and unstructured data
  • 14. @hughleo01 Liberty Information Technology 14Liberty IT @hughleo01 Setting up a mapping {"mappings":{"result":{ "properties":{ "project":{"type":"string"}, "environment":{"type":"string"}, "duration":{"type":"double"}, "name":{"type":"string","index":"not_analyzed"}, "classname":{"type":"string","index":"not_analyzed"}, "message":{"type":"string","index":"not_analyzed"}, "status":{"type":"string"}, "created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
  • 15. @hughleo01 Liberty Information Technology 15Liberty IT @hughleo01 Setting up a mapping {"mappings":{"result":{ "properties":{ "project":{"type":"string"}, "environment":{"type":"string"}, "duration":{"type":"double"}, "name":{"type":"string","index":"not_analyzed"}, "classname":{"type":"string","index":"not_analyzed"}, "message":{"type":"string","index":"not_analyzed"}, "status":{"type":"string"}, "created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
  • 16. @hughleo01 Liberty Information Technology 16Liberty IT @hughleo01 Setting up a mapping {"mappings":{"result":{ "properties":{ "project":{"type":"string"}, "environment":{"type":"string"}, "duration":{"type":"double"}, "name":{"type":"string","index":"not_analyzed"}, "classname":{"type":"string","index":"not_analyzed"}, "message":{"type":"string","index":"not_analyzed"}, "status":{"type":"string"}, "created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}}
  • 17. @hughleo01 Liberty Information Technology 17Liberty IT @hughleo01 Setting up a mapping {"mappings":{"result":{ "properties":{ "project":{"type":"string"}, "environment":{"type":"string"}, "duration":{"type":"double"}, "name":{"type":"string","index":"not_analyzed"}, "classname":{"type":"string","index":"not_analyzed"}, "message":{"type":"string","index":"not_analyzed"}, "status":{"type":"string"}, "created":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}} curl -XPUT “http://localhost:9200/test-result” – d @mapping_schema_framework.json
  • 18. @hughleo01 Liberty Information Technology 18Liberty IT @hughleo01 {"index":{"_index":"test-result","_type":"functional" }} {"project":“TestProject", "environment":"QA", "classname":“TestProject.Foo.Bar", "name":"ShouldBeAbleToAddItem", "duration":"47.12", "created":"2016-09-14 21:35:53", "status":"Passed"} curl -XPOST “http://localhost:9200/test- result/_bulk?pretty” --data-binary @testResults.json Posting of results
  • 19. @hughleo01 Liberty Information Technology 19Liberty IT @hughleo01 A Simple Process Flow RunTests Results Post Results Visualize
  • 20. @hughleo01 Liberty Information Technology 20Liberty IT @hughleo01 Kibana is an open source data visualization plugin for ElasticSearch
  • 21. @hughleo01 Liberty Information Technology 21Liberty IT @hughleo01
  • 22. @hughleo01 Liberty Information Technology 22Liberty IT @hughleo01 New visualization results in a single bucket
  • 23. @hughleo01 Liberty Information Technology 23Liberty IT @hughleo01 Splitting error messages into buckets of top occurring error messages
  • 24. @hughleo01 Liberty Information Technology 24Liberty IT @hughleo01 Showing top ten most occurring error messages
  • 25. @hughleo01 Liberty Information Technology 25Liberty IT @hughleo01 Splitting error messages into buckets of tests affected by that error
  • 26. @hughleo01 Liberty Information Technology 26Liberty IT @hughleo01 Error Messages Individual Tests Splitting error messages into buckets of tests affected by that error
  • 27. @hughleo01 Liberty Information Technology 27Liberty IT @hughleo01 Top Failure Reasons By Test
  • 28. @hughleo01 Liberty Information Technology 28Liberty IT @hughleo01 Top Slowest Tests
  • 29. @hughleo01 Liberty Information Technology 29Liberty IT @hughleo01 Top Failing Tests
  • 30. @hughleo01 Liberty Information Technology 30Liberty IT @hughleo01 Step Time Versus Number Of Times Executed
  • 31. @hughleo01 Liberty Information Technology 31Liberty IT @hughleo01 Step Time Versus Number Of Times Executed
  • 32. @hughleo01 Liberty Information Technology 32Liberty IT @hughleo01 Step Time Versus Number Of Times Executed
  • 33. @hughleo01 Liberty Information Technology 33Liberty IT @hughleo01 Example Dashboard With Time Filter
  • 34. @hughleo01 Liberty Information Technology 34Liberty IT @hughleo01 References http://martinfowler.com/articles/nonDeterminism.htm https://watirmelon.blog/2015/11/11/your-tests-arent-flaky/
  • 35. @hughleo01 Liberty Information Technology 35Liberty IT @hughleo01 Thank you Hugh McCamphill @hughleo01 Run Tests Results Post Results Visualize In Summary