On July 17, 2019 Inflectra and Neotys hosted a joint webinar called: Transform Software Testing & Quality with the Neotys-Inflectra Platform. The webinar demonstrated the powerful capabilities of this combined QA and performance testing platform vis-a-vis legacy tools.
The is the presentation used in this webinar.
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Transform Software Testing and Quality with the Neotys-Inflectra Platform
1. ®
Transform Software Testing & Quality
with the Neotys-Inflectra Platform
July 17th, 2019 – Henrik Rexed & Adam Sandman
2. ®®
Introducing Henrik
Henrik Rexed (khnightloader)*
12 years of Performance engineering
Technical Evangelist @Neotys
Lego Fan
Focus on technologies for Performance
engineers
3. ®®
Introducing Adam
Adam Sandman was a programmer
from the age of 10 and has been
working in the IT industry for the past
20 years.
Currently Adam is a Director of
Technology at Inflectra Corporation,
where he is interested in technology,
business and innovation.
Adam lives in Washington, DC, USA
5. ®
Agenda
Introduction
Overview of the products:
SpiraTeam + NeoLoad + Rapise
Top Features of our Platform
Live Demo
Testing applications for functionality and performance
Q & A
9. ®
Top 5 Benefits of SpiraTeam
1. Fast, Modern User Interface
2. Get up and Running in 10 Minutes
3. Integrated Requirements, Test & Defect
4. Agile Planning Right Out of The Box
5. Open Platform, Add Your Tools of Choice
13. ®®
Open, Extensible Platform
Developer IDEs
CI / Build Servers
Legacy Test Management
Modelling Tools
Unit Test Frameworks
Bug-Tracking Tools
Performance Testing Tools
Functional Testing Tools
14. ®®
Performance Testing
Platform
Web-SaaS
On-premise
Cloud
NeoLoad API
We Focus on Performance Testing, and Do it Better
We are at the forefront of
innovation
Only with Neotys
• Automatic User Path Update
• Script design automation for correlation,
parametrization and randomization
• Mobile fully integrated
SD Times Top 10 in Security & Performance 2017
Rely on global support
Testing as
a service
Consulting
MigrationTraining
16. ®
Top 6 Benefits of NeoLoad
1. Reduce the TCO
2. Helps to implement continuous testing
3. Technology support
4. Large Ecosystem
5. Collaboration between team
6. Flexible deployment model
17. ®®
NeoLoad : Reduce Total Cost of Ownership (TCO)
Digital, Web, Mobile, SAP Oracle EBS…
5-10x Faster Test Design
Scriptless test design and validation – reduce errors
Reduce time to market – spend more time on
performance engineering vs. script writing
20x Faster Script Maintenance – Agile/DevOps
Reduce spend on tools
Much Higher ROI
Flexible pricing model - hours/days/months/year…/
permanent
18. ®®
NeoLoad vs script-based tools
15 60 15 90 30 60 60 45
15 30 15 60 15 20 60 30
Test Tuning
Test Tuning
For new test designs average time
savings is 60% over script-based tools
like LoadRunner, Jmeter…
15 1 1 5 5 1 60 30
New Test Case with LoadRunner, Jmeter (other script-based tools)
For test maintenance, average time savings is 90% over script-
based tools like LoadRunner, Jmeter
Same (New) Test Case
Test Maintenance
19. ®®
Continuous Performance Testing
API Testing Integration Testing Application Testing
Test a Component Test a System Test Real World
Continuous Testing Embedded in CI/CD Pipelines
20. ®®
API Testing – Test Early, Test Often
Supports API Technologies
• SOAP / REST Support
• Assisted design with Swagger/Open API import & highly
efficient extractors for fast test design and fewer errors in
scripting
• Test as Code or GUI based productivity
Continuous APIs Performance Testing
• CI / CD automated API testing, Dynamic load generation
infrastructure, Automatic pass/fail based on SLAs
• Real-time test results for fast dev feedback loop if bottleneck
is identified
• API Performance trends
Testing API Based Applications
• Test standalone APIs or APIs within an end-to-end test. Same
platform, same skills
• Dev, QA, Ops share a common view of APIs performance
• Confidence in API and App Speed and Reliability
21. ®®
NeoLoad : Technology, Flexibily, DevOps
New Technology Support – HTTP/2, GWT, MQTT, IoT, WebSocket,
APIs, SAP, Oracle EBS, Oracle Forms, Flex, Micro Services,
REST-based API, SOAP Web Services, JSON, Asynchronous
JavaSripts and XML (AJAX), Video adaptive bitrate streaming and
more…
Load Generation – On-Premises / Cloud / Hybrid
Out-of-the-box Support for Mobile and Agile – Jenkins, Bamboo,
Team City, Network Virtualization, Selenium, Appium…
Comprehensive Support
22. ®®
Support Modern and Legacy Technologies
GUI
WEB
Micro-
Service
s
Component
s
Web
Mobile
Restfu
l
24. ®®
A large ecosystemFunctional Testing
APM
Cloud Provider
Assets
Technologies
Certified Service Partners
Continuous
Integration
Network WAN
emulation
Integration in progress
Misc.
25. ®®
Collaboration bewteen team
• Project team wants flexibility when building scenarios : Code or from the GUI
• But, unique performance test standard shared by the team!
28. ®
Top 5 Benefits of Rapise
1. Scriptless, easy to maintain test design
2. Powerful, intelligent locators for reliable recording
3. Web, desktop, mobile, and API testing in one package
4. Accelerators for apps such as Salesforce & Dynamics
5. Fully extensible with custom library support
32. ®®
Extensive Coverage of Applications
Web Applications
Cross-Platform Technologies
Unit Test Frameworks
Enterprise Applications
Mobile Applications
Windows Applications
Integrations
Third-Party Component Libraries
Our mission at Neotys is to help organizations accelerate Agile/DevOps with more performance testing, in less time – which means testing, and improving performance at every stage of the application development lifecycle, and doing this sooner and more frequently
We deliver this value with our load testing platform NeoLoad available for testers, developers and every person involved in performance engineering through a all channel required to fit with their development processes: web/SaaS/On premise platform, cloud and/or on premise load generation, integration with other systems with the NeoLoad API
Headquartered in France, we have offices around the globe – US east coast, India, UK, and Germany. We back-up our performance testing platform with a range of professional services:
Performance testing as a service
Consulting
Neotys Academy – In-person/online training, certification
Migration from legacy tools
The NeoLoad platform for performance feedback consists of three distinct arenas.
Our core load testing components: controllers and load generators, provided as java installers and Docker containers
NeoLoad Web, a browser-based solution to visualize load and performance testing trends and results in real-time and stream to other dashboards
Our SaaS offerings: a hosted version of NeoLoad Web and a global network of always-ready load generators in data centers around the world
Both NeoLoad Web and our core components are programmatically addressable via RESTful APIs as well as through CLI execution
NeoLoad is not just a cheaper alternative to HP LoadRunner – NeoLoad is a justifiably better load and performance testing tool for web, mobile, and other digital applications. This is especially true for applications developed in fast-paced environments practicing DevOps or Agile methodologies.
Faster Test Design
Most testing tools (including LoadRunner) have a GUI with record and playback capabilities that require some coding to further customize scripts, but NeoLoad is different because there’s no coding required at all.
Correlation can be run automatically at the end of each recording, parameterization and randomization of variables takes just a few clicks, and logical actions are all drag and drop. Customers tell us they’re 5-10 times faster at designing tests than they were with LoadRunner.
Script Maintenance Automation
Any regular LoadRunner user knows just how long it takes to maintain scripts that break any time their applications change. What good is running automated performance tests if your scripts break and it takes forever to fix them?
That’s why NeoLoad has a user path update feature that allows you to update a designed script with a quick recording and automatically keep variable extractors, think times, SLAs, logical actions, javascript, validations and more from the original user path design. This makes the script maintenance process 20 times faster so that performance testing can actually keep pace in Continuous Integration/Continuous Deployment environments.
Cheaper License, Much Higher ROI
Even if HP throws a fairly significant discount at you, the cost of a NeoLoad license is typically around one quarter the cost of a LoadRunner license. When you combine this with the huge efficiency gains you get with NeoLoad, the return on investment is massive.
For example: if you’re able to run 20 tests with NeoLoad in the same amount of time that it takes to run 4 tests with LoadRunner when the license cost is 4 times more with LoadRunner, your cost per test is 20 times more expensive with LoadRunner than it would be with NeoLoad.
Load testing is becoming more critical in todays SDLC, and also more complex because
Test cycles are becoming shorter with Agile and the requirement to release faster
Competion requires accelerated pace of digital innovation, for both customer facing and internal applications. Organizations adopt Agile and Devops as an answer to release faster and more regularly
• QA needs to be able to ensure quality at the pace of development
• This trend means you need to TEST FASTER
2) Testing is more complex as applications become more complex.
3) Speed and reliability of the application is critical to the user – so you need to test more as user expectations are raising.
• User expectations re: application performance are extremely high – seamless across devices
• Delivering applications which fail to meet user expectations can have crippling impact on the business:
- Loss of revenue
- User attrition/defection to competitor apps
- Brand degradation
Lost productivity for internal apps
When performance testing is done late in the SDLC (Pre-production) and defects are found, 100% of performance defects needed to be resolved in the production environment. In companies that implement a “Shift left, continuous testing approach, where performance considerations are taken into account at every stage of the life cycle, about 5% of performance defects need to be resolved in the production environment post-deployment.
Relative Cost of Fixing Defects
A study was performed by the IBM System Science Institute in order determine the relative cost in order to fix defects within the SDLC. The study concluded that defects found in testing were 15 times more costly than if they were found during the design phase and 2 times more than if found during implementation. (Source: IBM System Science Institute)
https://www.researchgate.net/publication/255965523_Integrating_Software_Assurance_into_the_Software_Development_Life_Cycle_SDLC
It is commonly believed that the earlier a defect is found the cheaper it is to fix it. For example, if a problem in the requirements is found only post-release, then it would cost 10–100 times more to fix than if it had already been found by the requirements review.” – (Source: Code Complete 2nd Edition, Steve McConnell)
Key benefits of using Continuous Performance Testing include:
Finding and Resolve Issues Earlier - Issues can be found at an earlier stage in the SDLC, then fixed quicker and at lower cost
Faster Release Cycles - The automation built into leading-edge performance testing platforms save a huge amount of time. With this approach, testing becomes just another part of the build - not a manual step performed as an afterthought
Deploy with Confidence & Lower Risk- Improved code quality since everything is KNOW to perform as designed - under real-world loads
Success is based on the ability to deliver both high quality testing and speed.
And this cannot be achieved with legacy methods and tools that make Performance testing the bottleneck in the SDLC!
Be confident your applications will perform in production
Leaders achieve Excellent performance testing while supporting the speed of Agile and this creates a competitive advantage for their organizations. No tradeoff
Organizations need to constantly monitor the performance of their applications, at every step of the Software Delivery Life Cycle (SDLC), from the development of the first component to the deployment of the end-to-end application in pre-production and production environments.
NeoLoad supports continuous testing requirements from API testing to enable teams test earlier and identify performance issues as early as possible
up to integration testing and to the large, complex end-to-end application performance and load testing to release with confidence in production.
NeoLoad is fully integrated into CI/CD pipelines so performance testing can be embedded into automated processes and executed continuously to let teams track non-regression performance trends and make sure their code is still performing, day after day.
NeoLoad provides the best capabilities for API testing in terms of
API technology support: SOAP and REST support
Ease of use to create tests for developers who are the most likely to do API testing
import swagger definitions to create scripts rapidly
define test scripts as code, in a Dev environment (or use the NeoLoad GUI productivity if you prefer)
Integration of API testing into a continuous process, fully automated
CI/CD integration
Automated pass/fail based on SLAs
Automatic provisioning of dynamic load testing infrastructure/load generators
Testing more than a standalone API
Test a system of APIs with more complex scenarios
Dev and QA use the same platform
Better Support for Web Technologies
Neotys has consistently beaten HP to market with support for new web protocols (sometimes by 2-3 years!), but it’s more than just saying your tool is able to test a particular technology – it’s how well your tool is able to test that technology.
Many current NeoLoad customers were once LoadRunner customers who made the switch because NeoLoad was able to more accurately capture and replay traffic from applications utilizing technologies like GWT, Oracle Forms, Flex, adaptive bitrate streaming and more.
Out-of-the-box Support for Mobile Applications
Every NeoLoad edition comes out-of-the-box with proxy recording of native mobile applications, mobile device simulation and network virtualization – no expensive mobile package to buy. You can even run tests with virtual mobile users on your web applications with just a few clicks.
NeoLoad’s “sweet spot” is in enabling large enterprises to secure their move to DevOps while ensuring testing of their legacy systems
Enterprise requirements
Large set of applications with wide business usage
Wide coverage of technology
Latest frameworks (AngularJS, Node.JS, WebSocket…)
Enterprise apps (Oracle, SAP, Kronos…)
Heterogeneous architectures and deployments (on-premise, public and private cloud applications)