Bugs happen! It is a developers life fact. Let'e explore one way we the developers can help customers to make batter bug reports. During lifecycle of systems and applications that support complex and long running business processes it is often the challenge to get accurate bug report. In this talk we will present one custom developed solution that we used on several our projects as well as our experiences in using this approach.

1. How to help customers report bugs
accurately
Miroslav Čerkez
Milan Jovanović

2. 2
Intro – how most user report bugs
➢
Friction locks cause throttle
levers to stick.
➢
Suspected crack in windshield.
➢
Aircraft handles funny.
➢
Something loose in cockpit.
➢
Evidence of leak on right main
landing gear.

3. 3
Facts
➢
Most users aren’t technical people
➢
Users don’t have same ideas what helps and
what doesn’t when debugging as developers
➢
Users just want to get their work done
➢
They will not spend a second more than is needed to report
bug
➢
Bugs can be subtle
➢
Bugs can happen in edge cases and are difficult
to explain

4. 4
Our case
➢
Complex system
➢
Supported business processes spanning
multiple years (no strict end state)
➢
Business rules depending on previous system
state
➢
The older the process possible code paths increases
➢
Current bug usually is cause by some undetected bug
upstream

5. 5
Our case - problem
➢
As development progressed bugs are more
difficult to reproduce
➢
Testing team bug reports had 4 pages of text and 3 MB of XML
➢
Onboarding new team members takes very long time
➢
We need a easier way get bug reports
➢
Once we get it; How do we reproduce error
locally?

6. 6
Our case – problem
➢
Bug report contain possible sensitive business
information
➢
If logic depends on data can’t be helped – requires non
technical solution (NDA) – tight control of access
Doctor (developer) can’t operate (fix
error) on a patient (program) without
examining it.

7. 7
Our case - solution
SCENARIO BUILDER
➢
Procedure to collect all relevant data to reproduce issue on a
developer system
➢
Application API for loading data into system (mock setup for
integrated systems etc.)
➢
Custom JUnit runner used to “drive” application into desired
state

8. 8
Scenario bulder – data collection
➢
Application stores all input/output data
➢
Current application state can be calculated by
reprocessing all input data
➢
There must be a finite (small) number of parameters that can
uniquely single out input/output dataset
➢
It is usually business oriented (top level object unique key is a
good candidate)
➢
Get everything developer need in a single zip archive

9. 9
Scenario bulder – data collection

10. 10
Scenario bulder – scenario.xml

11. 11
Scenario bulder – scenario.xml

12. 12
Scenario bulder – Junit runner
➢
Provides integration with build tools
➢
Run scenario test during CI/CD pipeline
➢
Great for regression testing
➢
Validate system for common happy path scenarios on every
build
➢
Simple to configure – annotated test job with folder where
scenario.zip is extracted

13. 13
Scenario bulder – Junit runner –
technical details
➢
Based on BlockJUnit4ClassRunner
➢
scenario.xml schema defines scenario format
➢
Junit runner parses scenario.xml used to drive test execution
➢
Send message to message queue
➢
Send request to HTTP API (public)
➢
Send request to HTTP API (private) – introspection if internal state

14. 14
Scenario bulder – benefits
➢
Core benefits
➢
Bug report consists of last action that resulted in error and
scenario unique id
➢
Short time to bug reproduction in development environment
➢
Side benefits (use cases discovered during
usage)
➢
Introspection of application state history – not only current
state - possible without storing all the states in database
➢
Business analysts began writing specification in technical
language (referent scenarios) [code]

15. 15
Scenario bulder – challanges
➢
Time independent logic
➢
current date, current timestamp == enemy
➢
Develop patterns to mitigate issue
➢
Integration with external systems
➢
Store all external interaction and when running Junit, mock
integration interaction
➢
Parameterization
➢
Attempted to write scenario packets using parameterized
input and output messages

16. 16
Thank you
➢
Thank you!
➢
Questions?