Slides from Confitura 2017: https://2017.confitura.pl/presentations

1. Performance tests with
Gatling
Andrzej Ludwikowski

2. About me
➔aludwikowski.blogspot.com
➔github.com/aludwiko
➔@aludwikowski

4. Performance tests - why so hard?
Simulate production as close as possible:
hardware
CPU, RAM, storage, ...
software
OS, Virtualization, DBs, …
load
isolation

5. Performance tests - why so hard?
Necessary to have:
infrastructure
monitoring
logging

6. Performance tests - why so hard?
Your performance intuition is wrong!

7. Performance tests - why so hard?
Lies, damned lies, and statistics:
arithmetic mean = 2.9
median = 1
standard deviation = 6 (only for normal distribution)
Use:
percentiles
50 = 1
70 = 1
90 = 2.9
95 = 11.45
99 = 18.29
Check percentiles implementation!

8. Performance tests - why so hard?
● Coordinated omission problem by Gil Tene
http://bravenewgeek.com/tag/coordinated-omission/

9. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)

10. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 31
50 % 1
70 % 1
90 % 1
95 % 1
99 % 12,89
99,9 % 28,30
99,99 % 29,82
avg 1,93

11. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 10001
50 % 1
70 % 1
90 % 1
95 % 1
99 % 1
99,9 % 1,15
99,99 % 28,01
avg 1,03

12. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)

13. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 60
50 % 1
70 % 12,6
90 % 24,3
95 % 27,2
99 % 29,45
99,9 % 29,95
99,99 % 29,99
avg 8,25

14. Performance tests - why so hard?

15. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s

16. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%

17. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%
expected time in % for 99% 50% - 1% = 49%

18. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%
expected time in % for 99% 50% - 1% = 49%
real time for 99% 60 s * 49% = 29,4 s

19. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/

20. Performance tests - why so hard?
Tests…

21. Performance tests - why so hard?
Tests…

22. Gatling

23. Gatling

24. Why Gatling?
non-blocking, asynchronous stack (scala, akka, netty)
scala !!!111oneoneone (maven, sbt support)
DSL
recorder
math is good
reports
integration & performance tests

25. Gatling DSL
class BasicSimulation extends Simulation {
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
.acceptEncodingHeader("gzip, deflate")
.acceptLanguageHeader("en-US,en;q=0.5")
.userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")
val firstScenario = scenario("First Scenario Name")
.exec(http("Request name").get("/"))
.pause(7)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

26. Gatling DSL
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

27. Gatling DSL
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

28. Gatling DSL
class ComplexScenario extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(http("request_1").get("/")).pause(7)
.exec(http("request_2").get("/computers?f=macbook")).pause(2)
.exec(http("request_3").get("/computers/6")).pause(3)
.exec(http("request_4").get("/")).pause(2)
.exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds)
.exec(http("request_6").get("/computers/new")).pause(1)
.exec(http("request_7")
.post("/computers")
.formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37"))
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

29. Gatling DSL
class ComplexScenarioV2 extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
.exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

30. Gatling DSL
class ComplexScenarioV3 extends MySimulation {
val search = exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
val addComputer = exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
val complexScenario = scenario("Complex demo scenario").exec(search, addComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

31. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(status.is(200))
.check(status.in(200 to 210))

32. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(regex("computers")
.find(1)
.exists)
https://www.pinterest.com/pin/491666484294006138/

33. Gatling DSL - checks
scenario("First Scenario Name")
.exec(http("Request name")
.get("/computers")
.check(jsonPath("$..foo.bar[2].baz").ofType[String].notExists)
.check(xpath("//input[@id='text1']/@value").is("test"))
.check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))

34. Gatling DSL - checks
scenario("DSL demo")
.exec(http("Authorize")
.get("/auth")
.check(regex("token").find(1).exists
.saveAs("authorizationToken")))
.exec(http("Authorized resource")
.get("/authorized_resource?token=${authorizationToken}"))

35. Gatling DSL - looping
repeat(5, "i") {
exec(goToPage("${i}".toInt))
.pause(1)
}
repeat
foreach
during
asLongAs
forever
https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr

36. Gatling DSL - polling
exec(
polling
.every(10 seconds)
.exec(searchFor("thinkpad"))
)
http://www.firmus-solutions.com/terms-conditions.html

37. Gatling DSL - conditions
doIf(session => session("user").as[String].startsWith("admin")) {
exec(goToAdminPage)
}
doIf
doIfEquals
doIfOrElse
doSwitch
doSwitchOrElse
randomSwitch
https://en.wikipedia.org/wiki/Decision_tree

38. Gatling DSL - error management
exec(sendMoney)
.tryMax(10){
exec(checkIfMoneyReceived)
}
tryMax
exitBlockOnFail
exitHereIfFailed
Alice Bob
Kafka

39. Gatling DSL - setup
setUp(myScenario
.inject(
nothingFor(4 seconds),
atOnceUsers(10),
rampUsers(10) over (5 seconds))
.protocols(httpConf))
.maxDuration(10 minutes)
constantUsersPerSec
rampUsersPerSec
splitUsers
heavisideUsers

40. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.assertions(
global.responseTime.max.lt(50),
global.failedRequests.percent.is(0)
)
http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html

41. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.throttle(
reachRps(100) in (30 second),
holdFor(1 minute),
jumpToRps(50),
holdFor(2 hours)
)

42. Gatling DSL - feeders
val companies = List("apple", "lenovo", "hp")
val feeder = Iterator.continually(
Map("company" -> companies(Random.nextInt(companies.size))))
val searching = scenario("Searching")
.feed(feeder)
.exec(searchFor("${company}"))
RecordSeqFeederBuilder
CSV
JSON
JDBC
Sitemap
Redis
… http://favim.com/orig/201104/23/Favim.com-22725.jpg

43. Gatling DSL - resource inferring
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("...")
.acceptEncodingHeader("...")
.acceptLanguageHeader("...")
.inferHtmlResources()
.userAgentHeader("...")

44. Gatling - launching
● gatling:test
● gatling:testOnly x.y.z.GetUserSimulation

45. Gatling DSL - other goodies
Custom validators
HTTP
SSL
SSE (Server Sent Event)
basic cookies support
WebSocket
JMS
Pluginable architecture:
cassandra plugin
kafka plugin
rabbitMQ
AMQP

46. Gatling DSL - logging/debugging
● logging ALL HTTP request and responses
<logger name="io.gatling.http.ahc" level="TRACE" />
<logger name="io.gatling.http.response" level="TRACE" />
logging ONLY FAILED HTTP request and responses
<logger name="io.gatling.http.ahc" level="DEBUG" />
<logger name="io.gatling.http.response" level="DEBUG" />

47. Gatling DSL - reporting
================================================================================
---- Global Information --------------------------------------------------------
> request count 10 (OK=10 KO=0 )
> min response time 40 (OK=40 KO=- )
> max response time 177 (OK=177 KO=- )
> mean response time 55 (OK=55 KO=- )
> std deviation 41 (OK=41 KO=- )
> response time 50th percentile 42 (OK=42 KO=- )
> response time 75th percentile 43 (OK=43 KO=- )
> response time 95th percentile 117 (OK=117 KO=- )
> response time 99th percentile 165 (OK=165 KO=- )
> mean requests/sec 0.909 (OK=0.909 KO=- )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms 10 (100%)
> 800 ms < t < 1200 ms 0 ( 0%)
> t > 1200 ms 0 ( 0%)
> failed 0 ( 0%)
================================================================================

48. Gatling DSL - distributed

49. Gatling DSL - distributed
IT system

50. Gatling DSL - distributed
● manually
● Gatling FrontLine
● Flood.io
IT system