"Today it's crystal clear why we need unit tests. Even integration and acceptance tests are quite common but who is making sure that your pages are working in production environment? I'd like to show in detail how smoke tests will help achieving this goal and why you should try to burn down your production server."
This is a hands-on talk about how we use excessive smoke tests in our continuous deployment to make sure a freshly rolled-out release can actually go live.
17. What are SmokeTests?
In computer programming and software testing, smoke testing is preliminary testing to
reveal simple failures severe enough to reject a prospective software release.
Source: https://en.wikipedia.org/wiki/Smoke_testing_(software)
18. SmokeTests should…
• … be simple
• … be fast
• … test pages with optional parameters too
• … cover at least all URLs in google index
• … use a manual maintained list of URLs
19. How do SmokeTests work?
https://www.my-application.com/foo
<html><body>…</body></html>
TTFB: 65ms
HTTP 1.1/200 OK
SmokeTest
Client
CI Server
Application
Under Test
Production Server
20. How do SmokeTests work?
https://www.my-application.com/foo
<html><body>…</body></html>
TTFB: 320ms
HTTP 1.1/200 OK
SmokeTest
Client
CI Server
Application
Under Test
Production Server
21. What should SmokeTests validate?
• Correct Server
• Status Code
• Time To First Byte
• If Body Is Provided
22. SmokeTests are NOTAcceptance Tests
SmokeTest
Client
HTTP 1.1/200 OK
<html>
<head>
<title>Foo</title>
<body>
<div id="bar"><span>foobar</span></div>
</body>
</html>
23. namespace KartenmachereiTesting;
use PHPUnit_Framework_TestCase;
class SmokeTest extends PHPUnit_Framework_TestCase
{
/**
* @dataProvider furyUrlProvider
*
* @param Url $url
*/
public function testFuryUrl(Url $url)
{
$result = $this->sendGetRequest($url);
$this->assertSame(200, $result->getStatusCode());
$this->assertNotEmpty($result->getBody());
$this->assertLessThanOrEqual(100, $result->getTimeToFirstByteInMilliseconds());
}
public function furyUrlProvider()
{
$urls = ['http://www.kartenmacherei.de', …];
$urlCollection = UrlCollection::fromStrings($urls);
return $urlCollection->asDataProviderArray($urlCollection);
}
33. class SmokeTest extends PHPUnit_Framework_TestCase
{
use SmokeTestTrait;
/**
* @dataProvider myDataProvider
*/
public function testExample(Result $result)
{
$this->assertSuccess($result);
$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);
$this->assertBodyNotEmpty($result);
$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);
}
public function myDataProvider()
{
$urls = ['http://www.kartenmacherei.de', …];
$options = new SmokeTestOptions(
UrlCollection::fromStrings($urls),
new RequestTimeout(2),
new FollowRedirects(true),
new Concurrency(3),
new BodyLength(500)
);
return $this->runSmokeTests($options);
}