The Evolution of Development Testing

The Evolution of
Development Testing

Overview
•

Automating

•

Testing Categories + Scope

•

Development Testing

•

Principles, Practices + Tool Support

Automating
•

We automate “stuff” for a living

•

Developer/Programmer/Engineer/Automator

•

When we’re doing things manually we’re usually missing
an opportunity to automate

Automating
•

The stuff we automate might be
•

validation of a web form

•

reporting of a message parsing error

•

calculation of an average execution price

•

execution + status reporting of tests for all of the above

Automating
•

That automation might look like
•

choose validation rules, implement, enforce, report

•

log parsing error, automate monitoring/reporting of error

•

deﬁne calculation rules, implement, provide result

•

deﬁne inputs, context + expectations, execute scenario,
determine + report status

Automating
•

Manual solutions might look like
•

review user errors later in workflow e.g.“back office” manual
activity

•

message loss caused an incident, manually track it down

•

export raw data to csv, load into Excel, apply formula

•

run UI functional scenarios that use/execute the code in mind
•

fire up debugger, step through mis-behaving code
•

add detailed logging, monitor logs for hints

Overview
•

Automating

•


Test Categorisation
•

Useful concept - up to a point

•

Blurred lines + Overloaded terminology

•

Difﬁculties conversing about testing

Test Categorisation
•

Unit

•

Build Acceptance

•

Component

•

System

•

API

•

End-to-End

•

Acceptance

•

User Acceptance

•

Integration

•

Non-functional

•

Functional

Overview
•

Automating

•


•

Development Testing

Development Testing
•

What it is

•

What it should not be

•

How it has evolved

What it is
•

A Development practice ﬁrst and foremost

•

White box / Black box / Grey box

•

Has Quality aims

•

Has Efﬁciency aims

JUnit
•

Test automation framework of choice for most Java
developers

•

Synonymous with Unit testing

•

Widely used for testing beyond the Unit scope

•

Ported to many languages - generically known as xUnit

JUnit
“Never in the field of software development have so
many owed so much to so few lines of code”
Martin Fowler
(Development big-wig)

What it is
•

Can take a technical perspective on scenarios

•

Can take a user functionality perspective

•

Exercises precise control over the test context

•

Used + extended by Dev during initial and future coding

What it is
•

Often used to discover and reﬁne solution design

•

Often leaves ﬁne-grained regression coverage safety net

•

Enables instant feedback on failures due to future
enhancements and refactorings

What it is not
•

Automated full System tests a.k.a E2E, System Integration

•

Zero control over the state of the system (the test context)

•

Test scope coupled to other external systems, reference
data quality, env availability

•

Heavily reliant on driving a UI to execute many ﬁnegrained scenarios

Its Evolution
•

OOP - slow evolution

•

Automated Developer Testing around a lot less time

•

A lot less mainstream than OOP

•

Growing body of knowledge / experience available

•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec

•

Practices + Tools for improving test qualities

Test structure
•

Arrange, Act, Assert

•

Specify Inputs

•

Given, When, Then

•

Specify the state of the System

•

Specify the Event

•

Specify the Expectations

Test structure
•

Naming conventions + structure
@Test
public
//

//

//
}

void methodUnderTest_GivenABC_ThenExpectXYZ() {
given
when
then

@Test
public void methodUnderTest_GivenInputsABC_AndSystemStateDEF_ThenExpectXYZ() {
// given - inputs

// given - system state
!

// when

// then
}

Test structure
•

IDE assistance for consistency + evolve conventions

•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec

“Any fool can write code
that a computer can understand”
!

Martin Fowler

Test Qualities
•

Readability
•

What is it? Why does it matter?

Test Qualities
•

Readability == …?

•

Code that you can understand with ease

•

Language affects Thought

•

Thought affects Action

Test Qualities
•

Tools can directly address Readability qualities

•

Domain Speciﬁc Languages = a fancy term
•

•

JUnit, Hamcrest, FEST, JMock, Mockito, Gherkin

Language -> Thought -> Action

“Test automation is a first class software
engineering problem”
!

Brian Marick?

Test Qualities
•

Maintainability
•

What is it? Why does it matter?

Test Qualities
•

Maintainability == …?

•

To change, evolve with ease

•

Keeping the cost of change down

Test Qualities
•

They can be context sensitive. Many are not.

•

You may opt to sacriﬁce:
•
•

Readability for Maintainability

•
•

DRY Principle for Readability

Maintainability for Obviousness

Expect debate

Specifying Expectations
•

Stating Expectations / Asking Questions

•

From Computer dialect ——> Human dialect
// then
assertTrue(trader.getPermissions().isEmpty());
assertThat(trader.getPermissions().isEmpty(), is(true));
assertThat(trader.getPermissions()).isEmpty();
assertThat(trader).hasNoPermissions();

//
//
//
//
//
//
//

Computer dialect
.
.
.
.
.
Human dialect

•

Stating Expectations / Asking Questions

•

From Computer dialect ——> Human dialect
// Using Basic JUnit
assertEquals(orders.size(), 2);
Iterator<Order> itr = orders.iterator();
assertEquals(itr.next(), order3);
assertEquals(itr.next(), order4);
// Using Hamcrest
assertThat(orders, hasItems(order3, order4));
assertThat(orders.size(), is(2));
// Using FEST Assertions
assertThat(orders).containsOnly(order3, order4);
// Using FEST Assertions (chained assertions)
assertThat(orders).containsOnly(order3, order4)
.containsSequence(order3, order4);

•

Easier + Faster to write expressive statements

•

Faster to read + review

•

Easier to maintain

•

Encourages other better practices

Specifying Inputs
•

Trivial example. Test with very narrow scope
•

e.g.Input = Single Object
@Test
public void getPermissions_GivenNewUser_ThenReturnsNoPermissions() {
// given
Trader trader = new Trader();
// when + then
assertThat(trader.getPermissions()).isEmpty();
}

Specifying Inputs
•

Non-trivial example. Test with broader scope
•

e.g. Input = Object Graph
•

i.e. objects linked to objects

@Test public void
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() {
// given - inputs
TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"),
new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE);
trader.setPermissions(Arrays.asList(new Permission(tradingAcct)));
String goldIsinCode = "COF24680";
Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123"));
Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456"));
Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode));
String orderId = "ordId";
Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500);
Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600);
Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300);
Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300);
OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4);
OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO);
// when
List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode);
// then
}

Specifying Inputs
•

Consider many non-trivial tests, with “raw” setup
•

Impact of changing one setter or constructor

•

Quantity of code there is to read, understand, change

Specifying Inputs - Techniques
•

Helper methods

•

Object Mother Pattern

•

Test Data Builder Pattern

@Test public void // Using helpers
// given - inputs
String accountCode = "TRDRZ";
String goldIsinCode = "GLD24680";
TradingAccount tradingAcct = createTradingAccount(accountCode);
Trader trader = createTraderWithPermissionsFor(tradingAcct);
Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123");
Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456");
Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode);
Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500));
Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600));
Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300));
Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300));
OrderSearchService orderSearchService = createOrderService(
createOrdersDAO(order1, order2, order3, order4));
// when
List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode);
// then
}

@Test public void // Using helpers + hiding irrelevant details
// given - inputs
String searchAccount = "TRDRZ", searchIsin = "GLD24680";
Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ"));
Order order1 = createOrder("TRDRZ", dummyFuture()),
order2 = createOrder("TRDRZ", dummyFuture()),
order3 = createOrder("TRDRZ", createFuture("GLD24680")),
order4 = createOrder("TRDRZ", createFuture("GLD24680"));
OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4));
// when
List<Order> orders = service.getOrders(trader, searchAccount, searchIsin);
// then
}

•

Object Mother Pattern

public class ObjectMother {

!
public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, Order order4) {
return new OrdersDAOInMemory(order1, order2, order3, order4);
}

!
public static OrderSearchService createOrderService(OrdersDAO ordersDAO) {
OrderSearchService orderSearchService = new OrderSearchServiceImpl(ordersDAO);
return orderSearchService;
}

private OrderSearchService prepareOrderService(Order order1, Order order2, Order order3, Order order4) {
OrdersDAO orderDAO = createOrdersDAO(order1, order2, order3, order4);
OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO);
return orderSearchService;
}

!
public static Order createOrder(String accountCode, Future dummyFuture) {
return createOrder(accountCode, dummyFuture, qty(0), price(0));
}

!
public static Order createOrder(String accountCode, Future future, int qty, int price) {
return new Order(nextOrderId(), accountCode, future, qty, price);
}

private Order createOrder(String orderId, TradingAccount tradingAcct, Future future, int qty, int price) {
return new Order(orderId, tradingAcct.getTradingFirm().getCode(), future, qty, price);
}

!
public static Future createFuture(String desc, String isinCode) {
return new Future(desc, new ISIN(isinCode));
}

public static Future createFuture(String isinCode) {
return createFuture(DEFAULT_FUTURE_DESC, isinCode);
}

!
public static Future dummyFuture() {
return createFuture(DEFAULT_FUTURE_DESC, DEFAULT_ISIN_CODE);
}

!

•

Test Data Builder Pattern

@Test public void // Using Test Data Builders
// given - inputs
Trader trader = aTrader().with(aPermission()
.with(aTradingAccount()
.with(aTradingFirm().withCode("TRDRZ")))).build();
OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ")
.with(aFuture().with(anISIN().withIsinCode("GLD24680")).build());
Order order1 = anOrder().withAccountCode("TRDRZ").build(),
order2 = anOrder().withAccountCode("TRDRZ").build(),
order3 = matchingOrder.build(),
order4 = matchingOrder.build();
// when
// then
}

public class TraderBuilder {
public static String DEFAULT_USERNAME = "trader1";
public static UserDetail DEFAULT_USERDETAIL = new UserDetail();
public static List<Permission> NO_PERMISSIONS = Collections.emptyList();
public static List<Restriction> NO_RESTRICTIONS = Collections.emptyList();

!

private String userName = DEFAULT_USERNAME;
private UserDetail userDetail = DEFAULT_USERDETAIL;
private List<Permission> permissions = NO_PERMISSIONS;
private List<Restriction> restrictions = NO_RESTRICTIONS;

private TraderBuilder() { }

public static TraderBuilder aTrader() {
return new TraderBuilder();
}

public Trader build() {
trader.setUserName(userName);
trader.setUserDetail(userDetail);
trader.setPermissions(permissions);
trader.setRestrictions(restrictions);
return trader;
}

public TraderBuilder withUserName(String userName) {
this.userName = userName;
return this;
}

!

public TraderBuilder with(UserDetail userDetail) {
this.userDetail = userDetail;
return this;
}

public TraderBuilder withPermissions(List<Permission> permissions) {
this.permissions = new ArrayList<>(permissions);
return this;
}

public TraderBuilder
this.permissions
return this;
}

public TraderBuilder
this.permissions
return this;
}

with(PermissionBuilder permission) {
= new ArrayList<>(Arrays.asList(permission.build()));

withNoPermissions() {
= Collections.emptyList();

!

public TraderBuilder with(RestrictionBuilder restriction) {
this.restrictions = new ArrayList<>(Arrays.asList(restriction.build()));
return this;
}

public TraderBuilder withRestrictions(List<Restriction> restrictions) {
this.restrictions = new ArrayList<>(restrictions);
return this;
}

public TraderBuilder but() {
return new TraderBuilder()
.withUserName(userName)
.with(userDetail)
.withPermissions(permissions)
.withRestrictions(restrictions);
}
}

@Test public void // Using Test Data Builders to create input data variations
getOrders_GivenTraderWithVariousPermissions_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v5() {
// given - inputs
TradingAccountBuilder tradingAccount = aTradingAccount().with(aTradingFirm().withCode("TRDRZ"));
Trader traderWithNoPerms = aTrader().withNoPermissions().build();
Trader traderWithPerms = aTrader().with(aPermission().with(tradingAccount)).build();
Trader traderWithInactivePerms = aTrader().with(aPermission().with(
tradingAccount.but().isInActive())).build();
OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ")
.with(aFuture().with(anISIN().withIsinCode("GLD24680")).build());
Order order1 = anOrder().withAccountCode("TRDRZ").build(),
order2 = anOrder().withAccountCode("TRDRZ").build(),
order3 = matchingOrder.build(),
order4 = matchingOrder.build();
// when
List<Order> orders1 = service.getOrders(traderWithNoPerms, searchAccount, searchIsin);
List<Order> orders2 = service.getOrders(traderWithPerms, searchAccount, searchIsin);
List<Order> orders3 = service.getOrders(traderWithInactivePerms, searchAccount, searchIsin);
// then
assertThat(orders1).isEmpty();
assertThat(orders2).containsOnly(order3, order4);
assertThat(orders3).isEmpty();
}

•

Test Data Builder Pattern - with tool support
•

Make-It-Easy - addresses Builder class boiler-plate

•

Model Citizen - annotation-based “Blueprints”

•

Test Data Builder Pattern - Advantages
•

Your test / spec documents only the inputs that matter

•

Decouples your test from being impacted by a wide
range of changes

•

Supports “declarative” code style + useful templates
•

user = EXPIRED_USER.but( with ( aBalance, 100));

•

Test Data Builder Pattern - Disadvantages
•

Requires much new test-supporting code

•

Chained methods take getting used to

Specifying the System State
•

State of the SUT / the Test Context

•

Communicating what the context, limitations and
boundaries are

•

Isolating your test from certain dependencies / interactions

•

Use of Test Doubles
•

Hand-rolled Test Double

•

common misnomer Mocking Frameworks

Specifying the System State
•

Problems + solutions for Specifying Inputs normally apply

•

Unique to System State is your Test Isolation requirements

Specifying the System State - Techniques
•

Hand-rolling Test Doubles

•

Using Test Isolation frameworks to provide Test Doubles

•

“Mocking” + “Mocks Aren’t Stubs” 
http://martinfowler.com/articles/mocksArentStubs.html

•

Dummy / Fake / Stubs / Mocks / Spies

•

Stub being used in a state-based test

@Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details
// given - inputs
Order order1 = createOrder("TRDRZ", createFuture("GLD24680")),
OrdersDAO ordersDAOStub = mock(OrdersDAO.class);
given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2));
OrderSearchService service = createOrderService(ordersDAOStub);
// when
// then
}

@Test public void // Stubbing system state + stubbing inputs (Mockito)
// given - inputs
// Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ"));
Trader traderStub = mock(Trader.class);
Permission permissionStub = mock(Permission.class);
given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub));
TradingAccount tradingAccountStub = mock(TradingAccount.class);
given(permissionStub.getTradingAccount()).willReturn(tradingAccountStub);
given(tradingAccountStub.isActive()).willReturn(Boolean.TRUE);
given(tradingAccountStub.getTradingFirm()).willReturn(new TradingFirm("", searchAccount));
// when
List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin);
// then
}

@Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs)
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3_1() {
// given - inputs
// Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ"));
Trader traderStub = mock(Trader.class);
Permission permissionStub = mock(Permission.class, RETURNS_DEEP_STUBS);
given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub));
given(permissionStub.getTradingAccount().isActive()).willReturn(Boolean.TRUE);
given(permissionStub.getTradingAccount().getTradingFirm().getCode()).willReturn(searchAccount);
// when
List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin);
// then
}

•

A Mock being used in a interaction-based test

@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito)
// given - inputs
PermissionService permissionServiceMock = mock(PermissionService.class);
OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock);
// when
// then
verify(permissionServiceMock).hasViewPermissions(trader, order1);
}

@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito)
// given - inputs
PermissionService permissionServiceMock = mock(PermissionService.class);
given(permissionServiceMock.hasViewPermissions(
any(Trader.class), any(Order.class))).willReturn(Boolean.TRUE);
OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock);
// when
// then
}

•

Often a test will have
•

Zero to many Stubs. Zero or one Mock

•

Mocks help you answer questions about what happened

•

When a Mock is present, you ask it something at the end

•

A Stub should not be asserted against

•

There to help create + control the System State for the test

Specifying the Event
•

Often trivial

•

Might be a sequence of events that we want to ﬁre

•

Sequence might actually be context / SUT setup

•

Practices? Tools?

Specifying it All Together
•

Spring MVC Test support / DSL

@Test
public void helloWorld() throws Exception {
mockMvc.perform(get("/hello").accept(MediaType.TEXT_PLAIN))
.andDo(print()) // print the request/response in the console
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.TEXT_PLAIN))
.andExpect(content().string("Hello World!"));
}

Continuous Testing
•

Continuous Integration - development practice

•

CI tools - run automated tests frequently

•

Continuous Testing - development practice

•

CT tools - run automated tests locally, frequently

•

e.g. Inﬁnitest, JUnitMax

Summary
•

Tools/APIs just help with heavy lifting

•

Sound coding principles are paramount

•

Creating unreadable and unmaintainable test, even with
“the right” tools

•

Unreadable + high maintenance tests will hinder efﬁciency

Summary
•

The power of marginal improvements

•

for … each, <> operator, closure support, Guava

•

Continuous iterative improvements nudge us forward

•

Continuous iterative improvements push down cost of
reading, writing, evolving tests

•

Drives the cost down to “mass market” level

•

Beyond Java

Resources
•

Books
•

GOOS - Growing Object Oriented Software, Guided By
Tests

•

Effective Unit Testing

•

Working Effectively with Legacy Code

•

xUnit Test Patterns - Refactoring Test Code

Resources
•

Podcasts
•
•

Hanselminutes - Roy Asherove - The Art of Unit Testing

•
•

Sustainable Test Driven Development (Series)

Hanselminutes - Quetzal Bradley - Beyond Unit Testing

All Code Examples are available at
•

http://github.com/cathalking/devtestevolution

The Evolution of Development Testing

The Evolution of Development Testing

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (8)

Similaire à The Evolution of Development Testing

Similaire à The Evolution of Development Testing (20)

Dernier

Dernier (20)

The Evolution of Development Testing