We shall explore the concept of web application testing through the verification of declarative constraints on the contents of its user interface --that is, DOM, CSS and behavioural properties. It turns out that a lot of non GUI (e.g. functional, security) problems can also be verified by observing the application's GUI.
33. Experience reveals that many bugs
manifest themselves through their UI
They can be detected by observing
elements of the UI...
...in a single snapshot, or across
a sequence of snapshots
34. A language for expressing "high"-level
oracles over web UIs
- DOM + CSS attributes
- First-order quantification
- Temporal operators
1.
35. For each $x in $(css-selector)
something about $x.
There exists $x in $(css-selector)
such that
something about $x.
36. $x's prop is greater than
$y's prop
$x's height
$x's text
$x's color
$x's prop matches "a string"
...and most DOM/CSS
attributes
...and most common
functions
37. something And something else
Always something
Eventually something
something Until something else
The next time something,
then something else
...in other words, propositional logic + LTL
38. We say that pattern
When something.
Example:
We say that $x is wider than $y
When
$x's width is greater
than $y's width.
User-defined predicates to improve
legibility and create "mini-DSLs"
39. We say that I click on Go when (
There exists $b in $(button) such that (
($b’s text is "Go")
And
($b’s event is "mouseup"))).
Always (
If (I click on Go) Then (
There exists $x in $(.value) such that (
The next time (I click on Go)
Then (
There exists $y in $(.value) such that (
$x’s text equals $y’s text ))))).
40. $(document).mouseUp(function(event) {
var e = arguments.callee;
if ($(event.target).text() === "Go") {
var current_values = [];
$(".value").each(
current_values.push($(this).text());
);
if (e.lastValues !== undefined) {
var found = false;
for (var v in current_values) {
if ($.inArray(v, e.lastValues)) {
found = true;
break;
}
}
}
if (!found)
console.log("Error");
e.lastValues = current_values;}
};);
The same in jQuery
41. An architecture to instrument a web app
and report relevant UI attributes to an
interpreter
2.
42. Server
Application under test
<script ...
"getprobe?uid=39f2ac8">
Probe
creationRequest for probe code
Probe
interaction
Probe code
Probe (JS object)
Probe status report
Probe instructions (optional)
Probe log
Probe repository
Probe
retrieval
Developer
Probe ID
Dashboard
(analytics)
Developer
1
234
5
6
7
8
9
12
10
11
13 16
14
15
43. ... $(ul li) ...
... 's width ...
... $(p#abc) ...
... 's color ...
The JavaScript probe is selective:
it only sends back the attributes
mentioned in the property
only for the elements matching one of
the selectors
44. A mechanism for explaining the violation
of a property when found
3.
45.
46.
47. Producing a witness for a violation does not
require to be coded in any way
It is just a by-product of evaluating the
logical expression
It works for temporal formulas
too!
48. See it in action
https://www.youtu.be/90YitGRRx2w
BEST
TOOL
Hallé, Bergeron, Guérin, Le Breton.
Testing Web Applications Through Layout
Constraints. Proc ICST 2015.
https://bitbucket.org/sylvainhalle/cornipickle
50. cIcI My 2 cents...
In UI-based bugs, the browser matters
Don't take design choices that limit you from
the start (Selenium, any browser plugin)
If the app runs in browser X, so should your
testing tool
This may mean a lot of work ahead!
51. cIcI My 2 cents...
For each $x in $(selector)
blabla...
There exists $y in $(selector)
blabla...
CSS selectors are fragile. Simple changes to
the layout may break the property
52. cIcI My 2 cents...
...in search of a
carpenter
Hammer and
nails...
53. We still have to write the
properties, so what's the point?*
We're too busy to learn
another tool!*
Using this requires a higher level of
education. We'll have to increase the wages!*
*Really happened
We can already do all this (with a bunch
of home-brewed Python scripts)!*
54. Turn specifications into
sets of test cases
Run Cornipickle in tandem
with a crawler
Hub for sharing
specifications (à la
Greasemonkey)