Talk given at WP Helsinki Meetup 7.11.2018 See also: * https://developer.wordpress.org/themes * https://developer.wordpress.org/plugins * https://travis-ci.org/Seravo * https://seravo.com/blog/coding-wordpress-in-style-with-phpcs/

1. Automatic testing
and quality assurance
for WordPress plugins
and themes
WP Helsinki Meetup 7.11.2018
Otto Kekäläinen
@ottokekalainen
WP-palvelu.fi / Seravo.com

2. ● WP-palvelu.fi – WordPress hosting
and upkeep
● CEO, sysadmin and developer
● Linux and open source advocate
● Contributed to WordPress Core, fi
and sv translations, Linux, Docker,
Nginx, Redis, MariaDB…
● Twitter:@ottokekalainen
Otto Kekäläinen

3. Enterprise grade
hosting and upkeep
for WordPress

4. FIRST THINGS FIRST
● Before you write any WordPress theme
or plugin code, please read up on the
basics at:
○ developer.wordpress.org/themes
○ developer.wordpress.org/plugins

5. CHALLENGE:
EVERYBODY WANTS QUALITY
Developers just don’t have enough
time or customers budget

6. Solution:
Automatic ~ zero cost

7. DO EVERYTHING THAT CAN BE AUTOMATED
● scan code to find errors
○ static analysis
● run code to find errors
○ unit and integration tests

8. TOOLS
1. What can test PHP code
2. What can automate tests

9. WHAT TO DO ABOUT PHP CODE
● PHP Code Sniffer
● PHP unit tests
● PhantomJS Headless Chrome
integration tests
● performance
○ execution time
○ memory usage

10. PHP CODE SNIFFER

11. PHPCS

12. PHPCFB AND GIT CITOOL

13. phpcs.xml
<?xml version="1.0"?>
<ruleset name="Seravo">
<!-- show progress -->
<arg value="p"/>
<!-- check current and all subfolders if no file parameter given -->
<file>.</file>
<rule ref="Squiz.PHP.CommentedOutCode"/>
<rule ref="Squiz.WhiteSpace.SuperfluousWhitespace"/>
<rule ref="Generic.CodeAnalysis.UnusedFunctionParameter"/>
<rule ref="Generic.Commenting.Todo"/>
<rule ref="Generic.ControlStructures.InlineControlStructure"/>
<rule ref="WordPress-Extra">
<exclude name="Generic.WhiteSpace.DisallowSpaceIndent"/>
<exclude name="Generic.WhiteSpace.ScopeIndent"/>
<exclude name="WordPress.WhiteSpace.PrecisionAlignment.Found" />
<exclude name="WordPress.PHP.YodaConditions" />
</rule>
</ruleset>

14. phpcs --standard=Security
FILE: tools/gapi.php
---------------------------------------------------------------------
FOUND 1 ERROR AND 8 WARNINGS AFFECTING 8 LINES
---------------------------------------------------------------------
35 | WARNING | Possible RFI detected with GADWP_DIR on include_once
51 | WARNING | Function array_map() that supports callback detected
148 | WARNING | Possible XSS detected with esc_url on echo
152 | WARNING | Possible XSS detected with __ on echo
156 | WARNING | Possible XSS detected with _e on echo
307 | WARNING | Crypto function crc32 used.
767 | WARNING | Function array_map() that supports callback
---------------------------------------------------------------------
More tips at seravo.com/coding-wordpress-in-style-with-phpcs

15. PHP UNIT
TESTS
assertEquals(a, b)
Lots of PHP Unit test
examples in WordPress
Core source

16. ACCEPTANCE
TESTS
Codeception PHP
framework
+
Headless Chromium

17. Example test code
<?php
class ExampleCest {
/**
* Open front page (/)
**/
public function openFrontPage(AcceptanceTester $I) {
$I->amOnPage('/');
$I->checkBrowserConsole();
$I->see('WordPress');
}
}
Read more at seravo.com/docs/tests/ng-integration-tests

18. VISUAL REGRESSION TESTS
$ gm compare -highlight-style assign
-highlight-color purple -file diff.png *.png

19. VISUAL REGRESSION TESTS
$ gm compare -verbose -metric mse *.png
Image Difference (MeanSquaredError):
Normalized Absolute
============ ==========
Red: 0.0319159868 8.1
Green: 0.0251841368 6.4
Blue: 0.0278537225 7.1
Opacity: 0.0000000000 0.0
Total: 0.0212384615 5.4

20. Where do you draw the line
between acceptable changes
and failures/regressions?

21. HOW TO AUTOMATE
● git pre-commit hook
○ local tests
● git receive hook on a remote server
○ Github + Travis-CI
○ Gitlab + Gitlab-CI
○ Bitbucket
○ etc..

22. GIT PRE-COMMIT HOOK IN ACTION

23. Example .git/hooks/pre-commit
# Loop all files that are about to be committed (diff of git head and staged)
echo "==> Checking syntax errors..."
for FILE in $(git diff --cached --name-only); do
resource="$REPO_DIR/$FILE"
##
# Test PHP syntax for all changed *.php and *.module files
##
if [[ "$FILE" =~ ^.+(php|module)$ ]]; then
if [[ -f $resource ]]; then
phpcs "$resource" 1> /dev/null
if [ $? -ne 0 ]; then
errors+=("PHP syntax Error: $FILE")
fi
fi
fi
done
See code at seravo.com/coding-wordpress-in-style-with-phpcs

24. TRAVIS-CI IN ACTION

25. TRAVIS-CI IN ACTION

26. TRAVIS-CI CHECKING EVERY COMMIT

27. ..AND PULL REQUESTS!

28. NOTIFICATION EMAILS THAT
CAN’T GO UNNOTICED

29. Example .travis.yml
before_install:
- if [[ "$SNIFF" == "1" ]]; then git clone …
script:
# Syntax check all php files and fail for any error text in STDERR
- '! find . -type f -name "*.php" -exec
php -d error_reporting=32767 -l {} ; 2>&1 >&- | grep "^"'
# More extensive PHP Style Check
- if [[ "$SNIFF" == "1" ]]; then $PHPCS_DIR/bin/phpcs -i;
$PHPCS_DIR/bin/phpcs --standard=phpcs.xml; fi
- phpunit

30. Live examples
github.com/Seravo/seravo-plugin
github.com/Seravo/linux-tuki.fi
travis-ci.org/Seravo

31. FREE FOR OPEN SOURCE CODE SERVICES
● circleci.com
● cocodacy.com
● codeclimate.com
● codeship.com
● coveralls.io
● coverity.com
● sourceclear.com
● travis-ci.org (.com for private repos)
Listed in alphabetic order, no preference.

32. Measure execution time and memory
echo "<!-- Measurements: ";
echo memory_get_usage();
echo " - ";
echo (microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"]);
echo " -->";
$ for x in {1..20}
do curl -s http://localhost | grep "Measurements"
done
<!-- Measurements: 5761720 - 0.2341411113739
<!-- Measurements: 5761720 - 0.24964690208435
<!-- Measurements: 5761704 - 0.25908708572388
<!-- Measurements: 5761720 - 0.23540115356445
...

33. Test with dummy data
● While developing a site, load lots of dummy data into it so
you can test how your site looks and performs with 100, 1000
or 100 000 posts.
● Basic: Import themeunittestdata.wordpress.xml
○ codex.wordpress.org/Theme_Unit_Test
● More data: wp post generate
○ curl http://loripsum.net/api/5 |
wp post generate --post_content --count=10
● More realism: wp-cli-fixtures
○ github.com/nlemoine/wp-cli-fixtures

34. WordPress plugins have
a reputation of low
quality. Help us prove
them wrong. Start using
automatic quality testing!

35. Hopefully automatic
quality testing will
be integrated into the
WordPress.org plugin
directory in the future.
See make.wordpress.org/tide

36. Extra tip:
use the WP plugin
boiler plate to start
with: wppb.io
WP Theme starter example:
github.com/aucor/aucor-starter

37. This presentation was about plugin and theme
development. How about testing a real
WordPress site to ensure updates don’t break it?
Let Seravo handle updates of production sites
for you. See our hosting and upkeep service at
Seravo.com

38. THANK YOU!
KIITOS!
@Seravo
@SeravoFi
@ottokekalainen