1. LibrePlan Software
Quality attributes
Javier Morán Rúa <jmoran@igalia.com>
November 2012
2. Software quality
● When has a software quality? Two types of quality:
● Functional quality: How well a program conforms to the
requirements (functional requirements).
– Measured with tests.
● Non-functional quality: Structural quality and other things
not related with the program behavior. Examples:
Documentation, Internationalization, Scalability,
Robustness, Usability, Testability ...
– Analyzed with analsys of code and also with tests.
www.libreplan.com
3. Software tests
● Difference between failure/error/malfunction:
● Error/Defect: Wrong or missing function in the
code
● Failure: is the result of executing a software when
an error happens.
● Malfunction. The system does not do the
expected in the specification.
www.libreplan.com
4. Software tests
● Testing definition: Execute the software with the
target of finding errors and malfunctions.
● Does a software with tests has more structural and
functional quality on average? Yes
● Guarantee there are not errors/malfunctions.
● Measure and know if you meet the expectations.
www.libreplan.com
5. Software tests
● With tests you do two processes: verification and
validation.
● Verification.
● Is the product right?
● checks that the software is working correctly.
Assures that implementation is right for an
specification.
www.libreplan.com
6. Software tests
● Validation.
● Are we building the right product?
● checks if user requirements and the results got
match. Reviews if the system is the correct one.
www.libreplan.com
7. Software tests
● Types of tests – Classification 1:
● Black Box: testing interaction is based on inputs and
outputs.
● White Box: centered in testing the logic of the software and
its structure
– All sentences are executed (coverage)
– All conditions are checked
– All loops are checked
– All paths are checked
www.libreplan.com
8. Software tests
● Types of tests – Classification 2:
● Unit tests. Isolated modules or classes. (verification)
● Integration tests. Interaction of modules and classes
(verification)
● System tests. Hardware and software (verification,
validation)
● Acceptance test. User validates against user
requirements (validation)
www.libreplan.com
9. Software tests
● Types of tests – Classification 2
● Regression test. Assurance of changes. (verification)
● Stress test. Performance. (verification, validation)
● Usability test. Human interaction. (verification, validation)
● Functional GUI testing. Tests interface and can be a type of
validation tests. (validation)
● Validation tests. Against software requirements (validation).
www.libreplan.com
10. Software tests
● Types of tests – Classification 3
● Manual. Perform by a person.
● Automatic. Perform by another program.
www.libreplan.com
11. Software tests
● LibrePlan has white box and black box tests.
● LibrePlan has unit tests,integration tests, regression
tests, acceptance tests, functional GUI tests, stress
tests, validation tests, acceptance tests
● LibrePlan has not systematized usability tests, nor
systematized system tests.
● LibrePlan has manual and automatic tests.
www.libreplan.com
12. Unit and Integration tests
● Benefits:
● Increase code coverage
● Increase team productivity
● Detect regressions and limit debugging
● Improve implementation
● Document expected behavior
www.libreplan.com
13. Unit and Integration tests
● LibrePlan uses JUnit and SpringTestContext to do integration
and unit tests.
● JUnit is the de facto standard for unit testing in Java platform.
● It was created by Kent Beck and Erich Gamma
● Kent Beck. Creator of Extreme Programming and Test Driven
Methodologies.
● Erich Gamma. One of the authors of Design Patterns: Elements
of Reusable Object-Oriented Software.
www.libreplan.com
14. Unit and Integration tests
● SpringTestContext is a module of Spring. In
LibrePlan we use to simplify:
● Management of Hibernate sessions.
● Automatic management of transactions. Rollback
after each test.
● Integration with Spring. We have access by DI to
the repositories.
www.libreplan.com
15. Unit and Integration tests
● In LibrePlan we have a lot of unit and integration
tests:
● org.libreplan.ganttzk : 306 tests
● org.libreplan.business : 1884 tests
● org.libreplan.web: 400 tests
www.libreplan.com
16. Test coverage
● Definition: It describes the degree to which the
source code of a program has been tested.
● In which does it help?
● To know the untested parts of the codebase.
● In LibrePlan we measure the code coverage of our
unit and integration tests.
www.libreplan.com
17. Test coverage
● In LibrePlan we use Cobertura for measuring the code
coverage of the unit and integration tests.
● Cobertura features:
● Generates reports in HTML and XML.
● Does several code coverage measurement types:
– Statement coverage. % of lines executed.
– Branch coverage. % of total branches tested.
– Other aggregations: % of classes, files, methods
www.libreplan.com
18. Test coverage
● Current test coverage LibrePlan measurements:
● Whole project:
– Statement coverage: 12%
– Branch coverage: 15%
– Files: 50% of files with some test.
– Methods: 20% of methods
– Classes: 29% of classes
www.libreplan.com
19. Test coverage
● Current test coverage LibrePlan measurements:
– Statement coverage: 12%
– Branch coverage: 15%
– Files: 50% of files with some test.
– Methods: 20% of methods
– Classes: 29% of classes
● These numbers are much better in the domain layer
and persistence layer.
www.libreplan.com
20. Web UI tests
● Web UI testing is a type of GUI software testing for
applications with web interface.
● GUI software testing: Type of tests which use the GUI
of a program to perform the sequence of steps of the
use cases of the application. In this way it is tested:
● That the program fulfills the requirements
● That the GUI has the way of interaction as
designed.
www.libreplan.com
21. Web UI tests
● Web UI testing can be:
● Automatic. Advantages:
– Repeatability.
– Speed of execution.
● Manual
● Automatic UI testing mimics the actions of a user
through the interface
www.libreplan.com
22. Web UI tests
● Web UI testing is a black box technique with a lower
coverage than integration and unit tests.
● To automate or not automate ?
● Advantages: Frequent regression testing, rapid
feedback for developers, unlimited execution of
tests cases, support extreme and agile
technologies, discipline in the documentation of
test cases
www.libreplan.com
23. Web UI tests
● To automate or not automate ?
● Disadvantages:
– Tight deadline. Short time
– Interface changes a lot
www.libreplan.com
24. Web UI tests
● LibrePlan has automatic Web UI tests.
● It has been a challenge to find a framework to make UI tests
with LibrePlan.
● We wanted a tool that allowed to test LibrePlan in several
browsers.
● LibrePlan is a RIA application with a lot of JS, HTML5,
CSS (some of the latest standards in web)
● With ZK we have to deal with an generated id HTML
attribute for each page rendering
www.libreplan.com
25. Sahi
● In LibrePlan we use Sahi as framework to build the
automatic web UI tests.
● Sahi aims to make easy test automation in web
technologies addressing most of the problems and
limitations in this area.
● Advantages over Selenium: recorder is browser
independent, waits for AJAX and page loads, it does not
use XPath intensively, works well with dynamic ids,
frames, iframes, popups.
www.libreplan.com
27. Sahi
● Explaining the architecture:
● Need to simulate browser events. The way of Sahi:
Use JS injected to do it. Why JS?
– Support to access the DOM.
– Runs in the browsers.
● The injected JS is generated and inserted by the
Sahi proxy.
www.libreplan.com
28. Sahi
● Explaining the architecture:
● Role of the proxy server:
– Recording sessions.
– Does playback of sessions.
– Create reports, save data to database,...
● It is used Sahi scripting (an extended JS to define the
tests)
www.libreplan.com
29. Sahi
● Programming the tests in Sahi script has the next
advantages:
● It is a programming language. Nothing better than
the power of programming to define tests.
● The best API to interact with the DOM.
● Sahi has a powerful human relative API to locate
things: _in, _near, _under, _table ...
www.libreplan.com
30. i18n
● Internationalization (i18n): It is the process of designing an
application so that it can be adapted to different languages and
regions without engineering changes.
● Characteristics of an internationalized software:
● You can add localized data and run the same executable.
● Textual elements are not hardcoded
● Support for cultural dependent data, such as currencies and
dates.
www.libreplan.com
31. i18n
● Localization: It is the process of adapting the
software for a specific language or region (translating
and adding specific UI components if needed).
● LibrePlan is an internationalized software:
● Currently present in English, Spanish, Galician,
Italian, French, Dutch, Catalan 100%
● Almost fully translated: Czech, German
● On-going translations: Polish, Portuguese, Russian
www.libreplan.com
32. Stress tests
● Stress tests in software: Tests to put the system to
test under heavy load to know the behavior of the
system regarding to availability, robustness and error
handling.
● In LibrePlan we do stress tests sometimes when we
want to:
● Know the best alternative to implement a feature.
● The response times under different load conditions.
www.libreplan.com
33. Stress tests
● At LibrePlan we use the tool JMeter to do stress tests.
● JMeter is a free software application designed initially
to do stress load tests of web applications.
● Now it can test several server types:
– Web: HTTP, HTTPS
– SOAP
– Database via JDBC, LDAP, JMS,...
www.libreplan.com
34. Stress tests
● Example of task where we used stress tests: To
appraise if the activation of the Hibernate second-
level cache was worth.
– http://wiki.libreplan.org/twiki/bin/view/LibrePlan/ItEr
● Results analysis:
● A 75% of gain in the maximum response time and
second-level cache was activated.
www.libreplan.com
35. i18n
● In Java i18n is provided in the Java SE.
● Internationalization done with Java SE:
● Locale class
● ResourceBoundles.
● Property files
● In LibrePlan we do not use the standard Java i18n
facilities.
www.libreplan.com
36. i18n
● In LibrePlan we use GNU Gettext
● GNU Gettext. Toolset to provide a framework to help
to internationalize other programs. The tools include:
● A set of conventions about how programs should be
written to support message catalogs.
● A directory and file name organization for the
message catalogs themselves.
● A runtime library to get internationalized messages.
www.libreplan.com
37. i18n
● GNU Gettext. Toolset to provide a framework to help
to internationalize other programs. The tools include:
● A few stand alone programs to help translators with
the translation of messages.
● A library supporting the parsing of source code and
the creation of files containing messages to
translate (POT files).
www.libreplan.com
38. i18n
● Advantages of using GNU Gettext in LibrePlan:
● Programmers have not to be worried of adding
explicitly messages to property files with ids.
● The generation of the messages to translate is
automatic.
www.libreplan.com
39. i18n
● LibrePlan uses a online internationalization tool
called Transifex, that has helped to increase the
number of languages in which LibrePlan is available.
● Transifex is free for open source projects.
www.libreplan.com
40. Continuous Integration
● Definition: It is the process of frequently integrating one's
new or changed code with the existing repository.
● It is one of the practices of XP (Extreme Programming).
● Advantages: detect and fix integration problems
continuously, immediate unit testing of all changes,
promote frequent check-in pushes developers doing less
complex code.
www.libreplan.com
41. Continuous Integration
● Effective practices to achieve CI:
● Maintain a single source repository.
● Automate the build.
● Make your build self-testing.
● Everyone commits to the mainline daily.
● Every commit should build the commit in an
integration machine
www.libreplan.com
42. Continuous Integration
● Effective practices to achieve CI:
● Keep the build fast.
● Test in a clone of the production environment.
● Make it easy for everyone get the latest
executable.
● Everyone can see what is happening.
● Automate deployment.
www.libreplan.com
43. Continuous Integration
● Continuous Integration can be set up using a
Continuous Integration server.
● CI server definition:Tool that lets the user plan
automatic builds:
● Periodically.
● Triggered after each commit.
www.libreplan.com
44. LibrePlan CI server
● LibrePlan developers use continuous integration and
use as continous integration server Jenkins
● LibrePlan continuous integration server is publicly
accessible: http://jenkins.libreplan.org
● Two automatic builds:
● LibrePlan Postgresql
● LibrePlan MySQL
www.libreplan.com
45. LibrePlan CI server
● Jenkins configuration at LibrePlan:
● The codebase that is built is master branch
(unstable branch that originates new releases).
● The software is built once per day (2:00 AM) if
during the last 24 hours some new changes has
been pushed into the master branch.
– If the build fails an e-mail is sent to libreplan-dev
mailing list.
www.libreplan.com
46. LibrePlan CI server
● After build the unit and integration tests are passed.
www.libreplan.com
47. LibrePlan CI server
● Test code coverage is analyzed with Cobertura.
www.libreplan.com
48. LibrePlan CI server
● If some unit or integration tests fails, then an automated mail
is sent to libreplan-dev mailing list.
● As soon as a developer sees a fail (because a software build
crash or some tests not passing), this gets maximum priority
and managed the fixing.
● Doing it by himself if he is able to do.
● Looking for help and entrusting the responsible to fix it.
● Many times the first thing done is to revert the conflicting
push and set Jenkins back to normal.
www.libreplan.com
49. LibrePlan CI server
● Currently in LibrePlan the Sahi automate web UI
tests are not integrated with Jenkins in the build
cycle.
● Web UI tests are passed manually always before
each release as part of the release process.
● Other occasions too.
www.libreplan.com
50. Sonar
● Definition: It is an open source web platform to
manage code quality.
● It is helped by other code static analysis tools like
CheckStyle, FindBugs, ...
● Languages covered: Java, C, C#, Natural, PHP,
Cobol and Visual Basic.
www.libreplan.com
51. Sonar
● It measures the quality in 7 axis:
● Comments
● Coding rules
● Potential bugs.
● Complexity.
● Unit tests.
● Duplications.
www.libreplan.com
52. Sonar
● In LibrePlan we use Sonar to control and monitor the
quality of the source code.
● LibrePlan Sonar installation is publicly available:
http://sonar.libreplan.org
www.libreplan.com
53. Sonar
● Size metrics:
● Lines of code: Physical lines.
● Comment lines: Number of comments.
● Density of comment lines.
● Number of packages, number of methods.
● Number of accessors (getter and setters)
www.libreplan.com
54. Sonar
● Tests metrics:
● Number of tests.
● Number of failures.
● Test success density.
● Code coverage:
– Line coverage
– Branch coverage
www.libreplan.com
55. Sonar
● Duplication metrics:
● Duplicated lines: Number of lines touched by a
duplication.
● Duplicated blocks: Number of duplicated blocks of
lines.
● Duplicated files: Number of files involved in a
duplication.
● Duplication density: Percentage of duplicated lines.
www.libreplan.com
56. Sonar
● Complexity metrics:
● Cyclotomatic complexity (McCabe Metric).
● Method distribution complexity.
● Average class complexity.
● Average file complexity.
www.libreplan.com
57. Sonar
● Violations metrics:
● Total number of violations
● Weighted violations = Sum of the violations
weighted by a coefficient of severity.
● Rule Compliance Index (RCI) = 100 – (weighted
violations/Number of Lines of Code)*100
www.libreplan.com
58. Sonar
● LibrePlan uses Sonar to monitor the quality of the
project.
● If there are blocker violations, these are solved.
● LibrePlan Sonar installation: http://sonar.libreplan.org
www.libreplan.com
