Raniero Virgilio

1. Jython for Embedded
Software Validation
Raniero Virgilio
IAG/ECG/PPD SW
Pycon Quattro, 8/5/2010
1

2. Raniero Virgilio
My Job
• Software Engineer in Intel Shannon, Ireland
• Responsible for Test Automation infrastructure
My Team
• PPD (Performance Product Division)
Embedded Software
• Complete SOCs for security, communications,
storage and embedded designs
• http://www.intel.com/embedded/index.htm
Intel Architecture Group
2

3. Agenda
• Validation in embedded systems
• Runtime Plug-in model
• Runtime Plug-in model with Jython
• Jython and Eclipse
• Results
Intel Architecture Group
3

4. Validation in
embedded systems
• A challenging environment
• Typical scenario
• Requirements and solutions
Intel Architecture Group
4

5. A challenging environment
Multiple hw/sw deployments
• Configuration set up needs to be fast and reliable
• Support for different OS types and versions
Consistent cooperation
• Several teams working on the same infrastructure
• Synchronization with overseas groups
Intel Architecture Group
5

6. Test Automation scenario
XML-RPC session
Server1 (Windows) Traffic Generator 1
XML-RPC session
Server2 (Windows) Traffic Generator 2
Telnet session
Test DUT1
Test Execution
XML-RPC Driver
Reports (Linux/Windows/FreeBSD)
session
Telnet session
DUT2
(Linux/Windows/FreeBSD)
Telnet session
DUT3
(Linux/Windows/FreeBSD)
Intel Architecture Group
6

7. Test Automation requirements
Programming language
• Platform-independent
• Libraries for connectivity and string manipulation
Test Framework design
• OOP to maximize reusability of test code
• Platform runtime extendibility
Sustainable implementation
• An effective interface between test execution
(local to DUT) and test reporting (distributed)
• Customizable open source solution
Intel Architecture Group
7

8. Proposed solution
Programming language
• Python
– telnetlib, xmlrpclib, SimpleXMLRPCServer
– re, xml.dom.minidom
Test Framework design
• Runtime plug-in model
Sustainable implementation
• Jython embedded in Eclipse
– Jython is a Python interpreter written in 100% pure Java
Intel Architecture Group
8

9. Runtime plug-in
model
• Definition
• Design Pattern
Intel Architecture Group
9

10. Runtime Plug-in Model
The term plug-in refers to a type of program
that contributes code to the system and
adds a special capability to it.
• The plug-in is unknown at compile-time of the
application for which the plug-in is designed.
• Plug-ins are dynamically plugged into the
application they are designed for at runtime
• The key for this model is Dynamic Code Loading
Intel Architecture Group
10

11. Design Pattern
• Extendibility
– Application capabilities are defined
by the supported Plug-in
– Plug-in can be loaded and unloaded
at runtime
• Plug-in development
– Each Plug-in shall implement a
specified interface
– The plug-in delegates the
Application for high-level services
• Parallel life-cycle
– Application and Plug-ins can be
developed independently
Intel Architecture Group
11

12. Runtime Plug-in
Model with Jython
• Dynamic code loading in
Java and Jython
• Multithreading
Intel Architecture Group
12

13. Dynamic code loading in Java
• Runtime source compilation
– StandardJavaFileManager handles the
source code
– JavaCompiler executes the compilation
and creates a class file
– DiagnosticCollector gathers compilation
information
• Class file deployment
– A new Classloader has to be defined
– It need to extend the previous one with the new class file
Intel Architecture Group
13

14. Dynamic code loading in Jython
• One step task
– The PluginHandler instantiates one PythonInterpreter
– The Plug-in path is added in the PythonInterpreter sys.path
– The Plug-in is imported in the embedded Python environment
• Lower complexity
– Implementation is much easier
– Code can be modified “on-the-fly” and reloaded
– First time Plug-in loading is usually quicker on Jython
Intel Architecture Group
14

15. Multithreaded environment
• Parallel PluginHandlers
– Independent Jython environments’
setups
– Each PluginHandler defines an
autonomous environment
Java Thread
• Multiple task execution
– When PluginHandler invokes a run() the execution is
encapsulated in a Java Thread
– Plug-ins expand the application concurrently
Intel Architecture Group
15

16. Jython and Eclipse
• XTP architecture
• The Execution Engines
• The JythonExchanger
Intel Architecture Group
16

17. The eXtendible Test Platform (XTP)
TestCase
Code TestCase
Docs
Java
Jython
Traffic
Generator A
C Files
DUT A DUT B DUT C DUT D
Intel Architecture Group
17

18. The Execution Engine in XTP
DUT A DUT B
• XTP 28.08.09
– provides high-level test services (logging, reporting,
multithreading) and an interface with frontend
• Execution Engine
– a self-contained software package, written in Python, that
implements the actual test logic and expands XTP enabling
communication of the test platform with DUTs, servers and test
equipments
Intel Architecture Group
18

19. Plugging an Execution Engine
DUT A DUT B
– A PythonInterpreter is created by XTP
28.08.09
– The activation sequences contained in the EE Manifest are
executed in the PythonInterpreter
– An instance of the (Java) JythonInterpreter class is created
by XTP and deployed into the relevant PythonInterpreter
Intel Architecture Group
19

20. The jythonExchanger
– The jythonExchanger is the interface between the EE and the
XTP environment
– Each EE uses it to access XTP high-level features
– XTP infrastructure uses it to retrieve test results from EEs and
show them in graphs and reports
– The overall consistency of the system is assured by the
Jython characteristic to be a 100% pure Java implementation.
Intel Architecture Group
20

21. Results
• Main benefits
• Common risks
Intel Architecture Group
21

22. Sustainable architecture
40 40
39 39
38 38
Java Heap Space
Java Heap Space
37 37
36 36
35 35
34 34
33 33
32 32
0 1 2 3 4 5 5 4 3 2 1 0
EEs Number EEs Number
• Example: load/unload 5 plug-ins at runtime
– Memory request increases without “exploding”
– Memory is freed when plug-ins are deactivated
Intel Architecture Group
22

23. Major benefits
“Calibrate your technology”
• Choose the solution that best fits your needs
– Python threads or Java threads (or Eclipse Jobs)?
• Transparent Python layer on DUTs
– XML-RPC Server + os.popen() have small CPU overhead
• Plug-in lifecycle
– Runtime plug-ins promote validation through cooperation
• Short rump-up time
– Perl users start using Python very intuitively
Intel Architecture Group
23

24. Common risks
“Design your architecture”
• Test planning
– To make the most of the Plug-in design pattern, point out
common test domains in advance
• Long-term investment
– Platform stability requires a significant effort
• Keep code under control
– Review carefully test code before integrating it
• Right-Size your hardware
– Concurrent executions may require performing machines
Intel Architecture Group
24

25. Q&A
(raniero.virgilio@intel.com)
Intel Architecture Group
25