Psi multi accessgateway_casestudy

PrimeSoft Solutions, Inc.

Multi Access Gateway: Case Study

August 26, 2010

This document contains information proprietary to PrimeSoft Solutions, Inc.
Reproduction, release, or use of the information contained in this document is
strictly prohibited without prior written permission from PrimeSoft Solutions,
Inc.

Multi Access Gateway : Case Study

Multi Access Gateway

Client Offshore Development Center (ODC)

This Santa Clara, CA, company develops and markets industry's first multi-access
convergence gateway designed for stateful management of tens of thousands of secure,
reliable, high-quality subscriber sessions over multiple network access types including Wi-
Fi and WiMAX.

To date, the company has raised over $50M and is backed by Sequoia Capital and Kleiner
Perkins among other venture capital firms.

Relationship Model
PrimeSoft is the chosen Outsourced Product Development partner for the client. PrimeSoft
developed this relationship by providing the following at our offshore development center

1. Building testing, automation and development teams
2. Identifying an Project Manager
3. Setting a Lab and setting it up with Client specific software and hardware
components
4. Providing supported from HR, Admin and Facilities Management functions
The offshore team consists of 18 engineering resources divided into QA, Automation and
Development. The teams function as an extension to the client’s team in USA, and
participate in various projects during a release.

Weekly status reports are provided by the Project Manager on all active offshore projects,
followed by conference calls to discuss any open issues.

UMA Handset Simulator
Technology Brief
Unlicensed Mobile Access (UMA) technology provides access to GSM and GPRS mobile
services over unlicensed spectrum technologies, including Bluetooth and 802.11. By
deploying UMA technology, service providers can enable subscribers to roam and perform
handover between cellular networks and unlicensed wireless networks using dual-mode
mobile handsets. With UMA, subscribers receive a consistent user experience for their
mobile voice and data services as they transition between networks.

Project Objectives
The objective of this project was to develop a UMA Handset Simulator. UMA simulator
came out as a requirement from the development and automation engineers to help them
perform unit testing and automation of UMA functionality.

PrimeSoft Confidential Page 2 of 11


UMA simulator was required support functional, to help automate more than 90% of all
functional tests and help development engineers run pre-commit unit tests to ensure basic
functionality.



Our Approach
The Team

PrimeSoft identified a technical lead, development & test engineers to staff this project.
The team had prior experience in working with telecom and data communications
protocols and call processing.

Project Execution

Requirements Analysis
A team from PrimeSoft traveled to US client site for gathering requirements. The
team had brainstorming sessions with client’s technical team and end users of the
UMA simulator.

The requirements were documented in a Functional Requirements Document
Template and shared with all the stake holders, followed by a formal review.
Review comments were incorporated and the updated document was checked into
CVS and posted on internal Wiki.

Functional Specifications & Software Design
Functional Requirements Document was an input for developing a Functional
Specifications Document. The HLD (High Level Design) and LLD (Low Level
Design) documents were developed based on the functional specifications. The
design document identified modules, interfaces, and presented pseudo code.

Coding, Unit Testing & Acceptance Testing
Coding of the tool involved setting up the environment and developing code based
on the low level design. The code was unit tested by development engineers.

System testing of the simulator was carried out by an independent team that
developed system test plan document based on the requirements document and
executed multiple test cycles and logged bugs using Bugzilla.

User acceptance test plan was developed by PrimeSoft QA team and shared with
the client. Acceptance testing was performed jointly by PrimeSoft and client
teams.

Salient Features

Object oriented Design approach provides well defined programming interfaces to
the scripting layer
Provides interfaces to build and parse and UMA and GSM Layer3 messages
Multithreaded implementation for RTP sends and receives for multiple handset
instances
Python extensions provided for abstracting the complex scripting functionality and
reduce scripting effort
Supports complex handover scenarios by simulating two UMA connections from a
single Handset



Project Deliverables
Deliverable Details
Functional Requiremnts Document
Functional Specifications Document
Documents Design Document
Users Guide
Release Notes
UMA Simulator Source Code (C/C++, Makefiles, Shell
Source Code
Scripts)
Unit Test Plan
Test Documents
Unit Test Matrix
Bugs BUGS Logged into Bugzilla
Sample Scripts Sample Automated Scripts
Users Guide Automation suite and scripts users guide

Traces all the requirements against Functional
Requirements Traceability
Specifications, Design, System Testing, Unit Testing
Matrix
and Users Guide

Tools & Technologies
Title Details
Protocols UMA, GSM, TCP/IP
Operating Systems Redhat Linux
Programming & Scripting C, C++, Python, Shell Scripts, Makefiles
Configuration Management CVS



Multi Access Gateway Testing
PrimeSoft houses the largest testing team for our client. Our testing team is involved in the
entire test life cycle of the product including test strategy development, test plans
development, tests cases development, manual testing, filing bugs and running regression
cycles.

Project Objectives
The main objective of the testing group is to work as an extension to client’s onsite
engineering teams, and be responsible for test and automation activities involved in the
project.

The offshore team is responsible of the following project activities
1. Defining QA and Automation Strategy for a identified release
2. Developing QA plan for the identified release
3. Developing Test Plans and Test Case documents based on
a. Marketing and Product Requirements Documents
b. Functional Specifications
c. Design Documents
d. RFC’s Supported
4. Performing Functional, System, Sanity, Regression, Load and Stress, Security and
Interop testing
5. Identifying and logging bugs, coordinating with developers in re-producing and
analyzing bugs
6. Developing test matrix and reporting on manual tests progress on a daily/weekly
basis

Testing Projects
PrimeSoft team worked with client on multiple releases of this product and was involved in
testing and owning QA and Automation for the following modules

 IPSec IKE v2, IKEv1
 MOBIKE (Mobile IKE)
 RIP/OSPF
 VLAN (Virtual LAN)
 Session Initiation Protocol (SIP)
 QoS (Quality of Service)
 AAA (Authentication, Authorization and Accounting) and RADIUS
 Change of Authorization (CoA)
 Internet Protocol Slow Path



 Point to Point over Ethernet (PPPoE)
 Dynamic Host Control Protocol (DHCP)
 PDIF (Protocol Data Interworking Function)
 CDR (Call Detail Records)
 IP-IP Encapsulation
 DOS Attacks
 ACL (Access Control List)
 SNMP (Simple Network Management Protocol)
 RTC (Real Time Charging)
 HA (High Availability)
 System Testing
 Performance Testing

Our Approach
The Team
We created a dedicated test team who were technically competent in networking
technologies. Project Manager and a core group of engineers started the project by
performing manual testing on one of the releases and logged bugs. The team developed
its competency on the product by organizing
Internal training seminars
Technical hands on sessions
Developing internal wiki pages
Going through past bugs reports
Communicating with developers and PLM teams
Test Beds

A Site-to-Site VPN was setup between PrimeSoft, and Client location in USA, allowing
engineers to reserve, configure and access remote test setups.

PrimeSoft also has a lab setup with client specific software and hardware available locally
for engineers.

Project Execution

Test Projects execution followed the following phases.

Analysis
The aim of this phase was to technically be competent in the technology and
enhance product understanding and gather all the information necessary to
develop a test plan document and completely define the scope of testing activities.

The team initiated the projects by going through RFC’s and product documentation
including, Requirements, Functional Specifications and High Level Design. Email



exchanges and conference calls were conducted to understand the nuances of
implementation on client’s product.

Test Plan Development
Test plan development involved, identification and documenting

1. Test Topology
2. Software and Hardware Tools
3. Entry and Exit Criteria
4. Conformance scenarios from RFC’s
5. Test cases against Requirements
6. CLI Related test cases
7. Functional test cases
8. Negative test cases
9. End-to-End tests
10. SNMP related test cases
11. High availability test cases

Internal & External Reviews
Internal peer reviews (board review) is performed and review comments are
updated into the document. An internal review checklist is used to ensure the
quality of the test plans is consistent.

The update document is sent out for an external/client review followed by a walk
through of the test plan through a conference call. Review comments received
from architects, designers, implementers and PLM are consolidated and updated
into the document.

Updated document is checked into CVS and posted on internal Wiki.

Manual Test Cycle
Manual test cycles involved setting up test bed and executing test cases against
the SUT (System Under Test) and tracking progress.

A Test Matrix is filled for each cycle of manual test cycles across multiple builds,
the test matrix captures testing results documented as PASS, FAIL or BLOCKED.
The matrix also provides key metrics of the testing cycle including testing effort,
bugs logged and passes percentage.

Bugs were logged into client maintained bugzilla, with information including
1. Bug description
2. Build version
3. Steps to reproduce



4. Logs and core files path
5. Initial diagnosis findings
6. Observation and Comments

Title Detail
Protocols IPSec, IKE v1/v2, AAA, CoA, PPPoE, DHCP, SIP, SNMP

Operating Systems REDHAT Linux, QNX
Scripting Python, Shell Scripts
Ethereal (Network Analyzer), Scapy (Packet Generator),
Tools Silvercreek, SIPp, QA Robot, Smartbits, Landslide
Avalanche, IXIA, IKEv1/v2 clients, DOS Attacks tools
Configuration Management CVS, Bugzilla


Deliverable Details

Test Plan Test Plan Document

Test Matrix Test Cycle Summary in a Test Matrix Document

Bugs BUGS Logged into Bugzilla

Checklist Test Plan Release Checklist
Requirements Traceability Traces all the requirements against the test cases and
Matrix bugs

Multi Access Gateway Automation
With a growing feature list in the product we sent out with a target of automating 50% of
our test cases, and this could significantly reduce the manual test cycle timeline.

PrimeSoft was involved in test automation from the very beginning and participated in
developing an automation framework, developing automated test scripts and suites.

Project Objectives
The objective of the automation group at PrimeSoft is to
Design, Develop and Enhance an Object Oriented Automated Framework
Develop automated test scripts
Run automated regression on daily/weekly builds



Develop test suites for pre-commit, sanity, weekly regression and feature specific
suites
Identify and log bugs
Fix scripts and maintain them across release and builds

Our Approach
The Team

We created a dedicated automation group within PrimeSoft who were technically
competent in scripting languages and had experience with relevant networking
technologies.

Framework Enhancements
Our team participated in design and development activities of the automation
framework. An object oriented framework was designed using Python as the
scripting language.

Enhancements to the framework were performed and tested (using automated unit
tests); and the framework was released to carry out the development of the scripts.

Automated Scripts
A team of automation engineers was selected to automate the test plans using the
enhanced automation framework. Automation engineers focused on developing
automated scripts and scripts development involved the following scripts
Manual testing
Develop and automated test script
Running the test on a stable build to verify the behavior of the script
Fixing all issues with the scripts
Adding the script in a suite and running the suite

Every developed script went through a through review with at least two reviewers
and focused on functionality and coding guidelines.
Functional Review: Review focused on the functionality of the scripts and
compared them with what is described in the test cases
Coding Guidelines Review: The review focused on usability, maintainability
and adherence to coding guidelines
Checked into CVS

Project Salient Features
Python based Object Oriented Automation Framework
o Can load/clear and check configuration on system under test
o Can load/clear and check configuration on Juniper Netscreen hardware
o Can initiate IKE v1 and v2 sessions from end clients
o Can initiate PPPoE sessions from Linux based end clients
o Can handle High Availability cases
Exposes library functions for script developers



Automated Tests
o Topology File: Can be changed to point to a new topology
o Configuration File: Captures all common configuration for a test
o Suite file: Can include multiple test cases to be run in a suite

Title Detail
SIP, SDP, RTP
Protocols IPSec IKE v1/v2
RIP, OSPF, BGP, DHCP, DNS, Radius, TCP/IP, PPPoE
Clients Hardware/Software Netscreen 5GT, XpressVPN, PPPoE Linux Client, SIPp
Used test tool, Cisco Terminal Server
Operating Systems Redhat Linux, QNX
Programming & Scripting Python, Perl, Shell Scripts
Configuration Management CVS, Bugzilla


Deliverable Details
Automated Scripts Developed as part of each
Scripts
functionality
Test Suits Automated test suites for various types of testing

Framework Libraries Framework Libraries and Documentation

Checklist Scripts Development and Review Checklist
Requirements Traceability Traces all the requirements against the test cases and
Matrix bugs


Psi multi accessgateway_casestudy

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Psi multi accessgateway_casestudy

Similaire à Psi multi accessgateway_casestudy (20)

Dernier

Dernier (20)

Psi multi accessgateway_casestudy