This document discusses Orange Polska's use of function point analysis (FPA) for estimating the size, effort, and pricing of IT projects, as well as their use of automated code review. It describes how Orange Polska counts over 1 million function points annually across 800 projects in many technologies. It also explains their four-step process for converting function points to price, including custom adjustment rules. Additionally, it outlines the quality metrics and tools used for automated code review of vendors' source code.
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
Automated Code Review and Function Point Analysis for IT Projects
1. 1
Orange Polska for IWSM Mensura, Cracow 2015
Orange Polska
IWSM Mensura 2015
Practical usage of FPA
and automated code
review
Piotr Popowski & Bogusz Jeliński
IT Applications
2. 2
Orange Polska for IWSM Mensura, Cracow 2015
What we measure, count or look for in Orange
Function Points
Manual, IFPUG-based counting for pricing
Quality of code (static review)
Violations of patterns
Weighted average of low-level grades
Defects per FP
Software process
maturity, repository, continuous integration
Automated FP
Complete source code
3. 3
Orange Polska for IWSM Mensura, Cracow 2015
Measurement in figures
MANY TECHNOLOGIES
JAVA, C/C++, PHP, PL/SQL, WEB METHODS, PEOPLE SOFT, POWER
BUILDER, ABAP, ORACLE FORMS, etc.
1 000 000
FP COUNTED
800
PROJECTS
(GO-LIVES)
1 500
CODE AUDITS
7
DOMAINS
5
VENDORS
4 800
ENHANCEMENT COUNTS (incl. CRs)
200
IT SYSTEMS
5 YEARS
EXPERIENCE
4. 4
Orange Polska for IWSM Mensura, Cracow 2015
Part 1
Estimating size and price for IT projects
using FPA method
Piotr Popowski – IT Applications, Orange Polska
Contact: piotr.popowski@orange.com
5. 5
Orange Polska for IWSM Mensura, Cracow 2015
Context
In 2010 Orange Polska transformed its sourcing strategy from multi-vendor to
consolidated vendor agreements. In return for a long-term contract, the
company offered its suppliers exclusive rights to deliver software changes in
particular IT areas.
Vendors
consolidation
Solution
Challenge Lack of free market pricing mechanisms, as implied by the contracts, imposed
the necessity to implement a dedicated sourcing and pricing method.
To meet the new challenges Orange Polska adopted a new method of
estimating IT development size, workload and finally, pricing.
6. 6
Orange Polska for IWSM Mensura, Cracow 2015
From FP to price
In four steps
FP and SNAP count based
on IFPUG and custom
rules
FP adjustment
Adjusting number of FP.
Impact ratios based on
COCOMO II
effort calculation
Contract-based
productivity ratios related
to vendor, system and
technology
price
Price calculation using
rates defined in contracts
FP
SP
Calculated by Orange and
vendor analysts
Verified and authorized by OPL
IFPUG certified team
Orange and vendor
analysts
Sourcing division Sourcing division
7. 7
Orange Polska for IWSM Mensura, Cracow 2015
Customization
Custom rules
The custom rules cover the areas where IFPUG FPA proved to be in some way deficient
Custom rules
5% of FPs is „Rule 27” algorithmic complexity - internal data processing where data does not cross application boundary
7% of FPs is „Rule 30” reusability – a reusable component, where development is made once but analysis and tests are conducted
for many elementary processes
20% of FPs – dedicated rules for 26 systems in the following areas
Integration
Business Intelligence (ex. Data Warehousing)
Wholesale
8. 8
Orange Polska for IWSM Mensura, Cracow 2015
Customization
FP adjusting
The main goal of adjusting is to assess effort
Adjustation is made separately for each BFC (Base Functional Component)
Impact ratios are defined based on COCOMO II Equation 4
Impacts are expert-estimated and mutually agreed
Also using NESMA impacts as reference
Creation or huge change Moderate change Small change Suppression or COTS
Effort
9. 9
Orange Polska for IWSM Mensura, Cracow 2015
Customization
Productivity
Productivity ratios were established during vendor selection process
COCOMO II factors were used as reference for determining productivity
Productivity
Technology System Vendor
Year of
contract
10. 10
Orange Polska for IWSM Mensura, Cracow 2015
Knowledge base and tools
FP calculation
CARTO – Excel spreadsheet for doing detailed calculations based
on HLD
Full decomposition: project > system > functionality > BFC
(or SCU for SNAP)
Additional information for effort calculation (e.g. technology,
impact)
FP adjustment
Document and report
FP Repository – all data from CARTOs recorded in a dedicated system
Over 30 on-line generated reports available from the repository
Progressive build of Application Count
HLD
CARTO
REPOSITORY
REPORTS
11. 11
Orange Polska for IWSM Mensura, Cracow 2015
Common pricing model
Instead of tenders, quotes and negotiations – objective pricing
Budget and capacity early estimation
Based on historical data
Project Roadmap management
Project assignment to particular path depending on path capacity
Path management – schedule and capacity
KPIs
Cost efficiency – spending/FP
Quality – errors/kFP
Process optimization
Better and more precise analytical documents (HLD)
Tracing volume of changes in IT projects (e.g. number of projects, number of change
requests, changing size of average project in particular domain)
Benefits
12. 12
Orange Polska for IWSM Mensura, Cracow 2015
Challenges
Introduce another measuring method in order to verify or improve
current model (e.g. Use Case Points)
Measuring Agile projects
Improving the way to measure configuration changes
Introduce NESMA impacts
13. 13
Orange Polska for IWSM Mensura, Cracow 2015
Orange Polska experience
Since 2010
Has a dedicated, internal, IFPUG certified Metric Team
Uses IFPUG FPA (and SNAP since 2012 ) for pricing IT projects
Uses metrics to manage Project Roadmap and IT budget
Dedicated Metric Team provides the following services
Design and implementation of effort estimation and pricing models for IT contracts
FPA, SNAP and custom trainings
Methodology, contract and project audits
Services are offered internally and externally, both on-site
(currently Warsaw and Cracow) as well as remotely
14. 14
Orange Polska for IWSM Mensura, Cracow 2015
Part 2
Automated code review
dr Bogusz Jeliński – IT Architecture Strategy, Orange Polska
Contact: bogusz.jelinski2@orange.com
15. 15
Orange Polska for IWSM Mensura, Cracow 2015
What we measure, count or look for in Orange
Function Points
Manual, IFPUG-based counting for pricing
Quality of code (static review)
Violations of patterns
Weighted average of low-level grades
Defects per FP
Software process
maturity, repository, continuous integration
Automated FP
Complete source code
16. 16
Orange Polska for IWSM Mensura, Cracow 2015
Code review – history & inspirations
1976 - Fagan’s classic article in IBM Systems Journal
1979 - Lint for Unix V7
1991 - ISO/IEC 9126
1992 - Technical debt metaphor by Ward
Cunningham
90’s - Sun Java Code Conventions
Our own manual review (WebMethods, Drools)
17. 17
Orange Polska for IWSM Mensura, Cracow 2015
Tools used for the static review of source code
pros & cons
Many supported technologies
Tracks cross-layer dependencies
Semi-automated count of function points
Cost of license
Needs extensive training to operate
Not automated (as its manufacturer might
claim)
Cannot be run by any vendor before code
hand-over
CAST
Sonar
No license fee
Integrated with dev tools
Supports few technologies
Some plugins are not open-source
HP Fortify
Security oriented
Only security oriented
Not free of charge
18. 18
Orange Polska for IWSM Mensura, Cracow 2015
Quality metrics
Content of contracts with vendors
Low level rules
Aggregate
Total
Grade
Expected
improvement
< 3.10 0,25
3.1 - 3.5 0,15
> 3.50 0,10
Expected
yearly
improvement
Tree-based aggregation with weights
Values from 1 (bad) to 4 (very good)
19. 19
Orange Polska for IWSM Mensura, Cracow 2015
We prevented the increase of technical debt
Technical
Quality
Index
Discovered acts of sabotage – removing comments to hamper the change of vendor
Created rules for the analyzer to prevent technology-specific anti-patterns
(based on test reports)
Fewer production incidents by 4%
Increase of code performance prevented infrastructure spending
Benefits
20. 20
Orange Polska for IWSM Mensura, Cracow 2015
Challenges
We found that vendors differ vastly
in terms of delivered quality
Reviews may be used to create
a ranking of software vendors
Functionalities always prevail over
internal quality
We need quality metrics in contracts
Vendors tend to claim too many
false-positives
Anti-patterns remain in the code,
aggregate metrics (CAST TQI) may be
used to exert pressure on the vendor
There have to be penalties in contracts
for poor quality. Or at least KPIs
inspected by top management
Good-will assumption does not work
21. 21
Orange Polska for IWSM Mensura, Cracow 2015
Service for third parties – scope and tools
Quality audits by Orange Polska*
Static code review
CAST & Sonar (cyclical & one shot), including adding new rules
Productivity review
CAST (automated function points)
Open-source licenses review
Palamida
Security audit
HP Fortify
Software process improvement
SVN, Git, Jenkins (continuous integration)
* provisioned in English, French and Polish
22. 22
Orange Polska for IWSM Mensura, Cracow 2015
Thank you
Bogusz Jeliński
bogusz.jelinski2@orange.com
Piotr Popowski
piotr.popowski@orange.com
IWSM Mensura 2015
Estimating size and price
for IT projects using FPA
method
Automated code review