Presentation from the IWSM Mensura 2015 conference held October 5-7 in Cracow, Poland

1. Accoun&ng for Non-­‐Func&onal and Project
requirements
in so9ware project performance measurement, benchmarking and es&ma&ng:
COSMIC and IFPUG developments
IWSM/Mensura
Conference,
Krakow,
October
2015
Talmon
Ben-­‐Cnaan
(IFPUG),
Charles
Symons
(COSMIC)

2. IWSM
MENSURA
Krakow,
October
2015
Agenda
• IntroducJon:
Why
the
need
for
standards
for
Non-­‐
FuncJonal
Requirements
(NFR)
and
for
Project
Requirements
and
Constraints
(PRC)?
• The
joint
COSMIC/IFPUG
development
of
a
Glossary
of
NFR
and
PRC
terms
• The
COSMIC
Guideline
on
how
to
consider
NFR
and
PRC
• The
IFPUG
SoTware
Non-­‐FuncJonal
Assessment
Process
(SNAP)
to
measure
NFR
• Conclusions
and
next
steps
• QuesJons
and
debate

3. IWSM
MENSURA
Krakow,
October
2015
The ac&vi&es of so9ware project performance
measurement, benchmarking and es&ma&ng
need consistent data and terminology
Measuring
project
performance
Project
estimating
Benchmarking
Project
data &
benchmark
repository
Recording
&
measuring
software
system
project
requirements

4. IWSM
MENSURA
Krakow,
October
2015
Func&onal size measurements are used
consistently across all ac&vi&es
• There
are
three
types
of
requirements
for
a
soTware
system
project
• FuncJonal
User
Requirements
(FUR)
• Non-­‐funcJonal
Requirements
(NFR)
• Project
Requirements
and
Constraints
(PRC)
• FuncJonal
size
measurement
methods,
e.g.
the
COSMIC
or
IFPUG
methods,
are
used
consistently
to
measure
the
size
of
FUR
across
all
acJviJes
But
what
about
NFR
and
PRC?

5. IWSM
MENSURA
Krakow,
October
2015
There is no good exis&ng defini&on of
Non-­‐Func&onal Requirements (NFR)
Example:
ISO
24756
definiJon
1):
“A
so%ware
requirement
that
describes
not
what
the
so%ware
will
do
but
how
the
so%ware
will
do
it.
Example:
so%ware
performance
requirements,
so%ware
external
interface
requirements,
so%ware
design
constraints,
and
so%ware
quality
a>ributes.”
Comment:
only
‘design
constraints’
define
‘how
the
soTware
will
do
it’

6. IWSM
MENSURA
Krakow,
October
2015
Next, each ac&vity has defined its own
data and terminology for NFR and PRC 2)
Requirements Recording
(50)
IEEE 804, ISO 9126
Wikipedia
Requirements Sizing
(36)
VAF, TCA, SNAP
Benchmarking
(48)
ISBSG, SEI
Project Estimating
(39)
COCOMO II
One common NFR
(= “Interfaces”)

7. IWSM
MENSURA
Krakow,
October
2015
Past surveys have found varying
numbers of NFR terms
• ‘at
least’
161
terms
(Chung
et
al
3))
• 122
terms
in
a
structured
hierarchy
(Saito
et
al
4))
• 108
terms
(Symons
2))
• ISO/IEC
SQuaRE
5)
standard
lists
32
Quality
terms
A
complete,
standard
list
of
NFR
and
PRC
terms
may
be
impossible

8. IWSM
MENSURA
Krakow,
October
2015
Past aSempts to measure a collec&ve
size of NFR are now discredited
Albrecht’s
10
components
of
the
‘Value
Adjustment
Factor’
6)
14
components
(IFPUG)
7)
19+
components
(MkII
FPA)
8)
…..
but
they
did
have
value
when
they
were
first
invented.

9. IWSM
MENSURA
Krakow,
October
2015
Conclusions
• There
is
no
common
understanding
in
the
soTware
industry
of
what
is
meant
by
NFR
• ExisJng
standards
for
NFR
and
PRC
are
not
coherent
• These
issues
handicap
acceptance
of
methods
for:
• quanJfying
requirements,
• developing
project
performance
benchmarks,
• project
esJmaJng.
These
were
the
drivers
for
COSMIC
and
IFPUG
to
develop
the
joint
Glossary
of
NFR
and
PRC
terms

10. IWSM
MENSURA
Krakow,
October
2015
Agenda
• IntroducJon:
Why
the
need
for
standards
for
Non-­‐
funcJonal
Requirements
(NFR)
and
for
Project
Requirements
and
Constraints
(PRC)?
• The
joint
COSMIC/IFPUG
development
of
a
Glossary
of
NFR
and
PRC
terms
• The
COSMIC
Guideline
on
how
to
consider
NFR
and
PRC
• The
IFPUG
SoTware
Non-­‐FuncJonal
Assessment
Process
(SNAP)
to
measure
NFR
• Conclusions
and
next
steps
• QuesJons
and
debate

11. IWSM
MENSURA
Krakow,
October
2015
Words
What
do
we
mean
when
saying
• A
date
is
a
date.
• Right!
• It
is
not
my
type

12. IWSM
MENSURA
Krakow,
October
2015
We have agreed many aspects of
NFR and PRC over the past year
• Scope
of
the
Glossary
9)
• Clarity
on:
• ‘Requirements’
versus
‘constraints’
• The
‘things’
that
NFR
and
PRC
apply
• DisJnguishing
and
defining
NFR
and
PRC
• Structuring
NFR
(aligned
with
SQuaRE)
and
PRC
to
classes
à
groups
à
terms
• The
terms:
• 67
NFR
terms,
mainly
from
ISO
and
ISBSG
(27
COSMIC/IFPUG)
• 19
PRC
terms,
mainly
from
PMI
and
ISBSG

13. IWSM
MENSURA
Krakow,
October
2015
We have clarified the terms
Requirements and Constraints
Thesaurus:
Requirement:
“a
thing
demanded
or
obligatory”
Constraint:
“limitaJon
or
restricJon”
The
difference
is
not
always
clear:
• A
requirement
that
the
so%ware
shall
use
C#
is
also
a
constraint
• ‘Latency’
can
be
a
requirement
for
some
real-­‐Fme
systems
but
a
constraint
for
the
Mars
Rover
system
• Staffing
skills
can
be
a
project
requirement
or
a
constraint
Constraints
Requirements
We
use
requirements
for
convenience.
We
use
constraints
only
when
it
is
helpful
to
disAnguish
constraints
from
requirements.

14. IWSM
MENSURA
Krakow,
October
2015
The scope of requirements and
constraints
Requirements
and
constraints
can
apply
to
• The
soCware;
• The
data
maintained
or
used
by
the
soTware;
• The
technology
to
be
used,
e.g.
the
planorms;
• Other
deliverables,
e.g.
documentaJon
or
training;
• The
combined
hardware/soCware
system,
e.g.
a
response
Jme;
and
some
constraints
come
from
the
environment

15. IWSM
MENSURA
Krakow,
October
2015
Data
Technology
Hardware/Software
System
Project
Environment
Other
Deliverables
Software
Project, System, Product
A
project:
a
temporary
endeavor
to
achieve
defined
objecJves
of
delivering
a
product
by
defined
dates.
Following
a
Project,
there
is
a
Product
in
place
(A
new
Product
or
a
Product
that
was
changed
by
the
project).
A
product
is
a
hardware/
soTware
system
or
an
item
of
soTware
such
as
a
soTware
package
Product
A
Product
C
Product
B

16. IWSM
MENSURA
Krakow,
October
2015
Requirements
for
a
SoTware
System
Project
Project
Requirements
and
Constraints
(PRC)
System
and
SoTware
Non-­‐FuncJonal
Requirements
(NFR)
SoTware
FuncJonal
User
Requirements
(FUR)
System
Environment
Requirements
Technical
Requirements
Quality
Requirements
(SoTware
System)
Quality
Requirements
(Data)
Classifica&on of Requirements

17. IWSM
MENSURA
Krakow,
October
2015
System
and
SoTware
Non-­‐FuncJonal
Requirements
(NFR)
System
Environment
Requirements
Technical
Requirements
Quality
Requirements
(SoTware
System)
Quality
Requirements
(Data)
Classifica&on of NFR
1 Quality
of
the
data
maintained
by
the
soTware
2 System
performance
3 CompaJbility
4 Ease
of
use
5 System
reliability
6 Control
of
access
7 Maintainability
8 Ease
of
deployment
9 System
or
soTware
architecture
or
design
1 Context
2 ApplicaJon
Domain
3 ImplementaJons
4 User
Base
1 OperaJonal
Planorm
2 Database
3 OperaJonal
Planorm
constraints
4 Development
requirements
Class
Group
Type

18. IWSM
MENSURA
Krakow,
October
2015
COSMIC and IFPUG defini&ons
FuncAonal
User
Requirements
(FUR)
-­‐
ISO/IEC
14143-­‐1
DefiniAon
Adopted
by
COSMIC
and
IFPUG
A
sub-­‐set
of
the
user
requirements.
Requirements
that
describe
what
the
soCware
shall
do,
in
terms
of
tasks
and
services.
NOTE:
FuncJonal
User
Requirements
relate
to
but
are
not
limited
to:
• data
transfer
(for
example
Input
customer
data;
Send
control
signal)
• data
transformaJon
(for
example
Calculate
bank
interest;
Derive
average
temperature)
• data
storage
(for
example
Store
customer
order;
Record
ambient
temperature
over
Jme)
• data
retrieval
(for
example
List
current
employees;
Retrieve
latest
aircraT
posiJon)

19. IWSM
MENSURA
Krakow,
October
2015
COSMIC and IFPUG defini&ons
Non-functional requirement – COSMIC and IFPUG
Definition
Any
requirement
for
a
soCware
system
or
for
a
soCware
product,
including
how
it
should
be
developed
and
maintained,
and
how
it
should
perform
in
operaAon,
except
any
funcAonal
user
requirement
for
the
soCware.
Non-­‐funcJonal
requirements
concern:
• the
soTware
system
or
soTware
product
quality;
• the
environment
in
which
the
soTware
system
or
soTware
product
must
be
implemented
and
which
it
must
serve;
• the
processes
and
technology
to
be
used
to
develop
and
maintain
the
soTware
system
or
soTware
product,
and
the
technology
to
be
used
for
their
execuJon.

20. IWSM
MENSURA
Krakow,
October
2015
COSMIC and IFPUG defini&ons
Project Requirements and Constraints - Definition
Requirements
that
define
how
a
soCware
system
project
should
be
managed
and
resourced,
or
constraints
that
affect
its
performance.
Requirements
may
include:
• The
targets
the
project
should
achieve
(e.g.
budget,
delivery
date,
product
quality);
• The
project
management
processes
that
should
be
used;
• How
the
project
should
be
governed
and
resourced.
Constraints
may
include:
• LimitaJons
on
the
project
resources;
• Dependencies
on
other
projects
outside
the
control
of
the
project
concerned.

21. IWSM
MENSURA
Krakow,
October
2015
67
Terms
Examples
COSMIC
/
IFPUG
DefiniJon
NFR
sub-­‐type
(Quality,
System
Environment
Requirements,
Technical
Requirements)
Quality:
9
groups
Environment:
4
groups
Technical:
4
groups
ISO
DefiniJon

22. IWSM
MENSURA
Krakow,
October
2015
COSMIC and IFPUG strongly
recommend their joint Glossary
• Developed
through
>
20
iteraJons
over
a
year
• Reviewed
by
many
experts
(academic
and
pracJJoners)
from
around
the
world
• Available
for
free
download
from:
• www.cosmic-­‐sizing.org
• www.ifpug.org
• We
welcome
feedback
and
comments

23. IWSM
MENSURA
Krakow,
October
2015
Agenda
• IntroducJon:
Why
the
need
for
standards
for
Non-­‐
funcJonal
Requirements
(NFR)
and
for
Project
Requirements
and
Constraints
(PRC)?
• The
joint
COSMIC/IFPUG
development
of
a
Glossary
of
NFR
and
PRC
terms
• The
COSMIC
Guideline
on
how
to
consider
NFR
and
PRC
• The
IFPUG
SoTware
Non-­‐FuncJonal
Assessment
Process
(SNAP)
to
measure
NFR
• Conclusions
and
next
steps
• QuesJons
and
debate

24. IWSM
MENSURA
Krakow,
October
2015
The COSMIC Guideline builds on
the joint COSMIC/IFPUG Glossary
Contents
• Terminology,
NFR/PRC
definiJons
• ClassificaJon
of
NFR/PRC
terms
• Glossary
of
NFR/PRC
terms
• EvoluJon
of
NFR
in
a
project
• How
to
deal
with
NFR/PRC
in
performance
measurement,
benchmarking
&
esJmaJng
• Measurement
of
NFR
(interim)
Glossary9)
ü
ü
ü
Guideline10)
ü
ü
ü
ü
ü
ü

25. IWSM
MENSURA
Krakow,
October
2015
Requirements that first appear as NFR
may evolve, wholly or partly, into FUR
Outline
Funct-
-ional
Requts.
Outline
NFR
Project
Requts. &
Constraints
Require-
ments
Analysis
Definition
& Design
Build, Test
&
Implement
A
r
c
h
i
t
e
c
t
u
r
e
Approx.
Funct-
-ional
Requts.
Detailed
NFR
Imple-
mented
software
system
or
software
product
Detailed
FUR
……
as
demonstrated
by
Al
Sarayreh,
Abran
and
others,
e.g.
[11]

26. IWSM
MENSURA
Krakow,
October
2015
The ‘addi&onal’ FUR can be measured by
approximate or standard COSMIC sizing
Outline
Funct-
-ional
Requts.
Outline
NFR
Project
Requts. &
Constraints
Require-
ments
Analysis
Definition
& Design
Build, Test
&
Implement
A
r
c
h
i
t
e
c
t
u
r
e
Imple-
mented
software
system
or
software
product
Approx.
Funct-
-ional
Requts.
Detailed
NFR
Detailed
FUR
Size by analogy or
expert judgement
Approx. COSMIC
size measurement
Precise COSMIC
size measurement

27. IWSM
MENSURA
Krakow,
October
2015
The Guideline gives examples for all
Quality NFR how they may evolve as a
project progresses
FuncAonality
that
may
evolve
from
the
NFR,
that
can
be
measured
‘Middleware’
funcJonality
to
enable
portability
across
mulJple
DBMS
or
OS
or
hardware
Graphical
User
Interface
funcJons;
and
FuncJonality
to
assist
users,
e.g.
‘Help’
funcJons
FuncJonality
to
import
external
data
in
real-­‐Jme
so
that
it
is
available
for
immediate
use
IniAal
NFR
term
Portability
Usability
Response
Jme
‘Residual’
or
‘Real’
NFR
statement
The
specific
environments
across
which
the
soTware
must
be
portable
The
specific
usability
requirements
(e.g.
‘must
be
usable
by
public
with
no
training’)
-­‐
The
response
Jme
target;
-­‐
Specific
hardware;
-­‐
Low-­‐level
programming
language.
Examples:

28. IWSM
MENSURA
Krakow,
October
2015
COSMIC has no plans to develop a
collec&ve size measure for NFR
Inherent
difficulJes:
1. How
to
form
a
collecJve
size
measure
that
will
add
pracJcal
value,
long-­‐term?
2. The
large
number
and
variety
of
NFR
3. How
to
collect
effort
data
related
to
NFR
when
NFR
evolve
into
FUR?

29. IWSM
MENSURA
Krakow,
October
2015
1. Collec&ve size measures are
common and o9en valuable
Examples
of
valuable
collecJve
size
measures.
(All
are
mathemaJcally
invalid,
but
they
work.)
• The
IFPUG
Value
Adjustment
Factor
(when
first
introduced)
• Indices
that
affect
stock
market
senJment,
e.g.
the
PMI
• Measures
of
‘biological
age’
• Total
Factor
ProducJvity
CondiJons
for
a
usable
collecJve
size
measure:
• A
finite,
stable,
well-­‐defined
set
of
components
• If
possible,
a
common
unit
of
measure
for
all
components
• A
simple
formula
for
the
index
(avoid
complex
maths)

30. IWSM
MENSURA
Krakow,
October
2015
2. There is a large number and variety of
NFR. Many overlap in meaning
How
many
separate
NFR
should
we
include
in
a
collecJve
size
measure?
(10,
14,
19+,
67,
108,
122,
161+
…?)
Maintain-­‐
ability
Test-­‐
ability
Flex-­‐
ibility
Adapt-­‐
ability
Extend-­‐
ability
Evolv-­‐
ability
Modula-­‐
rity
Modifia-­‐
ability
Reus-­‐
ability
Port-­‐
ability

31. IWSM
MENSURA
Krakow,
October
2015
3. How to capture the effort
associated with implemen&ng NFR?
….
given
that
NFR
evolve
as
a
project
progresses
into
FUR
…..
?
....
&
noJng
that
effort
to
implement
NFR
varies
with:
• soTware
domain,
• type
of
NFR,
• the
parJcular
‘soluJon’
for
the
NFR,
• technology,
re-­‐use,
etc.

32. IWSM
MENSURA
Krakow,
October
2015
Instead, we propose basic sets of NFR/PRC to
record and measure for performance
measurement, benchmarking, es&ma&ng
NFR and PRC
Class
Terms
Quality NFR Response time, Transaction rate
Security, Privacy
Maintainability, Reusability
Interfaces, Operational processing mode
System
Environment NFR
Application type, sub-type
Implementations (numbers of)
Maximum number of concurrent users
Technical NFR Operational platform type
Programming language
DBMS software
Project
Requirements and
Constraints (PRC)
Many, but not all of the 19 project requirements and constraints are worth
recording. Example: if staff experience levels, processes and methods
and tools are the same for all projects, then they need not be recorded
for internal purposes.

33. IWSM
MENSURA
Krakow,
October
2015
…. and advise how to deal with NFR in
performance measurement,
benchmarking and es&ma&ng
• EsJmate
and
measure
total
funcJonal
size,
including
funcJonality
that
evolves
from
NFR
• Within
a
defined
‘environment’,
i.e.
• soTware
domain
(business,
real-­‐Jme)
• soTware
architecture,
technology
• project
type
(new,
enhancement,
etc)
….
collect
and
record
data
on
‘real’
NFR
and
PRC
so
that
you
can
make
like-­‐for-­‐like
comparisons
(This
is
normal
pracJce
for
benchmarking
acJviJes,
plus
a
few
addiJonal
parameters.)

34. IWSM
MENSURA
Krakow,
October
2015
Guideline Conclusions
The
COSMIC
‘Guideline
for
dealing
with
Non-­‐
FuncFonal
Requirements
in
project
performance
measurement
&
esFmaFng’
• builds
on
the
joint
COSMIC/IFPUG
Glossary
• provides
pracJcal
advice
on
how
to
deal
with
manageable
sub-­‐sets
of
NFR
and
PRC
and
will
be
available
in
October
for
free
download
from
www.cosmic-­‐sizing.org

35. IWSM
MENSURA
Krakow,
October
2015
Agenda
• IntroducJon:
Why
the
need
for
standards
for
Non-­‐
funcJonal
Requirements
(NFR)
and
for
Project
Requirements
and
Constraints
(PRC)?
• The
joint
COSMIC/IFPUG
development
of
a
Glossary
of
NFR
and
PRC
terms
• The
COSMIC
Guideline
on
how
to
consider
NFR
and
PRC
• The
IFPUG
SoTware
Non-­‐FuncJonal
Assessment
Process
(SNAP)
to
measure
NFR
• Conclusions
and
next
steps
• QuesJons
and
debate

36. IWSM
MENSURA
Krakow,
October
2015
FPA and SNAP covers functional and
non-functional requirements
PRC
–
do
not
impact
soTware
size
NFR
–
measured
with
SNAP
Points
FUR
–
measured
with
FP
Guidelines
to
prevent
doubled
counJng

37. IWSM
MENSURA
Krakow,
October
2015
PRC affects produc&vity
and not size
• Effort
depends
on
PRC
classes.
No
standard
/
industry
mathemaJcal
formula
• Examples:
Project
type;
(same
size
different
effort)
• Effort
may
be
formulated
for
some
PRC
(“producJvity
drivers”)
• Examples:
Skill
level;
LocaJon;
Schedule
compression
(“crashing”)
(same
size
different
effort)
• Effort
(or
deviaJon
from
esJmaJon)
can
be
explained
by
some
PRC
• Examples:
Risk
that
was
materialized;
scope
creep

38. IWSM
MENSURA
Krakow,
October
2015
SNAP
• SNAP
idenJfies
fourteen
non-­‐funcJonal
characterisJcs
(“sub
categories”)
that
classify
the
way
NFR
are
met.
• In
each
sub-­‐category,
the
size
is
assessed
within
a
counJng
unit
(SCU).
The
SCU
is
part
of
the
definiJon
of
the
sub-­‐category.
• Non
funcJonal
size
is
determined
by
a
set
of
parameters
/
complexity
levels.
The
parameters
that
define
each
sub-­‐category
answer
the
quesJon
“what
factors
affect
the
effort
to
deliver
a
NFR”
• Example:
The
effort
to
add
input
methods
(e.g.
barcode
reader,
fax
server,
reading
CSV
file)
with
same
funcJonality
depends
on
number
of
added
input
methods

39. IWSM
MENSURA
Krakow,
October
2015
SNAP
• Parameters
can
be
used
as
a
table
of
values
(e.g.
“High,
medium,
low”,
“<10,
10
to
15,
16+”)
or
as
a
mulJplier
(“SP
=
3*#
addiJonal
input
methods),
or
a
combinaJon
of
the
above
• Once
the
parameters
are
assessed,
the
size
of
the
SCU
is
calculated.
SP
size
is
the
sum
of
the
SPs
of
the
SCUs

40. IWSM
MENSURA
Krakow,
October
2015
SNAP sub categories are independent of
the way NFR are classified / defined
Accuracy
Performance
Ease of use / customer
satisfaction
Response Time
Requirements
CharacterisJcs

41. IWSM
MENSURA
Krakow,
October
2015
SNAP sub-­‐categories are independent
of the way NFR are classified
Accuracy
Ease of use / customer
satisfaction
Response Time
ISO9126
ISO25010
COSMIC/IFPUGGlossary
Requirements
CharacterisJcs
Time Behavior
Performance

42. IWSM
MENSURA
Krakow,
October
2015
Data Func&ons and Transac&onal
Func&ons are not sufficient to size NFR
The effort to deliver NFR is also affected by
• Database ac&vi&es
• Dealing with UI proper&es
• Extensive Logical / Mathema&cal opera&ons
• Batch processes that do not cross the boundaries
and are not exposed to users
• Mul&ple inputs/output
• …..

43. IWSM
MENSURA
Krakow,
October
2015
Example 1 – The requirements
NFR:
Improve
CRM
system
performance
for
“Retrieve
and
View
Customer
Details“-­‐
by
loading
the
‘customer
details’
screen
in
3
seconds
or
less.
(Currently
it
takes
0.5
sec
to
6
sec.)
Problem
analysis:
The
system
is
slow
when
the
user
(call
center
representaJve)
opens
a
big
customer
with
many
products
and
many
interacJons.
It
also
happens
when
users’
virtual
memory
is
low

44. IWSM
MENSURA
Krakow,
October
2015
Example 1 -­‐ the design
1. Increase
the
virtual
memory
of
‘Windows’
to
maximum;
2. Create
a
database
view,
which
includes
data
from
three
tables
(Assigned
Products,
TransacJons,
and
Payments)
(Assuming
30
different
DETs
are
fetched).
3. Add
Logic
to
idenJfy
which
customers
will
be
loaded
to
the
new
view
4. Add
a
field
to
‘Customers’
table
to
indicate
whether
recent
informaJon
moved
to
the
new
view
and
add
logic
to
idenJfy
which
transacJons
will
be
copied
(not
seen
by
the
users,
hence
non-­‐
funcJonal
change).
5. A
batch
process
runs
in
the
background
every
2
hours,
refreshing
this
view
with
large
and
most
acJve
users

45. IWSM
MENSURA
Krakow,
October
2015
Example 1 – SNAP count
• Increasing
virtual
memory:
InstallaJon
of
a
larger
size
virtual
memory
is
a
hardware
configuraJon
and
not
a
soTware
change.
It
is
not
counted
using
soTware
sizing
methods.
• CreaJng
a
database
view
(CUST_DETAIL_SUMMARY_
TEMP)
and
adding
an
indicator
(field)
to
‘Customer’
table:
• Count
SP
using
3.2
Database
changes;
SNAP
count
is
based
on
two
database
changes,
20-­‐50
DETs,
2-­‐5
RETs
• Adding
a
batch
process:
(this
batch
job
does
not
cross
the
boundary,
hence
it
is
not
counted
using
FP)
• Count
SP
using
3.3
Batch
Processes.
SNAP
count
is
based
on
30
DETs
and
3
FTRs
• Adding
logic
to
meet
the
NFR:
Count
SP
using
1.2
Logical
and
MathemaJcal
OperaJons.
SNAP
count
is
based
on
3
FTRs,
type
=
‘Logical’
and
10
DETs
used
for
the
added
logic

46. IWSM
MENSURA
Krakow,
October
2015
IFPUG Coun&ng Process
Gather
available
documentaJon
Determine
counJng
purpose,
scope,
boundaries
and
parJJons
IdenJfy
requirements
as
FUR,
NFR.
Separate
mixed
requirements
to
FUR
and
NFR
Measure
Data
FuncJons
Measure
TransacJonal
FuncJons
Associate
NFR
to
sub-­‐
categories
Calculate
funcJonal
size
Determine
SNAP
size
of
each
sub
category
Calculate
SNAP
size
Document
and
report
FuncJonal
Non-­‐FuncJonal

47. IWSM
MENSURA
Krakow,
October
2015
Es&ma&on and benchmark
Es&ma&on
𝑬𝒇𝒇𝒐𝒓𝒕=
​[ 𝑷𝑫𝑹↓ 𝟏 (𝑷𝒓𝒐𝒋𝒆𝒄​ 𝒕↑′ 𝒔 𝑪𝒉𝒂𝒓𝒂𝒄𝒕𝒆𝒓𝒊𝒔𝒕𝒊𝒄𝒔)× 𝑭𝒖𝒏𝒄𝒕𝒊𝒐𝒏𝒂𝒍
𝑺𝒊𝒛𝒆]+[​ 𝑷 𝑫𝑹↓ 𝟐 (𝑷𝒓𝒐𝒋𝒆𝒄​ 𝒕↑′ 𝒔 𝑪𝒉𝒂𝒓𝒂𝒄𝒕𝒆𝒓𝒊𝒔𝒕𝒊𝒄𝒔)× 𝑵𝒐𝒏
−𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏𝒂𝒍 𝑺𝒊𝒛𝒆]
PDR
–
ProducJvity
Delivery
Rate
Note:
Feedback
from
users
show
that
PDR2
diverges
in
3
of
the
14
sub-­‐categories.
NSFFC
will
reformulate
them
based
on
collected
data
Benchmarking
• Based
on
linear
regression
to
extract
PDR1
and
PDR2:
• Two
different
baselines,
two
business
values
to
customers

48. IWSM
MENSURA
Krakow,
October
2015
Agenda
• IntroducJon:
Why
the
need
for
standards
for
Non-­‐
funcJonal
Requirements
(NFR)
and
for
Project
Requirements
and
Constraints
(PRC)?
• The
joint
COSMIC/IFPUG
development
of
a
Glossary
of
NFR
and
PRC
terms
• The
COSMIC
Guideline
on
how
to
consider
NFR
and
PRC
• The
IFPUG
SoTware
Non-­‐FuncJonal
Assessment
Process
(SNAP)
to
measure
NFR
• Conclusions
and
next
steps
• QuesJons
and
debate

49. IWSM
MENSURA
Krakow,
October
2015
COSMIC and IFPUG have come
closer over the past year
Close
exchange
of
ideas
has
led
to:
• Agreeing
to
disJnguish
NFR
and
PRC
• AdopJng
PMI
definiJons
for
PRC
terms
• Many
refinements
to
the
classificaJon
structure
and
Glossary
contents
–
through
>
20
iteraAons
Further,
we
agree
that:
• NFR
increase
the
size
of
the
soTware,
and
should
be
measured
• Performance
measurement,
benchmarking
and
project
esJmaJng
must
consider
both
FUR
and
NFR

50. IWSM
MENSURA
Krakow,
October
2015
Conclusions
Whilst
COSMIC
&
IFPUG
sJll
have
different
views
on
how
to
measure
NFR,
we
have
collaborated
very
well
to
agree:
• DefiniJons
and
classificaJon
of
NFR
and
PRC
• The
Glossary
of
NFR
and
PRC
terms,
v1.0
We
strongly
recommend
these
to
the
soTware
engineering
community

51. IWSM
MENSURA
Krakow,
October
2015
Next steps
• Enhance
the
Glossary
to
take
into
account
‘Data
Quality’
requirements
• Enhance
the
Glossary
(and
define
measurements?)
to
take
into
account
‘Other
Deliverables’
such
as
training
and
documentaJon
This
is
a
quesJon
to
YOU:
Should
we?????

52. Thank
you
for
your
a^enAon
Talmon
Ben-­‐Cnaan
(www.ifpug.org
);
talmonbc@amdocs.com
Charles
Symons
(www.cosmic-­‐sizing.org
);
cr.symons@bJnternet.com

53. IWSM
MENSURA
Krakow,
October
2015
References
1. ISO/IEC/IEEE
24765:2010
‘Systems
and
soTware
engineering
–
vocabulary’.
2. Symons,
C.R.,
‘AccounJng
for
Non-­‐FuncJonal
Requirements
in
ProducJvity
Measurement,
Benchmarking
and
EsJmaJng’,
UKSMA/COSMIC
InternaJonal
Conference
on
SoTware
Metrics
&
EsJmaJng,
London,
27/28
October
2011.
3. L.
Chung,
B.
Nixon,
E.
Yu,
and
J.
Mylopoulos,
"Non-­‐funcJonal
Requirements
in
SoTware
Engineering,“
Kluwer
Academic
Publishing,
2000.
4. Saito,
Y.,
Monden
A.,
Matsumoto
K.,
‘EvaluaJon
of
non-­‐funcJonal
requirements
in
a
request
for
proposal
(RFP)’,
Nara
InsJtute
of
Science
&
Technology,
Japan,
at
InternaJonal
Workshop
on
SoTware
Measurement
(IWSM),
Nara,
2012..
5. ISO/IEC
FCD
25010:
Systems
and
soTware
engineering,
–System
and
soTware
product
Quality
Requirements
and
EvaluaJon
(SQuaRE)
–System
and
soTware
quality
models
6. Albrecht,
A.
J.,
‘Measuring
applicaJon
development
producJvity’,
In
Proc.
of
the
IBM
ApplicaJons
Development
Symposium,
pp.
83-­‐-­‐92.
Monterey,
California
(1979)
7. IFPUG,
CounJng
PracJces
Manual,
Release
4.3.1
–
Appendix
C
8. Symons,
C.R.,
SoTware
sizing
&
esJmaJng:
Mk
II
FPA’,
John
Wiley
&
Sons,
1991,
ISBN
0
471
92985
9
9. ‘Glossary
of
terms
for
Non-­‐FuncJonal
Requirements
and
Project
Requirements
used
in
soTware
project
performance
measurement,
benchmarking
and
esJmaJng’,
Version
1.0,
September
2015
10. ‘Guideline
on
Non-­‐FuncJonal
&
Project
Requirements:
How
to
consider
non-­‐funcJonal
and
project
requirements
in
soTware
project
performance
measurement,
benchmarking
and
esJmaJng’,
version
1.0,
October
2015
11. Khalid
T.
Al-­‐Sarayreh,
Alain
Abran
and
Juan
J.
Cuadrado-­‐Gallego,
‘A
Standards-­‐based
model
of
system
maintainability
requirements’,
Journal
of
SoTware:
EvoluJon
and
Process,
2013,
Vol.
25,
no.
5,
pp.
459-­‐505