Industry data indicates that untrained and inexperienced requirements authors commonly inject thirty to fifty major defects per page of text. With many requirements specifications reaching several hundred pages, potentially thousands of defects are injected into the software development process. John Terzakis says training and mentoring of authors by a requirements coach is effective in reducing defect densities by an order of magnitude—when each coach is assigned only a few authors, they are collocated and, most importantly, experienced requirements coaches are available. So what happens if there are dozens of authors spread across multiple geographical sites and there are no requirements coaches at those sites? The Requirements Authors Mentoring Program (RAMP) provides the solution. John shares how a network of site-based requirements apprentices were recruited, trained to be coaches, and then assigned to requirements authors in their geographic region. John provides details on the defect detection and removal training, key success factors, and results from an actual Intel project. Learn how your organization can establish RAMP, which is applicable to both waterfall and agile software development lifecycles.

1. BT4
Business Analysis & Requirements
6/8/2017 11:30:00 AM
BT4 RAMP: Requirements Authors
Mentoring Program
Presented by:
John Terzakis
Intel
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888-­‐268-­‐8770 ·∙ 904-­‐278-­‐0524 - info@techwell.com - https://www.techwell.com/

2. John Terzakis
Intel
John Terzakis has more than thirty years of experience developing, writing, and
testing software. With Intel for eighteen years, John is a systems engineer
specializing in requirements engineering. He works with architects and engineers
to help specify, improve, verify, and manage requirements for Intel's next
generation cellular modem products. He is a certified Intel instructor for the
internal requirements engineering courses. John's prior experience includes
director and manager roles with Shiva, Racal InterLan, and Dataproducts. He was
a member of the technical staff at Bell Labs. John is a Fellow with the
International Academy, Research and Industry Association.

3. Version 1.0
RAMP: Requirements Authors Mentoring
Program
John Terzakis
Intel Corporation
June 8, 2017
BSC/ADC West 2017
Las Vegas, NV

4. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Legal Disclaimers
2
Intel Trademark Notice:
Intel and the Intel Logo are trademarks of Intel Corporation in the U.S. and other countries.
Non-Intel Trademark Notice:
*Other names and brands may be claimed as the property of others

5. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Agenda
3
• Problem Statements
• Background and Proposed Solution
• Recruiting and Training Coaches
• RAMP Results and Conclusions
• Future Work
• Questions & Answers

6. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Problem Statements
4

7. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Problem Statements
5
• Untrained or inexperienced authors typically inject 30-50 major
Defects Per logical Page (DPP) of requirements
• Industry data
• Requirements training lacking in undergraduate programs
• High quality requirements sets: < 2 DPP
• Sampled Intel requirements quality levels measured at 30+ DPP
• Too few requirements Subject Matter Experts (SMEs) to mentor
requirements authors in entire organization
• Only 1 SME
• Over 125 authors across all projects
• 15 global sites

8. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Problem Statements
6
• Poor requirements quality leads to:
• Poor product quality
• Schedule slips
• Overtime & rework
• “Defeaturing”
• Low customer satisfaction levels

9. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Background and Proposed Solution
7

10. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Background
8
Previous work1 demonstrated effectiveness of training and mentoring to
reduce requirements defects and improve product quality
• Small number of authors (1 initially -> 4 -> 12)
• One requirements SME
• Relatively low SME to author ratio
• Requirements defect density reduced from double digit DPP to <1 DPP over time
1J. Terzakis, “The Impact of Requirements on Software Quality across Three Product Generations,” Proceedings of the 21st International
Requirements Engineering Conference (RE ’13), 2013.

11. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
New Project
9
• “Alpha” project: silicon and real time firmware
• Next generation platform
• Alpha requirements based on existing catalog of legacy requirements
(~5,000) plus over 100 new requirements
• Development at many, geographically dispersed sites (several from
acquisitions): different processes, practices, artifacts and cultures
• 65 authors across 15 global sites including locations in the United
States, Europe, and Asia/Pacific

12. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
10
Requirements Defects
Requirements SME
Impossible for one SME to mentor that many authors

13. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Establishing the Baseline
11
• Measure Requirements Defect Density (ReqDD)
• ReqDD = total number of defects / total number of logical pages (DPP)
• 1 logical page = 400 words
• Reviewed sample of legacy requirements on two prior projects and new
requirements for Alpha:
Project # of Defects # of Logical Pages Defects/ Page (DPP)
Alpha Gen -2 246 7.36 33.4
Alpha Gen -1 370 7.76 47.7
Alpha 163 3.52 46.3

14. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Proposed Solution
12
• Train a network of site-based requirements coaches
• Key success factors:
 Recruit coaches with a deep interest in and aptitude for improving requirements
quality
 Commit coaches to at least 10% time per week initially
 Identify an experienced requirements SME to train them
 Require strong support from the management team
 Documented in coach development plan
 Participation recognized and rewarded
 Importance of RAMP communicated publicly
 RAMP visible as a continuous improvement activity with measurable goals

15. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Recruiting and Training Coaches
13

16. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Recruiting Coaches
14
• Identify students with strong interest during requirements training
• Solicit coaches via emails and other announcements
• Partner with management team to recommend coaches
• “Word of mouth”

17. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Training Coaches
15
• Weekly training sessions (SME with 2-5 coaches per session; multiple
sessions based on geographic location)
• Approximate 6-9 month total training period
• Three part plan:
1. Review key training materials (~2 months)
2. Exercises on actual requirements to locate defects (~3 months)
3. Practice rewriting requirements to remove defects (~3 months)
Review Exercises Rewrites

18. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Review Key Training Materials
16
• Defect definitions and common problems
• Weak words, subjectivity and ambiguity
• Unbounded lists
• Implicit collections
• Grammatical issues
• Missing imperatives
• Lack of common syntax
• Identifying missing information (very important)
• Checklists to identify defects
• Ensures consistency
• Ensures objectivity
Review

19. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Exercises to Locate Defects
17
• Focus on major defects for new and modified requirements
• Major: costs >10x to fix in future
• New and Modified: best return on investment (can’t correct all of
them)
• Coaches review requirements individually between meetings
• Group reviews in meetings
• One coach presents their findings and the others add their findings
• SME highlights any defects missed by the group
• “Wisdom of the Crowd”—demonstrates the value of multiple
perspectives during requirements reviews
• Coaches can then emphasize the importance of group reviews to their
authors
Exercises

20. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Practice Rewriting Requirements
18
• Identify questions to ask authors for rewrites
• Suggest alternate wording to replace located defects
• Highlight missing information (states, triggers, error conditions, etc.)
• Propose rewrites based on available data
• Use a notation for unknown information (e.g., <While in this state>)
• Provides a starting point for discussions with the authors
Rewrites

21. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Project Engagement
19
• RAMP coaches assigned to Alpha project requirements authors
• Focus on major defects for new and modified requirements
• 15 coaches across 11 development sites (4 sites without a coach)
• Approximate 1:4 coach to author ratio
• Initial sample of 137 new and updated requirements selected for improvement

22. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Issues with Initial Assignments
20
• Too few coaches at some sites
• Authors “clustered”
• Mitigation: coaches assigned from
other sites
• Number of requirements written
per author variable
• Only 26 of 64 authors involved initially
• # of requirements per author highly
distributional
• Mitigation: coaches with no or few
authors writing requirements assigned
to assist “overloaded” coaches

23. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
RAMP Results and Conclusions
21

24. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Results for Alpha Project
22
Sample Date # of Defects # of Logical Pages Defects/ Page (DPP)
Apr 2015 368 10.37 35.5
Jun 2015 38 3.40 11.2
Jul 2015 30 2.83 10.6
Aug 2015 19 2.56 7.42
Sep 2015 6 0.79 7.59
• RAMP started in April 2015
• Measured ReqDD for new and modified project requirements
• Pre-RAMP ReqDD: ~35 DPP
• Post-RAMP ReqDD: ~7 DPP achieved within 4 months

25. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Results for Requirements Catalog
23
Sample Date # of Defects # of Logical Pages Defects/ Page (DPP)
Q1 2016 43 7.06 6.1
Q2 2016 11 4.77 2.3
Q3 2016 5 2.09 2.4
Q4 2016 18 4.00 4.5
• RAMP started in April 2015
• Measured ReqDD for new requirements added to the catalog
• Pre-RAMP ReqDD: ~30 DPP (Q3 2014)
• Post-RAMP ReqDD: Single digit DPP

26. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Conclusions
24
• RAMP solved fundamental issue of lack of trained requirements SME
resources to assist with requirements rewrites
• RAMP coaches invested <10% of their time on a weekly basis but were
able to deliver ~10x improvement in requirements quality
• RAMP should result in:
• Less software rework due to requirements defects
• Improved efficiency
• Higher velocity
• Higher quality final product
• Based on its success, RAMP can be extended to other development
organizations

27. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
25
Requirements Defects
Requirements Coaches
Network of Coaches able to reduce the requirements defects “fire”

28. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Future Work
26

29. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Future Work
27
• Maintain coaching network over time
• Need to keep the coaching pipeline filled due to attrition
• Extend RAMP to other projects
• RAMP coaches assigned already to other projects
• Extend RAMP to other business units within Intel
• Training of coaches started in two other Intel organizations
• Measure the impact on overall product quality, execution to schedule,
program costs and customer satisfaction levels
• Measurements currently underway on product quality; other measures to
follow

30. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Q&A
28

31. Copyright © 2017 Intel Corporation. All Rights Reserved.
No part of this presentation may be copied without the written permission of Intel
Corporation.
Contact Information
29
Thank You!
For more information, please contact:
John Terzakis
john.terzakis@intel.com