Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

SWP Take Three - Lets Talk Agile - 27 Jul 2022b.pptx

Prochain SlideShare
Dev ops
Dev ops
Chargement dans…3

Consultez-les par la suite

1 sur 77 Publicité

Plus De Contenu Connexe

Similaire à SWP Take Three - Lets Talk Agile - 27 Jul 2022b.pptx (20)

Plus récents (20)


SWP Take Three - Lets Talk Agile - 27 Jul 2022b.pptx

  1. 1. 1 https://aaf.dau.edu/aaf/software/ DoD’s Software Acquisition Pathway Digital Delivery at the Speed of Relevance DAU – Let’s Talk Agile 27 Jul 2022 Mr. Sean Brady DoD Senior Lead for Software Acquisition OUSD(A&S)/Acq Enablers
  2. 2. DoD Leadership on Software “Now is the time to be bold!” “This requires the focus of DoD senior leadership…I expect all offices to provide the support necessary to make software modernization a reality.” Dr.Hicks, DEPSECDEF DoD Software Modernization Strategy 2 “We must improve our ability to acquire software.” Dr.LaPlante, USD(A&S) Confirmation Hearing Mission to Accelerate Software Modernization a Top Priority
  3. 3. Urgency to Modernize DoD Software 3 NDS DSB DIB 20+ software provisions in recent NDAAs DoD SW Mod Strategy Great Power Competition Software Central to Most DoD Missions, Systems, and Future Conflicts
  4. 4. • FY20 NDAA Sec 800 directed SWP establishment  Implements top DIB & DSB recommendations  Exempt: JCIDS, MDAP treatment  Accelerated delivery timelines (<annual)  Sub-paths: applications; embedded SW; (soon business systems) • Collaboratively developed DODI 5000.87 policy  Streamlined, tailored processes, documents, reviews for rapid SW  Active user engagements; MVP feedback cycles  Leverages enterprise DevSecOps platforms and services • Acquisition Enablers – accelerating programs  Robust web guidance, templates, workshops, consulting – enabling PM success Software Acquisition Pathway (SWP) SWP Enables Modern Software Development in DoD https://aaf.dau.edu/aaf/software/ 4 21st century digital product delivery based on Silicon Valley business models. Designed for rapid, iterative, and human–centered SW delivery. . . . Lean Startup cycles Software is immortal
  5. 5. 40 Software Acquisition Pathway Programs 5 Air Force AOC-WS ADCP C2IMERA JCC2 Mod & Sim NLCC-DSS T&G UP WARPspeed WDA Inc 5 SEWOL (USSF) Navy DCGS-N Inc 2 Navy EPS NCORS FLINT MTC2 NITESNext NCSA CEMI RTSO TRACKER JCW (USMC) LVC-TE (USMC) Army AIAMD IEWTPT RCV JCAP OWT TSS/TMT DOD NBIS MED-COP EMBM Catapult EDAE MARMS More on-ramping soon. . . SOCOM DCGS-SOF GAP MCS-COP SOF DE SOMPE Application 95% Embedded 5% Decision Support 17 C2 6 Cyber 5 Business System 5 Training 3 Digital Infrastructure 2 Weapons System 2 <$100M 20% $100-200M 20% $200-500M 37% >$500M 23% Navy 30% Air Force 27% Army 15% SOCOM 13% Defense Agencies 15%
  6. 6. Growth and Phases of SWP Programs 6 Oct 2021 Apr 2022 40 35 30 25 20 15 10 3QFY21 4QFY21 1QFY22 5 0 1QFY21 2QFY21 Air Force Navy Army OtherDoD DODI 5000.87 signed Steady Adoption of the SWP +5 new programs added since
  7. 7. GAO: SWP Implements Leading Commercial Practices GAO-22-104513 LEADING PRACTICES study looked at DHS, NASA, and DoD. Found DoD’s SWP exemplar in including policy and guidance to implement the key practices of leading companies such as Amazon, SpaceX, IBM, Qualcomm, and Virgin Orbit. GAO noted that the SWP enables • Iterative design and Agile development cycles to deliver MVPs • Prioritizing schedule throughout development (capability off-ramps) • Right sized, empowered teams • Soliciting early feedback from end users • Encourages applying SWP practices to other acquisitions 7
  8. 8. “Now is the time to be bold.” Vision: Deliver better software faster Tasked SW SSG to pursue 3 major efforts: 1. Execute DoD CIO Capability Programming Guidance for Cloud and DevSecOps investments 2. Implement innovative authorities DoD-wide such as the Software Acquisition Pathway per DODI 5000.87 3. Increased use of software factories, secure DevSecOps pipelines DSD: DoD SW Modernization Strategy 8 DoD Software Modernization Strategy Demands true process transformation to enable the software delivery pace required to compete.
  9. 9. https://aaf.dau.edu/aaf/software/ Software Acquisition Pathway 9
  10. 10. Commercial Tech Requirements & Planning Model Overarching Requirements Product Roadmaps Program Backlogs Capability Needs Statement (CNS) OR Software Initial Capabilities Document (SW-ICD) FY22 FY23 FY24 FY25 FY26 1 2 3 n 1 2 3 1 2 Overarching document Scope of user’s operational needs, threats, environments. Flexibility w/in 2-5 yr outlook. Visual plans for next 18- months to 5-years (FYDP) Foster stakeholder collaboration, plans, investments, priorities, interdependencies, etc. Dynamic, prioritized lists of tactical user needs. Each release focused on highest priority needs. Performance & user feedback shape future iterations. Details More Less Active Sponsor/User Engagement Early and Throughout Development Build Measure Learn: Evolving Mission, Adoption, Performance, Threats, Priorities, Tech 10
  11. 11. Minimum Requirement: Annually but can be more often as appropriate. Value Assessment Reporting Cycles 11 MVCR #1 Potential Scenarios Annual Cycle Value Assessment Release #2 Value Assessment Release #3 Value Assessment Release #4 Value Assessment MVCR #1 Release #2 Value Assessment Release #3 Release #4 Value Assessment MVCR #1 Release #2 Release #3 Release #4 Value Assessment #1 #2 #3 SWP Official Guidance Value assessments will be performed at least annually after the software is fielded to determine if the mission improvements or efficiencies realized from the delivered software are timely and worth the current and future investments from the end user perspective. More frequent VAs are encouraged • Timing of VA: negotiated between sponsor and PMO; capture in User Agreement • PMO gauge frequency of VAs based on desire by ops sponsor to provide formal feedback • Programs encouraged to make use of informal feedback to ensure regular user feedback Human Centered Design demands Ops community and PMO to co-create strategies and governance that support effective VAs VA Template: https://www.milsuite.mil/book/docs/DOC-952079 Deploy Frequency: visit AAF/software
  12. 12. The Subpaths Building a new path for some DBS programs within SWP 12 Updating guidance and templates for DBS/SWP programs Coming soon! Collaborative Partnership to Align/Integrate SWP and DBS Application Path Rapid development and deployment of software running on commercial hardware (including modified hardware) and cloud computing platforms. Embedded Software Path Rapid development and insertion of upgrades and improvements for software embedded in weapon systems and other military-unique hardware systems. Defense Business System Path Acquisition of DBS with high levels of configuration, interfaces, workflow development, and/or custom code
  13. 13. Decision Matrix to Use SWP SWP Using Modern Software Development Methods Using Agile, Iterative Release Approach Delivering Releases At Least Annually 13 Tailor Processes Decision Points Documentation User Governance for Defense Business Systems If the business system meets all 3 criteria, consider use of the SWP
  14. 14. Defense Business System Acquisition Approaches 14 Use SWP Planning Phase for Initial BCAC Steps Tailor Agile to Incorporate BCAC Tailor BCAC to Incorporate Agile Repeat BCAC process steps in Execution Phase to accelerate sw delivery Accelerate Into Execution Adopt modern SW development methods Incorporate SWP best practices • Iterative Releases • Flexible Planning • Shift Left Testing • Automation Use and tailor certain SWP artifacts • Use CNS to document high-level needs; update often • Use User Agreement to capture governance process • Use Program Roadmap to convey plan & progress • Use Program Backlog to prioritize user needs • Use Performance Metrics to monitor progress • Use Value Assessment to measure against outcomes Capability Need Identification Solution Analysis Functional Requirements& Acquisition Planning Business ProcessReengineering Many DBS programs already adopt agile practices
  15. 15. • Interoperability is crucial to  modern software  joint warfighting  JADC2  CDAO & AI superiority  Deputy SECDEF’s Data Decree  and is the #1 priority of Joint Staff • In contrast to commercial industry and modern web economy… …DoD lacks a coherent API ecosystem  Result: DoD suffers from informational silos… limiting connectivity; innovation; productivity  21st Century businesses know: Investing in an API strategy pays significant dividends  APIs/exposed interfaces (controlled access) promote interoperability, security, and scalability • How can we help DSD create data advantage and set conditions for change? • Create a DoD API Strategy and Ecosystem • Install concepts that build a data-exchange mindset into acquisitions – “design for / continuous interoperability” • Goal: – Anyone w/ validated mission need can access data/functionality anywhere in DoD – Future proof systems for potential joint warfighting Why a DoD API Tiger Team? Time for a DoD API-First Business Strategy 15
  16. 16. 1. Brief API manifesto/directive/memo  Reinforces DSD Data Decree, DoD Software Mod Strategy 2. API Template for SWP programs  Structure of key elements on API that SWP should capture in acquisition strategy or related document 3. API Guidance  Publish robust use cases & how-to on SWP website for PMOs 4. 5000.87 Policy Update  Include API expectations of SWP programs in .87 update 5. Pilots and Wargames  Real-world use cases captured in vignettes Through line – Path to Updated DODI 5000.87 Policy for APIs 16
  17. 17. • Related to the efficiency of development and delivery processes to identify issues and continuouslyimprove • Story Points, Velocity, Cycle Time, Lead Time, Cumulative Flow Robust Set of Metrics in the SWP SWP metrics-driven & draws on DSB, DIB, commercial sector practice Different Information Needs at Diff Levels (OSD Exec; PEO; PM Chief Engineer) Program Management OSD Enterprise/Exec • Real-time metrics to address team issues • Context understood due to proximity to teams • Analyzed in full context in Progress Reviews • Focused on trends over time • Metric data aggregated • Program-specific context may be required Large set of metrics exist and in use on the SWP A&S has robust guidance on modernmetrics Still iterating on the right metrics at each level; data highly contextual Process Efficiency • Related to the quality of the work delivered; issue resolution; and –illities (non functionalrequirements) • Defect count, escaped defects, code coverage, automated test coverage; availability;usability Quality • Related to the amount of value delivered to users and the efficiency of that valuedelivery • MVP/MVCR Date, Progress Against Roadmap; Backlog & SprintBurndown Progress • Related to the efficiency of the value delivery pipeline (DORAmetrics) • Deployment frequency; Deployment failure rate, security incident rate, mean time to remediate DevSecOps • Provide insight into the program budget and expenditure rate to guide resource decisions • Total cost estimate, cost per agile team, non-labor agile team costs, PM costs, burnrate. Cost • Related to evaluating the impact of the work, amount of value delivered, and overall customer satisfaction • Value assessment ratings, value delivered points, customer satisfaction, net promoter score Value 17
  18. 18. Multiple Lines of Effort for SW Modernization Process Reform Initiatives JCIDS: Partnered w/ J8, Services to drive big change to SW requirements process/docs - e.g., 60% reduction to JS validation timeline to only 40 days T&E: Partnered w/ DOT&E to dramatically streamline T&E docs, support shift-left testing - e.g., 90% reduction to legacy TEMP(10-20 pages) DBS: Partner w/ Services to develop all new DBS sub-path per Congressional direction Cost: Developed streamlined estimating and agile reporting approaches; - e.g., new modern SWP CARD (template reduced by magnitudes) SW Mod Strategy and NDAA demand streamlining enterprise processes for software Partnerships and Collaboration Software Alliance: identify pain; solve problems for MAJCOMs and PEOs Software PM Forums: Enable SWP programs to share practices, lessons, insights Cross-DoD: SW Acq modernization with ADNI; SOCOM, CYBERCOM, CDAO; JADC2 Outreach: Dozens of workshops, webinars w/ executives, practitioners Program Enablement A&S Team worked with dozens of programs to shape strategies, practices, roles, and guide them on new pathway to enable greater software success; - e.g. cut a PM’s SWP planning phase in half (12 mo to 6) Unity of Effort to Reform, Streamline, Tailor DoD Environment for Software DBS Ignite 18
  19. 19. SWP Insights & Work Ahead 19 • Encouraging early indicators • Active and early user/stakeholder collaboration • PMs/Users emphasize SWP amplified Warfighter voices • SWP driving continuous improvement to lean processes • Statute requires faster delivery cadences: • DoD cycle times for SW historically were multi-year • 71% SWP programs report delivery cadences <6 months • Statute requires streamlining processes for SW • SWP drives accelerated planning – 51% of SWP programs spend ≤ 6 months in Planning Phase Enterprise view More work needed to optimize DoD for speed • Costing/Budgeting: budgeting processes are bottlenecked; upfront detailed estimates do not align with dynamic SW model • Interoperability: move to API-based ecosystem and modernize DoD to continuously design/test for data-exchange • Drive lean processes: drive into practice at service level; • Transform legacy mindsets, policies Program view Navigating friction at Agile/Traditional Process intersects • Testing: DOT&E still in process of defining “shift left”/agile testing vs. “big bang”/traditional testing • Cost Estimation: full-lifecycle cost estimates versus “software is immortal” approach • PEOs/PMs leverage A&S/SWP team for advice/troubleshooting – High demand for support • SWP can drive larger SW ACQ reform – non-SWP programs adopting and learning from our concepts SWP, as an agile practice, is evolving requiring iterative development of policies, practices, and metrics
  20. 20. A&S Acquisition Enablers Mission Focused We enable programs to deliver better software faster Quick Review of Artifacts Consult / Co-develop innovative strategies Strategic Outreach Advocate for Buy-in / Support Growth mindset: learning while leading FY22 focus: - Scale Transformation (Fusion cell - spread TTPs across DoD) - Optimize / Modernize Remaining Bottleneck DoD Processes 20
  21. 21. SWP on AAF Website 21 https://aaf.dau.edu/aaf/software/ Integrated policies, guidance, and resources to navigate the SWP with greater speed and success. Recent Guidance Updates • Value Assessment • Cost Estimation • ADM Template • Registration, insight reporting • Management baselines • Webinar video/slides • SW in NDAAs • Transitioning • FAQs • Many References What Content Do You Find Valuable? What Else Do You Want to See? > 48,000 page views over last 12 months
  22. 22. Strategic Initiatives to Reform People, Process, Tech AE Strategy Creates Virtuous Cycle Between Software Modernization Lines of Effort Enable Program Success Transform Processes, Policies, and Guidance Enable DevSecOps Tools, Services, and Platforms Develop a Robust Digital Workforce DRIVES IMPROVEMENT 22 SETS CONDITIONS
  23. 23. Ask Me Anything 23 What else do you want to know about DoD Software Acquisition? How can we enable you to be successful in delivering software?
  24. 24. BACKUP SLIDES 24
  25. 25. Commercial Tech Insertion / Market Analysis Points 1 2 3 SWP purpose built to support rapid innovation and align with commercial approaches. Central design principle: rapid, iterative cycles, supporting incremental tech investments to break adversary’s OODA loop. Short, frequent cycles more responsive to changing tech, ops, and threats. 4 Tech insertion opportunities: 11.Market analysis at time of Acquisition Strategy identify commercial offerings that can help get to the solution faster; investigate available SW development infrastructures that promote focus on app innovation. CNS is defined at a high level that allows future evolution; does not unnecessarily precludecommercial components as part of the solution. 22.Every iteration an opportunity for tech insertion – tech solution always evolving; working versions deployed frequently. 33 .Annual value assessments provide periodic opportunities for re-examination of commercial market to understand whether more cost-effective opportunities exist for the capability. 44 .Open architecture: .87 Policy mandates strategies that plan for rapid tech insertion: “(h) Architecture strategies to enable a modular open systems approach that is interoperable with requiredsystems. 25
  26. 26. Evolving Software “Requirements” Draft CNS or SW-ICD Operations User Periodic updates Active user engagements Roadmap(s) Backlogs MVP MVCR Release2 Releasen Evolving Mission, Adoption, Performance, Threats, Priorities, Tech Strategic Dynamic processes with active feedback loop 26
  27. 27. Focuses on understanding the users’ and systems’ needs and planning the approach to deliver capabilities to meet those needs Key Artifacts • Capability Needs Statement or SW-ICD • User Agreement • Program Strategies  Acquisition Strategy  Contracting Strategy + IP Strategy  Test Strategy + Cybersecurity Strategy  Product Support Strategy • Cost Estimate Planning Phase 27 https://aaf.dau.edu/aaf/software/planning-phase/
  28. 28. A high-level capture of need with enough information to define the software solution space and consider the threat environment. •Capability Needs Statement (CNS) or Software Initial Capabilities Doc (SW-ICD)  SW-ICD required if JS views it has Joint equities •Sponsor and Requirements Manager ID operational software capabilities needed • Draft doc to start the Software Pathway • Refine during Planning Phase and approve prior to Execution Phase Software Requirements Document 28 Clear Understanding of What Capabilities Are Needed https://aaf.dau.edu/aaf/software/define-capability-needs/
  29. 29. User Agreement 29 Agreement between the operational and acquisition communities Active User Involvement • Ensure users from across ops community actively support development • Provide ops insights and user feedback on interim and fielded software • Resourcing co-location, travel, or virtual ceremonies, events, and meetings Informed Decision Making • Clear understanding of roles and responsibilities • Capturing and prioritizing requirements • Shaping roadmaps and backlogs • Integrating user inputs for value assessments https://aaf.dau.edu/aaf/software/user-agreement/ Drives Active User Engagement in Development
  30. 30. • Product Roadmap • Program Backlogs • Active User Engagements • Develop, Deliver Software • Track Metrics • Value Assessments Execution Phase 30 https://aaf.dau.edu/aaf/software/execution/ Continuous improvement to maximize mission impact.
  31. 31. Application Rapid development and deployment of software running on commercial hardware and cloud computing platforms. Embedded Software Rapid development and insertion of upgrades and improvements for software embedded in weapon systems. Business System (soon per FY21 NDAA) Rapid development and deployment of business system capability consisting of fully custom software or commercial software with significant customization. Three SWP Sub-Paths 31
  32. 32. MVP, MVCR, and Iterative Deliveries 32 Minimum Viable Product (MVP) Early version of software for users to evaluate and provide feedback. Minimum Viable Capability Release (MVCR) Fully tested set of value-added features fielded to an operational environment. Statute Requires Software Deliveries at Least Annually ASAP Within 12 months of obligating funds https://aaf.dau.edu/aaf/software/mvp-mvcr/
  33. 33. • The sponsor and user community shall perform a value assessment at least annually  Timing: negotiated between sponsor & PMO; capture in UA • Report card for acquirers and developers • Use to shape strategies, designs, requirements, and investment decisions • Focus on maximizing value to the Warfighters Value Assessment Value 33 Investment
  34. 34. SWP Interplay of Key Elements 34
  35. 35. Area Objective Requirements Drive a streamlined, dynamic approach to req mgmt. Metrics Shape what programs track, report to assess progress, impact Software Delivery Accelerate initial software releases, streamline future ones Interoperability Modernize design and verification, maximize use of APIs Test & Evaluation Increase automation, shift left and right, bake in cybersecurity Cost Estimation Streamline upfront estimates, doc, iterate during development Workforce Modernize training, guidance, and resources to enable success 35 Continuous Improvement to the SWP
  36. 36. Area Objective Requirements Drive a streamlined, dynamic approach to req mgmt. Metrics Shape what programs track, report to assess progress, impact Software Delivery Accelerate initial software releases, streamline future ones Interoperability Modernize design and verification, maximize use of APIs Test & Evaluation Increase automation, shift left and right, bake in cybersecurity Cost Estimation Streamline upfront estimates, doc, iterate during development Workforce Modernize training, guidance, and resources to enable success 36 Continuous Improvement to the SWP
  37. 37. How Software Is Different Traditional Acquisition (Hardware Centric) Capabilities time Define all requirements Design system Develop system Test system Produce system IOC FOC Capabilities time Design, Develop, Test, Deliver software Software Acquisition High level needs Iteratively define details 37
  38. 38. • High-level, adaptable visual summary • Maps out the vision and direction of product offerings over time • Illustrates the planned capabilities and delivery timelines • Regularly updated with inputs from key stakeholder groups to ensure alignment with priorities, budgets, platforms, and related systems Product Roadmap Consider multiple roadmap views: • 3-5 Year Long-Term (FYDP) and • 6-18 Month Detailed 38
  39. 39. Program Backlogs Less defined • Identify detailed user needs, not tasks/activities, often in “user stories”. • Agile planning focuses on “what” users want and leaves “how” to the team. • Backlogs are dynamically reprioritized, and stories can shift across backlogs • Development teams estimate size of each story and balance what can be done within each time-boxed sprint and release • Identified Issues, errors, and defects are added to the backlogs to address • Longer-term Program backlog typically contains items that are not decomposed 39 More defined
  40. 40. Operational Sponsor Overall program champion from an operational, requirements, and resourcing perspective. Product Owner Ensures active user engagement, shapes roadmaps and backlogs, and measures value. Users/User Reps Provides developers insight into operations, shapes tactical needs, feedback on interim/final software. Decision Authority Responsible for program oversight and key decisions. PEO for many, higher/lower official based on size. Program Manager Manages the program in partnership with the Product Owner to regularly deliver valued software. Notional Roles and Responsibilities in UA 40 Ops/Req Orgs Acquisition Orgs See details at: https://aaf.stage.dau.edu/aaf/software/user-agreement/
  41. 41. Strategy Development in SW Pathway 41 Planning Phase Execution Phase 1 0 Develop strategies and required artifacts for the DA to approve the program to begin Execution Phase. Continuous improvement of strategies based on user feedback, team and system performance, shifting priorities, integrating best practices, etc. Explicitly NOT a traditional acquisition milestone with dozens of major documents required. https://aaf.dau.edu/aaf/software/program-management/ Program documentation should be constrained to what is needed to effectively manage the program.
  42. 42. Entering the Planning Phase ADM signed by DA Draft Capability Needs Statement Entering the Execution Phase Capability Needs Statement or SW-ICD User Agreement Acquisition Strategy Cybersecurity Strategy Test Strategy IP Strategy Product Support Strategy Information Support Plan Program Cost Estimate and ICE CARD ADM signed by DA The acquisition strategy can include required contracting, test, cyber, IP,ISP, and support info. During the Execution Phase System Architecture Product Roadmap Program Backlogs Strategy Updates CARD/Cost Estimate Updates Value Assessment Program Metrics Program Reporting Balance speed with rigor: Focus on delivering software, not documents SWP Information Requirements 42 See details at: https://aaf.dau.edu/aaf/software/develop-strategies/
  43. 43. What Should a SWP AS Contain Minimum Elements  High-level User Needs  Operational Context  Initial Program Roadmap | 43 | Acquisition Plan Capability Description Contracting Strategy Technical Approach Test Strategy Program Governance Cost/ Funding Product Support  Business Case Summary  Iterative Approach  Team Roles and Responsibilities  SW Development Method  Architecture/Design  Enterprise Services/Tools  Metrics  Prioritization Schema  User Engagement Plan  Value AssessmentApproach  Competition Plan  Incentives  Progress Measures  IP Strategy  Organic Resources  Transition Plan  Interoperability  Definition of Done  Operational Fielding  5-Year Plan  Infrastructure + Labor  Appropriations Acq Strategies can cover most required info vice many standalone docs 43
  44. 44. Strategy: a data-centric DoD 44 DSD Hicks strategy: transform DoD into a data-centric org Goal: "improving warfighting performance and creating decision advantage at all echelons from the battlespace to the board room.” How do we operationalize this? Organizational behaviors/capabilities align Creating Data Advantage decree What must be built-in early to our acquisitions to ensure we have systems and platforms that meet the needs of a digital enterprise (APIs, interfaces, data rights and usage, etc.)?
  45. 45. Free the Data for Software “You have to make all of your data available. Not everyone is going to be able to access your data. There’s going to be credentials. Classification. Various impediments. “But you can’t hoard your data. It can’t be proprietary. You have to be able to make the data available to the broad world.” - Gen Hyten Implies that DoD must: • Move away from specific / narrow technology and standards compliance (e.g., limiting use to a particular standard/tech such as Stiches, CORBA or REST) • Move to open standards and mandated API for all applications and data (i.e., as long as interface is exposed; we can use any tech) • All SWP programs must have an API strategy • All software functions and data must be designed using API • Allows controlled access to internal and external app operations • Promotes interoperability and security (e.g., Zero Trust) and scalability 21st Century businesses know that investing in an API strategy can pay significant dividends • Time for DoD to invest in an API Strategy… Could unleash DoD innovation DoD API ecosystem: open or partner? 45
  46. 46. Modernizing Processes for Software Requirements Testing Cost Estimating Acquisition Contracting Developed new streamlined JCIDS document and accelerated approval process Developed detailed guidance and tailorable templates to speed entry into execution Developed test strategy template and guidance for incorporating testers early in the software lifecycle Developing detailed guidance and examples, working with industry to understand challenges Developed streamlined estimating approach and new templates for documenting program info A working group has been established for each area that includes Service representatives in collaboration with the appropriate OSD offices 46
  47. 47. Acquisition Enablers SWP Toolkit AE Serves as Rapid Response Force / SW Cadre to Address Program Pain Points Below products: direct response to program urgent needs • Outreach: PEO Roadshows, Targeted Discussions, Coaching • Guides and Playbooks: Interoperability, DBS sub-path, XaaS, Deployment Frequency, APB Alternatives, EVM, RAI, Metrics, Value Assessment, etc. • Vignettes: Portfolio adoption; MTA-to-SWP transition; MTA + SWP adoption • Web: SWP website, FAQs, welcome kit, COI, templates 47
  48. 48. 40 35 30 25 20 15 10 5 0 3QFY21 4QFY21 1QFY22 1QFY21 2QFY21 Air Force Navy Army OtherDoD Software Acquisition Pathway DODI 5000.87 signed • 21st century SW Requirements • Interoperability • Cost Estimation • T&E Modernization • New Biz System sub-path • Weapons Rapid Adoption of the SWP Reforming DoD Processes ToEnable Modern Software Practices https://aaf.dau.edu/aaf/software/
  49. 49. Expanding Applicability Defense Business Systems Addressed congressional direction to expand SWP to Defense Business Systems Developed SWP sub-path along with guidance and training for when to use A&S memo to acq workforce in final coordination https://aaf.dau.edu/aaf/software/ dbs-in-swp/ 49 Weapon Systems Applying SWP to enable safety-critical and nuclear- certified use Working directly with pilot programs to refine and expand guidance for DoD Will enable accelerated software delivery to DoD’s most sensitive systems
  50. 50. Ignite Innovation and Execution Requirements Cost Estimating and Budgeting Test and Evaluation Weapon and Business SW Ruthlessly Focused on DoD Unity of Effort to Dominate Digital Product Delivery Strategic Partnerships to Reform, Streamline, Tailor DoD Environment for Software DBS Ignite 50
  51. 51. Contracting Strategy 51 • Planned contracts (i.e., FAR 12, New/ Existing IDIQs, GSA Schedules, OTs) to develop modular contracting strategy* • Strategy to access top software development vendors • Approach for using commercial services • Source selection techniques that will be employed (i.e., oral proposals, challenge-based demonstrations) • Incentives to ensure high quality software delivery • Strategies to sustain competition throughout program lifecycle (preferably using modular contracting) *Modular Contracting - FAR39.103
  52. 52. Modular Contracting 52 Instead of a single monolithic contract… Portfolio of contracts enabling a program to scale and evolve over time Example Modular Contracting Strategy Requirement Contract Strategies Infrastructure-as-a- Service (IaaS) (Cloud solution) ESI Agreements DevSecOps Platform-as- a-Service (PaaS) (CI/CDPipeline) Existing IDIQ/MAC/ GWAC, BOAs (i.e., Platform One) Agile Software Development Teams (Services) GSA Schedules/BPAs, FAR 12, FAR 13.5, New/Existing IDIQs IT Services (Data Management, Independent Cyber Assessment) GSA Schedules/BPAs, FAR 12, FAR 13.5, New/Existing IDIQs Benefits • Collection of contracts for different types of requirements • Leverage mix of existing vehicles and new contracts to get “right” vendors • Mitigate risk of changing requirements • Continuous competition; performance incentive for future work Preferred approach for acquiring major information technology systems in accordance with 41 U.S.C. §2308; implemented at FAR 39.103
  53. 53. Contracting for DevSecOps Solutions 53 There is not a uniform set of DevSecOps practices or a standard DevSecOps environment Infrastructure-as-a-Service (IaaS) Computing, storage, network resources, and managed services to enable function, cybersecurity, and non-functional capabilities • Provides hosting environment for DevSecOps Platform and Software Factory • Solution may be a combination of cloud and on-prem IaaS subscriptions A program may need to contract for solutions such as the following to establish a DevSecOps Environment: DevSecOps Platform-as-a-Service (PaaS) Hardware and tools to establish DevSecOps platform Software Factory Processes and practices to establish dev environment where developers create software artifacts • Contains one or more CI/CD pipelines with automated set of processes and tools tailored for a specific program or mission • Multiple CI/CD pipelines may be needed for different software systems Contracting for IaaS Solutions • DISA ESI Agreements offer cloud solutions and support services • Leverage existing enterprise IaaS services before contracting for unique solutions Contracting for DevSecOps PaaS and Software Factory • Leverage existing BOA or IDIQ for DevSecOps platform and unique CI/CD pipeline • Existing DevSecOps PaaS/Software Factory solution offerings: Air Force Platform One; Navy Black Pearl; Army AFC Software Factory
  54. 54. Contracting for Agile SW Dev 54 Agile Software Dev Contracts Objective: Support small, frequent releases, respond to change, consider programmatic risks, and program scope/objectives Multiple contracts for software development can be executed to develop discrete capabilities of the overall solution managed by the program Ensure individual dev contracts align with architecture, vision, and other dev activities: • Gov has overall integrator role • Gov prioritizes product backlog • Gov manages overall dev processes Continuously assess performance via: • Collaborative Agile dev processes • Mutually agreed-upon definition of “done” for each sprint/release • Metrics informing quality of capability developed and delivered Contracting for Agile SW Dev Solutions • GSA Schedule awards and BPAs • FAR 12 and 13.5 for commercial solutions • Existing IDIQ awards; establish new IDIQ • Small business set-asides • Basic Ordering Agreements
  55. 55. Contracting for Agile SW Dev New SWP Contracting Content Coming Soon! https://aaf.dau.edu/aaf/software/contracting-strategy/ 55 Services vs Product Contracts • DoD programs have used both product and service contracts for software development teams • Both can be structured to describe objectives and desired outcomes rather than a detailed set of requirements • Specific definition of “done” not defined in the contract; established at start of each sprint or iteration, in collaboration with the government • Use a Statement of Objectives (SOO) or Performance Work Statement (PWS) to describe objectives and desired outcomes Software Development - Services Software Development - Product Gov contracts for time and expertise of a developer team to deliver applications or solutions. Gov contracts for delivery of a specific application or solution.
  56. 56. Contract Types • Contracts for software development teams have been executed using all contract types (FP, T&M, CR) • Contract type selection will depend on a variety of factors including:  Selected contract strategy (no CR for FAR 8.4/12; IDIQ contract type limitations)  Level of government involvement in development activities Contracting for Agile SW Dev Fixed-Price Time & Materials Cost-Reimbursement Use When: Estimates can be reasonably identified and estimated, usually by historical costs. Development team velocity is unknown. Government is integrator. Necessary labor hours/labor mix not well understood. Not concerned about excluding small businesses without certified systems. How to Use: Fixed price level-of-effort provided over specified time period enabling product backlog requirements to continuously evolve. Scope increments into task requirements aligned to the product backlog. Labor rates and ceiling price must be established. Fixed work cycles and software release cycles to enable government to prioritize work. New SWP Contracting Content Coming Soon! https://aaf.dau.edu/aaf/software/contracting-strategy/ 56
  57. 57. Contracting for SW Dev - Best Practices New SWP Contracting Content Coming Soon! https://aaf.dau.edu/aaf/software/contracting-strategy/ 57 Award Timelines • Select contract strategies that offer streamlined procedures:  i.e., existing IDIQs/BPAs, FAR 12, FAR 13.5, CSO Source Selection Techniques • Leverage oral proposals/tech demos to assess vendor solutions, reduce B&P cost/cycle time, streamline source selection process • Leverage phased evaluations and advisory down-selects to reduce number of proposals evaluated in full Deliverables • Extensive list of deliverable documents not required for Agile S/W development • Consider alternatives to traditional CDRLs:  Data Accession List (DAL)  Leverage DevSecOps environment/processes to deliver data
  58. 58. Contracting for IT Services Examples of IT Services Supporting SWP Programs • Cloud Services (IaaS) • Data Management Services • Identify Management Services • Quality Assurance Services • Lead System Integrator Services • Independent Cyber Assessment Services • Software Development Team Services • Agile Coaching Services • Subject Matter Expert Services (Software Eng, Architecture, etc.) Leverage Acquisition of Services (AoS) procedures to award IT services contracts as part of modular contracting strategy for SWP program execution SWP Acquisition Strategy; AoS Contracting Strategy: Why? • DODI 5000.74 section 1.1.b(8): the AoS pathway does NOT apply to: “Services that are managed and reviewed as part of acquisition programs under other pathways of the Adaptive Acquisition Framework.” • AoS Acquisition Strategy doesn’t address programmatic elements required of a SWP program • Services Category (S-CAT) decision authorities not authorized to serve as DA for defense acquisition program AoS procedures can be used to acquire IT Services but not to execute a SWP program 58
  59. 59. • Initial cost estimate completed prior to execution phase and updated annually • CAPE ICE currently required for SW programs over ACAT II threshold (based on DoDI 5000.73) • A CARD is a policy-driven information requirement to initiate the cost estimate. Cost Estimate Requirements 59
  60. 60. Contracting Insights 25 12 10 6 6 5 3 3 1 1 1 0 5 10 15 20 25 30 Programs Use a Mix of Approaches But Have Clear Preferences IDIQ (FAR 16.5) OT (10 USC 4022) Negotiated (FAR15) Commercial (FAR 12) Basic Agreements (FAR 16.7) Fed Supply (FAR 8.4) Simplified (FAR 13) Small Business (FAR 19.5) Interagency (FAR 17.5) BAA (FAR35.016) PIA (15 USC 3715) Contracting Strategies 28 28 9 3 3 2 1 0 5 10 15 20 25 30 Contract Types 60
  61. 61. 1. Timely. Should support short decision timelines. SWP programs are expected to move rapidly from planning to execution phase. A cost estimate that cannot inform key decision points or budget requests will be less useful. 2. Iterative. Cost estimates should be developed with the available information and refined as more information is presented. It is impracticable for an initial cost estimate to capture the entire scope of the program. 3. Informative. Cost estimates should incorporate all available information including artifacts that may not conform to standard legacy practices. 4. Flexible. Cost estimates should be responsive to shifting operational needs and program priorities. Software programs adopting modern software development approaches need an equally responsive cost estimating process. 5. Credible. Cost estimates should strive to achieve being as informative and flexible as possible while providing adequate justification to defend the proposed budget through the resourcing process. 61 What Do SW Programs Need from a Cost Estimate
  62. 62. Agile Cost Estimating Deltas 62 Cost Estimation for Traditional Development Cost Estimationfor Agile Development Exhaustive upfront analysiswith slight adjustments over time Estimating Process Initial lower-fidelitybaseline iteratively refined over time Focused on lifecycle as dictated by predictive program plans Duration Focused on near-term roadmap to inform resourcing cycle Milestone-Driven Update Timelines Regular updates as new information becomes available SME involvement upfront; disconnected from execution Team Interactions Cost analysts tightly coupled with SMEs throughout program Solution defined in detail upfront w/periodic updates Program Definition Near-term iterations defined with less fidelity for longer term objectives; regular updates Large set of program data available; solid analogous programs to reference Cost Data Emerging set of program actuals; along with metrics used to inform future iterations Focused on cost and schedule to meet fixed scope Uncertainty and Risk Analysis Focused on variable capability given cost and schedule No benefits are realized until the end of development when the software is fielded Benefit Analysis Benefits are realized more often with each fielded iteration of software
  63. 63. Develop and Deliver Software Operations release feedback • Small, frequent releases to maximize value delivered • Feedback is captured that changes work priorities • Modern software practices and tools (Agile, DevSecOps) • Automation everywhere - testing, cybersecurity, releases • T&E/Cyber pulled as close to development as possible • Leverage enterprise services, DevSecOps pipelines Development 63
  64. 64. • Your User Agreement defines users and anticipated engagement activities. • During execution, leverage the UA to ensure that appropriate stakeholders are actively and consistently engaged in activities and events captured in the UA. • This helps ensure robust feedback loops to guide future development efforts. • Without user feedback, you can’t adapt to maximize value! • If users aren’t engaged, take corrective action! Leverage Your User Agreement to Engage User Community 1 Development release feedback User Community 2 User Community 3 64
  65. 65. SWP Baselines and Progress Legacy HW Centric Systems • Define requirements upfront, years in advance of delivery • Detailed cost estimates • Baseline C/S/P in APB before any insights from dev • Measure performance vs APB, no ability to assess value until the end • Track contractor via schedule/ budget compliance and EVM data • Focus on compliance to plan, meeting schedule/budget • Punish program for poor CE Modern SW Practices • Iterative requirements that are responsive to change • Iterative, scalable, build to budget • Active user engagements and feedback incorporated iteratively • Real-time performance and value data based on working SW • Contractor performance based on delivery of valuable SW • Focus on mission impact, improving value delivery to users • Reward programs that deliver value 65 SWP programs should NOT baseline cost, schedule, and performance using traditional approaches. APB is NOT required and highly discouraged. DODI 5000.87 does NOT require an APB Programs migrating to SWP should sunset their APBs
  66. 66. Status Reporting • Registration (within 60 days of joining SWP)  High level info on program • Insight Reporting (every 6 months Apr/Oct)  Program Information  Key Planning & Execution Dates  Cost Estimate / Budget  Software Development  Contract Approach  Cyber Resilience  Performance Metrics Leanest Reporting of AAF Acquisition Pathways 66
  67. 67. Why are Program Management Metrics Important? • To inform decision makers to take action, guide decisions, and solve problems! • To improve the process for value delivery • To deliver more value faster • To provide areas of focus for continuous improvement experiments • To highlight areas of great success to mimic with other teams and programs 67
  68. 68. Types of Metrics to Consider •Related to the efficiency of development and delivery processes to identify issues and continuously improve •Story Points, Velocity, Cycle Time, Cumulative Flow, Burnup, Burndown Process Efficiency •Related to the quality of the work delivered and identifying quality issues for resolution •Defect count, escaped defects, code coverage, automated test coverage Quality •Related to the amount of value delivered to users and the efficiency of that value delivery •Deployment frequency, MVP/MVCR Date, Progress Against Roadmap Progress •Related to the efficiency of the value delivery pipeline •Deployment failure rate, security incident rate, mean time to remediate DevSecOps Cost •Provide insight into the program budget and expenditure rate to guide resource decisions •Total cost estimate, cost per agile team, non-labor agile team costs, PM costs, burn rate. •Related to evaluating the impact of the work, amount of value delivered, and overall customer satisfaction •Value assessment ratings, value delivered points, customer satisfaction, net promoter score Value 68 68
  69. 69. Metrics Guidance 69 69 Begin the end in mind – what key insights do you need to guide decisions at the team, program, and stakeholder levels? Don’t boil the ocean - focus on metrics that provide valuable insight. Always use metrics to identify and solve problems (not to punish or blame). Ensure alignment between leadership and team(s) on the key metrics and usage. Agile metrics place heavy emphasis on value delivered and team performance versus cost and schedule. Utilize metrics demonstrating value delivered against program goals/objectives and Agile commitment. Focus on real-time (versus lagging) metrics, so decision-makers have actionable information. Automate metrics, data collection and analysis to the greatest extent possible using Agile-based tools. Establish a blameless/safe culture to experiment, fail fast, continuously improve, and share knowledge. Review metrics on a recurring basis to continuously improve.
  70. 70. APBs vs a Smarter Paradigm focused on Value Creation and Agility In Era of Disruption: Playing Status Quo is a Losing Game / Creates Risk Let’s Not Lionize Traditions over Innovation (have they reduced DoD’s SW risk?)
  71. 71. APB vs Superior Insights/Value from SWP Frequent OODA cycles so programs can be responsive! Rapid feedback/decision-making driven by new info Single OODA - early decisions with the least knowledge lock in requirements
  72. 72. DevSecOps Reference Design Pillars 72 DoD Enterprise DevSecOps Reference Design “DevSecOps is the preferred software practice for DoD to deliver at speed of relevance” – DoD CIO, USD(A&S)
  73. 73. DevSecOps Maturity Very Difficult to Adopt – Requires time - $ 73 Monolithic Architecture, Manual Processes Agile, Microservices, Test Driven Development DevOps Continuous Integration End to end cycle time – Design to Delivery Iterative with Hybrid or SOA Monolithic Architectures DevSecOps Maturity High Low Shift Cybersecurity Left Continuous ATO (cATO) enables bug and security fixes in minutes instead of months to years and provides rapid deployment of critical capabilities to the war fighter at the speed of relevance. Agile DevOps DevSecOps ContinuousATO (cATO) DevSecOps Continuous Monitoring Telemetry Capture Service Mesh Secure Containers Adoption Challenge Difficult Significant investment of time, effort and tools are required to achieve high DevSecOps maturity Brady Stark Smith Triangle of DSOSuccess
  74. 74. How the SWP Can Improve the BCAC Process • Improves the ease of tailoring Decision Authority decision points or status checks which can reduce any delivery delays due to lead and coordination time • Negates Full Deployment and Capability Support phases of a program – adopts a software never done approach • Eliminates excessive tailoring that may be required for defense business efforts that deliver smaller and more frequent efforts • Adopts an agile cost estimating process focused on near-term SWP Designed for Maximum Flexibility with Rigor 74
  75. 75. Culture • Unlearning old practices – defining, controlling programs • Contractor, acquirer, operator, security separate groups Processes • Significant documentation and long timelines to acquire • Linear requirements, design, develop, test, and deliver Platforms and Services • Limited enterprise platforms, APIs, and services to use • Lack of investments ($, expertise) to shape and scale Workforce • Workforce lacks experience and training in modern SW • Struggle with new roles and teaming structures Industry and Contract Incentives • Traditional primes limited in modern SW expertise • Incentivized to remotely develop closed SW systems • Contract not structured for Agile/DevSecOps 75 Barriers to Modern Software Development
  76. 76. What YOU Can Do ToEnable SW Success 76 Train Execs, Workforce • Truly understand new software development paradigm • Teams and contractors go through training together Promote new Culture • Building on SOCOM’s values of teaming, speed, agility • Small empowered teams continuously improving • Value-creation focused: Fail fast; fail cheap Processes • Continue lean processes for SW requirements, T&E, etc. • Transform for small, frequent software releases • Robust how-to guidance, templates to enable workforce Platforms and Services • Invest in enterprise/portfolio platforms and services • Drive open standards, interoperability, & API ecosystem Industry • Seek out more software companies who “get it” • Integrate portfolios of SW vendors under Gov’t leads