Publicité
Publicité

Contenu connexe

Similaire à Open Source Compliance Automation Capability Map(20)

Publicité

Open Source Compliance Automation Capability Map

  1. Capability Map OC Tooling Reference Workgroup - v1.5.5 V1.5.5 by Open Chain Tooling Workgroup, July, 20th 2022 v1.5.4 by Open Chain Tooling Workgroup, July, 6th 2022 v1.5.3 by Open Chain Tooling Workgroup, June, 22nd 2022 v1.5.2 by Open Chain Tooling Workgroup, June, 8th 2022 v1.5.0 by Open Chain Tooling Workgroup, May 11th 2022 v1.4.0 by Open Chain Tooling Workgroup, 30.3.22 v1.3.2 by Dr. Peter Ellsiepen (ESA) & Jan Thielscher (TrustSource)
  2. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Changelog Version Date by Comments/Changes 1.2 3.12.19 Jan, Peter Initial draft 1.3 6.12.19 Jan Rename Case Data => Situation Data, delete „Compliance Artefacts“ as capability, change Mission of Snippet scanner 1.3.1 11.1.21 Jan Review spelling, add some Readme‘s in the surrounding, review & harmonize definitions 1.3.2. 11.1.21 Jan Added a few samples for capability mapping 1.4.0 30.3.22 Tooling WG Reviewed Capabilities Package Crawler, Scanners (Binary, Source and Container) as well as License & Copyright Scanner, added CI/CD rule enforcement 1.4.1 13.4.22 Tooling WG Reviewed changes, extended Snippet-Scanning, 1.5.0 11.5.22 Tooling WG Split Case Data into Case Data Analyzer & Collector Capabilities, re-arranged overview slide 1.5.2 8.6.22 Tooling WG Reviewed Legal Solver, Policies & Rules, 3rd party component data 1.5.3 22.6.22 Tooling WG Reviewed License Repositiry, Compliance Artefact Generator and Approval flow 1.5.4 6.7.22 Tooling WG Reviewed User & Role Management, Audit Log, started with Reporting & Analytics 1.5.5 20.7.22 Tooling WG Finalized Reporting & Analytics and reviewed Tool Orchestrator PLEASE NOTE: To keep an overview of working state, we mark the agreed capabilities with this symbol 2
  3. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Traceability of data sources, decisions and configs as a General Requirment We need to provide the general requirement, that all decisions, data and sources need to be tracible, so that it always is possible to track why and on what basis a decision has been made. This involves: • Provide all information available under which a certain decision is made and that point in time • Track changes and their originators • Archive sources / binaries that are used in a solution • Link notice files and other documentation with sources/binaries • Document decisions and choices made 3
  4. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Overview 4 Tool Orchestrator Reporting and Analytics Case Data Collector (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet & Similarity Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Metadata Repository Data Flow Data Sink CI/CD OSG Rule Enforcement EXCLUSION: At this point in time the model is not addressing Security or Export regulations Dependency Analyzer Source Container Binary Input Condition Management Case Data Data Analysis 1 2 3 17 18 19 4 5 16 15 14 6 8 9 10 7 11 12 20 13 Control Flow
  5. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Package Crawler/Finder Mission • Research information on (new) components such as locate the repository, current and former versions, project homepage and viability information Responsibilities • Collect and provide accurate information about the component • Alert, if component can’t be matched/found Tasks • Scan package managers for new packages or versions of packages • Collect package data • Transfer data into package repository Input • Component descriptor or component name Output • Component Information, such as: source repository url, version history, branches, commit count, stars, last commit date, etc. Comments => Distinguish between component loader & assessment or just cralwer for information 5
  6. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Dependency Analyzer (Source) Mission • Provide composition analysis of software to be built from these sources Responsibilities • Determine all packages and dependencies (incl. transitive) used to build the software • Determine the way of linking of dependencies Tasks • Integrate with build process (CI/CD) • Determine composition (_complete_ Bill of Materials) • Provide output for further analysis, e.g. as SPDX • Provide link between scanned source and BoM information, e.g. Commit ID Input • Build description, e.g. POM or requirements.txt Output • Bill of Materials (BoM) for particular build Comments Analysis and dependency resolution is highly language specific. Thus a language specific implementation might be required Discussion: Would it make sense to declare a task or responsibility to stop CI/CD in sit of violation? 6
  7. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Dependency Analyzer (Binary) Mission • Provide composition analysis of a software binary Responsibilities • Determine all packages and dependencies used within this binary Tasks • Download binary (if required) • Unpack binary • Assess content and determine used packages/components • Collect information and assemble Bill of Materials • Provide Bill of Materials (e.g. as SPDX) • Provide link between BoM and scanned artefact, e.g. binary repo ID • Hash to identify the binary scanned should be generated and archived Input • Binary or link to binary location Output • Bill of Materials (BoM) for particular binary • Status of processing (e.g. errors, inclompleteness, failures in processing) Comments 7
  8. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Depdendency Analyzer (Container) Mission • Provide composition analysis of a container Responsibilities • Determine all packages and dependencies used within this container Tasks • Download container (if necessary) • Assess container content/structure and determine used packages/components • Collect information and assemble Bill of Materials • Provide Bill of Materials (e.g. as SPDX) • Provide link between BoM and scanned container, e.g. Repo + image ID + tag ▪ Hash to identify the scanned container should be generated and archived Input • Container or link to container location Output • Bill of Materials (BoM) for particular container • Status of processing (e.g. errors, inclompleteness, failures in processing) Comments 8
  9. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - License, Copyright & Authors Scanner Mission • Precise scanning of sources to determine exact situation for proper compliance declarations Responsibility • Ensure completeness and correctness of compliance information Tasks • Identify & gather copyright statements • Identify & gather authors • Identify & gather effective licenses (e.g. license identifier & if available license text) • Identify & gather changes and / or additions to license terms Input • Repository or file(s) to scan Output • List of effective and declared licenses with links into code • List of changed licenses with links into code • List of copyright statements with links into code • List of author information with links into code • Status of processing (e.g. errors, inclompleteness, failures in processing) Comments • TODO: Clarify granularity required to differentiate between author, commiter and copyright holder 9
  10. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities – (CI/CD) OSG Rule Enforcement Mission • Ensure only compliant artifacts will leave the automated tool chain Responsibilities • Break build, deployment or packaging as long as compliance violations exist Tasks • Verify compliance state • Interrupt automated build/deployment processing in case of violations • Log event and causes • Alert Input • Automation event Output • „Confirmation“ or „break“ event – or any sort of recording of required action • Log entry Comments • The key of this is to ensure that no non-compliant artifact will leave the process. It must not be CI/CD driven, but it should ensure that a check happens 1 0 OSG = Open Source Governance Data Flow Data Flow
  11. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities – Input Condition Management Mission • Determine that all copyright holders of commits finally grant rights and will not claim back Responsibilities • Prevent code from entering the repository without the commiter having agreed to the terms seeked by repo-owner Tasks • Link confirmation into Pull-request • Provide sort of proof that code commited to repo went through this process • Log event and confirmations of commiters Input • Automation event Output • „Confirmation“ or „break“ event • Log entry Comments • One option could be to apply CLA-Assistant by SAP 1 1
  12. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Snippet & Similarity Scanner Mission • Identify pieces of original code (source, object, binary) by comparing against known codebase Responsibility • Ensure code is free from copyright infringements due to copying routines or third party code • Discover re-use of code • Determine modification of identified code Tasks • Scan files for copies • Scan sources for known snippets • Provide scan results including references to copies/identified origin (e.g. earliest known appearance) Input • Repository or file(s) to scan • Comparison basis (known data sets) Output • List of potential infringements with links to potential matches (e.g. in existing OSS) • Weighting/ordering of potential matches Comments • Snippet Scanning (e.g. plagiarism check), similarity scanning (rough check) and delta analysis (identify change) serve different purposes • While similarity analysis gives indication that something might require further analysis, Snippet scanning delivers proof of re-use • Similarity analysis also allows delta analysis to be performed 12
  13. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Package Metadata Repository Mission • Collect package information and clearing metadata on packages Responsibility • Single point of truth for package information Tasks • Store package metadata and quality verification status (of that metadata concenring completeness and correctness) • Support composition analysis (verification of dependency analysis) • Provide search capabilities to identify existing packages • Support authentication/authorization to ensure responsible data handling/editing Input • Package identifier (e.g. purl) + already identified metadata • Package metadata Output • Package metadata, including package type (e.g. OSS, COTS, internal) and completion/ verification status of associated metadata • Containment structures (consists of) • Dependency structures (depends on) • Optional: relate known vulnerability information (not OSC specific, but a good place) Comments • Archive should be provided by archive capability. Tools supporting both functions in one are not limited by the capabilities beeing separate. 1 3
  14. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Case Data Collector Mission • Provide bracket for all compliance relevant information that is not directly related to source of a product / distribution item Responsibility • Ensure completeness of case documentation Tasks • Collect all product specific information, including package change & linkage status (via history) • Follow the release cycle of a particular product, e.g. approvals • Build canvas for reporting and analysis of a given composition & in a given situation • Versioning of analysis results to map with input situations Input • Business context (business model, distribution, external contractual obligations, etc.) • Software Bill of Materials (SBOM) + Component meta data (see Package Metadata Repo) • External components, e.g. runtime environments, middleware or resources (as part of solution) • Type of delivery/distribution (binary, source (oss), source (proprietary & oss), source (proprietary, oss , COTS and combinations of these) • Participants / Stakeholders (audience) • Approval Feedback Output • Status Overview • History of events and changes to context and meta data Comments 1 4
  15. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities – Case Data Analyzer Mission • Interpret all collected case data in given context and determine deltas Responsibility • Identify obligations, violations and warnings Tasks • Check for completeness of information • Identify missing information (e.g. missing Copyright information) • Determine rights and obligations, compare with requirements from business context Input • Case Data (see 13. ToolChain Capabilities - Case Data (Structure of Solution...) • Policy & Rules • Legal interpretation Output • Analysis result for further processing Comments • Review after re-draw of model 15
  16. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Policies & Rules Mission • Capturing the Organisation specific interpretation of its obligations, objectives & goals Responsibility • Represent the rules derived from organisations legal understanding Tasks • Rules how to treat specific legal circumstances, e.g. commercial aspects, trade secrets or IP protection requirements, etc. • Translate human readable policies to machine readable instructions/rules (as input input for analysis) • Document / Track changes in project specific allow- lists or deny-lists (licenses, components, frameworks, etc.) • Allow managing groups of projects with consistent policies & rules • Optional: Store open source policy for reference Input • Legal requirements for particular application scenarios • Definition allow- and deny-lists • Project specific rules and policies (e.g. versions, OpenSSF Score, specific components, viability, etc.) Output • History of changes Comments 16
  17. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities – Management of 3rd party provided Components Mission • Manage Commercial-Off-The-Shelf (COTS) and infrastructure (open source or COTS) packages of a solution Responsibility • Allow tracking 3rd party components concerning vulnerability and compliance • Collect and provide meta data for 3rd party or infrastructure packages Tasks • Store package metadata or 3rd party components and quality verification status (of that metadata concenring completeness and correctness) • Store information about 3rd party/private commercial conditions (license information) • Allow to assemble reports like SOUP-lists • Optional: Review 3rd party assemblies for known vulnerabilities Input • Package data and metadata (if known) • Binary scan information (BoM) Output • Package data and metadata (updated) • License information about 3rd party components Comments • PLEASE NOTE: For full compliance a storage for 3rd party sources/binaries should be available and referenceable • PLEASE NOTE: Commercial Licenses may have different aspects involved like termination by time / renewable • SOUP lists will require additional meta information, which is not in the scope of open source components 17
  18. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Legal Solver Mission • Determine legal rights and obligations resulting from the usage of the listed packages within the project context Responsibility • Provide compliance requirements: obligations and violations (missing rights) • Verify license compatibility under given circumstances Tasks • Assess license information from all packages (recent BoMs, infrastructure and 3rd party) and circumstances of use (business model, licensing amibition, IP protection requirements) • Determine license obligations and potential violations Input • Composition analysis of all project related packages, their status (binding and modification status), and licenses • Legal circumstances and requirements of the project Output • List of legal obligations and missing rights (if) by package and mitigation hints • Information on license in-compatibility (yes, no, why?) Comments • Independent from package status the analysis results may vary depending on changes in the circumstances. Thus analysis results should be versioned to allow allocation to related circumstances. • How to handle jurisdiction specific decisions? Would this be the place to put the information? 18
  19. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - License Repository Mission • Capture and archive legal information & interpretation about licenses Responsibility • Manage and provide legal information about known licenses Tasks • Capture & Update all license information including derived requirements and exceptions • Provide reference for original license texts • Provide environment to allow license analysis • Track changes in license interpretation • Manage classification and tagging Input • License data + interpretations Output • License data (updated) machine readable format Comments • Could be combined with legal solver, but we decided to provide as separate capability. A solver requires the repository, but the solver also could be a human worker. • How to represent different jurisdictions (e.g. case law UK / US)? => probably overdone, stay with most restrictive interpretation to prevent failure 19
  20. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Compliance Artefact Generator Mission • Support provisioning of compliance documentation Responsibility • Ensure legally compliant documentation Tasks • Generate documentation according to requirements • Support Compliance Managers in completing the documentation • Assemble documentation parts, e.g. written offer, license texts, copyrights, modification statement, etc. • Link documentation with objects (version management / binary links) • Provide documentation in machine readable export formats, e.g. JSON, SPDX, CyDX, etc. Input • List of versioned packages to be documented (BoMs) and their meta data • Legal requirements with respect to particular circumstances Output • Stub with all documentation requirements • Pre-assembled stub with all existing information (e.g. from repositories) • Identified TODOs for missing bits Comments 20
  21. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Approval Flow Mission • Ensure that the outgoing documentation fits the purpose Responsibility • Provide approval flow appropriate for audit Tasks • Track all legally relevant changes to products and packages • Identify authors of change • Provide compliance status and overview • Allow to approve or reject an approval request • Document/archive all decisions (auditing) • Support for different roles / instances of approval flows Input • Artifacts to be approved and approval type (e.g. security, compliance, etc.) Output • State of compliance analysis for approval request • Approval / Rejection documentation Comments • The approval by a dedicated, skilled resource (Compliance Manager) combined with the automation support for all prior steps reduces the need for Compliance Managers • Could be used for other objects, e.g. completeness of list of packages, etc. 21
  22. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - User & Role Management Mission • Provide role based authorization Responsibility • Authenticate users • Manage and/or map roles and authorizations • Assign users to roles Tasks • Identify users (Login, oAuth, MFA) • Manage roles and related authorizations (permissions assigned to roles) • Manage programmatical access (e.g. API keys) Input • Users • Roles Output • Authenticated user and associated roles (e.g. via access token) Comments • Agreement that these „infrastructural capabilities“ should be added and described 22 TODO: Provide support for infrastructural services to other capabilities
  23. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Audit Log Mission • Maintain log of changes and user actions (create accountability) Responsibility • Ensure traceability of configuration changes • Ensure tracing and archiving of all user actions/decisions for auditing purposes Tasks • Track user activity and changes in settings, especially legal settings • Track and archive user decisions and related context to enable auditing • Confirmation of completeness (e.g. by project owner) • Derive configuration status at a certain point in history Input • User actions / events Output • History of changes with actors • History of changes, configurations and decisions that lead to a particular compliance artefact (e.g. version number of scanner, scan config, etc.) Comments 23
  24. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Reporting & Analytics Mission • Visualize current work status, todos, efforts spent and success of compliance initiative Responsibility • Provide insights into state of portfolio • Create overview of workload and help to assign priorities • Measure compliance related activity Tasks • Collect data from different capabilities to allow reporting • Report design Input • Report specific data required Output • Reports (human AND machine readable format) • Transparency Comments • Specific reports should be defined on org level • See Todo Group for potential KPI ideas , e.g. scans/period, num of products scanned, number of issues found , etc. 24
  25. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) ToolChain Capabilities - Tool Orchestrator Mission • Co-ordinate overall compliance workflow(s) Responsibility • Arrange combination of tools to cope with compliance challenge • Handle handover between capabilities Tasks • Trigger events Input • Events Output • Events Comments • Depending on the degree of process automation the orchestrator may be a combination of event driven rule engine or a ticket system 25
  26. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Open Questions for further discussions 1. How to capture policies & rules in a form that allows automation/repetition? (from Rules & polices) • What constitutes a policy? = document (statement of intent, limits, ownership…) • What makes a rule ? Allow / Deny a User or Group to execute an action 2. Defined list of use cases that should be covered (check at Todo Group) i. Product/Solution compliance (create the output) ii. Handling an inquiry (internal/external) iii. Running an audit iv. Maintain / update compliance documentation v. Finding specific components across the portfolio vi. Pre-analysis of potentially useful components (or contributions) vii. Verifying 3rd party components (COTS) viii. Showing progress in compliance (visualizing metrics) ix. Maintain proper functionality of tooling chain x. Update license list / interpretation & handling consequences of it 26
  27. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example BANG) Dependency Analyzer (Source) Dependency Analyzer (Binary) Dependency Analyzer (Container) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 2 7 Data Flow Data Sink BANG
  28. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example Software Heritage) Dependency Analyzer (Source) Dependency Analyzer (Binary) Dependency Analyzer (Container) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 2 8 Data Flow Data Sink
  29. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example TERN) Dependency Analyzer (Source) Dependency Analyzer (Binary) Dependency Analyzer (Container) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 2 9 Data Flow Data Sink
  30. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example ClearlyDefined) Dependency Analyzer (Source) Dependency Analyzer (Binary) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 3 0 Data Flow Data Sink Dependency Analyzer (Container)
  31. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example TrustSource Scanners) Dependency Analyzer (Source) Dependency Analyzer (Binary) Dependency Analyzer (Container) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator Snippet Scanner (forensics) License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Package Crawler Compliance Artefacts Legal Solver (determine obligations) COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 3 1 Data Flow Data Sink DeepScan
  32. Licensed under CC-BY-SA-4.0 Open Source Compliance Capability Model (v1.5.5) Tool Orchestrator Reporting and Analytics ToolChain Capabilities (v1.3.1) – Mapping of Tools (example SCANOSS) Dependency Analyzer (Source) Dependency Analyzer (Binary) Dependency Analyzer (Container) Case Data (Situation, Inputs, Status) Policies & Rules Approval Flow (WFE) Compliance Artefact Generator License, Copyright & Authors Scanner License Repository (license facts, rights obligations) Compliance Artefacts COTS Management User & Role Management Package Source Archive Audit Log Package Data Repository 3 2 Data Flow Data Sink Snippet Scanner (forensics) Legal Solver (determine obligations) Package Crawler
Publicité