Continuous Interop Testing
•
0 j'aime•315 vues
This presentation represents the dimensions I have worked with in the development, testing, and cloud space.
![Agenda What is TestForge? What are the issues with the current DOD/DISA T&E Process and Environment? Why are we developing TestForge? What are our goals and approach to TestForge? ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Continuous Interop Testing
Similaire à Continuous Interop Testing (20)
Dernier
Dernier (20)
Continuous Interop Testing
- 1. Continuous Interoperability Testing TestForgeStatement of Direction John Cocke Strategy Architect
- 3. IA, C&A
- 4. Pilots and Selection Criteria
- 5. What is the end state?
- 6. What is the roadmap for getting us to the end state?Development Release Schedule and % Complete?
- 7. What is TestForge? Testforge is an event-driven net-centric test bed capable of providing 24 x 7 real-time information regarding any code, functional, or performance changes to application components in a user’s particular Testforge environment. Testforge is offered as a tightly integrated service with Forge.mil or it can be delivered as a shrink-wrapped stand-alone capability for installation and use in a customer environment. TEMC offers Testforge as a service including developer/administrative to assist customers or as a platform service offering with no organizational support.
- 9. Operational Testing (OT)
- 10. Interoperability Testing (IOP)
- 11. Information Assurance (IA)
- 14. 3 Decision Makers – MDA, JS J6,DAAT&E Plan – Test – Report cycle can exceed six months!
- 15. 2hr. access window Under Construction Wrong test data Source Application Dependency Constraints System availability constraints restrict accessibility for development and delivery.
- 18. Centralized Test Repository
- 20. Pre-Certified Environments to enhance time to market
- 22. The automated build server constantly monitors the repository for changes (Hudson)
- 23. As changes are made, the new code is checked out from the build server
- 24. The new code is built and run through a series of smoke tests (unit and functional) and static code analysis (security, complexity, standards adherence, well-formed code, etc)
- 25. Issues are immediately reported to team members via email or other CommunityForge channels
- 26. Builds are packaged and can be automatically deployed to Testforge for execution of heavier tests such as regression, performance , scale testing, IA testing, and integration testing.
- 28. DISA/RACE Integration - notional
- 30. Test Bed Preparation (all a developer should need to know about what’s under the hood)
- 31. Build Management
- 33. Continuous InteroperabilityThe Testforge End State
- 34. John R. Cocke The Wave Group of VA, LLC 804-761-3863 jrcocke@gmail.com
Notes de l'éditeur
- Main Point: Dependency constraints are a critical problem for Netcentric applications. It’s a root cause of delays, increasing infrastructure costs, and quality problems.---------------------Development and testing should be an integrated activity, but we don’t have the tools to really make that work in parallel because of constraints and dependencies in the process. Constraints can take many forms:Still under developmentWrong data-can’t testOnly 2 hours of access
- Projects are initially stood up in Forge.mil using the SoftwareForge repository, in a future release cloud resources for all Forge.mil projects will be provisioned by the TestForge-provided cloud management capability(IaaS provisioning). This assures continuity and ease from development all the way through push to production. The PKI and CAC credentials (roles/permissions) inherent in the developers, testers, administrators, and project managers for “this project” are established at “SoftwareForge time”. The cloud resources used for Dev and TEST and PROD will be different, however the vision is for a single TestForge cloud management capability to handle all provisioning.Initial code and unit test is performed in SoftwareForge, however developers will still be able to access SoftwareForge codebases after projects have moved to TestForge. The build management capabilities, specifically notifications within SoftwareForge will be integrated with TestForge and will be extended to trigger TestForge’s “continuous interoperability” cycle (assumes clean compile and code analysis scores (if required).When the project is ready to move into TestForge, the TestForge “account management” operational area assures that the developers, testers, administrators, and project managers are correctly permissioned (based on what was established at SoftwareForge time. These people will have the CRUD rights over those objects needed to complete a test and IA cycle: assets, scenarios, physical services (including IA controls and available application services), virtual services, etc.) PKI and CAC credentials in the initial Forge.mil edition of TestForge will be acquired from Forge.mil. The “Workgroup Edition” of TestForgebeing developed for the 350th ELSW at Hanscom is not addressed in this document.The Forge.mil Build Management capability is transparently provided by SoftwareForge.mil to TestForge. Notifications of successful and failed builds will be messaged in the TestForge Console and reporting stores. Asset Management is the operational area within TestForge where “system modules” (mostly IaaS in nature) and their related “assets” (middleware, databases, etc.) are created and/or selected by a tester. TestForge testing concepts and terminology differ slightly from the SoftwareForge approach to testing. The goal is complete transparency. However, SoftwareForge testing assets (including access-related) will be migrated to TestForge repositories at the time the project is moved to TestForge. In addition, all codebase originally developed in SoftwareForge will remain available in SoftwareForge for changes and build management at any time during the TestForge cycle. Physical entry points to applications (where available) and virtual services are created. IA compliance requirements are identified and appropriate controls are created(reused) as physical or virtual services. TestForge seeks to minimize the need for physical SOA infrastructure. The key strategy for doing this is to use our virtual service testing capability (currently iTKO LISA and VSE) to abstract SOA components. Developers, testers, administrators work together to vision test cases which are built at this time. We now begin scenario construction. The “scenario” is what is actually tested once the codebase, physical entry points, virtual services, and IA controls are identified and included in the scenario. The scenario may include multiple “assets” (each with their own operating system, middleware, codebase, apps) plus test cases to be executed plus virtual services to be executed. Once a scenario with all of its components is assembled and a triggering event (could be a clean compile in SoftwareForge, a state change in a database or client app, a successful change and retest of a virtual service, etc.), this project can be added into the Continuous Interoperability (CI) stack.Real time C&A reporting requirements coming out of TestForge IA controls during CI cycles need to be defined and incorporated.