Not much has been shared and talked about on Architecture as practiced on Agile projects. In fact, there is a concern among organizations small and large in adopting Agile practices/process that they have to abandon architecture in favor of agility. However, from experts we hear about this "emerging design and architecture," but not much explained in a way that dispels the myth around architecture on Agile projects. I would like to show step by step how we have done it on a large government project. (includes workflow automation, transactions, and data warehousing solutions, as well as spans multiple legacy components, and multiple agencies). The participants will be able understand how architecture evolves on Agile projects and how to manage/guide this evolution of architecture in a way to meet the goals of the project.

1. Emergent Architecture on Agile Projects Syed Rayhan Co-founder, Code71, Inc. Contact:srayhan@code71.com Blog:http://blog.syedrayhan.com Company:http://www.code71.com Product:http://www.scrumpad.com

2. 2 Agenda Introduction Section 1 Section 2 Project summary Section 3 Initial Architecture Evolving initial architecture Section 4 Recap Section 5 Q&A Section 6

3. 3 Agenda Introduction

5. 15+ years of total experience

7. Agile technology planning and architecture

9. Scrum for small projects delivered remotely

10. Agile engineering practicesInterests

13. How emergent architecture actually is in practice

15. Who does architecture

17. Department of Transportation

19. Reduce traffic collisions

20. System under consideration is dependent on other systems that are also in the process of being replaced

21. Infrastructure is provided by VITA

22. New technologyArchitecture Landscape

23. 7 Agenda ProjectSummary

25. Duplicate data stores is causing poor data quality

26. Manual data entry is causing poor data quality as well as increased processing time and costs

28. Elimination of paper reports

29. Increasing the efficiency and improving data quality

30. Elimination of duplicate data entry

31. Electronic data reporting

32. Elimination of data entry backlogsGoals

34. Development started in 2007 May

35. 60 sprints completed to-date

37. $4.5 M

38. Contractors and consultants

39. Core team has 12 members with cross-functional

40. Located on-siteStaffing

41. 10 Project Context

42. 11 Project type Our project new Zone of death complex technology complex simple expert well understood Not in agreement requirements

43. 12 Existing Architecture data export App 3 image Agency 3 Image db 1 data entry data entry image copy update update Image db 2 update App 2 App 1 Agency 1 Main App Agency 2

44. 13 Existing Enterprise Architecture Standards Technology Platform VB.Net, MSSQL & Oracle Security Two-factor authentication (FOB based) Infrastructure Strict 3-tier architecture

45. 14 Agenda Initial Architecture

47. Reuse existing components

49. 17 Architecture Patterns MVC Hub & Spoke

50. 18 Logical Architecture- Initial Vision

54. Business Process Impact:: - Scan datavalidation - Automation of some current manual operations

55. Scan bubble data into CAP

56. Automate data presentation forSafetynet

57. Build TREDS/SFTP Interface

61. Build TREDS/RNS interface

63. TREDS/ CAP data synchronization willprovide system fall back during burn in

64. Policy and standards enhancements may occur due to human to system authentication and authorization

66. TREDS Client – Pilot 1

67. Build TREDS data management

68. Build centralized rules engine

69. Build TREDS Data Warehouse(Havana)

71. Build human to system authorization and authentication

72. 20 Agenda Evolution of Architecture

73. 21 1st Release Architecture scanned report Scanning App WS-API image Image db1 Controller WS data App 3 App 2 BTOrchestration WS image image FTP App 1 update Broker Image db2 DW unprocessed data data migration data Main App

74. 22 1st Release Recap Reduce data entry time by 50% Reduce architectural risk around infrastructure, Goals Existing system is still the system of records Electronic data capture, not delivery Capabilities OCR accuracy was lower than expected Initial data processing speed went down Result Form redesign should have followed data modeling and scanning technology selection Windows environment for efficient data entry poses different challenge Lessons Learned

75. 23 2nd Release Architecture scanned report Scanning App AzMan Web Client WS-API DB image Image db1 Controller WS data App 3 Workflow BRE App 2 BTOrchestration WS Model image FTP data App 1 update Broker Image db2 DB trigger processed data DW Audit DB Processed data Main App

76. 24 Defining Workflow using State Machine

77. 25 2nd Release Recap Introduce automated workflow for data processing Goals Existing system is still the system of records Automated work management of data entry clerks Capabilities Automated workflow was well received Improved the quality of management of the supervisors Result Too many data validation rules were too restrictive Lack of robust exception handling resulted in premature termination of workflows Lessons Learned

78. 26 3rd Release Architecture scanned report Scanning App AzMan Web Client WS-API DB image Image db1 Controller WS data Report WS App 4 Workflow BRE App 3 BTOrchestration WS Model image FTP App 2 trigger Broker Image db2 DB update processed data DW Audit DB Main App

79. 27 3rd Release Recap Integration with App2 Canned reporting from new DW Goals Main reporting from the new system Complete data entry from the new system Capabilities Reporting time reduced from a few weeks to a few minutes Reports requiring manual check reduced to 70%-80% due to auto acceptance Result Longer processing time at certain steps in the workflow caused user frustration Web service interface to BizTalk was causing timeouts Lessons Learned

80. 28 4th Release Architecture Client App Scanning App AzMan electronic report scanner image scanned report Web Client WS-API DB Image db1 Controller WS data Report WS App 4 Workflow BRE App 3 Req Q Req Q image Model BTOrchestration WS FTP App 2 update Image db2 Broker DB processed data DW Main App Audit DB

81. 29 Workflow with Asynch. System Processing

82. 30 4th Release Recap Improve response time using asynchronous processing Electronic delivery of reports from the field Goals Direct submission of reports (no scanning) to the system Complete data entry from the new system Capabilities Improved user satisfaction from reduction in response time Reports requiring manual check further reduced to 50%-60% due to auto acceptance Result Lessons Learned Change in workflow caused versionites Accelerate adoption of electronic delivery of reports from the field

83. 31 5th Release Architecture Client App Scanning App AzMan electronic report scanner image scanned report Web Client WS-API DB Image db1 WCF Controller WS data Report WS App 4 BRE App 3 Workflow Req Q Req Q image BTOrchestration WS FTP Model App 2 Image db2 Broker DB update processed data DW Main App Audit DB

84. 32 5th Release Recap Integration with App3 “Revision of Reports” handling Ad-hoc reporting from new DW Goals New system is self-sufficient Asynchronous processing of reports delivered electronically Capabilities Integration with other reporting tools for electronic reporting Old system is ready for sunset Result The definition of “revision of a report” changed (new capability changes business practices) Lessons Learned

86. Upgrade to Biztalk 2009

87. Upgrade to Visual Studio 2008

88. Keep an eye on new technology for new capability or easier maintainability

89. ASP.Net MVC

91. 35 Recap lessons learned Start with existing architecture standards, identify gaps, and know upcoming changes Influence / introduce new standards to fill the gap in existing architecture Align with upcoming changes to the existing architecture Let architecture evolve top-down (architect is responsible) and bottom-up (team is responsible) Use working “spikes” to determine how to adopt new technologies Manage evolution of architecture in small increments (a.k.arefactor) Document architecture using FAQ style and just-in-time Architecture evolves; when unmanaged, it becomes sprawling.

92. 36 Q&A Guided evolution of architecture improves Agility!

93. 37 Contact Please contact for on-site training/coaching or Webinar: Contact:srayhan@code71.com Blog:http://blog.syedrayhan.com Company:http://www.code71.com Product:http://www.scrumpad.com