Enterprise data architecture of complex distributed applications & services
•Télécharger en tant que PPTX, PDF•
0 j'aime•667 vues
Enterprise data architecture of complex distributed applications & services
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Enterprise data architecture of complex distributed applications & services
Similaire à Enterprise data architecture of complex distributed applications & services (20)
Dernier
Dernier (20)
Enterprise data architecture of complex distributed applications & services
- 1. Enterprise data architecture of complex distributed applications & services Davinder Kohli
- 2. Data Metamorphosis Process Requirements Architecture/Desig nBusi ness Data App Tech Implementation Business/Data Transfer Objects Deploy/Test Data Provisioning Test Scenarios Use Cases
- 3. organization Data is an asset Data has economic value Data must be shared and easily accessible Data must have common terminology & definitions Data needs to be secured
- 4. Case
- 5. UI Confused? Where do I start? Http Session Service A Service B Service C Service C1 Service C2 Service C3 Service A1 Service A2 Service A3 Service B1 Service B2 Service B3 Service A31 Service A32 Service A33 Service B31 Service B32 Service B33 Service C31 Service C32 Service C33 Service A2, A33, B1, B32, C1, C31
- 6. Data Flow/Mapping - facet of Data Architecture Why? Identify sources of data Define data interrelationship Flow of information through the app’s ecosystem Public/non-public information Data provisioning for testing What? UI - data rendering, form submissions Complex Services – requests, responses Unnecessary data – movement,
- 7. Absence of Data Flow/Mapping Implementation Longer implementation cycle Too much information to figure out Redundant data objects Too much data movement Testing Confusion during data provisioning Lack of coordinated datasets Longer testing cycle Quality More unit tests More lines of code Performance degradation
- 8. Challenges in data mapping Getting buy-in from stakeholders Lack of data dictionary Silo’d resources – technology, people, process (release cycle) Evolving interfaces – WSDL, DB Schema Long term maintenance
- 9. Approach/Solution Approach Top down – Business cases Bottom up – Existing interfaces, WSDLs Resource alignment – people, artifacts Artifacts Data Mapping/Flow Sheet Analysis of data flow/mapping Reduce data movement Identify redundancy of data sources Gaps in mapping SOT
- 10. How to build data mapping? Sample UI IxD CCL Use Cases Data Mapping WSDLs WSDLs WSDLs Bottom Up Top Down CSD
- 12. Artifact Creation Approach Data Model Beans WSDL(s) Data Mapping File UIWSDL(s) WSDL(s) Data Mapper Utility (Apache POI,JAXB) Data Model Beans WSDL to Java Manual Creation Java AdaptersXSLT Transformers Mapping done manually by developer Manual CreationManual Creation
- 13. Questions?
Notes de l'éditeur
- Need to understand where data fits within SDLC. Requirements: Use cases identify the data to be displayed and submitted by the user. Use cases identify the calculations to be performed on data submitted and presented back. Data may manifest itself as a report within your DC’s app or in another DC. Architecture: Data identified in use cases morphs into data models in architecture/design. In SOA, data resides in different silos but needs to come together to meet certain customer needs. Since data comes from different sources perhaps from across DCs, there is a need for data mapping. Implementation: The data model and mapping that you identify in architecture phase is used for deriving the DTO and BO in the implementation phase. Deploy/Test: Test Scenarios are based on use cases. Combination of test scenarios, data mappings, data sources, BO help provision data.
- Asset: Just like fixtures, data centers, data is also an asset. Economic value: Can be traded
- If you were to start implementing the UI application without data mappings and flow, you may be able to quickly build out partial JSPs based on your IxDs. However, you will soon be stuck trying to figure out how the information flows from page to page. You will be stuck trying to figure out the interrelationship between services. You will be stuck trying to figure out which service is responsible for doing computations on user submitted data.
- Goals Why: Distributed data sources Flow of info – building the app and later for triaging Identify what content can be displayed/not displayed to the user or should flow through a particular service What: Capture data that needs to be rendered on UI, user submitted data. Map responses to requests. Identify what data is not required, avoid unnecessary data movement.
- Cadillac with 3 wheels Readability and understandability of code
- Everyone involved in the project is a stakeholder – BAs, Architects, Developers, Testers, PMs Buy-in on the approach, unless it becomes TE guidance.
- We have agreed on the need for data mapping and flow, what is the approach? Artifacts available to work with: Use Case docs, IxDs, CCLs, Existing services WSDL Top down: Based on UI application flow, identify the flow that is longest pole in the tent and touches maximum number of services. For example, Multiple vehicles in multi-car state, paying by a new credit card, financial account owned by an uncle, paying in monthly installments, paying additional amount. Identify distinct flows – stop when there is a lot of overlap Bottom up: Use existing service WSDLs Create new WSDLs for new services Map request/responses of services in an orchestration flow. Map the 1st tiered services with UI pages. Not a single person effort, requires team work and collaboration Analysis: Intent of data mapping was to build the application in a correct manner. Information should flow through correctly such that meets all business use cases. Other benefits: Too much data movement – irrelevant information is travelling to other services. – Security concern? Data is persisted in multiple places. Gaps due to service schedules – identify the need for stubbing data, identify translations required (EAVS), build a data dictionary. SOT – source of truth. Data may have transformed during the flow. Instead of consuming data from it’s source, consuming data after it has been transformed and persisted in a different source may not yield the correct results.
- Often when this document is created there are many open questions. The source of this information can found many different places. Business Spaces now has all of the contracts for services. You may be able to start with existing services or coordinate with appropriate teams for data. You may not be given much detail at all. If you find that a lot of the details you need are loosely defined or unavailable. You may need to make some assumptions. Either way this will help track your data relationships.
- Team/single effort in producing this artifact. There needs to be coordination among DC teams. Show example from project on a high level Show example UI and walk though an example mapping. Go into detail about UI and Services mapping. Points: Focus on reducing redundant sources of information. Explain how to define UI model and handle order of execution. Show how to define data relationships. How to handle undefined data sources. How to handle logic within the model. Discuss populating the data, tracking missing fields, arrays, Show example initial interface map from generated example. Some pros and cons to this technique. Color mapping can become a problem for many reason. For example if the data is from 2 services. It is very consumable. The idea is to find a way to show the relationship between an UI or Service and underlying tiers. Changes happen. Changes to one tier are simple although multiple tiers becomes more complicated. Names can be misleading and may cause incorrect assumptions about the data. Changes to external wsdl must be maintained in the data mapping. A more meta friendly contsruct like XML instead of excel would be worth considering.
- POI automates the manual task of converting consumed service WSDLs into a rough initial data mapping. Once your project is configured, you simply drop your wsdl files into your source location WSDL to java is commonly used to create business objects. Ideally all data would go through the single data mapping file (This would greatly simplify version management). (Defined by black lines) Another method to create the sample request is to use soap ui. Make it easy to modify the same request and responses. Sample request and responses are not necessary to generate if the actual service is available. These request and responses can then be used for stub services. Works well for quick and dirty example data, not ideal for coordinated data sets or stateful response scenarios. All good reason this should be a multiple step process. Create Mapping Create Business Objects Populate Business Objects with data Map data via tool of choice. Ex. Mule Data Mapper, Dozer, SQL There are many different ways to handle this. This was one conceptual model that we liked for a mule orchestration. Apache POI, POI, Apache, the Apache feather logo, and the Apache POI project logo are trademarks of The Apache Software Foundation.
- Team/single effort in producing this artifact. There needs to be coordination among DC teams. Show example from project on a high level Show example UI and walk though an example mapping. Go into detail about UI and Services mapping. Points: Focus on reducing redundant sources of information. Explain how to define UI model and handle order of execution. Show how to define data relationships. How to handle undefined data sources. How to handle logic within the model. Discuss populating the data, tracking missing fields, arrays, Show example initial interface map from generated example. Some pros and cons to this technique. Color mapping can become a problem for many reason. For example if the data is from 2 services. It is very consumable. The idea is to find a way to show the relationship between an UI or Service and underlying tiers. Changes happen. Changes to one tier are simple although multiple tiers becomes more complicated. Names can be misleading and may cause incorrect assumptions about the data. Changes to external wsdl must be maintained in the data mapping. A more meta friendly contsruct like XML instead of excel would be worth considering.