Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Graphing Enterprise IT – Representing IT Infrastructure and Business Processes as a Graph - Alan Robertson @ GraphConnect SF 2013

7 153 vues

Publié le

The Assimilation Monitoring Project is an open source project providing both continuous Stealth Discovery(TM) and extreme scale monitoring.

Publié dans : Technologie
  • Soyez le premier à commenter

Graphing Enterprise IT – Representing IT Infrastructure and Business Processes as a Graph - Alan Robertson @ GraphConnect SF 2013

  1. 1. Modeling IT Infrastucture using The Assimilation Project #AssimProj @OSSAlanR http://assimproj.org/ http://bit.ly/AssimGC2013 Alan Robertson <alanr@unix.sh> Assimilation Systems Limited http://assimilationsystems.com G R A P H C O N N E C T
  2. 2. Upcoming Events GraphConnect San Francisco (today!) Open Source Monitoring Conference - Nürnberg NSA / Homeland Security Assimilation Technical Talk Large Installation System Administration Conference - DC Colorado Springs Open Source User’s Group linux.conf.au – Awesome Australian Linux Conf - Perth Details on http://assimilationsystems.com/ G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 2/25
  3. 3. Talk Outline ● Project Overview ● Infrastructure Schema ● Python Object-Graph-Mapping API G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 3/25
  4. 4. Assimilation Project History ● Inspired by 2 million core computer (cyclops64) ● Concerns for extreme scale ● Topology aware monitoring ● Topology discovery w/out security issues =►Discovery of everything! G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 4/25
  5. 5. Project Scope Zero-network-footprint continuous Discovery integrated with extreme-scale Monitoring ● Continuous extensible discovery – ● Extensible exception monitoring – ● systems, switches, services, dependencies – zero network footprint G R A P H C O N N E C T more than 100K systems All data goes into central graph database © 2013 Assimilation Systems Limited GraphConnect 4 October 2013 5/25
  6. 6. Discovery Discovering ● systems you've forgotten ● what you're not monitoring ● whatever you'd like ● without setting off security alarms G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 6/25
  7. 7. Why Discovery? ● Documentation: incomplete, incorrect ● Dependencies: unknown ● Planning: Needs accurate data ● ● Best Practices: Verification needs data ITIL CMDB (Configuration Mgmt DataBase) Our Discovery: continuous, low-profile © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 7/25
  8. 8. Why Neo4j (graph db)? ● ● ● ● ● ● Dependency & Discovery information: graph Speed of graph traversals depends on size of subgraph, not total graph size Root cause queries  graph traversals – notoriously slow in relational databases Visualization of relationships Schema-less design: good for constantly changing heterogeneous environment Graph Model === Object Model © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 8/25
  9. 9. Assimilation Communication Neighbor-Rings G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 9/25
  10. 10. Ring Representation Schema G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 10/25
  11. 11. ssh -> sshd dependency graph G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 11/25
  12. 12. Switch Discovery Data from LLDP (or CDP) CRM transforms LLDP (CDP) Data to JSON © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 12/25
  13. 13. OGM – Object Graph Mapping ● Managing the Graph Nodes “disappears” ● The Object Model is the Graph Model ● Significant Improvement in Thinking Clarity – The “mud” is gone G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 13/25
  14. 14. OGM rules ● ● ● Don't use Constructors Relationships replace hash tables and object references and so on Constructor parameter names match attribute names G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 14/25
  15. 15. OGM sample @RegisterGraphClass class Person(GraphNode): 'A Person - or at least an electronic representation of one' def __init__(self, firstname, lastname, dateofbirth=None): GraphNode.__init__(self, domain='global') self.firstname = firstname self.lastname = lastname if dateofbirth is not None: self.dateofbirth = dateofbirth else: self.dateofbirth='unknown' @staticmethod def __meta_keyattrs__(): 'Return our key attributes in order of significance' return ['lastname', 'firstname'] © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 15/25
  16. 16. OGM sample @RegisterGraphClass class TestSystem(GraphNode): 'Some kind of semi-intelligent system' def __init__(self, designation, roles=[]): GraphNode.__init__(self, domain) self.designation = designation.lower() if roles == []: roles = [''] self.roles = roles def addroles(self, role): if self.roles[0] == '''': del self.roles[0] if isinstance(role, list): for arole in role: self.addroles(arole) elif role not in self.roles: self.roles.append(role) © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 16/25
  17. 17. OGM sample @staticmethod def __meta_keyattrs__(): 'Return our key attributes in order of significance' return ['designation'] G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 17/25
  18. 18. OGM sample # (seven)-[:formerly]->(Annika) Annika = store.load_or_create(Person, firstname='Annika', lastname='Hansen') seven = store.load_or_create(Drone, designation='SevenOfNine', roles='Borg') store.relate(seven, 'formerly', Annika) store.commit() G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 18/25
  19. 19. OGM sample @RegisterGraphClass class TestDrone(TestSystem): def __init__(self, designation, roles=[]): TestSystem.__init__(self, designation=designation) if isinstance(roles, str): roles = [roles] roles.extend(['host', 'Drone']) System.__init__(self, designation, roles=roles) G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 19/25
  20. 20. Current State ● First release was April 2013 ● Great unit test infrastructure ● Nanoprobe code – works well ● Service monitoring works ● Lacks digital signatures, encryption, compression ● Reliable UDP comm code working ● Several discovery methods written ● CMA and database code restructuring near-complete ● G R A P H C O N N E C T UI development underway ● Licensed under the GPL, commercial license available © 2013 Assimilation Systems Limited GraphConnect 4 October 2013 20/25
  21. 21. Future Plans ● Production grade by end of year ● Purchased support ● “Real digital signatures, compression, encryption ● Other security enhancements ● Much more discovery ● GUI ● Alerting ● Reporting ● Add Statistical Monitoring ● Best Practice Audits ● Dynamic (aka cloud) specialization ● G R A P H C O N N E C T Hundreds more ideas – See: https://trello.com/b/OpaED3AT © 2013 Assimilation Systems Limited GraphConnect 4 October 2013 21/25
  22. 22. Get Involved! Powerful Ideas and Infrastucture Fun, ground-breaking project Looking for early adopters, testers!! Needs for every kind of skill ● ● ● ● ● ● ● Awesome User Interfaces (UI/UX) Evangelism, community building Test Code (simulate 106 servers!) Python, C, script coding Documentation Feedback: Testing, Ideas, Plans Many others! © 2013 Assimilation Systems Limited G R A P H C O N N E C T GraphConnect 4 October 2013 22/25
  23. 23. Resistance Is Futile! #AssimProj @OSSAlanR #AssimMon Project Web Site http://assimproj.org G R A P H C O N N E C T Blog techthoughts.typepad.com lists.community.tummy.com/cgi-bin/mailman/admin/assimilation © 2013 Assimilation Systems Limited GraphConnect 4 October 2013 23/25
  24. 24. The Elder GeekGirl G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 24/25
  25. 25. Younger GeekGirl's Computer Running Linux Of Course! G R A P H C O N N E C T GraphConnect 4 October 2013 © 2013 Assimilation Systems Limited 25/25

×