Powerful Google developer tools for immediate impact! (2023-24 C)
Nebula james Williams
1. NASA NebulaPast, Present, and Future A Story in Three Parts James F. Williams CIO, NASA Ames Research Center April 26, 2011
2. One of the first cloud computing platforms built for the Federal Government by the Federal Government Publicly launched IaaS with the White House as the first customer in production Basis of OpenStack Compute, aka “Nova” There are over 300 users across nine NASA Centers + JPL + HQ NASA Nebula Cloud Computing
3. Lots Been Said About Nebula….some of it crazy What do you get when you combine cloud computing and data center containers? You get NASA’s Nebula, the space agency’s new data powerhouse, which provides on-demand computing power for NASA researchers. http://www.datacenterknowledge.com/archives/2009/12/02/nasas-nebula-the-cloud-in-a-container/ “The world will end if we don’t do this.” Former NASA Center CIO “Will create world peace.” Future Miss America “Putting the ‘Space’ in Rackspace” Start-up Co-founder “This will can help solve NASA’s real compute issues” New NASA Center CIO “Because we need more developers” Well known cloud architect
4. Tell you the unofficial Story of how NASA Nebula started…..from my perspective. I was there for most of it. I’ve blacked out for some of it. Where we are today and what NASA is doing Our vision for NASA Nebula for the future Getting Past the Hype
6. 1000s of other NASA websites "A long time ago in a US Agency far, far away.... (well, DC is far)" www.nasa.gov Photo Credit: www.starwars,com Photo Credit: www.starwars,com
7. Why wouldn’t they join NASA.gov? www.nasa.gov Perception: Control Issues with CMS Flexibility Cost Photo Credit: www.starwars,com
8. There’s a better way to do this… Problem: How do we get these web developers to stop building out their own sites? Solution: Give Developers a better alternative to status quo NASA.net was born
9. NASA.net Setting: Basement of NASA Ames Research Center (ARC) Building 200, in an old conference room Imagine: Small team of developers working on Platform as a Service Code hosting Continuous integration Bug Tracking Best Practices in code development Making unicorns happy across NASA
10. But after working on Platform as a Service prototypes… Learned in order to run a web application framework properly as a service, we need elastic infrastructure
11. Over Indian Food in Mountain View Joshua McKenty pitched the “cloud” idea to us. I just ate curry. We decided build out an IaaS capability just to support NASA.net Chris didn’t pay for lunch. Photo Credit: http://www.tandooribistrosj.com/
12. Problem Us: “Thanks! We need funding..” Everyone: “Great Idea!”
13.
14. Why? FY09: Line item for cloud? FY06: Plan FY09 Budget How do we fund IT innovation?
15. At the same time, White House “cloud first” initiative was gaining traction Federal CIO, VivekKundra evangelized the idea of Cloud Computing In coordination with the data center consolidations, agencies should evaluate the potential to adopt cloud computing solutions by analyzing computing alternatives for IT investments in FY 2012. Agencies will be expected to adopt cloud computing solutions where they represent the best value at an acceptable level of risk.http://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-19.pdf
16. A team said “No” They just said No. A wise woman said. “Drop back. Punt. And wait to get the ball back.”
17. We didn’t have any money but we had popsicle sticks and string so we started anyway. Josh liked Indian food and was friends with Jesse These guys were the first IT hippies I met. I gave them free headbands. There eventually was a guy named Vish(from Iowa? Really?) and someone from ZZTop(“He’s got cloud, and he knows how to use them..”) I bought cases of Red Bull to cover up the fact I didn’t have money for chairs. But ARC is creative…..
18. USASpending.gov Then the White House wanted to use our cloud. Then they came to ARC and some NASA officials were there…... Somehow we got a container and a couple $$’s to do something but still no money for chairs. Lots of cats……no milk…. Herding was a problem…… Enter Soo and Ray and William……Finally a semblance of a team that started to make things happen.
20. Massive Technical Challenges ) Ran into every problem you can think of: Hardware, Database, Software, Client management Details: Jumbo frames causing VMs to kernel panic 2 minute network separation between cloud controller & VMs caused controller to decide to terminate instances Prepare your container for the cloud… it rains Pieces of their really cloud ready. Indexes didn’t fit in RAM (even with 96GB allocated to VMs)
21. Weekend Hack-a-Thon Decided to spend the weekend hacking a new open source cloud controller framework. This became “Nova” Some thoughts:Monolithic is bad – each component should scale independently APIs are good – you shouldn’t have to use web UI to configure Simple things should be simple, hard things possible… The hard parts (hypervisor, storage system, networking) is done by others. Build a cloud the same way the way you build a scalable web application… Nova has been empowering NASA users for a year this May.
23. OpenStack History 2010 2005 March May June July Rackspace Cloud developed Rackspace Decides to Open Source Cloud Software NASA Open Sources Nebula Platform OpenStack formed with contributions from Rackspace & NASA Inaugural Design Summit in Austin
24. Rackspace Called NASA Rackspace: “Wow. Can we meet your team of 400 developers?” NASA: “Sure. We got 8 developers. And not all of them full time. “
25. Why not a NASA-Driven Foundation? Resources Expertise Focus Not NASA’s mission
26. NASA Nebula’s contributions to OpenStack aligns with the Administrator's strategic goals Facilitate the success of a viable commercial space industry to provide assured U.S. access to low Earth orbit for cargo and crew and acquire, mature, and infuse commercial capabilities across all NASA activities Promote enhanced cooperation with international, industry, other U.S. government agency, and academic partners in the pursuit of our missions. 26 26
27. NASA Nebula + OpenStack supports the Agency’s Goals: Goal 6: Share NASA with the public, educators, and students to provide opportunities to participate in our mission, foster innovation and contribute to a strong National economy
29. Developers from all over the world want to contribute code because they “want to be part of the space exploration.” Overheard at the OpenStack Design Summit, “My code could be part of NASA Nebula. This is as close to being astronaut I am ever going to get!” Overheard at Design Summit
30.
31. In April 2010, the Alpha phase began with 20 users began
32. Even though we’ve been concentrating on ARC and GSFC, Beta closed with 240 IaaS users across 9 Centers, HQ, JPL, and NSSC through word of mouth
33. Nebula requires OCIO sponsorship to expand and become an institutionalized service for the entire AgencyGRC: 4 HQ: 8 GSFC: 104 MSFC: 2 ARC: 111 LaRC: 1 JSC: 4 JPL: 3 KSC: 1 NSSC: 2 SSC: 1 30
36. Requirements* Current Options* * Requirements and Options documented in over 30+ interviews with Ames scientists as part 2009 NASA Workstation project.
37. Offer scientists services to address the gap Desktop TARGET COMPUTE PLATFORM Server-based compute resources Excellent example of how OCIO-sponsored innovation can be rapidly transformed into services that address Agency mission needs High-end Compute Vast Storage High Speed Networking Super Computer 34
42. Significant system administration expense*Number of servers: Estimate based on data collected from NASA Workstation Project, inventory of Ames institiional data center, review of Ames IP address allocation, and consultation with Ames Network Engineers. **15% utilization based on two reports from Gartner Group, Cost of Traditional Data Centers (2009), and Data Center Efficiency (2010). 35
43. ROI and ARC Case Study POWER: Computers typically require 70% of their total power requirements to run at just 15% utilization. *15% utilization based on two reports from Gartner Group, Cost of Traditional Data Centers (2009), and Data Center Efficiency (2010). 36
44.
45. ROI and ARC Case Study Cost Avoidance Improved visibility of NASA server resources achieved by cloud-based resource pooling. Promotes standardization of both hardware and operating systems Security compliance and technical integration of new capabilities IT Security Enhancements: Implementing NASA’s Security Program will be much easier in a standardized environment We know where the servers are located: Physically locate assets involved in incidents Security posture will be improved simply due to enhanced visibility (by the NASA Security Team) into the resources being secured 38
46. ROI and ARC Case Study Operational Enhancements: Strict standardization of hardware and infrastructure software components Small numbers of system administratorsto manage the cloud and applications due to the cookie-cutter design of cloud components and support processes Failure of any single component within the Nebula cloud will not become reason for alarm 39
47. NASA has direct access to the Nebula cloud computing platform Mission Objectives Explore, Understand, and Share MISSION Exploration Space Ops Science Aeronautics Mission Support USE CASES Process Large Data Sets Scale-out for one-time events Require infrastructure on-demand Store mission & scientic data Share information with the public Run Compute Intensive Workloads OCIO INNOVATION High Compute Vast Storage High Speed Networking 40
48. Conversation with Scientists Test it for free. Cool. Let’s see what this baby can do… I love your cloud and want to start. How much does it cost?
58. Possible Cloud Applications Hundreds of galaxies can be hi-res processed to provide higher angular resolution images for better studies of star formation, galactic structure, etc. Thousands of galaxies could be hi-res processed to resolve & measure source sizes, etc. Hundreds of thousands of galaxies can be hi-res processed to differentiate point-like (active galactic nucleus or nuclear star formation) from distributed (merger or spiral galaxy) emission. Several large regions nearby can be surveyed for distributed star formation in our galaxy. A few large regions can be processed to tremendous depth. The whole sky can be hi-res processed. “conducted calculations that could not be done on our project's large server farm. ”
62. Nebula will continue to be part of the open source community Nebula is being incorporated as an option in the overarching computational services for the agency. And yes….we are looking at commercial clouds too. Operational management is being implemented. Nebula continues to be supported by the Agency and the Agency is working to define strategies for cloud use. The Agency is defining its current use cases for cloud to ensure adoption. ARC is currently developing a cloud migration strategy for business and operational organizations. Are you cloud ready? ARC is beginning to incorporate cloud into their mission proposal process. Security in the cloud continues to be a driver. Pushing towards moderate and ITAR usage. What NASA is doing………
68. Open source means moving faster and smarter Moving inertia takes great strength. People aren’t wrong in the way they do things. It still works for them. They just have to be shown why another way is better for the future. Messaging and direction should always be clear. The why needs to be solid. The how agreed upon. And the message accurate. Passion for the mission outweighs the paycheck. Yes some of us really do work for NASA because it is NASA. Never ask hippy developers to show up to an 8 a.m. meeting. What have I learned…..
