Osis18_Cloud : Software-heritage

•

1 j'aime•133 vues

The document discusses the Software Heritage initiative, which aims to collect, preserve, and share the source code of all publicly available software. It notes that software is culturally and economically important but fragile, as source code is often lost. The initiative builds an archive of software source code and history using free and open source methods. It has already archived over 150 terabytes of open source software from repositories like GitHub, and seeks more support to continue its work of preserving software for current and future use.

Business

Software Heritage
Why and How We Preserve all of Mankind’s Source Code
Roberto Di Cosmo
roberto@dicosmo.org
June 5th, 2018
THE GREAT LIBRARY OF SOURCE CODE
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 1 / 14

Outline
1 Software all around us
2 The Software Heritage initiative
3 Building for the long term
4 Conclusion
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 2 / 14

Software is everywhere
Software
Software embodies our collective Knowledge and Cultural Heritage
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 2 / 14

Software Source Code is special
Harold Abelson, Structure and Interpretation of Computer Programs (1st ed.) 1985
“Programs must be written for people to read, and only incidentally for machines to execute.”
Quake 2 source code (excerpt) Net. queue in Linux (excerpt)
Len Shustek, Computer History Museum
“Source code provides a view into the mind of the designer.”
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 3 / 14

~ 50 years, a lightning fast growth
Apollo 11 Guidance Computer (~60.000 lines), 1969
"When I first got into it, nobody knew what it was that we were doing. It
was like the Wild West."
Margaret Hamilton
Linux Kernel
... now in your pockets!
are we taking care of all this?
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 4 / 14

Software is spread all around
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 5 / 14

Software is fragile
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 6 / 14

Software lacks its own research infrastructure
Photo: ALMA(ESO/NAOJ/NRAO), R. Hills
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 7 / 14

The Software Heritage Project www.softwareheritage.org
Our mission
Collect, preserve and share the source code of all the software that is available
Past, present and future
Preserving the past, enhancing the present, preparing the future
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 8 / 14

A principled infrastructure http://bit.ly/swhpaper
Technology
transparency and FOSS
replicas all the way down
Content
intrinsic identifiers
facts and provenance
Organization
non-profit
multi-stakeholder
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 9 / 14

Architecture (simplified)
Git
loader
Mercurial
loader
Debian source
package loader
tar loader
.
.
.
Software Heritage Archive
Merkle DAG + blob storage
Loading
& deduplication
dsc
dsc
hg
hg
hg
git
git
git git
svn
svn
svn
tar
zip
software
origins
Package
repos
Forges
GitHub
lister
GitLab
lister
Debian
lister
PyPi
lister
.
.
.
Distros
...
Scheduling
Listing
(full/incremental)
full development history permanently archived
origins: GitHub (automated), Debian (automated), Gitorious, Google Code, GNU
~150Tb raw contents, ~10Tb graph (7+Bn nodes, 60+Bn edges)
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 10 / 14

Using the archive
Preview just for you: a "wayback machine" for archived code!
go to http://archive.softwareheritage.org/browse
use the credentials: devoxx / 2018
No time for a demo, let’s highlight some features...
Origin search
Directory browsing Revisions as diffs
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 11 / 14

Growing Support
Landmark Inria Unesco agreement, April 3rd, 2017
Sharing the vision
Contributing to the mission
>= 100Ke/year
>= 50Ke/year
>= 25Ke/year
>= 10Ke/year
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 12 / 14

You can help!
Coding, advice see forge.softwareheritage.org
Current development priorities ... all contributions equally welcome!
documentation
listers/loaders for unsupported forges, VCS
Web UI improvements
Yes, we’ll be hiring, see www.softwareheritage.org/jobs
Funding
make your company a sponsor :
sponsorship.softwareheritage.org
give your own contribution :
www.softwareheritage.org/donate
Spread the word!
follow and relay project
news
share the vision, tell others
how to support the mission
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 13 / 14

Come in, we’re open!
www.softwareheritage.org @swheritage
Grand opening, June 7th, UNESCO heaquarters!
Library of Alexandria of code
recover the past
structure the future
A CERN for Software
build better software
for industry
for society as a
whole
Roberto Di Cosmo www.dicosmo.org www.softwareheritage.org June 5th, 2018 14 / 14

Contenu connexe

Similaire à Osis18_Cloud : Software-heritage

Osgis 10 arnulf-christl

Arnulf Christl

Free/Open Source Software is now everywhere, but the risk of losing forever some of it is growing. Shutdowns of once popular forges are early warnings that we should not underestimate. How many million lines of code would we lose if development hubs that are hype today were to disappear 20 years from now? This talk will present Software Heritage, whose aim is to collect, preserve, and share all publicly available source code. Forever. Software Heritage has already archived 3 billion distinct source code files and 650 million commits, spanning more than 25 million development projects.

Software Heritage: let's build together the universal archive of our software...

Codemotion

The goal of the Software Heritage project is to collect, preserve, and share all publicly available software in source code form. Forever. By doing so Software Heritage will serve the needs of: Society, by preserving our collective technological heritage; Industry, by building the largest software provenance open database; Science, by assembling the largest curated archive for software research; and Education, by creating the ultimate anthology for programming curricula. Although still in Beta, Software Heritage has already archived more than 2.5 billion unique source code files and 600 million unique commits, spanning more than 20 million projects from major software development hubs, GNU/Linux distributions, and upstream software collections. Software Heritage is developed transparently as a collaborative project and all its own source code is available as Free/Open Source Software. Currently incubated by Inria, the project will graduate soon to an independent charitable, nonprofit organization.

Software Heritage: Building the Universal Software Archive, OW2con'16, Paris.

OW2

The Archives Forum - The National Archives - 02 March 2011

David F. Flanders

This talk presents an overview of FOSS4G and the issues and characteristics that makes it a mature alternative in the geo-technologies arena. The following questions will be addressed: What is FOSS4G? What is its history and current state of the art? Why you should consider it as an alternative to private/proprietary software options? What are the common concerns regarding the use of FOSS4G? Are they true? What mature FOSS4G is available for every geo-technology niche (Desktop GIS; Remote Sensing, etc..)? Is there a conflict between private/proprietary geospatial software and FOSS4G?

2015 FOSS4G Track: What is Free and Open Source Software for Geospatial Appli...

GIS in the Rockies

Static analysis for multiple programming languages

Lingoport (www.lingoport.com)

Our Team and Work Philosophy (Tavisca)

Tavisca Solutions

Has the time come to collaboratively develop the software the language industry needs? To be more creative than any single company by sharing ideas, effort and costs? Instead of being suspicious competitors, is the language industry ready to work together and share ideas? Many companies in the industry express the need for tools, which - truly support Open Standards - can be extended, customized and integrated the way they need it - are driven by the latest research - are well supported - are affordable The OSELI initiative hopes to achieve traction.

Oseli seville-monday

Serge Gladkoff

Will Postgres Live Forever?

EDB

OSGeo - Spatially Empowered Open Source Software FOSS4G 2009

Arnulf Christl

R. Di Cosmo - Software Heritage

LibreItalia

FEC2017-Introduction-to-programming

Henrikki Tenkanen

TODO group has published a series of useful guidelines on how to create an open source program office but the reality of implementing can come with unexpected challenges and surprising opportunities. The broad scope of an open source program requires cross-functional coordination between multiple teams with differing goals, culture, focus areas, perspectives, and priorities, including legal, engineering, design, product, business, marketing, and branding. The success of the open source movement has made the concept of open source software so pervasive that it created another layer of challenges in education and communication. This talk will provide the honest story of my one-year journey of building the open source program office at Uber. I will share my observations and takeaways, as well as what I found rewarding and impactful during the process.

Os summit jp 2019 untold story ospo

Brian Hsieh

Open core summit 2020: Building an Open Source office on a shoestring budget

Aliza Carpio

Developers Italia and the New Guidelines: Let the Open Source Revolution Start!

Team per la Trasformazione Digitale

Android Development: A meta-learning approach

Apostolos Kritikos

Resume Nov/2011

Industrial Technology Research Institute (ITRI)(工業技術研究院, 工研院)

Wageningen phenotype meeting

thehyve

San Diego Start Up Month: Rockstar Open Source

Aliza Carpio

Abstract and video link below) Presented at ICGSE 2016: Conference on Global Software Engineering (http://www.ics.uci.edu/~icgse2016/2_0cfp.html) Video link: https://www.youtube.com/watch?v=BsgnLwPMqWM&feature=youtu.be&list=PLcm9UtazJCOLBwPaaHNn_htAjPAXIdRGr Abstract: Software development stakeholders require a constellation of tools to support their communication, collaboration and coordination activities. But poor tool integration can lead to gaps in knowledge flow, or worse, to an overabundance of shared communication and information. The software development community is witnessing the rise of "social bots" to integrate diverse development and communication tools and to address the challenge of information overload. A bot is a conversational user interface that can automate rote or tedious tasks. It may fetch or share information, extract and analyze data, detect and monitor events and activities in communication and social media, connect developers with each other or with other tools, or it may provide feedback on individual and collaborative development tasks. Some bots are emerging as important team members, providing support for individual and team task management and for the automation of dev-ops and customer support. However, the rapid adoption of bots and the platforms that support them brings possible drawbacks. Designing effective platforms for bots is challenging and bots may introduce alienation among stakeholders or lead to other technical challenges. In this talk, I will discuss the emerging role of bots in software development and describe some of the advantages and challenges that may lie ahead.

To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...

Margaret-Anne Storey

Similaire à Osis18_Cloud : Software-heritage (20)

Osgis 10 arnulf-christl

Software Heritage: let's build together the universal archive of our software...

Software Heritage: Building the Universal Software Archive, OW2con'16, Paris.

The Archives Forum - The National Archives - 02 March 2011

2015 FOSS4G Track: What is Free and Open Source Software for Geospatial Appli...

Static analysis for multiple programming languages

Our Team and Work Philosophy (Tavisca)

Oseli seville-monday

Will Postgres Live Forever?

OSGeo - Spatially Empowered Open Source Software FOSS4G 2009

R. Di Cosmo - Software Heritage

FEC2017-Introduction-to-programming

Os summit jp 2019 untold story ospo

Open core summit 2020: Building an Open Source office on a shoestring budget

Developers Italia and the New Guidelines: Let the Open Source Revolution Start!

Android Development: A meta-learning approach

Resume Nov/2011

Wageningen phenotype meeting

San Diego Start Up Month: Rockstar Open Source

To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...

Plus de Pôle Systematic Paris-Region

Short-range wireless communication technologies such as Bluetooth or ZigBee represent an important part of the Internet of Things ecosystem. By design, this category of smart devices has physically limited reachability inside their Wireless Personal Area Network (WPAN) and are not directly compatible with the TCP/IP stack. However, users may need to access them from anywhere at any moment. To address this problem, we design a new application-agnostic approach called RCM (Remote Connection Manager) enabling transparent communication between an application and out-of-range devices. It creates new IoT use cases by seamlessly mixing remote and local devices. We implemented an open-source prototype for Bluetooth Low Energy (BLE) technology on top of Linux and Android BLE stacks and demonstrated its efficiency through experiments performed on real devices.

OSIS19_IoT :Transparent remote connectivity to short-range IoT devices, by Na...

Pôle Systematic Paris-Region

OSIS19_Cloud : SAFC: Scheduling and Allocation Framework for Containers in a ...

Pôle Systematic Paris-Region

On parle d’observabilité des services lorsque ceux-ci exposent des états et métriques internes pour améliorer la disponibilité globale. Qu’en est-il de l’observabilité des infrastructures sur lesquelles ils sont déployés, configurés et maintenus ? Les différents logs (centralisés, agrégés) permettent un bon début d’analyse mais il faut aussi observer les systèmes au fil de l’eau pour tracer chaque changement et les corréler avec le monitoring. Aujourd’hui, ces étapes de configuration IT devraient être prises en charge par les outils de gestion de configuration, qui deviennent la passerelle vers l’observabilité des opérations. Nous montrerons l'intérêt de cette approche pour la gestion IT moderne avec un retour d’expérience sur les challenges de leur mise en place dans Rudder, notre solution libre d’audit et de gestion de configuration en continu.

OSIS19_Cloud : Qu’apporte l’observabilité à la gestion de configuration? par ...

Pôle Systematic Paris-Region

My research is in virtualized infrastructure domain. I aim at minimizing electricity consumption while improving application performance. To achieve the first goal, I work both at the entire datacenter level (by providing better VM placement strategies) and at the physical machine level (by providing better power management policies). Concerning the second goal, I work both at the VM monitor level (for minimizing its overhead) and at the VM's operating system (OS) level (for making it aware of the fact that it is virtualized). In this talk I present two contributions of my research team, one for each objective. The first contribution presents Drowsy-DC, a novel way to reduce data center power consumption inspired by smartphones. The second contribution presents XPV (eXtended Para-Virtualization), a new principle for well virtualizing NUMA machines.

OSIS19_Cloud : Performance and power management in virtualized data centers, ...

Pôle Systematic Paris-Region

L'expérience du développement de CRESON, support pour des objets distants fortement cohérents dans Infinispan, par Etienne Riviere (UCLouvain). Cet exposé présentera des résultats obtenus dans le cadre du projet européen LEADS que j'ai coordonné et où l'entreprise Red Hat était partenaire. Le code produit a été intégré dans le “staging" de la base de données NoSQL Infinispan, et évalué avec un équivalent open source de Dropbox développé par CloudSpaces, un autre projet européen.

OSIS19_Cloud : Des objets dans le cloud, et qui y restent -- L'expérience du ...

Pôle Systematic Paris-Region

OSIS19_Cloud : Attribution automatique de ressources pour micro-services, Alt...

Pôle Systematic Paris-Region

OSIS19_IoT : State of the art in security for embedded systems and IoT, by Pi...

Pôle Systematic Paris-Region

Pointers are a notorious "defect attractor", in particular when dynamic memory management is involved. Ada mitigates these issues by having much less need for pointers overall (thanks to first-class arrays, parameter modes, generics) and stricter rules for pointer manipulations that limit access to dangling memory. Still, dynamic memory management in Ada may lead to use-after-free, double-free and memory leaks, and dangling memory issues may lead to runtime exceptions. The SPARK subset of Ada is focused on making it possible to guarantee properties of the program statically, in particular the absence of programming language errors, with a mostly automatic analysis. For that reason, and because static analysis of pointers is notoriously hard to automate, pointers have been forbidden in SPARK until now. We are working at AdaCore since 2017 on including pointer support in SPARK by restricting the use of pointers in programs so that they respect "ownership" constraints, like what is found in Rust. In this talk, I will present the current state of the ownership rules for pointer support in SPARK, and the current state of the implementation in the GNAT compiler and GNATprove prover, as well as our roadmap for the future.

Osis19_IoT: Proof of Pointer Programs with Ownership in SPARK, by Yannick Moy

Pôle Systematic Paris-Region

Osis18_Cloud : Pas de commun sans communauté ?

Pôle Systematic Paris-Region

Osis18_Cloud : Projet Wolphin

Pôle Systematic Paris-Region

La virtualisation est une technologie mature dont le surcoût est aujourd’hui marginal sur les machines grand public. Néamoins, ce surcoût augmente radicalement pour les machines reposant sur une architecture Non Uniform Memory Access (NUMA), omniprésentes dans les data centers. Les techniques de virtualisation actuelles exploitent mal cette architecture et causent une dégradation des performances des applications allant jusqu’à 700%. Cette présentation détaille les causes de telles dégradations et propose une méthode qui permet la virtualisation efficace d’architectures NUMA. Une évaluation de cette méthode montre qu’il est possible de multiplier par 2 ou plus la performance de 9 des 29 applications testées.

Osis18_Cloud : Virtualisation efficace d’architectures NUMA

Pôle Systematic Paris-Region

Nous présentons une solution Open Source de stockage et d’archivage distribué des données dont l’objectif est la pérennité des données. Il est basé sur le protocole BitTorrent et intègre un haut niveau de redondance, ainsi que d’un mécanisme de régénération automatique des données. Il peut être déployé à grande échelle en LAN et en WAN. Les agents sont compatibles avec des serveurs et postes clients Linux, Window ou Mac OS.

Osis18_Cloud : DeepTorrent Stockage distribué perenne basé sur Bittorrent

Pôle Systematic Paris-Region

Dans cet exposé, on présentera OMicroB, une machine virtuelle OCaml pour microcontrôleurs à faibles ressources, inspirée des travaux précédents sur le projet OCaPIC. Cette machine virtuelle, destinée à être exécutée sur diverses architectures matérielles (AVR, PIC, ARM, ...) permet ainsi de factoriser le développement d’applications, mais aussi de généraliser l’analyse et le débogage du bytecode associé, tout en permettant un usage précautionneux de la mémoire. On cible alors des programmes ludiques ou de domotiques destinés à être exécutés sur des microcontrôleurs à faibles ressources, en insistant sur les particularités inhérentes à la programmation de systèmes embarqués.

OSIS18_IoT: L'approche machine virtuelle pour les microcontrôleurs, le projet...

Pôle Systematic Paris-Region

OSIS18_IoT: La securite des objets connectes a bas cout avec l'os et riot

Pôle Systematic Paris-Region

OSIS18_IoT : Solution de mise au point pour les systemes embarques, par Julio...

Pôle Systematic Paris-Region

OSIS18_IoT : Securisation du reseau des objets connectes, par Nicolas LE SAUZ...

Pôle Systematic Paris-Region

The programming language Ada offers unique features to safely program a micro-controller. From the start, Ada was designed to make it difficult to introduce errors, and to make it easy to discover errors that were introduced. For example, language rules enforced at compile time make it possible to have safe concurrency by design. And run-time checking allows immediate detection of what would be "undefined behavior" in C/C++. In the first part of this presentation, we will present the benefits of using Ada for micro-controller programming, including support for debugging on a board. In the second part of this presentation, we will present how the Ada language and its subset SPARK provide a strong foundation for static analyzers, that make it possible to detect errors and provide guarantees on embedded software in Ada/SPARK.

OSIS18_IoT : Ada and SPARK - Defense in Depth for Safe Micro-controller Progr...

Pôle Systematic Paris-Region

OSIS18_IoT : RTEMS pour l'IoT professionnel, par Pierre Ficheux (Smile ECS)

Pôle Systematic Paris-Region

PyParis 2017 / Un mooc python, by thierry parmentelat

Pôle Systematic Paris-Region

PyParis2017 / Python pour les enseignants des classes préparatoires, by Olivi...

Pôle Systematic Paris-Region

Plus de Pôle Systematic Paris-Region (20)