HyperKitty, or how to get the best from mailing lists and forums

•

1 j'aime•3,526 vues

Anne Nicolas

https://distro-recipes.org

Business Technologie

HyperKitty
The next-gen. list archiver

Presented by
Aurélien Bompard
Fedora Engineering

FUDCon Lawrence 2013 − CC-BY

Today's Topics
Where we come from
Mailman today
Máirín's and Luke's vision

Where we are
Mailman 3
HyperKitty

Where we will be
Planned features
How you can help

Mailman 2.1
Maintenance branch
One release a year since 2006

A monolithic package :
Mailing-list engine
Web archiver (pipermail)
Web admin interface

What's wrong
Pipermail generates static HTML
Not directly searchable
No user-based customization
No summary / stats page
Read-only

Admin interface is CGI-based and generates
configuration files

Different environments, technologies,
skillsets

The vision
Early 2010 : Máirín & Luke

Complement the existing mailing-lists
Thread statistics (participants, replies)
List statistics (topics, people)
Post rating
User rating (“karma”)
Thread categories

A new hope
Early 2012 : Fedora Engineering

Refined mockup

Moar mockups !!1

Spring/summer: implementation starts
Toshio, Pingou, Syst3mW0rm, yours truly

Mailman 3
Modern and modular design
Mailman itself: mailing-list and REST API
Postorius: the web admin UI
HyperKitty: the web archiver
Queues, runners, TDD, ...

Key improvements
No more “happy Mailman day” !
No storing of plain-text passwords

When? RSN. Fall 2012: season of the delays

HyperKitty
An archiver plugin for Mailman3

KittyStore: the storage engine
SQL database (Storm ORM)
Analysis and statistics

HyperKitty: the web UI
Django-based
Interfaces with KittyStore
Implements “ModernArchiving” on list.org
Implement some of Mo's mockups

Planned features
Post from the web
A few challenges there (spam, SPF, etc.)

Many ideas in Mo's mockups
Rankings, categories
Improving signal-to-noise ratio

Ideas from the Debian community
Joey Hess (thread patterns)
A. Delanoë and B. Conein (socio-meter)

How you can help
HyperKitty needs you
UI and design
Django knowledge and optimization
Testing, installing, fuzzing, debugging

KittyStore needs you
Statistical analysis
Spam Troll detection

Frustrated with mailing-lists ? Every idea
counts.

Where you can help
Test the development version:
http://mm3test.fedoraproject.org

Report bugs:
http://fedorahosted.org/hyperkitty

Check out the code:
http://github.com/hyperkitty

Read the docs and install it:
http://hyperkitty.rtfd.org

Summary
Mailman 2.1 is growing old, Mailman 3 is
coming

HyperKitty will be its main archiver

Designed by the Fedora community for
everyone

Bridge the gap between mailing lists and
web forums

Questions?
Cool techs, cool designs, huge audience...
Come and be a part of it !

http://fedorahosted.org/hyperkitty
abompard@fedoraproject.org
FUDCon Lawrence 2013 − CC-BY

Contenu connexe

Similaire à HyperKitty, or how to get the best from mailing lists and forums

Clean out the cobwebs.. Joomla for Libraries (Oct-2008)

Mark A. Smith

COMMitMDE'18: Eclipse Hawk: model repository querying as a service

Antonio García-Domínguez

Building a Semantic Web of Comic Book Metadata: User Application Profiles for Publishing Linked Data in HTML/RDFa Kent State University - November 11, 2014 The objective of this research was to present a case study for developing a domain ontology, and explore methodologies for improving the usability and potential usage of that vocabulary through the development of interoperable metadata application profiles designed for specific groups of users within a community. This objective was realized by the development of a metadata vocabulary for comic books and comic book collections, and a series of metadata application profiles designed for publishing Linked Data in the content of existing information systems using HTML/RDFa. Semantic Web standards and technologies represent an opportunity for connecting data about comic books and graphic novels in LOD datasets with detailed, community-created data on the open Web. Recognizing the potential for an open exchange of data about comic books and graphic novels, a case study was designed to gain a comprehensive understanding of the domain and develop an effective data model. The initial phase of the study involved a review of information and reference resources, acquisition of example materials, and practical experience gained indexing comics in a collaborative Web database. A metamodel for comics was then developed and realized as an XML schema, with those elements mapped as properties to classes in an OWL ontology. In order to align the ontology with the wider Web environment and validate the model, the final phase of the case study explored external sources through a review of existing information systems and an analysis of their content. Results were then summarized as skeleton, data-driven user persona documents, which were used to guide the design of a series of metadata application profiles representing the functional requirements identified. The profiles build upon a core schema and incorporate elements from other Web vocabularies as necessary, focusing on publishing Linked Data in existing information systems using HTML/RDFa. Examples were explored and validated for their ability to link to LOD resources and produce meaningful, valid RDF data consistent with the Ontology. The final result is a flexible and extensible, semantic model for comics. The Comic Book Ontology (CBO) as an RDFS/OWL vocabulary is compatible with a variety of other systems, including next-generation library catalogs, where it can potentially be used in a collaborative exchange of data to describe relationships between comics material and content not previously available. This study demonstrates how an ontology can be applied to existing collaborative projects, database, content, or research to enhance the visibility, reference, and utilization of those endeavors through their publication as Linked Data.

Thesis Defense: Building a Semantic Web of Comic Book Metadata

Sean Petiya

Online community platforms and multimedia content delivery are merging in recent years. Current platforms like Facebook and YouTube are client-server based which result in high administration costs for the provider. In contrast to that peer-to-peer systems offer scalability and low costs, but are limited in their functionality. In this paper we present a framework for peer-to-peer based multimedia online communities.We identified the key challenges for this new application of the peer-to-peer paradigm and built a plugin based, easily extendible and multifunctional framework. Further, we identified distributed linked lists as valuable data structure to implement the user profiles, friend lists, groups, photo albums and more. Our framework aims at providing the functionality of common online community platforms combined with the multimedia delivery capabilities of modern peer-to-peer systems, e.g. direct multimedia delivery and access to a distributed multimedia pool.

IEEE ISM 2008: Kalman Graffi: A Distributed Platform for Multimedia Communities

Kalman Graffi

PHP, Lithium and MongoDB

Mitch Pirtle

Web 2.0: new definition of web

Neeraj Singh

Machine Learning - Know Enough To Be Dangerous #SearchLove

Britney Muller

SearchLove San Diego 2019 - Britney Muller - Machine Learning: Know Enough To...

Distilled

Web Futures: Implications For HE

lisbk

ProjectHub

Sematext Group, Inc.

Web 2.0 and LiveQuotes Presentation

Jamie Thingelstad

Tuenti teams - Php Conference

Guille -bisho-

Web browsers have become part of everyday life, and are relied upon by millions of internet citizens each day. The feature rich online world has turned the once simple web browser into a highly complex (and very often insecure) desktop application. As browser vendors have extended functionality and support to new technologies, security researchers and hackers are continuously looking for new vulnerabilities. In this talk, Roberto and Scott will share results of their assiduous browser bug hunting. The talk will examine techniques used to discover critical and less severe vulnerabilities in some of the most popular browsers on the market. This talk will focus heavily (but not exclusively) on the following areas: - Memory corruption bugs; - New approaches to DOM fuzzing; - Old school techniques against new browser technology; - Cross Context Scripting and injection attacks; - SOP Bypass; The presentation will conclude with a montage of on-stage demonstrations of previously unreleased vulnerabilities, including remote code execution, injections and other tailored browser exploits.

Window Shopping Browser - Bug Hunting in 2012

Roberto Suggi Liverani

Mozillamagazine

Marco Casteleijn

sidje

webuploader

Web2 0-SOA InterAct2008

guest1fb6e4

How To Implement a CMS

Jonathan Smith

Desktop integration & ECM

Richard van Delft

Social Web Technologies

hchen1

This session tries to give insights into what the people using, developing and supporting OpenOffice.org want. For this purpose, data from different OpenOffice.org surveys and metrics will be analyzed and presented. One key data source for the presentation will be the ongoing OpenOffice.org user survey which gets completed by about 2000 people every day. In addition, OpenOffice.org developer surveys, external surveys and various OpenOffice.org user and developer metrics will be brought into the picture as well. The session will try to provide answers to questions like “What functionality do different OpenOffice.org users need?”, “How do people want to contribute to OpenOffice.org?”, “What motivates people to contribute to OpenOffice.org?”, etc.

What open source lovers want?

Alexandro Colorado

Similaire à HyperKitty, or how to get the best from mailing lists and forums (20)

Clean out the cobwebs.. Joomla for Libraries (Oct-2008)

COMMitMDE'18: Eclipse Hawk: model repository querying as a service

Thesis Defense: Building a Semantic Web of Comic Book Metadata

IEEE ISM 2008: Kalman Graffi: A Distributed Platform for Multimedia Communities

PHP, Lithium and MongoDB

Web 2.0: new definition of web

Machine Learning - Know Enough To Be Dangerous #SearchLove

SearchLove San Diego 2019 - Britney Muller - Machine Learning: Know Enough To...

Web Futures: Implications For HE

ProjectHub

Web 2.0 and LiveQuotes Presentation

Tuenti teams - Php Conference

Window Shopping Browser - Bug Hunting in 2012

Mozillamagazine

sidje

Web2 0-SOA InterAct2008

How To Implement a CMS

Desktop integration & ECM

Social Web Technologies

What open source lovers want?

Plus de Anne Nicolas

Wanting to avoid the Android experience, Google developers always aimed to make their Chrome OS Linux kernels as close to mainline as possible. However, when Chromebooks were first created, Google was left with no choice, the mainline kernel, in some subsystems, still did not have all the functionalities needed by Chromebooks. Hence, similarly to Android, Chrome OS had to develop their own out-of-tree code for the kernel and maintain that for a few different kernel versions. Luckily, over the last few years a strong and consistent effort has been happening to bring Chromebook devices closer to mainline. It has led to significant improvements that now make it possible to run mainline on Chrome OS devices. And not only Chromebooks, as these significant strides are also improving Arm-based SOCs and other key components of the rich Chromebook hardware ecosystem. In this talk, we will look at how and why upstream support for Chromebooks improved, the current status of various models, and what we expect in the future. Enric Balletbò i Serra

Kernel Recipes 2019 - Driving the industry toward upstream first

Anne Nicolas

As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread. AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU. This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it. Julien Thierry

Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMI

Anne Nicolas

At a rate of almost 9 changes per hour (24/7), the Linux kernel is definitely a scary beast. Bugs are introduced on a daily basis and, through the use of multiple code analyzers, *some* of them are detected and fixed before they hit mainline. Over the course of the last few years, Gustavo has been fixing such bugs and many different issues in every corner of the Linux kernel. Recently, he was in charge of leading the efforts to globally enable -Wimplicit-fallthrough; which appears by default in Linux v5.3. This presentation is a report on all the stuff Gustavo has found and fixed in the kernel with the support of the Core Infrastructure Initiative. Gustavo A.R. Silva

Kernel Recipes 2019 - Hunting and fixing bugs all over the Linux kernel

Anne Nicolas

Kernel Recipes 2019 - Metrics are money

Anne Nicolas

The Linux kernel project includes a huge amount of documentation, but that information has seen little in the way of care over the years. The amount of care has increased significantly recently, though, and things are improving quickly. Listen as the kernel’s documentation maintainer discusses the current state of the kernel’s docs, how we got here, where we’re trying to go, and how you can help. Jonathan Corbet

Kernel Recipes 2019 - Kernel documentation: past, present, and future

Anne Nicolas

Modern SoC designs incorporate technologies from numerous vendors, each with their own inconsistent, confusing, undocumented and even contradictory terminology. The result is a mess of acronyms and product names which have a surprising impact on the ability to develop reusable, modular code thanks to the nature of the underlying IP being obscured. This presentation will dive into some of the misnomers plaguing the Arm ecosystem, with the aim of explaining why things are like they are, how they fit together under the architectural umbrella and how you, as a developer, can decipher the baffling ingredients list of your next SoC design! Will Deacon

Embedded Recipes 2019 - Knowing your ARM from your ARSE: wading through the t...

Anne Nicolas

GNU poke is a new interactive editor for binary data. Not limited to editing basic ntities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Once a user has defined a structure for binary data (usually matching some file format) she can search, inspect, create, shuffle and modify abstract entities such as ELF relocations, MP3 tags, DWARF expressions, partition table entries, and so on, with primitives resembling simple editing of bits and bytes. The program comes with a library of already written descriptions (or “pickles” in poke parlance) for many binary formats. GNU poke is useful in many domains. It is very well suited to aid in the development of programs that operate on binary files, such as assemblers and linkers. This was in fact the primary inspiration that brought me to write it: easily injecting flaws into ELF files in order to reproduce toolchain bugs. Also, due to its flexibility, poke is also very useful for reverse engineering, where the real structure of the data being edited is discovered by experiment, interactively. It is also good for the fast development of prototypes for programs like linkers, compressors or filters, and it provides a convenient foundation to write other utilities such as diff and patch tools for binary files. This talk (unlike Gaul) is divided into four parts. First I will introduce the program and show what it does: from simple bits/bytes editing to user-defined structures. Then I will show some of the internals, and how poke is implemented. The third block will cover the way of using Poke to describe user data, which is to say the art of writing “pickles”. The presentation ends with a status of the project, a call for hackers, and a hint at future works. Jose E. Marchesi

Kernel Recipes 2019 - GNU poke, an extensible editor for structured binary data

Anne Nicolas

Operating system distributors often face challenges that are somewhat different from that of upstream kernel developers. For instance, some kernel updates often need to stay at least binary compatible with modules that might be “out of tree” for some time. In that context, being able to automatically detect and analyze changes to the binary interface exposed by the kernel to its module does have some noticeable value. The Libabigail framework is capable of analyzing ELF binaries along with their accompanying debug info in the DWARF format, detect and report changes in types, functions, variables and ELF symbols. It has historically supported that for user space shared libraries and application so we worked to make it understand the Linux kernel binaries. In this presentation, we are going to present the current support of ABI analysis for Linux Kernel binaries, the challenges we face, how we address them and the plans we have for the future. Dodji Seketeli, Jessica Yu, Matthias Männich

Kernel Recipes 2019 - Analyzing changes to the binary interface exposed by th...

Anne Nicolas

Different upgrade and update strategies exist when it comes to embedded Linux system. If at development time none of these strategies have been chosen, adding them afterwards can be tedious task. Even harder it gets when the system is already deployed in the field and only accessible via a 3G connection. This talk is a developer experience of putting in place exactly that. Giving a return of experience on one way of doing it on a system running Barebox and a Yocto-based distribution. Patrick Boettcher

Embedded Recipes 2019 - Remote update adventures with RAUC, Yocto and Barebox

Anne Nicolas

Traditionally graphics drivers were one of the last hold-outs of proprietary software in an embedded Linux system. This situation is changing with open-source graphics drivers showing up for almost all of the graphics acceleration peripherals on the market right now. This talk will show how open-source graphics drivers are making embedded systems less special, as well as trying to provide an overview of the Linux graphics stack, de-mystifying what is often seen as black magic GPU stuff from outside observers. Lucas Stach

Embedded Recipes 2019 - Making embedded graphics less special

Anne Nicolas

This talk will explore Open Source Hardware projects relevant to Linux, including boards like BeagleBone, Olimex OLinuXino, Giant board and more. Looking at the benefits and challenges of designing Open Source Hardware for a Linux system, along with BeagleBoard.org’s experience of working with community, manufacturers, and distributors to create an Open Source Hardware platform. In closing also looking at the future, Libre Silicon like RISC-V designs, and where this might take Linux. Drew Fustini

Embedded Recipes 2019 - Linux on Open Source Hardware and Libre Silicon

Anne Nicolas

The I2C subsystem is not the shiniest part of the Linux Kernel. For embedded devices, though, it is one of the many puzzle pieces which just have to work. Wolfram Sang has the experience of maintaining this subsystem for nearly 7 years now. This talk gives a short overview of how maintaining works in general and specifically in this subsystem. But mainly, it will highlight noteworthy points in the timeline and lessons learnt from that. It will present trends, not so much regarding I2C but more the Linux Kernel and the embedded ecosystem in general. And of course, there will be plenty of anecdotes and bits from behind the scenes for your entertainment. Wolfram Sang

Embedded Recipes 2019 - From maintaining I2C to the big (embedded) picture

Anne Nicolas

ITRenew is selling recertified OCP servers under the Sesame brand, those servers come either with their original UEFI BIOS or with LinuxBoot. The LinuxBoot project is pushing the Linux kernel inside bios flash and using userland programs as bootloader. To achieve quality on our software stack, as any project, we need to test it. Traditional BIOS are tested by hand, this is 2019 we need to do it automatically! We already presented the hardware setup behind the LinuxBoot CI, this talk will focus on the software. We use u-root for our userland bootloader; this software is written in Go so we naturally choose to use Go for our testing too. We will present how we are using and extending the Go native test framework `go test` for testing embedded systems (serial console) and improving the report format for integration to a CI. Julien Viard de Galbert

Embedded Recipes 2019 - Testing firmware the devops way

Anne Nicolas

About 60% of the Linux kernel source tree is devoted to drivers for a large variety of supported hardware components. Especially in the embedded world, the number of different SoC families, versions, and revisions, integrating a myriad of “IP cores”, keeps on growing. In this presentation, Geert will explain how to match drivers against hardware, and how to support a wide variety of (dis)similar devices, without turning platform and driver code into an entangled bowl of spaghetti.tra Starting with a brief history of driver matching in Linux, he will fast-forward to device-tree based matching. He will discuss ways to handle slight variations of the same hardware devices, and different SoC revisions, each with their own quirks and bugs. Finally, Geert will show best practices for evolving device drivers in a maintainable way, based on his experiences as an embedded Linux kernel developer and maintainer. Geert Uytterhoeven

Embedded Recipes 2019 - Herd your socs become a matchmaker

Anne Nicolas

Buildroot is a popular and easy to use embedded Linux build system. It generates, in few minutes, lightweight and customized Linux systems, including the cross-compilation toolchain, kernel and bootloader images, as well as a wide variety of userspace libraries and programs. This talk is about the integration of LLVM/clang into Buildroot. In 2018, Valentin Korenblit, supervised by Romain Naour, worked on this topic during his internship at Smile ECS. After a short introduction about llvm/clang and Buildroot, this talk will go through the numerous issues discovered while adding llvm/clang componants and how these issues were fixed. Romain will also detail the work in progress and the work to be done based on llvm/clang libraries (OpenCL, Compiler-rt, BCC. Chromium, ldd). Romain Naour

Embedded Recipes 2019 - LLVM / Clang integration

Anne Nicolas

This talk introduces JTAG debugging capabilities, both for debugging hardware and software. Marek first explains what the JTAG stands for and explains the operation of the JTAG state machine. This is followed by an introduction to free software JTAG tools, OpenOCD and urJTAG. Marek shortly explains how to debug software using those tools and how that ties into the JTAG state machine. However, JTAG was designed for testing hardware. Marek explains what boundary scan testing (BST) is, what are BSDL files and their format, and practically demonstrates how to blink an LED using BST and only free software tools. Marek Vasut

Embedded Recipes 2019 - Introduction to JTAG debugging

Anne Nicolas

PipeWire is an open source project that aims to greatly improve audio and video handling under Linux. Utilising a fresh design, it bridges use cases that have been previously addressed by different tools – or not addressed at all -, providing ground for building complex, yet secure and efficient, multimedia systems. In this talk, Julien is going to present the PipeWire project and the concepts that make up its design. In addition, he is going to give an update of the current and future work going on around PipeWire, both upstream and in Automotive Grade Linux, an early adopter that Julien is actively working on. Julian Bouzas

Embedded Recipes 2019 - Pipewire a new foundation for embedded multimedia

Anne Nicolas

Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls! Steven Rostedt

Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started

Anne Nicolas

Suricata is a network threat detection engine using network packets capture to reconstruct the traffic till the application layer and find threats on the network using rules that define behavior to detect. This task is really CPU intensive and discarding non interesting traffic is a solution to enable a scaling of Suricata to 40gbps and other. This talk will present the latest evolution of Suricata that knows uses eBPF and XDP to bypass traffic. Suricata 5.0 is supporting the hardware XDP to provide ypass with network card such as Netronome. It also takes advantage of pinned maps to get persistance of the bypassed flows. This talk will cover the different usage of XDP and eBPF in Suricata and shows how it impact performance and usability. If development time permit, the talk will also cover AF_XDP and the impact on this new capture method on Suricata. Eric Leblond

Kernel Recipes 2019 - Suricata and XDP

Anne Nicolas

System memory configuration is a transparent operation nowadays, something that we all came to expect to just work out of the box. Still, it does happen behind the scenes every single time we boot our computers. This requires the cooperation of hardware components on the mainboard and on memory modules themselves, as well as firmware code to drive these. While it is possible to just let it happen, having a deeper understanding of how it works makes it possible to access valuable information from the operating system at run-time. I will take you through the history of system memory configuration from the mid 70s to now. We will explore the different types of memory modules, how their configuration data is stored and how the firmware can access them. We will see which problems had to be solved along the way and how they were solved. Lastly we will see how Linux supports reading the memory configuration information and what you can do with that information. Jean Delvare

Kernel Recipes 2019 - Marvels of Memory Auto-configuration (SPD)

Anne Nicolas

Plus de Anne Nicolas (20)