With the availability of relatively cheap, large capacity hard disks, the likelihood of running out of disk space is relatively low. However, we may be in the process of re-organising our data storage by increasing the number, or capacity, of our disk array. Or, we may wish to substitute SSDs for our conventional HDDs.
Anybody who has run out of disk capacity will know just how frustrating it is. Equally, the re-organisation of our data is also time consuming and can be fraught with difficulty. If we have a RAID we can, of course, remove and replace drives on the fly as they become full, or even fail. With LVM we can extend, or shrink, file systems on the fly by changing the capacity of the data pool.
Wouldn't it be great, though, if we could combine all these functions, and more besides, into one set of tools? Furthermore, what if we could have these tools working at the file system level, rather than on a virtual layer above the file system?
Well, Sun Microsystems thought so too and developed the ZFS file system for use in its Solaris OS. Sun is now owned by Oracle but ZFS is still very much alive. Unfortunately, for GNU/Linux users, the CDDL licence that came with ZFS was not compatible with the GPL, so inclusion in the Linux kernel was just not possible. ZFS can be built as a module, but this is not very convenient for a file system, hence, the birth of Btrfs.
Btrfs could be seen as a ZFS for GNU/Linux. It is younger, and therefore not as complete as ZFS, but it is getting there. Linus Torvalds claims that it will be the default GNU/Linux file system when it is ready for production use, so it is worth becoming familiar with it now. The Btrfs file system is built into recent kernels, and additional userland tools, btrfs-progs, are provided for manipulating the file system.
This talk will be a basic introduction for those not yet familiar with the Btrfs file system.
Case study of BtrFS: A fault tolerant File systemKumar Amit Mehta
A case study of Fault Tolerance features of BTRFS. These slides were prepared for the coursework for a Masters level program at Tallinn University of Technology, Estonia. A lot of materials in the slides are taken from the materials in the public domain. Many thanks to the people on BTRFS IRC Channel.
This document compares the file systems BTRFS and ZFS. Both support copy-on-write, snapshots, RAID, compression and other features. BTRFS is the default file system for Linux while ZFS is more robust but has no official Linux kernel support. For Linux systems, BTRFS is recommended over ZFS due to its native support and active development, although ZFS remains more mature and feature-rich. A newer file system called BcacheFS also aims to provide similar functionality.
The Btrfs filesystem was jointly developed by several companies including Oracle and Redhat. It uses copy-on-write and maintains CRCs for metadata and data. Features include snapshots, multi-device support, online resizing and compression. Performance and stability have improved through fixes and new tools allow automated repair of corrupt blocks. Storage is allocated through extent and chunk trees which map logical to physical blocks and allow different RAID levels for data and metadata. Btrfs works well with flash storage through trim support and hints to optimize device lifetime and performance.
Btrfs and Snapper - The Next Steps from Pure Filesystem Features to Integrati...Gábor Nyers
These are the slides of our SUSECon 2013 presentation with Arvin (the inventor of Snapper)
Btrfs as technology has been getting a lot of attention over the past few years. While interesting for its feature set alone, like checksums, copy on write, snapshots and built-in device management, without proper management tooling and integration with other parts of the operating system, it is difficult for the average user to use Btrfs to its full potential.
This session will help you understand the features of Btrfs and how Snapper can be used for snapshot management in SUSE Linux Enterprise. We also will provide use cases and an outlook for future functionality.
Describe the following information.
- Background (Why Fujitsu has contributed to Btrfs)
- Introduction to core features
- Development statistics
- Future prospects
This slide is almost the same as my presentation at LinuxCon Europe 2014. The difference is just minor fixes.
The document discusses the BTRFS file system. It begins with an introduction to BTRFS, describing its creation in 2007 and some of its key features like copy-on-write, snapshots, checksums, and support by some Linux distributions. It then discusses reasons to use BTRFS over other file systems like ZFS or LVM, highlighting advantages like improved integration with Linux and new capabilities for snapshots, online resizing, and checksums. The remainder outlines implementation details of BTRFS and demonstrates some of its features through commands.
The document discusses the Btrfs file system. It was jointly developed by Oracle, Red Hat, Intel and others as a new copy-on-write file system. Btrfs addresses limitations of other Linux file systems by adding features like snapshots, checksums, and defragmentation. It continues to be developed with goals of adding encryption, deduplication, and RAID 5/6 support in the future.
Case study of BtrFS: A fault tolerant File systemKumar Amit Mehta
A case study of Fault Tolerance features of BTRFS. These slides were prepared for the coursework for a Masters level program at Tallinn University of Technology, Estonia. A lot of materials in the slides are taken from the materials in the public domain. Many thanks to the people on BTRFS IRC Channel.
This document compares the file systems BTRFS and ZFS. Both support copy-on-write, snapshots, RAID, compression and other features. BTRFS is the default file system for Linux while ZFS is more robust but has no official Linux kernel support. For Linux systems, BTRFS is recommended over ZFS due to its native support and active development, although ZFS remains more mature and feature-rich. A newer file system called BcacheFS also aims to provide similar functionality.
The Btrfs filesystem was jointly developed by several companies including Oracle and Redhat. It uses copy-on-write and maintains CRCs for metadata and data. Features include snapshots, multi-device support, online resizing and compression. Performance and stability have improved through fixes and new tools allow automated repair of corrupt blocks. Storage is allocated through extent and chunk trees which map logical to physical blocks and allow different RAID levels for data and metadata. Btrfs works well with flash storage through trim support and hints to optimize device lifetime and performance.
Btrfs and Snapper - The Next Steps from Pure Filesystem Features to Integrati...Gábor Nyers
These are the slides of our SUSECon 2013 presentation with Arvin (the inventor of Snapper)
Btrfs as technology has been getting a lot of attention over the past few years. While interesting for its feature set alone, like checksums, copy on write, snapshots and built-in device management, without proper management tooling and integration with other parts of the operating system, it is difficult for the average user to use Btrfs to its full potential.
This session will help you understand the features of Btrfs and how Snapper can be used for snapshot management in SUSE Linux Enterprise. We also will provide use cases and an outlook for future functionality.
Describe the following information.
- Background (Why Fujitsu has contributed to Btrfs)
- Introduction to core features
- Development statistics
- Future prospects
This slide is almost the same as my presentation at LinuxCon Europe 2014. The difference is just minor fixes.
The document discusses the BTRFS file system. It begins with an introduction to BTRFS, describing its creation in 2007 and some of its key features like copy-on-write, snapshots, checksums, and support by some Linux distributions. It then discusses reasons to use BTRFS over other file systems like ZFS or LVM, highlighting advantages like improved integration with Linux and new capabilities for snapshots, online resizing, and checksums. The remainder outlines implementation details of BTRFS and demonstrates some of its features through commands.
The document discusses the Btrfs file system. It was jointly developed by Oracle, Red Hat, Intel and others as a new copy-on-write file system. Btrfs addresses limitations of other Linux file systems by adding features like snapshots, checksums, and defragmentation. It continues to be developed with goals of adding encryption, deduplication, and RAID 5/6 support in the future.
Presentation slides for running MySQL (InnoDB) on ZFS. Since most databases have analogues to optimisation targets mentioned, it is more broadly applicable.
ZFS provides several advantages over traditional block-based filesystems when used with PostgreSQL, including preventing bitrot, improved compression ratios, and write locality. ZFS uses copy-on-write and transactional semantics to ensure data integrity and allow for snapshots and clones. Proper configuration such as enabling compression and using ZFS features like intent logging can optimize performance when used with PostgreSQL's workloads.
ZFS is an innovative file system that provides immense storage capacity, data integrity, and simplified administration. It was developed by Sun Microsystems in 2000 and released in 2005. Some key features of ZFS include its ability to detect and correct errors, provide end-to-end data integrity checks, flexibly pool storage devices, and scale to exabytes of data. While it has limitations like a lack of boot support and encryption, ZFS is widely used with Solaris and is being ported to other platforms like Linux and FreeBSD.
ZFS is a modern file system that provides data integrity through checksums and a copy-on-write design. It simplifies storage administration with a pooled model where space can be shared across file systems. ZFS supports features like snapshots, clones, compression, and deduplication. It can leverage solid state drives for caching and improve performance.
ZFS is a file system developed by Sun Microsystems that provides advanced storage capabilities such as data integrity checking, snapshots and cloning. Some key features of ZFS include using copy-on-write storage, end-to-end checksumming of data to prevent silent data corruption, transactional semantics for consistency, and pooled storage that allows for thin provisioning and easy management of storage resources. ZFS aims to eliminate many of the issues with traditional file systems through its novel approach to data storage and management.
BTRFS is a copy-on-write filesystem for Linux that aims to provide scalable storage, simplified administration, fault tolerance, and easy snapshots. It combines the functionality of a filesystem, volume manager, and RAID controller. While not fully implemented yet, BTRFS goals include self-healing capabilities to avoid data corruption, dynamic allocation of inodes, and space-efficient storage of small files. It shares some similarities with ZFS but was designed for the Linux kernel. Subvolumes allow independent filesystem-like structures that share storage. Snapshots capture the state of subvolumes but writable snapshots are currently the only option. BTRFS continues developing features like read-only snapshots and self-healing capabilities.
ZFS is a filesystem and logical volume manager that provides advanced features like snapshots, compression, and checksumming. It was originally developed at Sun and is now open source under the OpenZFS project. This document discusses how to use and manage ZFS on FreeNAS and PC-BSD operating systems, including creating ZFS pools and datasets, using snapshots and scrubs, and adding features like SLOG and L2ARC devices. Management utilities for these tasks are provided in the GUI and CLI of each OS.
ZFS is a new type of file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability. It uses a pooled storage model that eliminates volumes and partitions, and associated problems. All operations are copy-on-write transactions to keep the on-disk state valid. ZFS provides data integrity through checksumming, self-healing capabilities, and disk scrubbing to detect and repair errors. It is highly scalable with no limits on file systems, files per directory, or other aspects. Snapshots provide space-efficient incremental backups. ZFS delivers improved performance through pipelined I/O and built-in compression.
This document discusses RAID, replication using ZFS, and their benefits. It explains that software RAID like ZFS and Btrfs can provide parity and mirroring with flexibility. ZFS asynchronous replication allows block-level copying of snapshots to another system, preserving metadata and with less load than rsync. This allows fast, efficient off-site backups of entire filesystems including snapshots over time. The document promotes these features as essential for small businesses to reliably protect their data.
The document discusses Linux file systems. It begins with an overview of file system architecture, including inodes, dentries, superblocks, and how data is never erased but overwritten. It then covers various local file systems like Ext2, Ext3, Ext4, ReiserFS, and XFS. Next it discusses log-structured and pseudo file systems. It also covers network file systems like NFS and CIFS. Finally it summarizes cluster, distributed, and Hadoop file systems. The document provides a technical overview of Linux file system types, structures, features and capabilities.
This document provides an overview of the features and management utilities of the ZFS filesystem for FreeNAS and PC-BSD operating systems. It describes key ZFS concepts like pools, RAIDZ levels, datasets, snapshots, scrubs, deduplication, and boot environments. It also outlines how to perform tasks like creating pools and datasets, adding disks, taking and restoring snapshots, and scheduling scrubs on both FreeNAS and PC-BSD. Additional resources for learning more about best practices and advanced ZFS topics are also referenced.
The document discusses Linux file systems and partitioning. It describes how to use the fdisk command to view and create partitions, and supported local file systems like Ext2, Ext3, Vfat, and ISO9660. It provides details on Ext3 file system structure, creation, conversion from Ext2, and tools like dumpe2fs, fsck, and tune2fs. It also covers mounting file systems using mount, automatic mounting from /etc/fstab, and unmounting file systems with umount.
The document provides an overview of a ZFS tutorial presented at USENIX 2009. It discusses the history and features of ZFS, including its layered architecture, use of copy-on-write, pooled storage, and virtual devices (vdevs) like mirroring and RAID-Z. It also covers topics like dynamic striping, checksumming, self-healing, and the ability to replace and resize vdevs without downtime.
ZFS is a modern filesystem designed to add features not found in traditional filesystems. It provides massive storage pools, data integrity checks, snapshots and clones for backup/restore, and more. This document discusses how to use ZFS features in FreeNAS and PC-BSD operating systems through their management utilities, including creating ZFS storage pools and datasets, adding SLOG/L2ARC devices, taking snapshots, and restoring from snapshots. It also covers scrubbing for disk errors and boot environments for rolling back upgrades. Additional resources are provided for learning more about administering ZFS.
The document discusses best practices for deploying MongoDB including sizing hardware with sufficient memory, CPU and I/O; using an appropriate operating system and filesystem; installing and upgrading MongoDB; ensuring durability with replication and backups; implementing security, monitoring performance with tools, and considerations for deploying on Amazon EC2.
The document discusses key features of the ZFS file system including its use of pooled storage, data integrity through checksumming, snapshots and clones, scalability, and dynamic striping. Some advantages of ZFS over traditional file systems are more efficient storage utilization through pooled devices, self-healing capabilities, and faster administration through instant snapshots and clones. Challenges of ZFS include limited adoption, higher CPU and power usage than some traditional file systems, and lack of built-in encryption.
This document discusses configuring and using disk quotas on Linux. It describes:
1. Enabling disk quotas by modifying /etc/fstab and remounting filesystems.
2. Initializing quota tracking with quotacheck and assigning quotas to users with edquota.
3. Creating a virtual ext3 filesystem with quotas to test on, and configuring a shared directory with group quotas.
This document discusses disk and file system troubleshooting and repair. It covers using the newfs command to create file systems, mounting file systems manually and via fstab, checking and repairing file systems with fsck, identifying mounted file systems, unmounting file systems, and configuring and using volume manager. It also discusses recognizing and repairing soft and hard disk errors.
Presentation slides for running MySQL (InnoDB) on ZFS. Since most databases have analogues to optimisation targets mentioned, it is more broadly applicable.
ZFS provides several advantages over traditional block-based filesystems when used with PostgreSQL, including preventing bitrot, improved compression ratios, and write locality. ZFS uses copy-on-write and transactional semantics to ensure data integrity and allow for snapshots and clones. Proper configuration such as enabling compression and using ZFS features like intent logging can optimize performance when used with PostgreSQL's workloads.
ZFS is an innovative file system that provides immense storage capacity, data integrity, and simplified administration. It was developed by Sun Microsystems in 2000 and released in 2005. Some key features of ZFS include its ability to detect and correct errors, provide end-to-end data integrity checks, flexibly pool storage devices, and scale to exabytes of data. While it has limitations like a lack of boot support and encryption, ZFS is widely used with Solaris and is being ported to other platforms like Linux and FreeBSD.
ZFS is a modern file system that provides data integrity through checksums and a copy-on-write design. It simplifies storage administration with a pooled model where space can be shared across file systems. ZFS supports features like snapshots, clones, compression, and deduplication. It can leverage solid state drives for caching and improve performance.
ZFS is a file system developed by Sun Microsystems that provides advanced storage capabilities such as data integrity checking, snapshots and cloning. Some key features of ZFS include using copy-on-write storage, end-to-end checksumming of data to prevent silent data corruption, transactional semantics for consistency, and pooled storage that allows for thin provisioning and easy management of storage resources. ZFS aims to eliminate many of the issues with traditional file systems through its novel approach to data storage and management.
BTRFS is a copy-on-write filesystem for Linux that aims to provide scalable storage, simplified administration, fault tolerance, and easy snapshots. It combines the functionality of a filesystem, volume manager, and RAID controller. While not fully implemented yet, BTRFS goals include self-healing capabilities to avoid data corruption, dynamic allocation of inodes, and space-efficient storage of small files. It shares some similarities with ZFS but was designed for the Linux kernel. Subvolumes allow independent filesystem-like structures that share storage. Snapshots capture the state of subvolumes but writable snapshots are currently the only option. BTRFS continues developing features like read-only snapshots and self-healing capabilities.
ZFS is a filesystem and logical volume manager that provides advanced features like snapshots, compression, and checksumming. It was originally developed at Sun and is now open source under the OpenZFS project. This document discusses how to use and manage ZFS on FreeNAS and PC-BSD operating systems, including creating ZFS pools and datasets, using snapshots and scrubs, and adding features like SLOG and L2ARC devices. Management utilities for these tasks are provided in the GUI and CLI of each OS.
ZFS is a new type of file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability. It uses a pooled storage model that eliminates volumes and partitions, and associated problems. All operations are copy-on-write transactions to keep the on-disk state valid. ZFS provides data integrity through checksumming, self-healing capabilities, and disk scrubbing to detect and repair errors. It is highly scalable with no limits on file systems, files per directory, or other aspects. Snapshots provide space-efficient incremental backups. ZFS delivers improved performance through pipelined I/O and built-in compression.
This document discusses RAID, replication using ZFS, and their benefits. It explains that software RAID like ZFS and Btrfs can provide parity and mirroring with flexibility. ZFS asynchronous replication allows block-level copying of snapshots to another system, preserving metadata and with less load than rsync. This allows fast, efficient off-site backups of entire filesystems including snapshots over time. The document promotes these features as essential for small businesses to reliably protect their data.
The document discusses Linux file systems. It begins with an overview of file system architecture, including inodes, dentries, superblocks, and how data is never erased but overwritten. It then covers various local file systems like Ext2, Ext3, Ext4, ReiserFS, and XFS. Next it discusses log-structured and pseudo file systems. It also covers network file systems like NFS and CIFS. Finally it summarizes cluster, distributed, and Hadoop file systems. The document provides a technical overview of Linux file system types, structures, features and capabilities.
This document provides an overview of the features and management utilities of the ZFS filesystem for FreeNAS and PC-BSD operating systems. It describes key ZFS concepts like pools, RAIDZ levels, datasets, snapshots, scrubs, deduplication, and boot environments. It also outlines how to perform tasks like creating pools and datasets, adding disks, taking and restoring snapshots, and scheduling scrubs on both FreeNAS and PC-BSD. Additional resources for learning more about best practices and advanced ZFS topics are also referenced.
The document discusses Linux file systems and partitioning. It describes how to use the fdisk command to view and create partitions, and supported local file systems like Ext2, Ext3, Vfat, and ISO9660. It provides details on Ext3 file system structure, creation, conversion from Ext2, and tools like dumpe2fs, fsck, and tune2fs. It also covers mounting file systems using mount, automatic mounting from /etc/fstab, and unmounting file systems with umount.
The document provides an overview of a ZFS tutorial presented at USENIX 2009. It discusses the history and features of ZFS, including its layered architecture, use of copy-on-write, pooled storage, and virtual devices (vdevs) like mirroring and RAID-Z. It also covers topics like dynamic striping, checksumming, self-healing, and the ability to replace and resize vdevs without downtime.
ZFS is a modern filesystem designed to add features not found in traditional filesystems. It provides massive storage pools, data integrity checks, snapshots and clones for backup/restore, and more. This document discusses how to use ZFS features in FreeNAS and PC-BSD operating systems through their management utilities, including creating ZFS storage pools and datasets, adding SLOG/L2ARC devices, taking snapshots, and restoring from snapshots. It also covers scrubbing for disk errors and boot environments for rolling back upgrades. Additional resources are provided for learning more about administering ZFS.
The document discusses best practices for deploying MongoDB including sizing hardware with sufficient memory, CPU and I/O; using an appropriate operating system and filesystem; installing and upgrading MongoDB; ensuring durability with replication and backups; implementing security, monitoring performance with tools, and considerations for deploying on Amazon EC2.
The document discusses key features of the ZFS file system including its use of pooled storage, data integrity through checksumming, snapshots and clones, scalability, and dynamic striping. Some advantages of ZFS over traditional file systems are more efficient storage utilization through pooled devices, self-healing capabilities, and faster administration through instant snapshots and clones. Challenges of ZFS include limited adoption, higher CPU and power usage than some traditional file systems, and lack of built-in encryption.
This document discusses configuring and using disk quotas on Linux. It describes:
1. Enabling disk quotas by modifying /etc/fstab and remounting filesystems.
2. Initializing quota tracking with quotacheck and assigning quotas to users with edquota.
3. Creating a virtual ext3 filesystem with quotas to test on, and configuring a shared directory with group quotas.
This document discusses disk and file system troubleshooting and repair. It covers using the newfs command to create file systems, mounting file systems manually and via fstab, checking and repairing file systems with fsck, identifying mounted file systems, unmounting file systems, and configuring and using volume manager. It also discusses recognizing and repairing soft and hard disk errors.
The document discusses the Linux boot process and management tools Grub and Dracut. It provides an overview of the BIOS and UEFI boot methods, the kernel loading process, and the role of the initramfs and systemd. It describes Grub fundamentals like the multi-stage boot process and configuration via grub.cfg. Diagnostic tools like the Grub shell, initramfs shell, and systemd targets are covered. The document concludes with demonstrations of BIOS vs UEFI boot and recovering from a boot failure.
This document outlines steps to mirror the boot disk and partitions across two disks for redundancy. It involves:
1. Creating metadatabases and metamirrors for the root disk (/), swap, and partitions across the two disks.
2. Configuring /etc/vfstab to reference the new metadevices instead of physical disks.
3. Editing /etc/system to set mirrored_root_flag=1 to allow booting from the mirror if 50% of databases are available.
4. Rebooting and attaching the remaining submirrors, then verifying sync progress and boot device configuration.
The document provides information about the boot process on Linux systems from BIOS to kernel loading. It discusses:
- The boot sequence from BIOS performing self-test to loading the master boot record from the boot drive, then the boot loader (LILO or GRUB), and finally the kernel.
- How LILO and GRUB allow selecting operating systems and passing options to the kernel like the root filesystem.
- Common boot loader characteristics and how GRUB uses device names differently than Linux.
- How to recover from a failed boot by booting from removable media and reinstalling LILO or GRUB in the master boot record.
- Common kernel parameters and where to
This document provides instructions on installing a boot manager in Linux. It discusses both LILO and GRUB as options. It describes how LILO works, including creating the /etc/lilo.conf configuration file, running /sbin/lilo to install it, and some common configuration options in lilo.conf like boot devices, timeouts, default boots, and dual booting with other operating systems. The document also provides a long example lilo.conf file with explanations of each setting.
This document provides an overview of installing Linux, including planning partitions and file systems, hardware requirements, choosing between Fedora Core and Red Hat Enterprise Linux, and performing a fresh vs upgraded installation. It discusses setting up partitions, including primary/extended partitions and logical volume management. Recommendations are given for recommended partition sizes. The document also briefly discusses RAID levels and using ISO images to install from CD/DVD.
101 4.3 control mounting and unmounting of filesystemsAcácio Oliveira
Mount points allow partitions and removable drives to be mounted and integrated into the Linux file system hierarchy. Filesystems can be manually mounted using the mount command and specifying the device and mount point directory. Filesystems are unmounted using the umount command on the device or mount point. Mount points are also configured automatically at boot through entries in /etc/fstab that specify the device, filesystem type, and mount options.
This document provides instructions for installing Gentoo Linux. It begins with partitioning the disk using fdisk to create /boot, swap, and encrypted root partitions. The boot and root partitions are formatted with ext2 and ext4 respectively. The encrypted root partition is encrypted with cryptsetup/LUKS. The partitions are mounted and the system is chrooted into. Configuration files like make.conf are edited to optimize the system. Packages are then synced and updated using portage and emerge. Finally, GRUB is installed as the boot loader.
The document provides information about designing hard disk layouts in Linux systems. It discusses partitioning schemes and the use of extended partitions to allow for more than 4 primary partitions. It also covers creating filesystems and swap spaces on partitions using tools like mkfs, mkswap, and mke2fs. Mount points are explained as directories where partitions can be mounted to make their contents accessible in the file system hierarchy.
This document provides instructions for installing, securing, and maintaining FreeBSD servers. It discusses pre-installation planning including partitioning, software selection, and kernel customization. Post-installation tasks covered include rebuilding the operating system to incorporate updates, installing software via packages and ports, and preparing for automated upgrades. The goal is to provide a secure, optimized system tailored to the server's purpose through careful configuration and removal of unnecessary components.
The document discusses how to install and configure a boot loader in Linux. It describes LILO, the Linux Loader, which acts as the boot manager. It explains how LILO works by creating a configuration file, mapping disk locations, and installing the boot loader on the disk. The /etc/lilo.conf file determines LILO's configuration by specifying options like the boot device, default boot target, timeout, and boot images. Running /sbin/lilo installs LILO based on this configuration file. The document also provides an example lilo.conf file and describes configuring LILO for dual booting with another operating system.
The document discusses designing hard disk layouts in Linux systems. It describes partitioning schemes, including extended and logical partitions. It explains how to create filesystems and swap spaces using tools like fdisk, mkfs, mkswap. It also covers formatting disks or partitions, and the various Linux filesystem types and standards like FHS.
Btrfs: Design, Implementation and the Current StatusLukáš Czerner
This document provides an overview of the Btrfs file system including its design, implementation, features and current status. Btrfs was created to address limitations in other Linux file systems like ext4 and XFS by providing improved scalability, reliability, and advanced features. It uses a copy-on-write design with checksumming and flexible btrees to store metadata and data. Key current features include snapshots, checksumming, scrubbing, and volume management. The file system is actively developed but aspects like the file system check and raid 5/6 support remain incomplete.
The document provides an overview of device driver development in Linux, including character device drivers. It discusses topics such as device driver types, kernel subsystems, compiling and loading kernel modules, the __init and __exit macros, character device registration, and issues around reference counting when removing modules. It also provides sample code for a basic character device driver that prints information to the kernel log.
This document provides examples of how to mount and unmount filesystems and partitions in Linux using the mount and umount commands. It discusses:
1. Mounting partitions and filesystems using mount, including CD-ROMs, viewing all mounts, mounting all filesystems listed in /etc/fstab, mounting a specific filesystem from /etc/fstab, viewing mounts of a specific type like ext2, mounting a floppy disk, bind mounting, accessing contents from a new mount point, mounting without writing to /etc/mtab, and mounting with read-only or read-write access.
2. Unmounting partitions and filesystems using umount, including unmounting all filesystems
The document discusses designing hard disk layouts in Linux systems. It covers key areas like allocating filesystems and swap space to separate partitions, tailoring the design to the intended system use, and ensuring boot partition requirements are met. It provides details on partitioning schemes, creating and formatting partitions and filesystems, swap space creation, and the Linux disk naming convention. The goal is to help administrators properly layout disks and partitions for Linux installation and package management.
The document discusses the Linux booting procedure. It describes how when a computer is turned on, the BIOS loads the MBR which contains the bootloader. The bootloader then loads the Linux kernel and initrd. The kernel mounts the root filesystem and executes init to start essential processes and reach the default runlevel. Popular init systems include SystemV, Upstart, and Systemd which boot Linux in different ways. The document also provides an overview of the Linux filesystem structure and organization of important directories.
Similaire à Introduction to Btrfs - FLOSS UK Spring Conference York 2015 (20)
XP 2024 presentation: A New Look to Leadershipsamililja
Presentation slides from XP2024 conference, Bolzano IT. The slides describe a new view to leadership and combines it with anthro-complexity (aka cynefin).
This presentation by Tim Capel, Director of the UK Information Commissioner’s Office Legal Service, was made during the discussion “The Intersection between Competition and Data Privacy” held at the 143rd meeting of the OECD Competition Committee on 13 June 2024. More papers and presentations on the topic can be found at oe.cd/ibcdp.
This presentation was uploaded with the author’s consent.
This presentation by OECD, OECD Secretariat, was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
This presentation by Thibault Schrepel, Associate Professor of Law at Vrije Universiteit Amsterdam University, was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
This presentation by OECD, OECD Secretariat, was made during the discussion “The Intersection between Competition and Data Privacy” held at the 143rd meeting of the OECD Competition Committee on 13 June 2024. More papers and presentations on the topic can be found at oe.cd/ibcdp.
This presentation was uploaded with the author’s consent.
Carrer goals.pptx and their importance in real lifeartemacademy2
Career goals serve as a roadmap for individuals, guiding them toward achieving long-term professional aspirations and personal fulfillment. Establishing clear career goals enables professionals to focus their efforts on developing specific skills, gaining relevant experience, and making strategic decisions that align with their desired career trajectory. By setting both short-term and long-term objectives, individuals can systematically track their progress, make necessary adjustments, and stay motivated. Short-term goals often include acquiring new qualifications, mastering particular competencies, or securing a specific role, while long-term goals might encompass reaching executive positions, becoming industry experts, or launching entrepreneurial ventures.
Moreover, having well-defined career goals fosters a sense of purpose and direction, enhancing job satisfaction and overall productivity. It encourages continuous learning and adaptation, as professionals remain attuned to industry trends and evolving job market demands. Career goals also facilitate better time management and resource allocation, as individuals prioritize tasks and opportunities that advance their professional growth. In addition, articulating career goals can aid in networking and mentorship, as it allows individuals to communicate their aspirations clearly to potential mentors, colleagues, and employers, thereby opening doors to valuable guidance and support. Ultimately, career goals are integral to personal and professional development, driving individuals toward sustained success and fulfillment in their chosen fields.
This presentation by Katharine Kemp, Associate Professor at the Faculty of Law & Justice at UNSW Sydney, was made during the discussion “The Intersection between Competition and Data Privacy” held at the 143rd meeting of the OECD Competition Committee on 13 June 2024. More papers and presentations on the topic can be found at oe.cd/ibcdp.
This presentation was uploaded with the author’s consent.
This presentation by Yong Lim, Professor of Economic Law at Seoul National University School of Law, was made during the discussion “Artificial Intelligence, Data and Competition” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/aicomp.
This presentation was uploaded with the author’s consent.
This presentation by Professor Giuseppe Colangelo, Jean Monnet Professor of European Innovation Policy, was made during the discussion “The Intersection between Competition and Data Privacy” held at the 143rd meeting of the OECD Competition Committee on 13 June 2024. More papers and presentations on the topic can be found at oe.cd/ibcdp.
This presentation was uploaded with the author’s consent.
This presentation by OECD, OECD Secretariat, was made during the discussion “Pro-competitive Industrial Policy” held at the 143rd meeting of the OECD Competition Committee on 12 June 2024. More papers and presentations on the topic can be found at oe.cd/pcip.
This presentation was uploaded with the author’s consent.
Collapsing Narratives: Exploring Non-Linearity • a micro report by Rosie WellsRosie Wells
Insight: In a landscape where traditional narrative structures are giving way to fragmented and non-linear forms of storytelling, there lies immense potential for creativity and exploration.
'Collapsing Narratives: Exploring Non-Linearity' is a micro report from Rosie Wells.
Rosie Wells is an Arts & Cultural Strategist uniquely positioned at the intersection of grassroots and mainstream storytelling.
Their work is focused on developing meaningful and lasting connections that can drive social change.
Please download this presentation to enjoy the hyperlinks!
This presentation by OECD, OECD Secretariat, was made during the discussion “Competition and Regulation in Professions and Occupations” held at the 77th meeting of the OECD Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found at oe.cd/crps.
This presentation was uploaded with the author’s consent.
Suzanne Lagerweij - Influence Without Power - Why Empathy is Your Best Friend...Suzanne Lagerweij
This is a workshop about communication and collaboration. We will experience how we can analyze the reasons for resistance to change (exercise 1) and practice how to improve our conversation style and be more in control and effective in the way we communicate (exercise 2).
This session will use Dave Gray’s Empathy Mapping, Argyris’ Ladder of Inference and The Four Rs from Agile Conversations (Squirrel and Fredrick).
Abstract:
Let’s talk about powerful conversations! We all know how to lead a constructive conversation, right? Then why is it so difficult to have those conversations with people at work, especially those in powerful positions that show resistance to change?
Learning to control and direct conversations takes understanding and practice.
We can combine our innate empathy with our analytical skills to gain a deeper understanding of complex situations at work. Join this session to learn how to prepare for difficult conversations and how to improve our agile conversations in order to be more influential without power. We will use Dave Gray’s Empathy Mapping, Argyris’ Ladder of Inference and The Four Rs from Agile Conversations (Squirrel and Fredrick).
In the session you will experience how preparing and reflecting on your conversation can help you be more influential at work. You will learn how to communicate more effectively with the people needed to achieve positive change. You will leave with a self-revised version of a difficult conversation and a practical model to use when you get back to work.
Come learn more on how to become a real influencer!
The importance of sustainable and efficient computational practices in artificial intelligence (AI) and deep learning has become increasingly critical. This webinar focuses on the intersection of sustainability and AI, highlighting the significance of energy-efficient deep learning, innovative randomization techniques in neural networks, the potential of reservoir computing, and the cutting-edge realm of neuromorphic computing. This webinar aims to connect theoretical knowledge with practical applications and provide insights into how these innovative approaches can lead to more robust, efficient, and environmentally conscious AI systems.
Webinar Speaker: Prof. Claudio Gallicchio, Assistant Professor, University of Pisa
Claudio Gallicchio is an Assistant Professor at the Department of Computer Science of the University of Pisa, Italy. His research involves merging concepts from Deep Learning, Dynamical Systems, and Randomized Neural Systems, and he has co-authored over 100 scientific publications on the subject. He is the founder of the IEEE CIS Task Force on Reservoir Computing, and the co-founder and chair of the IEEE Task Force on Randomization-based Neural Networks and Learning Systems. He is an associate editor of IEEE Transactions on Neural Networks and Learning Systems (TNNLS).
3. Why Btrfs and how does it differ
from other file systems?
● Pooling file system
● Own built-in version of RAID
● Built-in support for SSDs
● Obviates the need for traditional partitions
● Can enable subvolumes and quotas instead of partitions
● Allows snapshots of subvolumes
● Use snapshots for backups
● Use snapshots to roll-back to an earlier state
● Once set up almost everything can be done on the fly
4. Pooling file system – my definition
A pooling file system creates an area of common storage where
data is stored. The pool can be increased in size by adding more
devices, or decreased by removing them, and this can be achieved
on the fly. This is different from the traditional device partitioning
scheme where fixed areas of storage, or partitions, are rigidly
allocated to various parts of the operating system tree.
6. RAID
Btrfs has its own version of RAID built-in
BUT
It differs from MDADM RAID
7. IMPORTANT!
Btrfs RAID 1 keeps only two copies of data and
metadata no matter how many devices you have
in the pool!!!
And those two copies may be held on any two
devices
8. Built-in support for SSDs
Your /etc/fstab mount options could include
ssd,discard
9. Obviates need for traditional
partitioning...
run
mkfs.btrfs /dev/sdx
to format a bare device with the Btrfs file system
11. You can convert an existing partition
1) Unmount the system
umount /dev/sdxx
2) Run a file system check
fsck.extN -f /dev/sdxx
3) Do the conversion
btrfs-convert /dev/sdxx
12. Change your fstab settings
/etc/fstab
new UUID /mount/point btrfs options 0 0
13. Reboot and if all is well delete
ext2_saved
ext2_saved is the subvolume saved for you by
the btrfs-convert tool
Example
btrfs subvolume delete ext2_saved
14. If there is a problem you can
roll-back
So don't delete but rather mount ext2_saved
which has been saved as a subvolume
16. Using a bootable USB flash drive to
boot into a Btrfs RAID 1 rootfs
The boobtable USB flash looks like this from gdisk
Number Start (sector) End (sector) Size Code Name
1 2048 206847 100.0 MiB 8300 Boot
2 206848 19081215 9.0 GiB 8200 Swap
3 19081216 30282974 5.9 GiB 8300 Resc
17. Syslinux can be found here
If anybody is interested
https://www.kernel.org/pub/linux/utils/boot/syslinux/
It's worth it, believe me
18. Download the latest kernel and
btrfs-progs
The latest stable kernel is 3.19.2 from kernel.org
For the latest btrfs-progs run
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
If you don't have git install it first
I guarantee the distro repositories won't have
anything like the latest versions.
19. Let's format our two-device Btrfs
RAID 1 rootfs
mkfs.btrfs -m raid1 -d raid1 /dev/sda /dev/sdb
Clearly, you need to do this from a rescue
system, or externally connect the two devices,
via USB, to a build system, or another system
20. Change your fstab settings again
/etc/fstab
UUID / btrfs device=/dev/sda,device=/dev/sdb 0 0
The mount options where the devices are listed
would be added to with a comma separated list
of your other options. The full list of possible
mount options can be found here
https://btrfs.wiki.kernel.org/index.php/Mount_options
25. Default subvolume
The default subvolume created automatically is
0/5 into which the system boots if no other
subvolume is passed to mount
This can never be deleted
The default subvolume can be toggled with
btrfs subvolume set-default <id> <path>
26. Quotas
Quotas can be applied to subvolumes
Run btrfs quota enable /
Before creating your subvolumes
27. Applying limits to quotas
Example
Run btrfs qgroup limit 1g /var
To limit the size of /var to 1GB
28. Snapshots
Snapshots are just special subvolumes
Basic example for snapshotting
btrfs subvolume snapshot /home/ /home/snapshot
29. Snapshot and backup tools using
Btrfs send/receive
Choose one of these tools if you wish to
automate your backups
https://github.com/masc3d/btrfs-sxbackup # written in python
http://www.digint.ch/btrbk/doc/readme.html # written in perl
https://github.com/ruediste/btrbck # written in java
My favourite is the second written in perl
30. Kernel patch for booting without an
initrd
If you are booting without an initrd you may find
that you need to hack the kernel to stop the
rootfs attempting to load too fast. You can find a
patch here
https://gist.github.com/the8lack8ox/2706705
This will not work with every kernel version, so
you may need to add the appropriate lines of
code manually. The good news is there's not
much to change in the file
31. Testing the file system
After the system boots successfully run
btrfs fi show /
it may return
Label: 'raid 1' uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Total devices 2 FS bytes used 6.63GiB
devid 2 size 29.82GiB used 8.03GiB path /dev/sda1
*** Some devices missing
32. Getting rid of an annoying message
Run: btrfs device scan
You will find that the message has disappeared
btrfs fi show /
Label: 'raid 1' uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Total devices 2 FS bytes used 6.63GiB
devid 1 size 29.82GiB used 8.03GiB path /dev/sdb1
devid 2 size 29.82GiB used 8.03GiB path /dev/sda1
You can add btrfs dev scan to a boot script
33. Scrub your file system
Make sure you run this periodically
btrfs scrub start /
You can test the progress with
btrfs scrub status /
You can run this as a cron job
34. Booting a corrupted root file system
Boot with
rootflags=degraded
as a kernel boot parameter on whichever boot
loader you use
36. Device missing?
If you receive a “device missing” warning when
you run
btrfs fi show /
Then run
btrfs device delete missing /
37. Summary of steps necessary to
replace a failed device
1) Mount in degraded mode
2) Add a new device
3) Remove the missing device
4) Rebalance the data and metadata with
btrfs balance start -dconvert=raid1 -mconvert=raid1 /
38. Checking for free disk space
Use this invocation
btrfs fi df <btrfs mountpoint>
And not this
df -h <btrfs mountpoint>
You can test each and see the difference
40. There is no definitive list of shortened forms
You can make them up as you go along
41. A “shortened-form-failed” example
If instead of btrfs subvolume list we attempted
btrfs s l
The parser would return
: ambiguous token 's'
Did you mean one of these ?
subvolume
scrub
send
42. Shortened form corrected
So if we run
btrfs su l
The command completes as expected
The parser operates on a contextual basis
43. Help from the Web
There are many howtos on the web
Here are links to three
https://btrfs.wiki.kernel.org/index.php/Main_Page
https://wiki.archlinux.org/index.php/Btrfs
https://www.howtoforge.com/a-beginners-guide-to-btrfs
44. Acknowledgements
Gerard Beekmans – The man behind Linux From
Scratch who made it possible initially for us to
build our battery-powered solid-state servers
#btrfs on freenode – a great source of help and
advice
Chris Mason – lead developer of Btrfs
This will be a Simple introduction
Anybody used it
Stop me at any time if you have a question
Other options open single device, RAID 0, RAID 10
Too fully featured to cover everything
Not enough time today
But I wanted to start at the beginning as one would when installing Btrfs for the first time. Listing the necessary steps and flagging possible pitfalls.
The first thing that Btrfs does is to pool all devices.
RAID 0 is much the same as MDADM RAID 0.
Data and metadata striped across as many disks as you have but with no redundancy.
Btrfs RAID 10 is built on those precepts.
You could have added 2 disks or you could have added 20 but Btrfs keeps only two copies.
SSD has optimisations built-in
Rotational in /sys should be set at 0
Discard brings back into service discarded blocks that are no longer in use.
You might still need a partition if you wish to:-
Use different file systems on the same device
Have a swap file
Or you might want to just try btrfs out on a spare partition
If you don&apos;t every action to your new system gets mirrored on the backup subvolume
Don&apos;t forget that the saved subvolume is a copy of your original OS and that wasn&apos;t Btrfs, so your trying to maintain two different file systems.
As a point of interest, I always boot my systems from a 16GB USB flash drive.
I create three GPT partitions. I take it everybody knows what I&apos;m talking about when I say GPT?
“Globally Unique Identifiers Partition Table”. It&apos;s the successor to the legacy MBR style partitioning scheme where we&apos;re stuck with only four primary partitions and are forced to use the extended partition kludge to achieve a greater number of logical partitions.
use Syslinux rather than Grub2 because, while I really liked legacy Grub, I think that Grub2 has become excessively bloated.
Syslinux, on the other hand, is small, light, fast and under constant development, and it&apos;s a pleasure to use.
The other benefit here is that once set up the flash drive can be cloned with dd to boot other boxes from flash drives.
I&apos;m not an expert on distros
Btrfs enabled in your kerne –latest kernel
built-in and not as a module
the userspace btrfs-progs – latest from the kernel devs
major distributions already have Btrfs loaded in the kernel, with the btrfs-progs in the repository
I&apos;m no expert on the various distributions.
In fact, come to think of it, I would never refer to myself as an “expert” on anything. Why is that you may ask? Or maybe not. However, the term “expert” has unfortunate connotations. Maybe some of you know this. Many “experts” that I&apos;ve met over the years, and as you can see there have been many of those years. I&apos;ve been around the block a few times. Well, many of those “experts”, unfortunately, have turned out to be of the type where X was the unknown quantity, and spurt was the drip under pressure. I don&apos;t wish to be associated with those people.
Subvolumes are the key to flexibility
This is where the root file system will be mounted
You don&apos;t have to call it root
These will be mounted on the corresponding directories
Once you have your subvolumes with quotas applied they behave very much like a traditional partition.
The main difference is that they can be shrunk, extended, deleted re-created on a running system.
You don&apos;t have to reboot.
I&apos;m a big fan of Erlang.
Erlang is a fault tolerant language developed by Ericsson back in the 1980s for its telephone switches. It has concurrency built in, and Ericcson claims to have a running system in its labs with the elusive nine nines uptime. An Erlang mantra is “let it crash”, meaning processes can crash and be restarted on a running system.
I&apos;m not saying that Btrfs works in the same manne but there are a lot of cool tings you can do on a running system at the file system level
“missing” is a Btrfs term for a specific device
The GNU Coreutils df tool does not return the correct free space.
I think this is really cool
If you have a number of commands to enter it can save a great deal of time
Some may disagree – it&apos;s a free country
Official Btrfs wiki
Archlinux geared towards systemd and we don&apos;t use that.
We generally build our own systems, although we do use Linux Mint – generally as a demonstration for Windows users to show them what a nice environment they could be working in.
Third one is simple and good but with a caveat: the final column of the fstab should be 0 and not 1