This session will first discuss basic analysis of a Linux system with pre-installed tools. You will learn several commands to help you understand Linux issues better. In the second step, you will learn how to fix a broken Linux system. The session will teach some major troubleshooting techniques and will be filled with lots of demos that will show you how to fix real problems on real broken systems.

1. Diagnostics of a
Linux System
Understanding Your Operating System
Stephan Barth Sander van Vugt
Senior Technical Support Engineer Independent Trainer and Consultant
stephan.barth@novell.com mail@sandervanvugt.nl

2. Overview
A Few Words About Us
Basic Parts of a Linux System (theory)
Analyzing and Debugging
Other Important Tools and Commands
Demo: Troubleshooting Server Issues
2 © Novell, Inc. All rights reserved.

3. Basic Parts of a Linux System

4. Basic Parts of a Linux System
Boot loader, e.g. Grub, LILO
Kernel
glibc
init
4 © Novell, Inc. All rights reserved.

5. Basic Parts of a Linux System
• Monolithic kernel with modules
• Compiled in options and drivers
– zcat /proc/config.gz
• Modules that are being loaded
– lsmod
– dmesg
– /proc and /sys
5 © Novell, Inc. All rights reserved.

6. Basic Parts of a Linux System
• Glibc
– # /lib/libc.so.6
– /etc/ld.so.conf
– /etc/ld.so.cache
– ldconfig
– Timezone
– rpm -q --provides glibc
– ldd and nm
– More info with: info glibc
• Try to compile packages on your own
6 © Novell, Inc. All rights reserved.

7. Basic Parts of a Linux System
• init
• ldd /sbin/init
• /etc/inittab
– initial boot
– runlevels
– tty's
Demo
7 © Novell, Inc. All rights reserved.

8. Understanding Your Operating System
• What happens during boot?
– Kernel start up and initrd
– boot options
– boot.msg
> Examples
– module options
– debug kernel
– MCE
– hal
– lspci, lsusb
8 © Novell, Inc. All rights reserved.

9. Understanding Kernel Loading
from GRUB
• GRUB is the first thing that runs when booting the server
• Responsible for loading kernel and initrd
• root file system is activated as a kernel argument
• Specific options can be loaded from the boot prompt
Demo
9 © Novell, Inc. All rights reserved.

10. Analyzing and Debugging

11. Application Debugging
• ldd $(which gpg)
• rpm -q --requires gpg2
• strace / ltrace
• objdump
• GNU debugger gdb
• Valgrind
• Closed source software
11 © Novell, Inc. All rights reserved.

12. Running Services
strace/ltrace
Increase debug level
Change init script – very last option
12 © Novell, Inc. All rights reserved.

13. Memory and Storage
• Memory debugging is difficult
– ps
– top
– sar
• Disk usage
– df vs. du
13 © Novell, Inc. All rights reserved.

14. Other Important Tools and Commands

15. Miscellaneous Tools
• Parted
• Ethtool
• RPM
– rpm -qf
– rpm -qa --last
– zypper
• How to find files, packages: ARCHIVES.gz, pin, distpin
15 © Novell, Inc. All rights reserved.

16. Looking at Startup
• Different stages in startup
– GRUB
– loading of kernel and initrd
– init taking over and doing initial boot
– init starting runlevels
Demo
16 © Novell, Inc. All rights reserved.

17. Novell Tools ®
supportconfig
Novell Support Advisor
17 © Novell, Inc. All rights reserved.

18. Demo: Troubleshooting Server Issues

20. Unpublished Work of Novell, Inc. All Rights Reserved.
This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc.
Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope
of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified,
translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc.
Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General Disclaimer
This document is not to be construed as a promise by any participating company to develop, deliver, or market a
product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in
making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents
of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any
particular purpose. The development, release, and timing of features or functionality described for Novell products
remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to
make changes to its content, at any time, without obligation to notify any person or entity of such revisions or
changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc.
in the United States and other countries. All third-party trademarks are the property of their respective owners.