This chapter describes Resource Monitoring and management

1. CHAPTER FOUR
Resource Monitoring and management

2. process and Disks
System and NetworkAdministration
● Controlling Processes
– Components of a process
– Life cycle of a process
– Signals
– Send signals using kill and killall
– Process states
– Influence scheduling priority with nice and renice
– Monitoring processes with ps and top
– Runaway processes
– Periodic processes

3. Components of a process
– A process is the instantiation of a program.
– From the kernel's perspective, a process is:
● An address space (the set of memory pages with code,
libraries, and data)
● Set of data structures (within the kernel)
– The process's address space map
– Current status
– Execution priority
– Resources used
– Signal mask (which signals are blocked)
– The owner
– Which instructions are currently being executed

4. Process attributes
● Process ID – PID
– Unique identifier, wraps around
● Parent PID – PPID
– When a process is cloned, there is a parent and a child
● Real and effective user ID – UID and EUID
– EUID is used to determine what permissions the process has
– Also records original EUID (saved UID)
● Can be re-accessed later in program (even after changing EUID)
● Real and effective group ID – GID and EGID
● Niceness
– The CPU time available depends on its scheduling priority
– Users can make their processes 'nicer' to the rest of the system
● Control terminal – where stdin, stdout, stderr are attached

5. Signals
● Signals are process-level interrupt requests
● Uses
– Inter-process communication
– Terminal driver can kill, interrupt or suspend
processes (Ctrl-C, Ctrl-Z)
– Can be sent by admin (with kill) for various purposes
– Can be sent by kernel when process breaks a rule
● e.g., division by zero
– Can be sent by kernel for i/o available, death of child

6. Handling signals
– Process can designate a signal handler for a
particular signal
– If no handler, kernel takes some default action
– When handler is finished catching signal, execution
continues where the signal was received
– Process can request that particular signals be
ignored, or blocked
– If signal is received while blocked, one instance of that
signal is buffered until it is unblocked

7. Important signals

8. Sending signals
# kill [-signal] pid
# kill sends TERM signal by default
# kill -9 pid === kill -KILL pid
– “Guarantees” that the process will die
# kill -USR1 910 3044
# sudo killall -USR1 httpd
– killall removes need for pid

9. Process states
● Process exist in one of four states
– Runnable – can be executed
– Sleeping – waiting for some resources
● Gets no CPU time until resource is available
– Zombie – trying to die (parent hasn't waited)
– Stopped – process is suspended (i.e., not
permitted to run)
● Like sleeping, but can't wake until CONT received

10. Scheduling priority
– “Niceness” is hint to kernel about how often to
schedule the process
– Linux ranges from -20 (high priority, not nice) to +19 (low
priority, very nice), 0 is default
– User/process can raise, but not lower niceness
● Root can lower
– Examples
% nice +5 ~/bin/longtask
% renice -5 8829
% sudo renice 5 -u boggs

11. Monitoring processes: ps
● /bin/ps primary tool
● Shows
– PID, UID, priority, control terminal
– Memory usage, CPU time, status
● Multiple variations of ps
– ps -aux (BSD, Linux)
– ps -Af (Solaris)

12. Example ps output

13. Monitoring processes: top
● /usr/bin/top is optional in some OSes
● Shows top-n CPU-using processes
– Plus other stats, like memory usage and
availability, system load
– Can renice within top
– Automatically refreshes screen every 5 seconds
– Can focus on a particular user

14. Sample top output

15. Runaway processes
● What can you do about processes using an
unusual amount of resources (memory, CPU,
disk space)?
– Identify resource hogs using top and/or ps
– Contact owner and ask about resource usage
– Suspend using STOP signal (might break job)
● Contact owner, restart or kill later
– Renice CPU hog

16. Creating periodic processes
● Automation, as you've heard, is key to efficiency
● Instead of manually performing tasks daily,
weekly, or monthly, you can schedule them
– cron
– anacron
● Includes tasks like:
– monitoring, log rotation, backups, file distribution

17. cron
– cron daemon performs tasks at scheduled times
– crontab files are examined by cron for schedule
● /etc/crontab, /etc/cron.d/*, /var/spool/cron/*
– cron wakes up each minute and checks to see if
anything needs to be executed
– cron is susceptible to changes in time
● doesn't compensate for when machine is down, or time
changes (clock adjustments or daylight savings time) that are
sufficiently large (3 hours, at least for some implementations)
– anacron works daily
● records when task last performed, and will catch up with
missing time

18. crontab files
– Filename provides username in /var/spool/cron/
– Example crontab entries:
# run make at 2:30 each Monday morning
30 2 * * 1 (cd /home/joe4/project; make)

19. Managing crontabs
● Use crontab -e to edit
– Checks out a copy
– Uses EDITOR environment variable
– Resubmits it to the /var/spool/cron/ directory
● crontab -l will list the contents to stdout
● /etc/cron.allow and /etc/cron.deny can control access to
cron facilities

20. Using cron
– Distributions set up crontab entries to automatically
run scripts in
● /etc/cron.monthly/
● /etc/cron.weekly/
● /etc/cron.daily/
● /etc/cron.hourly/
– Typical tasks:
● Cleaning the filesystem (editor files, core files) using find
● Distributing files (mail aliases, sendmail config, etc.)
using: rsync, rdist, or expect
● Log rotation

21. Disks
● Partitions
● Volumes
● Filesystems
● Files

22. Adding a disk to Linux
STEP-BY-STEP (w/out LVM)
– Install new hardware
● verify that hardware is recognized by BIOS or controller
– Boot, make certain device files already exist in /dev
● e.g., /dev/sdc
– Use fdisk/to partition the drive
● Verify the system type on each partition
– Use mke2fs (-t ext4) on each regular partition
● To create (an ext4) filesystem
– Use mkswap to initialize swap partitions
– Add entries to /etc/fstab
– Mount by hand, then reboot to verify everything

23. hdparm: test/set hd params
● hdparm will do simple performance tests

24. Disk partitions
– Drives are divided into one or more partitions that are
treated independently
● Partitions make backups easier, confine damage
– Typically have at least two or three
● root partition (one)
– everything needed to bring system up in single-user mode
(often copied onto another disk for emergencies)
● swap partition (at least one)
– stores virtual memory when physical memory is insufficient
● user partition(s)
– home directories, data files, etc.
● boot partition - boot loader, kernel, etc.

25. Logical Volumes
– Partitions are static, and sometimes you want to
change them
– LVM (Linux Logical Volume Manager) lets you combine
partitions and drives to present an aggregate volume as a
regular block device (just like a disk or partition)
● Use and allocate storage more efficiently
● Move logical volumes among different physical devices
● Grow and shrink logical volume sizes on the fly
● Take “snapshots” of whole filesystems
● Replace on-line drives without interrupting service
– Similar systems are available for other OSes

26. LVM
Sample organization:

27. Filesystems
– Linux filesystems are created in partitions or volumes
● ext2fs (2nd Extended File System) is old
● ext3fs (3rd Extended File System) is common
● ext4fs (Fourth Extended File System) is modern
– Speeds large directories
– Compatible with ext2 and ext3
● Other filesystems also supported
– ReiserFS, IBM's JFS, SGI's XFS
● Can read foreign filesystems (e.g., FAT, NTFS, ISO 9660)

28. ext# filesystems
– For ext2/ext3/ext4, mke2fs is used, which creates
● A set of inode storage cells
– each holds info about one file
● A set of scattered “superblocks”
– holds global filesystem info (multiple copies for reliability)
– size and location of inode tables, block map and usage,
etc.
● A map of the disk blocks in the filesystem (used and free)
● The set of data blocks

29. Mounting a filesystem
– Filesystem must be mounted before use
● Must be made part of root filesystem
– Can be mounted on (top of) any directory
# mount /dev/sda1 /usr/local
# df /usr/local
– Use /mnt for temporary mounts
– Want to set up automatic mounting

30. 4.1. printers
● Printing and print services
– Printing policies and architecture
– Printing terms
– Types of printers
– Adding a printer
– Common printing software

31. Where should printers be located?
– Some want a printer on their
own desk
● Very convenient but expensive
– Some want to be able to print
to any printer, no matter where it is
● Flexible, able to borrow specialty printers as needed
– Finance people want to centralize everything
● A single high-speed printer, single high-quality printer,
and one color printer per building (most cost-effective)
– Others want to charge every expense
● Regardless of how much is out there, those who use it,
pay for it

32. Real world
● People need to be able to print to any printer
they have permission to use
● Centralized printing services can save money
– Ten people who might otherwise buy slow, lowquality
personal printers
● The sysadmin only has to support one printer
driver/printer rather than 10

33. Print architecture
● How centralized will printing be?
– How many people will share a printer for general
printing?
– Who qualifies for a personal printer?
– How will they be networked?
● Network printers can benefit from a central print-spool
– How will they be maintained?

34. Printing terms
● spooler
– Daemon that receives print
jobs,
stores, prioritizes, and sends
them sequentially to be printed
● PDL
– Page Description Language,
usually device and resolution
independent
– PostScript, PCL, PDF
● bitmap
– JPEG, TIFF, GIF, PNG
● RIP
– Raster image processor
– Accepts PDL input, generates
bitmap appropriate for a
particular device
● filters
– Modify print jobs on their way
to a printer
● PostScript
– Most common PDL – also a
full programming language

35. Types of printers
– Classified by connection interface
● Serial and parallel printers
– USB faster and the default today for personal printers
● Network printers
– Contain network interfaces (e.g., ethernet or wifi)
– Accept jobs via one or more printing protocols
● including via, IPP, HP JetDirect

36. CUPS
– Common UNIX Printing System
● Latest rewrite of the printing system
– Also supports secure printing (SSL, etc.)
– Implements IPP: Internet Printing Protocol (HTTPbased)
– Supports load-balancing across a class of printers
– Supports automatic network configuration
– Standard in most Linux distributions
http://www.cups.org/

37. Network and system monitoring tools
• The tools that monitor the infrastructure should be
designed to add the least performance impact on the
system that is being monitored. In other words, these
tools need to be as unobtrusive as possible to avoid any
added burden on the production services.
• Monitoring tools can be categorized as:
Command Line Tools
Desktop Monitoring Tools
Infrastructure Monitoring Tools
Log Monitoring Tools
Network Monitoring Tools

38. Considerations of Selecting Network
monitoring Tools
• When looking for a system monitoring software package
there are many things to consider. Apart from the basic
monitoring functionalities such as processes and system
performance, it is essential that the tool allows the
following features
Customizable alerts and notifications.
 Auto-discovery of systems.
Easy graphical user interface.
Automatic update measurements.
Easy Installation and operation.

39. Project on Network and system
monitoring tools
• Introduction about the tool
• Applications (usage) of tool
• Installation steps
• Additional tools or plugins that can enhance performance
of the tool
• Configuring network parameters
• Special features of the tool
• Steps to run the tool features (Note: support the steps
with example)
• Conclusion

40. Tools
1. Nagios
2. Gnome system monitor
3. Cacti
4. KDE system Guard
5. Solarwinds SAM (Server & Application Monitor)
6. PRTG by Paessler
7. OPManager by ManageEngine
8. WhatsUp Gold
9. Zabbix
10. Spiceworks
11. Sarg (Squid Analysis Report Generator)
12. GoAccess

41. Next chapter 5: Data center security