LVE Stats2 is a complete re-write of our customer’s statistics module in CloudLinux OS. It features more detailed charts, flexible architecture, and ability to extend the functionality. In this presentation, Igor Seletskiy, our CEO, discusses LVE Stats2.

1. LVE Stats 2 Training

2. • Modular, extendable and efficient
architecture
o Can monitor anything
o Customize what is monitored
 Use RES mem instead of LVE PMEM
o In memory data analysis - better
performance
o Easy to extend - suspend, notify, etc…
Why LVE Stats v2?

3. • Better precision
o Only 1 month of data vs 2 months for v1
o Yet, 1 minute intervals vs 1 hour intervals
Why LVE Stats v2?

4. • Better charts (SVG)
• Include DB Governor statistics
Why LVE Stats v2?

5. • Faults for CPU/IO
o Not ‘faultable’ resources, just throttled
o We count it as a ‘fault’, once per minute, if limit reached
o Easier to identify/see when user hits the limit
Why LVE Stats v2?

6. • Collects snapshots
o URL
o Process
o Query
Why LVE Stats v2?

7. • Backwards compatible by default
o By default lve-stats-2.x runs in backward
compatibility mode.
 lveinfo and lvechart run as before
 Will be changed in ~12 months
• Make new mode default by setting mode=v2 in
/etc/sysconfig/lvestats2
o Or use -m v1|v2 option in lveinfo.
Backwards compatible

8. • lveinfo/lvechart are backward
compatible
o Old options should work the same
• /var/lve/info populated as before
o Every 5 seconds, used by control panel to
display ‘live’ information
What stayed the same?

9. • --dbgov -- allows to manipulate DB Governor stats
• -c/--csv [FILE] -- saves data in CSV format
• -j/--json -- json output
• --time-unit -- averages time in output based on the
time unit
• --servers-info for multi-server install, lists servers &
LVE version
• -m v1/v2 -- v1 display output in backward
compatible mode, v2 - new mode
• --blank-value [val] -- substitute instead of
unsupported limit, default -
lveinfo

10. • -u/--user -- show stats for particular MySQL user
• -l/--limit -- max number of entries in output
• -o/--order-by -- order output by
con/cpu/read/write
• -b/--format -- columns to show
• --csv/--json -- display data in csv or json formats
• --from/--to/--period -- same as lveinfo
lveinfo --dbgov

11. • Includes DB Governor data on CPU/IO charts
• --format {svg,png} -- allows to use SVG or PNG format, default SVG
What’s new: lvechart

12. • Top like utility for LVE
o will replace lvetop
• Powers new UI
• Includes DB Governor data
• Supports --json only for now (more later, including live/on screen)
o Used in the next version UI for cPanel/DA/Plesk
• Takes data from:
o /var/lve/cloudlinux_top.json file
o Updated by lvestast v2 every 5 seconds
What’s new: cloudlinux-top

13. • - hide-mysql - don’t show DB Governor related data
• - u/--username - show data only for specific user,
substring matching
• - d/--domain - show data only for specific domain,
substring matching
• - m/--max - max number of entries to show, default 25
cloudlinux-top options:

14. • dbgovchart - show DBGovernor charts
• lve-create-db -- creates or re-creates database
for LVE Stats v2.0
• lve-read-snapshot -- reads snapshots for end
user. Can produce data in json, or produce
‘summary’ stats for a particular period
More commands...

15. • Not on CL5
Storing History: Supported Databases

16. • Automated upgrade from lve-stats 1.x
once released to production channel
• Automated settings migrations
• Data migrated automatically within first
24 hours
o Throttled, so disks are not overloaded
o No support for migrating stats on centralized
storage
Upgrading: Migrating data

17. • /var/lve/lvestats2.db -- SQLite db
with all the data
• /var/lve/info -- ‘real time’ usage to
show in control panels
• /var/lve/cloudlinux_top.json -- data
for cloudlinux-top command
Important files

18. • /etc/sysconfig/lvestats2 -- main config file
• /etc/sysconfig/lvestats.config/ -- config
files for Plugins
o HistoryCleaner.cfg -- how often to remove
data from db
o SnapshotSaver.cfg -- snapshot settings (how
many, how often)
o StatsNotifier.cfg -- Notifications settings
Config files

19. • db_type = sqlite
• plugins=/usr/share/lve-stats/plugins
• keep_history_days=30
• server_id=7269a505-3
• mode=v1
...
/etc/sysconfig/lvestats2

20. • NOTIFY_FROM_EMAIL=main_admin@host.com
• NOTIFY_ADMIN=N
• NOTIFY_RESELLER=N
• NOTIFY_CUSTOMER=N
• NOTIFY_INCLUDE_RESELLER_CUSTOMER=N
• NOTIFY_CPU=Y
• NOTIFY_IO=Y
• NOTIFY_IOPS=Y
• NOTIFY_MEMORY=Y
• NOTIFY_EP=Y
• NOTIFY_NPROC=Y
• NOTIFY_MIN_FAULTS_ADMIN=1
• NOTIFY_MIN_FAULTS_USER=1
• NOTIFY_INTERVAL_ADMIN=12h
• NOTIFY_INTERVAL_USER=12h
StatsNotifer.cfg –
Notify when limited

21. Notification message example

22. • period_between_incidents=300
• snapshots_per_minute=2
• max_snapshots_per_incident=10
SnapshotSaver.cfg

23. • Easy way to extend lvestats
• Simple Architecture
• Python based
• http://docs.cloudlinux.com/index.html?creating_a_
plugin.html
Plugins

24. • Collectors
• Analyzers
• Persistors
• Notifiers
collectors
analyzers
Persistors
Notifiers
Plugin Types

25. • Generic / always included
• Optional
o LVEDestroyer
o ResMEMCollector
• Custom
More about plugins

26. • period -- how often to execute, default 5
seconds
• order -- order in the execution sequence,
to decide which plugin goes next
• Any other settings can be set using config
files
Plugin Properties

27. Name Period Order Default
LVECollector 5 1000 Y
CPUInfoCollector 5 2000 Y
LVEUsernamesCollector 3600 3000 Y
LVEUsageAnalyzer 5 4000 Y
LveUsageAggregator 60 5000 Y
DBGovSaver 5 6000 Y
FileSaver 5 7000 Y
CloudLinuxTopFileSaver 60 8000 Y
Plugins

28. DBSaver 60 9000 Y
DbUsernamesSaver 3600 10000 Y
DBSaverX60 3600 11000 Y
SnapshotSaver 30 12000 Y
StatsNotifier varied 13000 Y
HistoryCleaner 3600 14000 Y
ResMEMCollector 30 1500 N
LVEDestroyer 5 - N
Plugins
Name Period Order Default

29. • To enable plugin, copy or link it to /usr/share/lve-
stats/plugins director
• For example to enable ResMEMCollector plugin, do
ln -s /opt/alt/python27/lib/python2.7/site-
packages/lvestats/plugins/generic/res_mem_collecto
r.py /usr/share/lve-stats/plugins/
Adding plugins

30. # FSize_watcher_collector.py
# Example plugin for monitoring file size.
# Part 1. Collector
import os
from lvestats.core.plugin import LveStatsPlugin
# Key name
COLLECTOR_KEY = 'FSizeWatcher_fsize'
COLLECTOR_KEY_FILENAME = 'FSizeWatcher_fname'
class FSize_watcher_collector (LveStatsPlugin):
# this plugin should be first in chain
order = 0
# File to monitoring
file_to_monitoring = None
def __init__(self):
pass
# Sets configuration to plugin
def set_config(self, config):
self.file_to_monitoring = config.get('file_to_monitoring', None)
pass
# Work method
def execute(self, lve_data):
try:
# if monitoring file absent, do nothing
if self.file_to_monitoring is None or not os.path.exists(self.file_to_monitoring):
return
# Get file size
stat_info = os.stat(self.file_to_monitoring)
fsize = stat_info.st_size
# Place file name and file size to server data dictionary
lve_data[COLLECTOR_KEY_FILENAME] = self.file_to_monitoring
lve_data[COLLECTOR_KEY] = fsize
except (OSError, IOError):
# file absent or any other error - remove size from dictionary
del lve_data[COLLECTOR_KEY]
Example Plugin

31. • LVE_VERSION LVE version.
• stats -- Dictionary, that contains lve id’s as keys and LVEStat class objects as values. Every LVEStat object
contains values of usages and limits taken from /proc/lve/list file for every LVE Id. Dictionary keys –
integer lve id, including 0 for “default” LVE. This dictionary is updated on each iteration of lvestats-server
(every 5 seconds by default).
LVEStat – is a standard server class, it can be imported with the command from lvestats.core.lvestat
import LVEStat.
The class is described in the file /opt/alt/python27/lib/python2.7/site-packages/lvestats/core/lvestat.py.
Here you can find the whole list of data fields and their functions.
• old_stats -- stats content from the previous iteration. Before the first iteration – empty dictionary.
• totalHz -- When LVE_VERSION is 4, real CPU frequency in Hz multiplied by number of cores. When
LVE_VERSION > 4, CPU speed is in conventional units and equals to 1000000000 * cores (1 GHz per
core).
• procs -- Quantity of CPU/cores.
• lve_usages -- Contains accumulated LVE statistics for each 5-seconds interval in current minute. Cleared
each minute.
• lve_usage -- Contains aggregated LVE Statistics for “previous” minute to store to database. Overwritten
each minute.
Plugin Data Convention

32. • Python 2.7 (www.python.org): No comment;
• Sqlite (sqlite.org), MySQL (www.mysql.com), MariaDB
(mariadb.org), PostgreSQL (www.postgresql.org)
• Sqlalchemy (sqlalchemy.org): the Python SQL Toolkit and
Object Relational Mapper;
• Alembic (alembic.zzzcomputing.com): database migration tool
for usage with the SQLAlchemy;
• Numpy (numpy.org): package for scientific computing with
Python;
• Sentry (sentry.io): real-time error tracking;
• Prospector (prospector.landscape.io): python static analysis
tool.
Stack technology

33. Questions?
Twitter.com/CloudLinuxOS Visit CloudLinux.com