This talk will demonstrate the capabilities of HyClops (open source), integrating it for Zabbix management The presentation will overview the benefits of three components of HyClops - job monitoring module, proxy auto management module and external job management tool (JobShceduler), as well as automatic distribution control of Zabbix Proxy based on rules, discussing the path of future development and operation possibilities. Zabbix Conference 2015

1. TIS Inc. Daisuke IKEDA
11 September,2015
Riga,Latvia
Advanced SysOps
by using HyClops
Copyright © 2015 TIS Inc. All rights reserved. -1-

2. Agenda
About myself
- Server Engineer working at OSS promotion office
Zabbix Conference “2012” and then
- Released HyClops for Zabbix
Approaching to Advanced SysOps with OSS ecosystem
- Autonomous SysOps at cloud,multi-cloud, container based env.
Our Activity 2012-2015
- Cooperation with Job Management tool and Zabbix
- Proxy setting automation tool etc..
HyClops roadmap 1.0
- The architecture will be improved more Pluggable.
- Add features monitoring detail info from AWS
Agenda
Copyright © 2015 TIS Inc. All rights reserved. -2-

3. Daisuke IKEDA
TIS Inc.
OSS Promotion Office
(Main)Provide Zabbix support service.
Search for the way of effective SysOps.
I wrote a Zabbix book in Japanese(2014)
About Myself
Copyright © 2015 TIS Inc. All rights reserved. -3-
Belongs To
Division
My work
Recent
Activities
http://www.amazon.co.jp/dp/4774162884

4. Talked about
“Zabbix for Hybrid Cloud Management”
Zabbix Conference 2012
Copyright © 2015 TIS Inc. All rights reserved. -4-

5. Released “HyClops for Zabbix”
July 12, 2013
And then
Copyright © 2015 TIS Inc. All rights reserved. -5-
Released at
GitHub URL https://github.com/tech-sketch/hyclops
License GPL version 2
This tool achieves...
- Auto registration of AWS or vSphere info
- Show instance list at Zabbix Dashboard
- Operate for multi instances on Zabbix Dashboard

6. HyClops screen capture
Copyright © 2015 TIS Inc. All rights reserved. -6-
AWS or vSphere
status summary

7. HyClops screen capture
Copyright © 2015 TIS Inc. All rights reserved. -7-
AWS or vSphere
status summary
Operate to
some instances
popup

8. HyClops screen capture
Copyright © 2015 TIS Inc. All rights reserved. -8-
Connect to SSH server
via GateOne

9. Make a change
Copyright © 2015 TIS Inc. All rights reserved. -9-
Only enough?
Integration & Automation

10. Make a change
Copyright © 2015 TIS Inc. All rights reserved. -10-
Not enough!

11. Advanced SysOps
Our concepts
Copyright © 2015 TIS Inc. All rights reserved. -11-
Change Infrastructure
Change Requirement
Physical Virtual Cloud Multi-cloud Container
Evaluate the each server resource
Service
Evaluate the service status

12. OSS Ecosystem
Our concepts
Copyright © 2015 TIS Inc. All rights reserved. -12-
Zabbix
Job
Management
Incident
Management
Data
Analytics
LogData
Collector

13. Copyright © 2015 TIS Inc. All rights reserved. -13-
Activities in 2012-2015

14. 1. CloudModule for Zabbix
Activity in 2012-2015
Copyright © 2015 TIS Inc. All rights reserved. -14-
GitHub URL https://github.com/ike-dai/zabbix-cloud-module
Slideshare
URL
http://www.slideshare.net/ikedai/cloudmodule-for-zabbix
Zabbix
CloudModule
Deltacloud
AWS
Loadable Module
cloud.instance.discovery
cloud.metric.discovery
CloudCache
(Shared memory)
cloud.instance.info
cloud.metric

15. 2. HyClops JobMonitoring
Activity in 2012-2015
Copyright © 2015 TIS Inc. All rights reserved. -15-
[Monitoring]
Zabbix
[JobController]
JobScheduler
GitHub URL https://github.com/tech-sketch/hyclops_jm
This tool achieves...
- Monitoring Job elapse time
- Monitoring Job failure information
- Adjust trigger automatically in executing some jobs
Cooperation

16. Auto Adjust Zabbix trigger
Activity in 2012-2015
Copyright © 2015 TIS Inc. All rights reserved. -16-
Zabbix
Change
Trigger Job
Return
Trigger Job
Main Job
Change
Trigger Job
Template
Return
Trigger Job
Template
Job Template
Original Trigger
Disable
Temp Trigger
Create
e.g.) CPU Util >70%
via Zabbix API
timeline
e.g.) CPU Util >90%
Enable
Delete
JobScheduler

17. JobScheduler
[Reference] About JobScheduler
Copyright © 2015 TIS Inc. All rights reserved. -17-
What?
Who?
Features
OSS Job control software (GPL ver.2)
SOS GmbH in Berlin, Germany
JobScheduler Engine
JobScheduler Engine
HA or Cluster support
JobScheduler Agent
SSH Agent
Agent or Agentless support
Advantage More detailed job controlling
All operation are executed by JobScheduler API
http://www.sos-berlin.com

18. Copyright © 2015 TIS Inc. All rights reserved. -18-
3. Proxy scheduler
Other activity
GitHub URL https://github.com/ike-dai/proxy_scheduler_for_zabbix
Scheduler
Zabbix
rules.json
{
"agent_ip": {
"proxy-01": {
"from": "10.1.1.1",
"to": "10.1.2.20"
},
“proxy-02": {
"from": "10.1.2.21",
"to": "10.2.1.40"
}
}
}
Execute caller_schedule.py
Read rule from rule.json
Decide target proxy
Register the proxy setting
via Zabbix API(host.update)
Modify the agent config
(zabbix_agentd.conf)
Action
Source: Auto Registration
Operations: Remote command
caller_schedule.py <params>

19. Copyright © 2015 TIS Inc. All rights reserved. -19-
HyClops roadmap

20. Improve the Base architecture
HyClops roadmap
Copyright © 2015 TIS Inc. All rights reserved. -20-
Message Queue
Message Queue
1. Discovery
….
2.PreOperation 3.Registration
AWS Discovery
Job Discovery
Target Proxy
Discovery
….
Filter Operation
Machine
Learning
Operation
….
Zabbix API
Registration
Zabbix Sender
Registration
Zabbix LLD
Registration
….

21. Towards Advanced SysOps
HyClops roadmap
Copyright © 2015 TIS Inc. All rights reserved. -21-
- Job execution target
- Point of failure
in Service trouble
etc....
e.g.)
Auto decision
Effective use of Zabbix.
Auto collect cloud info or
job info via HyClops
Generate synergy with
some data at the system
centered on HyClops
Now

22. HyClops 1.0
- Improve the base architecture
- Enhance the AWS monitoring feature
- AWS service discovery plugin
- AWS EC2/RDS/ELB/CloudWatch discovery
HyClops 1.0 features
Copyright © 2015 TIS Inc. All rights reserved. -22-

23. Mailing list
hyclops@ml.tis.co.jp
<Please send e-mails.>
GitHub repository
https://github.com/tech-sketch/hyclops
<Please send issues or Pull requests.>
Please contact us!
Copyright © 2015 TIS Inc. All rights reserved. -23-

24. Questions?
Q&A
Copyright © 2015 TIS Inc. All rights reserved. -24-

25. Thank You!
The End
Twitter : @ike_dai
Facebook: https://www.facebook.com/ikedai
Copyright © 2015 TIS Inc. All rights reserved. -25-