Hammer of the Sysadmins based on puppet, git and redmine. Managing complex infrastructure producing 1/3 of slovak peering traffic in peaks. joj.sk, noviny.sk, huste.tv, tipsportextraliga.sk, prohokej.sk
9. …ouch and we are hiring
• job@bonetics.com (for developers)
• job@bonet.sk (for sysadmins)
10. Autoscaling
• Vmware clean system snapshot
• Provisioning profile for each node type
• Zabbix collecting performance data
• Zabbix trigger for new server deployment
• 10s of new webservers in 3 minutes at your
service
Notes de l'éditeur
Introduction to customer’s infrastructureShort overview of most critical issuesWhat we want to achieve in new infrastructureWhich tools were used to implement these solutionsLive demonstrationIf we have some extra time, we could drop a word about autoscaling techniques
TV JOJ is one of the two biggest commercial TV stations in slovakia broadcasting 3 channelsThey’re running a lot of web sites for channels, own TV shows, sports (streaming slovakicehockey play-off for example)and online news paper site ‘noviny.sk’You could predict traffic for sport events, but it’s much harder for news.And of course, we are running lot of technologies in our app stack.
This is traffic generated by whole infrastructure ontuesday
We need consistent configuration across all servers (if you have 5 balanced web servers, you definitely need to have same config on all of them)There is strong requirement to track all changes in infrastructure and applicationProper tools for change requests, configuration reviews and approvals are priceless when your team and stack becomes bigger.
Puppet– You definitely need somebody with object oriented design experience in your team- The common mistake is defining processes instead of proposed system state. You should think about where you want to be instead of how to get there (that’s puppet’s work).Redmine – issue tracking, integrated with version control (code, config, one place) - code review plugin to add more orgaiozational hierarchy in productionGit – comments with redmine issue references / fixesBash – master files deployment automation
Here we have some security breach and leaking some top secret information about unnamned celebrityWe should report it to our sysadmin teamAnd now we should assign it to proper sysadmin and apply some hotfix through apache configuration change.