Dan Lagos “AdmFord" - Cyphercon 6 - 2023
When Management Asks You:
“Do You Accept Agile as Your
Lord and Savior?”
Agile in Operations, can it actually be done?

What is Agile in the first place?
The Principles
• Our highest priority is to satisfy the customer through early
and continuous delivery of valuable software
• Welcome changing requirements, even late in development.
Agile processes harness change for the customer’s
competitive advantage
• Deliver working software frequently, from a couple of weeks to
a couple of months, with a preference to the shorter timescale
• Business people and developers must work together daily
throughout the project
• Build projects around motivated individuals. Give them the
environment and support they need, and trust them to ge the
job done
• The most e
ffi
cient and e
ff
ective method of conveying
information to and within a development team is face-to-face
conversation
• Working software is the primary measure of progress
• Agile processes promote sustainable development. The
sponsors, developers, and users should be able to
maintain a constant pace inde
fi
nitely
• Continuous attention to technical excellence and good
design enhances agility
• Simplicity — the art of maximizing the amount of work
not done — is essential
• The best architectures, requirements, and designs
emerge from self-organizing teams
• At regular intervals, the team re
fl
ects on how to
become more e
ff
ective, then tunes and adjust its
behavior accordingly

Origins of Agile
From the Factory Floor, to the O
ffi
ce Floor
• Theory of Constrains
Any system is limited from achieving more of
its goals by a small number of constraints, and
by focusing to identify and restructure the
organization around them, we limit their
impact.
1. Identify the constraints (bottlenecks)
2. Decide how to exploit the constraints
3. Subordinate everything to the above decision
4. Elevate the system’s constraints
5. If the constraint breaks (is eliminated), go
back to step 1. But don’t allow inertia to
cause a new constraint
• TOC advocates keeping inventory of
products low.
• You don’t make new inventory when
you can’t sell it
• You keep in-work inventory low by
using small batches
• Time to Process an element of
inventory is key to
fi
nding where
bottlenecks appear
• Inventory is anything that requires
labor to create, or modify

Origins of Agile
The Toyota Way
• Continuous Improvement
• Challenge
Long term vision for company/program
• Kaizen
Continuous Improvement
• Gench Genbutsu
Go to the source to
fi
nd facts to make correct decisions
• Respect for people
• Respect
Respect others, and take responsibility to do our best
to build mutual trust in a team
• Teamwork
Stimulate personal and professional growth, share
opportunities of development and maximize individual
and team performance
• Right Process
• Create a process that can bring problems to the surface
• Use a pull system to avoid overproduction
• Level out the workload (don’t overload workers)
• Build a culture where it’s ok to stop a process in order to
fi
x it,
when problems are identi
fi
ed
• Standardize tasks and processes
• Use visual control so no problems are hidden
• Use already tested technology
• Add value to the organization
• Grow leaders who understand the work, live the philosophy, teach
it to others
• Develop people and teams who follow the company’s philosophy
• Respect partners and suppliers by challenging and helping them
to improve
• Continuously solve root problems drives organizational learning
• Go see for yourself the situation to understand it
• Make decisions by consensus, considering all options, then
implementing the solution quickly
• Become a learning organization through re
fl
ection and continuous
learning

It’s basically project management
But without the project managers (to a point)
• Every Plan Fails when Confronted with the Enemy
• The further you plan out work, the more of a chance unexpected work or
circumstances will wreck your schedule
• Waterfall works when plans don’t change
• Increasing Flexibility Means Breaking work up into smaller more manageable
chunks
• Iteration based on the chunk size, work with team leads, management, and
employees to identify issues and solutions quickly.

Methodology Letter Jumble
SCRUM, KANBAN, SCRUMBAN, etc…
SCRUM KANBAN SCRUMBAN
Team Members
Recommended members
between 3-9
No speci
fi
c limitation on the
number of team members
No Speci
fi
c Limitations on the
number of team members
Team Roles
Members are assigned di
ff
erent
roles & responsibilities
Members are generalists or
specialists
No Roles
Work Cycles
Sprints that can last from 1 to 4
weeks
Continuous Work
fl
ow
2-Week Iterations with continuity
(the board is not cleared)
Rules Follows Strict Rules Relaxed and Flexible
Finds the middle grounds between
scrum & kanban with moderate rules
Task Assignments Assigned to the team members
Members choose their fast-
paced tasks
Members choose their tasks
Limits Based on the current sprint
Limits placed on the work-in-
progress
Limits placed on the work-in-
progress

Scrum Meetings
The Structured Work Process (That can, or usually be, a PITA)
Frequency Attendees Time Needed Objectives
Sprint Lasts anywhere from 1 to 4 weeks
Sprint Planning First day of sprint
Product Owner, Scrum
Master, Scrum Team
Potentially an hour,
depending on team size
Set a sprint goal
Daily Scrum Daily
Scrum Master & Scrum
Team
No more than 15
minutes
What did you do yesterday?
What are you doing today?
Any Blockers/Issues?
Sprint Review
At the end of a
sprint
Scrum Team, Product
Owner, Scrum Master
2 - 4 hours
Demo of work done
user stories - con
fi
rm and decide on
incomplete ones
Assesement of Backlog
Sprint
Retrospective
Between Review &
Planning meetings
Scrum Team, Scrum
Master, Product Owner
An hour
What was done well
What didn’t go as planned
Improvements for next sprint
Backlog
Re
fi
nement
Every other week
depending on the
duration of the sprint
Scrum Master, Product Owner,
Potentially Scrum Team
No de
fi
ned duration
Prioritize backlog items
Alight backlog to KPIs
Appropriate sizing of backlog items
Add more detail

Agile Is Not Simply Implementing a Methodology
Stay away from “Cargo Cult Agile”
• An Organization that wants to implement “Agile" can’t just order the use of
Scrum or Kansan across the board.
• Every Operations Team will have di
ff
erent organizational, and business
requirements
• it’s a two way street
• Methodologies can be implemented partially, or in a hybrid manner
(SCRUMBAN)
• But institutional organization also must change to give Project Managers/
Product Owners / Team Leads more initiative to initiate & lead changes

Methodologies and Team Evolution
Using Scrum to Grow your Team
• Methodologies, while not necessary to fully implement, can be used to help in team evolution.
• Stages of Team Development
• Forming & Storming
• More structured work environment is bene
fi
cial
• Norming
• As people get used to working together & with their tools, the structure can be eased as needed
• Performing
• Less outright structure is needed as team members know what they are supposed to do and are active in
choosing task to complete
• Adjourning, or changes in structure
• Other than closing teams, people will also come and go. So it can be useful to re-implement more structure
from methodologies to get identify issues and corrective measures.

Actually Putting Agile to work, DevOps
The Culmination of Agile in the Development World
• In development, the culmination of Agile can be said to be DevOps.
• DevOps has the same principles as Agile, while not necessarily following strict
methodologies.
• It de
fi
nes objectives regarding the types of work and techniques to achieve
them
• It expects good knowledge of what the team members are doing, and
creating/implement tools that they can implement changes frequently.
• We see this in things like the Software Development Lifecycle (SDLC)

Types of Work and Priorities
The Four Types of work
• Gene Kim in the Phoenix Project and DevOps handbook de
fi
ned the four types of work in a business
• Business Projects (Highest Priority)
• Business Initiatives, most of development work
• Internal IT Projects (Highest Priority)
• Infrastructure and IT Operations
• Updates and Changes (Normal Priority)
• Often Generated from the two previous types of work
• Unplanned Work or Recovery Work (should be limited as much as possible)
• Incidents and problems generated by other work

The Three Ways of DevOps
• Systems Thinking/Flow of Work
• All work should ideally
fl
ow in one direction, such as across a
KANBAN board
• Any
fl
ow back in the process means that there are unidenti
fi
ed
potential issues or constraints that need to be addressed
• Amplify Feedback Loops
• Learn from current processes and
fi
nd improvements
• Implement improvements to the processes
• Experimentation
• Try multiple ways of improving processes and work and test
which work better
• In development, the best example of this is A/B testing

DevOps foundations in Security (DevSecOps?)
Enterprise vs MSSP
• Enterprise & Team Projects take priority
• Maintenance and Operations work
(tasks for other teams, installation of
software, etc) is lower priority
• Incidents take engineers away from
improvements and should be limited.
• 80% Project & Ops work
20% Incident work
• Automate Incidents and Operations to
meet KPIs and team SLAs
• MSSP Improvements, Maintenance &
Operations work may be handled by a
separate team compared to Security
Monitoring
• Incidents are the main work done
• 80% Incident work
20% identifying improvements
• Based on Agreement with other
Organizations, apply automations and
process improvements to meet KPIs
and SLAs

Starting the Change to Agile
People, Process, Tools
• Change has to come from both the top (Senior Management & Management)
and within the teams themselves
• Identify all the processes that your team is involved with. That is tasks
received from other teams. Incidents your team have de
fi
ned and how to
resolve them. Common operations work that is done frequently
• SOAR is your friend. It is never too early to start. Playbooks in Word,
Sharepoint, OneNote, or Wiki are
fi
ne. But a playbook within your ticketing
system (such as the SecOps module in ServiceNow), forces analysts to follow
the work
fl
ow, ensuring repeatability. This also creates the foundations for
automating the Incident Response process.

Constraints and Bottlenecks
Time is literally money
• IT operations often doesn’t have many bottlenecks regarding infrastructure, that is when
certain commands/tools taking too much time. But there is one bottleneck that is universal.
• Man/Woman-hours of work.
• Don’t expect 8 hours of fully productive work from a person
• Lose 2 hours for lunch, breaks, and meetings
• Consider the
fi
rst hour and last hour as “half productivity”, as they’re getting up to
speed for the day, or thinking what needs to get done when they get back home
• 4 hours of fully productive work, 2 hours at 50% productivity, 2 hours of zero productivity
• Find the people that are always being called to put out
fi
res, or take on the most work for
themselves.

From Process to Pipeline
Identify the work process,
fi
nd improvements, rinse and repeat
1. Knowing your team’s work processes is foundational. Since it is literally understanding what work is
done, and how it is done.
2. Interactive Playbooks are a
fi
rst step to standardize work
3. Identify dependencies (where other teams are needed) and constraints (certain tasks always falling to
a single team member)
4. Implement improvements
A. Automate parts of the playbooks
B. Request that other teams automate some of their work that’s related to your own team’s
5. If Incident Response to alerts has been fully automated, create new alerts based on frequency of the
automated ones, to
fi
nd discrepancies to normal daily
fl
ow
6. New services, products and technologies are implemented regularly, so go back to step 1

Software Development Life Cycle (SDLC)
The foundation behind development and DevOps
1. Planning
2. Analysis
3. Design
4. Implementation
5. Testing & Integration
6. Maintenance

Software Development Lifecycle
Adapting the Dev process into SecOps
1. Planning: Identify IOC that you want to create an alert for
2. Analysis: Research IOC
3. Design: Identify availability of log sources to see if IOC would be visible
4. Implementation: Write the initial query/alert for the SIEM, ideally in a code repository with version tracking
5. Testing & Integration: Test the query in the SIEM for existing indicators from log archive
A. If successful, push query to SIEM as a new alert rule
B. If unsuccessful, push query to SIEM with limited alert generation (potential new alerts are only assigned to the
user)
C. Assign Purple Team member to write test article on dedicated machine to con
fi
rm alert will actually
fi
re.
6. Maintenance: Run regular “drills”, making sure that all know alerts are able to detonated at will by the team to
make sure everything works correctly (think, "
fi
re drills”)
SIEM
https://learn.microsoft.com/en-us/azure/architecture/example-scenario/devops/automate-sentinel-integration

Ticket Metrics
The Subtle Art of Of Proving a team’s Work
• Number of tickets closed is a disingenuous metric. It takes in consideration the di
ffi
culty or the
result (true positive or false positive).
• Automatically assigning tickets to users randomly is good, but you can’t use the count of how
many tickets are closed to measure employee performance.
• Pick and pull of tickets at discretion also doesn’t work, as employees with experience can identify
potentially easy tickets and horde them.
• Classifying tickets based on priority level (Critical, High, Medium, Low) and assigning a points value
to them can help, but frequency of occurrence of the tickets and the results (true or false positives)
still aren’t considered. Allowing employees to potentially game the results in their favor.
• But SCRUM o
ff
ers something else, “Planning Poker”, assigning a weight for a task based on the
amount or di
ffi
culty of work to complete it. Assigning a separate weight to True Positive incidents,
and False Positives can normalize the amount of work done between team members.

Planning Poker
Stealing From Scrum, to Make Metrics Work
• Planning Poker is usually done during backlog re
fi
nement.
• Values are decided on how much work a task takes, and NEVER how much time it
would. People in general cannot give good estimates of the amount of time things will
take. But we’re more visually oriented so measuring things by estimating size or
quantity can actually be more accurate.
• Planning Poker uses a modi
fi
ed
fi
bonacci sequence to estimate work.
0, 1, 2, 3, 5, 8, 13, 20, 40, 100
• In doing so, you do not take in consideration SLAs or KPIs
• Track these separately, as adjacent goals to achieve
• The law of averages in the end comes to our rescue regarding metrics

x̄ and σ
Average and Standard Deviation in a Normal Distribution
• In a Normal Distribution, the average
(mean), x̄, is at the central point of
the curve
• The standard deviation, σ, measures
how dispersed the data is compared
to the average (mean)

Out of Bounds Tickets
When Response times exceed Standard Deviation (both Positive & Negative)
• When they take more time
• Dependencies can be to blame,
team members are waiting for other
teams to complete associated work
• Employee needs additional training
or is a new team member and needs
to improve
• There potentially is a problem with a
tool the team uses
• These can be the
fi
rst targets to quickly
improve SLA results by automation
• When they take less time
• An employee has found a way to
automate their work
• Congratulate them, and see about
standardizing their procedure!
• An Employee has been closing tickets
by only doing the work partially.
• Retraining might be required
• Closer monitoring of employee’s
work
• Termination of employee

Track the Averages
Track Team Progress
• The average time it takes for a job with a certain Estimation of E
ff
ort (Planning Poker value),
even when not linked to an KPI or SLA, can provide valuable information on the team.
• As the team evolves (forming, storming, norming, performing), the average time it takes for
work should drop month to month.
• This includes performance improvements done with automations and improvements done
in conjunction with other teams.
• Time saved = Money Saved. Doing more tickets in less time and improved e
ffi
ciency means
management can o
ff
er a measure of money saved over time to senior leaders at an
organization.
• Changes to teams can raise these averages, seeing how fast they start dropping again can
indicate how well the team is working together.

Useful Books
Some Study Materials
• Foundational Texts
• "The Goal: A Process of Ongoing Improvement” by Dr. Eliahu Goldratt
• “Toyota Kata” by Mike Rother
• “The DevOps Handbook” by Gene Kim
• Useful in my opinion, as Security can also be a creative enterprise, I would
also suggest the following book:
• "Creativity, Inc.: Overcoming Unseen Forces That Stand in the Way of True
Inspiration" by Ed Catmull

Certs?
When you need some credentials
• Scrum?
• Useful to have if you’re doing Scrum Master work.
• Professional Scrum Master (PSM) by Scrum.org
• No need to take a class. Higher minimal passing score needed. Scrum.org pushes more for understanding how to
use scrum and adapt it to your requirements.
• Certi
fi
ed Scrum Master (CSM) by Scrum Alliance
• Requires you to take a class. Comparatively low passing score. The exam concentrates more on wrote knowledge
of Scrum, and not as much how to apply or adapt it.
• Agile
• Anything potentially from PMI
• If you don’t have the work history for a PMP
• Try the CompTIA Project+
• Or do the Certi
fi
ed Associate in Project Management (CAPM) by PMI

Thank You
Twitter: @admford
Mastodon: @admford@infosec.exchange
Linkedin: www.linkedin.com/in/danlagos/