Transcript of a BriefingsDirect sponsored podcast on ISM3 and security standards recorded live at The Open Group’s Enterprise Architecture Practitioners Conference in Seattle, Washington.

1. ISM3 Helps Describe Security in Language Business Leaders
Can Understand
Transcript of a BriefingsDirect sponsored podcast on ISM3 and security standards recorded live
at The Open Group’s Enterprise Architecture Practitioners Conference in Seattle, Washington.
Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Sponsor: The Open Group.
Dana Gardner: Hi, this is Dana Gardner, principal analyst at Interarbor Solutions, and you’re
listening to BriefingsDirect.
Today, we present a sponsored podcast discussion coming to you from The Open Group’s
Enterprise Architecture Practitioners Conference in Seattle, the week of February 1, 2010.
We've assembled a panel to examine the need for IT security to run more like a
data-driven science, rather than a mysterious art form. Rigorously applying data
and metrics to security can dramatically improve IT results and reduce overall
risk to the business.
By employing and applying more metrics and standards to security, both outside
the firewall and inside, the protection of IT becomes better, and the known threats can become
evaluated. People can understand better what they are up against, perhaps in close to real time,
and they can also know what's working or is not working, both inside and outside their
organization.
Standards like ISM3 are helping to not only gain greater visibility, but also are allowing IT
leaders to scale security best practices repeatably and reliably.
We're here to determine the strategic imperatives for security metrics, and to discuss how to use
them to change the outcomes in terms of IT’s value to the business. Please join me in welcoming
a security executive from The Open Group, as well as two experts on security who are presenting
here at the Security Practitioners Conference. I want to welcome Jim Hietala, Vice President for
Security at The Open Group. Hi, Jim.
Jim Hietala: Hi Dana.
Gardner: We are also here with Adam Shostack, co-author of 'The New School of Information
Security.' Welcome, Adam.
Adam Shostack: Hey, Dana. Great to be here.
Gardner: And also Vicente Aceituno, director of the ISM3 Consortium. Welcome.
Vicente Aceituno: Thank you very much.

2. Gardner: Now that we have got a sense of this need for better metrics and better visibility, I
wonder if I could go to you Jim. What is it to be a data-driven security organization, versus the
alternative?
Hietala: In a sentence, it's using information to make decisions, as opposed to what vendors are
pitching at you or your gut reaction. It's getting a little more scientific about gathering data on the
kinds of attacks you're seeing and the kinds of threats that you face, and using that data to inform
the decisions around the right set of controls to put in place to effectively secure the organization.
Gardner: Is it fair to say that organizations are largely not doing this now?
All over the map
Hietala: It's probably not a fair characterization to say that they're not. A presentation we had
today from an analyst firm talked about people being all over the map. I wouldn’t say there's a lot
of rigor and standardization around the kinds of data that’s being collected to
inform decisions, but there is some of that work going on in very large
organizations. There, you typically see a little more mature metrics program. In
smaller organizations, not so much. It's a little all over the map.
Gardner: Perhaps it's time to standardize this a little bit?
Hietala: We think so. We think there's a contribution to make from The Open
Group, in terms of developing the ISM3, standard and getting it out there more widely.
Gardner: Adam, what, in your perception, is different now in terms of security than say two,
three, or four years ago?
Shostack: The big change we've seen is that people have started to talk about the problems that
they are having, as a result of laws passed in California and elsewhere that require them to say,
"We made a mistake with data that we hold about you," and to tell their
customers.
We've seen that a lot of the things we feared would happen haven't come to pass.
We used to say that your company would go out of business and your customers
would all flee. It's not happening that way. So, we're getting an opportunity today
to share data in a way that’s never been possible before.
Gardner: Is it fair to say we are getting real about security?
Shostack: We've been real about security for a long time, but we have an opportunity to be a
heck of a lot more effective than we have been. We can say, "This control that we all thought was
a really good idea -- well, everyone is doing it, and it's not having the impact that we would
like." So, we can reassess how we're getting real, where we're putting our dollars.

3. Gardner: Vicente, perhaps you could help us understand the application of metrics and data for
security with external factors, and then internal. What's the difference?
Aceituno: Well, you can only use metrics to manage internal factors, because metrics are all
about controlling what you do and being able to manage the outputs that you produce and that
contribute value to the business.
I don’t think it brings a bigger return on investment (ROI) to collect metrics on external things
that you can't control. It’s like hearing the news. What can you do about it? You're not the
government or you're not directly involved. It's only the internal metrics that really make sense.
Gardner: From your perception, what needs to be a top priority in terms of this data-driven
approach to security inside your own organization?
What you measure
Aceituno: The top priority should be to make sure that the things you measure are things that
are contributing positivity to the value that you're bringing to business as a information security
management (ISM) practitioner. That’s the focus. Are you measuring things that are actually
bringing value or are you measuring things that are fancy or look good?
Gardner: We've heard "fit for purpose" applied to some other aspects of
architecture and IT. How does this notion, being fit for purpose, apply to your
security efforts?
Aceituno: Basically, we link business goals, business objectives, and security
objectives in a way that’s never been done before, because we are painfully detailed when we
express the outcomes that you are supposed to get from your ISM system. That will make it far
easier for practitioners to actually measure the things that matter.
Gardner: We've been talking fairly generally about metrics and data. Jim, what do we really talk
about? What are we defining here? Is this about taxonomy and categories, metadata, all the above
-- or is there something a bit more defined that we're trying to measure?
Hietala: There's some taxonomy work to be done. One of the real issues in security is that when
I say "threat," do other people have the same understanding? Risk management is rife with
different terms that mean different things to different people. So, getting a common taxonomy is
something that makes sense.
The kinds of metrics we're collecting can be all over the map, but generally they're the things that
would guide the right kind of decision making within an IT security organization around the
question, "Are we doing the right things?"

4. Today, Vicente used an example of looking at vulnerabilities that are found in web applications.
A critical metric was how long those vulnerabilities are out there
before they get fixed by different lines of business, by different parts
of the business, looking at how the organization is responding to that.
We're trying to drive that metric towards the vulnerabilities being
open for less time and getting fixed quicker.
Gardner: Adam, in your book, I believe you addressed some of these issues. How do look at
metrics? How do you characterize them? I know it could go on for an hour about that, but at the
high level.
Shostack: At the high level, Vicente’s point about measuring the things you can control is
critical. Oftentimes in security, we don’t like to admit that we've made mistakes and we conceal
some of the issues that are happening. A metrics initiative gives you the opportunity to get out
there and talk about what's going on, not in a finger pointing way, which has happened so often
in the past, but in an objective and numerically centered way. That gives us opportunity to
improve.
Gardner: I suppose this is a maturation of security. Is that fair to say that we're bringing this to
where some other aspects of business may have been, in say manufacturing, 30, 40, or 50 years
ago?
Learning from other disciplines
Shostack: I think that’s a fair statement. We're learning a lot from other fields. We're learning a
lot from other disciplines. Elements of that are going to uncomfortable for some practitioners,
and there are elements that will really enable practitioners to connect what they are doing to the
business.
Gardner: The stakes here, I imagine, are quite high. This is about the trust you have with your
partners, your customers, and the brand equity you have in your company. These are not small
considerations, Jim?
Hietala: No, they're big considerations, and they do have a big effect on the business. Also, the
important outputs of a good metrics program can be that it gives you a different way to talk to
your senior management about the progress that you're making against the business objectives
and security objectives.
That’s been an area of enormous disconnect. Security professionals have tended to talk about
viruses, worms, relatively technical things, but haven't been able to show a trend to senior
management that justifies the kind of spending they have been doing and the kind of spending
they need to do in the future. Business language around some of that is needed in this area.

5. Gardner: I have to imagine, too, that if we formalize, structure, and standardize, we can make
these repeatable. There's not that risk of personnel leaving and taking a lot of the tribal
knowledge with them. Is that fair?
Hietala: That's fair as well. That's something that came out today in some of the discussions.
Documenting the processes and what you're doing makes it easier to transition to new personnel
and that kind of thing.
Gardner: Vicente, tell us a little bit about the ISM3 Consortium, its history, and what it is that
you are principally involved with at this time.
Aceituno: The main task of the ISM3 Consortium so far was to manage the ISM3 standard. I'm
very happy to say that The Open Group and ISM3 Consortium reached an agreement and, with
this agreement, The Open Group will be managing ISM3 from here on. We'll be devoting our
time to other things, like teaching and consulting services in Spain, which is our main market. I
can't think of anything better than for ISM3 to be managed from The Open Group from here on.
Gardner: Adam, do you have a sense of this particular standard, the ISM3? Where do you see it
fitting in?
Shostack: Actually, I don't have a great sense of where it fits in. There are a tremendous number
of standards out there, and what I heard today I am very impressed by. I'm going to go read more
about it, but it's not something I have a lot of operational exposure to that really lets me say,
"This is where it's working for me."
Gardner: Jim, do you have a sense of where it fits in, and perhaps for those of our listeners who
are not that familiar, can you give a quick tutorial?
Business value approach
Hietala: Sure. In terms of where I'd place it in the information security community, it adds a
business value approach to information security, a metrics and maturity model approach that you
had not necessarily had there with some of the other standards that are out there.
I'd also say that it's approachable from the standpoint that it's geared towards having different
target maturity levels for different kinds of enterprises. That makes sense.
One of the things we talk about is that there's an 80-20 rule. You get 80 percent of the benefit
from a subset of security controls. You can tailor ISM3 to the organization and get some benefit
out of it, without setting the bar so high that it's unachievable for a mid-size or small business.
That's the way I would characterize it.
Gardner: I think it's really important that these things are developed and brought into an
organization at a practical level for those people who are in the trenches and are down there

6. doing the work. Is there anything about this particular standard that you think is really not
academic, but something quite effective in practice?
Hietala: Well, it spans the breadth of information security. You have metrics and control
approaches in various areas and you can pick a starting point. You can come at this top-down, if
you're trying to implement a big program. Or, you come at it bottoms-up and pick a niche, where
you know you are not doing well and want to establish some rigor around what you are doing.
You can do a smaller implementation and get some benefit out of it. It's approachable either way.
Gardner: Adam, any thoughts about this issue of practicality when it comes to security,
something that's more scientific and not perhaps a mysterious dark art of some kind?
Shostack: I really liked seeing the practical extracted. "Here are the things we're measuring.
Here is why it matters to the business." That's what Vicente was talking about with regards to
ISM3 through the day. Getting away from these very broad, hand wavy measures of risk or
improvement, down to, "We are measuring this precise thing and this is why we need it to
improve," is refreshing.
Gardner: Vincente, do you have any examples of organizations that have taken a lead on this
and what sort of results have they been able to provide?
Aceituno: At this moment, the one organization that has implemented the ISM3 is Caja Madrid,
which is the fourth biggest financial institution in Spain, and they had very impressive results.
We found six times as many vulnerabilities. We were making more than twice as many ethical
hacking tests. We could bring down the cost of unethical hacking by a big percentage, and we
were getting more vulnerabilities fixed.
It was easier to communicate with other teams, and we had metrics to understand the results we
were getting from making changes in the process. We have knowledge management that allows
us to change the whole team of people and still carry on doing exactly the same thing in the same
way that we were doing it.
I think that Caja Madrid is very happy and, actually, the director of security at Caja Madrid is
very impressed with ISM3.
Gardner: Who typically are the folks who would be bringing this into an organization? I
suppose there is some variability and the organizational landscape is still quite diverse, but is
there a methodology in terms of how to bring this into an organization?
Works either way
Aceituno: It could work either way. Either you're a top-level manager, the CISO, or whatever,
and you can think, "Okay, I want to do this" and you can implement a top-down implementation
of the method.

7. Or, you can have no support from higher management and understand that you need to put in
some rigor for management and you can think, "Okay, I'm going to organize my own work
around this framework."
It can work either way, as Jim was saying before. You can implement it top down or bottom up
and get benefit from it.
Gardner: Jim, this is a specific Open Group question. Does this work well inside of some other
framework activity or architectural initiatives? Are there some other ITIL related activities? Does
this have a brotherhood, if you will, in terms of standards and approaches that The Open Group's
heritage is a bit more attuned to?
Hietala: I don't know that there's a direct statement you can make about how well this will work
in an enterprise architecture framework or something like that. This is more about managing
security objectives and operational things that you are going to do in a information security
frame within an enterprise.
It's process-oriented. So, in terms of working well with other things, it works well with ITIL.
Some of the early implementations have suggested that, but there is a good synergy there. I'll
leave it there.
Gardner: Adam, any thoughts, from your perspective, on how this fits into some larger
initiatives around security?
Language they understand
Shostack: We've seen over the last few years that those security programs that succeed are the
ones that talk to the business needs and talk to the executive suite in language that the executives
understand.
The real success here and the real step with ISM3 is that it gives people a prescriptive way to get
started on building those metrics. You can pick it up and look at it and say, "Okay, I'm going to
measure these things. I'm going to trend on them." And, I'm going to report on them." As we get
towards a place, where more people are talking about those things, we'll start to see an
expectation that security is a little bit different. There is a risk environment that's very outside of
people's control, but this gives people a way to get a handle on it.
Gardner: Vicente, it seems quite important, as a first step, to know where you are, in order to
know how you've progressed. This seems to be an essential ingredient to being able to ascertain
your risks over time.
Aceituno: The very first step, when it comes to the usual implementing, is to understand the
needs and the goals of the business and the obligations of the business, because that's what drives
the whole design of the ISM system There is no need to align security goals and business goals,
because there are no goals outside of business goals. You have to serve the business first.

8. Gardner: There really isn't much difference between the goals of security and the general goals
of the business. They are inexorably tied.
Aceituno: Yes, of course, they are.
Gardner: We've been learning more about security, some new metrics, and the ability to tie this
into business outcomes. I want to thank our panel. We've been talking to Jim Hietala, Vice
President for Security at the Open Group. Thank you, Jim.
Hietala: Thank you, Dana.
Gardner: Adam Shostack, co-author of the book, 'The New School of Information Security.'
Thank you.
Shostack: Thank you.
Gardner: And, also Vicente Aceituno, who is the Director of the ISM3 Consortium. Thank you.
Aceituno: Thanks so much.
Gardner: We are coming to you from The Open Group Security Practitioners Conference in
Seattle, the week of February 1, 2010.
This is Dana Gardner, principal analyst at Interarbor Solutions. Thanks for listening to this
BriefingsDirect podcast, and come back next time.
Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Sponsor: The Open Group.
Transcript of a BriefingsDirect sponsored podcast on ISM3 and security standards recorded live
at The Open Group’s Enterprise Architecture Practitioners Conference in Seattle, Washington.
Copyright Interarbor Solutions, LLC, 2005-2010. All rights reserved.
You may also be interested in:
• Security, simplicity and control ease make desktop virtualization ready for enterprise
uptake
• Mainframes provide fast-track access to private cloud benefits for enterprises, process
ecosystems
• Webinar: Modernization pulls new value from legacy and client-server enterprise
applications