2. An event is an action, (grouping or listing of such actions called event log
file or just log file)
The action itself can be as simple as a successful or failed or some device
actions. For example, event generated whenever we log on to the computer
at work.
Devices generate records of the events that occur. Some are routine. Others
are indicators of a decline in system health or attempted security breaches.
When an event occurs, Administrators/professionals must determine
– what was the event?
– what caused the event?
– What is the necessary action?
Event
3. The event log files can play several roles:
Event log provides the most basic piece of information on every action or
incident that occurs on a network (or attempts to get in to a network, audit
trail for user activity, both internal and external).
It is helpful if applications, the operating system, and other system services
record important events such as low-memory conditions or excessive
attempts to access a disk.
Event log files provide the most valuable information a network professional
could desire about the health and monitor the network against attacks
(vulnerable devices, brute force attacks, etc) and correlate events in order to
identify hidden treats (DDOS, scanning, worms) and to identify business and
operational frauds.
Event Logs
4. The concept of total event log management includes the
following components:
Monitoring
Real time monitoring of log files with notification capability
Collecting
Collection and consolidation of event logs
Reporting
The ability to quickly refine and filter the logs and provide a report on
specific events or event trends
Concept Of Log Management
5. Identifying the right SIM tool for Log Monitoring which has Analyzing,
Customized Filtering and Reporting capabilities.
Are we receiving logs from all the devices ? Is the monitoring Tool identifies
and understands (supports) the logs from all the devices. If not what is the
solution ?
How our SIM tool segregates the logs of various Customers ?
Filtering out the Unnecessary logs ? What are the critical Events ? What report
Customer wants Exactly ? Do we have real-time alerts and notification
method in case of any suspicious logs detected ?
Is the Correlation rules and Alerts working Properly without false-
positives/false-negatives ?
Are we analyzing the database’s growth and free space at regular intervals ?
Are we doing archiving of old Logs ? Is database backup taking place
regularly ? If so where is the Backup stored ?
Strategies in Log Monitoring
6. Event Type varies with devices. Here are some common Event types
explained.
Error - Error events are logged when an application or a system
component actually failed some part of its functionality. For example,
an inability to write data to a disk, which resulted in data loss.
Warning - Warning events signify potential or future problem
situations. For example, relatively low memory or disk space, which
might become problematic if resources continue to be consumed.
Information - Information events indicate a situation or an operation
that occurred that is not problematic to the application or system. For
example, the starting or stopping of a service application.
Common Event Types
7. How to justify the event is suspicious or not?
How to filter out those Duplicated Event logs further?
We need to keep an eye on any new patch/updates, it may add
some new events which we have not captured.
Logs are obtained but useless if don’t know how to review
Logs are reviewed but useless if there is no process to response
Challenges in Log Monitoring
8. SIEM Tool
Security information Enterprise Management Tool
To monitor the network against attacks (vulnerable devices,
brute force attacks, etc)
To monitor user activities
To correlate events in order to identify hidden threats (DDOS,
scanning, worms)
To identify business and operational frauds.
Collects log from various supported Devices and provides User
interface for easy monitoring ,analyzing and reporting.
Raise Alarms if anything found suspicious based on the rules
configured.
SIEM Tool
11. Event Life Cycle
Agent :-
Phase 2
Manager:-
Phase 3
Console/Web:-
Phase 4
· Collect Event data
· Normalize Event data
· categorization
· Aggregate and Filter Events
· Tag Customer and Zone information
· Prioritize the Events
Phase 1
Phase 2
Phase 3
Correlation Engine evaluates the Event Stream
Generates correlated Events
· Filters
· Rules
· Data monitors
Write Events to database
Phase 4
Monitoring :-
· Active Channel
· Reports
· Event graphs
Raw Data from devices
12. Phase 1
Events begin at network devices that can sense and record instances
of security-sensitive activity.
Examples include a database record change, a syslog entry, a firewall
transit, a router access, or scanning a door access card.
Such initial events are typically recorded in logs, and are sometimes
called base or raw events.
Phase 2
When these Raw Events reaches Agents, the following things will
happen
Normalization
categorization
Aggregation
Filtration
13. Phase 3
When Events reaches Manager, the following things will happen
Correlation
Filters will be applied on Real-time data for Rules, dashboards etc
Writes data in to the database.
Phase 4
Use the Historical data (stored in database) in Active Channels,
Reports etc..
15. Smart Connectors are softwares, Can be installed as dedicated servers or
installed directly on machines (co-hosting -- standard pc running software
based application such as ISS RealSecure, Snort).
Smart Connectors collect logs from Devices
- OS:- Windows, UNIX, Linux, Mainframes etc
- Applications: Web servers, email servers, proxy, application servers, anti
virus etc
- Databases: Oracle, MSSQL, Sybase etc
- Devices: Routers, Firewalls, IDS/IPS, Switch, VPN Concentrators etc
Smart Connectors are Product Specific. i.e. Connector installed for UNIX
different from Windows.
FlexConnector – To create custom connectors ( Connectors which are not
defined by ArcSight)
ArcSight Smart Connectors/Agents
16. Aggregation
When these events reach ArcSight Smart
Agents, several things can happen.
All received events are normalized .As
each device has a different logging
format and reporting mechanism, Agent
evaluates which fields are relevant and
arranges them in a common format.
All received events are categorized
using Arc Sight's event categorization
taxonomy. Categories are Object,
Behavior, Outcome, Technique, Device
group and significance.
If appropriate and the Smart Agent is
configured to aggregated the events to
issue fewer and more meaningful events
and to reduce network traffic.
If appropriate and the Smart Agent is
configured to filtered out the selected
events, to eliminate them as a further
traffic or processing burden.
18. Customer URI :-
carries customer information. Used to segregate the events from
different customers.
“Customer URI” is specified in the Connectors during
Configuration.
“Customer URI” field is added as Tags to the logs which are
forwarded to the Manager
ArcSight Smart Connectors/Agents
19. Aggregation is done by Agents to reduce the volume of events
flowing to the manager which saves the network bandwidth.
Aggregation happens if some events occurred contains specified
matching fields within a specified time frame.
Eg. Aggregate if 10 Events received within 2 minutes having the
following fields Matches
Source IP Address
Destination IP Address
Source port
Destination Port
Name
Agent aggregates the events if above given conditions satisfied i.e.
it will send only a single event to manager instead of 10 events by
adding a field “Aggregated Event Count “ (which shows the actual
number of Events).
In this example the aggregated Event Count is 10.
Aggregation
22. The ArcSight Manager writes events to the ArcSight database.
Manager passes queries and fetches the events from database
for the requests raised from ArcSight Console.
Once an event is received by the ArcSight Manager, it is cached
and correlated .Then its data fields are stored in the ArcSight
Database with a normalized schema.
Manager triggers an alert if any event matches the Real-Time
Rules
ArcSight Manager
23. ArcSight Manager-Correlation
When Rule conditions are met Arcsight generates a special
internal event called a correlation event.
Successive failed logins could imply a Brute-Force Attack
(attempts to guess a password).
Port Scan :- Multiple packets From a Source Address with
Source port is same but Destination Port is different.
25. ArcSight Database is based on Oracle.
The ArcSight Database is the relational database repository that is used to
store all captured events
Manager writes data to the database in Normalized schema. This enables
ArcSight to collect the events for later analysis and reference.
ArcSight Database
26. Online Retention period :- Period of time for the data to be available
online. Logs which are older than retention Period will get Archived.
Retention period is specified at the time of Database installation. Online
Retention period for
India ArcSight – 37 days
FGB ArcSight– 45 days
Archive partitions :- Older logs will get archived automatically based on
the Online retention period. To access the archived logs we need to
reactivate the partitions. Archived partitions are stored in Archived Volume
directory.
ArcSight Database
28. Provides Graphical user interface for easy Monitoring, Analyzing and
Reporting.
To set up filters and create customized rules to display and process
events, define notification and escalation procedures, actions, manages
users and sets permissions etc..
ArcSight Console elements
Navigator Panel
Viewer Panel
Inspect/Edit Panel
Message bar
ArcSight Console
29. Navigator Panel
To access ArcSight Resources . Resource includes Active channel, Reports,
Rules, Agents, Active lists, Customers, Notifications etc
ArcSight Console
30. Viewer Panel
To View Dashboards, Active Channels , agents and manger
status, Notifications etc..
ArcSight Console
31. Inspect /Edit Panel
To Examine the Details of Events that appears in Active Channel
To Modify resources like Reports, Active channels, Filters, Rules,
Dashboards etc..
ArcSight Console
32. Message Bar
Displays Error Messages and Notifications from the System
ArcSight Console
33. Filter
- Filters are conditions that reduces the volume of Events.
- Can be applied at the Connectors to reduce the volume of events sent to
Manager.
- Can be applied in Reports, Active Channels, Rules etc to retrieve the exact
events.
Filter Statements are constructed using Boolean Logic Operators and operators
& AND
= OR
!= NOT
Operators
=
!=
Contains
In
Startswith
Endswith
Like
Is
InSubnet
Between
<
<=
>
>=
ArcSight Console Resources - Filter
On
InGroup
BitAnd
34. ArcSight Console Resources
Data Type Operators Used Example
Number or
Integer
=, !=, <, <=, >=, <, and In
CustomNumber1 = 50
Aggregated Event Count >= 10
String
=, !=, In, Contains,
Matches, Starts With,
Ends With, and Like
ArcSightCategory StartsWith /Attack
or
ArcSightCategory = /AttackSuccess
Date Time
=, !=, Between, In, and
On
End Time Between 03/06/2009 15:00:00,03/06/2009 16:00:00
IP Address
=, !=, In, InSubnet, and
between
Target Address = 178.168.11.211
Target Address In 178.168.11.211, 178.168.11.212,
178.168.11.213
Target Address InSubnet 172.168.11.0/24
In the case sensitive column, select the check box if the data field value must
be case sensitive.
In the negate condition column, select the check box to change the condition
statement to an “all except this condition statement”.
35. Field Sets
Group of fields
Shows Exact contents/informations in an Event
Used in Active Channel (in grid View) to limit the columns that are
displayed, Reports ,etc
Sortable Field Sets
– Composed of Fields with sorting Index enabled
– Denoted by UP and Down arrows
– Field sorting number
Unsortable Field Sets
ArcSight Console Resources
37. Time Parameters
M - Month
m – Minute
h – Hour
d – Day
w – Week
Eg. $Now - 1d
Custom Time Parameter
- Used in Report Scheduling
ArcSight Console Resources
38. Active Channel
- Display Events that match an existing filter over a fixed or rolling time
frame.
- Active Channel Elements
Header
Provides overview of the active Channel.
It includes the Time frame, Filter, Events criticality etc
Radar
Bar Chart overview Active Channel events
It represents the group of events (in segments) with the same
End time
Channel Viewer
Grid
Graph
Image
ArcSight Console Resources - Active Channel
41. Report
Summary of Events (information) captured in PDF,HTML,CSV or RTF
ArcSight Console Resources - Report
42. Name
The name of the field appears as a column heading in the report unless you
specify an alias
Alias
An alternate name that replace the original field name as the column heading in
the report
COL (Column)
Decides the alignment of fields in the report i.e. which column come first
SORT ORD (Sort Order)
Specify which column you want sorted first, second, and so forth, in your report.
SRT DIR (Sort Direction)
Decides the sort direction (ascending, descending, or none) for each column.
The "none“ option defaults to ascending.
ArcSight Console Resources - Report
43. SRT BY (Sort By)
Sort by data field values, COUNT (by the total number, for numeric values), SUM
(by total values), AVG (by average value), MAX (by maximum values), or MIN (by
minimum values).
GRP BY (Group By)
For grouping (aggregating) the items in the report. When you select a field to use
as a "group by," also choose a Function by which to evaluate the grouping.
Function
When you select a field to use as a "group by" factor in a report, also choose a
function by which to evaluate the grouping. These are the same functions described
above for SRT BY.
PGE BRK (Page Break)
Select a field if you want page breaks to occur when there are changes in that field's
sorted content. You can select multiple fields.
ArcSight Console Resources - Report
44. Scheduled and
Archived Reports
Archived reports are
retrieved for immediate
viewing, without required
to rerun the report. In
addition, we can
schedule a report for
automatic archiving, on
a yearly, monthly,
weekly, daily, or hourly
basis
ArcSight Console Resources - Report
49. 9) Replace the certificate
Go to Console Installation
Path.
In this example, I installed
console in the location
D:/ArcSight
Go to the location
D:ArcSight Consolecurrent
jrelibsecurity
replace the “cacert” certificate file
Directory path
ArcSight Console Installation
54. ArcSight Web
ArcSight Web is an ArcSight-specific web server that
provides a personalized web-based interface that is accessed
to monitor events, dashboards ,etc.
55. Client Verifies
The hostname is identical to
the one with which it initiated
communication
Compares the Certificate with
the certificate in its Trust Store
(ARCSIGHT_HOMEjrelibsecurit
ycacert)
Client Generates
If the certificate is validated,
the client generates a random
session key, encrypts it using
the server’s public key, and
sends it to the Manager
Manager decrypts the session
key using its private key.
This session key is used to
encrypt and decrypt data
exchanged between the
Manager and the client from
this point forward
ArcSight
Components
ArcSight
Manager
Connecting to
Manager.sim.paladion.net
Manager’s SSL Certificate
Contains
CN = Hostname
Issued by :- Issuer
Public Key
Manager sends its SSL
Certificate
Encrypted Session Key
Client Connects to the
Hostname
(Manager.sim.paladion.net)
SSL Communication Between Arcsight manager and Arcsight Components
Cipher text
Cipher text
:
:
Client Decrypts data using
session key
Client encrypts data using
session key
56. Effect on communication when components
fail
If any of the ArcSight components is unavailable, it can affect
communication between other components.
If the database is unavailable for any reason, such as database capacity
is full or the database hardware is down, the Manager stops accepting
events and caches any events that were not committed to the database.
The agents start caching new events they receive, so there is no event
data loss.
The Consoles are disconnected. All existing ArcSight Web connections
are disconnected and no new login requests to the Web server are
accepted until the database is up and running again.
