WIXLIB

The following table summarizes these ideas:I. Detector capabilityReliabilitySensitivityPropertyThe level of certainty providedby our detector when wereceive warning of a possibleeventThe capability our detector hasfor extensive and complexanalysis in locating possibleattacksResult in its absenceFalse PositivesFalse negativesThe Detection ProcessWe will refer to the global process developed by the SIM as the detection process, including theorganization’s various detectors and monitors as well as those executed by the system toprocess this information.The detection process normally involves three well-defined phases: Preprocessing: Detection itself, in which detectors generate alerts and information isconsolidated before being sent. Collection: All the information from these detectors is sent and received at a centrallocation. Post-processing: What we do with the information after we have it all centralized.Post-processingPreprocessing and collection are traditional capabilities and do not contribute anything new toour solution. But in post-processing, once we have all the information in one place, we canimplement mechanisms that will improve detection sensitivity and reliability. We increase thecomplexity of the analysis by including methods to discard false positives or on the other handprioritize or discover more complex patterns that our detectors have overlooked.In OSSIM we use three post-processing methods:1. Prioritization: We prioritize alerts received using a contextualization process developed bydefining a topological security policy in combination with the inventory of our systems.2. Risk assessment: Each event is evaluated in relation to its associated risk, in other words,in proportion to the assets at risk, the threat represented by the event, and the probability it isreal.3. Correlation: We analyze a collection of events to obtain more valuable information.9

The following figure shows how the properties mentioned above are affected by theseprocesses:II. Post-processingPrioritizationRisk assessmentCorrelationProcessingEffectAssess the threat byImproves reliabilitycontextualizing an eventAssess the risk in relationImproves reliabilityto the value of assetsCompare various eventsImproves reliability, sensitivity,to obtain more valuableand abstractioninformationSo after processing them, our system will use the alerts provided by detectors to produce whatwill be referred to in this document as alarms.An alarm will usually be the result of various alerts and have a higher level of abstraction thatallows us to identify more complex patterns—and it will provide better reliability.The Detection MapWith the goal of defining detection capability, OSSIM will develop a detection map, which willinvolve categorizing the possibilities in the detection of attacks and security events.Auditing Detection CapabilityUsing this detection map we will be able to define a new auditing method that will allow us tomeasure the situation and needs of an organization in relation to the effectiveness of itsdetection systems when an attack is in place.This perspective is very different from the traditional audit or intrusion test, since we areinterested not in locating security failures but in the capability to detect the eventual exploitationof these failures.In this project we will therefore develop a Detection Capability Auditing Procedure with which wewill provide a mechanism for assessing an organization’s situation in relation to its capacity fordetecting attacks.10

3. FunctionalityIn order to understand what OSSIM offers we can define the functionality of the system using asimplified, graphical form with the following nine levels:We discuss each of these levels below to help give a practical description of our system:3.1 Pattern DetectorsMost traditional detectors operate using patterns, the best example of which is the IDS orintrusion detection system, which is capable of detecting patterns defined using signatures orrules.There is another class of pattern detectors that are included in most devices like routers andfirewalls, and they are capable of detecting, for example, port scans, spoofing attempts, andpossible fragmentation attacks.We also have detectors for security events within an operating system. They are capable ofsending alerts for possible security problems, and they almost all include their own logger, likesyslog for UNIX.Any element in the network, such as a router, a work station, a firewall, etc., has some capacityfor detection. In our system, we are interested in collecting events from all critical systems inorder to achieve one of our principle objectives: a comprehensive view of the network.11

3.2 Anomaly DetectorsThe ability to detect anomalies is more recent than that for patterns. In this case we do not haveto tell the detection system what is good and what is bad; it can learn on its own and alert uswhen behavior deviates enough from what it has learned is normal.Since the two processes work in opposite ways, this new functionality provides a point of viewthat is both different and complementary to pattern detection.Anomaly detection can be especially useful for preventing, for example, perimeter attacks,which are one continuous anomaly: in the direction of the communications and the path theydefine, in the flow of data, in their size, duration, time, content, etc.This technique provides a solution—until now beyond reach—for controlling access of privilegedusers, as in internal attacks by, for example, disloyal employees, in which no policies areviolated and no exploits carried out. Yet they represent an anomaly in the use and manner ofuse of a service.Now let's look at some other examples in which these detectors would be useful: A new attack for which there still are no signatures could produce an obvious anomalyyet circumvent pattern detection systems. A worm that has been introduced into the organization, a spamming attack, and eventhe use of P2P programs would generate a number of anomalous connections that areeasy to detect. We could likewise detect:o Use of services that is abnormal in origin and destinationo Use at abnormal timeso Excess use of traffic or connectionso Abnormal copying of files on the internal networko Changes in a machine's operating systemo Etc.We might think that, as an undesirable byproduct, these detectors will generate a number ofnew alerts, amplifying our signal and making our problem worse (our objective is to limit thenumber of alerts). However, if we take them as additional information that complements thetraditional pattern alerts, we will be able to evaluate and therefore differentiate those that couldresult in a higher risk situation.12

3.3 Centralization and NormalizationNormalization and centralization (or aggregation) are aimed at unifying security events from allcritical systems throughout the organization in a single format on just one console.All security products normally have a capacity for centralized management using standardprotocols; hence, aggregation is simple using these protocols. In OSSIM we attempt to avoidregular use of agents and instead use forms of communication that are native to systems.Normalization requires a parser or translator familiar with the types and formats of alerts comingfrom different detectors. We will need to organize the database and adapt the forensic consolein order to homogenize processing and display of all these events.That way we will be able to observe all security events for a particular moment in time—whetherthey come from a router, a firewall, an IDS, or a UNIX server—on the same screen and in thesame format.When we have all network events centralized in the same database, we achieve a considerablycomprehensive view of what’s going on throughout the network, which, as we will see shortly,allows us to develop processes that enable us to detect more complex and widely dispersedpatterns.13

3.4 PrioritizationThe priority of an alert should depend on the topology and inventory of the organization’ssystems. The reasons are quite clear, as demonstrated by the following examples:a. If a machine running the UNIX operating system and Apache web server receivesan alert about an attack on Microsoft IIS, the alert should be deprioritized.b. If a user makes a suspicious connection to a server, the system should:oooGive it maximum priority if the user is external to the network and attackingthe client database.Give it low priority if the user is internal to the network and attacking anetwork printer.Discard it if the user is someone who normally tests development servers.By prioritization we mean the process of contextualization, in other words, the evaluation of analert’s importance in relation to the organization’s environment, which is described in aknowledge base for the network comprised of: An inventory of machines and networks (identifiers, operating systems, services,etc.)An access policy (whether access is permitted or prohibited, and from where towhere)To perform these tasks (as in risk assessment, as explained in the following section), we have aframework in which we can configure the following:1. Security policy, or assessment of asset-threat pairs according to topology and dataflow2. Inventory3. Asset assessment4. Risk assessment (prioritization of alerts)5. Assessment of the reliability of each alert6. Alarm definitionPrioritization is one of the most important steps in filtering alerts received by detectors andshould be executed using a continuous process of fine-tuning and feedback from theorganization.14

3.5 Risk AssessmentThe importance given to an event depends on these three factors:a. The value of the assets associated with the eventb. The threat represented by the eventc.The probability that the event will occurIntrinsic RiskThese three factors are the building blocks for the traditional definition of risk: a measure of thepotential impact of a threat on assets given the probability that it will occur.Traditionally risk assessment is concerned with intrinsic risks, or latent risks, in other words,risks that an organization assumes by virtue of both the assets it possesses for the purpose ofdeveloping its business and circumstantial threats to those assets.Immediate RiskIn our case, due to real-time capabilities we can measure the risk associated with the currentsituation in immediate terms.In this case the measurement of risk is weighted by the damage it would produce and theprobability that the threat is occurring in the present.That probability, which is a derivative of the imperfection of our sensors, turns out to be nothingmore than the degree of reliability of our sensors in detecting the potential intrusion-in-progress.By immediate risk we mean the state of risk produced when an alert is received and assessedinstantaneously as a measure of the damage an attack would produce, weighted by thereliability of the detector that made the report.OSSIM calculates the immediate risk of each event received, and this will be the objectivemeasure we use to assess the event's importance in terms of security. We assess the need toact using this measure only.Our system likewise includes a risk monitor (described below) that assesses the riskaccumulated over time of networks and groups of machines related to an event.15

3.6 CorrelationWe define the correlation function as an algorithm that executes an operation on input data andreturns output data.We must consider the information collected by our detectors and monitors to be specific yetpartial; it illuminates only small areas along the spectrum defined by all the information wewould really like to have.We can think of correlation as the ability to take advantage of these systems and, using a newlayer of processing, fill in more areas along that infinite spectrum of all possible informationabout a network.If we got carried away, this idea could lead us to try to install one single system with a detectorthat can gather all possible information about the network, but it would require a display thatshows absolutely everything in one place and almost limitless memory and storage capacity.Therefore, correlation systems merely supplement the inadequate sensitivity, reliability, andlimited range of our detectors.Input and OutputIn simple terms, our architecture has two clearly defined elements that provide information to itscorrelation functions: Monitors, which normally provide indicators Detectors, which normally provide alertsThe output will also be one of these two elements: alerts or indicators. Our correlation functionsbecome new detectors and monitors.3.6.1 Correlation ModelThe OSSIM correlation model has ambitious objectives:1. Develop specific patterns for detecting the known and detectable2. Develop nonspecific patterns for detecting the unknown and the undetectable3. Provide an inference machine that can be configured using interrelated rules and that hasthe ability to describe more complex patterns4. The ability to link detectors and monitors recursively to create more abstract and usefulobjects5. Develop algorithms for displaying a general view of the security situation16

3.6.2 Correlation MethodsTo achieve these objectives we employ two very different correlation methods, based on thefollowing two principles: Correlation using sequences of events, focused on known and detectable attacks,relates the known patterns and behaviors that define an attack using rules implementedby a state machine. Correlation using heuristic algorithms. Using an opposite approach, we implementalgorithms that attempt to detect risky situations using heuristic analysis. In an effort tocompensate for the shortcomings of other methods, this one detects situations withoutknowing or displaying the details. It is useful for detecting unknown attacks anddisplaying a general view of the security state for a large number of systems.3.6.2.1 Method 1: Correlation Using Heuristic AlgorithmsFor correlation OSSIM will implement a simple heuristic algorithm using event accumulation inorder to obtain an indicator or snapshot of the general security state of the network.In this process our first objective is to obtain what we defined earlier as immediate risk andsubsequently a value we could call accumulated risk.This provides high level monitoring that we can use as a “thermometer” for risky situationswithout ever knowing any details about the characteristics of the problem.Continuing with that analogy, we will construct a sensitive thermometer that will display the totalaccumulated risk in a certain time frame. The thermometer will go up proportionately to theamount of events received recently and to how “hot" they are, and it will cool off as time goes onif no new events are received.This correlation method supplements correlation using sequences of events with an oppositeapproach—in the latter we attempt to characterize possible attacks to the highest level of detail.Consequently, the value of correlation using heuristic algorithms is twofold: It provides a quick global view of the situation. It detects possible patterns that other correlation systems might overlook, eitherbecause the attacks are unknown or due to some shortcoming.CALMCALM (Compromise and Attack Level Monitor) is an assessment algorithm that uses eventaccumulation with recovery over time. Its input is a high volume of events, and its output is asingle indicator of the general state of security.This accumulation is executed for any subject on the network, including any machine, group ofmachines, network segment, path, etc. that we are interested in monitoring.Event AccumulationAccumulation is calculated simply by the sum of two state variables that represent theimmediate risk of each event:17

“C” or level of compromise, which measures the probability that a machine iscompromised “A” or level of attack to which a system is subjected, which measures the potentialrisk due to attacks launchedWhy separate these two variables for monitoring? First, because they characterize differentsituations: the level of attack indicates the probability that an attack has been launched, anattack that may or may not be successful; while the level of compromise provides directevidence that there has been an attack and that it has been successful.Second, the importance of both variables depends on the situation of the machine. Mainly dueto the exposed state of perimeter networks, which are exposed to an enormous number ofattacks, most of them automated, unfortunately a high level-of-attack value is a “normal”situation. However, any sign of compromise or movement that might lead us to think there is anattacker residing in these networks should be immediately pointed out and reviewed.On the other hand, there are cases in which a machine generates anomalies within the networkdue to the nature of its function—such as a security scanner, a service with random passiveports, development, etc.—and these will normally have a high C and a low A.A value is assigned to the C or A variable for a machine on the network according to three rules:1. Any possible attack launched from machine 1 on machine 2 will increase the A (level ofattacks experienced) of machine 2 and the C

OSSIM was conceived as an integration project, and our intent is not to develop new capabilities but to take advantage of the wealth of free software "gems," programs developed and inspired by the best programmers in the world (including snort, rrd, nmap, nessus, and