Network Event Correlation Using Unsupervised Machine Learning Algorithms

PDF

Authors
  1. Dondo, M.
  2. Mason, P.
  3. Japkowicz, N.
  4. Smith, R.
Corporate Authors
Defence R&D Canada - Ottawa, Ottawa ONT (CAN)
Abstract
We have successfully implemented a two-stage event correlation model for intrusion detection system (IDS) alerts. The model is designed to automate alert and incidents management and reduce the workload on an IDS analyst. We achieve this correlation by clustering similar alerts together, thus allowing the analyst to only look at a few clusters instead of hundreds or thousands of alerts. The first stage of this model uses an artificial neural network (ANN)-based autoassociator. The autoassociator is trained to reproduce each alert at its output, and it uses the error metric between its input and output to cluster similar alerts together. The accuracy of the system is improved by adding another machine-learning stage which attempts to combine closely related clusters produced by the first stage into super-clusters. The second stage uses the Expectation–Maximisation (EM) clustering algorithm. The model and performance of this model are tested with intrusion alerts generated by a Snort IDS on DARPA’s 1999 IDS evaluation data as well as incidents.org alerts.

Il y a un résumé en français ici.

Report Number
DRDC-OTTAWA-TM-2006-193 — Technical Memorandum
Date of publication
01 Oct 2006
Number of Pages
108
DSTKIM No
CA028454
CANDIS No
526639
Format(s):
CD ROM

Permanent link

Document 1 of 1

Date modified: