Design and Implementation of a Distributed Platform for Sharing IP Flow Records

Abstract

Experiments using real traffic traces are of key importance in many network management research fields, such as traffic characterization, intrusion detection, and accounting. Access to such traces is often restricted due to privacy issues; research institutions typically have to sign non-disclosure agreements before accessing such traces from a network operator. Having such restrictions, researchers rarely have more than one source of traffic traces on which to run and validate their experiments.
Therefore, this paper develops a Distributed Platform for Sharing IP Flows (DipSIF) based on NetFlow records between multiple institutions. It is assumed that NetFlow traces collected by each participant are archived on separate storage hosts within their premises and then made available to others using a server that acts as a gateway to the storage. Due to privacy reasons the platform presented here uses a prefix-preserving, cryptography-based, and consistent anonymization algorithm in order to comply to different regulations determining the exchange of traffic traces data.