The release of the system for capturing, storing and indexing network packets Arkime 3.1 has been prepared , which provides tools for visually assessing traffic flows and searching for information related to network activity. The project was originally developed by AOL with the goal of creating an open and deployable replacement for commercial network packet processing platforms on its servers , capable of scalable to handle traffic at speeds of tens of gigabits per second. The traffic capture component code is written in C, and the interface is implemented in Node.js / JavaScript. The source code is distributed under the Apache 2.0 license. Work in Linux and FreeBSD is supported. Ready packages are prepared for Arch, CentOS and Ubuntu.
Arkime includes tools for capturing and indexing traffic in native PCAP format, and provides tools for quick access to indexed data. The use of the PCAP format greatly simplifies integration with existing traffic analyzers such as Wireshark. The amount of stored data is limited only by the size of the available disk array. Session metadata is indexed in a cluster based on the Elasticsearch engine .
To analyze the accumulated information, a web interface is proposed that allows navigation, search and export of samples. The web interface provides several viewing modes – from general statistics, connection maps and visual graphs with data on changes in network activity to tools for studying individual sessions, analyzing activity in the context of the protocols used, and parsing data from PCAP dumps. An API is also provided to allow third-party applications to pass captured packet data in PCAP format and parsed sessions in JSON format.
Arkime consists of three basic components:
- Traffic Capture System is a multithreaded C application for monitoring traffic, writing PCAP dumps to disk, parsing captured packets, and sending session metadata (Stateful packet inspection) (SPI) and protocols to the Elasticsearch cluster. Encrypted storage of PCAP files is possible.
- A web interface based on the Node.js platform that runs on each traffic capture server and handles requests related to accessing indexed data and transferring PCAP files through the API .
- Elasticsearch-based metadata store.
In the new release :
- Added support for IETF QUIC, GENEVE, VXLAN-GPE protocols.
- Added support for Q-in-Q (Double VLAN) type, which allows encapsulating VLAN tags into second-level tags to expand the number of VLANs to 16 million.
- Added support for the “float” field type.
- The Amazon Elastic Compute Cloud writer has been moved to use the IMDSv2 (Instance Metadata Service) protocol.
- Refactoring of the code to add UDP tunnels.
- Added support for elasticsearchAPIKey and elasticsearchBasicAuth.
