How to get flow record details of a netflow packet
Categories:
Unpacking NetFlow: Extracting Flow Record Details from UDP Packets
Learn how to parse and interpret NetFlow (v5/v9) and IPFIX packets to extract valuable network flow record details using Java, focusing on UDP packet capture and data interpretation.
NetFlow, J-Flow, and IPFIX are essential protocols for network monitoring, providing detailed insights into network traffic. They work by exporting 'flow records' – summaries of network conversations – from routers and switches to collectors. Understanding how to capture and parse these UDP packets is crucial for anyone looking to build network analysis tools or integrate with existing monitoring systems. This article will guide you through the process of extracting flow record details, with a focus on Java-based implementation.
Understanding NetFlow Packet Structure
Before diving into code, it's vital to grasp the structure of NetFlow packets. NetFlow v5 is a fixed-format protocol, while NetFlow v9 and IPFIX are template-based, offering greater flexibility. Regardless of the version, the core idea is to encapsulate flow records within UDP datagrams. A NetFlow packet typically consists of a header followed by one or more flow records (v5) or template/data flowsets (v9/IPFIX).
flowchart TD
A[NetFlow UDP Packet] --> B{NetFlow Header}
B --> C{FlowSet 1}
C --> D[Template FlowSet (v9/IPFIX)]
C --> E[Data FlowSet (v9/IPFIX)]
B --> F{FlowSet 2}
F --> G[Template FlowSet (v9/IPFIX)]
F --> H[Data FlowSet (v9/IPFIX)]
E --> I[Flow Record 1]
E --> J[Flow Record 2]
H --> K[Flow Record 3]
H --> L[Flow Record 4]High-level structure of a NetFlow v9/IPFIX UDP packet.
Capturing NetFlow UDP Packets in Java
The first step in processing NetFlow data is to capture the UDP packets sent by your network devices. This involves setting up a DatagramSocket to listen on the designated NetFlow collector port (commonly 2055, 9995, or 9996). Once a packet is received, its raw byte content can be read and then parsed according to the NetFlow protocol specification.
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.SocketException;
public class NetFlowCollector {
private static final int NETFLOW_PORT = 2055; // Common NetFlow port
private static final int BUFFER_SIZE = 4096; // Max UDP packet size for NetFlow
public static void main(String[] args) {
try (DatagramSocket socket = new DatagramSocket(NETFLOW_PORT)) {
System.out.println("NetFlow collector listening on port " + NETFLOW_PORT);
byte[] buffer = new byte[BUFFER_SIZE];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
while (true) {
socket.receive(packet);
System.out.println("Received NetFlow packet from: " + packet.getAddress().getHostAddress() + ":" + packet.getPort());
// Process the packet data
processNetFlowPacket(packet.getData(), packet.getLength());
packet.setLength(buffer.length); // Reset packet length for next receive
}
} catch (SocketException e) {
System.err.println("Socket error: " + e.getMessage());
} catch (IOException e) {
System.err.println("I/O error: " + e.getMessage());
}
}
private static void processNetFlowPacket(byte[] data, int length) {
// This is where the parsing logic will go
System.out.println("Packet length: " + length + " bytes");
// For now, just print a snippet
// System.out.println("First 16 bytes: " + bytesToHex(data, 0, Math.min(length, 16)));
}
// Helper to convert bytes to hex string (for debugging)
private static String bytesToHex(byte[] bytes, int offset, int length) {
StringBuilder sb = new StringBuilder();
for (int i = 0; i < length; i++) {
sb.append(String.format("%02x ", bytes[offset + i]));
}
return sb.toString();
}
}
Basic Java code to set up a UDP listener for NetFlow packets.
Parsing NetFlow v5 Flow Records
NetFlow v5 is simpler due to its fixed format. Each packet starts with a 24-byte header, followed by a series of 48-byte flow records. To extract details, you'll need to read specific byte offsets and convert them to appropriate data types (e.g., IP addresses, port numbers, byte counts). The header contains information like the NetFlow version, count of records, sequence number, and system uptime.
flowchart LR
A[UDP Packet Data] --> B{Read NetFlow Header (24 bytes)}
B --> C["Version (2 bytes)"]
B --> D["Count (2 bytes)"]
B --> E["SysUptime (4 bytes)"]
B --> F["UnixSecs (4 bytes)"]
B --> G["UnixNanos (4 bytes)"]
B --> H["FlowSequence (4 bytes)"]
B --> I["EngineType/ID (2 bytes)"]
D -- "Loop 'Count' times" --> J{Read Flow Record (48 bytes)}
J --> K["SrcAddr (4 bytes)"]
J --> L["DstAddr (4 bytes)"]
J --> M["NextHop (4 bytes)"]
J --> N["Input/Output (2 bytes each)"]
J --> O["Pkts/Bytes (4 bytes each)"]
J --> P["Start/End Time (4 bytes each)"]
J --> Q["SrcPort/DstPort (2 bytes each)"]
J --> R["TCP Flags/Protocol/ToS (1 byte each)"]
J --> S["SrcMask/DstMask (1 byte each)"]NetFlow v5 packet parsing flow, showing header and flow record fields.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.net.InetAddress;
import java.net.UnknownHostException;
public class NetFlowV5Parser {
public static void parseV5Packet(byte[] data, int length) {
ByteBuffer buffer = ByteBuffer.wrap(data, 0, length);
buffer.order(ByteOrder.BIG_ENDIAN); // NetFlow uses network byte order (big-endian)
// Parse NetFlow v5 Header (24 bytes)
int version = buffer.getShort() & 0xFFFF; // Unsigned short
if (version != 5) {
System.err.println("Unsupported NetFlow version: " + version);
return;
}
int count = buffer.getShort() & 0xFFFF;
long sysUptime = buffer.getInt() & 0xFFFFFFFFL;
long unixSecs = buffer.getInt() & 0xFFFFFFFFL;
long unixNanos = buffer.getInt() & 0xFFFFFFFFL;
long flowSequence = buffer.getInt() & 0xFFFFFFFFL;
int engineType = buffer.get() & 0xFF;
int engineId = buffer.get() & 0xFF;
int samplingInterval = buffer.getShort() & 0xFFFF; // Or sampling rate
System.out.println("\n--- NetFlow v5 Header ---");
System.out.println("Version: " + version);
System.out.println("Count: " + count);
System.out.println("SysUptime: " + sysUptime + " ms");
System.out.println("Unix Secs: " + unixSecs);
System.out.println("Flow Sequence: " + flowSequence);
// Parse Flow Records (48 bytes each)
for (int i = 0; i < count; i++) {
if (buffer.remaining() < 48) {
System.err.println("Not enough bytes for flow record " + (i + 1));
break;
}
System.out.println("\n--- Flow Record " + (i + 1) + " ---");
try {
InetAddress srcAddr = InetAddress.getByAddress(readBytes(buffer, 4));
InetAddress dstAddr = InetAddress.getByAddress(readBytes(buffer, 4));
InetAddress nextHop = InetAddress.getByAddress(readBytes(buffer, 4));
int input = buffer.getShort() & 0xFFFF;
int output = buffer.getShort() & 0xFFFF;
long dPkts = buffer.getInt() & 0xFFFFFFFFL;
long dOctets = buffer.getInt() & 0xFFFFFFFFL;
long first = buffer.getInt() & 0xFFFFFFFFL;
long last = buffer.getInt() & 0xFFFFFFFFL;
int srcPort = buffer.getShort() & 0xFFFF;
int dstPort = buffer.getShort() & 0xFFFF;
int tcpFlags = buffer.get() & 0xFF;
int protocol = buffer.get() & 0xFF;
int tos = buffer.get() & 0xFF;
int srcAs = buffer.getShort() & 0xFFFF;
int dstAs = buffer.getShort() & 0xFFFF;
int srcMask = buffer.get() & 0xFF;
int dstMask = buffer.get() & 0xFF;
buffer.getShort(); // Unused padding
System.out.println(" Source IP: " + srcAddr.getHostAddress());
System.out.println(" Destination IP: " + dstAddr.getHostAddress());
System.out.println(" Source Port: " + srcPort);
System.out.println(" Destination Port: " + dstPort);
System.out.println(" Protocol: " + protocol + " (" + getProtocolName(protocol) + ")");
System.out.println(" Packets: " + dPkts);
System.out.println(" Bytes: " + dOctets);
System.out.println(" Start Time: " + (unixSecs - (sysUptime - first)/1000) + "s"); // Approximation
System.out.println(" End Time: " + (unixSecs - (sysUptime - last)/1000) + "s"); // Approximation
System.out.println(" TCP Flags: 0x" + String.format("%02X", tcpFlags));
} catch (UnknownHostException e) {
System.err.println("Error parsing IP address: " + e.getMessage());
}
}
}
private static byte[] readBytes(ByteBuffer buffer, int count) {
byte[] bytes = new byte[count];
buffer.get(bytes);
return bytes;
}
private static String getProtocolName(int protocol) {
switch (protocol) {
case 1: return "ICMP";
case 6: return "TCP";
case 17: return "UDP";
// Add more as needed
default: return "Unknown";
}
}
}
Java code snippet for parsing NetFlow v5 header and flow records.