Conntrack

Overview

The system conntrack feature allows network connections to be tracked through OSDx devices. The conntrack feature provides the Netfilter connection tracking system with a userspace interface, thus enabling the monitoring and management of all active connections.

This chapter covers the basics of connection tracking, application detection capabilities, and advanced traffic classification features for policy-based routing and traffic filtering.

Basic Connection Tracking

Connection Format

Connections are displayed including the source and reply information, along with metadata. Here’s an example:

icmp     1 25 src=10.0.0.1 dst=10.0.0.2 type=8 code=0 id=18723 vrf=RED packets=2 bytes=168 src=10.0.0.2 dst=10.0.0.1 type=0 code=0 id=18723 vrf=RED packets=2 bytes=168 mark=33 use=1 appdetect[L3:1]

Connection Fields:

  • Protocol: icmp - Protocol identifier (icmp, tcp, udp, etc.)

  • Protocol Number: 1 - Numeric protocol identifier

  • TTL: 25 - Time remaining before connection expires

  • Origin: src=10.0.0.1 dst=10.0.0.2 - Source and destination of original packet

  • Reply: src=10.0.0.2 dst=10.0.0.1 - Source and destination of reply packet

  • VRF: vrf=RED - Virtual routing and forwarding instance

  • Statistics: packets=2 bytes=168 - Packet and byte counters

  • Metadata:

    • mark=33 - Packet mark for policy routing

    • use=1 - Usage counter

    • appdetect[L3:1] - Application detection data

    • [OFFLOAD] or [HW_OFFLOAD] - Hardware offload status

    • Sc: not-bypass - Security bypass status

For more information on traffic policies and marks, see Traffic Routing and Firewall Service.

Connection States

Running system conntrack monitor displays connections with state information:

  • NEW: Newly established connections

  • UPDATE: Updated connection information

  • DESTROY: Terminated connections

Logging

The system conntrack logging feature sends connection tracking information to syslog, enabling centralized monitoring and auditing.

Configuration

Event Types

Configure which connection events to log (default: destroy):

  • NEW: Log new connections

  • UPDATE: Log connection updates

  • DESTROY: Log connection termination

  • ALL: Log all events

Log Level

The log-level option controls the syslog severity level for logged messages (default: info):

View logs using system journal monitor or system journal show:

osdx ulogd[9431]: [NEW] ORIG: SRC=10.0.0.1 DST=10.0.0.2 VRF=RED PROTO=TCP SPT=55406 DPT=80 PKTS=6 BYTES=577 , REPLY: SRC=10.0.0.2 DST=10.0.0.1 VRF=RED PROTO=TCP SPT=80 DPT=55406 PKTS=4 BYTES=653 MARK=33 LABELS=LOCAL APPDETECT[L4:80 http-host:10.0.0.1]

Log messages include:

  • Origin and reply connection information

  • Traffic policy labels

  • Application detection data (app-id and detected protocols)

Custom Identity

Customize the log identifier using system conntrack logging identity <txt>:

# Changes "ulogd" to "OSDx_DUT0"
set system conntrack logging identity OSDx_DUT0

Examples: Conntrack Logging Examples

Application Detection (App-Detect)

Overview

system conntrack app-detect is an implementation based on RFC6759 that allows applications to be identified in connection tracking flows using Deep Packet Inspection (DPI).

Application detection provides traffic flows with:

  • Application ID (App-ID): Identifies the application or protocol, the APP-ID consists of:

    • Engine ID: Network layer (L3, L4, U)

    • Selector ID: Protocol or service identifier (protocol number, port number, custom ID, provider ID)

  • Detected Data: Protocol-specific information (hostnames, URLs, etc.)

The App-ID assigned can be used by traffic selectors for traffic policy matching (see Traffic Policy).

Display application detection information using:

Output format: appdetect[engineID:selectorID "detected data"]

Application ID Structure

The Application ID is made up of two components:

Engine ID

Indicates the network layer associated with the flow:

  • L3: Layer 3 Internet Protocol Number (IANA-L3) - E.g., ICMP, protocol code 1 per RFC792

  • L4: Layer 4 Transport Protocol Port Number (IANA-L4) - E.g., HTTP on port 80

  • U: Upper-layer application number (Layer 7 applications) - Custom or provider application identifiers (e.g. U6 for custom dictionary)

Selector ID

Specific identifier within the engine type:

  • L3: Protocol number (e.g., 1 for ICMP)

  • L4: Service port number (e.g., 80 for HTTP, 443 for HTTPS)

    • Application detectors can override port-based assignment (e.g., L4:80 assigned to traffic on port 8080 if HTTP is detected)

  • U: Upper-layer application identifier number set via traffic policy or dictionary

System Architecture

Session-Based Packet Processing

OSDx conntrack implements session-based packet processing to efficiently classify and handle IP packets. When enabled, the system automatically associates IP (IPv4/IPv6) packets with connection tracking sessions.

Session Creation:

When a packet is processed:

  1. The system attempts to find the packet’s corresponding session among existing connections

  2. If no matching session exists, a new session is created and assigned to the packet

  3. Subsequent packets belonging to the same flow are associated with this session

Session Definition:

Sessions are identified by:

  • Source IP address

  • Destination IP address

  • Protocol (e.g., TCP, UDP, ICMP)

  • Source and destination ports (for TCP/UDP protocols)

This means a session can represent various types of traffic flows, such as:

  • A TCP connection between devices

  • ICMP echo requests and replies

Application Identification Integration

To boost session identification, an Application ID (App-ID) is linked to each session through the application detection subsystem (app-detect).

The App-ID assigned provides:

  • Session classification based on detected applications

  • Traffic policy matching via traffic selectors

  • Enhanced visibility in connection tracking logs and netflow exportation

How Detection Works:

When app-detect is configured via system conntrack app-detect, OSDx automatically enables application detection for all network traffic passing through the device. The system processes both incoming traffic and locally-generated traffic.

The detection process works as follows:

  1. Cache Lookup: When a new connection starts, OSDx first checks if the destination IP address has a recently detected cached App-ID:

    • If found, the cached App-ID is assigned to the connection

  2. Deep Packet Inspection: DPI runs on each packet until the application is identified or detection completes. Exceptionally, DNS inspection could continue detecting the host even after initial resolution. The number of packets required, if fragmentation isn’t applied, varies by protocol:

    • DNS (UDP): 2 packets (query and response)

    • DNS (TCP): 5 packets (TCP handshake, query, and response)

    • HTTP: 4 packets (TCP handshake and HTTP request)

    • SSL/TLS: 4 packets (TCP handshake and Client Hello)

    During inspection, the system:

    • Examines protocol-specific patterns (HTTP requests, SSL/TLS handshakes, DNS messages)

    • Extracts relevant information (hostnames, URLs, query names)

    • Searches configured application dictionaries for matching patterns

    • Continues inspection until the application is identified or deemed undetectable

  3. App-ID Assignment: Once the application is identified, the App-ID is:

    • Stored in the connection tracking entry

    • Cached for the destination IP address to speed up detection in future connections and perform First Packet Detection (FPD)

    • Immediately available for use in traffic policy rules

  4. Subsequent Packets: All remaining packets in the connection inherit the assigned App-ID without further inspection.

The assigned App-ID can be referenced in traffic policy rules to enable application-aware classification for policy-based routing, QoS, and security filtering.

Note

When system offload is configured, once an application is detected for a session, subsequent packets can be offloaded to the fast path.

Application Detectors

Application detectors use DPI to identify specific applications by analyzing packet content. The system examines protocol-specific patterns and structures to identify DNS, HTTP, and SSL/TLS traffic.

DNS Detection

Detects DNS protocol traffic by analyzing DNS message structure and validating message format.

  • Protocol: DNS (UDP/TCP)

  • Detection Method: Analyzes DNS header fields and message structure

  • App-ID Assignment: L4:53 (or L4:5353 for multicast DNS), regardless of actual server port

  • Detection Mode:

HTTP Detection

Detects HTTP protocol traffic (versions 1.0 and 1.1) by analyzing HTTP request methods and headers.

  • Protocol: HTTP on TCP connections

  • Detection Method: Validates HTTP request methods and parses request headers

  • App-ID Assignment: L4:80, regardless of actual server port

  • Detection Scope: Only the initial session request (persistent connections show first request only)

SSL/TLS Detection

Detects SSL/TLS encrypted traffic by analyzing the TLS handshake.

  • Protocol: SSL/TLS (all versions)

  • Detection Method: Parses the TLS Client Hello message to identify encrypted connections

  • App-ID Assignment: Port-based (e.g., L4:443 for HTTPS)

    • SSL/TLS is a transport layer protocol, not a final application identifier

    • The server port provides better application context

  • Detection Scope: Only the initial Client Hello message

Data Extraction

In addition to protocol detection, app-detect can extract protocol-specific metadata from detected traffic. The data extracted is displayed alongside the App-ID in connection tracking output and logs.

DNS Data Extraction

Extracts domain names from DNS queries and responses.

HTTP Data Extraction

Extracts various fields from HTTP request headers.

SSL/TLS Data Extraction

Extracts server hostname from SSL/TLS handshakes.

Application ID Storage Modes

OSDx supports two storage modes for Application IDs, configured via system conntrack app-detect app-id-storage.

Override Mode (Default)

In override mode (system conntrack app-detect app-id-storage override), only the highest layer Application ID is stored for each traffic session. When a higher-layer application is detected, it replaces the previous App-ID.

Example:

A connection initially classified as L4:443 (HTTPS port) may be reclassified as U128:42 when a dictionary match identifies the specific application.

Chained Mode

In chained mode (system conntrack app-detect app-id-storage chained), all detected Application IDs are stored for each traffic session. This preserves the complete detection history, allowing traffic selectors to match any of the detected App-IDs in the chain.

Use Cases:

  • Track App-ID transitions (e.g., HTTP detection followed by dictionary-based reclassification)

  • Match traffic based on detection identifier tuples (e.g., together L3 protocol, L4 port, custom dictionary and provider dictionary properties like category and reputation)

  • Policy enforcement requiring multiple classification criteria

  • Enhanced visibility for logging and Netflow exportation

Example:

A connection might have an App-ID chain of [L3:6;L4:443;U130:15;U131:85] where:

  • L3:6 identifies the TCP protocol

  • L4:443 identifies the HTTPS protocol

  • U130:15 is the category from a remote dictionary

  • U131:85 is the reputation score from a remote dictionary

Note

Dictionary Priority: The dictionary number in system conntrack app-detect dictionary <u32> serves as the priority value, affecting the search order and App-ID resolution when multiple dictionaries are configured. Lower numbers indicate higher priority (dictionary 1 has highest priority). In override mode, higher priority dictionaries are preferred. In chained mode, priority has no effect since all resolved App-IDs are stored.

Remote Dictionary

Overview

Remote dictionaries enable cloud-based application categorization by connecting to external classification services. When OSDx detects traffic requiring classification (via DNS, HTTP or SSL detectors), it communicates with the remote dictionary server for remote App-ID resolution.

Remote dictionaries support retrieving following classification properties:

  • Category: Application category ID (Classification Engine ID U130)

  • Reputation: Reputation score (Classification Engine ID U131)

Configuration

Configure a remote dictionary using system conntrack app-detect dictionary <u32> remote:

Required Parameters:

  • url / encrypted-url: Remote dictionary server URL

  • key / encrypted-key: API key for authentication

  • property category or property reputation: Classification property to retrieve

Optional Parameters:

  • ssl-allow-insecure: Disable SSL certificate verification (not recommended for production)

  • alarm connection-error: Alarm to trigger on connection failures

Binding Options:

Control the network interface or address used to connect to the remote server:

  • local-address: Bind to a specific IPv4 address

  • local-interface: Bind to a specific network interface

  • local-vrf: Bind to a specific VRF

Traffic Marking:

Mark outgoing remote dictionary traffic for routing or QoS purposes:

  • mark: Set a specific numeric mark

  • vrf-mark: Set the mark based on a VRF

Traffic Selector Integration

The app-detect traffic selector provides comprehensive matching capabilities for application detection data, including App-ID values and detection states.

App-ID Matching

Match traffic based on Application ID values using traffic selector <txt> rule <u32> app-detect app-id:

# Match by L3 protocol (e.g., ICMP = 1)
set traffic selector ICMP rule 1 app-detect app-id l3 1

# Match by L4 port (e.g., HTTPS = 443)
set traffic selector HTTPS rule 1 app-detect app-id l4 443

# Match by custom dictionary App-ID
set traffic selector CUSTOM rule 1 app-detect app-id custom 42

# Match by specific classification engine and selector
set traffic selector ENGINE rule 1 app-detect app-id engine 128 selector 30

# Match by remote dictionary category
set traffic selector CATEGORY rule 1 app-detect app-id category 15

# Match by remote dictionary reputation
set traffic selector REPUTATION rule 1 app-detect app-id reputation greater-than 70

Category and Reputation Classification

Remote dictionaries assign Application IDs using Classification Engine IDs 130 (category) and 131 (reputation).

Category (Engine 130):

Categories are numeric identifiers representing application types (e.g., social media, streaming, business). The specific category values depend on the remote dictionary service provider.

Reputation (Engine 131):

Reputation scores indicate the trustworthiness or risk level of a destination. Higher values typically indicate lower risk.

Traffic Selector Matching:

Use traffic selectors to match category and reputation values:

# Match traffic with category 15
set traffic selector SOCIAL rule 1 app-detect app-id category 15

# Match traffic with reputation below 50 (higher risk)
set traffic selector RISKY rule 1 app-detect app-id reputation less-than 50

# Match traffic with reputation above 80 (lower risk)
set traffic selector TRUSTED rule 1 app-detect app-id reputation greater-than 80

Detection State Matching

Match traffic based on the current detection state using traffic selector <txt> rule <u32> app-detect state:

  • detecting: Traffic session currently being analyzed by the detection engine

  • detected: Traffic session where an application has been successfully identified

  • host-detected: Traffic session where DPI has detected a host (FQDN)

Example:

# Match traffic still being analyzed
set traffic selector ANALYZING rule 1 app-detect state detecting

# Match traffic identified by any DNS, HTTP or SSL application detectors
set traffic selector IDENTIFIED rule 1 app-detect state detected

# Match traffic where a host has been detected
set traffic selector HOST_FOUND rule 1 app-detect state host-detected

Application Identification Methods

Assign App-ID via Traffic Policy

Assign App-IDs through interface-applied traffic policies.

Example:

set traffic policy Test rule 1 set app-id custom 33
set interfaces ethernet eth4 traffic policy in Test

Send traffic and view the result:

# system conntrack show
icmp     1 28 src=10.215.168.1 dst=10.215.168.64 type=8 code=0 id=17504 packets=2 bytes=168 src=10.215.168.64 dst=10.215.168.1 type=0 code=0 id=17504 packets=2 bytes=168 mark=0 use=1 appdetect[U6:33]

The custom App-ID (U6:33) is displayed.

Dictionary-Based Classification

Application dictionaries provide structured sets of identifiable applications with names, App-IDs, and matching criteria (FQDN patterns or IP ranges).

CLI Dictionary Configuration

Create custom dictionaries directly via CLI commands:

# Define custom app with FQDN pattern
set system conntrack app-detect dictionary 1 custom app-id 42 fqdn enterprise.opentok.com

# Add application name
set system conntrack app-detect dictionary 1 custom app-id 42 name "Teldat Test"

  • Classification Engine ID: 6 (for CLI-defined dictionaries)

XML Dictionary Files

Define dictionaries in XML format (with optional GZIP compression):

<?xml version='1.0' encoding='UTF-8'?>
    <provider engine_id="128" name="Teldat">
        <app id="30" name="Teldat Test" version="1">
            <fqdn_list>
                <fqdn>10.215.168.1</fqdn>
            </fqdn_list>
        </app>
        <app id="31" name="Teldat Test 2" version="1">
            <address_list>
                <range id="1">
                    <net_address>10.215.168.64</net_address>
                    <net_mask>255.255.255.192</net_mask>
                </range>
            </address_list>
        </app>
    </provider>

Application Elements:

  • engine_id: Classification engine identifier (custom value, e.g., 128)

  • id: Application selector ID

  • name: Human-readable application name

  • version: Application definition version

  • fqdn_list: FQDN patterns for matching

  • address_list: IP address ranges for matching

Load Dictionary:

set system conntrack app-detect dictionary 1 filename running://path/to/dict.xml

Example Result:

After loading the dictionary and enabling private IP matching, establish an SSH connection to 10.215.168.64:

tcp      6 3597 ESTABLISHED src=10.215.168.1 dst=10.215.168.64 sport=52360 dport=22 packets=18 bytes=1016 src=10.215.168.64 dst=10.215.168.1 sport=22 dport=52360 packets=21 bytes=2452 [ASSURED] mark=0 use=1 appdetect[U128:31]

The flow is classified with App-ID U128:31 based on the destination IP range match.

Note

Dictionary-based classification can override default port-based assignment. In this example, SSH (normally L4:22) is reclassified as U128:31 per the dictionary definition.

Configuration Commands

Configuration commands

Operational Commands

Operational commands