========= Conntrack ========= Overview ======== The :osdx:cfg:`system conntrack` feature allows network connections to be tracked through OSDx devices. The conntrack feature provides the Netfilter connection tracking system with a userspace interface, thus enabling the monitoring and management of all active connections. This chapter covers the basics of connection tracking, application detection capabilities, and advanced traffic classification features for policy-based routing. Basic Connection Tracking ========================== Connection Format ----------------- Connections are displayed including the source and reply information, along with metadata. Here's an example: .. code-block:: bash icmp 1 25 src=10.0.0.1 dst=10.0.0.2 type=8 code=0 id=18723 vrf=RED packets=2 bytes=168 src=10.0.0.2 dst=10.0.0.1 type=0 code=0 id=18723 vrf=RED packets=2 bytes=168 mark=33 use=1 appdetect[L3:1] **Connection Fields:** * **Protocol**: ``icmp`` - Protocol identifier (icmp, tcp, udp, etc.) * **Protocol Number**: ``1`` - Numeric protocol identifier * **TTL**: ``25`` - Time remaining before connection expires * **Origin**: ``src=10.0.0.1 dst=10.0.0.2`` - Source and destination of original packet * **Reply**: ``src=10.0.0.2 dst=10.0.0.1`` - Source and destination of reply packet * **VRF**: ``vrf=RED`` - Virtual routing and forwarding instance * **Statistics**: ``packets=2 bytes=168`` - Packet and byte counters * **Metadata**: * ``mark=33`` - Packet mark for policy routing * ``use=1`` - Usage counter * ``appdetect[L3:1]`` - Application detection data * ``[OFFLOAD]`` or ``[HW_OFFLOAD]`` - Hardware offload status * ``Sc: not-bypass`` - Security bypass status For more information on traffic policies and marks, see :doc:`Traffic Routing ` and :doc:`Firewall Service `. Connection States ----------------- Running :osdx:op:`system conntrack monitor` displays connections with state information: * **NEW**: Newly established connections * **UPDATE**: Updated connection information * **DESTROY**: Terminated connections Logging ======= The :osdx:cfg:`system conntrack logging` feature sends connection tracking information to syslog, enabling centralized monitoring and auditing. Configuration ------------- **Event Types** Configure which connection events to log (default: **destroy**): * **NEW**: Log new connections * **UPDATE**: Log connection updates * **DESTROY**: Log connection termination * **ALL**: Log all events **Log Level** The **log-level** option controls the syslog severity level for logged messages (default: **info**): View logs using :osdx:op:`system journal monitor` or :osdx:op:`system journal show`: .. code-block:: bash osdx ulogd[9431]: [NEW] ORIG: SRC=10.0.0.1 DST=10.0.0.2 VRF=RED PROTO=TCP SPT=55406 DPT=80 PKTS=6 BYTES=577 , REPLY: SRC=10.0.0.2 DST=10.0.0.1 VRF=RED PROTO=TCP SPT=80 DPT=55406 PKTS=4 BYTES=653 MARK=33 LABELS=LOCAL APPDETECT[L4:80 http-host:10.0.0.1] Log messages include: * Origin and reply connection information * Traffic policy labels * Application detection data (app-id and detected protocols) **Custom Identity** Customize the log identifier using :osdx:cfg:`system conntrack logging identity *`: .. code-block:: bash # Changes "ulogd" to "OSDx_DUT0" set system conntrack logging identity OSDx_DUT0 Examples: :doc:`Conntrack Logging Examples ` Application Detection (App-Detect) =================================== Overview -------- :osdx:cfg:`system conntrack app-detect` is an implementation based on `RFC6759 `_ that allows applications to be identified in connection tracking flows using Deep Packet Inspection (DPI). Application detection provides traffic flows with: * **Application ID (App-ID)**: Identifies the application or protocol, the APP-ID consists of: * **Engine ID**: Network layer (L3, L4, U) * **Selector ID**: Protocol or service identifier (protocol number, port number, custom ID) * **Detected Data**: Protocol-specific information (hostnames, URLs, etc.) The App-ID assigned can be used by traffic selectors for traffic policy matching (see :doc:`Traffic Policy `). Display application detection information using: * :osdx:op:`system conntrack show` * :osdx:op:`system conntrack monitor` * :osdx:cfg:`system conntrack logging` Output format: ``appdetect[engineID:selectorID "detected data"]`` Application ID Structure ------------------------- The Application ID is made up of two components: Engine ID ********* Indicates the network layer associated with the flow: * **L3**: Layer 3 protocols (e.g., ICMP, protocol code 1 per `RFC792 `_) * **L4**: Layer 4 protocols with port-based identification (e.g., HTTP on port 80) * **U**: Userspace (Layer 7) - Custom application identifiers Selector ID *********** Specific identifier within the engine type: * **L3**: Protocol number (e.g., 1 for ICMP) * **L4**: Service port number (e.g., 80 for HTTP, 443 for HTTPS) * Application detectors can override port-based assignment (e.g., L4:80 assigned to traffic on port 8080 if HTTP is detected) * **U**: Custom application identifier set via traffic policy or dictionary System Architecture ------------------- Session-Based Packet Processing ******************************** OSDx conntrack implements session-based packet processing to efficiently classify and handle IP packets. When enabled, the system automatically associates IP (IPv4/IPv6) packets with connection tracking sessions. **Session Creation:** When a packet is processed: 1. The system attempts to find the packet's corresponding session among existing connections 2. If no matching session exists, a new session is created and assigned to the packet 3. Subsequent packets belonging to the same flow are associated with this session **Session Definition:** Sessions are identified by: * Source IP address * Destination IP address * Protocol (e.g., TCP, UDP, ICMP) * Source and destination ports (for TCP/UDP protocols) This means a session can represent various types of traffic flows, such as: * A TCP connection between devices * ICMP echo requests and replies Application Identification Integration *************************************** To boost session identification, an Application ID (App-ID) is linked to each session through the application detection subsystem (app-detect). The App-ID assigned provides: * Session classification based on detected applications * Traffic policy matching via traffic selectors * Enhanced visibility in connection tracking logs and netflow exportation **How Detection Works:** When app-detect is configured via :osdx:cfg:`system conntrack app-detect`, OSDx automatically enables application detection for all network traffic passing through the device. The system processes both incoming traffic and locally-generated traffic. The detection process works as follows: 1. **Cache Lookup**: When a new connection starts, OSDx first checks if the destination IP address has a recently detected cached App-ID: * If found, the cached App-ID is assigned to the connection 2. **Deep Packet Inspection**: DPI runs on each packet until the application is identified or detection completes (DNS inspection could continue detecting the host even after initial resolution). The number of packets required, if fragmentation isn't applied, varies by protocol: * **DNS (UDP)**: 2 packets (query and response) * **DNS (TCP)**: 5 packets (TCP handshake, query, and response) * **HTTP**: 4 packets (TCP handshake and HTTP request) * **SSL/TLS**: 4 packets (TCP handshake and Client Hello) During inspection, the system: * Examines protocol-specific patterns (HTTP requests, SSL/TLS handshakes, DNS messages) * Extracts relevant information (hostnames, URLs, query names) * Searches configured application dictionaries for matching patterns * Continues inspection until the application is identified or deemed undetectable 3. **App-ID Assignment**: Once the application is identified, the App-ID is: * Stored in the connection tracking entry * Cached for the destination IP address to speed up detection in future connections and perform First Packet Detection (FPD) * Immediately available for use in traffic policy rules 4. **Subsequent Packets**: All remaining packets in the connection inherit the assigned App-ID without further inspection. The assigned App-ID can be referenced in traffic policy rules to enable application-aware classification for policy-based routing, QoS, and security filtering. .. note:: When :osdx:cfg:`system offload` is configured, once an application is detected for a session, subsequent packets can be offloaded to the fast path. Application Detectors ---------------------- Application detectors use DPI to identify specific applications by analyzing packet content. The system examines protocol-specific patterns and structures to identify DNS, HTTP, and SSL/TLS traffic. DNS Detection ************* Detects DNS protocol traffic by analyzing DNS message structure and validating message format. * **Protocol**: DNS (UDP/TCP) * **Detection Method**: Analyzes DNS header fields and message structure * **App-ID Assignment**: L4:53 (or L4:5353 for multicast DNS), regardless of actual server port * **Detection Mode**: * Continuously inspects DNS flows by default to detect each query in the session * When :osdx:cfg:`system conntrack app-detect dns-host disable-continuous-resolution` is enabled, inspects only the first query HTTP Detection ************** Detects HTTP protocol traffic (versions 1.0 and 1.1) by analyzing HTTP request methods and headers. * **Protocol**: HTTP on TCP connections * **Detection Method**: Validates HTTP request methods and parses request headers * **App-ID Assignment**: L4:80, regardless of actual server port * **Detection Scope**: Only the initial session request (persistent connections show first request only) SSL/TLS Detection ***************** Detects SSL/TLS encrypted traffic by analyzing the TLS handshake. * **Protocol**: SSL/TLS (all versions) * **Detection Method**: Parses the TLS Client Hello message to identify encrypted connections * **App-ID Assignment**: Port-based (e.g., L4:443 for HTTPS) * SSL/TLS is a transport layer protocol, not a final application identifier * The server port provides better application context * **Detection Scope**: Only the initial Client Hello message Data Extraction --------------- In addition to protocol detection, app-detect can extract protocol-specific metadata from detected traffic. The data extracted is displayed alongside the App-ID in connection tracking output and logs. DNS Data Extraction ******************* Extracts domain names from DNS queries and responses. * **Configuration**: :osdx:cfg:`system conntrack app-detect dns-host` * **Extracted Data**: * DNS query names (hostnames being resolved) HTTP Data Extraction ******************** Extracts various fields from HTTP request headers. * **Available Fields**: * :osdx:cfg:`system conntrack app-detect http-host` - Extracts Host header (domain name) * :osdx:cfg:`system conntrack app-detect http-url` - Extracts complete request URL * :osdx:cfg:`system conntrack app-detect http-referer` - Extracts Referer header * :osdx:cfg:`system conntrack app-detect http-user-agent` - Extracts User-Agent string SSL/TLS Data Extraction *********************** Extracts server hostname from SSL/TLS handshakes. * **Configuration**: :osdx:cfg:`system conntrack app-detect ssl-host` * **Extracted Data**: Server hostname from Server Name Indication (SNI) extension Application Identification Methods =================================== Assign App-ID via Traffic Policy --------------------------------- Assign App-IDs through interface-applied traffic policies. **Example:** .. code-block:: none set traffic policy Test rule 1 set app-id custom 33 set interfaces ethernet eth4 traffic policy in Test Send traffic and view the result: .. code-block:: bash # system conntrack show icmp 1 28 src=10.215.168.1 dst=10.215.168.64 type=8 code=0 id=17504 packets=2 bytes=168 src=10.215.168.64 dst=10.215.168.1 type=0 code=0 id=17504 packets=2 bytes=168 mark=0 use=1 appdetect[U6:33] The custom App-ID (U6:33) is displayed. Dictionary-Based Classification -------------------------------- Application dictionaries provide structured sets of identifiable applications with names, App-IDs, and matching criteria (FQDN patterns or IP ranges). CLI Dictionary Configuration ***************************** Create custom dictionaries directly via CLI commands: .. code-block:: bash # Define custom app with FQDN pattern set system conntrack app-detect dictionary 1 custom app-id 42 fqdn enterprise.opentok.com # Add application name set system conntrack app-detect dictionary 1 custom app-id 42 name "Teldat Test" * **Classification Engine ID**: 6 (for CLI-defined dictionaries) XML Dictionary Files ******************** Define dictionaries in XML format (with optional GZIP compression): .. code-block:: xml 10.215.168.1 10.215.168.64 255.255.255.192 **Application Elements:** * ``engine_id``: Classification engine identifier (custom value, e.g., 128) * ``id``: Application selector ID * ``name``: Human-readable application name * ``version``: Application definition version * ``fqdn_list``: FQDN patterns for matching * ``address_list``: IP address ranges for matching **Load Dictionary:** .. code-block:: bash set system conntrack app-detect dictionary 1 filename running://path/to/dict.xml **Example Result:** After loading the dictionary and enabling private IP matching, establish an SSH connection to 10.215.168.64: .. code-block:: bash tcp 6 3597 ESTABLISHED src=10.215.168.1 dst=10.215.168.64 sport=52360 dport=22 packets=18 bytes=1016 src=10.215.168.64 dst=10.215.168.1 sport=22 dport=52360 packets=21 bytes=2452 [ASSURED] mark=0 use=1 appdetect[U128:31] The flow is classified with App-ID U128:31 based on the destination IP range match. .. note:: Dictionary-based classification can override default port-based assignment. In this example, SSH (normally L4:22) is reclassified as U128:31 per the dictionary definition. Configuration Commands ---------------------- .. osdx:cmdtree:: cfg :maxdepth: 2 system conntrack Operational Commands -------------------- .. osdx:cmdtree:: op :maxdepth: 2 system conntrack