Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 18561 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.629 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.629/0.629/0.629/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.529 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.529/0.529/0.529/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 43648 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 117 MBytes 982 Mbits/sec 291 1.29 MBytes [ 5] 1.00-2.00 sec 134 MBytes 1.12 Gbits/sec 0 1.38 MBytes [ 5] 2.00-3.00 sec 146 MBytes 1.23 Gbits/sec 0 1.44 MBytes [ 5] 3.00-4.00 sec 119 MBytes 996 Mbits/sec 0 1.49 MBytes [ 5] 4.00-5.00 sec 100 MBytes 839 Mbits/sec 0 1.54 MBytes [ 5] 5.00-6.00 sec 132 MBytes 1.11 Gbits/sec 0 1.60 MBytes [ 5] 6.00-7.00 sec 145 MBytes 1.22 Gbits/sec 0 1.67 MBytes [ 5] 7.00-8.00 sec 152 MBytes 1.28 Gbits/sec 239 1.22 MBytes [ 5] 8.00-9.00 sec 122 MBytes 1.03 Gbits/sec 0 1.37 MBytes [ 5] 9.00-10.00 sec 140 MBytes 1.17 Gbits/sec 0 1.48 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.28 GBytes 1.10 Gbits/sec 530 sender [ 5] 0.00-10.01 sec 1.28 GBytes 1.10 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:10:28.111963 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43644 -> 40.0.0.2:5001 01/27/2026-11:10:28.113332 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43648 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23774 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.588 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.588/0.588/0.588/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.563 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.563/0.563/0.563/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 41220 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 247 MBytes 2.07 Gbits/sec 11 1.63 MBytes [ 5] 1.00-2.00 sec 239 MBytes 2.00 Gbits/sec 0 1.77 MBytes [ 5] 2.00-3.00 sec 216 MBytes 1.81 Gbits/sec 128 1.35 MBytes [ 5] 3.00-4.00 sec 202 MBytes 1.70 Gbits/sec 0 1.45 MBytes [ 5] 4.00-5.00 sec 232 MBytes 1.95 Gbits/sec 0 1.56 MBytes [ 5] 5.00-6.00 sec 224 MBytes 1.88 Gbits/sec 0 1.66 MBytes [ 5] 6.00-7.00 sec 266 MBytes 2.23 Gbits/sec 22 1.24 MBytes [ 5] 7.00-8.00 sec 250 MBytes 2.10 Gbits/sec 0 1.38 MBytes [ 5] 8.00-9.00 sec 260 MBytes 2.18 Gbits/sec 0 1.50 MBytes [ 5] 9.00-10.00 sec 286 MBytes 2.40 Gbits/sec 0 1.64 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.37 GBytes 2.03 Gbits/sec 161 sender [ 5] 0.00-10.00 sec 2.37 GBytes 2.03 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:11:03.110327 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41214 -> 40.0.0.2:5001 01/27/2026-11:11:03.111484 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41220 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jan 27 11:10:54.298615 osdx systemd-journald[1949]: Runtime Journal (/run/log/journal/19d27b7fd9034c15b59c452d6ca2fcd4) is 1.7M, max 13.8M, 12.0M free. Jan 27 11:10:54.301369 osdx systemd-journald[1949]: Received client request to rotate journal, rotating. Jan 27 11:10:54.301425 osdx systemd-journald[1949]: Vacuuming done, freed 0B of archived journals from /run/log/journal/19d27b7fd9034c15b59c452d6ca2fcd4. Jan 27 11:10:54.309472 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'system journal clear'. Jan 27 11:10:54.563393 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'system coredump delete all'. Jan 27 11:10:54.779302 osdx OSDxCLI[105994]: User 'admin' entered the configuration menu. Jan 27 11:10:54.874029 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jan 27 11:10:54.943443 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jan 27 11:10:55.037532 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jan 27 11:10:55.104197 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jan 27 11:10:55.235702 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jan 27 11:10:55.292270 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jan 27 11:10:55.402949 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'show working'. Jan 27 11:10:55.473270 osdx ubnt-cfgd[106486]: inactive Jan 27 11:10:55.536238 osdx INFO[106492]: FRR daemons did not change Jan 27 11:10:55.573373 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jan 27 11:10:55.589086 osdx WARNING[106532]: No supported link modes on interface eth1 Jan 27 11:10:55.590524 osdx modulelauncher[106532]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Jan 27 11:10:55.590535 osdx modulelauncher[106532]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Jan 27 11:10:55.591935 osdx modulelauncher[106532]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Jan 27 11:10:55.591943 osdx modulelauncher[106532]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jan 27 11:10:55.638511 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jan 27 11:10:55.683537 osdx WARNING[106612]: No supported link modes on interface eth0 Jan 27 11:10:55.684936 osdx modulelauncher[106612]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Jan 27 11:10:55.684948 osdx modulelauncher[106612]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Jan 27 11:10:55.686081 osdx modulelauncher[106612]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Jan 27 11:10:55.686089 osdx modulelauncher[106612]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jan 27 11:10:55.696470 osdx (udev-worker)[106628]: Network interface NamePolicy= disabled on kernel command line. Jan 27 11:10:55.876085 osdx cfgd[1624]: [105994]Completed change to active configuration Jan 27 11:10:55.888027 osdx OSDxCLI[105994]: User 'admin' committed the configuration. Jan 27 11:10:55.904759 osdx OSDxCLI[105994]: User 'admin' left the configuration menu. Jan 27 11:10:58.247916 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jan 27 11:10:58.329445 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jan 27 11:10:58.436063 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jan 27 11:10:58.960335 osdx file_operation[106807]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jan 27 11:10:58.988818 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jan 27 11:10:59.147898 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jan 27 11:10:59.285115 osdx OSDxCLI[105994]: User 'admin' entered the configuration menu. Jan 27 11:10:59.354878 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jan 27 11:10:59.477443 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jan 27 11:10:59.597149 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jan 27 11:10:59.751704 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jan 27 11:10:59.865552 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jan 27 11:10:59.978789 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jan 27 11:11:00.076541 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jan 27 11:11:00.144706 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jan 27 11:11:00.233637 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jan 27 11:11:00.299202 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jan 27 11:11:00.393050 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jan 27 11:11:00.456696 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jan 27 11:11:00.636814 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jan 27 11:11:00.719975 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jan 27 11:11:00.867770 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jan 27 11:11:00.963055 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jan 27 11:11:01.080560 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jan 27 11:11:01.177105 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jan 27 11:11:01.247937 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jan 27 11:11:01.369265 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jan 27 11:11:01.460368 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'show working'. Jan 27 11:11:01.550515 osdx ubnt-cfgd[106860]: inactive Jan 27 11:11:01.728056 osdx INFO[106908]: FRR daemons did not change Jan 27 11:11:02.183720 osdx systemd[1]: Reloading. Jan 27 11:11:02.249376 osdx systemd-sysv-generator[106973]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jan 27 11:11:02.365825 osdx systemd[1]: Starting logrotate.service - Rotate log files... Jan 27 11:11:02.370455 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jan 27 11:11:02.393076 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jan 27 11:11:02.399749 osdx systemd[1]: logrotate.service: Deactivated successfully. Jan 27 11:11:02.399928 osdx systemd[1]: Finished logrotate.service - Rotate log files. Jan 27 11:11:02.646114 osdx INFO[106952]: Rules successfully loaded Jan 27 11:11:02.646749 osdx cfgd[1624]: [105994]Completed change to active configuration Jan 27 11:11:02.649151 osdx OSDxCLI[105994]: User 'admin' committed the configuration. Jan 27 11:11:02.728278 osdx OSDxCLI[105994]: User 'admin' left the configuration menu. Jan 27 11:11:03.113396 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=17571 DF PROTO=TCP SPT=41214 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jan 27 11:11:03.113470 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=24178 DF PROTO=TCP SPT=41220 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jan 27 11:11:13.388988 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 47046 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 196 MBytes 1.65 Gbits/sec 70 1.30 MBytes [ 5] 1.00-2.00 sec 242 MBytes 2.03 Gbits/sec 0 1.42 MBytes [ 5] 2.00-3.00 sec 215 MBytes 1.80 Gbits/sec 0 1.53 MBytes [ 5] 3.00-4.00 sec 206 MBytes 1.73 Gbits/sec 0 1.62 MBytes [ 5] 4.00-5.00 sec 216 MBytes 1.81 Gbits/sec 0 1.71 MBytes [ 5] 5.00-6.00 sec 214 MBytes 1.79 Gbits/sec 0 1.80 MBytes [ 5] 6.00-7.00 sec 204 MBytes 1.71 Gbits/sec 85 1.33 MBytes [ 5] 7.00-8.00 sec 250 MBytes 2.10 Gbits/sec 0 1.47 MBytes [ 5] 8.00-9.00 sec 245 MBytes 2.06 Gbits/sec 43 1.18 MBytes [ 5] 9.00-10.01 sec 301 MBytes 2.51 Gbits/sec 0 1.35 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 2.24 GBytes 1.92 Gbits/sec 198 sender [ 5] 0.00-10.01 sec 2.23 GBytes 1.92 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:11:03.110327 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41214 -> 40.0.0.2:5001 01/27/2026-11:11:03.111484 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41220 -> 40.0.0.2:5001 01/27/2026-11:11:19.011648 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47042 -> 40.0.0.2:5001 01/27/2026-11:11:19.013598 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47046 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jan 27 11:10:54.298615 osdx systemd-journald[1949]: Runtime Journal (/run/log/journal/19d27b7fd9034c15b59c452d6ca2fcd4) is 1.7M, max 13.8M, 12.0M free. Jan 27 11:10:54.301369 osdx systemd-journald[1949]: Received client request to rotate journal, rotating. Jan 27 11:10:54.301425 osdx systemd-journald[1949]: Vacuuming done, freed 0B of archived journals from /run/log/journal/19d27b7fd9034c15b59c452d6ca2fcd4. Jan 27 11:10:54.309472 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'system journal clear'. Jan 27 11:10:54.563393 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'system coredump delete all'. Jan 27 11:10:54.779302 osdx OSDxCLI[105994]: User 'admin' entered the configuration menu. Jan 27 11:10:54.874029 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jan 27 11:10:54.943443 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jan 27 11:10:55.037532 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jan 27 11:10:55.104197 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jan 27 11:10:55.235702 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jan 27 11:10:55.292270 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jan 27 11:10:55.402949 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'show working'. Jan 27 11:10:55.473270 osdx ubnt-cfgd[106486]: inactive Jan 27 11:10:55.536238 osdx INFO[106492]: FRR daemons did not change Jan 27 11:10:55.573373 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jan 27 11:10:55.589086 osdx WARNING[106532]: No supported link modes on interface eth1 Jan 27 11:10:55.590524 osdx modulelauncher[106532]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Jan 27 11:10:55.590535 osdx modulelauncher[106532]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Jan 27 11:10:55.591935 osdx modulelauncher[106532]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Jan 27 11:10:55.591943 osdx modulelauncher[106532]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jan 27 11:10:55.638511 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jan 27 11:10:55.683537 osdx WARNING[106612]: No supported link modes on interface eth0 Jan 27 11:10:55.684936 osdx modulelauncher[106612]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Jan 27 11:10:55.684948 osdx modulelauncher[106612]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Jan 27 11:10:55.686081 osdx modulelauncher[106612]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Jan 27 11:10:55.686089 osdx modulelauncher[106612]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jan 27 11:10:55.696470 osdx (udev-worker)[106628]: Network interface NamePolicy= disabled on kernel command line. Jan 27 11:10:55.876085 osdx cfgd[1624]: [105994]Completed change to active configuration Jan 27 11:10:55.888027 osdx OSDxCLI[105994]: User 'admin' committed the configuration. Jan 27 11:10:55.904759 osdx OSDxCLI[105994]: User 'admin' left the configuration menu. Jan 27 11:10:58.247916 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jan 27 11:10:58.329445 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jan 27 11:10:58.436063 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jan 27 11:10:58.960335 osdx file_operation[106807]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jan 27 11:10:58.988818 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jan 27 11:10:59.147898 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jan 27 11:10:59.285115 osdx OSDxCLI[105994]: User 'admin' entered the configuration menu. Jan 27 11:10:59.354878 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jan 27 11:10:59.477443 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jan 27 11:10:59.597149 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jan 27 11:10:59.751704 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jan 27 11:10:59.865552 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jan 27 11:10:59.978789 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jan 27 11:11:00.076541 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jan 27 11:11:00.144706 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jan 27 11:11:00.233637 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jan 27 11:11:00.299202 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jan 27 11:11:00.393050 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jan 27 11:11:00.456696 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jan 27 11:11:00.636814 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jan 27 11:11:00.719975 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jan 27 11:11:00.867770 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jan 27 11:11:00.963055 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jan 27 11:11:01.080560 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jan 27 11:11:01.177105 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jan 27 11:11:01.247937 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jan 27 11:11:01.369265 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jan 27 11:11:01.460368 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'show working'. Jan 27 11:11:01.550515 osdx ubnt-cfgd[106860]: inactive Jan 27 11:11:01.728056 osdx INFO[106908]: FRR daemons did not change Jan 27 11:11:02.183720 osdx systemd[1]: Reloading. Jan 27 11:11:02.249376 osdx systemd-sysv-generator[106973]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jan 27 11:11:02.365825 osdx systemd[1]: Starting logrotate.service - Rotate log files... Jan 27 11:11:02.370455 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jan 27 11:11:02.393076 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jan 27 11:11:02.399749 osdx systemd[1]: logrotate.service: Deactivated successfully. Jan 27 11:11:02.399928 osdx systemd[1]: Finished logrotate.service - Rotate log files. Jan 27 11:11:02.646114 osdx INFO[106952]: Rules successfully loaded Jan 27 11:11:02.646749 osdx cfgd[1624]: [105994]Completed change to active configuration Jan 27 11:11:02.649151 osdx OSDxCLI[105994]: User 'admin' committed the configuration. Jan 27 11:11:02.728278 osdx OSDxCLI[105994]: User 'admin' left the configuration menu. Jan 27 11:11:03.113396 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=17571 DF PROTO=TCP SPT=41214 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jan 27 11:11:03.113470 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=24178 DF PROTO=TCP SPT=41220 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jan 27 11:11:13.388988 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Jan 27 11:11:13.549900 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'system journal show | cat'. Jan 27 11:11:13.755448 osdx OSDxCLI[105994]: User 'admin' entered the configuration menu. Jan 27 11:11:13.821182 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jan 27 11:11:13.922001 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jan 27 11:11:13.987788 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jan 27 11:11:14.188241 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jan 27 11:11:14.259934 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jan 27 11:11:14.346165 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jan 27 11:11:14.541018 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jan 27 11:11:14.603190 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jan 27 11:11:14.726996 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jan 27 11:11:14.807642 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jan 27 11:11:14.889453 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jan 27 11:11:14.944158 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jan 27 11:11:15.033906 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jan 27 11:11:15.097276 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jan 27 11:11:15.187547 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jan 27 11:11:15.246764 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jan 27 11:11:15.340506 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jan 27 11:11:15.420961 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jan 27 11:11:15.580784 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jan 27 11:11:15.652338 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jan 27 11:11:15.767788 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Jan 27 11:11:15.862720 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Jan 27 11:11:16.066708 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Jan 27 11:11:16.169423 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Jan 27 11:11:16.320145 osdx OSDxCLI[105994]: User 'admin' added a new cfg line: 'show changes'. Jan 27 11:11:16.424108 osdx ubnt-cfgd[107100]: inactive Jan 27 11:11:16.482401 osdx INFO[107120]: FRR daemons did not change Jan 27 11:11:16.712732 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Jan 27 11:11:18.466830 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Jan 27 11:11:18.466938 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Jan 27 11:11:18.466968 osdx systemd[1]: suricata@FW.service: Consumed 1.863s CPU time. Jan 27 11:11:18.489647 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jan 27 11:11:18.513598 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jan 27 11:11:18.746108 osdx INFO[107147]: Rules successfully loaded Jan 27 11:11:18.746794 osdx cfgd[1624]: [105994]Completed change to active configuration Jan 27 11:11:18.749268 osdx OSDxCLI[105994]: User 'admin' committed the configuration. Jan 27 11:11:18.768548 osdx OSDxCLI[105994]: User 'admin' left the configuration menu. Jan 27 11:11:19.013413 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=25407 DF PROTO=TCP SPT=47042 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jan 27 11:11:19.017888 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=47103 DF PROTO=TCP SPT=47046 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jan 27 11:11:24.032159 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Jan 27 11:11:29.267879 osdx OSDxCLI[105994]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 4345 0 --:--:-- --:--:-- --:--:-- 4448
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.528 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.528/0.528/0.528/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.524 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.524/0.524/0.524/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 36528 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.01 sec 194 MBytes 1.62 Gbits/sec 583 1.80 MBytes [ 5] 1.01-2.00 sec 172 MBytes 1.46 Gbits/sec 41 1.36 MBytes [ 5] 2.00-3.00 sec 216 MBytes 1.81 Gbits/sec 0 1.47 MBytes [ 5] 3.00-4.00 sec 180 MBytes 1.51 Gbits/sec 59 1.11 MBytes [ 5] 4.00-5.00 sec 159 MBytes 1.33 Gbits/sec 0 1.21 MBytes [ 5] 5.00-6.00 sec 238 MBytes 1.98 Gbits/sec 0 1.34 MBytes [ 5] 6.00-7.00 sec 122 MBytes 1.03 Gbits/sec 0 1.41 MBytes [ 5] 7.00-8.00 sec 208 MBytes 1.75 Gbits/sec 96 1.06 MBytes [ 5] 8.00-9.00 sec 191 MBytes 1.60 Gbits/sec 0 1.19 MBytes [ 5] 9.00-10.00 sec 246 MBytes 2.07 Gbits/sec 0 1.31 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.88 GBytes 1.62 Gbits/sec 779 sender [ 5] 0.00-10.00 sec 1.88 GBytes 1.61 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:11:54.095734 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36514 -> 40.0.0.2:5001 01/27/2026-11:11:54.096705 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36528 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=229 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=229 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=36528 dport=5001 packets=1395655 bytes=2093472353 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=36528 packets=107994 bytes=5610920 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=70 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=70 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=36514 dport=5001 packets=14 bytes=1196 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=36514 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 53264 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 305 MBytes 2.56 Gbits/sec 25 1.43 MBytes [ 5] 1.00-2.00 sec 299 MBytes 2.51 Gbits/sec 0 1.57 MBytes [ 5] 2.00-3.00 sec 288 MBytes 2.41 Gbits/sec 57 1.25 MBytes [ 5] 3.00-4.00 sec 246 MBytes 2.07 Gbits/sec 0 1.38 MBytes [ 5] 4.00-5.00 sec 298 MBytes 2.49 Gbits/sec 0 1.53 MBytes [ 5] 5.00-6.00 sec 321 MBytes 2.70 Gbits/sec 3 1.26 MBytes [ 5] 6.00-7.00 sec 291 MBytes 2.44 Gbits/sec 0 1.42 MBytes [ 5] 7.00-8.00 sec 274 MBytes 2.30 Gbits/sec 0 1.55 MBytes [ 5] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 104 1.25 MBytes [ 5] 9.00-10.00 sec 324 MBytes 2.72 Gbits/sec 0 1.43 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.86 GBytes 2.45 Gbits/sec 189 sender [ 5] 0.00-10.00 sec 2.85 GBytes 2.45 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:11:54.095734 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36514 -> 40.0.0.2:5001 01/27/2026-11:11:54.096705 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36528 -> 40.0.0.2:5001 01/27/2026-11:12:09.815892 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53256 -> 40.0.0.2:5001 01/27/2026-11:12:09.816925 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53264 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=53264 dport=5001 packets=2117489 bytes=3176216057 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53264 packets=268540 bytes=13959192 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=53256 dport=5001 packets=16 bytes=1301 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53256 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 7162 0 --:--:-- --:--:-- --:--:-- 7407
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.569 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.569/0.569/0.569/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.524 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.524/0.524/0.524/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
01/27/2026-11:12:51.999970 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47044 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=232 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=232 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=73 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=73 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=47044 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=47044 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
01/27/2026-11:12:51.999970 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47044 -> 40.0.0.2:5000 01/27/2026-11:12:58.395010 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47044 -> 40.0.0.2:5000 01/27/2026-11:13:00.003347 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47282 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=47282 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=47282 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 18019 0 --:--:-- --:--:-- --:--:-- 18428
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.609 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.609/0.609/0.609/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.529 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.529/0.529/0.529/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
01/27/2026-11:13:26.309000 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37272 -> 40.0.0.2:5001 01/27/2026-11:13:26.310418 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37288 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=76 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=76 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=235 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=235 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=37272 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=37272 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=37288 dport=5001 packets=46831 bytes=70237297 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=37288 packets=4529 bytes=236224 [ASSURED] [OFFLOAD, packets=46818 bytes=70222096 packets=4527 bytes=236112] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 11665 0 --:--:-- --:--:-- --:--:-- 12090
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
01/27/2026-11:13:26.309000 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37272 -> 40.0.0.2:5001 01/27/2026-11:13:26.310418 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37288 -> 40.0.0.2:5001 01/27/2026-11:13:29.142516 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37292 -> 40.0.0.2:5001 01/27/2026-11:13:29.145952 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:38361 -> 40.0.0.2:5001 01/27/2026-11:13:29.156623 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:38361 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
udp 17 src=20.0.0.2 dst=40.0.0.2 sport=38361 dport=5001 packets=11 bytes=14792 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=38361 packets=1 bytes=32 [OFFLOAD, packets=7 bytes=10332 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=37292 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=37292 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 37085 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.649 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.649/0.649/0.649/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.612 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.612/0.612/0.612/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
01/27/2026-11:13:50.735818 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47722 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 47722 yes 201 0 0 0 20.0.0.2 40.0.0.2 47722 5000 yes 201 0 7 615
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250313s ending at 1769512434.374069 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
01/27/2026-11:13:50.735818 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47722 -> 40.0.0.2:5000 01/27/2026-11:13:54.545337 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:49533 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 47722 yes 201 0 0 0 40.0.0.2 20.0.0.2 5001 49533 no 201 0 0 0 20.0.0.2 40.0.0.2 49533 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 47722 5000 yes 201 0 11 847
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250131s ending at 1769512438.185999 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)