Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23875 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.560 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.560/0.560/0.560/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.422 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.422/0.422/0.422/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 53414 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 50.5 MBytes 424 Mbits/sec 0 2.38 MBytes [ 5] 1.00-2.00 sec 81.2 MBytes 682 Mbits/sec 20 1.88 MBytes [ 5] 2.00-3.00 sec 96.2 MBytes 807 Mbits/sec 89 1.43 MBytes [ 5] 3.00-4.00 sec 96.2 MBytes 807 Mbits/sec 0 1.52 MBytes [ 5] 4.00-5.00 sec 91.2 MBytes 766 Mbits/sec 0 1.58 MBytes [ 5] 5.00-6.00 sec 88.8 MBytes 744 Mbits/sec 0 1.63 MBytes [ 5] 6.00-7.00 sec 91.2 MBytes 765 Mbits/sec 0 1.66 MBytes [ 5] 7.00-8.00 sec 95.0 MBytes 797 Mbits/sec 171 1.18 MBytes [ 5] 8.00-9.00 sec 82.5 MBytes 692 Mbits/sec 0 1.27 MBytes [ 5] 9.00-10.00 sec 97.5 MBytes 818 Mbits/sec 0 1.33 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 870 MBytes 730 Mbits/sec 280 sender [ 5] 0.00-10.01 sec 868 MBytes 728 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:39:52.433884 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53400 -> 40.0.0.2:5001 03/24/2026-12:39:52.434840 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53414 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 37207 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.539 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.539/0.539/0.539/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.470 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.470/0.470/0.470/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 38240 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 313 MBytes 2.63 Gbits/sec 176 1.36 MBytes [ 5] 1.00-2.00 sec 308 MBytes 2.58 Gbits/sec 0 1.52 MBytes [ 5] 2.00-3.00 sec 324 MBytes 2.72 Gbits/sec 0 1.67 MBytes [ 5] 3.00-4.00 sec 318 MBytes 2.66 Gbits/sec 0 1.80 MBytes [ 5] 4.00-5.00 sec 308 MBytes 2.58 Gbits/sec 0 1.93 MBytes [ 5] 5.00-6.00 sec 295 MBytes 2.47 Gbits/sec 137 1.47 MBytes [ 5] 6.00-7.00 sec 314 MBytes 2.63 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 320 MBytes 2.68 Gbits/sec 0 1.75 MBytes [ 5] 8.00-9.00 sec 321 MBytes 2.69 Gbits/sec 0 1.88 MBytes [ 5] 9.00-10.00 sec 366 MBytes 3.07 Gbits/sec 0 2.01 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec 313 sender [ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:40:23.980997 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38230 -> 40.0.0.2:5001 03/24/2026-12:40:23.981901 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38240 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 24 12:40:16.000398 osdx systemd-timedated[249852]: Changed local time to Tue 2026-03-24 12:40:16 UTC Mar 24 12:40:16.002030 osdx systemd-journald[1674]: Time jumped backwards, rotating. Mar 24 12:40:16.002403 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'set date 2026-03-24 12:40:16'. Mar 24 12:40:16.317711 osdx systemd-journald[1674]: Runtime Journal (/run/log/journal/aec016c8ce304ac68362b205c4156884) is 1.9M, max 13.8M, 11.8M free. Mar 24 12:40:16.318275 osdx systemd-journald[1674]: Received client request to rotate journal, rotating. Mar 24 12:40:16.318317 osdx systemd-journald[1674]: Vacuuming done, freed 0B of archived journals from /run/log/journal/aec016c8ce304ac68362b205c4156884. Mar 24 12:40:16.329425 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'system journal clear'. Mar 24 12:40:16.535347 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'system coredump delete all'. Mar 24 12:40:16.754017 osdx OSDxCLI[249431]: User 'admin' entered the configuration menu. Mar 24 12:40:16.830665 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 24 12:40:16.914332 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 24 12:40:16.970205 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 24 12:40:17.068341 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 24 12:40:17.127742 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 24 12:40:17.224337 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 24 12:40:17.292898 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'show working'. Mar 24 12:40:17.392357 osdx ubnt-cfgd[249884]: inactive Mar 24 12:40:17.424847 osdx INFO[249894]: FRR daemons did not change Mar 24 12:40:17.442023 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 24 12:40:17.510025 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 24 12:40:17.531952 osdx (udev-worker)[250016]: Network interface NamePolicy= disabled on kernel command line. Mar 24 12:40:17.668267 osdx cfgd[1472]: [249431]Completed change to active configuration Mar 24 12:40:17.679787 osdx OSDxCLI[249431]: User 'admin' committed the configuration. Mar 24 12:40:17.703785 osdx OSDxCLI[249431]: User 'admin' left the configuration menu. Mar 24 12:40:19.711127 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 24 12:40:19.790401 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 24 12:40:19.868231 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 24 12:40:20.348880 osdx file_operation[250181]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 24 12:40:20.374230 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 24 12:40:20.547401 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 24 12:40:20.709511 osdx OSDxCLI[249431]: User 'admin' entered the configuration menu. Mar 24 12:40:20.797737 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 24 12:40:20.908124 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 24 12:40:20.975970 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 24 12:40:21.084363 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 24 12:40:21.152352 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 24 12:40:21.255645 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 24 12:40:21.342895 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 24 12:40:21.423342 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 24 12:40:21.484747 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 24 12:40:21.585145 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 24 12:40:21.645572 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 24 12:40:21.753275 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 24 12:40:21.818652 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 24 12:40:21.919285 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 24 12:40:21.990315 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 24 12:40:22.101864 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 24 12:40:22.157494 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 24 12:40:22.274357 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 24 12:40:22.374156 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 24 12:40:22.439025 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 24 12:40:22.556585 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'show working'. Mar 24 12:40:22.642158 osdx ubnt-cfgd[250232]: inactive Mar 24 12:40:22.702357 osdx INFO[250269]: FRR daemons did not change Mar 24 12:40:23.160235 osdx systemd[1]: Reloading. Mar 24 12:40:23.206030 osdx systemd-sysv-generator[250329]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 24 12:40:23.326414 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 24 12:40:23.343851 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 24 12:40:23.540003 osdx INFO[250311]: Rules successfully loaded Mar 24 12:40:23.540789 osdx cfgd[1472]: [249431]Completed change to active configuration Mar 24 12:40:23.543045 osdx OSDxCLI[249431]: User 'admin' committed the configuration. Mar 24 12:40:23.593914 osdx OSDxCLI[249431]: User 'admin' left the configuration menu. Mar 24 12:40:23.982032 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=50123 DF PROTO=TCP SPT=38230 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 24 12:40:23.982114 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62076 DF PROTO=TCP SPT=38240 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 24 12:40:34.179135 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 47658 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 266 MBytes 2.23 Gbits/sec 38 1.93 MBytes [ 5] 1.00-2.00 sec 370 MBytes 3.10 Gbits/sec 0 2.09 MBytes [ 5] 2.00-3.00 sec 369 MBytes 3.09 Gbits/sec 0 2.22 MBytes [ 5] 3.00-4.01 sec 306 MBytes 2.55 Gbits/sec 168 1.19 MBytes [ 5] 4.01-5.00 sec 330 MBytes 2.79 Gbits/sec 0 1.38 MBytes [ 5] 5.00-6.00 sec 384 MBytes 3.22 Gbits/sec 0 1.57 MBytes [ 5] 6.00-7.00 sec 352 MBytes 2.96 Gbits/sec 0 1.73 MBytes [ 5] 7.00-8.00 sec 328 MBytes 2.75 Gbits/sec 0 1.86 MBytes [ 5] 8.00-9.00 sec 339 MBytes 2.84 Gbits/sec 0 1.99 MBytes [ 5] 9.00-10.00 sec 338 MBytes 2.83 Gbits/sec 32 1.48 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.30 GBytes 2.84 Gbits/sec 238 sender [ 5] 0.00-10.00 sec 3.30 GBytes 2.83 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:40:23.980997 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38230 -> 40.0.0.2:5001 03/24/2026-12:40:23.981901 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38240 -> 40.0.0.2:5001 03/24/2026-12:40:39.792283 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47656 -> 40.0.0.2:5001 03/24/2026-12:40:39.793323 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47658 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 24 12:40:16.000398 osdx systemd-timedated[249852]: Changed local time to Tue 2026-03-24 12:40:16 UTC Mar 24 12:40:16.002030 osdx systemd-journald[1674]: Time jumped backwards, rotating. Mar 24 12:40:16.002403 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'set date 2026-03-24 12:40:16'. Mar 24 12:40:16.317711 osdx systemd-journald[1674]: Runtime Journal (/run/log/journal/aec016c8ce304ac68362b205c4156884) is 1.9M, max 13.8M, 11.8M free. Mar 24 12:40:16.318275 osdx systemd-journald[1674]: Received client request to rotate journal, rotating. Mar 24 12:40:16.318317 osdx systemd-journald[1674]: Vacuuming done, freed 0B of archived journals from /run/log/journal/aec016c8ce304ac68362b205c4156884. Mar 24 12:40:16.329425 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'system journal clear'. Mar 24 12:40:16.535347 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'system coredump delete all'. Mar 24 12:40:16.754017 osdx OSDxCLI[249431]: User 'admin' entered the configuration menu. Mar 24 12:40:16.830665 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 24 12:40:16.914332 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 24 12:40:16.970205 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 24 12:40:17.068341 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 24 12:40:17.127742 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 24 12:40:17.224337 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 24 12:40:17.292898 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'show working'. Mar 24 12:40:17.392357 osdx ubnt-cfgd[249884]: inactive Mar 24 12:40:17.424847 osdx INFO[249894]: FRR daemons did not change Mar 24 12:40:17.442023 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 24 12:40:17.510025 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 24 12:40:17.531952 osdx (udev-worker)[250016]: Network interface NamePolicy= disabled on kernel command line. Mar 24 12:40:17.668267 osdx cfgd[1472]: [249431]Completed change to active configuration Mar 24 12:40:17.679787 osdx OSDxCLI[249431]: User 'admin' committed the configuration. Mar 24 12:40:17.703785 osdx OSDxCLI[249431]: User 'admin' left the configuration menu. Mar 24 12:40:19.711127 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 24 12:40:19.790401 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 24 12:40:19.868231 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 24 12:40:20.348880 osdx file_operation[250181]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 24 12:40:20.374230 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 24 12:40:20.547401 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 24 12:40:20.709511 osdx OSDxCLI[249431]: User 'admin' entered the configuration menu. Mar 24 12:40:20.797737 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 24 12:40:20.908124 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 24 12:40:20.975970 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 24 12:40:21.084363 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 24 12:40:21.152352 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 24 12:40:21.255645 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 24 12:40:21.342895 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 24 12:40:21.423342 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 24 12:40:21.484747 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 24 12:40:21.585145 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 24 12:40:21.645572 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 24 12:40:21.753275 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 24 12:40:21.818652 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 24 12:40:21.919285 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 24 12:40:21.990315 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 24 12:40:22.101864 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 24 12:40:22.157494 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 24 12:40:22.274357 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 24 12:40:22.374156 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 24 12:40:22.439025 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 24 12:40:22.556585 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'show working'. Mar 24 12:40:22.642158 osdx ubnt-cfgd[250232]: inactive Mar 24 12:40:22.702357 osdx INFO[250269]: FRR daemons did not change Mar 24 12:40:23.160235 osdx systemd[1]: Reloading. Mar 24 12:40:23.206030 osdx systemd-sysv-generator[250329]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 24 12:40:23.326414 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 24 12:40:23.343851 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 24 12:40:23.540003 osdx INFO[250311]: Rules successfully loaded Mar 24 12:40:23.540789 osdx cfgd[1472]: [249431]Completed change to active configuration Mar 24 12:40:23.543045 osdx OSDxCLI[249431]: User 'admin' committed the configuration. Mar 24 12:40:23.593914 osdx OSDxCLI[249431]: User 'admin' left the configuration menu. Mar 24 12:40:23.982032 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=50123 DF PROTO=TCP SPT=38230 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 24 12:40:23.982114 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62076 DF PROTO=TCP SPT=38240 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 24 12:40:34.179135 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Mar 24 12:40:34.274841 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'system journal show | cat'. Mar 24 12:40:34.564237 osdx OSDxCLI[249431]: User 'admin' entered the configuration menu. Mar 24 12:40:34.669789 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 24 12:40:34.882444 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 24 12:40:34.942525 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 24 12:40:35.044241 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 24 12:40:35.113616 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 24 12:40:35.233841 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 24 12:40:35.340582 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 24 12:40:35.441788 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 24 12:40:35.510474 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 24 12:40:35.642138 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 24 12:40:35.709511 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 24 12:40:35.820732 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 24 12:40:35.888993 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 24 12:40:35.989640 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 24 12:40:36.042282 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 24 12:40:36.140783 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 24 12:40:36.196103 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 24 12:40:36.295043 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 24 12:40:36.372399 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 24 12:40:36.469479 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 24 12:40:36.541492 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Mar 24 12:40:36.647868 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Mar 24 12:40:36.723196 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Mar 24 12:40:36.828513 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Mar 24 12:40:36.892807 osdx OSDxCLI[249431]: User 'admin' added a new cfg line: 'show changes'. Mar 24 12:40:37.009955 osdx ubnt-cfgd[250455]: inactive Mar 24 12:40:37.475073 osdx INFO[250471]: FRR daemons did not change Mar 24 12:40:37.702110 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Mar 24 12:40:39.319626 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Mar 24 12:40:39.319766 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Mar 24 12:40:39.319806 osdx systemd[1]: suricata@FW.service: Consumed 1.666s CPU time. Mar 24 12:40:39.338375 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 24 12:40:39.355569 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 24 12:40:39.551472 osdx INFO[250496]: Rules successfully loaded Mar 24 12:40:39.552393 osdx cfgd[1472]: [249431]Completed change to active configuration Mar 24 12:40:39.554617 osdx OSDxCLI[249431]: User 'admin' committed the configuration. Mar 24 12:40:39.589854 osdx OSDxCLI[249431]: User 'admin' left the configuration menu. Mar 24 12:40:39.794028 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=27694 DF PROTO=TCP SPT=47656 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 24 12:40:39.794097 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=38999 DF PROTO=TCP SPT=47658 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 24 12:40:46.032121 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Mar 24 12:40:49.984962 osdx OSDxCLI[249431]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23973 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.504 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.504/0.504/0.504/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.481 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.481/0.481/0.481/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 43794 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 359 MBytes 3.01 Gbits/sec 14 1.92 MBytes [ 5] 1.00-2.00 sec 348 MBytes 2.91 Gbits/sec 0 2.08 MBytes [ 5] 2.00-3.00 sec 321 MBytes 2.70 Gbits/sec 0 2.21 MBytes [ 5] 3.00-4.00 sec 299 MBytes 2.51 Gbits/sec 1 1.66 MBytes [ 5] 4.00-5.00 sec 326 MBytes 2.74 Gbits/sec 0 1.79 MBytes [ 5] 5.00-6.00 sec 328 MBytes 2.75 Gbits/sec 0 1.92 MBytes [ 5] 6.00-7.00 sec 312 MBytes 2.62 Gbits/sec 0 2.03 MBytes [ 5] 7.00-8.00 sec 314 MBytes 2.63 Gbits/sec 0 2.14 MBytes [ 5] 8.00-9.00 sec 318 MBytes 2.66 Gbits/sec 309 1.65 MBytes [ 5] 9.00-10.00 sec 352 MBytes 2.96 Gbits/sec 28 1.27 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.20 GBytes 2.75 Gbits/sec 352 sender [ 5] 0.00-10.00 sec 3.20 GBytes 2.74 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:41:13.362076 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43778 -> 40.0.0.2:5001 03/24/2026-12:41:13.363072 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43794 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=81 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=81 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=43794 dport=5001 packets=2371276 bytes=3556901941 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43794 packets=344601 bytes=17924372 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=498 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=498 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=43778 dport=5001 packets=16 bytes=1302 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43778 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 42140 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 323 MBytes 2.71 Gbits/sec 53 1.93 MBytes [ 5] 1.00-2.00 sec 336 MBytes 2.82 Gbits/sec 0 2.08 MBytes [ 5] 2.00-3.00 sec 336 MBytes 2.82 Gbits/sec 36 1.59 MBytes [ 5] 3.00-4.00 sec 390 MBytes 3.27 Gbits/sec 0 1.76 MBytes [ 5] 4.00-5.00 sec 408 MBytes 3.42 Gbits/sec 0 1.92 MBytes [ 5] 5.00-6.00 sec 365 MBytes 3.06 Gbits/sec 227 1.46 MBytes [ 5] 6.00-7.00 sec 312 MBytes 2.62 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 390 MBytes 3.27 Gbits/sec 0 1.77 MBytes [ 5] 8.00-9.00 sec 308 MBytes 2.58 Gbits/sec 0 1.88 MBytes [ 5] 9.00-10.00 sec 326 MBytes 2.74 Gbits/sec 132 1.45 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.41 GBytes 2.93 Gbits/sec 448 sender [ 5] 0.00-10.00 sec 3.41 GBytes 2.93 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:41:13.362076 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43778 -> 40.0.0.2:5001 03/24/2026-12:41:13.363072 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43794 -> 40.0.0.2:5001 03/24/2026-12:41:28.246846 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42138 -> 40.0.0.2:5001 03/24/2026-12:41:28.247963 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42140 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=42138 dport=5001 packets=16 bytes=1302 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42138 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=42140 dport=5001 packets=2530410 bytes=3795603717 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42140 packets=278312 bytes=14464836 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 43280 0 --:--:-- --:--:-- --:--:-- 50000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.600 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.600/0.600/0.600/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.433 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.433/0.433/0.433/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/24/2026-12:42:03.271514 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58712 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=501 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=501 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=84 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=84 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=58712 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=58712 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/24/2026-12:42:03.271514 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58712 -> 40.0.0.2:5000 03/24/2026-12:42:09.656435 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58712 -> 40.0.0.2:5000 03/24/2026-12:42:11.133195 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39508 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=39508 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=39508 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 32958 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.544 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.544/0.544/0.544/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.460 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.460/0.460/0.460/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/24/2026-12:42:34.738418 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44186 -> 40.0.0.2:5001 03/24/2026-12:42:34.739306 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44200 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=44200 dport=5001 packets=58370 bytes=87546357 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44200 packets=9292 bytes=483880 [ASSURED] [OFFLOAD, packets=58359 bytes=87534156 packets=9290 bytes=483768] mark=129834765 use=2 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=504 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=504 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=87 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=87 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=44186 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44186 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 27087 0 --:--:-- --:--:-- --:--:-- 33250
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
03/24/2026-12:42:34.738418 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44186 -> 40.0.0.2:5001 03/24/2026-12:42:34.739306 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44200 -> 40.0.0.2:5001 03/24/2026-12:42:36.752155 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44208 -> 40.0.0.2:5001 03/24/2026-12:42:36.753450 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52493 -> 40.0.0.2:5001 03/24/2026-12:42:36.764229 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52493 -> 40.0.0.2:5001 03/24/2026-12:42:36.776211 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52493 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
udp 17 src=20.0.0.2 dst=40.0.0.2 sport=52493 dport=5001 packets=16 bytes=22172 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52493 packets=1 bytes=32 [OFFLOAD, packets=12 bytes=17712 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=44208 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44208 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 50050 0 --:--:-- --:--:-- --:--:-- 66666
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.521 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.521/0.521/0.521/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.503 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.503/0.503/0.503/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/24/2026-12:42:55.045923 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52310 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 52310 5000 yes 201 0 8 673 40.0.0.2 20.0.0.2 5000 52310 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250121s ending at 1774356178.761068 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
03/24/2026-12:42:55.045923 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52310 -> 40.0.0.2:5000 03/24/2026-12:42:58.909257 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:39941 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5001 39941 no 201 0 0 0 20.0.0.2 40.0.0.2 39941 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 52310 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5000 52310 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250101s ending at 1774356182.625559 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)