Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23428 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.535 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.535/0.535/0.535/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.490 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.490/0.490/0.490/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57354 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 55.6 MBytes 467 Mbits/sec 0 2.53 MBytes [ 5] 1.00-2.00 sec 57.5 MBytes 482 Mbits/sec 76 1.35 MBytes [ 5] 2.00-3.00 sec 56.2 MBytes 472 Mbits/sec 0 1.44 MBytes [ 5] 3.00-4.00 sec 105 MBytes 881 Mbits/sec 0 1.51 MBytes [ 5] 4.00-5.00 sec 97.5 MBytes 818 Mbits/sec 0 1.56 MBytes [ 5] 5.00-6.00 sec 96.2 MBytes 807 Mbits/sec 0 1.59 MBytes [ 5] 6.00-7.00 sec 95.0 MBytes 797 Mbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 97.5 MBytes 818 Mbits/sec 0 1.62 MBytes [ 5] 8.00-9.00 sec 98.8 MBytes 828 Mbits/sec 0 1.64 MBytes [ 5] 9.00-10.00 sec 97.5 MBytes 818 Mbits/sec 0 1.68 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 857 MBytes 719 Mbits/sec 76 sender [ 5] 0.00-10.01 sec 855 MBytes 716 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:26:28.554804 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57342 -> 40.0.0.2:5001 11/25/2025-11:26:28.556282 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57354 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 3515 0 --:--:-- --:--:-- --:--:-- 3583
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.618 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.618/0.618/0.618/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.431 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.431/0.431/0.431/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57988 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 163 MBytes 1.37 Gbits/sec 314 1.28 MBytes [ 5] 1.00-2.00 sec 284 MBytes 2.38 Gbits/sec 0 1.43 MBytes [ 5] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 1.57 MBytes [ 5] 3.00-4.00 sec 264 MBytes 2.21 Gbits/sec 99 1.23 MBytes [ 5] 4.00-5.00 sec 296 MBytes 2.49 Gbits/sec 0 1.39 MBytes [ 5] 5.00-6.00 sec 311 MBytes 2.61 Gbits/sec 0 1.55 MBytes [ 5] 6.00-7.00 sec 285 MBytes 2.39 Gbits/sec 0 1.68 MBytes [ 5] 7.00-8.00 sec 278 MBytes 2.33 Gbits/sec 0 1.79 MBytes [ 5] 8.00-9.00 sec 348 MBytes 2.92 Gbits/sec 0 1.92 MBytes [ 5] 9.00-10.00 sec 300 MBytes 2.52 Gbits/sec 0 2.03 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.74 GBytes 2.36 Gbits/sec 413 sender [ 5] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:27:02.450933 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57984 -> 40.0.0.2:5001 11/25/2025-11:27:02.452019 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57988 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Nov 25 11:26:53.359753 osdx systemd-journald[1857]: Runtime Journal (/run/log/journal/3dabd6c33b8e497d9fdbc41689ab0c32) is 1.7M, max 13.8M, 12.0M free. Nov 25 11:26:53.361841 osdx systemd-journald[1857]: Received client request to rotate journal, rotating. Nov 25 11:26:53.361888 osdx systemd-journald[1857]: Vacuuming done, freed 0B of archived journals from /run/log/journal/3dabd6c33b8e497d9fdbc41689ab0c32. Nov 25 11:26:53.369941 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'system journal clear'. Nov 25 11:26:53.585159 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'system coredump delete all'. Nov 25 11:26:53.814498 osdx OSDxCLI[289853]: User 'admin' entered the configuration menu. Nov 25 11:26:53.896886 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Nov 25 11:26:53.985487 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Nov 25 11:26:54.042259 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Nov 25 11:26:54.142397 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Nov 25 11:26:54.205202 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Nov 25 11:26:54.297938 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Nov 25 11:26:54.365219 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'show working'. Nov 25 11:26:54.463724 osdx ubnt-cfgd[290300]: inactive Nov 25 11:26:54.512944 osdx INFO[290310]: FRR daemons did not change Nov 25 11:26:54.533811 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Nov 25 11:26:54.597820 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Nov 25 11:26:54.619734 osdx (udev-worker)[290432]: Network interface NamePolicy= disabled on kernel command line. Nov 25 11:26:54.662644 osdx (udev-worker)[290438]: Network interface NamePolicy= disabled on kernel command line. Nov 25 11:26:54.772315 osdx cfgd[1655]: [289853]Completed change to active configuration Nov 25 11:26:54.786672 osdx OSDxCLI[289853]: User 'admin' committed the configuration. Nov 25 11:26:54.803618 osdx OSDxCLI[289853]: User 'admin' left the configuration menu. Nov 25 11:26:58.176417 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Nov 25 11:26:58.254684 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Nov 25 11:26:58.375715 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Nov 25 11:26:58.893007 osdx file_operation[290597]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Nov 25 11:26:58.950917 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Nov 25 11:26:59.124362 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Nov 25 11:26:59.267004 osdx OSDxCLI[289853]: User 'admin' entered the configuration menu. Nov 25 11:26:59.331606 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Nov 25 11:26:59.432158 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Nov 25 11:26:59.496812 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Nov 25 11:26:59.587790 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Nov 25 11:26:59.652107 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Nov 25 11:26:59.751007 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Nov 25 11:26:59.841259 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Nov 25 11:26:59.969532 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Nov 25 11:27:00.034461 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Nov 25 11:27:00.160714 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Nov 25 11:27:00.222424 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Nov 25 11:27:00.333191 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Nov 25 11:27:00.396709 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Nov 25 11:27:00.495778 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Nov 25 11:27:00.555420 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Nov 25 11:27:00.671699 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Nov 25 11:27:00.728688 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Nov 25 11:27:00.826253 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Nov 25 11:27:00.909891 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Nov 25 11:27:01.018576 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Nov 25 11:27:01.120972 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'show working'. Nov 25 11:27:01.250942 osdx ubnt-cfgd[290648]: inactive Nov 25 11:27:01.317934 osdx INFO[290685]: FRR daemons did not change Nov 25 11:27:01.591255 osdx systemd[1]: Reloading. Nov 25 11:27:01.633794 osdx systemd-sysv-generator[290738]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Nov 25 11:27:01.786080 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Nov 25 11:27:01.809740 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Nov 25 11:27:02.002443 osdx INFO[290720]: Rules successfully loaded Nov 25 11:27:02.003173 osdx cfgd[1655]: [289853]Completed change to active configuration Nov 25 11:27:02.005482 osdx OSDxCLI[289853]: User 'admin' committed the configuration. Nov 25 11:27:02.052421 osdx OSDxCLI[289853]: User 'admin' left the configuration menu. Nov 25 11:27:02.453814 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=17066 DF PROTO=TCP SPT=57984 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Nov 25 11:27:02.453901 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=47509 DF PROTO=TCP SPT=57988 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Nov 25 11:27:12.642187 osdx systemd[1]: Starting logrotate.service - Rotate log files... Nov 25 11:27:12.666353 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Nov 25 11:27:12.668916 osdx systemd[1]: logrotate.service: Deactivated successfully. Nov 25 11:27:12.669032 osdx systemd[1]: Finished logrotate.service - Rotate log files.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 40530 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 198 MBytes 1.66 Gbits/sec 103 1.31 MBytes [ 5] 1.00-2.00 sec 262 MBytes 2.20 Gbits/sec 0 1.45 MBytes [ 5] 2.00-3.00 sec 282 MBytes 2.37 Gbits/sec 0 1.58 MBytes [ 5] 3.00-4.00 sec 290 MBytes 2.43 Gbits/sec 0 1.71 MBytes [ 5] 4.00-5.00 sec 272 MBytes 2.29 Gbits/sec 0 1.82 MBytes [ 5] 5.00-6.00 sec 289 MBytes 2.42 Gbits/sec 0 1.91 MBytes [ 5] 6.00-7.00 sec 266 MBytes 2.23 Gbits/sec 0 2.01 MBytes [ 5] 7.00-8.00 sec 281 MBytes 2.36 Gbits/sec 0 2.10 MBytes [ 5] 8.00-9.00 sec 271 MBytes 2.28 Gbits/sec 0 2.19 MBytes [ 5] 9.00-10.00 sec 281 MBytes 2.36 Gbits/sec 40 1.70 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec 143 sender [ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:27:02.450933 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57984 -> 40.0.0.2:5001 11/25/2025-11:27:02.452019 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57988 -> 40.0.0.2:5001 11/25/2025-11:27:17.315401 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40526 -> 40.0.0.2:5001 11/25/2025-11:27:17.316579 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40530 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Nov 25 11:26:53.359753 osdx systemd-journald[1857]: Runtime Journal (/run/log/journal/3dabd6c33b8e497d9fdbc41689ab0c32) is 1.7M, max 13.8M, 12.0M free. Nov 25 11:26:53.361841 osdx systemd-journald[1857]: Received client request to rotate journal, rotating. Nov 25 11:26:53.361888 osdx systemd-journald[1857]: Vacuuming done, freed 0B of archived journals from /run/log/journal/3dabd6c33b8e497d9fdbc41689ab0c32. Nov 25 11:26:53.369941 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'system journal clear'. Nov 25 11:26:53.585159 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'system coredump delete all'. Nov 25 11:26:53.814498 osdx OSDxCLI[289853]: User 'admin' entered the configuration menu. Nov 25 11:26:53.896886 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Nov 25 11:26:53.985487 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Nov 25 11:26:54.042259 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Nov 25 11:26:54.142397 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Nov 25 11:26:54.205202 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Nov 25 11:26:54.297938 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Nov 25 11:26:54.365219 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'show working'. Nov 25 11:26:54.463724 osdx ubnt-cfgd[290300]: inactive Nov 25 11:26:54.512944 osdx INFO[290310]: FRR daemons did not change Nov 25 11:26:54.533811 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Nov 25 11:26:54.597820 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Nov 25 11:26:54.619734 osdx (udev-worker)[290432]: Network interface NamePolicy= disabled on kernel command line. Nov 25 11:26:54.662644 osdx (udev-worker)[290438]: Network interface NamePolicy= disabled on kernel command line. Nov 25 11:26:54.772315 osdx cfgd[1655]: [289853]Completed change to active configuration Nov 25 11:26:54.786672 osdx OSDxCLI[289853]: User 'admin' committed the configuration. Nov 25 11:26:54.803618 osdx OSDxCLI[289853]: User 'admin' left the configuration menu. Nov 25 11:26:58.176417 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Nov 25 11:26:58.254684 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Nov 25 11:26:58.375715 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Nov 25 11:26:58.893007 osdx file_operation[290597]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Nov 25 11:26:58.950917 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Nov 25 11:26:59.124362 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Nov 25 11:26:59.267004 osdx OSDxCLI[289853]: User 'admin' entered the configuration menu. Nov 25 11:26:59.331606 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Nov 25 11:26:59.432158 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Nov 25 11:26:59.496812 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Nov 25 11:26:59.587790 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Nov 25 11:26:59.652107 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Nov 25 11:26:59.751007 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Nov 25 11:26:59.841259 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Nov 25 11:26:59.969532 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Nov 25 11:27:00.034461 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Nov 25 11:27:00.160714 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Nov 25 11:27:00.222424 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Nov 25 11:27:00.333191 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Nov 25 11:27:00.396709 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Nov 25 11:27:00.495778 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Nov 25 11:27:00.555420 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Nov 25 11:27:00.671699 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Nov 25 11:27:00.728688 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Nov 25 11:27:00.826253 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Nov 25 11:27:00.909891 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Nov 25 11:27:01.018576 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Nov 25 11:27:01.120972 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'show working'. Nov 25 11:27:01.250942 osdx ubnt-cfgd[290648]: inactive Nov 25 11:27:01.317934 osdx INFO[290685]: FRR daemons did not change Nov 25 11:27:01.591255 osdx systemd[1]: Reloading. Nov 25 11:27:01.633794 osdx systemd-sysv-generator[290738]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Nov 25 11:27:01.786080 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Nov 25 11:27:01.809740 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Nov 25 11:27:02.002443 osdx INFO[290720]: Rules successfully loaded Nov 25 11:27:02.003173 osdx cfgd[1655]: [289853]Completed change to active configuration Nov 25 11:27:02.005482 osdx OSDxCLI[289853]: User 'admin' committed the configuration. Nov 25 11:27:02.052421 osdx OSDxCLI[289853]: User 'admin' left the configuration menu. Nov 25 11:27:02.453814 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=17066 DF PROTO=TCP SPT=57984 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Nov 25 11:27:02.453901 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=47509 DF PROTO=TCP SPT=57988 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Nov 25 11:27:12.642187 osdx systemd[1]: Starting logrotate.service - Rotate log files... Nov 25 11:27:12.666353 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Nov 25 11:27:12.668916 osdx systemd[1]: logrotate.service: Deactivated successfully. Nov 25 11:27:12.669032 osdx systemd[1]: Finished logrotate.service - Rotate log files. Nov 25 11:27:12.814764 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'system journal show | cat'. Nov 25 11:27:12.980444 osdx OSDxCLI[289853]: User 'admin' entered the configuration menu. Nov 25 11:27:13.043634 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Nov 25 11:27:13.149888 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Nov 25 11:27:13.215532 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Nov 25 11:27:13.326134 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Nov 25 11:27:13.400496 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Nov 25 11:27:13.622017 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Nov 25 11:27:13.837955 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Nov 25 11:27:13.927556 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Nov 25 11:27:13.984788 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Nov 25 11:27:14.097587 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Nov 25 11:27:14.166887 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Nov 25 11:27:14.228756 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Nov 25 11:27:14.321167 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Nov 25 11:27:14.383805 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Nov 25 11:27:14.480604 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Nov 25 11:27:14.540263 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Nov 25 11:27:14.652219 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Nov 25 11:27:14.722350 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Nov 25 11:27:14.820202 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Nov 25 11:27:14.883195 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Nov 25 11:27:14.977817 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Nov 25 11:27:15.042413 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Nov 25 11:27:15.156182 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Nov 25 11:27:15.221572 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Nov 25 11:27:15.333463 osdx OSDxCLI[289853]: User 'admin' added a new cfg line: 'show changes'. Nov 25 11:27:15.407579 osdx ubnt-cfgd[290867]: inactive Nov 25 11:27:15.459753 osdx INFO[290883]: FRR daemons did not change Nov 25 11:27:15.702668 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Nov 25 11:27:16.811860 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Nov 25 11:27:16.811999 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Nov 25 11:27:16.812039 osdx systemd[1]: suricata@FW.service: Consumed 1.101s CPU time. Nov 25 11:27:16.842200 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Nov 25 11:27:16.866415 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Nov 25 11:27:17.065741 osdx INFO[290908]: Rules successfully loaded Nov 25 11:27:17.066485 osdx cfgd[1655]: [289853]Completed change to active configuration Nov 25 11:27:17.068358 osdx OSDxCLI[289853]: User 'admin' committed the configuration. Nov 25 11:27:17.093085 osdx OSDxCLI[289853]: User 'admin' left the configuration menu. Nov 25 11:27:17.317799 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=32386 DF PROTO=TCP SPT=40526 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Nov 25 11:27:17.317884 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=39193 DF PROTO=TCP SPT=40530 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Nov 25 11:27:23.032277 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Nov 25 11:27:27.520562 osdx OSDxCLI[289853]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23791 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.693 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.693/0.693/0.693/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.495 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.495/0.495/0.495/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51648 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 65.9 MBytes 553 Mbits/sec 114 1.69 MBytes [ 5] 1.00-2.00 sec 149 MBytes 1.25 Gbits/sec 193 1.29 MBytes [ 5] 2.00-3.00 sec 261 MBytes 2.19 Gbits/sec 0 1.42 MBytes [ 5] 3.00-4.00 sec 286 MBytes 2.40 Gbits/sec 0 1.55 MBytes [ 5] 4.00-5.00 sec 258 MBytes 2.16 Gbits/sec 0 1.66 MBytes [ 5] 5.00-6.00 sec 276 MBytes 2.32 Gbits/sec 0 1.77 MBytes [ 5] 6.00-7.00 sec 268 MBytes 2.24 Gbits/sec 17 1.37 MBytes [ 5] 7.00-8.00 sec 272 MBytes 2.29 Gbits/sec 0 1.50 MBytes [ 5] 8.00-9.00 sec 321 MBytes 2.69 Gbits/sec 0 1.62 MBytes [ 5] 9.00-10.00 sec 299 MBytes 2.51 Gbits/sec 0 1.73 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.40 GBytes 2.06 Gbits/sec 324 sender [ 5] 0.00-10.00 sec 2.40 GBytes 2.06 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:27:49.245624 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51636 -> 40.0.0.2:5001 11/25/2025-11:27:49.246787 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51648 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=143 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=143 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=51648 dport=5001 packets=1778651 bytes=2667965561 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51648 packets=200502 bytes=10419944 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=51636 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51636 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=19 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=19 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 44666 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 290 MBytes 2.43 Gbits/sec 79 1.92 MBytes [ 5] 1.00-2.00 sec 340 MBytes 2.85 Gbits/sec 0 2.08 MBytes [ 5] 2.00-3.00 sec 278 MBytes 2.33 Gbits/sec 105 1.55 MBytes [ 5] 3.00-4.00 sec 281 MBytes 2.36 Gbits/sec 0 1.68 MBytes [ 5] 4.00-5.00 sec 301 MBytes 2.53 Gbits/sec 0 1.80 MBytes [ 5] 5.00-6.00 sec 340 MBytes 2.85 Gbits/sec 0 1.93 MBytes [ 5] 6.00-7.00 sec 258 MBytes 2.16 Gbits/sec 215 1.46 MBytes [ 5] 7.00-8.00 sec 285 MBytes 2.39 Gbits/sec 0 1.60 MBytes [ 5] 8.00-9.00 sec 265 MBytes 2.22 Gbits/sec 0 1.72 MBytes [ 5] 9.00-10.00 sec 276 MBytes 2.32 Gbits/sec 37 1.29 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.85 GBytes 2.44 Gbits/sec 436 sender [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:27:49.245624 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51636 -> 40.0.0.2:5001 11/25/2025-11:27:49.246787 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51648 -> 40.0.0.2:5001 11/25/2025-11:28:03.057277 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44656 -> 40.0.0.2:5001 11/25/2025-11:28:03.058216 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44666 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=44666 dport=5001 packets=2109933 bytes=3164887817 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44666 packets=283707 bytes=14748864 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=44656 dport=5001 packets=16 bytes=1297 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44656 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 1489 0 --:--:-- --:--:-- --:--:-- 1492
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.544 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.544/0.544/0.544/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.490 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.490/0.490/0.490/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
11/25/2025-11:28:35.726090 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42664 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=22 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=22 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=42664 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=42664 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=146 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=146 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
11/25/2025-11:28:35.726090 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42664 -> 40.0.0.2:5000 11/25/2025-11:28:41.973682 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42664 -> 40.0.0.2:5000 11/25/2025-11:28:43.459288 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42670 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=42670 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=42670 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 21329 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.528 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.528/0.528/0.528/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.441 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.441/0.441/0.441/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
11/25/2025-11:29:05.979813 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48088 -> 40.0.0.2:5001 11/25/2025-11:29:05.981523 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48100 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48088 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48088 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48100 dport=5001 packets=13791 bytes=20680753 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48100 packets=1954 bytes=101616 [ASSURED] [OFFLOAD, packets=13778 bytes=20665552 packets=1952 bytes=101504] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=25 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=25 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=149 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=149 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 11340 0 --:--:-- --:--:-- --:--:-- 12090
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
11/25/2025-11:29:05.979813 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48088 -> 40.0.0.2:5001 11/25/2025-11:29:05.981523 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48100 -> 40.0.0.2:5001 11/25/2025-11:29:07.974112 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48112 -> 40.0.0.2:5001 11/25/2025-11:29:07.975696 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:42562 -> 40.0.0.2:5001 11/25/2025-11:29:07.986370 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:42562 -> 40.0.0.2:5001 11/25/2025-11:29:07.998405 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:42562 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
udp 17 src=20.0.0.2 dst=40.0.0.2 sport=42562 dport=5001 packets=17 bytes=23648 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42562 packets=1 bytes=32 [OFFLOAD, packets=13 bytes=19188 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48112 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48112 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 69013 0 --:--:-- --:--:-- --:--:-- 97k
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.655 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.655/0.655/0.655/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.681 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.681/0.681/0.681/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
11/25/2025-11:29:28.264950 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33656 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 33656 yes 201 0 0 0 20.0.0.2 40.0.0.2 33656 5000 yes 201 0 9 731
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250139s ending at 1764070171.963315 XDP_DROP 10 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9701492 pkts ( 8 pps) 12701679 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
11/25/2025-11:29:28.264950 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33656 -> 40.0.0.2:5000 11/25/2025-11:29:32.114588 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:48984 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 33656 yes 201 0 0 0 20.0.0.2 40.0.0.2 48984 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5001 48984 no 201 0 0 0 20.0.0.2 40.0.0.2 33656 5000 yes 201 0 12 905
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250123s ending at 1764070175.789796 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9701530 pkts ( 0 pps) 12701682 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)