Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 8819 0 --:--:-- --:--:-- --:--:-- 9214
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.456 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.456/0.456/0.456/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.581 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.581/0.581/0.581/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 34474 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 75.4 MBytes 632 Mbits/sec 38 1.72 MBytes [ 5] 1.00-2.00 sec 58.8 MBytes 493 Mbits/sec 0 1.90 MBytes [ 5] 2.00-3.00 sec 80.0 MBytes 671 Mbits/sec 0 2.05 MBytes [ 5] 3.00-4.00 sec 105 MBytes 881 Mbits/sec 159 1.54 MBytes [ 5] 4.00-5.00 sec 101 MBytes 849 Mbits/sec 0 1.62 MBytes [ 5] 5.00-6.00 sec 98.8 MBytes 828 Mbits/sec 0 1.67 MBytes [ 5] 6.00-7.00 sec 96.2 MBytes 807 Mbits/sec 0 1.71 MBytes [ 5] 7.00-8.00 sec 70.0 MBytes 587 Mbits/sec 0 1.74 MBytes [ 5] 8.00-9.00 sec 52.5 MBytes 440 Mbits/sec 0 1.75 MBytes [ 5] 9.00-10.00 sec 50.0 MBytes 419 Mbits/sec 0 1.76 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 788 MBytes 661 Mbits/sec 197 sender [ 5] 0.00-10.01 sec 787 MBytes 659 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:26:47.207560 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34470 -> 40.0.0.2:5001 08/20/2025-11:26:47.208557 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34474 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 27558 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.658 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.658/0.658/0.658/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.529 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.529/0.529/0.529/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59568 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 80.0 MBytes 671 Mbits/sec 9 1.72 MBytes [ 5] 1.00-2.00 sec 120 MBytes 1.01 Gbits/sec 0 1.93 MBytes [ 5] 2.00-3.00 sec 272 MBytes 2.28 Gbits/sec 0 2.07 MBytes [ 5] 3.00-4.00 sec 280 MBytes 2.35 Gbits/sec 0 2.19 MBytes [ 5] 4.00-5.00 sec 276 MBytes 2.32 Gbits/sec 17 1.65 MBytes [ 5] 5.00-6.00 sec 288 MBytes 2.41 Gbits/sec 148 1.31 MBytes [ 5] 6.00-7.00 sec 294 MBytes 2.46 Gbits/sec 0 1.47 MBytes [ 5] 7.00-8.00 sec 274 MBytes 2.30 Gbits/sec 0 1.59 MBytes [ 5] 8.00-9.00 sec 299 MBytes 2.51 Gbits/sec 66 1.27 MBytes [ 5] 9.00-10.00 sec 326 MBytes 2.74 Gbits/sec 0 1.42 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.45 GBytes 2.10 Gbits/sec 240 sender [ 5] 0.00-10.00 sec 2.45 GBytes 2.10 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:27:23.480971 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59562 -> 40.0.0.2:5001 08/20/2025-11:27:23.481850 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59568 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Aug 20 11:27:15.001225 osdx systemd-timedated[282844]: Changed local time to Wed 2025-08-20 11:27:15 UTC Aug 20 11:27:15.002606 osdx systemd-journald[1956]: Time jumped backwards, rotating. Aug 20 11:27:15.003046 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'set date 2025-08-20 11:27:15'. Aug 20 11:27:15.330587 osdx systemd-journald[1956]: Runtime Journal (/run/log/journal/5531ee11c77d43acb462311bc7f53dec) is 2.1M, max 15.3M, 13.2M free. Aug 20 11:27:15.334608 osdx systemd-journald[1956]: Received client request to rotate journal, rotating. Aug 20 11:27:15.334661 osdx systemd-journald[1956]: Vacuuming done, freed 0B of archived journals from /run/log/journal/5531ee11c77d43acb462311bc7f53dec. Aug 20 11:27:15.340699 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'system journal clear'. Aug 20 11:27:15.566729 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'system coredump delete all'. Aug 20 11:27:15.854104 osdx OSDxCLI[282419]: User 'admin' entered the configuration menu. Aug 20 11:27:15.948264 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Aug 20 11:27:16.058316 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Aug 20 11:27:16.127155 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Aug 20 11:27:16.200248 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Aug 20 11:27:16.382008 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Aug 20 11:27:16.437267 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Aug 20 11:27:16.558112 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'show working'. Aug 20 11:27:16.625351 osdx ubnt-cfgd[282876]: inactive Aug 20 11:27:16.657557 osdx INFO[282886]: FRR daemons did not change Aug 20 11:27:16.678619 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Aug 20 11:27:16.742614 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Aug 20 11:27:16.763025 osdx (udev-worker)[283006]: Network interface NamePolicy= disabled on kernel command line. Aug 20 11:27:16.806458 osdx (udev-worker)[283013]: Network interface NamePolicy= disabled on kernel command line. Aug 20 11:27:16.923889 osdx cfgd[1656]: [282419]Completed change to active configuration Aug 20 11:27:16.936239 osdx OSDxCLI[282419]: User 'admin' committed the configuration. Aug 20 11:27:16.989724 osdx OSDxCLI[282419]: User 'admin' left the configuration menu. Aug 20 11:27:19.312694 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Aug 20 11:27:19.393701 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Aug 20 11:27:19.510274 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Aug 20 11:27:20.165385 osdx file_operation[283173]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Aug 20 11:27:20.193856 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Aug 20 11:27:20.342690 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Aug 20 11:27:20.484892 osdx OSDxCLI[282419]: User 'admin' entered the configuration menu. Aug 20 11:27:20.547903 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Aug 20 11:27:20.648450 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Aug 20 11:27:20.706324 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Aug 20 11:27:20.805735 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Aug 20 11:27:20.865301 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Aug 20 11:27:20.965326 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Aug 20 11:27:21.052034 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Aug 20 11:27:21.137512 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Aug 20 11:27:21.191673 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Aug 20 11:27:21.308580 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Aug 20 11:27:21.368217 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Aug 20 11:27:21.472476 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Aug 20 11:27:21.531769 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Aug 20 11:27:21.635813 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Aug 20 11:27:21.691637 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Aug 20 11:27:21.798576 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Aug 20 11:27:21.858266 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Aug 20 11:27:21.957987 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Aug 20 11:27:22.016721 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Aug 20 11:27:22.118850 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Aug 20 11:27:22.217518 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'show working'. Aug 20 11:27:22.312315 osdx ubnt-cfgd[283220]: inactive Aug 20 11:27:22.387667 osdx INFO[283257]: FRR daemons did not change Aug 20 11:27:22.657651 osdx systemd[1]: Reloading. Aug 20 11:27:22.706609 osdx systemd-sysv-generator[283310]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Aug 20 11:27:22.818893 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Aug 20 11:27:22.838242 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Aug 20 11:27:23.021564 osdx INFO[283292]: Rules successfully loaded Aug 20 11:27:23.022991 osdx cfgd[1656]: [282419]Completed change to active configuration Aug 20 11:27:23.025065 osdx OSDxCLI[282419]: User 'admin' committed the configuration. Aug 20 11:27:23.053361 osdx OSDxCLI[282419]: User 'admin' left the configuration menu. Aug 20 11:27:23.482627 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=56543 DF PROTO=TCP SPT=59562 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Aug 20 11:27:23.482693 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19334 DF PROTO=TCP SPT=59568 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Aug 20 11:27:33.782069 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 56998 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 206 MBytes 1.73 Gbits/sec 6 1.31 MBytes [ 5] 1.00-2.00 sec 281 MBytes 2.36 Gbits/sec 0 1.46 MBytes [ 5] 2.00-3.00 sec 265 MBytes 2.22 Gbits/sec 0 1.58 MBytes [ 5] 3.00-4.00 sec 261 MBytes 2.19 Gbits/sec 0 1.69 MBytes [ 5] 4.00-5.00 sec 271 MBytes 2.28 Gbits/sec 0 1.80 MBytes [ 5] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec 0 1.90 MBytes [ 5] 6.00-7.00 sec 298 MBytes 2.50 Gbits/sec 0 2.02 MBytes [ 5] 7.00-8.00 sec 275 MBytes 2.31 Gbits/sec 0 2.11 MBytes [ 5] 8.00-9.00 sec 278 MBytes 2.33 Gbits/sec 2 1.62 MBytes [ 5] 9.00-10.00 sec 254 MBytes 2.13 Gbits/sec 0 1.76 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.60 GBytes 2.24 Gbits/sec 8 sender [ 5] 0.00-10.00 sec 2.60 GBytes 2.24 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:27:23.480971 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59562 -> 40.0.0.2:5001 08/20/2025-11:27:23.481850 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59568 -> 40.0.0.2:5001 08/20/2025-11:27:39.312607 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56982 -> 40.0.0.2:5001 08/20/2025-11:27:39.313555 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56998 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Aug 20 11:27:15.001225 osdx systemd-timedated[282844]: Changed local time to Wed 2025-08-20 11:27:15 UTC Aug 20 11:27:15.002606 osdx systemd-journald[1956]: Time jumped backwards, rotating. Aug 20 11:27:15.003046 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'set date 2025-08-20 11:27:15'. Aug 20 11:27:15.330587 osdx systemd-journald[1956]: Runtime Journal (/run/log/journal/5531ee11c77d43acb462311bc7f53dec) is 2.1M, max 15.3M, 13.2M free. Aug 20 11:27:15.334608 osdx systemd-journald[1956]: Received client request to rotate journal, rotating. Aug 20 11:27:15.334661 osdx systemd-journald[1956]: Vacuuming done, freed 0B of archived journals from /run/log/journal/5531ee11c77d43acb462311bc7f53dec. Aug 20 11:27:15.340699 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'system journal clear'. Aug 20 11:27:15.566729 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'system coredump delete all'. Aug 20 11:27:15.854104 osdx OSDxCLI[282419]: User 'admin' entered the configuration menu. Aug 20 11:27:15.948264 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Aug 20 11:27:16.058316 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Aug 20 11:27:16.127155 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Aug 20 11:27:16.200248 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Aug 20 11:27:16.382008 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Aug 20 11:27:16.437267 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Aug 20 11:27:16.558112 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'show working'. Aug 20 11:27:16.625351 osdx ubnt-cfgd[282876]: inactive Aug 20 11:27:16.657557 osdx INFO[282886]: FRR daemons did not change Aug 20 11:27:16.678619 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Aug 20 11:27:16.742614 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Aug 20 11:27:16.763025 osdx (udev-worker)[283006]: Network interface NamePolicy= disabled on kernel command line. Aug 20 11:27:16.806458 osdx (udev-worker)[283013]: Network interface NamePolicy= disabled on kernel command line. Aug 20 11:27:16.923889 osdx cfgd[1656]: [282419]Completed change to active configuration Aug 20 11:27:16.936239 osdx OSDxCLI[282419]: User 'admin' committed the configuration. Aug 20 11:27:16.989724 osdx OSDxCLI[282419]: User 'admin' left the configuration menu. Aug 20 11:27:19.312694 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Aug 20 11:27:19.393701 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Aug 20 11:27:19.510274 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Aug 20 11:27:20.165385 osdx file_operation[283173]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Aug 20 11:27:20.193856 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Aug 20 11:27:20.342690 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Aug 20 11:27:20.484892 osdx OSDxCLI[282419]: User 'admin' entered the configuration menu. Aug 20 11:27:20.547903 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Aug 20 11:27:20.648450 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Aug 20 11:27:20.706324 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Aug 20 11:27:20.805735 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Aug 20 11:27:20.865301 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Aug 20 11:27:20.965326 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Aug 20 11:27:21.052034 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Aug 20 11:27:21.137512 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Aug 20 11:27:21.191673 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Aug 20 11:27:21.308580 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Aug 20 11:27:21.368217 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Aug 20 11:27:21.472476 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Aug 20 11:27:21.531769 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Aug 20 11:27:21.635813 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Aug 20 11:27:21.691637 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Aug 20 11:27:21.798576 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Aug 20 11:27:21.858266 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Aug 20 11:27:21.957987 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Aug 20 11:27:22.016721 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Aug 20 11:27:22.118850 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Aug 20 11:27:22.217518 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'show working'. Aug 20 11:27:22.312315 osdx ubnt-cfgd[283220]: inactive Aug 20 11:27:22.387667 osdx INFO[283257]: FRR daemons did not change Aug 20 11:27:22.657651 osdx systemd[1]: Reloading. Aug 20 11:27:22.706609 osdx systemd-sysv-generator[283310]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Aug 20 11:27:22.818893 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Aug 20 11:27:22.838242 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Aug 20 11:27:23.021564 osdx INFO[283292]: Rules successfully loaded Aug 20 11:27:23.022991 osdx cfgd[1656]: [282419]Completed change to active configuration Aug 20 11:27:23.025065 osdx OSDxCLI[282419]: User 'admin' committed the configuration. Aug 20 11:27:23.053361 osdx OSDxCLI[282419]: User 'admin' left the configuration menu. Aug 20 11:27:23.482627 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=56543 DF PROTO=TCP SPT=59562 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Aug 20 11:27:23.482693 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19334 DF PROTO=TCP SPT=59568 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Aug 20 11:27:33.782069 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Aug 20 11:27:33.887382 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'system journal show | cat'. Aug 20 11:27:34.043592 osdx OSDxCLI[282419]: User 'admin' entered the configuration menu. Aug 20 11:27:34.109968 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Aug 20 11:27:34.310523 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Aug 20 11:27:34.372040 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Aug 20 11:27:34.488928 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Aug 20 11:27:34.546139 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Aug 20 11:27:34.637085 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Aug 20 11:27:34.983295 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Aug 20 11:27:35.042405 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Aug 20 11:27:35.144874 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Aug 20 11:27:35.219891 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Aug 20 11:27:35.330099 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Aug 20 11:27:35.394681 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Aug 20 11:27:35.567923 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Aug 20 11:27:35.631689 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Aug 20 11:27:35.732345 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Aug 20 11:27:35.794039 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Aug 20 11:27:35.898148 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Aug 20 11:27:35.976413 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Aug 20 11:27:36.192966 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Aug 20 11:27:36.250421 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Aug 20 11:27:36.350998 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Aug 20 11:27:36.408267 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Aug 20 11:27:36.513521 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Aug 20 11:27:36.581253 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Aug 20 11:27:36.709169 osdx OSDxCLI[282419]: User 'admin' added a new cfg line: 'show changes'. Aug 20 11:27:36.807845 osdx ubnt-cfgd[283436]: inactive Aug 20 11:27:36.857917 osdx INFO[283452]: FRR daemons did not change Aug 20 11:27:37.102940 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Aug 20 11:27:38.813822 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Aug 20 11:27:38.813923 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Aug 20 11:27:38.813957 osdx systemd[1]: suricata@FW.service: Consumed 1.666s CPU time. Aug 20 11:27:38.847000 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Aug 20 11:27:38.864205 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Aug 20 11:27:39.070180 osdx INFO[283477]: Rules successfully loaded Aug 20 11:27:39.070985 osdx cfgd[1656]: [282419]Completed change to active configuration Aug 20 11:27:39.072792 osdx OSDxCLI[282419]: User 'admin' committed the configuration. Aug 20 11:27:39.089569 osdx OSDxCLI[282419]: User 'admin' left the configuration menu. Aug 20 11:27:39.314651 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=1452 DF PROTO=TCP SPT=56982 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Aug 20 11:27:39.314723 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=39051 DF PROTO=TCP SPT=56998 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Aug 20 11:27:45.032947 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Aug 20 11:27:49.565503 osdx OSDxCLI[282419]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 1782 0 --:--:-- --:--:-- --:--:-- 1791
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.568 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.568/0.568/0.568/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.485 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.485/0.485/0.485/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51948 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 82.4 MBytes 691 Mbits/sec 47 1.77 MBytes [ 5] 1.00-2.00 sec 116 MBytes 975 Mbits/sec 0 1.94 MBytes [ 5] 2.00-3.00 sec 245 MBytes 2.06 Gbits/sec 13 1.47 MBytes [ 5] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec 0 1.58 MBytes [ 5] 4.00-5.00 sec 260 MBytes 2.18 Gbits/sec 0 1.69 MBytes [ 5] 5.00-6.00 sec 260 MBytes 2.18 Gbits/sec 0 1.79 MBytes [ 5] 6.00-7.00 sec 281 MBytes 2.36 Gbits/sec 0 1.90 MBytes [ 5] 7.00-8.00 sec 275 MBytes 2.31 Gbits/sec 224 1.42 MBytes [ 5] 8.00-9.00 sec 345 MBytes 2.89 Gbits/sec 0 1.57 MBytes [ 5] 9.00-10.00 sec 341 MBytes 2.86 Gbits/sec 0 1.70 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.41 GBytes 2.07 Gbits/sec 284 sender [ 5] 0.00-10.01 sec 2.41 GBytes 2.07 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:28:14.645638 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51932 -> 40.0.0.2:5001 08/20/2025-11:28:14.646455 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51948 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=233 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=233 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=51948 dport=5001 packets=1785360 bytes=2678027541 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51948 packets=201411 bytes=10471556 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=51932 dport=5001 packets=16 bytes=1301 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51932 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=60 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=60 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51312 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 247 MBytes 2.07 Gbits/sec 195 1.66 MBytes [ 5] 1.00-2.00 sec 289 MBytes 2.42 Gbits/sec 0 1.81 MBytes [ 5] 2.00-3.00 sec 276 MBytes 2.32 Gbits/sec 0 1.92 MBytes [ 5] 3.00-4.00 sec 279 MBytes 2.34 Gbits/sec 0 2.01 MBytes [ 5] 4.00-5.00 sec 295 MBytes 2.47 Gbits/sec 0 2.08 MBytes [ 5] 5.00-6.00 sec 301 MBytes 2.53 Gbits/sec 0 2.16 MBytes [ 5] 6.00-7.00 sec 278 MBytes 2.33 Gbits/sec 1 1.61 MBytes [ 5] 7.00-8.00 sec 276 MBytes 2.32 Gbits/sec 0 1.73 MBytes [ 5] 8.00-9.00 sec 315 MBytes 2.64 Gbits/sec 19 1.36 MBytes [ 5] 9.00-10.00 sec 279 MBytes 2.34 Gbits/sec 0 1.51 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.77 GBytes 2.38 Gbits/sec 215 sender [ 5] 0.00-10.00 sec 2.77 GBytes 2.38 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:28:14.645638 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51932 -> 40.0.0.2:5001 08/20/2025-11:28:14.646455 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51948 -> 40.0.0.2:5001 08/20/2025-11:28:30.544429 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51298 -> 40.0.0.2:5001 08/20/2025-11:28:30.546176 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51312 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=51312 dport=5001 packets=2052464 bytes=3078683821 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51312 packets=321182 bytes=16694408 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=51298 dport=5001 packets=16 bytes=1299 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51298 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 11797 0 --:--:-- --:--:-- --:--:-- 12500
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.652 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.652/0.652/0.652/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.430 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.430/0.430/0.430/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
08/20/2025-11:29:05.148285 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34366 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=236 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=236 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=34366 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=34366 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=63 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=63 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
08/20/2025-11:29:05.148285 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34366 -> 40.0.0.2:5000 08/20/2025-11:29:11.468973 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34366 -> 40.0.0.2:5000 08/20/2025-11:29:13.001302 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43618 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=43618 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=43618 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 18397 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.529 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.529/0.529/0.529/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.446 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.446/0.446/0.446/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
08/20/2025-11:29:36.362322 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53964 -> 40.0.0.2:5001 08/20/2025-11:29:36.363158 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53972 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=66 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=66 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=239 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=239 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53964 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53964 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53972 dport=5001 packets=16350 bytes=24519253 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53972 packets=2288 bytes=118984 [ASSURED] [OFFLOAD, packets=16337 bytes=24504052 packets=2286 bytes=118872] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 11339 0 --:--:-- --:--:-- --:--:-- 12090
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
08/20/2025-11:29:36.362322 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53964 -> 40.0.0.2:5001 08/20/2025-11:29:36.363158 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53972 -> 40.0.0.2:5001 08/20/2025-11:29:39.331812 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53974 -> 40.0.0.2:5001 08/20/2025-11:29:39.333132 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:33407 -> 40.0.0.2:5001 08/20/2025-11:29:39.343852 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:33407 -> 40.0.0.2:5001 08/20/2025-11:29:39.355916 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:33407 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53974 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53974 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=33407 dport=5001 packets=35 bytes=50216 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33407 packets=1 bytes=32 [OFFLOAD, packets=31 bytes=45756 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 46707 0 --:--:-- --:--:-- --:--:-- 50000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.584 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.584/0.584/0.584/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=4.29 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.294/4.294/4.294/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
08/20/2025-11:30:01.508976 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33700 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 33700 5000 yes 201 0 8 673 40.0.0.2 20.0.0.2 5000 33700 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250127s ending at 1755689405.285224 XDP_DROP 9 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9324987 pkts ( 8 pps) 12199165 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
08/20/2025-11:30:01.508976 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33700 -> 40.0.0.2:5000 08/20/2025-11:30:05.447566 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:35040 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 33700 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5001 35040 no 201 0 0 0 20.0.0.2 40.0.0.2 35040 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 33700 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250129s ending at 1755689409.226235 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9325025 pkts ( 0 pps) 12199168 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)