Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19068 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.830 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.830/0.830/0.830/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.947 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.947/0.947/0.947/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 36638 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 157 MBytes 1.32 Gbits/sec 13 1.83 MBytes [ 5] 1.00-2.00 sec 156 MBytes 1.31 Gbits/sec 311 1.38 MBytes [ 5] 2.00-3.00 sec 110 MBytes 923 Mbits/sec 0 1.45 MBytes [ 5] 3.00-4.00 sec 139 MBytes 1.16 Gbits/sec 0 1.50 MBytes [ 5] 4.00-5.00 sec 98.8 MBytes 828 Mbits/sec 0 1.54 MBytes [ 5] 5.00-6.00 sec 164 MBytes 1.37 Gbits/sec 0 1.62 MBytes [ 5] 6.00-7.00 sec 162 MBytes 1.36 Gbits/sec 0 1.69 MBytes [ 5] 7.00-8.00 sec 152 MBytes 1.28 Gbits/sec 0 1.76 MBytes [ 5] 8.00-9.00 sec 151 MBytes 1.27 Gbits/sec 0 1.82 MBytes [ 5] 9.00-10.00 sec 142 MBytes 1.20 Gbits/sec 0 1.87 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.40 GBytes 1.20 Gbits/sec 324 sender [ 5] 0.00-10.00 sec 1.40 GBytes 1.20 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:36:58.097596 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36632 -> 40.0.0.2:5001 04/16/2026-21:36:58.100421 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36638 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 21794 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.11 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.108/1.108/1.108/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.755 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.755/0.755/0.755/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 41630 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 263 MBytes 2.20 Gbits/sec 143 1.74 MBytes [ 5] 1.00-2.01 sec 284 MBytes 2.37 Gbits/sec 10 1.37 MBytes [ 5] 2.01-3.00 sec 212 MBytes 1.79 Gbits/sec 0 1.47 MBytes [ 5] 3.00-4.00 sec 308 MBytes 2.58 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 292 MBytes 2.45 Gbits/sec 0 1.74 MBytes [ 5] 5.00-6.00 sec 219 MBytes 1.83 Gbits/sec 232 1.30 MBytes [ 5] 6.00-7.00 sec 275 MBytes 2.31 Gbits/sec 0 1.43 MBytes [ 5] 7.00-8.00 sec 259 MBytes 2.17 Gbits/sec 0 1.56 MBytes [ 5] 8.00-9.00 sec 216 MBytes 1.81 Gbits/sec 0 1.65 MBytes [ 5] 9.00-10.00 sec 246 MBytes 2.07 Gbits/sec 1 1.28 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec 386 sender [ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:37:32.421110 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41626 -> 40.0.0.2:5001 04/16/2026-21:37:32.422254 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41630 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 16 21:37:24.531497 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Apr 16 21:37:24.000322 osdx systemd-timedated[873299]: Changed local time to Thu 2026-04-16 21:37:24 UTC Apr 16 21:37:24.001181 osdx systemd-journald[126917]: Time jumped backwards, rotating. Apr 16 21:37:24.001708 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'set date 2026-04-16 21:37:24'. Apr 16 21:37:24.355383 osdx systemd-journald[126917]: Runtime Journal (/run/log/journal/a3fecc5be5e949c7a083b0adeea4bb21) is 1.9M, max 13.8M, 11.9M free. Apr 16 21:37:24.357181 osdx systemd-journald[126917]: Received client request to rotate journal, rotating. Apr 16 21:37:24.357247 osdx systemd-journald[126917]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a3fecc5be5e949c7a083b0adeea4bb21. Apr 16 21:37:24.368504 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'system journal clear'. Apr 16 21:37:24.694805 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'system coredump delete all'. Apr 16 21:37:25.000373 osdx OSDxCLI[872824]: User 'admin' entered the configuration menu. Apr 16 21:37:25.146633 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 16 21:37:25.241745 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 16 21:37:25.361549 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 16 21:37:25.455985 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 16 21:37:25.521801 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 16 21:37:25.627681 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 16 21:37:25.710178 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'show working'. Apr 16 21:37:25.793295 osdx ubnt-cfgd[873332]: inactive Apr 16 21:37:25.823318 osdx INFO[873338]: FRR daemons did not change Apr 16 21:37:25.865189 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 16 21:37:25.886008 osdx WARNING[873378]: No supported link modes on interface eth1 Apr 16 21:37:25.887601 osdx modulelauncher[873378]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Apr 16 21:37:25.887615 osdx modulelauncher[873378]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Apr 16 21:37:25.889173 osdx modulelauncher[873378]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Apr 16 21:37:25.889183 osdx modulelauncher[873378]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Apr 16 21:37:25.925200 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 16 21:37:25.978665 osdx WARNING[873458]: No supported link modes on interface eth0 Apr 16 21:37:25.980538 osdx modulelauncher[873458]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Apr 16 21:37:25.980553 osdx modulelauncher[873458]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Apr 16 21:37:25.982217 osdx modulelauncher[873458]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Apr 16 21:37:25.982228 osdx modulelauncher[873458]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Apr 16 21:37:25.994857 osdx (udev-worker)[873474]: Network interface NamePolicy= disabled on kernel command line. Apr 16 21:37:26.036225 osdx (udev-worker)[873486]: Network interface NamePolicy= disabled on kernel command line. Apr 16 21:37:26.157869 osdx cfgd[1833]: [872824]Completed change to active configuration Apr 16 21:37:26.170158 osdx OSDxCLI[872824]: User 'admin' committed the configuration. Apr 16 21:37:26.189620 osdx OSDxCLI[872824]: User 'admin' left the configuration menu. Apr 16 21:37:28.341776 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 16 21:37:28.429499 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 16 21:37:28.520674 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 16 21:37:29.035504 osdx file_operation[873654]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 16 21:37:29.061952 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 16 21:37:29.191952 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 16 21:37:29.337391 osdx OSDxCLI[872824]: User 'admin' entered the configuration menu. Apr 16 21:37:29.399081 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 16 21:37:29.497749 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 16 21:37:29.551604 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 16 21:37:29.650883 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 16 21:37:29.729001 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 16 21:37:29.825493 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 16 21:37:29.921021 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 16 21:37:29.994533 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 16 21:37:30.099419 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 16 21:37:30.155627 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 16 21:37:30.248004 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 16 21:37:30.303050 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 16 21:37:30.397141 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 16 21:37:30.455294 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 16 21:37:30.542155 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 16 21:37:30.602841 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 16 21:37:30.691466 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 16 21:37:30.747167 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 16 21:37:30.840296 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 16 21:37:30.893793 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 16 21:37:31.038786 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'show working'. Apr 16 21:37:31.111243 osdx ubnt-cfgd[873702]: inactive Apr 16 21:37:31.207658 osdx INFO[873747]: FRR daemons did not change Apr 16 21:37:31.464214 osdx systemd[1]: Reloading. Apr 16 21:37:31.509189 osdx systemd-sysv-generator[873802]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 16 21:37:31.653512 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 16 21:37:31.676051 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 16 21:37:31.883830 osdx INFO[873784]: Rules successfully loaded Apr 16 21:37:31.884414 osdx cfgd[1833]: [872824]Completed change to active configuration Apr 16 21:37:31.886746 osdx OSDxCLI[872824]: User 'admin' committed the configuration. Apr 16 21:37:31.909400 osdx OSDxCLI[872824]: User 'admin' left the configuration menu. Apr 16 21:37:32.425578 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14929 DF PROTO=TCP SPT=41626 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 16 21:37:32.425937 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=48495 DF PROTO=TCP SPT=41630 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 16 21:37:42.603885 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 44830 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 295 MBytes 2.48 Gbits/sec 88 1.38 MBytes [ 5] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec 54 1.07 MBytes [ 5] 2.00-3.00 sec 214 MBytes 1.79 Gbits/sec 0 1.20 MBytes [ 5] 3.00-4.00 sec 298 MBytes 2.50 Gbits/sec 0 1.37 MBytes [ 5] 4.00-5.00 sec 284 MBytes 2.38 Gbits/sec 0 1.52 MBytes [ 5] 5.00-6.00 sec 260 MBytes 2.18 Gbits/sec 0 1.63 MBytes [ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec 0 1.74 MBytes [ 5] 7.00-8.00 sec 359 MBytes 3.01 Gbits/sec 0 1.88 MBytes [ 5] 8.00-9.00 sec 326 MBytes 2.74 Gbits/sec 185 1.47 MBytes [ 5] 9.00-10.00 sec 352 MBytes 2.96 Gbits/sec 0 1.63 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.86 GBytes 2.46 Gbits/sec 327 sender [ 5] 0.00-10.00 sec 2.86 GBytes 2.45 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:37:32.421110 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41626 -> 40.0.0.2:5001 04/16/2026-21:37:32.422254 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41630 -> 40.0.0.2:5001 04/16/2026-21:37:48.250950 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44818 -> 40.0.0.2:5001 04/16/2026-21:37:48.252048 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44830 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 16 21:37:24.531497 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Apr 16 21:37:24.000322 osdx systemd-timedated[873299]: Changed local time to Thu 2026-04-16 21:37:24 UTC Apr 16 21:37:24.001181 osdx systemd-journald[126917]: Time jumped backwards, rotating. Apr 16 21:37:24.001708 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'set date 2026-04-16 21:37:24'. Apr 16 21:37:24.355383 osdx systemd-journald[126917]: Runtime Journal (/run/log/journal/a3fecc5be5e949c7a083b0adeea4bb21) is 1.9M, max 13.8M, 11.9M free. Apr 16 21:37:24.357181 osdx systemd-journald[126917]: Received client request to rotate journal, rotating. Apr 16 21:37:24.357247 osdx systemd-journald[126917]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a3fecc5be5e949c7a083b0adeea4bb21. Apr 16 21:37:24.368504 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'system journal clear'. Apr 16 21:37:24.694805 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'system coredump delete all'. Apr 16 21:37:25.000373 osdx OSDxCLI[872824]: User 'admin' entered the configuration menu. Apr 16 21:37:25.146633 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 16 21:37:25.241745 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 16 21:37:25.361549 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 16 21:37:25.455985 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 16 21:37:25.521801 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 16 21:37:25.627681 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 16 21:37:25.710178 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'show working'. Apr 16 21:37:25.793295 osdx ubnt-cfgd[873332]: inactive Apr 16 21:37:25.823318 osdx INFO[873338]: FRR daemons did not change Apr 16 21:37:25.865189 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 16 21:37:25.886008 osdx WARNING[873378]: No supported link modes on interface eth1 Apr 16 21:37:25.887601 osdx modulelauncher[873378]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Apr 16 21:37:25.887615 osdx modulelauncher[873378]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Apr 16 21:37:25.889173 osdx modulelauncher[873378]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Apr 16 21:37:25.889183 osdx modulelauncher[873378]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Apr 16 21:37:25.925200 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 16 21:37:25.978665 osdx WARNING[873458]: No supported link modes on interface eth0 Apr 16 21:37:25.980538 osdx modulelauncher[873458]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Apr 16 21:37:25.980553 osdx modulelauncher[873458]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Apr 16 21:37:25.982217 osdx modulelauncher[873458]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Apr 16 21:37:25.982228 osdx modulelauncher[873458]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Apr 16 21:37:25.994857 osdx (udev-worker)[873474]: Network interface NamePolicy= disabled on kernel command line. Apr 16 21:37:26.036225 osdx (udev-worker)[873486]: Network interface NamePolicy= disabled on kernel command line. Apr 16 21:37:26.157869 osdx cfgd[1833]: [872824]Completed change to active configuration Apr 16 21:37:26.170158 osdx OSDxCLI[872824]: User 'admin' committed the configuration. Apr 16 21:37:26.189620 osdx OSDxCLI[872824]: User 'admin' left the configuration menu. Apr 16 21:37:28.341776 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 16 21:37:28.429499 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 16 21:37:28.520674 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 16 21:37:29.035504 osdx file_operation[873654]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 16 21:37:29.061952 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 16 21:37:29.191952 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 16 21:37:29.337391 osdx OSDxCLI[872824]: User 'admin' entered the configuration menu. Apr 16 21:37:29.399081 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 16 21:37:29.497749 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 16 21:37:29.551604 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 16 21:37:29.650883 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 16 21:37:29.729001 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 16 21:37:29.825493 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 16 21:37:29.921021 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 16 21:37:29.994533 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 16 21:37:30.099419 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 16 21:37:30.155627 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 16 21:37:30.248004 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 16 21:37:30.303050 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 16 21:37:30.397141 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 16 21:37:30.455294 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 16 21:37:30.542155 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 16 21:37:30.602841 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 16 21:37:30.691466 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 16 21:37:30.747167 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 16 21:37:30.840296 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 16 21:37:30.893793 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 16 21:37:31.038786 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'show working'. Apr 16 21:37:31.111243 osdx ubnt-cfgd[873702]: inactive Apr 16 21:37:31.207658 osdx INFO[873747]: FRR daemons did not change Apr 16 21:37:31.464214 osdx systemd[1]: Reloading. Apr 16 21:37:31.509189 osdx systemd-sysv-generator[873802]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 16 21:37:31.653512 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 16 21:37:31.676051 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 16 21:37:31.883830 osdx INFO[873784]: Rules successfully loaded Apr 16 21:37:31.884414 osdx cfgd[1833]: [872824]Completed change to active configuration Apr 16 21:37:31.886746 osdx OSDxCLI[872824]: User 'admin' committed the configuration. Apr 16 21:37:31.909400 osdx OSDxCLI[872824]: User 'admin' left the configuration menu. Apr 16 21:37:32.425578 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14929 DF PROTO=TCP SPT=41626 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 16 21:37:32.425937 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=48495 DF PROTO=TCP SPT=41630 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 16 21:37:42.603885 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Apr 16 21:37:42.727102 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'system journal show | cat'. Apr 16 21:37:42.929044 osdx OSDxCLI[872824]: User 'admin' entered the configuration menu. Apr 16 21:37:43.028978 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 16 21:37:43.130992 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 16 21:37:43.230181 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 16 21:37:43.328077 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 16 21:37:43.383740 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 16 21:37:43.547770 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 16 21:37:43.687193 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 16 21:37:43.807468 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 16 21:37:43.873187 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 16 21:37:43.969516 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 16 21:37:44.032017 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 16 21:37:44.136953 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 16 21:37:44.194677 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 16 21:37:44.314150 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 16 21:37:44.373925 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 16 21:37:44.489006 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 16 21:37:44.542128 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 16 21:37:44.652099 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 16 21:37:44.764738 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 16 21:37:44.891714 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 16 21:37:45.016215 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Apr 16 21:37:45.131594 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Apr 16 21:37:45.249151 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Apr 16 21:37:45.370021 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Apr 16 21:37:45.472343 osdx OSDxCLI[872824]: User 'admin' added a new cfg line: 'show changes'. Apr 16 21:37:45.549784 osdx ubnt-cfgd[873931]: inactive Apr 16 21:37:45.632064 osdx INFO[873951]: FRR daemons did not change Apr 16 21:37:45.887648 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Apr 16 21:37:47.772794 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Apr 16 21:37:47.772945 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Apr 16 21:37:47.772984 osdx systemd[1]: suricata@FW.service: Consumed 2.029s CPU time. Apr 16 21:37:47.801566 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 16 21:37:47.818633 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 16 21:37:48.017114 osdx INFO[873978]: Rules successfully loaded Apr 16 21:37:48.017909 osdx cfgd[1833]: [872824]Completed change to active configuration Apr 16 21:37:48.020229 osdx OSDxCLI[872824]: User 'admin' committed the configuration. Apr 16 21:37:48.036181 osdx OSDxCLI[872824]: User 'admin' left the configuration menu. Apr 16 21:37:48.253196 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=39558 DF PROTO=TCP SPT=44818 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 16 21:37:48.253265 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=31358 DF PROTO=TCP SPT=44830 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 16 21:37:54.030790 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Apr 16 21:37:58.412661 osdx OSDxCLI[872824]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 25338 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.966 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.966/0.966/0.966/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.536 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.536/0.536/0.536/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 34874 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 252 MBytes 2.11 Gbits/sec 275 1.28 MBytes [ 5] 1.00-2.00 sec 301 MBytes 2.53 Gbits/sec 0 1.44 MBytes [ 5] 2.00-3.00 sec 319 MBytes 2.67 Gbits/sec 0 1.59 MBytes [ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec 0 1.71 MBytes [ 5] 4.00-5.00 sec 296 MBytes 2.49 Gbits/sec 43 1.31 MBytes [ 5] 5.00-6.00 sec 286 MBytes 2.40 Gbits/sec 0 1.45 MBytes [ 5] 6.00-7.00 sec 306 MBytes 2.57 Gbits/sec 58 1.16 MBytes [ 5] 7.00-8.00 sec 261 MBytes 2.19 Gbits/sec 0 1.32 MBytes [ 5] 8.00-9.00 sec 268 MBytes 2.24 Gbits/sec 0 1.45 MBytes [ 5] 9.00-10.00 sec 278 MBytes 2.33 Gbits/sec 0 1.58 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec 376 sender [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:38:22.199711 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34862 -> 40.0.0.2:5001 04/16/2026-21:38:22.200960 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34874 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=444 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=444 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=34874 dport=5001 packets=2063294 bytes=3094924029 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34874 packets=185470 bytes=9636660 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=34862 dport=5001 packets=14 bytes=1194 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34862 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=103 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=103 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 41200 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 282 MBytes 2.37 Gbits/sec 160 1.32 MBytes [ 5] 1.00-2.00 sec 205 MBytes 1.72 Gbits/sec 0 1.43 MBytes [ 5] 2.00-3.00 sec 224 MBytes 1.88 Gbits/sec 0 1.53 MBytes [ 5] 3.00-4.00 sec 236 MBytes 1.98 Gbits/sec 0 1.64 MBytes [ 5] 4.00-5.00 sec 288 MBytes 2.41 Gbits/sec 0 1.75 MBytes [ 5] 5.00-6.00 sec 239 MBytes 2.00 Gbits/sec 14 1.37 MBytes [ 5] 6.00-7.00 sec 278 MBytes 2.33 Gbits/sec 0 1.49 MBytes [ 5] 7.00-8.00 sec 218 MBytes 1.82 Gbits/sec 0 1.59 MBytes [ 5] 8.00-9.00 sec 268 MBytes 2.24 Gbits/sec 88 1.23 MBytes [ 5] 9.00-10.00 sec 272 MBytes 2.29 Gbits/sec 0 1.38 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.45 GBytes 2.10 Gbits/sec 262 sender [ 5] 0.00-10.00 sec 2.45 GBytes 2.10 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:38:22.199711 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34862 -> 40.0.0.2:5001 04/16/2026-21:38:22.200960 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34874 -> 40.0.0.2:5001 04/16/2026-21:38:37.661284 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41194 -> 40.0.0.2:5001 04/16/2026-21:38:37.662306 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41200 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=41200 dport=5001 packets=1815794 bytes=2723681069 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=41200 packets=136347 bytes=7082376 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=41194 dport=5001 packets=17 bytes=1641 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=41194 packets=13 bytes=1030 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 33647 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.555 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.555/0.555/0.555/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.723 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.723/0.723/0.723/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/16/2026-21:39:15.868555 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54894 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=106 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=106 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=54894 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=54894 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=447 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=447 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 10 445 814 2 - 5 5 369 369 ------------------------------------------------------------------ Total 10 10 814 814
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/16/2026-21:39:15.868555 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54894 -> 40.0.0.2:5000 04/16/2026-21:39:22.197035 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54894 -> 40.0.0.2:5000 04/16/2026-21:39:23.950897 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49618 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=49618 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=49618 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 445 775 2 - 5 5 330 330 --------------------------------------------------------------------- Total 10 10 775 775
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 18115 0 --:--:-- --:--:-- --:--:-- 18428
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.972 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.972/0.972/0.972/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.626 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.626/0.626/0.626/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/16/2026-21:39:49.065402 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38518 -> 40.0.0.2:5001 04/16/2026-21:39:49.066781 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38524 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=450 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=450 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=109 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=109 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=38524 dport=5001 packets=38317 bytes=57471201 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=38524 packets=3757 bytes=197220 [ASSURED] [OFFLOAD, packets=38304 bytes=57456000 packets=3755 bytes=197108] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=38518 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=38518 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 17390 0 --:--:-- --:--:-- --:--:-- 19000
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
04/16/2026-21:39:49.065402 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38518 -> 40.0.0.2:5001 04/16/2026-21:39:49.066781 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38524 -> 40.0.0.2:5001 04/16/2026-21:39:51.248173 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38536 -> 40.0.0.2:5001 04/16/2026-21:39:51.250088 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41893 -> 40.0.0.2:5001 04/16/2026-21:39:51.261072 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41893 -> 40.0.0.2:5001 04/16/2026-21:39:51.273054 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41893 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=38536 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=38536 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=3 bytes=298 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=41893 dport=5001 packets=17 bytes=23648 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=41893 packets=1 bytes=32 [OFFLOAD, packets=13 bytes=19188 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 53966 0 --:--:-- --:--:-- --:--:-- 66666
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.06 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.059/1.059/1.059/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.599 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.599/0.599/0.599/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/16/2026-21:40:12.720352 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48996 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 48996 5000 yes 201 0 7 602 40.0.0.2 20.0.0.2 5000 48996 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250144s ending at 1776375616.443656 XDP_DROP 8 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 11097733 pkts ( 8 pps) 15149713 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
04/16/2026-21:40:12.720352 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48996 -> 40.0.0.2:5000 04/16/2026-21:40:16.658095 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:54317 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 48996 5000 yes 201 0 10 776 20.0.0.2 40.0.0.2 54317 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 48996 yes 201 0 0 0 40.0.0.2 20.0.0.2 5001 54317 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250141s ending at 1776375620.313417 XDP_DROP 10 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 11097773 pkts ( 0 pps) 15149716 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)