Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 3386 0 --:--:-- --:--:-- --:--:-- 3486
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=21.7 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 21.735/21.735/21.735/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=1.30 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.303/1.303/1.303/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33730 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 34.5 MBytes 289 Mbits/sec 58 1.49 MBytes [ 5] 1.00-2.01 sec 45.0 MBytes 375 Mbits/sec 0 1.32 MBytes [ 5] 2.01-3.00 sec 41.2 MBytes 348 Mbits/sec 20 1024 KBytes [ 5] 3.00-4.00 sec 46.2 MBytes 388 Mbits/sec 0 1.07 MBytes [ 5] 4.00-5.00 sec 43.8 MBytes 367 Mbits/sec 0 1.12 MBytes [ 5] 5.00-6.00 sec 46.2 MBytes 388 Mbits/sec 0 1.16 MBytes [ 5] 6.00-7.00 sec 46.2 MBytes 388 Mbits/sec 0 1.18 MBytes [ 5] 7.00-8.00 sec 60.0 MBytes 503 Mbits/sec 0 1.19 MBytes [ 5] 8.00-9.00 sec 101 MBytes 850 Mbits/sec 0 1.21 MBytes [ 5] 9.00-10.00 sec 109 MBytes 912 Mbits/sec 0 1.27 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 573 MBytes 481 Mbits/sec 78 sender [ 5] 0.00-10.01 sec 571 MBytes 479 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:48:39.770151 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33714 -> 40.0.0.2:5001 03/20/2026-10:48:39.777551 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33730 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 44467 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.688 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.688/0.688/0.688/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=3.24 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 3.242/3.242/3.242/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 37274 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 97.8 MBytes 820 Mbits/sec 40 1.53 MBytes [ 5] 1.00-2.00 sec 165 MBytes 1.38 Gbits/sec 0 1.67 MBytes [ 5] 2.00-3.00 sec 140 MBytes 1.17 Gbits/sec 0 1.78 MBytes [ 5] 3.00-4.00 sec 196 MBytes 1.65 Gbits/sec 114 1.35 MBytes [ 5] 4.00-5.00 sec 182 MBytes 1.53 Gbits/sec 0 1.44 MBytes [ 5] 5.00-6.00 sec 164 MBytes 1.37 Gbits/sec 0 1.52 MBytes [ 5] 6.00-7.00 sec 201 MBytes 1.69 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 181 MBytes 1.52 Gbits/sec 0 1.69 MBytes [ 5] 8.00-9.01 sec 55.0 MBytes 456 Mbits/sec 201 1.36 MBytes [ 5] 9.01-10.00 sec 63.8 MBytes 541 Mbits/sec 30 1.04 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.41 GBytes 1.21 Gbits/sec 385 sender [ 5] 0.00-10.01 sec 1.41 GBytes 1.21 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:49:19.261688 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37268 -> 40.0.0.2:5001 03/20/2026-10:49:19.280616 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37274 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 20 10:49:05.705139 osdx systemd-journald[2054]: Runtime Journal (/run/log/journal/ba147585e3944580985006d3d45a280a) is 2.1M, max 17.2M, 15.1M free. Mar 20 10:49:05.710408 osdx systemd-journald[2054]: Received client request to rotate journal, rotating. Mar 20 10:49:05.710483 osdx systemd-journald[2054]: Vacuuming done, freed 0B of archived journals from /run/log/journal/ba147585e3944580985006d3d45a280a. Mar 20 10:49:05.728572 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'system journal clear'. Mar 20 10:49:06.182627 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'system coredump delete all'. Mar 20 10:49:06.675530 osdx OSDxCLI[393028]: User 'admin' entered the configuration menu. Mar 20 10:49:06.850556 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 20 10:49:06.975391 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 20 10:49:07.157032 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 20 10:49:07.334144 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 20 10:49:07.469519 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 20 10:49:07.584730 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 20 10:49:07.776321 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'show working'. Mar 20 10:49:07.958118 osdx ubnt-cfgd[393535]: inactive Mar 20 10:49:08.035593 osdx INFO[393541]: FRR daemons did not change Mar 20 10:49:08.134476 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 20 10:49:08.201488 osdx WARNING[393581]: No supported link modes on interface eth1 Mar 20 10:49:08.203892 osdx modulelauncher[393581]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Mar 20 10:49:08.203908 osdx modulelauncher[393581]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Mar 20 10:49:08.205759 osdx modulelauncher[393581]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Mar 20 10:49:08.205769 osdx modulelauncher[393581]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Mar 20 10:49:08.290617 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 20 10:49:08.403688 osdx WARNING[393661]: No supported link modes on interface eth0 Mar 20 10:49:08.405545 osdx modulelauncher[393661]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Mar 20 10:49:08.405557 osdx modulelauncher[393661]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Mar 20 10:49:08.406984 osdx modulelauncher[393661]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Mar 20 10:49:08.406996 osdx modulelauncher[393661]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Mar 20 10:49:08.449417 osdx (udev-worker)[393677]: Network interface NamePolicy= disabled on kernel command line. Mar 20 10:49:08.876370 osdx cfgd[1833]: [393028]Completed change to active configuration Mar 20 10:49:08.906305 osdx OSDxCLI[393028]: User 'admin' committed the configuration. Mar 20 10:49:08.993975 osdx OSDxCLI[393028]: User 'admin' left the configuration menu. Mar 20 10:49:12.738748 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 20 10:49:12.896804 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 20 10:49:13.025528 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 20 10:49:14.194971 osdx file_operation[393856]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 20 10:49:14.244600 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 20 10:49:14.496304 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 20 10:49:14.738180 osdx OSDxCLI[393028]: User 'admin' entered the configuration menu. Mar 20 10:49:14.924929 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 20 10:49:15.041834 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 20 10:49:15.177912 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 20 10:49:15.300412 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 20 10:49:15.431849 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 20 10:49:15.532722 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 20 10:49:15.743370 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 20 10:49:15.875546 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 20 10:49:15.981817 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 20 10:49:16.089787 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 20 10:49:16.190603 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 20 10:49:16.340880 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 20 10:49:16.471216 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 20 10:49:16.566473 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 20 10:49:16.702059 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 20 10:49:16.882285 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 20 10:49:16.996467 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 20 10:49:17.162605 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 20 10:49:17.275360 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 20 10:49:17.398953 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 20 10:49:17.537215 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'show working'. Mar 20 10:49:17.709260 osdx ubnt-cfgd[393909]: inactive Mar 20 10:49:17.832979 osdx INFO[393954]: FRR daemons did not change Mar 20 10:49:18.094571 osdx systemd[1]: Reloading. Mar 20 10:49:18.152967 osdx systemd-sysv-generator[394010]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 20 10:49:18.317735 osdx systemd[1]: Starting logrotate.service - Rotate log files... Mar 20 10:49:18.322349 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 20 10:49:18.359500 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 20 10:49:18.360961 osdx systemd[1]: logrotate.service: Deactivated successfully. Mar 20 10:49:18.361089 osdx systemd[1]: Finished logrotate.service - Rotate log files. Mar 20 10:49:18.654862 osdx INFO[393991]: Rules successfully loaded Mar 20 10:49:18.655790 osdx cfgd[1833]: [393028]Completed change to active configuration Mar 20 10:49:18.660257 osdx OSDxCLI[393028]: User 'admin' committed the configuration. Mar 20 10:49:18.691553 osdx OSDxCLI[393028]: User 'admin' left the configuration menu. Mar 20 10:49:19.265153 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=45523 DF PROTO=TCP SPT=37268 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 20 10:49:19.285261 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14515 DF PROTO=TCP SPT=37274 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 20 10:49:29.563911 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 42236 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 105 MBytes 883 Mbits/sec 15 2.51 MBytes [ 5] 1.00-2.00 sec 76.2 MBytes 640 Mbits/sec 321 1.30 MBytes [ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 1 1.32 MBytes [ 5] 3.00-4.00 sec 214 MBytes 1.79 Gbits/sec 64 1.91 MBytes [ 5] 4.00-5.00 sec 200 MBytes 1.68 Gbits/sec 173 683 KBytes [ 5] 5.00-6.00 sec 154 MBytes 1.29 Gbits/sec 0 901 KBytes [ 5] 6.00-7.00 sec 112 MBytes 944 Mbits/sec 0 1.07 MBytes [ 5] 7.00-8.00 sec 102 MBytes 860 Mbits/sec 0 1.23 MBytes [ 5] 8.00-9.00 sec 149 MBytes 1.25 Gbits/sec 0 1.36 MBytes [ 5] 9.00-10.00 sec 144 MBytes 1.21 Gbits/sec 0 1.46 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.23 GBytes 1.05 Gbits/sec 574 sender [ 5] 0.00-10.00 sec 1.23 GBytes 1.05 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:49:19.261688 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37268 -> 40.0.0.2:5001 03/20/2026-10:49:19.280616 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37274 -> 40.0.0.2:5001 03/20/2026-10:49:35.915710 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42228 -> 40.0.0.2:5001 03/20/2026-10:49:35.917732 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42236 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 20 10:49:05.705139 osdx systemd-journald[2054]: Runtime Journal (/run/log/journal/ba147585e3944580985006d3d45a280a) is 2.1M, max 17.2M, 15.1M free. Mar 20 10:49:05.710408 osdx systemd-journald[2054]: Received client request to rotate journal, rotating. Mar 20 10:49:05.710483 osdx systemd-journald[2054]: Vacuuming done, freed 0B of archived journals from /run/log/journal/ba147585e3944580985006d3d45a280a. Mar 20 10:49:05.728572 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'system journal clear'. Mar 20 10:49:06.182627 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'system coredump delete all'. Mar 20 10:49:06.675530 osdx OSDxCLI[393028]: User 'admin' entered the configuration menu. Mar 20 10:49:06.850556 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 20 10:49:06.975391 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 20 10:49:07.157032 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 20 10:49:07.334144 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 20 10:49:07.469519 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 20 10:49:07.584730 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 20 10:49:07.776321 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'show working'. Mar 20 10:49:07.958118 osdx ubnt-cfgd[393535]: inactive Mar 20 10:49:08.035593 osdx INFO[393541]: FRR daemons did not change Mar 20 10:49:08.134476 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 20 10:49:08.201488 osdx WARNING[393581]: No supported link modes on interface eth1 Mar 20 10:49:08.203892 osdx modulelauncher[393581]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Mar 20 10:49:08.203908 osdx modulelauncher[393581]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Mar 20 10:49:08.205759 osdx modulelauncher[393581]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Mar 20 10:49:08.205769 osdx modulelauncher[393581]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Mar 20 10:49:08.290617 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 20 10:49:08.403688 osdx WARNING[393661]: No supported link modes on interface eth0 Mar 20 10:49:08.405545 osdx modulelauncher[393661]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Mar 20 10:49:08.405557 osdx modulelauncher[393661]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Mar 20 10:49:08.406984 osdx modulelauncher[393661]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Mar 20 10:49:08.406996 osdx modulelauncher[393661]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Mar 20 10:49:08.449417 osdx (udev-worker)[393677]: Network interface NamePolicy= disabled on kernel command line. Mar 20 10:49:08.876370 osdx cfgd[1833]: [393028]Completed change to active configuration Mar 20 10:49:08.906305 osdx OSDxCLI[393028]: User 'admin' committed the configuration. Mar 20 10:49:08.993975 osdx OSDxCLI[393028]: User 'admin' left the configuration menu. Mar 20 10:49:12.738748 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 20 10:49:12.896804 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 20 10:49:13.025528 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 20 10:49:14.194971 osdx file_operation[393856]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 20 10:49:14.244600 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 20 10:49:14.496304 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 20 10:49:14.738180 osdx OSDxCLI[393028]: User 'admin' entered the configuration menu. Mar 20 10:49:14.924929 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 20 10:49:15.041834 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 20 10:49:15.177912 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 20 10:49:15.300412 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 20 10:49:15.431849 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 20 10:49:15.532722 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 20 10:49:15.743370 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 20 10:49:15.875546 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 20 10:49:15.981817 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 20 10:49:16.089787 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 20 10:49:16.190603 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 20 10:49:16.340880 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 20 10:49:16.471216 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 20 10:49:16.566473 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 20 10:49:16.702059 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 20 10:49:16.882285 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 20 10:49:16.996467 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 20 10:49:17.162605 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 20 10:49:17.275360 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 20 10:49:17.398953 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 20 10:49:17.537215 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'show working'. Mar 20 10:49:17.709260 osdx ubnt-cfgd[393909]: inactive Mar 20 10:49:17.832979 osdx INFO[393954]: FRR daemons did not change Mar 20 10:49:18.094571 osdx systemd[1]: Reloading. Mar 20 10:49:18.152967 osdx systemd-sysv-generator[394010]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 20 10:49:18.317735 osdx systemd[1]: Starting logrotate.service - Rotate log files... Mar 20 10:49:18.322349 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 20 10:49:18.359500 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 20 10:49:18.360961 osdx systemd[1]: logrotate.service: Deactivated successfully. Mar 20 10:49:18.361089 osdx systemd[1]: Finished logrotate.service - Rotate log files. Mar 20 10:49:18.654862 osdx INFO[393991]: Rules successfully loaded Mar 20 10:49:18.655790 osdx cfgd[1833]: [393028]Completed change to active configuration Mar 20 10:49:18.660257 osdx OSDxCLI[393028]: User 'admin' committed the configuration. Mar 20 10:49:18.691553 osdx OSDxCLI[393028]: User 'admin' left the configuration menu. Mar 20 10:49:19.265153 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=45523 DF PROTO=TCP SPT=37268 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 20 10:49:19.285261 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14515 DF PROTO=TCP SPT=37274 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 20 10:49:29.563911 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Mar 20 10:49:29.763984 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'system journal show | cat'. Mar 20 10:49:30.047119 osdx OSDxCLI[393028]: User 'admin' entered the configuration menu. Mar 20 10:49:30.120933 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 20 10:49:30.253795 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 20 10:49:30.360272 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 20 10:49:30.497721 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 20 10:49:30.598447 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 20 10:49:30.707088 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 20 10:49:30.916269 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 20 10:49:31.020506 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 20 10:49:31.142539 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 20 10:49:31.249497 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 20 10:49:31.391608 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 20 10:49:31.499734 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 20 10:49:31.579256 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 20 10:49:31.760614 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 20 10:49:31.854744 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 20 10:49:31.934349 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 20 10:49:32.089046 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 20 10:49:32.238508 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 20 10:49:32.369689 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 20 10:49:32.438297 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 20 10:49:32.562253 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Mar 20 10:49:32.679512 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Mar 20 10:49:32.800452 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Mar 20 10:49:32.920960 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Mar 20 10:49:33.013297 osdx OSDxCLI[393028]: User 'admin' added a new cfg line: 'show changes'. Mar 20 10:49:33.158489 osdx ubnt-cfgd[394143]: inactive Mar 20 10:49:33.272291 osdx INFO[394163]: FRR daemons did not change Mar 20 10:49:33.512596 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Mar 20 10:49:35.035385 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Mar 20 10:49:35.343695 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Mar 20 10:49:35.343833 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Mar 20 10:49:35.343895 osdx systemd[1]: suricata@FW.service: Consumed 2.034s CPU time. Mar 20 10:49:35.357406 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 20 10:49:35.387852 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 20 10:49:35.662866 osdx INFO[394190]: Rules successfully loaded Mar 20 10:49:35.663469 osdx cfgd[1833]: [393028]Completed change to active configuration Mar 20 10:49:35.665632 osdx OSDxCLI[393028]: User 'admin' committed the configuration. Mar 20 10:49:35.682619 osdx OSDxCLI[393028]: User 'admin' left the configuration menu. Mar 20 10:49:35.917049 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=30556 DF PROTO=TCP SPT=42228 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 20 10:49:35.921225 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=28905 DF PROTO=TCP SPT=42236 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 20 10:49:46.223223 osdx OSDxCLI[393028]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 3809 0 --:--:-- --:--:-- --:--:-- 3909
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=9.41 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 9.411/9.411/9.411/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.810 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.810/0.810/0.810/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59336 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 144 MBytes 1.21 Gbits/sec 90 1.51 MBytes [ 5] 1.00-2.00 sec 176 MBytes 1.48 Gbits/sec 0 1.65 MBytes [ 5] 2.00-3.00 sec 160 MBytes 1.34 Gbits/sec 0 1.76 MBytes [ 5] 3.00-4.00 sec 131 MBytes 1.10 Gbits/sec 72 1.17 MBytes [ 5] 4.00-5.00 sec 136 MBytes 1.14 Gbits/sec 0 1.38 MBytes [ 5] 5.00-6.00 sec 150 MBytes 1.26 Gbits/sec 0 1.45 MBytes [ 5] 6.00-7.00 sec 232 MBytes 1.95 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 271 MBytes 2.28 Gbits/sec 0 1.66 MBytes [ 5] 8.00-9.00 sec 260 MBytes 2.18 Gbits/sec 19 1.24 MBytes [ 5] 9.00-10.00 sec 290 MBytes 2.43 Gbits/sec 0 1.39 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.91 GBytes 1.64 Gbits/sec 181 sender [ 5] 0.00-10.00 sec 1.90 GBytes 1.64 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:50:13.136893 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59326 -> 40.0.0.2:5001 03/20/2026-10:50:13.138141 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59336 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=509 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=509 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=80 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=80 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=59336 dport=5001 packets=1412895 bytes=2119335257 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59336 packets=124133 bytes=6449852 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=59326 dport=5001 packets=18 bytes=1703 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59326 packets=14 bytes=1083 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 48614 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 296 MBytes 2.48 Gbits/sec 21 1.34 MBytes [ 5] 1.00-2.00 sec 250 MBytes 2.10 Gbits/sec 0 1.47 MBytes [ 5] 2.00-3.00 sec 269 MBytes 2.25 Gbits/sec 0 1.59 MBytes [ 5] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec 84 1.23 MBytes [ 5] 4.00-5.00 sec 278 MBytes 2.33 Gbits/sec 0 1.39 MBytes [ 5] 5.00-6.00 sec 310 MBytes 2.60 Gbits/sec 0 1.54 MBytes [ 5] 6.00-7.00 sec 314 MBytes 2.63 Gbits/sec 0 1.68 MBytes [ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec 0 1.79 MBytes [ 5] 8.00-9.00 sec 312 MBytes 2.62 Gbits/sec 0 1.92 MBytes [ 5] 9.00-10.00 sec 349 MBytes 2.93 Gbits/sec 0 2.05 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.85 GBytes 2.45 Gbits/sec 105 sender [ 5] 0.00-10.00 sec 2.85 GBytes 2.44 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:50:13.136893 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59326 -> 40.0.0.2:5001 03/20/2026-10:50:13.138141 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59336 -> 40.0.0.2:5001 03/20/2026-10:50:27.696969 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48612 -> 40.0.0.2:5001 03/20/2026-10:50:27.698116 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48614 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=48612 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48612 packets=13 bytes=1017 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=48614 dport=5001 packets=2111835 bytes=3167737505 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48614 packets=275353 bytes=14309784 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 3813 0 --:--:-- --:--:-- --:--:-- 3773
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.765 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.765/0.765/0.765/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.590 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.590/0.590/0.590/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/20/2026-10:51:14.294390 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51068 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=51068 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=51068 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=512 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=512 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=83 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=83 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 10 409 778 2 - 5 5 369 369 ------------------------------------------------------------------ Total 10 10 778 778
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/20/2026-10:51:14.294390 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51068 -> 40.0.0.2:5000 03/20/2026-10:51:20.549096 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51068 -> 40.0.0.2:5000 03/20/2026-10:51:21.551388 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51068 -> 40.0.0.2:5000 03/20/2026-10:51:23.061728 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36728 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=36728 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=36728 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 409 690 2 - 5 5 281 281 --------------------------------------------------------------------- Total 10 10 690 690
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 15462 0 --:--:-- --:--:-- --:--:-- 16125
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.646 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.646/0.646/0.646/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.888 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.888/0.888/0.888/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/20/2026-10:51:48.876111 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58030 -> 40.0.0.2:5001 03/20/2026-10:51:48.877745 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58046 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=58030 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=58030 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=86 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=86 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=515 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=515 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=58046 dport=5001 packets=33290 bytes=49930701 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=58046 packets=3265 bytes=169788 [ASSURED] [OFFLOAD, packets=33277 bytes=49915500 packets=3263 bytes=169676] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 4757 0 --:--:-- --:--:-- --:--:-- 4925
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
03/20/2026-10:51:48.876111 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58030 -> 40.0.0.2:5001 03/20/2026-10:51:48.877745 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58046 -> 40.0.0.2:5001 03/20/2026-10:51:51.563814 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58056 -> 40.0.0.2:5001 03/20/2026-10:51:51.565483 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:37686 -> 40.0.0.2:5001 03/20/2026-10:51:51.576155 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:37686 -> 40.0.0.2:5001 03/20/2026-10:51:51.588226 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:37686 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=58056 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=58056 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=37686 dport=5001 packets=11 bytes=14792 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=37686 packets=1 bytes=32 [OFFLOAD, packets=7 bytes=10332 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 31162 0 --:--:-- --:--:-- --:--:-- 33333
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.805 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.805/0.805/0.805/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.558 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.558/0.558/0.558/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/20/2026-10:52:12.636355 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46222 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 46222 5000 yes 201 0 7 615 40.0.0.2 20.0.0.2 5000 46222 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250142s ending at 1774003936.258478 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
03/20/2026-10:52:12.636355 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46222 -> 40.0.0.2:5000 03/20/2026-10:52:16.415511 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:37103 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5001 37103 no 201 0 0 0 20.0.0.2 40.0.0.2 37103 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 46222 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5000 46222 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250140s ending at 1774003940.055283 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)