Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 9299 0 --:--:-- --:--:-- --:--:-- 9923
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.560 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.560/0.560/0.560/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.544 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.544/0.544/0.544/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 52436 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 110 MBytes 923 Mbits/sec 41 1.18 MBytes [ 5] 1.00-2.00 sec 132 MBytes 1.11 Gbits/sec 0 1.29 MBytes [ 5] 2.00-3.00 sec 151 MBytes 1.27 Gbits/sec 0 1.38 MBytes [ 5] 3.00-4.00 sec 155 MBytes 1.30 Gbits/sec 0 1.44 MBytes [ 5] 4.00-5.00 sec 156 MBytes 1.31 Gbits/sec 0 1.49 MBytes [ 5] 5.00-6.00 sec 146 MBytes 1.23 Gbits/sec 0 1.52 MBytes [ 5] 6.00-7.00 sec 131 MBytes 1.10 Gbits/sec 0 1.58 MBytes [ 5] 7.00-8.00 sec 150 MBytes 1.26 Gbits/sec 0 1.65 MBytes [ 5] 8.00-9.00 sec 146 MBytes 1.23 Gbits/sec 0 1.71 MBytes [ 5] 9.00-10.00 sec 156 MBytes 1.31 Gbits/sec 0 1.78 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.40 GBytes 1.20 Gbits/sec 41 sender [ 5] 0.00-10.01 sec 1.40 GBytes 1.20 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:55:45.405063 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52428 -> 40.0.0.2:5001 03/05/2026-19:55:45.406069 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52436 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 42588 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.591 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.591/0.591/0.591/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.512 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.512/0.512/0.512/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 49978 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 149 MBytes 1.25 Gbits/sec 93 1.82 MBytes [ 5] 1.00-2.00 sec 222 MBytes 1.87 Gbits/sec 0 1.99 MBytes [ 5] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec 17 1.50 MBytes [ 5] 3.00-4.00 sec 261 MBytes 2.19 Gbits/sec 0 1.62 MBytes [ 5] 4.00-5.00 sec 331 MBytes 2.78 Gbits/sec 0 1.76 MBytes [ 5] 5.00-6.00 sec 326 MBytes 2.74 Gbits/sec 123 1.33 MBytes [ 5] 6.00-7.00 sec 341 MBytes 2.86 Gbits/sec 0 1.51 MBytes [ 5] 7.00-8.00 sec 336 MBytes 2.82 Gbits/sec 0 1.66 MBytes [ 5] 8.00-9.00 sec 299 MBytes 2.51 Gbits/sec 18 1.30 MBytes [ 5] 9.00-10.00 sec 332 MBytes 2.79 Gbits/sec 0 1.48 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.81 GBytes 2.41 Gbits/sec 251 sender [ 5] 0.00-10.00 sec 2.81 GBytes 2.41 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:56:21.173696 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49974 -> 40.0.0.2:5001 03/05/2026-19:56:21.174907 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49978 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 05 19:56:11.322591 osdx systemd-journald[466780]: Runtime Journal (/run/log/journal/78445f9299f9414990e66dc8aa510838) is 1.7M, max 13.8M, 12.0M free. Mar 05 19:56:11.324275 osdx systemd-journald[466780]: Received client request to rotate journal, rotating. Mar 05 19:56:11.324330 osdx systemd-journald[466780]: Vacuuming done, freed 0B of archived journals from /run/log/journal/78445f9299f9414990e66dc8aa510838. Mar 05 19:56:11.333225 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'system journal clear'. Mar 05 19:56:11.570824 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'system coredump delete all'. Mar 05 19:56:11.793877 osdx OSDxCLI[733738]: User 'admin' entered the configuration menu. Mar 05 19:56:11.881308 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 05 19:56:12.159363 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 05 19:56:12.317289 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 05 19:56:12.432228 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 05 19:56:12.529237 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 05 19:56:12.580912 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 05 19:56:12.711050 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'show working'. Mar 05 19:56:12.772458 osdx ubnt-cfgd[734256]: inactive Mar 05 19:56:12.797580 osdx INFO[734262]: FRR daemons did not change Mar 05 19:56:12.828281 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 05 19:56:12.846401 osdx WARNING[734302]: No supported link modes on interface eth1 Mar 05 19:56:12.847776 osdx modulelauncher[734302]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Mar 05 19:56:12.847789 osdx modulelauncher[734302]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Mar 05 19:56:12.848893 osdx modulelauncher[734302]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Mar 05 19:56:12.848902 osdx modulelauncher[734302]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Mar 05 19:56:12.884285 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 05 19:56:12.931385 osdx WARNING[734382]: No supported link modes on interface eth0 Mar 05 19:56:12.932810 osdx modulelauncher[734382]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Mar 05 19:56:12.932824 osdx modulelauncher[734382]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Mar 05 19:56:12.933998 osdx modulelauncher[734382]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Mar 05 19:56:12.934005 osdx modulelauncher[734382]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Mar 05 19:56:12.944212 osdx (udev-worker)[734398]: Network interface NamePolicy= disabled on kernel command line. Mar 05 19:56:12.983178 osdx (udev-worker)[734410]: Network interface NamePolicy= disabled on kernel command line. Mar 05 19:56:13.094881 osdx cfgd[1863]: [733738]Completed change to active configuration Mar 05 19:56:13.110171 osdx OSDxCLI[733738]: User 'admin' committed the configuration. Mar 05 19:56:13.128490 osdx OSDxCLI[733738]: User 'admin' left the configuration menu. Mar 05 19:56:15.354430 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 05 19:56:15.436645 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 05 19:56:15.556715 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 05 19:56:16.035513 osdx file_operation[734577]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 05 19:56:16.057535 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 05 19:56:16.216706 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 05 19:56:16.349514 osdx OSDxCLI[733738]: User 'admin' entered the configuration menu. Mar 05 19:56:16.408684 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 05 19:56:16.504776 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 05 19:56:16.564636 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 05 19:56:16.654638 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 05 19:56:16.706450 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 05 19:56:16.800709 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 05 19:56:16.887187 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 05 19:56:16.954819 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 05 19:56:17.044598 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 05 19:56:17.107817 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 05 19:56:17.203286 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 05 19:56:17.258239 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 05 19:56:17.353407 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 05 19:56:17.407695 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 05 19:56:17.501867 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 05 19:56:17.560504 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 05 19:56:17.646948 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 05 19:56:18.648215 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 05 19:56:18.713200 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 05 19:56:18.815723 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 05 19:56:18.899700 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'show working'. Mar 05 19:56:18.997893 osdx ubnt-cfgd[734630]: inactive Mar 05 19:56:19.074668 osdx INFO[734675]: FRR daemons did not change Mar 05 19:56:19.276722 osdx systemd[1]: Reloading. Mar 05 19:56:19.340357 osdx systemd-sysv-generator[734733]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 05 19:56:19.464593 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 05 19:56:19.481327 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 05 19:56:19.684735 osdx INFO[734712]: Rules successfully loaded Mar 05 19:56:19.685320 osdx cfgd[1863]: [733738]Completed change to active configuration Mar 05 19:56:19.687263 osdx OSDxCLI[733738]: User 'admin' committed the configuration. Mar 05 19:56:20.836435 osdx OSDxCLI[733738]: User 'admin' left the configuration menu. Mar 05 19:56:21.176296 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=49080 DF PROTO=TCP SPT=49974 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 05 19:56:21.176382 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=49843 DF PROTO=TCP SPT=49978 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 05 19:56:31.327560 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 54574 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 298 MBytes 2.50 Gbits/sec 88 1.85 MBytes [ 5] 1.00-2.00 sec 245 MBytes 2.06 Gbits/sec 0 2.01 MBytes [ 5] 2.00-3.00 sec 230 MBytes 1.93 Gbits/sec 385 1.51 MBytes [ 5] 3.00-4.00 sec 269 MBytes 2.25 Gbits/sec 0 1.63 MBytes [ 5] 4.00-5.00 sec 254 MBytes 2.13 Gbits/sec 0 1.73 MBytes [ 5] 5.00-6.00 sec 326 MBytes 2.74 Gbits/sec 0 1.86 MBytes [ 5] 6.00-7.00 sec 255 MBytes 2.14 Gbits/sec 316 1.43 MBytes [ 5] 7.00-8.00 sec 290 MBytes 2.43 Gbits/sec 0 1.57 MBytes [ 5] 8.00-9.00 sec 321 MBytes 2.69 Gbits/sec 99 1.26 MBytes [ 5] 9.00-10.00 sec 314 MBytes 2.63 Gbits/sec 0 1.43 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec 888 sender [ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:56:21.173696 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49974 -> 40.0.0.2:5001 03/05/2026-19:56:21.174907 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49978 -> 40.0.0.2:5001 03/05/2026-19:56:35.944744 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54558 -> 40.0.0.2:5001 03/05/2026-19:56:35.945781 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54574 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 05 19:56:11.322591 osdx systemd-journald[466780]: Runtime Journal (/run/log/journal/78445f9299f9414990e66dc8aa510838) is 1.7M, max 13.8M, 12.0M free. Mar 05 19:56:11.324275 osdx systemd-journald[466780]: Received client request to rotate journal, rotating. Mar 05 19:56:11.324330 osdx systemd-journald[466780]: Vacuuming done, freed 0B of archived journals from /run/log/journal/78445f9299f9414990e66dc8aa510838. Mar 05 19:56:11.333225 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'system journal clear'. Mar 05 19:56:11.570824 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'system coredump delete all'. Mar 05 19:56:11.793877 osdx OSDxCLI[733738]: User 'admin' entered the configuration menu. Mar 05 19:56:11.881308 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 05 19:56:12.159363 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 05 19:56:12.317289 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 05 19:56:12.432228 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 05 19:56:12.529237 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 05 19:56:12.580912 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 05 19:56:12.711050 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'show working'. Mar 05 19:56:12.772458 osdx ubnt-cfgd[734256]: inactive Mar 05 19:56:12.797580 osdx INFO[734262]: FRR daemons did not change Mar 05 19:56:12.828281 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 05 19:56:12.846401 osdx WARNING[734302]: No supported link modes on interface eth1 Mar 05 19:56:12.847776 osdx modulelauncher[734302]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Mar 05 19:56:12.847789 osdx modulelauncher[734302]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Mar 05 19:56:12.848893 osdx modulelauncher[734302]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Mar 05 19:56:12.848902 osdx modulelauncher[734302]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Mar 05 19:56:12.884285 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 05 19:56:12.931385 osdx WARNING[734382]: No supported link modes on interface eth0 Mar 05 19:56:12.932810 osdx modulelauncher[734382]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Mar 05 19:56:12.932824 osdx modulelauncher[734382]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Mar 05 19:56:12.933998 osdx modulelauncher[734382]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Mar 05 19:56:12.934005 osdx modulelauncher[734382]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Mar 05 19:56:12.944212 osdx (udev-worker)[734398]: Network interface NamePolicy= disabled on kernel command line. Mar 05 19:56:12.983178 osdx (udev-worker)[734410]: Network interface NamePolicy= disabled on kernel command line. Mar 05 19:56:13.094881 osdx cfgd[1863]: [733738]Completed change to active configuration Mar 05 19:56:13.110171 osdx OSDxCLI[733738]: User 'admin' committed the configuration. Mar 05 19:56:13.128490 osdx OSDxCLI[733738]: User 'admin' left the configuration menu. Mar 05 19:56:15.354430 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 05 19:56:15.436645 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 05 19:56:15.556715 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 05 19:56:16.035513 osdx file_operation[734577]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 05 19:56:16.057535 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 05 19:56:16.216706 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 05 19:56:16.349514 osdx OSDxCLI[733738]: User 'admin' entered the configuration menu. Mar 05 19:56:16.408684 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 05 19:56:16.504776 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 05 19:56:16.564636 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 05 19:56:16.654638 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 05 19:56:16.706450 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 05 19:56:16.800709 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 05 19:56:16.887187 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 05 19:56:16.954819 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 05 19:56:17.044598 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 05 19:56:17.107817 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 05 19:56:17.203286 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 05 19:56:17.258239 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 05 19:56:17.353407 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 05 19:56:17.407695 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 05 19:56:17.501867 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 05 19:56:17.560504 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 05 19:56:17.646948 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 05 19:56:18.648215 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 05 19:56:18.713200 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 05 19:56:18.815723 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 05 19:56:18.899700 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'show working'. Mar 05 19:56:18.997893 osdx ubnt-cfgd[734630]: inactive Mar 05 19:56:19.074668 osdx INFO[734675]: FRR daemons did not change Mar 05 19:56:19.276722 osdx systemd[1]: Reloading. Mar 05 19:56:19.340357 osdx systemd-sysv-generator[734733]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 05 19:56:19.464593 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 05 19:56:19.481327 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 05 19:56:19.684735 osdx INFO[734712]: Rules successfully loaded Mar 05 19:56:19.685320 osdx cfgd[1863]: [733738]Completed change to active configuration Mar 05 19:56:19.687263 osdx OSDxCLI[733738]: User 'admin' committed the configuration. Mar 05 19:56:20.836435 osdx OSDxCLI[733738]: User 'admin' left the configuration menu. Mar 05 19:56:21.176296 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=49080 DF PROTO=TCP SPT=49974 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 05 19:56:21.176382 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=49843 DF PROTO=TCP SPT=49978 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 05 19:56:31.327560 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Mar 05 19:56:31.434804 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'system journal show | cat'. Mar 05 19:56:31.575002 osdx OSDxCLI[733738]: User 'admin' entered the configuration menu. Mar 05 19:56:31.674888 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 05 19:56:31.733264 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 05 19:56:31.825833 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 05 19:56:31.880681 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 05 19:56:31.973936 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 05 19:56:32.027132 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 05 19:56:32.175111 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 05 19:56:32.243923 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 05 19:56:32.337687 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 05 19:56:32.395957 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 05 19:56:32.498654 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 05 19:56:32.557080 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 05 19:56:32.653141 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 05 19:56:32.718948 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 05 19:56:32.796200 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 05 19:56:32.852183 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 05 19:56:32.946934 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 05 19:56:33.004047 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 05 19:56:33.097100 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 05 19:56:33.153082 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 05 19:56:33.248067 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Mar 05 19:56:33.300072 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Mar 05 19:56:33.400314 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Mar 05 19:56:33.456942 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Mar 05 19:56:33.551150 osdx OSDxCLI[733738]: User 'admin' added a new cfg line: 'show changes'. Mar 05 19:56:33.618479 osdx ubnt-cfgd[734853]: inactive Mar 05 19:56:33.670012 osdx INFO[734873]: FRR daemons did not change Mar 05 19:56:33.923442 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Mar 05 19:56:35.468304 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Mar 05 19:56:35.468427 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Mar 05 19:56:35.468459 osdx systemd[1]: suricata@FW.service: Consumed 1.513s CPU time. Mar 05 19:56:35.500711 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 05 19:56:35.517513 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 05 19:56:35.722913 osdx INFO[734900]: Rules successfully loaded Mar 05 19:56:35.723519 osdx cfgd[1863]: [733738]Completed change to active configuration Mar 05 19:56:35.726105 osdx OSDxCLI[733738]: User 'admin' committed the configuration. Mar 05 19:56:35.745621 osdx OSDxCLI[733738]: User 'admin' left the configuration menu. Mar 05 19:56:35.948298 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=15756 DF PROTO=TCP SPT=54558 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 05 19:56:35.948465 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=60872 DF PROTO=TCP SPT=54574 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 05 19:56:41.039267 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Mar 05 19:56:46.343586 osdx OSDxCLI[733738]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 24358 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.706 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.706/0.706/0.706/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.638 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.638/0.638/0.638/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 35882 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 148 MBytes 1.24 Gbits/sec 181 1.76 MBytes [ 5] 1.00-2.00 sec 231 MBytes 1.94 Gbits/sec 109 1.37 MBytes [ 5] 2.00-3.00 sec 274 MBytes 2.30 Gbits/sec 3 1.09 MBytes [ 5] 3.00-4.00 sec 326 MBytes 2.74 Gbits/sec 0 1.29 MBytes [ 5] 4.00-5.00 sec 318 MBytes 2.66 Gbits/sec 0 1.46 MBytes [ 5] 5.00-6.00 sec 265 MBytes 2.22 Gbits/sec 0 1.58 MBytes [ 5] 6.00-7.00 sec 285 MBytes 2.39 Gbits/sec 0 1.70 MBytes [ 5] 7.00-8.00 sec 311 MBytes 2.61 Gbits/sec 12 1.34 MBytes [ 5] 8.00-9.00 sec 340 MBytes 2.85 Gbits/sec 0 1.51 MBytes [ 5] 9.00-10.00 sec 298 MBytes 2.50 Gbits/sec 0 1.64 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec 305 sender [ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:57:12.986184 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35878 -> 40.0.0.2:5001 03/05/2026-19:57:12.987508 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35882 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=140 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=140 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=822 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=822 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=35882 dport=5001 packets=2023987 bytes=3035968193 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=35882 packets=234864 bytes=12205992 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=35878 dport=5001 packets=14 bytes=1199 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=35878 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59218 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 329 MBytes 2.76 Gbits/sec 36 1.87 MBytes [ 5] 1.00-2.00 sec 338 MBytes 2.83 Gbits/sec 70 1.45 MBytes [ 5] 2.00-3.00 sec 340 MBytes 2.85 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 352 MBytes 2.96 Gbits/sec 0 1.76 MBytes [ 5] 4.00-5.00 sec 325 MBytes 2.73 Gbits/sec 0 1.89 MBytes [ 5] 5.00-6.00 sec 332 MBytes 2.79 Gbits/sec 0 2.02 MBytes [ 5] 6.00-7.00 sec 326 MBytes 2.73 Gbits/sec 0 2.13 MBytes [ 5] 7.00-8.00 sec 310 MBytes 2.61 Gbits/sec 578 1.10 MBytes [ 5] 8.00-9.00 sec 318 MBytes 2.66 Gbits/sec 0 1.30 MBytes [ 5] 9.00-10.00 sec 272 MBytes 2.29 Gbits/sec 0 1.44 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.17 GBytes 2.72 Gbits/sec 684 sender [ 5] 0.00-10.00 sec 3.16 GBytes 2.72 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:57:12.986184 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35878 -> 40.0.0.2:5001 03/05/2026-19:57:12.987508 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35882 -> 40.0.0.2:5001 03/05/2026-19:57:27.826201 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59212 -> 40.0.0.2:5001 03/05/2026-19:57:27.827141 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59218 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=59212 dport=5001 packets=16 bytes=1303 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59212 packets=13 bytes=1016 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=59218 dport=5001 packets=2347400 bytes=3521087109 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59218 packets=275529 bytes=14329076 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 3382 0 --:--:-- --:--:-- --:--:-- 3389
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.71 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.714/1.714/1.714/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.765 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.765/0.765/0.765/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/05/2026-19:58:00.812837 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41052 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=41052 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=41052 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=825 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=825 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=143 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=143 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 10 445 814 2 - 5 5 369 369 ------------------------------------------------------------------ Total 10 10 814 814
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/05/2026-19:58:00.812837 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41052 -> 40.0.0.2:5000 03/05/2026-19:58:06.951146 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41052 -> 40.0.0.2:5000 03/05/2026-19:58:08.454480 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54028 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=54028 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=54028 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 445 775 2 - 5 5 330 330 --------------------------------------------------------------------- Total 10 10 775 775
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19187 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.647 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.647/0.647/0.647/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.662 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.662/0.662/0.662/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/05/2026-19:58:32.249008 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34044 -> 40.0.0.2:5001 03/05/2026-19:58:32.251296 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34050 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=34050 dport=5001 packets=9732 bytes=14593701 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34050 packets=1246 bytes=64800 [ASSURED] [OFFLOAD, packets=9719 bytes=14578500 packets=1244 bytes=64688] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=34044 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34044 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=828 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=828 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=146 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=146 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 1446 0 --:--:-- --:--:-- --:--:-- 1461
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
03/05/2026-19:58:32.249008 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34044 -> 40.0.0.2:5001 03/05/2026-19:58:32.251296 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34050 -> 40.0.0.2:5001 03/05/2026-19:58:35.096879 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44738 -> 40.0.0.2:5001 03/05/2026-19:58:35.098307 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:49383 -> 40.0.0.2:5001 03/05/2026-19:58:35.109177 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:49383 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
udp 17 src=20.0.0.2 dst=40.0.0.2 sport=49383 dport=5001 packets=10 bytes=13316 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=49383 packets=1 bytes=32 [OFFLOAD, packets=6 bytes=8856 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=44738 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44738 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 37112 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.777 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.777/0.777/0.777/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.563 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.563/0.563/0.563/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/05/2026-19:58:59.672896 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57304 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 57304 yes 201 0 0 0 20.0.0.2 40.0.0.2 57304 5000 yes 201 0 6 544
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250379s ending at 1772740743.360772 XDP_DROP 7 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
03/05/2026-19:58:59.672896 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57304 -> 40.0.0.2:5000 03/05/2026-19:59:03.555094 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:35202 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 57304 yes 201 0 0 0 40.0.0.2 20.0.0.2 5001 35202 no 201 0 0 0 20.0.0.2 40.0.0.2 57304 5000 yes 201 0 10 776 20.0.0.2 40.0.0.2 35202 5001 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250163s ending at 1772740747.378883 XDP_DROP 10 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)