Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 22517 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.615 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.615/0.615/0.615/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.509 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.509/0.509/0.509/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50208 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 48.9 MBytes 410 Mbits/sec 0 1.66 MBytes [ 5] 1.00-2.00 sec 115 MBytes 965 Mbits/sec 21 1.30 MBytes [ 5] 2.00-3.00 sec 109 MBytes 912 Mbits/sec 0 1.37 MBytes [ 5] 3.00-4.00 sec 149 MBytes 1.25 Gbits/sec 0 1.44 MBytes [ 5] 4.00-5.00 sec 155 MBytes 1.30 Gbits/sec 0 1.51 MBytes [ 5] 5.00-6.00 sec 154 MBytes 1.29 Gbits/sec 0 1.59 MBytes [ 5] 6.00-7.00 sec 155 MBytes 1.30 Gbits/sec 0 1.66 MBytes [ 5] 7.00-8.00 sec 151 MBytes 1.27 Gbits/sec 222 1.23 MBytes [ 5] 8.00-9.00 sec 151 MBytes 1.27 Gbits/sec 0 1.37 MBytes [ 5] 9.00-10.00 sec 152 MBytes 1.28 Gbits/sec 0 1.47 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.31 GBytes 1.12 Gbits/sec 243 sender [ 5] 0.00-10.01 sec 1.31 GBytes 1.12 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:02:51.461129 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50206 -> 40.0.0.2:5001 12/11/2025-20:02:51.462081 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50208 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 3350 0 --:--:-- --:--:-- --:--:-- 3394
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.546 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.546/0.546/0.546/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.462 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.462/0.462/0.462/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51020 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 49.0 MBytes 411 Mbits/sec 0 1.53 MBytes [ 5] 1.00-2.00 sec 249 MBytes 2.09 Gbits/sec 74 1.81 MBytes [ 5] 2.00-3.00 sec 261 MBytes 2.19 Gbits/sec 0 1.95 MBytes [ 5] 3.00-4.00 sec 278 MBytes 2.33 Gbits/sec 0 2.07 MBytes [ 5] 4.00-5.00 sec 268 MBytes 2.24 Gbits/sec 0 2.17 MBytes [ 5] 5.00-6.00 sec 272 MBytes 2.29 Gbits/sec 53 1.57 MBytes [ 5] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec 0 1.69 MBytes [ 5] 7.00-8.00 sec 258 MBytes 2.16 Gbits/sec 0 1.80 MBytes [ 5] 8.00-9.00 sec 232 MBytes 1.95 Gbits/sec 46 1.36 MBytes [ 5] 9.00-10.00 sec 251 MBytes 2.11 Gbits/sec 0 1.49 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.34 GBytes 2.01 Gbits/sec 173 sender [ 5] 0.00-10.00 sec 2.34 GBytes 2.01 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:03:27.592759 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51006 -> 40.0.0.2:5001 12/11/2025-20:03:27.593881 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51020 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Dec 11 20:03:19.334965 osdx systemd-journald[265387]: Runtime Journal (/run/log/journal/ff8de1b7feed4cd0a923a6e53f75b1b9) is 1.7M, max 13.8M, 12.0M free. Dec 11 20:03:19.338514 osdx systemd-journald[265387]: Received client request to rotate journal, rotating. Dec 11 20:03:19.338589 osdx systemd-journald[265387]: Vacuuming done, freed 0B of archived journals from /run/log/journal/ff8de1b7feed4cd0a923a6e53f75b1b9. Dec 11 20:03:19.347201 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'system journal clear'. Dec 11 20:03:19.564602 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'system coredump delete all'. Dec 11 20:03:19.813504 osdx OSDxCLI[745927]: User 'admin' entered the configuration menu. Dec 11 20:03:19.899474 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Dec 11 20:03:19.968041 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Dec 11 20:03:20.062990 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Dec 11 20:03:20.124620 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Dec 11 20:03:20.234410 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Dec 11 20:03:20.300892 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Dec 11 20:03:20.413466 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'show working'. Dec 11 20:03:20.477785 osdx ubnt-cfgd[746424]: inactive Dec 11 20:03:20.510041 osdx INFO[746430]: FRR daemons did not change Dec 11 20:03:20.538506 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Dec 11 20:03:20.557738 osdx WARNING[746470]: No supported link modes on interface eth1 Dec 11 20:03:20.559183 osdx modulelauncher[746470]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Dec 11 20:03:20.559195 osdx modulelauncher[746470]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Dec 11 20:03:20.560410 osdx modulelauncher[746470]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Dec 11 20:03:20.560420 osdx modulelauncher[746470]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 11 20:03:20.602489 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Dec 11 20:03:20.646110 osdx WARNING[746550]: No supported link modes on interface eth0 Dec 11 20:03:20.647541 osdx modulelauncher[746550]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Dec 11 20:03:20.647555 osdx modulelauncher[746550]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Dec 11 20:03:20.648753 osdx modulelauncher[746550]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Dec 11 20:03:20.648763 osdx modulelauncher[746550]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 11 20:03:20.657778 osdx (udev-worker)[746566]: Network interface NamePolicy= disabled on kernel command line. Dec 11 20:03:20.696591 osdx (udev-worker)[746579]: Network interface NamePolicy= disabled on kernel command line. Dec 11 20:03:20.813609 osdx cfgd[1647]: [745927]Completed change to active configuration Dec 11 20:03:20.827883 osdx OSDxCLI[745927]: User 'admin' committed the configuration. Dec 11 20:03:20.856564 osdx OSDxCLI[745927]: User 'admin' left the configuration menu. Dec 11 20:03:23.194220 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Dec 11 20:03:23.285617 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Dec 11 20:03:23.376062 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Dec 11 20:03:23.916516 osdx file_operation[746745]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Dec 11 20:03:23.977055 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Dec 11 20:03:24.197092 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Dec 11 20:03:24.353514 osdx OSDxCLI[745927]: User 'admin' entered the configuration menu. Dec 11 20:03:24.424620 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 11 20:03:24.526604 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 11 20:03:24.588928 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 11 20:03:24.690537 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 11 20:03:24.778122 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 11 20:03:24.867593 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 11 20:03:25.032598 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 11 20:03:25.092354 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 11 20:03:25.198832 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 11 20:03:25.265104 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 11 20:03:25.382013 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 11 20:03:25.465882 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 11 20:03:25.562086 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 11 20:03:25.635112 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 11 20:03:25.739867 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 11 20:03:25.799319 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 11 20:03:25.889877 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 11 20:03:26.084191 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 11 20:03:26.140631 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 11 20:03:26.276474 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 11 20:03:26.359438 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'show working'. Dec 11 20:03:26.454896 osdx ubnt-cfgd[746798]: inactive Dec 11 20:03:26.543684 osdx INFO[746843]: FRR daemons did not change Dec 11 20:03:26.764635 osdx systemd[1]: Reloading. Dec 11 20:03:26.798497 osdx systemd-sysv-generator[746899]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Dec 11 20:03:26.930799 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 11 20:03:26.946859 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 11 20:03:27.136720 osdx INFO[746880]: Rules successfully loaded Dec 11 20:03:27.137345 osdx cfgd[1647]: [745927]Completed change to active configuration Dec 11 20:03:27.139580 osdx OSDxCLI[745927]: User 'admin' committed the configuration. Dec 11 20:03:27.183874 osdx OSDxCLI[745927]: User 'admin' left the configuration menu. Dec 11 20:03:27.594502 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=57970 DF PROTO=TCP SPT=51006 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 11 20:03:27.594584 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=53016 DF PROTO=TCP SPT=51020 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 11 20:03:37.908741 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 55272 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 223 MBytes 1.87 Gbits/sec 140 1.65 MBytes [ 5] 1.00-2.00 sec 212 MBytes 1.78 Gbits/sec 57 1.25 MBytes [ 5] 2.00-3.00 sec 244 MBytes 2.05 Gbits/sec 0 1.38 MBytes [ 5] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec 0 1.51 MBytes [ 5] 4.00-5.00 sec 270 MBytes 2.26 Gbits/sec 0 1.64 MBytes [ 5] 5.00-6.00 sec 271 MBytes 2.28 Gbits/sec 0 1.75 MBytes [ 5] 6.00-7.00 sec 252 MBytes 2.12 Gbits/sec 241 1.36 MBytes [ 5] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 1.49 MBytes [ 5] 8.00-9.00 sec 265 MBytes 2.22 Gbits/sec 0 1.62 MBytes [ 5] 9.00-10.00 sec 278 MBytes 2.33 Gbits/sec 0 1.74 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.50 GBytes 2.14 Gbits/sec 438 sender [ 5] 0.00-10.00 sec 2.49 GBytes 2.14 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:03:27.592759 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51006 -> 40.0.0.2:5001 12/11/2025-20:03:27.593881 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51020 -> 40.0.0.2:5001 12/11/2025-20:03:43.511686 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55264 -> 40.0.0.2:5001 12/11/2025-20:03:43.512656 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55272 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Dec 11 20:03:19.334965 osdx systemd-journald[265387]: Runtime Journal (/run/log/journal/ff8de1b7feed4cd0a923a6e53f75b1b9) is 1.7M, max 13.8M, 12.0M free. Dec 11 20:03:19.338514 osdx systemd-journald[265387]: Received client request to rotate journal, rotating. Dec 11 20:03:19.338589 osdx systemd-journald[265387]: Vacuuming done, freed 0B of archived journals from /run/log/journal/ff8de1b7feed4cd0a923a6e53f75b1b9. Dec 11 20:03:19.347201 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'system journal clear'. Dec 11 20:03:19.564602 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'system coredump delete all'. Dec 11 20:03:19.813504 osdx OSDxCLI[745927]: User 'admin' entered the configuration menu. Dec 11 20:03:19.899474 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Dec 11 20:03:19.968041 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Dec 11 20:03:20.062990 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Dec 11 20:03:20.124620 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Dec 11 20:03:20.234410 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Dec 11 20:03:20.300892 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Dec 11 20:03:20.413466 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'show working'. Dec 11 20:03:20.477785 osdx ubnt-cfgd[746424]: inactive Dec 11 20:03:20.510041 osdx INFO[746430]: FRR daemons did not change Dec 11 20:03:20.538506 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Dec 11 20:03:20.557738 osdx WARNING[746470]: No supported link modes on interface eth1 Dec 11 20:03:20.559183 osdx modulelauncher[746470]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Dec 11 20:03:20.559195 osdx modulelauncher[746470]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Dec 11 20:03:20.560410 osdx modulelauncher[746470]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Dec 11 20:03:20.560420 osdx modulelauncher[746470]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 11 20:03:20.602489 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Dec 11 20:03:20.646110 osdx WARNING[746550]: No supported link modes on interface eth0 Dec 11 20:03:20.647541 osdx modulelauncher[746550]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Dec 11 20:03:20.647555 osdx modulelauncher[746550]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Dec 11 20:03:20.648753 osdx modulelauncher[746550]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Dec 11 20:03:20.648763 osdx modulelauncher[746550]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 11 20:03:20.657778 osdx (udev-worker)[746566]: Network interface NamePolicy= disabled on kernel command line. Dec 11 20:03:20.696591 osdx (udev-worker)[746579]: Network interface NamePolicy= disabled on kernel command line. Dec 11 20:03:20.813609 osdx cfgd[1647]: [745927]Completed change to active configuration Dec 11 20:03:20.827883 osdx OSDxCLI[745927]: User 'admin' committed the configuration. Dec 11 20:03:20.856564 osdx OSDxCLI[745927]: User 'admin' left the configuration menu. Dec 11 20:03:23.194220 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Dec 11 20:03:23.285617 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Dec 11 20:03:23.376062 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Dec 11 20:03:23.916516 osdx file_operation[746745]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Dec 11 20:03:23.977055 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Dec 11 20:03:24.197092 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Dec 11 20:03:24.353514 osdx OSDxCLI[745927]: User 'admin' entered the configuration menu. Dec 11 20:03:24.424620 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 11 20:03:24.526604 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 11 20:03:24.588928 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 11 20:03:24.690537 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 11 20:03:24.778122 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 11 20:03:24.867593 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 11 20:03:25.032598 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 11 20:03:25.092354 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 11 20:03:25.198832 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 11 20:03:25.265104 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 11 20:03:25.382013 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 11 20:03:25.465882 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 11 20:03:25.562086 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 11 20:03:25.635112 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 11 20:03:25.739867 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 11 20:03:25.799319 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 11 20:03:25.889877 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 11 20:03:26.084191 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 11 20:03:26.140631 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 11 20:03:26.276474 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 11 20:03:26.359438 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'show working'. Dec 11 20:03:26.454896 osdx ubnt-cfgd[746798]: inactive Dec 11 20:03:26.543684 osdx INFO[746843]: FRR daemons did not change Dec 11 20:03:26.764635 osdx systemd[1]: Reloading. Dec 11 20:03:26.798497 osdx systemd-sysv-generator[746899]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Dec 11 20:03:26.930799 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 11 20:03:26.946859 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 11 20:03:27.136720 osdx INFO[746880]: Rules successfully loaded Dec 11 20:03:27.137345 osdx cfgd[1647]: [745927]Completed change to active configuration Dec 11 20:03:27.139580 osdx OSDxCLI[745927]: User 'admin' committed the configuration. Dec 11 20:03:27.183874 osdx OSDxCLI[745927]: User 'admin' left the configuration menu. Dec 11 20:03:27.594502 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=57970 DF PROTO=TCP SPT=51006 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 11 20:03:27.594584 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=53016 DF PROTO=TCP SPT=51020 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 11 20:03:37.908741 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Dec 11 20:03:38.042444 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'system journal show | cat'. Dec 11 20:03:38.398717 osdx OSDxCLI[745927]: User 'admin' entered the configuration menu. Dec 11 20:03:38.466854 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 11 20:03:38.608177 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 11 20:03:38.723493 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 11 20:03:38.897598 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 11 20:03:38.950698 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 11 20:03:39.053560 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 11 20:03:39.162787 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 11 20:03:39.239310 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 11 20:03:39.351910 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 11 20:03:39.420861 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 11 20:03:39.511550 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 11 20:03:39.775165 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 11 20:03:39.872930 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 11 20:03:40.010186 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 11 20:03:40.123073 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 11 20:03:40.232227 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 11 20:03:40.352512 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 11 20:03:40.495196 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 11 20:03:40.564823 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 11 20:03:40.673409 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 11 20:03:40.752654 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Dec 11 20:03:40.846326 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Dec 11 20:03:40.962379 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Dec 11 20:03:41.055161 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Dec 11 20:03:41.157444 osdx OSDxCLI[745927]: User 'admin' added a new cfg line: 'show changes'. Dec 11 20:03:41.259121 osdx ubnt-cfgd[747026]: inactive Dec 11 20:03:41.352680 osdx INFO[747046]: FRR daemons did not change Dec 11 20:03:41.605074 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Dec 11 20:03:42.944955 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Dec 11 20:03:42.945078 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Dec 11 20:03:42.945109 osdx systemd[1]: suricata@FW.service: Consumed 1.304s CPU time. Dec 11 20:03:42.958831 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 11 20:03:42.975544 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 11 20:03:43.206097 osdx INFO[747073]: Rules successfully loaded Dec 11 20:03:43.206646 osdx cfgd[1647]: [745927]Completed change to active configuration Dec 11 20:03:43.208724 osdx OSDxCLI[745927]: User 'admin' committed the configuration. Dec 11 20:03:43.230727 osdx OSDxCLI[745927]: User 'admin' left the configuration menu. Dec 11 20:03:43.514684 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14460 DF PROTO=TCP SPT=55264 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Dec 11 20:03:43.514771 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=52018 DF PROTO=TCP SPT=55272 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Dec 11 20:03:49.031600 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Dec 11 20:03:53.705274 osdx OSDxCLI[745927]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19337 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.558 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.558/0.558/0.558/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.489 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.489/0.489/0.489/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 48872 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 80.4 MBytes 674 Mbits/sec 4 1.76 MBytes [ 5] 1.00-2.00 sec 251 MBytes 2.11 Gbits/sec 0 1.95 MBytes [ 5] 2.00-3.00 sec 276 MBytes 2.32 Gbits/sec 0 2.10 MBytes [ 5] 3.00-4.00 sec 284 MBytes 2.38 Gbits/sec 7 1.58 MBytes [ 5] 4.00-5.00 sec 270 MBytes 2.26 Gbits/sec 0 1.70 MBytes [ 5] 5.00-6.00 sec 281 MBytes 2.36 Gbits/sec 0 1.82 MBytes [ 5] 6.00-7.00 sec 279 MBytes 2.34 Gbits/sec 0 1.93 MBytes [ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec 0 2.03 MBytes [ 5] 8.00-9.00 sec 271 MBytes 2.28 Gbits/sec 0 2.13 MBytes [ 5] 9.00-10.00 sec 261 MBytes 2.19 Gbits/sec 11 1.63 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.47 GBytes 2.12 Gbits/sec 22 sender [ 5] 0.00-10.00 sec 2.47 GBytes 2.12 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:04:15.971922 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48862 -> 40.0.0.2:5001 12/11/2025-20:04:15.972808 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48872 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
nfct_labelmap_new: No such file or directory tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=48872 dport=5001 packets=1834200 bytes=2751290381 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48872 packets=305174 bytes=15861148 [ASSURED] (Sc: not-bypass) mark=129834765 conntag=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=48862 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48862 packets=13 bytes=1016 [ASSURED] (Sc: not-bypass) mark=129834765 conntag=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=110 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=110 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=785 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=785 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 52194 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 203 MBytes 1.70 Gbits/sec 104 1.85 MBytes [ 5] 1.00-2.00 sec 271 MBytes 2.28 Gbits/sec 0 2.02 MBytes [ 5] 2.00-3.00 sec 258 MBytes 2.16 Gbits/sec 0 2.16 MBytes [ 5] 3.00-4.00 sec 259 MBytes 2.17 Gbits/sec 30 1.61 MBytes [ 5] 4.00-5.00 sec 236 MBytes 1.98 Gbits/sec 0 1.71 MBytes [ 5] 5.00-6.00 sec 248 MBytes 2.08 Gbits/sec 0 1.81 MBytes [ 5] 6.00-7.00 sec 236 MBytes 1.98 Gbits/sec 289 1.35 MBytes [ 5] 7.00-8.00 sec 222 MBytes 1.87 Gbits/sec 0 1.50 MBytes [ 5] 8.00-9.00 sec 241 MBytes 2.02 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 226 MBytes 1.90 Gbits/sec 0 1.70 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.34 GBytes 2.01 Gbits/sec 423 sender [ 5] 0.00-10.00 sec 2.34 GBytes 2.01 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:04:15.971922 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48862 -> 40.0.0.2:5001 12/11/2025-20:04:15.972808 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48872 -> 40.0.0.2:5001 12/11/2025-20:04:30.777593 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52184 -> 40.0.0.2:5001 12/11/2025-20:04:30.778498 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52194 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
nfct_labelmap_new: No such file or directory tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=52194 dport=5001 packets=1738003 bytes=2606987497 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52194 packets=247584 bytes=12867824 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 conntag=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=52184 dport=5001 packets=16 bytes=1299 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52184 packets=13 bytes=1015 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 conntag=0 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 3681 0 --:--:-- --:--:-- --:--:-- 3703
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.575 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.575/0.575/0.575/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.526 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.526/0.526/0.526/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/11/2025-20:05:02.920864 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51002 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
nfct_labelmap_new: No such file or directory icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=113 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=113 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=788 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=788 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=51002 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=51002 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 conntag=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/11/2025-20:05:02.920864 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51002 -> 40.0.0.2:5000 12/11/2025-20:05:09.271477 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51002 -> 40.0.0.2:5000 12/11/2025-20:05:10.776809 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36924 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
nfct_labelmap_new: No such file or directory tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=36924 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=36924 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 conntag=0 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19102 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.682 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.682/0.682/0.682/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.552 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.552/0.552/0.552/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/11/2025-20:05:34.316923 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49276 -> 40.0.0.2:5001 12/11/2025-20:05:34.318177 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49288 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
nfct_labelmap_new: No such file or directory tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=49276 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=49276 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 conntag=0 use=3 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=116 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=116 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=791 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=791 packets=1 bytes=84 (Sc: not-bypass) mark=0 conntag=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=49288 dport=5001 packets=9506 bytes=14253253 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=49288 packets=1417 bytes=73692 [ASSURED] [OFFLOAD, packets=9493 bytes=14238052 packets=1415 bytes=73580] mark=129834765 conntag=0 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 4465 0 --:--:-- --:--:-- --:--:-- 4586
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
12/11/2025-20:05:34.316923 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49276 -> 40.0.0.2:5001 12/11/2025-20:05:34.318177 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49288 -> 40.0.0.2:5001 12/11/2025-20:05:37.498501 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54332 -> 40.0.0.2:5001 12/11/2025-20:05:37.500383 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52070 -> 40.0.0.2:5001 12/11/2025-20:05:37.510972 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52070 -> 40.0.0.2:5001 12/11/2025-20:05:37.522941 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52070 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
nfct_labelmap_new: No such file or directory udp 17 src=20.0.0.2 dst=40.0.0.2 sport=52070 dport=5001 packets=12 bytes=16268 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52070 packets=1 bytes=32 [OFFLOAD, packets=8 bytes=11808 packets=0 bytes=0] mark=129834765 conntag=0 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=54332 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=54332 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 conntag=0 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 28425 0 --:--:-- --:--:-- --:--:-- 28571
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.953 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.953/0.953/0.953/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.614 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.614/0.614/0.614/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/11/2025-20:05:57.128994 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52202 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 52202 5000 yes 201 0 7 615 40.0.0.2 20.0.0.2 5000 52202 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250155s ending at 1765483560.753449 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9412273 pkts ( 0 pps) 12157262 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
12/11/2025-20:05:57.128994 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52202 -> 40.0.0.2:5000 12/11/2025-20:06:00.917340 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:51988 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 52202 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5001 51988 no 201 0 0 0 20.0.0.2 40.0.0.2 51988 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 52202 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250356s ending at 1765483564.550732 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9412313 pkts ( 0 pps) 12157265 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)