Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 29799 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.539 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.539/0.539/0.539/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.556 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.556/0.556/0.556/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 46816 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 53.2 MBytes 447 Mbits/sec 0 2.31 MBytes [ 5] 1.00-2.00 sec 101 MBytes 849 Mbits/sec 57 1.86 MBytes [ 5] 2.00-3.00 sec 132 MBytes 1.11 Gbits/sec 33 1.41 MBytes [ 5] 3.00-4.00 sec 142 MBytes 1.20 Gbits/sec 0 1.50 MBytes [ 5] 4.00-5.00 sec 139 MBytes 1.16 Gbits/sec 0 1.57 MBytes [ 5] 5.00-6.00 sec 141 MBytes 1.18 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 140 MBytes 1.17 Gbits/sec 0 1.66 MBytes [ 5] 7.00-8.00 sec 141 MBytes 1.18 Gbits/sec 0 1.73 MBytes [ 5] 8.00-9.00 sec 145 MBytes 1.22 Gbits/sec 0 1.79 MBytes [ 5] 9.00-10.00 sec 140 MBytes 1.17 Gbits/sec 12 1.37 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.25 GBytes 1.07 Gbits/sec 102 sender [ 5] 0.00-10.00 sec 1.24 GBytes 1.07 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:08:48.309731 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46812 -> 40.0.0.2:5001 12/17/2025-21:08:48.310772 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46816 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 2804 0 --:--:-- --:--:-- --:--:-- 2866
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.627 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.627/0.627/0.627/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.550 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.550/0.550/0.550/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57460 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 78.6 MBytes 659 Mbits/sec 24 1.70 MBytes [ 5] 1.00-2.00 sec 141 MBytes 1.18 Gbits/sec 45 1.29 MBytes [ 5] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 1.42 MBytes [ 5] 3.00-4.00 sec 298 MBytes 2.50 Gbits/sec 0 1.57 MBytes [ 5] 4.00-5.00 sec 311 MBytes 2.61 Gbits/sec 0 1.71 MBytes [ 5] 5.00-6.00 sec 319 MBytes 2.67 Gbits/sec 0 1.84 MBytes [ 5] 6.00-7.00 sec 315 MBytes 2.64 Gbits/sec 0 1.96 MBytes [ 5] 7.00-8.00 sec 311 MBytes 2.61 Gbits/sec 0 2.07 MBytes [ 5] 8.00-9.00 sec 305 MBytes 2.56 Gbits/sec 31 1.54 MBytes [ 5] 9.00-10.00 sec 291 MBytes 2.44 Gbits/sec 0 1.70 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.59 GBytes 2.22 Gbits/sec 100 sender [ 5] 0.00-10.00 sec 2.59 GBytes 2.22 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:09:27.414370 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57450 -> 40.0.0.2:5001 12/17/2025-21:09:27.417316 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57460 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Dec 17 21:09:17.000710 osdx systemd-timedated[444379]: Changed local time to Wed 2025-12-17 21:09:17 UTC Dec 17 21:09:17.001586 osdx systemd-journald[123332]: Time jumped backwards, rotating. Dec 17 21:09:17.002067 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'set date 2025-12-17 21:09:17'. Dec 17 21:09:17.520656 osdx systemd-journald[123332]: Runtime Journal (/run/log/journal/a189e667ab9f46898dbfc92a68a94f73) is 1.9M, max 13.8M, 11.8M free. Dec 17 21:09:17.521585 osdx systemd-journald[123332]: Received client request to rotate journal, rotating. Dec 17 21:09:17.521648 osdx systemd-journald[123332]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a189e667ab9f46898dbfc92a68a94f73. Dec 17 21:09:17.532841 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'system journal clear'. Dec 17 21:09:17.775347 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'system coredump delete all'. Dec 17 21:09:18.073153 osdx OSDxCLI[443939]: User 'admin' entered the configuration menu. Dec 17 21:09:18.502038 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Dec 17 21:09:18.601770 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Dec 17 21:09:18.684648 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Dec 17 21:09:18.807285 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Dec 17 21:09:18.896507 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Dec 17 21:09:18.989884 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Dec 17 21:09:19.057222 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'show working'. Dec 17 21:09:19.289117 osdx ubnt-cfgd[444412]: inactive Dec 17 21:09:19.322128 osdx INFO[444418]: FRR daemons did not change Dec 17 21:09:19.345600 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Dec 17 21:09:19.367897 osdx WARNING[444457]: No supported link modes on interface eth1 Dec 17 21:09:19.369531 osdx modulelauncher[444457]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Dec 17 21:09:19.369552 osdx modulelauncher[444457]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Dec 17 21:09:19.371001 osdx modulelauncher[444457]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Dec 17 21:09:19.371011 osdx modulelauncher[444457]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 17 21:09:19.405590 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Dec 17 21:09:19.460012 osdx WARNING[444536]: No supported link modes on interface eth0 Dec 17 21:09:19.461820 osdx modulelauncher[444536]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Dec 17 21:09:19.461836 osdx modulelauncher[444536]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Dec 17 21:09:19.463396 osdx modulelauncher[444536]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Dec 17 21:09:19.463407 osdx modulelauncher[444536]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 17 21:09:19.475742 osdx (udev-worker)[444552]: Network interface NamePolicy= disabled on kernel command line. Dec 17 21:09:19.658845 osdx cfgd[1460]: [443939]Completed change to active configuration Dec 17 21:09:20.135775 osdx OSDxCLI[443939]: User 'admin' committed the configuration. Dec 17 21:09:20.195314 osdx OSDxCLI[443939]: User 'admin' left the configuration menu. Dec 17 21:09:22.743596 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Dec 17 21:09:22.820705 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Dec 17 21:09:22.927383 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Dec 17 21:09:23.462693 osdx file_operation[444715]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Dec 17 21:09:23.529803 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Dec 17 21:09:23.695023 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Dec 17 21:09:23.855307 osdx OSDxCLI[443939]: User 'admin' entered the configuration menu. Dec 17 21:09:23.918150 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 17 21:09:24.020771 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 17 21:09:24.137981 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 17 21:09:24.252686 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 17 21:09:24.363489 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 17 21:09:24.459770 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 17 21:09:24.617609 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 17 21:09:24.690457 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 17 21:09:24.805552 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 17 21:09:24.871962 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 17 21:09:24.990169 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 17 21:09:25.049658 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 17 21:09:25.168611 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 17 21:09:25.282407 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 17 21:09:25.399291 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 17 21:09:25.496393 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 17 21:09:25.591390 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 17 21:09:25.725821 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 17 21:09:25.810549 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 17 21:09:25.909884 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 17 21:09:25.999437 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'show working'. Dec 17 21:09:26.098684 osdx ubnt-cfgd[444766]: inactive Dec 17 21:09:26.186610 osdx INFO[444811]: FRR daemons did not change Dec 17 21:09:26.419117 osdx systemd[1]: Reloading. Dec 17 21:09:26.461596 osdx systemd-sysv-generator[444867]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Dec 17 21:09:26.593992 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 17 21:09:26.618579 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 17 21:09:26.901544 osdx INFO[444848]: Rules successfully loaded Dec 17 21:09:26.902608 osdx cfgd[1460]: [443939]Completed change to active configuration Dec 17 21:09:26.970147 osdx OSDxCLI[443939]: User 'admin' committed the configuration. Dec 17 21:09:26.992774 osdx OSDxCLI[443939]: User 'admin' left the configuration menu. Dec 17 21:09:27.419226 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=16934 DF PROTO=TCP SPT=57450 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 17 21:09:27.419317 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19998 DF PROTO=TCP SPT=57460 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 17 21:09:37.635793 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33886 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 299 MBytes 2.51 Gbits/sec 84 1.37 MBytes [ 5] 1.00-2.00 sec 300 MBytes 2.52 Gbits/sec 0 1.52 MBytes [ 5] 2.00-3.00 sec 319 MBytes 2.67 Gbits/sec 0 1.67 MBytes [ 5] 3.00-4.00 sec 328 MBytes 2.75 Gbits/sec 0 1.81 MBytes [ 5] 4.00-5.00 sec 304 MBytes 2.55 Gbits/sec 0 1.92 MBytes [ 5] 5.00-6.00 sec 291 MBytes 2.44 Gbits/sec 5 1.43 MBytes [ 5] 6.00-7.00 sec 271 MBytes 2.28 Gbits/sec 19 1.13 MBytes [ 5] 7.00-8.00 sec 292 MBytes 2.45 Gbits/sec 0 1.30 MBytes [ 5] 8.00-9.00 sec 308 MBytes 2.58 Gbits/sec 0 1.47 MBytes [ 5] 9.00-10.00 sec 286 MBytes 2.40 Gbits/sec 0 1.60 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.93 GBytes 2.51 Gbits/sec 108 sender [ 5] 0.00-10.05 sec 2.93 GBytes 2.50 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:09:27.414370 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57450 -> 40.0.0.2:5001 12/17/2025-21:09:27.417316 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57460 -> 40.0.0.2:5001 12/17/2025-21:09:43.366980 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33880 -> 40.0.0.2:5001 12/17/2025-21:09:43.368038 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33886 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Dec 17 21:09:17.000710 osdx systemd-timedated[444379]: Changed local time to Wed 2025-12-17 21:09:17 UTC Dec 17 21:09:17.001586 osdx systemd-journald[123332]: Time jumped backwards, rotating. Dec 17 21:09:17.002067 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'set date 2025-12-17 21:09:17'. Dec 17 21:09:17.520656 osdx systemd-journald[123332]: Runtime Journal (/run/log/journal/a189e667ab9f46898dbfc92a68a94f73) is 1.9M, max 13.8M, 11.8M free. Dec 17 21:09:17.521585 osdx systemd-journald[123332]: Received client request to rotate journal, rotating. Dec 17 21:09:17.521648 osdx systemd-journald[123332]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a189e667ab9f46898dbfc92a68a94f73. Dec 17 21:09:17.532841 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'system journal clear'. Dec 17 21:09:17.775347 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'system coredump delete all'. Dec 17 21:09:18.073153 osdx OSDxCLI[443939]: User 'admin' entered the configuration menu. Dec 17 21:09:18.502038 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Dec 17 21:09:18.601770 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Dec 17 21:09:18.684648 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Dec 17 21:09:18.807285 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Dec 17 21:09:18.896507 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Dec 17 21:09:18.989884 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Dec 17 21:09:19.057222 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'show working'. Dec 17 21:09:19.289117 osdx ubnt-cfgd[444412]: inactive Dec 17 21:09:19.322128 osdx INFO[444418]: FRR daemons did not change Dec 17 21:09:19.345600 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Dec 17 21:09:19.367897 osdx WARNING[444457]: No supported link modes on interface eth1 Dec 17 21:09:19.369531 osdx modulelauncher[444457]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Dec 17 21:09:19.369552 osdx modulelauncher[444457]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Dec 17 21:09:19.371001 osdx modulelauncher[444457]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Dec 17 21:09:19.371011 osdx modulelauncher[444457]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 17 21:09:19.405590 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Dec 17 21:09:19.460012 osdx WARNING[444536]: No supported link modes on interface eth0 Dec 17 21:09:19.461820 osdx modulelauncher[444536]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Dec 17 21:09:19.461836 osdx modulelauncher[444536]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Dec 17 21:09:19.463396 osdx modulelauncher[444536]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Dec 17 21:09:19.463407 osdx modulelauncher[444536]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Dec 17 21:09:19.475742 osdx (udev-worker)[444552]: Network interface NamePolicy= disabled on kernel command line. Dec 17 21:09:19.658845 osdx cfgd[1460]: [443939]Completed change to active configuration Dec 17 21:09:20.135775 osdx OSDxCLI[443939]: User 'admin' committed the configuration. Dec 17 21:09:20.195314 osdx OSDxCLI[443939]: User 'admin' left the configuration menu. Dec 17 21:09:22.743596 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Dec 17 21:09:22.820705 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Dec 17 21:09:22.927383 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Dec 17 21:09:23.462693 osdx file_operation[444715]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Dec 17 21:09:23.529803 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Dec 17 21:09:23.695023 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Dec 17 21:09:23.855307 osdx OSDxCLI[443939]: User 'admin' entered the configuration menu. Dec 17 21:09:23.918150 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 17 21:09:24.020771 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 17 21:09:24.137981 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 17 21:09:24.252686 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 17 21:09:24.363489 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 17 21:09:24.459770 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 17 21:09:24.617609 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 17 21:09:24.690457 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 17 21:09:24.805552 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 17 21:09:24.871962 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 17 21:09:24.990169 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 17 21:09:25.049658 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 17 21:09:25.168611 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 17 21:09:25.282407 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 17 21:09:25.399291 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 17 21:09:25.496393 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 17 21:09:25.591390 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 17 21:09:25.725821 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 17 21:09:25.810549 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 17 21:09:25.909884 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 17 21:09:25.999437 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'show working'. Dec 17 21:09:26.098684 osdx ubnt-cfgd[444766]: inactive Dec 17 21:09:26.186610 osdx INFO[444811]: FRR daemons did not change Dec 17 21:09:26.419117 osdx systemd[1]: Reloading. Dec 17 21:09:26.461596 osdx systemd-sysv-generator[444867]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Dec 17 21:09:26.593992 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 17 21:09:26.618579 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 17 21:09:26.901544 osdx INFO[444848]: Rules successfully loaded Dec 17 21:09:26.902608 osdx cfgd[1460]: [443939]Completed change to active configuration Dec 17 21:09:26.970147 osdx OSDxCLI[443939]: User 'admin' committed the configuration. Dec 17 21:09:26.992774 osdx OSDxCLI[443939]: User 'admin' left the configuration menu. Dec 17 21:09:27.419226 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=16934 DF PROTO=TCP SPT=57450 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 17 21:09:27.419317 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19998 DF PROTO=TCP SPT=57460 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 17 21:09:37.635793 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Dec 17 21:09:37.762963 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'system journal show | cat'. Dec 17 21:09:37.995035 osdx OSDxCLI[443939]: User 'admin' entered the configuration menu. Dec 17 21:09:38.242242 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 17 21:09:38.394721 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 17 21:09:38.457407 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 17 21:09:38.555202 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 17 21:09:38.612510 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 17 21:09:38.715612 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 17 21:09:38.806761 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 17 21:09:38.888048 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 17 21:09:38.982849 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 17 21:09:39.110894 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 17 21:09:39.177502 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 17 21:09:39.299437 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 17 21:09:39.417187 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 17 21:09:39.607504 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 17 21:09:39.689017 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 17 21:09:39.776188 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 17 21:09:39.830835 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 17 21:09:39.951515 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 17 21:09:40.009789 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 17 21:09:40.102679 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 17 21:09:40.163934 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Dec 17 21:09:40.255533 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Dec 17 21:09:40.322435 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Dec 17 21:09:40.420747 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Dec 17 21:09:40.501786 osdx OSDxCLI[443939]: User 'admin' added a new cfg line: 'show changes'. Dec 17 21:09:40.603009 osdx ubnt-cfgd[444993]: inactive Dec 17 21:09:40.657903 osdx INFO[445013]: FRR daemons did not change Dec 17 21:09:40.913212 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Dec 17 21:09:42.746359 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Dec 17 21:09:42.746514 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Dec 17 21:09:42.746555 osdx systemd[1]: suricata@FW.service: Consumed 1.879s CPU time. Dec 17 21:09:42.766049 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 17 21:09:42.784235 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 17 21:09:42.990738 osdx INFO[445040]: Rules successfully loaded Dec 17 21:09:42.991301 osdx cfgd[1460]: [443939]Completed change to active configuration Dec 17 21:09:42.993582 osdx OSDxCLI[443939]: User 'admin' committed the configuration. Dec 17 21:09:43.059704 osdx OSDxCLI[443939]: User 'admin' left the configuration menu. Dec 17 21:09:43.369601 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=34602 DF PROTO=TCP SPT=33880 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Dec 17 21:09:43.369683 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=38863 DF PROTO=TCP SPT=33886 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Dec 17 21:09:47.038162 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Dec 17 21:09:53.588819 osdx OSDxCLI[443939]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 10642 0 --:--:-- --:--:-- --:--:-- 10750
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.642 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.642/0.642/0.642/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.590 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.590/0.590/0.590/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51172 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 102 MBytes 860 Mbits/sec 127 1.85 MBytes [ 5] 1.00-2.00 sec 172 MBytes 1.45 Gbits/sec 20 1.39 MBytes [ 5] 2.00-3.00 sec 235 MBytes 1.97 Gbits/sec 0 1.49 MBytes [ 5] 3.00-4.00 sec 291 MBytes 2.44 Gbits/sec 0 1.63 MBytes [ 5] 4.00-5.00 sec 286 MBytes 2.40 Gbits/sec 0 1.75 MBytes [ 5] 5.00-6.00 sec 289 MBytes 2.42 Gbits/sec 0 1.86 MBytes [ 5] 6.00-7.01 sec 304 MBytes 2.52 Gbits/sec 0 1.98 MBytes [ 5] 7.01-8.00 sec 236 MBytes 2.00 Gbits/sec 149 1.54 MBytes [ 5] 8.00-9.00 sec 264 MBytes 2.22 Gbits/sec 0 1.66 MBytes [ 5] 9.00-10.00 sec 281 MBytes 2.36 Gbits/sec 42 1.24 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.40 GBytes 2.06 Gbits/sec 338 sender [ 5] 0.00-10.00 sec 2.40 GBytes 2.06 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:10:23.685847 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51166 -> 40.0.0.2:5001 12/17/2025-21:10:23.686920 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51172 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=68 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=68 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=51172 dport=5001 packets=1781588 bytes=2672365269 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51172 packets=220535 bytes=11463108 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=432 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=432 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=51166 dport=5001 packets=14 bytes=1194 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51166 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 46624 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 260 MBytes 2.18 Gbits/sec 145 1.32 MBytes [ 5] 1.00-2.00 sec 262 MBytes 2.20 Gbits/sec 0 1.45 MBytes [ 5] 2.00-3.00 sec 279 MBytes 2.34 Gbits/sec 0 1.58 MBytes [ 5] 3.00-4.00 sec 238 MBytes 1.99 Gbits/sec 0 1.68 MBytes [ 5] 4.00-5.00 sec 296 MBytes 2.49 Gbits/sec 35 1.30 MBytes [ 5] 5.00-6.00 sec 265 MBytes 2.22 Gbits/sec 0 1.44 MBytes [ 5] 6.00-7.00 sec 302 MBytes 2.54 Gbits/sec 0 1.59 MBytes [ 5] 7.00-8.00 sec 296 MBytes 2.49 Gbits/sec 0 1.71 MBytes [ 5] 8.00-9.00 sec 289 MBytes 2.42 Gbits/sec 186 1.35 MBytes [ 5] 9.00-10.00 sec 309 MBytes 2.59 Gbits/sec 0 1.50 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 366 sender [ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:10:23.685847 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51166 -> 40.0.0.2:5001 12/17/2025-21:10:23.686920 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51172 -> 40.0.0.2:5001 12/17/2025-21:10:38.125234 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46618 -> 40.0.0.2:5001 12/17/2025-21:10:38.126249 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46624 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=46618 dport=5001 packets=16 bytes=1302 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=46618 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=46624 dport=5001 packets=2024747 bytes=3037109081 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=46624 packets=266589 bytes=13857152 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 421 0 --:--:-- --:--:-- --:--:-- 421
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.755 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.755/0.755/0.755/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.548 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.548/0.548/0.548/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/17/2025-21:11:16.809251 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54012 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=435 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=435 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=71 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=71 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=54012 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=54012 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 10 409 778 2 - 5 5 369 369 ------------------------------------------------------------------ Total 10 10 778 778
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/17/2025-21:11:16.809251 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54012 -> 40.0.0.2:5000 12/17/2025-21:11:23.057947 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:54012 -> 40.0.0.2:5000 12/17/2025-21:11:24.558963 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53418 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 30 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=53418 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=53418 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19023 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.714 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.714/0.714/0.714/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.578 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.578/0.578/0.578/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/17/2025-21:11:49.695468 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34716 -> 40.0.0.2:5001 12/17/2025-21:11:49.697472 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34730 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=34716 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34716 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=438 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=438 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=34730 dport=5001 packets=7225 bytes=10833201 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34730 packets=1132 bytes=58872 [ASSURED] [OFFLOAD, packets=7212 bytes=10818000 packets=1130 bytes=58760] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=74 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=74 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 4714 0 --:--:-- --:--:-- --:--:-- 4750
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
12/17/2025-21:11:49.695468 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34716 -> 40.0.0.2:5001 12/17/2025-21:11:49.697472 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34730 -> 40.0.0.2:5001 12/17/2025-21:11:52.513147 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34740 -> 40.0.0.2:5001 12/17/2025-21:11:52.514615 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:54690 -> 40.0.0.2:5001 12/17/2025-21:11:52.525737 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:54690 -> 40.0.0.2:5001 12/17/2025-21:11:52.537709 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:54690 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=34740 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34740 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=54690 dport=5001 packets=22 bytes=31028 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=54690 packets=1 bytes=32 [OFFLOAD, packets=18 bytes=26568 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 39231 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.642 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.642/0.642/0.642/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.552 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.552/0.552/0.552/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/17/2025-21:12:14.847238 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49952 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 49952 yes 201 0 0 0 20.0.0.2 40.0.0.2 49952 5000 yes 201 0 9 731
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250141s ending at 1766005938.469009 XDP_DROP 9 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
12/17/2025-21:12:14.847238 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49952 -> 40.0.0.2:5000 12/17/2025-21:12:18.655466 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:49255 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 49952 yes 201 0 0 0 20.0.0.2 40.0.0.2 49255 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5001 49255 no 201 0 0 0 20.0.0.2 40.0.0.2 49952 5000 yes 201 0 12 905
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250149s ending at 1766005942.306530 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 34 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)