Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 40553 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.93 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.932/1.932/1.932/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.802 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.802/0.802/0.802/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 44562 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 27.9 MBytes 234 Mbits/sec 0 1.26 MBytes [ 5] 1.00-2.00 sec 43.8 MBytes 367 Mbits/sec 89 1.28 MBytes [ 5] 2.00-3.00 sec 40.0 MBytes 336 Mbits/sec 0 1.40 MBytes [ 5] 3.00-4.00 sec 32.5 MBytes 273 Mbits/sec 0 1.49 MBytes [ 5] 4.00-5.00 sec 35.0 MBytes 294 Mbits/sec 1 1.12 MBytes [ 5] 5.00-6.00 sec 33.8 MBytes 283 Mbits/sec 0 1.18 MBytes [ 5] 6.00-7.00 sec 33.8 MBytes 284 Mbits/sec 0 1.23 MBytes [ 5] 7.00-8.00 sec 32.5 MBytes 272 Mbits/sec 0 1.26 MBytes [ 5] 8.00-9.00 sec 48.8 MBytes 410 Mbits/sec 0 1.28 MBytes [ 5] 9.00-10.00 sec 56.2 MBytes 472 Mbits/sec 0 1.28 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 384 MBytes 322 Mbits/sec 90 sender [ 5] 0.00-10.02 sec 383 MBytes 320 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:44:18.700234 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44550 -> 40.0.0.2:5001 07/28/2025-07:44:18.702429 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44562 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 18316 0 --:--:-- --:--:-- --:--:-- 18428
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.03 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.025/1.025/1.025/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=1.11 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.112/1.112/1.112/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 38046 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 63.9 MBytes 536 Mbits/sec 340 1.26 MBytes [ 5] 1.00-2.00 sec 57.5 MBytes 482 Mbits/sec 0 1.35 MBytes [ 5] 2.00-3.00 sec 118 MBytes 986 Mbits/sec 0 1.41 MBytes [ 5] 3.00-4.00 sec 106 MBytes 891 Mbits/sec 2 1.05 MBytes [ 5] 4.00-5.00 sec 52.5 MBytes 440 Mbits/sec 0 1.11 MBytes [ 5] 5.00-6.00 sec 71.2 MBytes 598 Mbits/sec 0 1.16 MBytes [ 5] 6.00-7.00 sec 104 MBytes 870 Mbits/sec 0 1.19 MBytes [ 5] 7.00-8.00 sec 101 MBytes 849 Mbits/sec 0 1.25 MBytes [ 5] 8.00-9.00 sec 56.2 MBytes 472 Mbits/sec 0 1.28 MBytes [ 5] 9.00-10.00 sec 98.8 MBytes 828 Mbits/sec 0 1.33 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 829 MBytes 695 Mbits/sec 342 sender [ 5] 0.00-10.00 sec 827 MBytes 693 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:45:06.725247 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38032 -> 40.0.0.2:5001 07/28/2025-07:45:06.747410 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38046 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jul 28 07:44:50.634501 osdx systemd-journald[1773]: Runtime Journal (/run/log/journal/a46937b51a3a4c469575696f63c9d620) is 1.9M, max 15.3M, 13.3M free. Jul 28 07:44:50.636184 osdx systemd-journald[1773]: Received client request to rotate journal, rotating. Jul 28 07:44:50.636265 osdx systemd-journald[1773]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a46937b51a3a4c469575696f63c9d620. Jul 28 07:44:50.670856 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'system journal clear'. Jul 28 07:44:51.179659 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'system coredump delete all'. Jul 28 07:44:51.848644 osdx OSDxCLI[38992]: User 'admin' entered the configuration menu. Jul 28 07:44:52.053381 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jul 28 07:44:52.198446 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jul 28 07:44:52.401387 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jul 28 07:44:52.580985 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jul 28 07:44:52.883143 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jul 28 07:44:52.982833 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jul 28 07:44:53.143438 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'show working'. Jul 28 07:44:53.284020 osdx ubnt-cfgd[39440]: inactive Jul 28 07:44:53.353985 osdx INFO[39450]: FRR daemons did not change Jul 28 07:44:53.392189 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jul 28 07:44:53.540185 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jul 28 07:44:53.591965 osdx (udev-worker)[39572]: Network interface NamePolicy= disabled on kernel command line. Jul 28 07:44:53.916880 osdx cfgd[1473]: [38992]Completed change to active configuration Jul 28 07:44:53.998247 osdx OSDxCLI[38992]: User 'admin' committed the configuration. Jul 28 07:44:54.049600 osdx OSDxCLI[38992]: User 'admin' left the configuration menu. Jul 28 07:44:58.240846 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jul 28 07:44:58.447099 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jul 28 07:44:58.631110 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jul 28 07:44:59.608801 osdx file_operation[39737]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jul 28 07:44:59.653879 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jul 28 07:44:59.910489 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jul 28 07:45:00.175147 osdx OSDxCLI[38992]: User 'admin' entered the configuration menu. Jul 28 07:45:00.309240 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jul 28 07:45:00.432811 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jul 28 07:45:00.570988 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jul 28 07:45:00.720479 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jul 28 07:45:00.836417 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jul 28 07:45:00.955070 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jul 28 07:45:01.174965 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jul 28 07:45:01.358877 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jul 28 07:45:01.606154 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jul 28 07:45:01.834449 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jul 28 07:45:02.003644 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jul 28 07:45:02.176290 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jul 28 07:45:02.407391 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jul 28 07:45:02.551867 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jul 28 07:45:02.742500 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jul 28 07:45:02.882389 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jul 28 07:45:03.029929 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jul 28 07:45:03.250905 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jul 28 07:45:03.379082 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jul 28 07:45:03.507432 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jul 28 07:45:03.763646 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'show working'. Jul 28 07:45:03.918903 osdx ubnt-cfgd[39791]: inactive Jul 28 07:45:04.102224 osdx INFO[39828]: FRR daemons did not change Jul 28 07:45:04.579019 osdx systemd[1]: Reloading. Jul 28 07:45:04.660264 osdx systemd-sysv-generator[39889]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jul 28 07:45:04.905000 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jul 28 07:45:04.942819 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jul 28 07:45:05.530084 osdx INFO[39870]: Rules successfully loaded Jul 28 07:45:05.531773 osdx cfgd[1473]: [38992]Completed change to active configuration Jul 28 07:45:05.535778 osdx OSDxCLI[38992]: User 'admin' committed the configuration. Jul 28 07:45:05.589250 osdx OSDxCLI[38992]: User 'admin' left the configuration menu. Jul 28 07:45:06.745796 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14391 DF PROTO=TCP SPT=38032 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jul 28 07:45:06.754347 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=52704 DF PROTO=TCP SPT=38046 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jul 28 07:45:17.048757 osdx systemd[1]: Starting logrotate.service - Rotate log files... Jul 28 07:45:17.090726 osdx systemd[1]: logrotate.service: Deactivated successfully. Jul 28 07:45:17.091155 osdx systemd[1]: Finished logrotate.service - Rotate log files. Jul 28 07:45:17.132863 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50644 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 128 MBytes 1.07 Gbits/sec 284 1.32 MBytes [ 5] 1.00-2.00 sec 131 MBytes 1.10 Gbits/sec 0 1.41 MBytes [ 5] 2.00-3.00 sec 132 MBytes 1.11 Gbits/sec 0 1.47 MBytes [ 5] 3.00-4.00 sec 110 MBytes 923 Mbits/sec 0 1.52 MBytes [ 5] 4.00-5.00 sec 81.2 MBytes 682 Mbits/sec 0 1.55 MBytes [ 5] 5.00-6.00 sec 96.2 MBytes 807 Mbits/sec 26 1.12 MBytes [ 5] 6.00-7.00 sec 75.0 MBytes 629 Mbits/sec 0 1.19 MBytes [ 5] 7.00-8.00 sec 66.2 MBytes 556 Mbits/sec 0 1.25 MBytes [ 5] 8.00-9.00 sec 67.5 MBytes 566 Mbits/sec 0 1.29 MBytes [ 5] 9.00-10.00 sec 104 MBytes 870 Mbits/sec 0 1.31 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 992 MBytes 832 Mbits/sec 310 sender [ 5] 0.00-10.01 sec 990 MBytes 829 Mbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:45:06.725247 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38032 -> 40.0.0.2:5001 07/28/2025-07:45:06.747410 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38046 -> 40.0.0.2:5001 07/28/2025-07:45:23.696440 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50642 -> 40.0.0.2:5001 07/28/2025-07:45:23.698156 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50644 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jul 28 07:44:50.634501 osdx systemd-journald[1773]: Runtime Journal (/run/log/journal/a46937b51a3a4c469575696f63c9d620) is 1.9M, max 15.3M, 13.3M free. Jul 28 07:44:50.636184 osdx systemd-journald[1773]: Received client request to rotate journal, rotating. Jul 28 07:44:50.636265 osdx systemd-journald[1773]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a46937b51a3a4c469575696f63c9d620. Jul 28 07:44:50.670856 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'system journal clear'. Jul 28 07:44:51.179659 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'system coredump delete all'. Jul 28 07:44:51.848644 osdx OSDxCLI[38992]: User 'admin' entered the configuration menu. Jul 28 07:44:52.053381 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jul 28 07:44:52.198446 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jul 28 07:44:52.401387 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jul 28 07:44:52.580985 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jul 28 07:44:52.883143 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jul 28 07:44:52.982833 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jul 28 07:44:53.143438 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'show working'. Jul 28 07:44:53.284020 osdx ubnt-cfgd[39440]: inactive Jul 28 07:44:53.353985 osdx INFO[39450]: FRR daemons did not change Jul 28 07:44:53.392189 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jul 28 07:44:53.540185 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jul 28 07:44:53.591965 osdx (udev-worker)[39572]: Network interface NamePolicy= disabled on kernel command line. Jul 28 07:44:53.916880 osdx cfgd[1473]: [38992]Completed change to active configuration Jul 28 07:44:53.998247 osdx OSDxCLI[38992]: User 'admin' committed the configuration. Jul 28 07:44:54.049600 osdx OSDxCLI[38992]: User 'admin' left the configuration menu. Jul 28 07:44:58.240846 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jul 28 07:44:58.447099 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jul 28 07:44:58.631110 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jul 28 07:44:59.608801 osdx file_operation[39737]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jul 28 07:44:59.653879 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jul 28 07:44:59.910489 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jul 28 07:45:00.175147 osdx OSDxCLI[38992]: User 'admin' entered the configuration menu. Jul 28 07:45:00.309240 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jul 28 07:45:00.432811 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jul 28 07:45:00.570988 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jul 28 07:45:00.720479 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jul 28 07:45:00.836417 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jul 28 07:45:00.955070 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jul 28 07:45:01.174965 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jul 28 07:45:01.358877 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jul 28 07:45:01.606154 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jul 28 07:45:01.834449 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jul 28 07:45:02.003644 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jul 28 07:45:02.176290 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jul 28 07:45:02.407391 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jul 28 07:45:02.551867 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jul 28 07:45:02.742500 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jul 28 07:45:02.882389 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jul 28 07:45:03.029929 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jul 28 07:45:03.250905 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jul 28 07:45:03.379082 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jul 28 07:45:03.507432 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jul 28 07:45:03.763646 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'show working'. Jul 28 07:45:03.918903 osdx ubnt-cfgd[39791]: inactive Jul 28 07:45:04.102224 osdx INFO[39828]: FRR daemons did not change Jul 28 07:45:04.579019 osdx systemd[1]: Reloading. Jul 28 07:45:04.660264 osdx systemd-sysv-generator[39889]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jul 28 07:45:04.905000 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jul 28 07:45:04.942819 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jul 28 07:45:05.530084 osdx INFO[39870]: Rules successfully loaded Jul 28 07:45:05.531773 osdx cfgd[1473]: [38992]Completed change to active configuration Jul 28 07:45:05.535778 osdx OSDxCLI[38992]: User 'admin' committed the configuration. Jul 28 07:45:05.589250 osdx OSDxCLI[38992]: User 'admin' left the configuration menu. Jul 28 07:45:06.745796 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=14391 DF PROTO=TCP SPT=38032 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jul 28 07:45:06.754347 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=52704 DF PROTO=TCP SPT=38046 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jul 28 07:45:17.048757 osdx systemd[1]: Starting logrotate.service - Rotate log files... Jul 28 07:45:17.090726 osdx systemd[1]: logrotate.service: Deactivated successfully. Jul 28 07:45:17.091155 osdx systemd[1]: Finished logrotate.service - Rotate log files. Jul 28 07:45:17.132863 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Jul 28 07:45:17.304946 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'system journal show | cat'. Jul 28 07:45:17.590768 osdx OSDxCLI[38992]: User 'admin' entered the configuration menu. Jul 28 07:45:17.700985 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jul 28 07:45:17.834632 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jul 28 07:45:17.936775 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jul 28 07:45:18.045215 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jul 28 07:45:18.146663 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jul 28 07:45:18.254757 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jul 28 07:45:18.429531 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jul 28 07:45:18.579444 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jul 28 07:45:18.733277 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jul 28 07:45:18.870347 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jul 28 07:45:18.988814 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jul 28 07:45:19.107804 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jul 28 07:45:19.241608 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jul 28 07:45:19.365087 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jul 28 07:45:19.455642 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jul 28 07:45:19.595443 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jul 28 07:45:19.725965 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jul 28 07:45:19.907493 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jul 28 07:45:20.003478 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jul 28 07:45:20.033287 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Jul 28 07:45:20.141539 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jul 28 07:45:20.282334 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Jul 28 07:45:20.385067 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Jul 28 07:45:20.492019 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Jul 28 07:45:20.602675 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Jul 28 07:45:20.772973 osdx OSDxCLI[38992]: User 'admin' added a new cfg line: 'show changes'. Jul 28 07:45:20.988402 osdx ubnt-cfgd[40021]: inactive Jul 28 07:45:21.074667 osdx INFO[40037]: FRR daemons did not change Jul 28 07:45:21.397398 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Jul 28 07:45:22.972077 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Jul 28 07:45:22.972350 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Jul 28 07:45:22.972395 osdx systemd[1]: suricata@FW.service: Consumed 1.745s CPU time. Jul 28 07:45:22.992764 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jul 28 07:45:23.019004 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jul 28 07:45:23.381269 osdx INFO[40062]: Rules successfully loaded Jul 28 07:45:23.382354 osdx cfgd[1473]: [38992]Completed change to active configuration Jul 28 07:45:23.385286 osdx OSDxCLI[38992]: User 'admin' committed the configuration. Jul 28 07:45:23.414692 osdx OSDxCLI[38992]: User 'admin' left the configuration menu. Jul 28 07:45:23.700279 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=29561 DF PROTO=TCP SPT=50642 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jul 28 07:45:23.700365 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=33881 DF PROTO=TCP SPT=50644 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jul 28 07:45:34.313134 osdx OSDxCLI[38992]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 10499 0 --:--:-- --:--:-- --:--:-- 10750
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=3.44 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 3.443/3.443/3.443/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=6.77 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 6.765/6.765/6.765/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 53242 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 81.9 MBytes 687 Mbits/sec 125 1.50 MBytes [ 5] 1.00-2.00 sec 91.2 MBytes 765 Mbits/sec 137 1.15 MBytes [ 5] 2.00-3.00 sec 72.5 MBytes 606 Mbits/sec 0 1.22 MBytes [ 5] 3.00-4.00 sec 76.2 MBytes 642 Mbits/sec 0 1.28 MBytes [ 5] 4.00-5.00 sec 70.0 MBytes 587 Mbits/sec 0 1.32 MBytes [ 5] 5.00-6.00 sec 43.8 MBytes 367 Mbits/sec 0 1.34 MBytes [ 5] 6.00-7.01 sec 38.8 MBytes 323 Mbits/sec 0 1.36 MBytes [ 5] 7.01-8.00 sec 57.5 MBytes 486 Mbits/sec 0 1.36 MBytes [ 5] 8.00-9.00 sec 82.5 MBytes 692 Mbits/sec 0 1.38 MBytes [ 5] 9.00-10.00 sec 66.2 MBytes 556 Mbits/sec 0 1.41 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 681 MBytes 571 Mbits/sec 262 sender [ 5] 0.00-10.03 sec 679 MBytes 568 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:46:12.095759 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53230 -> 40.0.0.2:5001 07/28/2025-07:46:12.097978 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53242 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 18 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=47 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=47 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=53230 dport=5001 packets=14 bytes=1209 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53230 packets=14 bytes=1071 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=53242 dport=5001 packets=492709 bytes=739059201 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53242 packets=19475 bytes=1014056 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 18 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=90 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=90 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57494 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 92.4 MBytes 775 Mbits/sec 438 1.72 MBytes [ 5] 1.00-2.00 sec 128 MBytes 1.07 Gbits/sec 26 1.29 MBytes [ 5] 2.00-3.00 sec 106 MBytes 891 Mbits/sec 0 1.35 MBytes [ 5] 3.00-4.00 sec 116 MBytes 975 Mbits/sec 0 1.40 MBytes [ 5] 4.00-5.00 sec 108 MBytes 902 Mbits/sec 3 1.01 MBytes [ 5] 5.00-6.01 sec 77.5 MBytes 645 Mbits/sec 0 1.08 MBytes [ 5] 6.01-7.00 sec 87.5 MBytes 740 Mbits/sec 0 1.14 MBytes [ 5] 7.00-8.00 sec 109 MBytes 912 Mbits/sec 0 1.18 MBytes [ 5] 8.00-9.00 sec 140 MBytes 1.17 Gbits/sec 0 1.26 MBytes [ 5] 9.00-10.00 sec 93.8 MBytes 786 Mbits/sec 0 1.31 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.03 GBytes 887 Mbits/sec 467 sender [ 5] 0.00-10.00 sec 1.03 GBytes 884 Mbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:46:12.095759 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53230 -> 40.0.0.2:5001 07/28/2025-07:46:12.097978 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53242 -> 40.0.0.2:5001 07/28/2025-07:46:30.279636 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57486 -> 40.0.0.2:5001 07/28/2025-07:46:30.284801 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57494 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=57494 dport=5001 packets=764770 bytes=1147144189 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=57494 packets=41897 bytes=2179624 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=57486 dport=5001 packets=14 bytes=1197 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=57486 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 47147 0 --:--:-- --:--:-- --:--:-- 50000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=3.96 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 3.961/3.961/3.961/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=12.7 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 12.729/12.729/12.729/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
07/28/2025-07:47:15.159209 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44646 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 25 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=93 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=93 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 25 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=50 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=50 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=44646 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=44646 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 8 263 491 2 - 3 3 228 228 ------------------------------------------------------------------ Total 8 8 491 491 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
07/28/2025-07:47:15.159209 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44646 -> 40.0.0.2:5000 07/28/2025-07:47:23.786148 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44646 -> 40.0.0.2:5000 07/28/2025-07:47:26.193438 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38980 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=38980 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=38980 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 7 263 376 2 - 2 2 113 113 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 10 499 740 2 - 4 4 241 241 --------------------------------------------------------------------- Total 10 10 740 740
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 34968 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=12.6 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 12.599/12.599/12.599/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=4.18 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.175/4.175/4.175/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
07/28/2025-07:48:06.422571 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56320 -> 40.0.0.2:5001 07/28/2025-07:48:06.425490 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56334 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 28 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=53 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=53 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=56320 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=56320 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 icmp 1 28 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=96 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=96 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=56334 dport=5001 packets=8806 bytes=13204005 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=56334 packets=1200 bytes=62408 [ASSURED] [OFFLOAD, packets=8793 bytes=13188804 packets=1198 bytes=62296] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 15363 0 --:--:-- --:--:-- --:--:-- 16625
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
07/28/2025-07:48:06.422571 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56320 -> 40.0.0.2:5001 07/28/2025-07:48:06.425490 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56334 -> 40.0.0.2:5001 07/28/2025-07:48:10.169358 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56340 -> 40.0.0.2:5001 07/28/2025-07:48:10.174259 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52271 -> 40.0.0.2:5001 07/28/2025-07:48:10.185233 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52271 -> 40.0.0.2:5001 07/28/2025-07:48:10.196265 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:52271 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=56340 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=56340 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=246 packets=4 bytes=211] mark=129834765 use=2 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=52271 dport=5001 packets=40 bytes=57596 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52271 packets=1 bytes=32 [OFFLOAD, packets=36 bytes=53136 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 22836 0 --:--:-- --:--:-- --:--:-- 25000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=4.95 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.953/4.953/4.953/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.835 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.835/0.835/0.835/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
07/28/2025-07:48:45.531979 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57878 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 57878 yes 201 0 0 0 20.0.0.2 40.0.0.2 57878 5000 yes 201 0 8 673
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250417s ending at 1753688930.112990 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 3028342 pkts ( 0 pps) 4275403 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
07/28/2025-07:48:45.531979 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57878 -> 40.0.0.2:5000 07/28/2025-07:48:50.480692 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:32882 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 32882 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 57878 yes 201 0 0 0 20.0.0.2 40.0.0.2 57878 5000 yes 201 0 10 789 40.0.0.2 20.0.0.2 5001 32882 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250624s ending at 1753688934.604343 XDP_DROP 10 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 3028382 pkts ( 0 pps) 4275406 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)