Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 10480 0 --:--:-- --:--:-- --:--:-- 10750
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.488 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.488/0.488/0.488/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.442 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.442/0.442/0.442/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 36070 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 104 MBytes 871 Mbits/sec 66 1.68 MBytes [ 5] 1.00-2.00 sec 114 MBytes 954 Mbits/sec 0 1.83 MBytes [ 5] 2.00-3.00 sec 104 MBytes 870 Mbits/sec 0 1.96 MBytes [ 5] 3.00-4.00 sec 52.5 MBytes 441 Mbits/sec 0 2.05 MBytes [ 5] 4.00-5.00 sec 62.5 MBytes 524 Mbits/sec 10 1.53 MBytes [ 5] 5.00-6.00 sec 51.2 MBytes 430 Mbits/sec 81 1.15 MBytes [ 5] 6.00-7.00 sec 48.8 MBytes 409 Mbits/sec 0 1.21 MBytes [ 5] 7.00-8.00 sec 48.8 MBytes 409 Mbits/sec 0 1.25 MBytes [ 5] 8.00-9.00 sec 48.8 MBytes 409 Mbits/sec 0 1.28 MBytes [ 5] 9.00-10.00 sec 50.0 MBytes 419 Mbits/sec 0 1.29 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 684 MBytes 574 Mbits/sec 157 sender [ 5] 0.00-10.01 sec 681 MBytes 571 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:08:18.991726 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36060 -> 40.0.0.2:5001 10/20/2025-16:08:18.992565 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36070 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23268 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.495 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.495/0.495/0.495/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.435 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.435/0.435/0.435/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 39630 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 108 MBytes 905 Mbits/sec 95 1.71 MBytes [ 5] 1.00-2.00 sec 195 MBytes 1.64 Gbits/sec 0 1.87 MBytes [ 5] 2.00-3.00 sec 314 MBytes 2.63 Gbits/sec 78 1.44 MBytes [ 5] 3.00-4.00 sec 305 MBytes 2.56 Gbits/sec 0 1.59 MBytes [ 5] 4.00-5.00 sec 320 MBytes 2.68 Gbits/sec 0 1.73 MBytes [ 5] 5.00-6.00 sec 308 MBytes 2.58 Gbits/sec 0 1.85 MBytes [ 5] 6.00-7.00 sec 309 MBytes 2.59 Gbits/sec 0 1.97 MBytes [ 5] 7.00-8.00 sec 316 MBytes 2.65 Gbits/sec 0 2.09 MBytes [ 5] 8.00-9.00 sec 338 MBytes 2.83 Gbits/sec 6 1.61 MBytes [ 5] 9.00-10.00 sec 335 MBytes 2.81 Gbits/sec 0 1.75 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec 179 sender [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:08:55.615051 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39620 -> 40.0.0.2:5001 10/20/2025-16:08:55.615959 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39630 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Oct 20 16:08:47.344479 osdx systemd-journald[222010]: Runtime Journal (/run/log/journal/82a9756ca47e4d589aa55e1b1a6b94de) is 1.7M, max 13.8M, 12.0M free. Oct 20 16:08:47.347567 osdx systemd-journald[222010]: Received client request to rotate journal, rotating. Oct 20 16:08:47.347616 osdx systemd-journald[222010]: Vacuuming done, freed 0B of archived journals from /run/log/journal/82a9756ca47e4d589aa55e1b1a6b94de. Oct 20 16:08:47.353658 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'system journal clear'. Oct 20 16:08:47.571686 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'system coredump delete all'. Oct 20 16:08:47.786712 osdx OSDxCLI[342301]: User 'admin' entered the configuration menu. Oct 20 16:08:47.862995 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Oct 20 16:08:47.947669 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Oct 20 16:08:48.003306 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Oct 20 16:08:48.101835 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Oct 20 16:08:48.472527 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Oct 20 16:08:48.527185 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Oct 20 16:08:48.637557 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'show working'. Oct 20 16:08:48.702837 osdx ubnt-cfgd[342754]: inactive Oct 20 16:08:48.736963 osdx INFO[342765]: FRR daemons did not change Oct 20 16:08:48.755575 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Oct 20 16:08:48.823572 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Oct 20 16:08:48.846788 osdx (udev-worker)[342888]: Network interface NamePolicy= disabled on kernel command line. Oct 20 16:08:48.883266 osdx (udev-worker)[342894]: Network interface NamePolicy= disabled on kernel command line. Oct 20 16:08:48.986973 osdx cfgd[1655]: [342301]Completed change to active configuration Oct 20 16:08:48.998882 osdx OSDxCLI[342301]: User 'admin' committed the configuration. Oct 20 16:08:49.016489 osdx OSDxCLI[342301]: User 'admin' left the configuration menu. Oct 20 16:08:51.306910 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Oct 20 16:08:51.384968 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Oct 20 16:08:51.503113 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Oct 20 16:08:51.943204 osdx file_operation[343053]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Oct 20 16:08:51.967603 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Oct 20 16:08:52.092872 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Oct 20 16:08:52.240474 osdx OSDxCLI[342301]: User 'admin' entered the configuration menu. Oct 20 16:08:52.303719 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 20 16:08:52.402601 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 20 16:08:52.467108 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 20 16:08:52.555347 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 20 16:08:52.613123 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 20 16:08:52.712226 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 20 16:08:52.798070 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 20 16:08:52.881167 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 20 16:08:52.935382 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 20 16:08:53.034173 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 20 16:08:53.089440 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 20 16:08:53.301990 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 20 16:08:53.358192 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 20 16:08:53.452343 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 20 16:08:53.515968 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 20 16:08:53.608701 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 20 16:08:53.663181 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 20 16:08:53.763381 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 20 16:08:53.843625 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 20 16:08:53.927459 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 20 16:08:54.010641 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'show working'. Oct 20 16:08:54.101099 osdx ubnt-cfgd[343104]: inactive Oct 20 16:08:54.172972 osdx INFO[343141]: FRR daemons did not change Oct 20 16:08:54.413829 osdx systemd[1]: Reloading. Oct 20 16:08:54.463567 osdx systemd-sysv-generator[343193]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Oct 20 16:08:54.595810 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 20 16:08:54.613118 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 20 16:08:54.801075 osdx INFO[343176]: Rules successfully loaded Oct 20 16:08:54.803151 osdx cfgd[1655]: [342301]Completed change to active configuration Oct 20 16:08:54.805373 osdx OSDxCLI[342301]: User 'admin' committed the configuration. Oct 20 16:08:55.270566 osdx OSDxCLI[342301]: User 'admin' left the configuration menu. Oct 20 16:08:55.615622 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=4783 DF PROTO=TCP SPT=39620 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 20 16:08:55.619577 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=43883 DF PROTO=TCP SPT=39630 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 20 16:09:05.798496 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 36378 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 284 MBytes 2.38 Gbits/sec 72 1.91 MBytes [ 5] 1.00-2.00 sec 319 MBytes 2.67 Gbits/sec 28 1.45 MBytes [ 5] 2.00-3.00 sec 328 MBytes 2.75 Gbits/sec 0 1.60 MBytes [ 5] 3.00-4.00 sec 319 MBytes 2.67 Gbits/sec 0 1.74 MBytes [ 5] 4.00-5.00 sec 388 MBytes 3.25 Gbits/sec 0 1.90 MBytes [ 5] 5.00-6.00 sec 321 MBytes 2.69 Gbits/sec 0 2.01 MBytes [ 5] 6.00-7.00 sec 320 MBytes 2.68 Gbits/sec 0 2.13 MBytes [ 5] 7.00-8.00 sec 325 MBytes 2.73 Gbits/sec 10 1.59 MBytes [ 5] 8.00-9.00 sec 321 MBytes 2.69 Gbits/sec 0 1.76 MBytes [ 5] 9.00-10.00 sec 53.8 MBytes 451 Mbits/sec 135 1.33 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.91 GBytes 2.50 Gbits/sec 245 sender [ 5] 0.00-10.00 sec 2.91 GBytes 2.50 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:08:55.615051 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39620 -> 40.0.0.2:5001 10/20/2025-16:08:55.615959 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39630 -> 40.0.0.2:5001 10/20/2025-16:09:11.214934 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36376 -> 40.0.0.2:5001 10/20/2025-16:09:11.215841 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36378 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Oct 20 16:08:47.344479 osdx systemd-journald[222010]: Runtime Journal (/run/log/journal/82a9756ca47e4d589aa55e1b1a6b94de) is 1.7M, max 13.8M, 12.0M free. Oct 20 16:08:47.347567 osdx systemd-journald[222010]: Received client request to rotate journal, rotating. Oct 20 16:08:47.347616 osdx systemd-journald[222010]: Vacuuming done, freed 0B of archived journals from /run/log/journal/82a9756ca47e4d589aa55e1b1a6b94de. Oct 20 16:08:47.353658 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'system journal clear'. Oct 20 16:08:47.571686 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'system coredump delete all'. Oct 20 16:08:47.786712 osdx OSDxCLI[342301]: User 'admin' entered the configuration menu. Oct 20 16:08:47.862995 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Oct 20 16:08:47.947669 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Oct 20 16:08:48.003306 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Oct 20 16:08:48.101835 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Oct 20 16:08:48.472527 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Oct 20 16:08:48.527185 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Oct 20 16:08:48.637557 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'show working'. Oct 20 16:08:48.702837 osdx ubnt-cfgd[342754]: inactive Oct 20 16:08:48.736963 osdx INFO[342765]: FRR daemons did not change Oct 20 16:08:48.755575 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Oct 20 16:08:48.823572 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Oct 20 16:08:48.846788 osdx (udev-worker)[342888]: Network interface NamePolicy= disabled on kernel command line. Oct 20 16:08:48.883266 osdx (udev-worker)[342894]: Network interface NamePolicy= disabled on kernel command line. Oct 20 16:08:48.986973 osdx cfgd[1655]: [342301]Completed change to active configuration Oct 20 16:08:48.998882 osdx OSDxCLI[342301]: User 'admin' committed the configuration. Oct 20 16:08:49.016489 osdx OSDxCLI[342301]: User 'admin' left the configuration menu. Oct 20 16:08:51.306910 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Oct 20 16:08:51.384968 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Oct 20 16:08:51.503113 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Oct 20 16:08:51.943204 osdx file_operation[343053]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Oct 20 16:08:51.967603 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Oct 20 16:08:52.092872 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Oct 20 16:08:52.240474 osdx OSDxCLI[342301]: User 'admin' entered the configuration menu. Oct 20 16:08:52.303719 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 20 16:08:52.402601 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 20 16:08:52.467108 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 20 16:08:52.555347 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 20 16:08:52.613123 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 20 16:08:52.712226 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 20 16:08:52.798070 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 20 16:08:52.881167 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 20 16:08:52.935382 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 20 16:08:53.034173 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 20 16:08:53.089440 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 20 16:08:53.301990 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 20 16:08:53.358192 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 20 16:08:53.452343 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 20 16:08:53.515968 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 20 16:08:53.608701 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 20 16:08:53.663181 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 20 16:08:53.763381 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 20 16:08:53.843625 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 20 16:08:53.927459 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 20 16:08:54.010641 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'show working'. Oct 20 16:08:54.101099 osdx ubnt-cfgd[343104]: inactive Oct 20 16:08:54.172972 osdx INFO[343141]: FRR daemons did not change Oct 20 16:08:54.413829 osdx systemd[1]: Reloading. Oct 20 16:08:54.463567 osdx systemd-sysv-generator[343193]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Oct 20 16:08:54.595810 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 20 16:08:54.613118 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 20 16:08:54.801075 osdx INFO[343176]: Rules successfully loaded Oct 20 16:08:54.803151 osdx cfgd[1655]: [342301]Completed change to active configuration Oct 20 16:08:54.805373 osdx OSDxCLI[342301]: User 'admin' committed the configuration. Oct 20 16:08:55.270566 osdx OSDxCLI[342301]: User 'admin' left the configuration menu. Oct 20 16:08:55.615622 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=4783 DF PROTO=TCP SPT=39620 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 20 16:08:55.619577 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=43883 DF PROTO=TCP SPT=39630 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 20 16:09:05.798496 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Oct 20 16:09:05.972133 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'system journal show | cat'. Oct 20 16:09:06.118658 osdx OSDxCLI[342301]: User 'admin' entered the configuration menu. Oct 20 16:09:06.222783 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 20 16:09:06.281628 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 20 16:09:06.377007 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 20 16:09:06.477753 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 20 16:09:06.534237 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 20 16:09:06.631846 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 20 16:09:06.719661 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 20 16:09:06.793228 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 20 16:09:06.889646 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 20 16:09:06.947262 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 20 16:09:07.043896 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 20 16:09:07.103295 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 20 16:09:07.209155 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 20 16:09:07.268165 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 20 16:09:07.361104 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 20 16:09:07.418449 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 20 16:09:07.518015 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 20 16:09:07.577400 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 20 16:09:07.687889 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 20 16:09:07.766815 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 20 16:09:07.883273 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Oct 20 16:09:07.951336 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Oct 20 16:09:08.057801 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Oct 20 16:09:08.140972 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Oct 20 16:09:08.259917 osdx OSDxCLI[342301]: User 'admin' added a new cfg line: 'show changes'. Oct 20 16:09:08.340296 osdx ubnt-cfgd[343319]: inactive Oct 20 16:09:08.382140 osdx INFO[343335]: FRR daemons did not change Oct 20 16:09:08.592043 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Oct 20 16:09:10.588499 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Oct 20 16:09:10.588607 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Oct 20 16:09:10.588637 osdx systemd[1]: suricata@FW.service: Consumed 2.108s CPU time. Oct 20 16:09:10.640628 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 20 16:09:10.677175 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 20 16:09:10.866148 osdx INFO[343360]: Rules successfully loaded Oct 20 16:09:10.866621 osdx cfgd[1655]: [342301]Completed change to active configuration Oct 20 16:09:10.868419 osdx OSDxCLI[342301]: User 'admin' committed the configuration. Oct 20 16:09:10.885163 osdx OSDxCLI[342301]: User 'admin' left the configuration menu. Oct 20 16:09:11.215581 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=43830 DF PROTO=TCP SPT=36376 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Oct 20 16:09:11.219658 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=44043 DF PROTO=TCP SPT=36378 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Oct 20 16:09:12.628037 osdx systemd[1]: Starting logrotate.service - Rotate log files... Oct 20 16:09:12.667200 osdx systemd[1]: logrotate.service: Deactivated successfully. Oct 20 16:09:12.667442 osdx systemd[1]: Finished logrotate.service - Rotate log files. Oct 20 16:09:17.031740 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Oct 20 16:09:21.431486 osdx OSDxCLI[342301]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 28943 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.599 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.599/0.599/0.599/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.497 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.497/0.497/0.497/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 55128 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 156 MBytes 1.31 Gbits/sec 116 1.29 MBytes [ 5] 1.00-2.00 sec 141 MBytes 1.18 Gbits/sec 0 1.37 MBytes [ 5] 2.00-3.00 sec 302 MBytes 2.54 Gbits/sec 0 1.50 MBytes [ 5] 3.00-4.00 sec 304 MBytes 2.55 Gbits/sec 0 1.64 MBytes [ 5] 4.00-5.00 sec 275 MBytes 2.31 Gbits/sec 162 1.27 MBytes [ 5] 5.00-6.00 sec 302 MBytes 2.54 Gbits/sec 0 1.43 MBytes [ 5] 6.00-7.00 sec 271 MBytes 2.27 Gbits/sec 0 1.56 MBytes [ 5] 7.00-8.00 sec 292 MBytes 2.45 Gbits/sec 0 1.69 MBytes [ 5] 8.00-9.00 sec 305 MBytes 2.56 Gbits/sec 0 1.81 MBytes [ 5] 9.00-10.00 sec 355 MBytes 2.98 Gbits/sec 0 1.95 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.64 GBytes 2.27 Gbits/sec 278 sender [ 5] 0.00-10.00 sec 2.64 GBytes 2.27 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:09:49.368115 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55126 -> 40.0.0.2:5001 10/20/2025-16:09:49.369005 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55128 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=475 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=475 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=88 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=88 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=55126 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55126 packets=13 bytes=1016 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=55128 dport=5001 packets=1958633 bytes=2937936273 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55128 packets=247032 bytes=12842556 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51482 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 300 MBytes 2.52 Gbits/sec 293 1.32 MBytes [ 5] 1.00-2.00 sec 314 MBytes 2.63 Gbits/sec 0 1.48 MBytes [ 5] 2.00-3.00 sec 315 MBytes 2.64 Gbits/sec 0 1.63 MBytes [ 5] 3.00-4.00 sec 322 MBytes 2.71 Gbits/sec 0 1.77 MBytes [ 5] 4.00-5.00 sec 289 MBytes 2.42 Gbits/sec 0 1.89 MBytes [ 5] 5.00-6.00 sec 315 MBytes 2.64 Gbits/sec 0 2.00 MBytes [ 5] 6.00-7.00 sec 328 MBytes 2.75 Gbits/sec 0 2.12 MBytes [ 5] 7.00-8.00 sec 331 MBytes 2.78 Gbits/sec 4 1.63 MBytes [ 5] 8.00-9.00 sec 331 MBytes 2.78 Gbits/sec 0 1.78 MBytes [ 5] 9.00-10.00 sec 314 MBytes 2.63 Gbits/sec 77 1.35 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.08 GBytes 2.65 Gbits/sec 374 sender [ 5] 0.00-10.00 sec 3.08 GBytes 2.65 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:09:49.368115 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55126 -> 40.0.0.2:5001 10/20/2025-16:09:49.369005 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55128 -> 40.0.0.2:5001 10/20/2025-16:10:04.220903 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51468 -> 40.0.0.2:5001 10/20/2025-16:10:04.221676 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51482 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=51468 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51468 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=51482 dport=5001 packets=2287424 bytes=3431125373 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51482 packets=374851 bytes=19486704 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 38692 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.496 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.496/0.496/0.496/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.439 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.439/0.439/0.439/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/20/2025-16:10:40.993547 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48078 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=48078 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=48078 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=91 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=91 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=478 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=478 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/20/2025-16:10:40.993547 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48078 -> 40.0.0.2:5000 10/20/2025-16:10:47.254961 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48078 -> 40.0.0.2:5000 10/20/2025-16:10:48.744049 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42546 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=42546 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=42546 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 24889 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.524 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.524/0.524/0.524/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.422 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.422/0.422/0.422/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/20/2025-16:11:11.701073 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43920 -> 40.0.0.2:5001 10/20/2025-16:11:11.701920 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43936 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43936 dport=5001 packets=24759 bytes=37126961 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43936 packets=2690 bytes=143484 [ASSURED] [OFFLOAD, packets=24746 bytes=37111760 packets=2688 bytes=143372] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=94 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=94 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=481 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=481 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43920 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43920 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 32407 0 --:--:-- --:--:-- --:--:-- 33250
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
10/20/2025-16:11:11.701073 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43920 -> 40.0.0.2:5001 10/20/2025-16:11:11.701920 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43936 -> 40.0.0.2:5001 10/20/2025-16:11:13.600453 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43942 -> 40.0.0.2:5001 10/20/2025-16:11:13.602167 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:56597 -> 40.0.0.2:5001 10/20/2025-16:11:13.613293 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:56597 -> 40.0.0.2:5001 10/20/2025-16:11:13.625302 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:56597 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43942 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43942 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=56597 dport=5001 packets=14 bytes=19220 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=56597 packets=1 bytes=32 [OFFLOAD, packets=10 bytes=14760 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 75642 0 --:--:-- --:--:-- --:--:-- 97k
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.551 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.551/0.551/0.551/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.468 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.468/0.468/0.468/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/20/2025-16:11:34.199228 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55580 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 55580 yes 201 0 0 0 20.0.0.2 40.0.0.2 55580 5000 yes 201 0 8 673
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250290s ending at 1760976697.889447 XDP_DROP 9 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 14 pkts ( 4 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
10/20/2025-16:11:34.199228 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55580 -> 40.0.0.2:5000 10/20/2025-16:11:38.062529 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41436 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 41436 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 55580 yes 201 0 0 0 20.0.0.2 40.0.0.2 55580 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5001 41436 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250132s ending at 1760976701.779728 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)