Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 13824 0 --:--:-- --:--:-- --:--:-- 14333
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.518 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.518/0.518/0.518/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.528 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.528/0.528/0.528/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 45940 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 45.9 MBytes 383 Mbits/sec 0 2.09 MBytes [ 5] 1.00-2.00 sec 76.2 MBytes 643 Mbits/sec 53 1.80 MBytes [ 5] 2.00-3.00 sec 108 MBytes 902 Mbits/sec 0 1.96 MBytes [ 5] 3.00-4.00 sec 105 MBytes 881 Mbits/sec 74 1.46 MBytes [ 5] 4.00-5.00 sec 102 MBytes 860 Mbits/sec 0 1.55 MBytes [ 5] 5.00-6.00 sec 57.5 MBytes 482 Mbits/sec 1 1.69 MBytes [ 5] 6.00-7.00 sec 65.0 MBytes 545 Mbits/sec 38 1.73 MBytes [ 5] 7.00-8.00 sec 51.2 MBytes 430 Mbits/sec 0 1.88 MBytes [ 5] 8.00-9.00 sec 52.5 MBytes 440 Mbits/sec 0 1.99 MBytes [ 5] 9.00-10.00 sec 50.0 MBytes 419 Mbits/sec 0 2.08 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 713 MBytes 598 Mbits/sec 166 sender [ 5] 0.00-10.02 sec 712 MBytes 596 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:11:58.504995 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:45930 -> 40.0.0.2:5001 03/10/2025-13:11:58.506099 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:45940 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23968 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.480 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.480/0.480/0.480/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.486 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.486/0.486/0.486/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33670 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 47.6 MBytes 399 Mbits/sec 0 1.48 MBytes [ 5] 1.00-2.00 sec 100 MBytes 839 Mbits/sec 48 1.80 MBytes [ 5] 2.00-3.00 sec 248 MBytes 2.08 Gbits/sec 0 1.99 MBytes [ 5] 3.00-4.00 sec 242 MBytes 2.03 Gbits/sec 5 1.49 MBytes [ 5] 4.00-5.00 sec 259 MBytes 2.17 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 258 MBytes 2.16 Gbits/sec 0 1.72 MBytes [ 5] 6.00-7.00 sec 250 MBytes 2.10 Gbits/sec 0 1.83 MBytes [ 5] 7.00-8.00 sec 258 MBytes 2.16 Gbits/sec 0 1.93 MBytes [ 5] 8.00-9.00 sec 251 MBytes 2.11 Gbits/sec 0 2.02 MBytes [ 5] 9.00-10.00 sec 246 MBytes 2.07 Gbits/sec 0 2.10 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.11 GBytes 1.81 Gbits/sec 53 sender [ 5] 0.00-10.00 sec 2.11 GBytes 1.81 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:12:29.874279 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33662 -> 40.0.0.2:5001 03/10/2025-13:12:29.875145 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33670 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 10 13:12:21.000459 osdx systemd-timedated[370179]: Changed local time to Mon 2025-03-10 13:12:21 UTC Mar 10 13:12:21.002305 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'set date 2025-03-10 13:12:21'. Mar 10 13:12:21.002494 osdx systemd-journald[51744]: Time jumped backwards, rotating. Mar 10 13:12:21.336673 osdx systemd-journald[51744]: Runtime Journal (/run/log/journal/3fdd2a0ddf0a4f0d80cd50f7e198c3e6) is 2.1M, max 15.3M, 13.2M free. Mar 10 13:12:21.338477 osdx systemd-journald[51744]: Received client request to rotate journal, rotating. Mar 10 13:12:21.338530 osdx systemd-journald[51744]: Vacuuming done, freed 0B of archived journals from /run/log/journal/3fdd2a0ddf0a4f0d80cd50f7e198c3e6. Mar 10 13:12:21.346355 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'system journal clear'. Mar 10 13:12:21.673381 osdx osdx-coredump[370197]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Mar 10 13:12:21.681248 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'system coredump delete all'. Mar 10 13:12:22.242132 osdx OSDxCLI[369836]: User 'admin' entered the configuration menu. Mar 10 13:12:22.317682 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 10 13:12:22.405879 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 10 13:12:22.463492 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 10 13:12:22.560612 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 10 13:12:22.620843 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 10 13:12:22.716822 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 10 13:12:22.792020 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'show working'. Mar 10 13:12:22.899816 osdx INFO[370227]: FRR daemons did not change Mar 10 13:12:22.918479 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 10 13:12:22.994475 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 10 13:12:23.020421 osdx (udev-worker)[370328]: Network interface NamePolicy= disabled on kernel command line. Mar 10 13:12:23.221477 osdx cfgd[1455]: [369836]Completed change to active configuration Mar 10 13:12:23.252270 osdx OSDxCLI[369836]: User 'admin' committed the configuration. Mar 10 13:12:23.269781 osdx OSDxCLI[369836]: User 'admin' left the configuration menu. Mar 10 13:12:25.407113 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 10 13:12:25.515776 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 10 13:12:25.597474 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 10 13:12:26.138382 osdx file_operation[370418]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 10 13:12:26.164464 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 10 13:12:26.303093 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 10 13:12:26.455949 osdx OSDxCLI[369836]: User 'admin' entered the configuration menu. Mar 10 13:12:26.541151 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 10 13:12:26.648634 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 10 13:12:26.718583 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 10 13:12:26.818679 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 10 13:12:26.897844 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 10 13:12:26.996832 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 10 13:12:27.075002 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 10 13:12:27.289519 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 10 13:12:27.361814 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 10 13:12:27.480063 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 10 13:12:27.546065 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 10 13:12:27.649447 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 10 13:12:27.730236 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 10 13:12:27.840315 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 10 13:12:27.910451 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 10 13:12:28.017358 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 10 13:12:28.090848 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 10 13:12:28.228604 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 10 13:12:28.281691 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 10 13:12:28.380225 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 10 13:12:28.493948 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'show working'. Mar 10 13:12:28.645730 osdx INFO[370504]: FRR daemons did not change Mar 10 13:12:28.887257 osdx systemd[1]: Reloading. Mar 10 13:12:29.006498 osdx systemd-sysv-generator[370556]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 10 13:12:29.130734 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 10 13:12:29.155689 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 10 13:12:29.376612 osdx INFO[370539]: Rules successfully loaded Mar 10 13:12:29.377881 osdx cfgd[1455]: [369836]Completed change to active configuration Mar 10 13:12:29.381876 osdx OSDxCLI[369836]: User 'admin' committed the configuration. Mar 10 13:12:29.417701 osdx OSDxCLI[369836]: User 'admin' left the configuration menu. Mar 10 13:12:29.874482 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=61995 DF PROTO=TCP SPT=33662 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 10 13:12:29.878484 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=13137 DF PROTO=TCP SPT=33670 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 10 13:12:40.106567 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57728 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 194 MBytes 1.62 Gbits/sec 66 1.87 MBytes [ 5] 1.00-2.00 sec 274 MBytes 2.30 Gbits/sec 0 2.03 MBytes [ 5] 2.00-3.00 sec 278 MBytes 2.33 Gbits/sec 37 1.53 MBytes [ 5] 3.00-4.00 sec 265 MBytes 2.22 Gbits/sec 0 1.65 MBytes [ 5] 4.00-5.00 sec 252 MBytes 2.12 Gbits/sec 0 1.76 MBytes [ 5] 5.00-6.00 sec 264 MBytes 2.21 Gbits/sec 0 1.86 MBytes [ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec 0 1.96 MBytes [ 5] 7.00-8.00 sec 264 MBytes 2.21 Gbits/sec 0 2.06 MBytes [ 5] 8.00-9.00 sec 276 MBytes 2.32 Gbits/sec 0 2.15 MBytes [ 5] 9.00-10.00 sec 264 MBytes 2.21 Gbits/sec 1 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.53 GBytes 2.17 Gbits/sec 104 sender [ 5] 0.00-10.00 sec 2.53 GBytes 2.17 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:12:29.874279 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33662 -> 40.0.0.2:5001 03/10/2025-13:12:29.875145 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33670 -> 40.0.0.2:5001 03/10/2025-13:12:45.592293 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57726 -> 40.0.0.2:5001 03/10/2025-13:12:45.593336 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57728 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Mar 10 13:12:21.000459 osdx systemd-timedated[370179]: Changed local time to Mon 2025-03-10 13:12:21 UTC Mar 10 13:12:21.002305 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'set date 2025-03-10 13:12:21'. Mar 10 13:12:21.002494 osdx systemd-journald[51744]: Time jumped backwards, rotating. Mar 10 13:12:21.336673 osdx systemd-journald[51744]: Runtime Journal (/run/log/journal/3fdd2a0ddf0a4f0d80cd50f7e198c3e6) is 2.1M, max 15.3M, 13.2M free. Mar 10 13:12:21.338477 osdx systemd-journald[51744]: Received client request to rotate journal, rotating. Mar 10 13:12:21.338530 osdx systemd-journald[51744]: Vacuuming done, freed 0B of archived journals from /run/log/journal/3fdd2a0ddf0a4f0d80cd50f7e198c3e6. Mar 10 13:12:21.346355 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'system journal clear'. Mar 10 13:12:21.673381 osdx osdx-coredump[370197]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Mar 10 13:12:21.681248 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'system coredump delete all'. Mar 10 13:12:22.242132 osdx OSDxCLI[369836]: User 'admin' entered the configuration menu. Mar 10 13:12:22.317682 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Mar 10 13:12:22.405879 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Mar 10 13:12:22.463492 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Mar 10 13:12:22.560612 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Mar 10 13:12:22.620843 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Mar 10 13:12:22.716822 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Mar 10 13:12:22.792020 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'show working'. Mar 10 13:12:22.899816 osdx INFO[370227]: FRR daemons did not change Mar 10 13:12:22.918479 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Mar 10 13:12:22.994475 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Mar 10 13:12:23.020421 osdx (udev-worker)[370328]: Network interface NamePolicy= disabled on kernel command line. Mar 10 13:12:23.221477 osdx cfgd[1455]: [369836]Completed change to active configuration Mar 10 13:12:23.252270 osdx OSDxCLI[369836]: User 'admin' committed the configuration. Mar 10 13:12:23.269781 osdx OSDxCLI[369836]: User 'admin' left the configuration menu. Mar 10 13:12:25.407113 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Mar 10 13:12:25.515776 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Mar 10 13:12:25.597474 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Mar 10 13:12:26.138382 osdx file_operation[370418]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Mar 10 13:12:26.164464 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Mar 10 13:12:26.303093 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Mar 10 13:12:26.455949 osdx OSDxCLI[369836]: User 'admin' entered the configuration menu. Mar 10 13:12:26.541151 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 10 13:12:26.648634 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 10 13:12:26.718583 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 10 13:12:26.818679 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 10 13:12:26.897844 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 10 13:12:26.996832 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 10 13:12:27.075002 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 10 13:12:27.289519 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 10 13:12:27.361814 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 10 13:12:27.480063 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 10 13:12:27.546065 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 10 13:12:27.649447 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 10 13:12:27.730236 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 10 13:12:27.840315 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 10 13:12:27.910451 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 10 13:12:28.017358 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 10 13:12:28.090848 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 10 13:12:28.228604 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 10 13:12:28.281691 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 10 13:12:28.380225 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 10 13:12:28.493948 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'show working'. Mar 10 13:12:28.645730 osdx INFO[370504]: FRR daemons did not change Mar 10 13:12:28.887257 osdx systemd[1]: Reloading. Mar 10 13:12:29.006498 osdx systemd-sysv-generator[370556]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Mar 10 13:12:29.130734 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 10 13:12:29.155689 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 10 13:12:29.376612 osdx INFO[370539]: Rules successfully loaded Mar 10 13:12:29.377881 osdx cfgd[1455]: [369836]Completed change to active configuration Mar 10 13:12:29.381876 osdx OSDxCLI[369836]: User 'admin' committed the configuration. Mar 10 13:12:29.417701 osdx OSDxCLI[369836]: User 'admin' left the configuration menu. Mar 10 13:12:29.874482 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=61995 DF PROTO=TCP SPT=33662 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 10 13:12:29.878484 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=13137 DF PROTO=TCP SPT=33670 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Mar 10 13:12:40.106567 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Mar 10 13:12:40.207458 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'system journal show | cat'. Mar 10 13:12:40.513767 osdx OSDxCLI[369836]: User 'admin' entered the configuration menu. Mar 10 13:12:40.573791 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Mar 10 13:12:40.677618 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Mar 10 13:12:40.733803 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Mar 10 13:12:40.838449 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Mar 10 13:12:40.897616 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Mar 10 13:12:40.996992 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Mar 10 13:12:41.086311 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Mar 10 13:12:41.161238 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Mar 10 13:12:41.258120 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Mar 10 13:12:41.317819 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Mar 10 13:12:41.415160 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Mar 10 13:12:41.636029 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Mar 10 13:12:41.694502 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Mar 10 13:12:41.796636 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Mar 10 13:12:41.849213 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Mar 10 13:12:41.953623 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Mar 10 13:12:42.010294 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Mar 10 13:12:42.129103 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Mar 10 13:12:42.188279 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Mar 10 13:12:42.288772 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Mar 10 13:12:42.347215 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Mar 10 13:12:42.444329 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Mar 10 13:12:42.504737 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Mar 10 13:12:42.618572 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Mar 10 13:12:42.710871 osdx OSDxCLI[369836]: User 'admin' added a new cfg line: 'show changes'. Mar 10 13:12:42.831079 osdx INFO[370693]: FRR daemons did not change Mar 10 13:12:43.140389 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Mar 10 13:12:45.128369 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Mar 10 13:12:45.128482 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Mar 10 13:12:45.128512 osdx systemd[1]: suricata@FW.service: Consumed 2.142s CPU time. Mar 10 13:12:45.142841 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Mar 10 13:12:45.159799 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Mar 10 13:12:45.354262 osdx INFO[370718]: Rules successfully loaded Mar 10 13:12:45.354887 osdx cfgd[1455]: [369836]Completed change to active configuration Mar 10 13:12:45.357163 osdx OSDxCLI[369836]: User 'admin' committed the configuration. Mar 10 13:12:45.384527 osdx OSDxCLI[369836]: User 'admin' left the configuration menu. Mar 10 13:12:45.594492 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=55058 DF PROTO=TCP SPT=57726 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 10 13:12:45.594571 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=32548 DF PROTO=TCP SPT=57728 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Mar 10 13:12:51.031941 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Mar 10 13:12:55.822106 osdx OSDxCLI[369836]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 22599 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.574 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.574/0.574/0.574/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.553 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.553/0.553/0.553/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 48378 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 51.6 MBytes 433 Mbits/sec 0 1.90 MBytes [ 5] 1.00-2.00 sec 189 MBytes 1.58 Gbits/sec 6 1.82 MBytes [ 5] 2.00-3.00 sec 269 MBytes 2.25 Gbits/sec 0 1.98 MBytes [ 5] 3.00-4.00 sec 245 MBytes 2.06 Gbits/sec 0 2.11 MBytes [ 5] 4.00-5.00 sec 298 MBytes 2.50 Gbits/sec 0 2.21 MBytes [ 5] 5.00-6.00 sec 248 MBytes 2.08 Gbits/sec 365 1.61 MBytes [ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec 0 1.71 MBytes [ 5] 7.00-8.00 sec 261 MBytes 2.19 Gbits/sec 0 1.82 MBytes [ 5] 8.00-9.00 sec 240 MBytes 2.01 Gbits/sec 0 1.91 MBytes [ 5] 9.00-10.00 sec 234 MBytes 1.96 Gbits/sec 180 1.43 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.24 GBytes 1.93 Gbits/sec 551 sender [ 5] 0.00-10.00 sec 2.24 GBytes 1.92 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:13:17.923543 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48364 -> 40.0.0.2:5001 03/10/2025-13:13:17.924397 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48378 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=48364 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48364 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 udp 17 28 src=127.0.0.1 dst=127.0.0.1 sport=56415 dport=2055 packets=1 bytes=506 [UNREPLIED] src=127.0.0.1 dst=127.0.0.1 sport=2055 dport=56415 packets=0 bytes=0 mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=118 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=118 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=571 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=571 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=48378 dport=5001 packets=1662026 bytes=2493028277 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48378 packets=255003 bytes=13256324 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 conntrack v1.4.7 (conntrack-tools): 5 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 52406 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 198 MBytes 1.66 Gbits/sec 60 1.32 MBytes [ 5] 1.00-2.00 sec 234 MBytes 1.96 Gbits/sec 0 1.43 MBytes [ 5] 2.00-3.00 sec 252 MBytes 2.12 Gbits/sec 0 1.56 MBytes [ 5] 3.00-4.00 sec 235 MBytes 1.97 Gbits/sec 0 1.66 MBytes [ 5] 4.00-5.00 sec 270 MBytes 2.26 Gbits/sec 0 1.74 MBytes [ 5] 5.00-6.00 sec 258 MBytes 2.16 Gbits/sec 228 1.35 MBytes [ 5] 6.00-7.00 sec 286 MBytes 2.40 Gbits/sec 0 1.47 MBytes [ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec 0 1.60 MBytes [ 5] 8.00-9.00 sec 278 MBytes 2.33 Gbits/sec 0 1.72 MBytes [ 5] 9.00-10.00 sec 246 MBytes 2.07 Gbits/sec 0 1.82 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.48 GBytes 2.13 Gbits/sec 288 sender [ 5] 0.00-10.01 sec 2.47 GBytes 2.12 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:13:17.923543 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48364 -> 40.0.0.2:5001 03/10/2025-13:13:17.924397 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48378 -> 40.0.0.2:5001 03/10/2025-13:13:32.725498 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52404 -> 40.0.0.2:5001 03/10/2025-13:13:32.726327 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52406 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=52406 dport=5001 packets=1836056 bytes=2754076333 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52406 packets=233254 bytes=12124580 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=52404 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52404 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 846 0 --:--:-- --:--:-- --:--:-- 847
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.538 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.538/0.538/0.538/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.432 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.432/0.432/0.432/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/10/2025-13:14:06.215490 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35898 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=121 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=121 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=35898 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=35898 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=574 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=574 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/10/2025-13:14:06.215490 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35898 -> 40.0.0.2:5000 03/10/2025-13:14:12.630462 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35898 -> 40.0.0.2:5000 03/10/2025-13:14:14.098761 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44654 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=44654 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=44654 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 54338 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.610 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.610/0.610/0.610/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.459 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.459/0.459/0.459/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
03/10/2025-13:14:37.051463 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33878 -> 40.0.0.2:5001 03/10/2025-13:14:37.052607 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33888 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=124 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=124 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33888 dport=5001 packets=14301 bytes=21447201 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33888 packets=1976 bytes=102760 [ASSURED] [OFFLOAD, packets=14288 bytes=21432000 packets=1974 bytes=102648] mark=129834765 use=2 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=577 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=577 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33878 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33878 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=5 bytes=388 packets=4 bytes=211] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 3666 0 --:--:-- --:--:-- --:--:-- 3694
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
03/10/2025-13:14:37.051463 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33878 -> 40.0.0.2:5001 03/10/2025-13:14:37.052607 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33888 -> 40.0.0.2:5001 03/10/2025-13:14:40.177152 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33904 -> 40.0.0.2:5001 03/10/2025-13:14:40.178797 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:33641 -> 40.0.0.2:5001 03/10/2025-13:14:40.189363 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:33641 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33904 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33904 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=5 bytes=406 packets=4 bytes=211] mark=129834765 use=2 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=33641 dport=5001 packets=16 bytes=22172 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33641 packets=1 bytes=32 [OFFLOAD, packets=13 bytes=19188 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 35417 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.490 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.490/0.490/0.490/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.507 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.507/0.507/0.507/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
03/10/2025-13:15:00.124198 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40874 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 40874 yes 201 0 0 0 20.0.0.2 40.0.0.2 40874 5000 yes 201 0 9 731
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250141s ending at 1741612503.845812 XDP_DROP 10 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9419936 pkts ( 8 pps) 12092229 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
03/10/2025-13:15:00.124198 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40874 -> 40.0.0.2:5000 03/10/2025-13:15:03.997674 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43197 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 40874 yes 201 0 0 0 40.0.0.2 20.0.0.2 5001 43197 no 201 0 0 0 20.0.0.2 40.0.0.2 43197 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 40874 5000 yes 201 0 12 905
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250129s ending at 1741612507.693981 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 9419974 pkts ( 0 pps) 12092232 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)