Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 10638 0 --:--:-- --:--:-- --:--:-- 10750
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.659 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.659/0.659/0.659/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.436 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.436/0.436/0.436/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 56408 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 58.2 MBytes 489 Mbits/sec 459 1.56 MBytes [ 5] 1.00-2.00 sec 78.8 MBytes 661 Mbits/sec 0 1.73 MBytes [ 5] 2.00-3.00 sec 46.2 MBytes 388 Mbits/sec 319 214 KBytes [ 5] 3.00-4.00 sec 51.2 MBytes 430 Mbits/sec 0 488 KBytes [ 5] 4.00-5.00 sec 50.0 MBytes 419 Mbits/sec 0 732 KBytes [ 5] 5.00-6.00 sec 76.2 MBytes 640 Mbits/sec 0 938 KBytes [ 5] 6.00-7.00 sec 104 MBytes 870 Mbits/sec 0 1.08 MBytes [ 5] 7.00-8.00 sec 102 MBytes 860 Mbits/sec 0 1.22 MBytes [ 5] 8.00-9.00 sec 101 MBytes 849 Mbits/sec 0 1.32 MBytes [ 5] 9.00-10.00 sec 70.0 MBytes 587 Mbits/sec 1 1.41 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 738 MBytes 619 Mbits/sec 779 sender [ 5] 0.00-11.09 sec 737 MBytes 557 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/04/2024-15:38:30.176107 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56400 -> 40.0.0.2:5001 12/04/2024-15:38:30.177112 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56408 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 3489 0 --:--:-- --:--:-- --:--:-- 3583
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.867 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.867/0.867/0.867/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.873 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.873/0.873/0.873/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59482 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 56.6 MBytes 475 Mbits/sec 208 1.41 KBytes [ 5] 1.00-2.00 sec 7.50 MBytes 62.9 Mbits/sec 78 2.29 MBytes [ 5] 2.00-3.00 sec 296 MBytes 2.49 Gbits/sec 74 1.85 MBytes [ 5] 3.00-4.00 sec 295 MBytes 2.47 Gbits/sec 324 1.43 MBytes [ 5] 4.00-5.00 sec 179 MBytes 1.50 Gbits/sec 1 1.41 KBytes [ 5] 5.00-6.00 sec 259 MBytes 2.17 Gbits/sec 81 1.93 MBytes [ 5] 6.00-7.00 sec 318 MBytes 2.66 Gbits/sec 0 2.09 MBytes [ 5] 7.00-8.00 sec 270 MBytes 2.26 Gbits/sec 0 2.21 MBytes [ 5] 8.00-9.00 sec 184 MBytes 1.54 Gbits/sec 562 1.39 MBytes [ 5] 9.00-10.00 sec 295 MBytes 2.47 Gbits/sec 0 1.53 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.11 GBytes 1.81 Gbits/sec 1328 sender [ 5] 0.00-10.00 sec 2.11 GBytes 1.81 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/04/2024-15:39:02.620938 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59478 -> 40.0.0.2:5001 12/04/2024-15:39:02.621949 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59482 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Dec 04 15:38:54.000254 osdx systemd-timedated[233927]: Changed local time to Wed 2024-12-04 15:38:54 UTC Dec 04 15:38:54.000833 osdx systemd-journald[1574]: Time jumped backwards, rotating. Dec 04 15:38:54.001087 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Dec 04 15:38:54.001535 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'set date 2024-12-04 15:38:54'. Dec 04 15:38:54.376446 osdx systemd-journald[1574]: Runtime Journal (/run/log/journal/2754912ae611401eb118833e63cf0c07) is 3.4M, max 15.3M, 11.8M free. Dec 04 15:38:54.376953 osdx systemd-journald[1574]: Received client request to rotate journal, rotating. Dec 04 15:38:54.376994 osdx systemd-journald[1574]: Vacuuming done, freed 0B of archived journals from /run/log/journal/2754912ae611401eb118833e63cf0c07. Dec 04 15:38:54.386208 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'system journal clear'. Dec 04 15:38:54.750273 osdx osdx-coredump[233944]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Dec 04 15:38:54.757729 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'system coredump delete all'. Dec 04 15:38:55.197543 osdx OSDxCLI[233601]: User 'admin' entered the configuration menu. Dec 04 15:38:55.262591 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Dec 04 15:38:55.349380 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Dec 04 15:38:55.403533 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Dec 04 15:38:55.498858 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Dec 04 15:38:55.567218 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Dec 04 15:38:55.671184 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Dec 04 15:38:55.762745 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'show working'. Dec 04 15:38:55.900745 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Dec 04 15:38:55.980756 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Dec 04 15:38:56.008511 osdx (udev-worker)[234066]: Network interface NamePolicy= disabled on kernel command line. Dec 04 15:38:56.190651 osdx cfgd[1239]: [233601]Completed change to active configuration Dec 04 15:38:56.218095 osdx OSDxCLI[233601]: User 'admin' committed the configuration. Dec 04 15:38:56.232854 osdx OSDxCLI[233601]: User 'admin' left the configuration menu. Dec 04 15:38:58.352288 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Dec 04 15:38:58.423339 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Dec 04 15:38:58.529695 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Dec 04 15:38:59.073785 osdx file_operation[234150]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Dec 04 15:38:59.131283 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Dec 04 15:38:59.321594 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Dec 04 15:38:59.472491 osdx OSDxCLI[233601]: User 'admin' entered the configuration menu. Dec 04 15:38:59.553774 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Dec 04 15:38:59.661438 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Dec 04 15:38:59.730797 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Dec 04 15:38:59.828040 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Dec 04 15:38:59.903312 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Dec 04 15:39:00.004058 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Dec 04 15:39:00.086365 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Dec 04 15:39:00.168981 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Dec 04 15:39:00.222020 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Dec 04 15:39:00.315425 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Dec 04 15:39:00.368176 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Dec 04 15:39:00.472021 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Dec 04 15:39:00.527964 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Dec 04 15:39:00.625179 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Dec 04 15:39:00.678658 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Dec 04 15:39:00.784868 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Dec 04 15:39:00.838951 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Dec 04 15:39:00.933687 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Dec 04 15:39:00.986354 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Dec 04 15:39:01.079128 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Dec 04 15:39:01.170043 osdx OSDxCLI[233601]: User 'admin' added a new cfg line: 'show working'. Dec 04 15:39:01.791844 osdx systemd[1]: Reloading. Dec 04 15:39:01.900713 osdx systemd-sysv-generator[234289]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Dec 04 15:39:02.017322 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Dec 04 15:39:02.036163 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Dec 04 15:39:02.222139 osdx INFO[234271]: Rules successfully loaded Dec 04 15:39:02.222930 osdx cfgd[1239]: [233601]Completed change to active configuration Dec 04 15:39:02.224816 osdx OSDxCLI[233601]: User 'admin' committed the configuration. Dec 04 15:39:02.257752 osdx OSDxCLI[233601]: User 'admin' left the configuration menu. Dec 04 15:39:02.624907 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=30599 DF PROTO=TCP SPT=59478 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 04 15:39:02.624987 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=21454 DF PROTO=TCP SPT=59482 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Dec 04 15:39:12.817105 osdx systemd[1]: Starting logrotate.service - Rotate log files... Dec 04 15:39:12.825194 osdx OSDxCLI[233601]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Dec 04 15:39:12.848521 osdx systemd[1]: logrotate.service: Deactivated successfully. Dec 04 15:39:12.848631 osdx systemd[1]: Finished logrotate.service - Rotate log files.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then, the test is broaden by configuring “Capture bypass drop” to also avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 21938 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.857 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.857/0.857/0.857/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.893 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.893/0.893/0.893/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 53044 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 167 MBytes 1.40 Gbits/sec 35 1.26 MBytes [ 5] 1.00-2.00 sec 292 MBytes 2.45 Gbits/sec 0 1.42 MBytes [ 5] 2.00-3.00 sec 316 MBytes 2.65 Gbits/sec 0 1.57 MBytes [ 5] 3.00-4.00 sec 290 MBytes 2.43 Gbits/sec 0 1.70 MBytes [ 5] 4.00-5.00 sec 292 MBytes 2.45 Gbits/sec 0 1.82 MBytes [ 5] 5.00-6.00 sec 282 MBytes 2.37 Gbits/sec 153 1.35 MBytes [ 5] 6.00-7.00 sec 295 MBytes 2.47 Gbits/sec 0 1.51 MBytes [ 5] 7.00-8.00 sec 329 MBytes 2.76 Gbits/sec 0 1.65 MBytes [ 5] 8.00-9.00 sec 294 MBytes 2.46 Gbits/sec 0 1.78 MBytes [ 5] 9.00-10.00 sec 352 MBytes 2.96 Gbits/sec 0 1.91 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec 188 sender [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/04/2024-15:39:33.272094 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53036 -> 40.0.0.2:5001 12/04/2024-15:39:33.273088 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53044 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 8 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=53044 dport=5001 packets=2107813 bytes=3161709905 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53044 packets=294972 bytes=15329232 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 18 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=53036 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53036 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 18 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=142 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=142 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 18 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=41 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=41 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 1626 0 --:--:-- --:--:-- --:--:-- 1639
Step 10: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 11: Modify the following configuration lines in DUT0:
set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW ruleset file 'running://drop-performance.rules' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 13: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/04/2024-15:39:33.272094 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53036 -> 40.0.0.2:5001 12/04/2024-15:39:33.273088 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53044 -> 40.0.0.2:5001 12/04/2024-15:39:48.124648 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38048 -> 40.0.0.2:5000
Step 14: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=38048 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=38048 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 15: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP 4 5 210 270 2 FW_SEL_ENQUEUE 1 1 60 60 --------------------------------------------------------------------- Total 5 5 270 270 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP 5 8 409 610 2 FW_SEL_ENQUEUE 3 3 201 201 --------------------------------------------------------------------- Total 8 8 610 610
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 56479 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system conntrack timeout tcp established 60 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set system offload timeout 60 set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.651 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.651/0.651/0.651/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.566 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.566/0.566/0.566/0.000 ms
Step 6: Run command monitor test performance server port 5001 at DUT1.
Step 7: Run command monitor test performance client 40.0.0.2 duration 10 port 5001 at DUT2.
Step 8: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
12/04/2024-15:40:16.468379 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50490 -> 40.0.0.2:5001 12/04/2024-15:40:16.469321 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50492 -> 40.0.0.2:5001
Step 9: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=44 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=44 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=145 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=145 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=50490 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50490 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=5 bytes=388 packets=4 bytes=211] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=50492 dport=5001 packets=16638 bytes=24952701 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50492 packets=2150 bytes=111808 [ASSURED] [OFFLOAD, packets=16625 bytes=24937500 packets=2148 bytes=111696] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 36886 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.520 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.520/0.520/0.520/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.551 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.551/0.551/0.551/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
12/04/2024-15:40:35.387030 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58520 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 58520 yes 201 0 0 0 20.0.0.2 40.0.0.2 58520 5000 yes 201 0 9 731
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250165s ending at 1733326839.259860 XDP_DROP 9 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
12/04/2024-15:40:35.387030 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58520 -> 40.0.0.2:5000 12/04/2024-15:40:39.412292 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:45079 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 58520 yes 201 0 0 0 20.0.0.2 40.0.0.2 58520 5000 yes 201 0 12 905 40.0.0.2 20.0.0.2 5001 45079 no 201 0 0 0 20.0.0.2 40.0.0.2 45079 5001 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250127s ending at 1733326843.038708 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)