Hash-Policy

These scenarios cover some aspects related to link-aggregation. This particular one covers the hash-policy configuration needed to setup the desired load balancing. In OSDx, this can be achieved using bonding interfaces in either xor-hash or 802.3ad modes, or transmit-load-balance and adaptive-load-balance in static mode.

../../../../_images/topology1.svg

Test hash-policy

Description

A bonding interface is configured in DUT0 using xor-hash mode. This scenario shows how to set the hash-policy option to manage load balancing.

Scenario

Step 1: Set the following configuration in DUT0 :

set interfaces bonding bond0 address 192.168.100.10/24
set interfaces bonding bond0 miimon 100
set interfaces bonding bond0 mode xor-hash
set interfaces ethernet eth0 bond-group bond0
set interfaces ethernet eth1 bond-group bond0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 2: Set the following configuration in DUT1 :

set interfaces bridge br0 address 192.168.100.20/24
set interfaces bridge br0 stp version rstp
set interfaces ethernet eth0 bridge-group bridge br0
set interfaces ethernet eth1 bridge-group bridge br0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 3: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 4: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 5: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=443 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 442.942/442.942/442.942/0.000 ms

Step 6: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer2 (0)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 7: Run command interfaces ethernet clear at DUT0.

Step 8: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 37802 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 37818 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 37830 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 37840 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 37844 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 37846 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 37854 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 37868 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 37880 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 37892 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  59.2 MBytes   497 Mbits/sec   12    581 KBytes
[  7]   0.00-1.00   sec  37.9 MBytes   318 Mbits/sec   67    218 KBytes
[  9]   0.00-1.00   sec  43.1 MBytes   362 Mbits/sec  134    250 KBytes
[ 11]   0.00-1.00   sec  36.3 MBytes   304 Mbits/sec   25    233 KBytes
[ 13]   0.00-1.00   sec  35.5 MBytes   298 Mbits/sec   56    206 KBytes
[ 15]   0.00-1.00   sec  27.4 MBytes   230 Mbits/sec  149    160 KBytes
[ 17]   0.00-1.00   sec  24.9 MBytes   209 Mbits/sec   72    148 KBytes
[ 19]   0.00-1.00   sec  43.6 MBytes   366 Mbits/sec   45    291 KBytes
[ 21]   0.00-1.00   sec  33.8 MBytes   283 Mbits/sec   50    197 KBytes
[ 23]   0.00-1.00   sec  32.5 MBytes   272 Mbits/sec   92    202 KBytes
[SUM]   0.00-1.00   sec   374 MBytes  3.14 Gbits/sec  702
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  37.5 MBytes   315 Mbits/sec  125    181 KBytes
[  7]   1.00-2.00   sec  30.3 MBytes   254 Mbits/sec   77    116 KBytes
[  9]   1.00-2.00   sec  32.7 MBytes   275 Mbits/sec  135    150 KBytes
[ 11]   1.00-2.00   sec  34.6 MBytes   290 Mbits/sec   62    177 KBytes
[ 13]   1.00-2.00   sec  29.7 MBytes   249 Mbits/sec   80    116 KBytes
[ 15]   1.00-2.00   sec  30.1 MBytes   253 Mbits/sec   13    182 KBytes
[ 17]   1.00-2.00   sec  26.9 MBytes   226 Mbits/sec   51    110 KBytes
[ 19]   1.00-2.00   sec  37.3 MBytes   313 Mbits/sec   25    235 KBytes
[ 21]   1.00-2.00   sec  33.2 MBytes   279 Mbits/sec    5    245 KBytes
[ 23]   1.00-2.00   sec  35.2 MBytes   295 Mbits/sec   57    199 KBytes
[SUM]   1.00-2.00   sec   328 MBytes  2.75 Gbits/sec  630
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  42.5 MBytes   357 Mbits/sec   11    270 KBytes
[  7]   2.00-3.00   sec  38.3 MBytes   321 Mbits/sec   26    146 KBytes
[  9]   2.00-3.00   sec  35.7 MBytes   299 Mbits/sec   67    134 KBytes
[ 11]   2.00-3.00   sec  34.0 MBytes   285 Mbits/sec  133    120 KBytes
[ 13]   2.00-3.00   sec  31.4 MBytes   263 Mbits/sec   68    137 KBytes
[ 15]   2.00-3.00   sec  39.8 MBytes   334 Mbits/sec   31    137 KBytes
[ 17]   2.00-3.00   sec  35.4 MBytes   297 Mbits/sec   26    160 KBytes
[ 19]   2.00-3.00   sec  37.6 MBytes   315 Mbits/sec   70    127 KBytes
[ 21]   2.00-3.00   sec  43.6 MBytes   366 Mbits/sec  125    218 KBytes
[ 23]   2.00-3.00   sec  38.7 MBytes   324 Mbits/sec   10    143 KBytes
[SUM]   2.00-3.00   sec   377 MBytes  3.16 Gbits/sec  567
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  41.2 MBytes   346 Mbits/sec   49    168 KBytes
[  7]   3.00-4.00   sec  31.9 MBytes   267 Mbits/sec   48    134 KBytes
[  9]   3.00-4.00   sec  32.7 MBytes   274 Mbits/sec   20    188 KBytes
[ 11]   3.00-4.00   sec  36.7 MBytes   308 Mbits/sec   23    187 KBytes
[ 13]   3.00-4.00   sec  35.8 MBytes   300 Mbits/sec   36    157 KBytes
[ 15]   3.00-4.00   sec  38.2 MBytes   321 Mbits/sec   57    192 KBytes
[ 17]   3.00-4.00   sec  37.0 MBytes   311 Mbits/sec    4    163 KBytes
[ 19]   3.00-4.00   sec  38.7 MBytes   324 Mbits/sec    0    272 KBytes
[ 21]   3.00-4.00   sec  40.3 MBytes   338 Mbits/sec   34    192 KBytes
[ 23]   3.00-4.00   sec  30.3 MBytes   254 Mbits/sec   64    132 KBytes
[SUM]   3.00-4.00   sec   363 MBytes  3.04 Gbits/sec  335
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  32.5 MBytes   273 Mbits/sec   66    192 KBytes
[  7]   4.00-5.00   sec  31.0 MBytes   260 Mbits/sec   31    173 KBytes
[  9]   4.00-5.00   sec  37.9 MBytes   318 Mbits/sec   23    226 KBytes
[ 11]   4.00-5.00   sec  34.0 MBytes   285 Mbits/sec   23    199 KBytes
[ 13]   4.00-5.00   sec  28.1 MBytes   236 Mbits/sec   42    173 KBytes
[ 15]   4.00-5.00   sec  34.0 MBytes   285 Mbits/sec   88    198 KBytes
[ 17]   4.00-5.00   sec  35.3 MBytes   296 Mbits/sec    2    240 KBytes
[ 19]   4.00-5.00   sec  38.8 MBytes   326 Mbits/sec   87    219 KBytes
[ 21]   4.00-5.00   sec  34.6 MBytes   290 Mbits/sec   51    191 KBytes
[ 23]   4.00-5.00   sec  33.2 MBytes   278 Mbits/sec   26    140 KBytes
[SUM]   4.00-5.00   sec   339 MBytes  2.85 Gbits/sec  439
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   213 MBytes   357 Mbits/sec  263             sender
[  5]   0.00-5.00   sec   210 MBytes   353 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   169 MBytes   284 Mbits/sec  249             sender
[  7]   0.00-5.00   sec   168 MBytes   282 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   182 MBytes   306 Mbits/sec  379             sender
[  9]   0.00-5.00   sec   180 MBytes   301 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   176 MBytes   294 Mbits/sec  266             sender
[ 11]   0.00-5.00   sec   175 MBytes   293 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   160 MBytes   269 Mbits/sec  282             sender
[ 13]   0.00-5.00   sec   159 MBytes   267 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   170 MBytes   285 Mbits/sec  338             sender
[ 15]   0.00-5.00   sec   168 MBytes   282 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   159 MBytes   268 Mbits/sec  155             sender
[ 17]   0.00-5.00   sec   158 MBytes   265 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   196 MBytes   329 Mbits/sec  227             sender
[ 19]   0.00-5.00   sec   195 MBytes   327 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   186 MBytes   311 Mbits/sec  265             sender
[ 21]   0.00-5.00   sec   184 MBytes   309 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   170 MBytes   285 Mbits/sec  249             sender
[ 23]   0.00-5.00   sec   168 MBytes   282 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.74 GBytes  2.99 Gbits/sec  2673             sender
[SUM]   0.00-5.00   sec  1.72 GBytes  2.96 Gbits/sec                  receiver

iperf Done.

Step 9: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

\s+up\s+(?:[1-9]?\d)\s+\d+\s+\d+\s+(?:[1-9]?\d)
\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        258236  17071474          0     1282797  1942076825          0
eth1  up             0         0          0           0           0          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, traffic is going out through only one of the bonded interfaces.

Step 10: Modify the following configuration lines in DUT0 :

set interfaces bonding bond0 hash-policy layer3+4

Step 11: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 12: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 13: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=0.250 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.250/0.250/0.250/0.000 ms

Step 14: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer3+4 (1)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 15: Run command interfaces ethernet clear at DUT0.

Step 16: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 54948 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 54956 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 54968 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 54980 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 54982 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 54996 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 55000 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 55012 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 55026 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 55030 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  27.1 MBytes   227 Mbits/sec   87    137 KBytes
[  7]   0.00-1.00   sec  28.2 MBytes   237 Mbits/sec   48    188 KBytes
[  9]   0.00-1.00   sec  57.2 MBytes   480 Mbits/sec  172    188 KBytes
[ 11]   0.00-1.00   sec  32.9 MBytes   276 Mbits/sec   26    204 KBytes
[ 13]   0.00-1.00   sec  67.8 MBytes   569 Mbits/sec   94    239 KBytes
[ 15]   0.00-1.00   sec  25.1 MBytes   211 Mbits/sec   58    174 KBytes
[ 17]   0.00-1.00   sec  32.7 MBytes   275 Mbits/sec   89    124 KBytes
[ 19]   0.00-1.00   sec  51.4 MBytes   431 Mbits/sec  110    240 KBytes
[ 21]   0.00-1.00   sec  35.6 MBytes   298 Mbits/sec   96    180 KBytes
[ 23]   0.00-1.00   sec  47.8 MBytes   401 Mbits/sec  160    185 KBytes
[SUM]   0.00-1.00   sec   406 MBytes  3.40 Gbits/sec  940
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  39.2 MBytes   329 Mbits/sec    0    280 KBytes
[  7]   1.00-2.00   sec  29.0 MBytes   243 Mbits/sec   63   76.4 KBytes
[  9]   1.00-2.00   sec  39.8 MBytes   334 Mbits/sec   63    243 KBytes
[ 11]   1.00-2.00   sec  32.6 MBytes   274 Mbits/sec   64    209 KBytes
[ 13]   1.00-2.00   sec  46.7 MBytes   392 Mbits/sec   47    133 KBytes
[ 15]   1.00-2.00   sec  27.3 MBytes   229 Mbits/sec   58    148 KBytes
[ 17]   1.00-2.00   sec  26.5 MBytes   222 Mbits/sec   10    165 KBytes
[ 19]   1.00-2.00   sec  37.7 MBytes   316 Mbits/sec   50    147 KBytes
[ 21]   1.00-2.00   sec  43.8 MBytes   367 Mbits/sec   41    153 KBytes
[ 23]   1.00-2.00   sec  44.6 MBytes   374 Mbits/sec   72    153 KBytes
[SUM]   1.00-2.00   sec   367 MBytes  3.08 Gbits/sec  468
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  37.2 MBytes   312 Mbits/sec   52    219 KBytes
[  7]   2.00-3.00   sec  18.3 MBytes   153 Mbits/sec   39    136 KBytes
[  9]   2.00-3.00   sec  44.8 MBytes   376 Mbits/sec    0    342 KBytes
[ 11]   2.00-3.00   sec  32.6 MBytes   274 Mbits/sec   21    184 KBytes
[ 13]   2.00-3.00   sec  33.2 MBytes   279 Mbits/sec   18    173 KBytes
[ 15]   2.00-3.00   sec  24.6 MBytes   206 Mbits/sec   27    174 KBytes
[ 17]   2.00-3.00   sec  32.7 MBytes   275 Mbits/sec   28    212 KBytes
[ 19]   2.00-3.00   sec  23.9 MBytes   201 Mbits/sec   23    174 KBytes
[ 21]   2.00-3.00   sec  36.8 MBytes   309 Mbits/sec   31    184 KBytes
[ 23]   2.00-3.00   sec  29.9 MBytes   251 Mbits/sec   47    153 KBytes
[SUM]   2.00-3.00   sec   314 MBytes  2.63 Gbits/sec  286
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  35.4 MBytes   297 Mbits/sec   87    117 KBytes
[  7]   3.00-4.00   sec  31.6 MBytes   265 Mbits/sec   85    127 KBytes
[  9]   3.00-4.00   sec  43.5 MBytes   365 Mbits/sec  120    229 KBytes
[ 11]   3.00-4.00   sec  42.7 MBytes   358 Mbits/sec   66    221 KBytes
[ 13]   3.00-4.00   sec  49.2 MBytes   413 Mbits/sec   75    194 KBytes
[ 15]   3.00-4.00   sec  33.6 MBytes   282 Mbits/sec   60    107 KBytes
[ 17]   3.00-4.00   sec  31.8 MBytes   267 Mbits/sec   56    132 KBytes
[ 19]   3.00-4.00   sec  26.5 MBytes   223 Mbits/sec   43    117 KBytes
[ 21]   3.00-4.00   sec  53.4 MBytes   448 Mbits/sec   42    198 KBytes
[ 23]   3.00-4.00   sec  44.6 MBytes   374 Mbits/sec   67    185 KBytes
[SUM]   3.00-4.00   sec   392 MBytes  3.29 Gbits/sec  701
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  33.4 MBytes   280 Mbits/sec   13    216 KBytes
[  7]   4.00-5.00   sec  25.5 MBytes   214 Mbits/sec   86    117 KBytes
[  9]   4.00-5.00   sec  40.5 MBytes   340 Mbits/sec  107    157 KBytes
[ 11]   4.00-5.00   sec  46.0 MBytes   386 Mbits/sec   32    197 KBytes
[ 13]   4.00-5.00   sec  49.2 MBytes   413 Mbits/sec   63    161 KBytes
[ 15]   4.00-5.00   sec  25.0 MBytes   210 Mbits/sec   86    120 KBytes
[ 17]   4.00-5.00   sec  39.3 MBytes   329 Mbits/sec   36    197 KBytes
[ 19]   4.00-5.00   sec  30.8 MBytes   259 Mbits/sec   14    146 KBytes
[ 21]   4.00-5.00   sec  54.2 MBytes   454 Mbits/sec   27    216 KBytes
[ 23]   4.00-5.00   sec  52.6 MBytes   441 Mbits/sec   67    215 KBytes
[SUM]   4.00-5.00   sec   396 MBytes  3.33 Gbits/sec  531
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   172 MBytes   289 Mbits/sec  239             sender
[  5]   0.00-5.01   sec   171 MBytes   286 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   133 MBytes   223 Mbits/sec  321             sender
[  7]   0.00-5.01   sec   132 MBytes   220 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   226 MBytes   379 Mbits/sec  462             sender
[  9]   0.00-5.01   sec   223 MBytes   374 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   187 MBytes   314 Mbits/sec  209             sender
[ 11]   0.00-5.01   sec   185 MBytes   310 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   246 MBytes   413 Mbits/sec  297             sender
[ 13]   0.00-5.01   sec   243 MBytes   408 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   136 MBytes   228 Mbits/sec  289             sender
[ 15]   0.00-5.01   sec   134 MBytes   225 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   163 MBytes   274 Mbits/sec  219             sender
[ 17]   0.00-5.01   sec   162 MBytes   271 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   170 MBytes   286 Mbits/sec  240             sender
[ 19]   0.00-5.01   sec   168 MBytes   282 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   224 MBytes   375 Mbits/sec  237             sender
[ 21]   0.00-5.01   sec   222 MBytes   372 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   219 MBytes   368 Mbits/sec  413             sender
[ 23]   0.00-5.01   sec   218 MBytes   365 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.83 GBytes  3.15 Gbits/sec  2926             sender
[SUM]   0.00-5.01   sec  1.81 GBytes  3.11 Gbits/sec                  receiver

iperf Done.

Step 17: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

eth0\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
eth1\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        187059  12367103          0      853803  1292439945          0
eth1  up        123889   8197780          0      496222   751185720          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, this time traffic is going out through both bonded interfaces.