Hash-Policy

These scenarios cover some aspects related to link-aggregation. This particular one covers the hash-policy configuration needed to setup the desired load balancing. In OSDx, this can be achieved using bonding interfaces in either xor-hash or 802.3ad modes, or transmit-load-balance and adaptive-load-balance in static mode.

../../../../_images/topology1.svg

Test hash-policy

Description

A bonding interface is configured in DUT0 using xor-hash mode. This scenario shows how to set the hash-policy option to manage load balancing.

Scenario

Step 1: Set the following configuration in DUT0 :

set interfaces bonding bond0 address 192.168.100.10/24
set interfaces bonding bond0 miimon 100
set interfaces bonding bond0 mode xor-hash
set interfaces ethernet eth0 bond-group bond0
set interfaces ethernet eth1 bond-group bond0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 2: Set the following configuration in DUT1 :

set interfaces bridge br0 address 192.168.100.20/24
set interfaces bridge br0 stp version rstp
set interfaces ethernet eth0 bridge-group bridge br0
set interfaces ethernet eth1 bridge-group bridge br0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 3: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 4: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 5: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=466 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 466.487/466.487/466.487/0.000 ms

Step 6: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer2 (0)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 7: Run command interfaces ethernet clear at DUT0.

Step 8: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 36762 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 36774 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 36778 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 36786 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 36794 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 36796 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 36810 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 36818 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 36826 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 36838 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  33.7 MBytes   283 Mbits/sec   80    219 KBytes
[  7]   0.00-1.00   sec  35.6 MBytes   299 Mbits/sec   23    250 KBytes
[  9]   0.00-1.00   sec  30.5 MBytes   256 Mbits/sec  167    153 KBytes
[ 11]   0.00-1.00   sec  27.3 MBytes   229 Mbits/sec   64    191 KBytes
[ 13]   0.00-1.00   sec  25.2 MBytes   211 Mbits/sec   25    182 KBytes
[ 15]   0.00-1.00   sec  32.6 MBytes   273 Mbits/sec  123    219 KBytes
[ 17]   0.00-1.00   sec  33.9 MBytes   284 Mbits/sec  121    246 KBytes
[ 19]   0.00-1.00   sec  35.0 MBytes   294 Mbits/sec   95    257 KBytes
[ 21]   0.00-1.00   sec  25.4 MBytes   213 Mbits/sec   82    185 KBytes
[ 23]   0.00-1.00   sec  20.5 MBytes   172 Mbits/sec   29    158 KBytes
[SUM]   0.00-1.00   sec   300 MBytes  2.51 Gbits/sec  809
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  30.0 MBytes   252 Mbits/sec   28    140 KBytes
[  7]   1.00-2.00   sec  30.4 MBytes   255 Mbits/sec   74    130 KBytes
[  9]   1.00-2.00   sec  30.8 MBytes   259 Mbits/sec   14    192 KBytes
[ 11]   1.00-2.00   sec  31.9 MBytes   268 Mbits/sec   13    205 KBytes
[ 13]   1.00-2.00   sec  29.1 MBytes   244 Mbits/sec   85    117 KBytes
[ 15]   1.00-2.00   sec  33.7 MBytes   283 Mbits/sec   28    206 KBytes
[ 17]   1.00-2.00   sec  29.8 MBytes   250 Mbits/sec   46    185 KBytes
[ 19]   1.00-2.00   sec  29.2 MBytes   245 Mbits/sec   95    143 KBytes
[ 21]   1.00-2.00   sec  31.3 MBytes   262 Mbits/sec   21    204 KBytes
[ 23]   1.00-2.00   sec  28.2 MBytes   237 Mbits/sec   18    157 KBytes
[SUM]   1.00-2.00   sec   304 MBytes  2.55 Gbits/sec  422
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  32.6 MBytes   273 Mbits/sec   28    212 KBytes
[  7]   2.00-3.00   sec  32.0 MBytes   268 Mbits/sec    5    143 KBytes
[  9]   2.00-3.00   sec  34.4 MBytes   288 Mbits/sec   44    122 KBytes
[ 11]   2.00-3.00   sec  34.1 MBytes   286 Mbits/sec   48    211 KBytes
[ 13]   2.00-3.00   sec  28.3 MBytes   237 Mbits/sec   37    178 KBytes
[ 15]   2.00-3.00   sec  33.8 MBytes   283 Mbits/sec   21    206 KBytes
[ 17]   2.00-3.00   sec  37.8 MBytes   317 Mbits/sec    0    294 KBytes
[ 19]   2.00-3.00   sec  32.6 MBytes   274 Mbits/sec   51    215 KBytes
[ 21]   2.00-3.00   sec  35.0 MBytes   294 Mbits/sec   21    236 KBytes
[ 23]   2.00-3.00   sec  33.4 MBytes   280 Mbits/sec   63    218 KBytes
[SUM]   2.00-3.00   sec   334 MBytes  2.80 Gbits/sec  318
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  39.8 MBytes   334 Mbits/sec   26    232 KBytes
[  7]   3.00-4.00   sec  29.1 MBytes   244 Mbits/sec   43    154 KBytes
[  9]   3.00-4.00   sec  32.9 MBytes   276 Mbits/sec    9    191 KBytes
[ 11]   3.00-4.00   sec  35.2 MBytes   295 Mbits/sec   18    201 KBytes
[ 13]   3.00-4.00   sec  30.9 MBytes   259 Mbits/sec   87    154 KBytes
[ 15]   3.00-4.00   sec  32.2 MBytes   270 Mbits/sec   26    163 KBytes
[ 17]   3.00-4.00   sec  33.2 MBytes   278 Mbits/sec  128    154 KBytes
[ 19]   3.00-4.00   sec  32.9 MBytes   276 Mbits/sec   32    164 KBytes
[ 21]   3.00-4.00   sec  33.4 MBytes   280 Mbits/sec   37    174 KBytes
[ 23]   3.00-4.00   sec  32.3 MBytes   271 Mbits/sec   58    170 KBytes
[SUM]   3.00-4.00   sec   332 MBytes  2.78 Gbits/sec  464
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  26.0 MBytes   218 Mbits/sec   67    233 KBytes
[  7]   4.00-5.00   sec  22.4 MBytes   188 Mbits/sec   62    141 KBytes
[  9]   4.00-5.00   sec  24.7 MBytes   207 Mbits/sec   45    212 KBytes
[ 11]   4.00-5.00   sec  24.3 MBytes   204 Mbits/sec   72    164 KBytes
[ 13]   4.00-5.00   sec  22.2 MBytes   186 Mbits/sec   12    201 KBytes
[ 15]   4.00-5.00   sec  23.0 MBytes   193 Mbits/sec    2    195 KBytes
[ 17]   4.00-5.00   sec  22.4 MBytes   188 Mbits/sec   30    189 KBytes
[ 19]   4.00-5.00   sec  23.0 MBytes   193 Mbits/sec   45    195 KBytes
[ 21]   4.00-5.00   sec  25.6 MBytes   215 Mbits/sec    0    253 KBytes
[ 23]   4.00-5.00   sec  25.5 MBytes   214 Mbits/sec    0    249 KBytes
[SUM]   4.00-5.00   sec   239 MBytes  2.01 Gbits/sec  335
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   162 MBytes   272 Mbits/sec  229             sender
[  5]   0.00-5.02   sec   161 MBytes   268 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   150 MBytes   251 Mbits/sec  207             sender
[  7]   0.00-5.02   sec   148 MBytes   247 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   153 MBytes   257 Mbits/sec  279             sender
[  9]   0.00-5.02   sec   152 MBytes   254 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   153 MBytes   256 Mbits/sec  215             sender
[ 11]   0.00-5.02   sec   151 MBytes   253 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   136 MBytes   228 Mbits/sec  246             sender
[ 13]   0.00-5.02   sec   135 MBytes   225 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   155 MBytes   260 Mbits/sec  200             sender
[ 15]   0.00-5.02   sec   154 MBytes   257 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   157 MBytes   264 Mbits/sec  325             sender
[ 17]   0.00-5.02   sec   156 MBytes   260 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   153 MBytes   256 Mbits/sec  318             sender
[ 19]   0.00-5.02   sec   151 MBytes   252 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   151 MBytes   253 Mbits/sec  161             sender
[ 21]   0.00-5.02   sec   149 MBytes   250 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   140 MBytes   235 Mbits/sec  168             sender
[ 23]   0.00-5.02   sec   139 MBytes   232 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.47 GBytes  2.53 Gbits/sec  2348             sender
[SUM]   0.00-5.02   sec  1.46 GBytes  2.50 Gbits/sec                  receiver

iperf Done.

Step 9: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

\s+up\s+(?:[1-9]?\d)\s+\d+\s+\d+\s+(?:[1-9]?\d)
\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        202659  13402218          0     1086496  1644878697          0
eth1  up             0         0          0           0           0          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, traffic is going out through only one of the bonded interfaces.

Step 10: Modify the following configuration lines in DUT0 :

set interfaces bonding bond0 hash-policy layer3+4

Step 11: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 12: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 13: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=0.349 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.349/0.349/0.349/0.000 ms

Step 14: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer3+4 (1)
Show output
Ethernet Channel Bonding Driver: v6.1.162

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 15: Run command interfaces ethernet clear at DUT0.

Step 16: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 36860 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 36868 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 36874 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 36888 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 36902 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 36910 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 36920 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 36924 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 36926 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 36940 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  26.2 MBytes   220 Mbits/sec   44    191 KBytes
[  7]   0.00-1.00   sec  70.6 MBytes   592 Mbits/sec  344    198 KBytes
[  9]   0.00-1.00   sec  39.3 MBytes   330 Mbits/sec   48    240 KBytes
[ 11]   0.00-1.00   sec  65.9 MBytes   552 Mbits/sec  307    230 KBytes
[ 13]   0.00-1.00   sec  49.8 MBytes   417 Mbits/sec  111    273 KBytes
[ 15]   0.00-1.00   sec  57.2 MBytes   480 Mbits/sec  150    315 KBytes
[ 17]   0.00-1.00   sec  21.5 MBytes   181 Mbits/sec  106    110 KBytes
[ 19]   0.00-1.00   sec  26.4 MBytes   221 Mbits/sec  136    124 KBytes
[ 21]   0.00-1.00   sec  19.0 MBytes   160 Mbits/sec   30    161 KBytes
[ 23]   0.00-1.00   sec  17.6 MBytes   148 Mbits/sec   92   96.2 KBytes
[SUM]   0.00-1.00   sec   394 MBytes  3.30 Gbits/sec  1368
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  33.1 MBytes   278 Mbits/sec   32    119 KBytes
[  7]   1.00-2.00   sec  41.2 MBytes   346 Mbits/sec    5    259 KBytes
[  9]   1.00-2.00   sec  35.7 MBytes   299 Mbits/sec   35    209 KBytes
[ 11]   1.00-2.00   sec  35.0 MBytes   294 Mbits/sec   73    178 KBytes
[ 13]   1.00-2.00   sec  41.2 MBytes   346 Mbits/sec   49    202 KBytes
[ 15]   1.00-2.00   sec  38.3 MBytes   321 Mbits/sec   16    235 KBytes
[ 17]   1.00-2.00   sec  21.5 MBytes   180 Mbits/sec   46   94.7 KBytes
[ 19]   1.00-2.00   sec  26.0 MBytes   218 Mbits/sec   88    122 KBytes
[ 21]   1.00-2.00   sec  29.6 MBytes   249 Mbits/sec   26    156 KBytes
[ 23]   1.00-2.00   sec  31.7 MBytes   266 Mbits/sec    0    238 KBytes
[SUM]   1.00-2.00   sec   333 MBytes  2.80 Gbits/sec  370
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  25.4 MBytes   213 Mbits/sec   16    184 KBytes
[  7]   2.00-3.00   sec  43.8 MBytes   367 Mbits/sec   48    296 KBytes
[  9]   2.00-3.00   sec  46.9 MBytes   394 Mbits/sec   92    259 KBytes
[ 11]   2.00-3.00   sec  33.8 MBytes   283 Mbits/sec   84    174 KBytes
[ 13]   2.00-3.00   sec  37.6 MBytes   316 Mbits/sec   70    209 KBytes
[ 15]   2.00-3.00   sec  37.4 MBytes   314 Mbits/sec  116    221 KBytes
[ 17]   2.00-3.00   sec  18.5 MBytes   155 Mbits/sec   67    103 KBytes
[ 19]   2.00-3.00   sec  21.3 MBytes   179 Mbits/sec   72    113 KBytes
[ 21]   2.00-3.00   sec  32.3 MBytes   271 Mbits/sec   32    175 KBytes
[ 23]   2.00-3.00   sec  43.7 MBytes   366 Mbits/sec   53    262 KBytes
[SUM]   2.00-3.00   sec   341 MBytes  2.86 Gbits/sec  650
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  36.0 MBytes   302 Mbits/sec   36    182 KBytes
[  7]   3.00-4.00   sec  42.5 MBytes   357 Mbits/sec   94    199 KBytes
[  9]   3.00-4.00   sec  47.2 MBytes   396 Mbits/sec   40    211 KBytes
[ 11]   3.00-4.00   sec  31.2 MBytes   262 Mbits/sec   67    168 KBytes
[ 13]   3.00-4.00   sec  42.5 MBytes   357 Mbits/sec   27    281 KBytes
[ 15]   3.00-4.00   sec  39.2 MBytes   329 Mbits/sec   82    150 KBytes
[ 17]   3.00-4.00   sec  31.8 MBytes   267 Mbits/sec    7    223 KBytes
[ 19]   3.00-4.00   sec  27.5 MBytes   231 Mbits/sec   66    168 KBytes
[ 21]   3.00-4.00   sec  39.1 MBytes   328 Mbits/sec    4    253 KBytes
[ 23]   3.00-4.00   sec  37.2 MBytes   312 Mbits/sec   76    192 KBytes
[SUM]   3.00-4.00   sec   374 MBytes  3.14 Gbits/sec  499
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  47.4 MBytes   398 Mbits/sec   17    239 KBytes
[  7]   4.00-5.00   sec  38.8 MBytes   325 Mbits/sec   18    216 KBytes
[  9]   4.00-5.00   sec  48.4 MBytes   406 Mbits/sec   39    174 KBytes
[ 11]   4.00-5.00   sec  28.8 MBytes   241 Mbits/sec   39    110 KBytes
[ 13]   4.00-5.00   sec  35.1 MBytes   295 Mbits/sec   71    140 KBytes
[ 15]   4.00-5.00   sec  25.9 MBytes   217 Mbits/sec   75    158 KBytes
[ 17]   4.00-5.00   sec  40.2 MBytes   337 Mbits/sec   66    110 KBytes
[ 19]   4.00-5.00   sec  28.5 MBytes   239 Mbits/sec   25    167 KBytes
[ 21]   4.00-5.00   sec  32.9 MBytes   276 Mbits/sec   79    124 KBytes
[ 23]   4.00-5.00   sec  38.0 MBytes   318 Mbits/sec   36    209 KBytes
[SUM]   4.00-5.00   sec   364 MBytes  3.05 Gbits/sec  465
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   168 MBytes   282 Mbits/sec  145             sender
[  5]   0.00-5.01   sec   167 MBytes   280 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   237 MBytes   397 Mbits/sec  509             sender
[  7]   0.00-5.01   sec   234 MBytes   393 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   218 MBytes   365 Mbits/sec  254             sender
[  9]   0.00-5.01   sec   216 MBytes   362 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   195 MBytes   327 Mbits/sec  570             sender
[ 11]   0.00-5.01   sec   191 MBytes   320 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   206 MBytes   346 Mbits/sec  328             sender
[ 13]   0.00-5.01   sec   204 MBytes   342 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   198 MBytes   332 Mbits/sec  439             sender
[ 15]   0.00-5.01   sec   196 MBytes   328 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   134 MBytes   224 Mbits/sec  292             sender
[ 17]   0.00-5.01   sec   132 MBytes   221 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   130 MBytes   218 Mbits/sec  387             sender
[ 19]   0.00-5.01   sec   129 MBytes   216 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   153 MBytes   257 Mbits/sec  171             sender
[ 21]   0.00-5.01   sec   152 MBytes   254 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   168 MBytes   282 Mbits/sec  257             sender
[ 23]   0.00-5.01   sec   167 MBytes   280 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.76 GBytes  3.03 Gbits/sec  3352             sender
[SUM]   0.00-5.01   sec  1.75 GBytes  3.00 Gbits/sec                  receiver

iperf Done.

Step 17: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

eth0\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
eth1\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        136204   9004486          0      678681  1027408784          0
eth1  up        139658   9242058          0      620577   939459275          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, this time traffic is going out through both bonded interfaces.