ECMP Support in OpenStack Contrail Environments

The
Avi Load Balancer
can manage load balancing capacity for a virtual service by dynamically scaling it out or in on additional or fewer SEs. By default, the primary SE for the virtual service co-ordinates distribution of traffic flow amongst the secondary SEs, including itself. On OpenStack with Contrail, the
Avi Load Balancer
can take advantage of ECMP support offered by Contrail and manage the orchestration of ECMP routes as part of virtual service placement.
ECMP functionality can be homed at the following locations:
  1. The upstream edge router (for instance, Juniper MX or similar ones).
  2. The Contrail vRouter on the host hypervisor.
Example
In Neutron, the VIP 19.1.1.7 is associated with the interface port of the SE named
Avi-se-czpey
.
root@dc11-cfg-1:~# neutron port-list +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+ | dc4695c6-6d16-4ee0-ab05-774537a33ab2 | Avi-Data...| 02:dc:46:95:c6:6d | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.6"} | | | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} | ... +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
In Contrail, the VIP 19.1.1.7 has the same linkage to port
dc4695c6-6d16-4ee0-ab05-774537a33ab2
and one route to the SE VM running on hypervisor 172.16.11.101.

Virtual Service Scaleout

Using
Avi Load Balancer
CLI, setting
scaleout_ecmp
to
True
enables a scaleout of vs2 as follows:
configure virtualservice vs2 scaleout_ecmp save +----------------------------------+-----------------------------------------------------+ | Field | Value | +----------------------------------+-----------------------------------------------------+ | uuid | virtualservice-2f8770d0-20cc-477f-9419-b206b9624389 | | name | vs2 | | scaleout_ecmp | True | | vip[1] | | | vip_id | 0 | | ip_address | 19.1.1.7 | | enabled | True | | network_ref | 02b846e6-cd68-4c84-b7ab-3709946ed28b | | port_uuid | c0e4cb54-8706-4bed-97bb-271a603f858f | | subnet_uuid | a834986a-385e-4616-9d9c-6b91cfaa51e0 | | subnet | 19.1.1.0/24 | | auto_allocate_ip | True | | auto_allocate_floating_ip | False | ... +----------------------------------+-----------------------------------------------------+ scaleout virtualservice vs2 vip_id 0
After the scaleout, vs2 runs on two SEs,
Avi-se-czpey
and
Avi-se-pnjbb
, as shown in the following summary output.
show virtualservice vs2 summary +-----------------------+-----------------------------------------+ | Field | Value | +-----------------------+-----------------------------------------+ | oper_status | | | state | OPER_UP | | percent_ses_up | 100 | | vip_summary[1] | | | vip_id | 0 | | oper_status | | | state | OPER_UP | | service_engine[1] | | | ref | Avi-se-czpey | | primary | True | | standby | False | | service_engine[2] | | | ref | Avi-se-pnjbb | | primary | False | | standby | False | | num_se_requested | 2 | | num_se_assigned | 2 | +-----------------------+-----------------------------------------+
In Neutron, the VIP 19.1.1.7 is now associated with the interface ports of SE
Avi-se-czpey
and
Avi-se-pnjbb
as well.
root@dc11-cfg-1:~# neutron port-list +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+ | dc4695c6-6d16-4ee0-ab05-774537a33ab2 | Avi-Data...| 02:dc:46:95:c6:6d | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.6"} | | | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} | | 492ee72a-c647-490f-9648-77124e1cf0b6 | Avi-Data...| 02:49:2e:e7:2a:c6 | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.4"} | | | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} | +--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
In Contrail, the VIP 19.1.1.7 has the same linkage to ports
dc4695c6-6d16-4ee0-ab05-774537a33ab2 and 492ee72a-c647-490f-9648-77124e1cf0b6
, and two routes to the SE VM running on hypervisors 172.16.11.101 and 172.16.11.102.
If there are multiple SE instances for a VIP address, the vRouter will receive multiple XMPP routes towards the final destination. The XMPP routes will have different route distinguishers (RDs) to keep them distinct, and they have different next-hops and MPLS labels to identify the different SE instances.

Flow Resiliency During Scale Out/ In

The flow is 5-tuple:
src-IP
,
src-port
,
dst-IP
,
dst-port
, and
protocol
. Routers perform a hash of the 5-tuple to pick a path to use. Contrail vRouter uses flow-tables to preserve the chosen SE destination for a flow. So, even when the number of SE instances in the ECMP group changes, a flow will never move.

Floating-IP ECMP

Using
Avi Load Balancer
CLI, associate a floating IP address with virtual service
vs2
as follows:
configure virtualservice vs2 vip vip_id 0 auto_allocate_floating_ip Overwriting the previously entered value for auto_allocate_floating_ip floating_subnet_uuid 010a4d99-794c-42a9-948c-080014463217 save save +----------------------------------+-----------------------------------------------------+ | Field | Value | +----------------------------------+-----------------------------------------------------+ | uuid | virtualservice-2f8770d0-20cc-477f-9419-b206b9624389 | | name | vs2 | | scaleout_ecmp | True | | vip[1] | | | vip_id | 0 | | ip_address | 19.1.1.7 | | enabled | True | | network_ref | 02b846e6-cd68-4c84-b7ab-3709946ed28b | | port_uuid | c0e4cb54-8706-4bed-97bb-271a603f858f | | subnet_uuid | a834986a-385e-4616-9d9c-6b91cfaa51e0 | | subnet | 19.1.1.0/24 | | auto_allocate_ip | True | | auto_allocate_floating_ip | False | | floating_ip | 10.1.11.4 | | auto_allocate_floating_ip | True | | floating_subnet_uuid | 010a4d99-794c-42a9-948c-080014463217 | ... +----------------------------------+-----------------------------------------------------+
In Contrail, the
floating_ip
is linked to the interface ports of both SEs and two routes to the SE VM running on hypervisors 172.16.11.101 and 172.16.11.102.

Virtual Service Traffic Stats

show virtualservice vs2 detail filter disable_aggregate se | grep connections_handled | connections_handled | 102 | | connections_handled | 49 |
The following screenshot shows the ECMP FIP linkage details.
The following screenshot shows the ECMP FIP routing table details.
The following screenshot shows the ECMP linkage details.
The following screenshot shows the non-ECMP linkage details.
The following screenshot shows the Non-ECMP route 1 details.
The following screenshot shows the ECMP route 2 details.