avi-logo

The control plane VM was unable to authenticate to the load balancer AVI

With vSphere 7 Update 3 beeing released, I wanted to try out the new vSphere with Tanzu installer and check the new options. While my NSX-T environment worked like a charm, I had some trouble with my NSX Advanced LoadBalancer environment (aka AVI).

After running multiple attempts to install it, trying different network setups, checking compatibility guides and even precisely following the installation steps from the documentation, I always ended up with this error message:

The control plane VM 42048b94fa433bbd1e8381f2f7060791 was unable to authenticate to the load balancer (Avi – https://avi-ctrl.vraccoon.lab:443/api/cluster) with the username ‘svc_svc-avi@vraccoon.lab’ and the supplied password. Validate the Supervisor cluster load balancer’s authentication configuration.

It became even more confusing when I checked AVI and noticed that the VirtualServices were actually created (although not up).

And the VM from the error message was actually able to log in.

Neither wcp logs nor ako logs or kube-api logs were showing any errors, which left me puzzled.
Thankfully, there is a great community, where @masanara provided the decisive tip – AVI Controllers are not allowing basic authentication by default.
Not sure if the default has changed on AVI site or if the SupervisorCluster is using another authentication mode than before, but after changing this setting, the installation succeeded.

Configure Basic Authentication

Allowing basic authentication in avi is pretty easy.
SSH to the controller and enter the AVI shell:

admin@avi-ctrl-1's password:

Last login: Sat Oct 30 10:16:47 2021 from 10.135.0.215
admin@avi-ctrl-1:~$
admin@avi-ctrl-1:~$
admin@avi-ctrl-1:~$
admin@avi-ctrl-1:~$ shell --user admin
Password:

[admin:avi-ctrl-1]: >

From here, move to systemconfiguration

[admin:avi-ctrl-1]: > configure systemconfiguration
Updating an existing object. Currently, the object is:
+----------------------------------+------------------------------------+
| Field                            | Value                              |
+----------------------------------+------------------------------------+
| uuid                             | default                            |
| dns_configuration                |                                    |
|   server_list[1]                 | 172.31.1.10                        |
|   search_domain                  | vraccoon.lab                       |
| ntp_configuration                |                                    |
|   ntp_servers[1]                 |                                    |
|     server                       | 0.us.pool.ntp.org                  |
|   ntp_servers[2]                 |                                    |
|     server                       | 1.us.pool.ntp.org                  |
|   ntp_servers[3]                 |                                    |
|     server                       | 2.us.pool.ntp.org                  |
|   ntp_servers[4]                 |                                    |
|     server                       | 3.us.pool.ntp.org                  |
| portal_configuration             |                                    |
|   enable_https                   | True                               |
|   redirect_to_https              | True                               |
|   enable_http                    | True                               |
|   sslkeyandcertificate_refs[1]   | avi-ctrl-crt                       |
|   use_uuid_from_input            | False                              |
|   sslprofile_ref                 | System-Standard-Portal             |
|   enable_clickjacking_protection | True                               |
|   allow_basic_authentication     | False                              |
|   password_strength_check        | True                               |
|   disable_remote_cli_shell       | False                              |
|   disable_swagger                | False                              |
|   api_force_timeout              | 24 hours                           |
|   minimum_password_length        | 8                                  |
| global_tenant_config             |                                    |
|   tenant_vrf                     | False                              |
|   se_in_provider_context         | True                               |
|   tenant_access_to_provider_se   | True                               |
| email_configuration              |                                    |
|   smtp_type                      | SMTP_NONE                          |
|   from_email                     | admin@avicontroller.net            |
|   mail_server_name               | localhost                          |
|   mail_server_port               | 25                                 |
|   disable_tls                    | False                              |
| admin_auth_configuration         |                                    |
|   auth_profile_ref               | vraccoon-AD                        |
|   mapping_rules[1]               |                                    |
|     index                        | 0                                  |
|     group_match                  |                                    |
|       criteria                   | AUTH_MATCH_CONTAINS                |
|       groups[1]                  | g_avi-admins                       |
|     is_superuser                 | True                               |
|   allow_local_user_login         | True                               |
| docker_mode                      | False                              |
| ssh_ciphers[1]                   | aes128-ctr                         |
| ssh_ciphers[2]                   | aes256-ctr                         |
| ssh_hmacs[1]                     | hmac-sha2-512-etm@openssh.com      |
| ssh_hmacs[2]                     | hmac-sha2-256-etm@openssh.com      |
| ssh_hmacs[3]                     | hmac-sha2-512                      |
| default_license_tier             | ENTERPRISE                         |
| secure_channel_configuration     |                                    |
|   sslkeyandcertificate_refs[1]   | System-Default-Secure-Channel-Cert |
| welcome_workflow_complete        | True                               |
| fips_mode                        | False                              |
| enable_cors                      | False                              |
| common_criteria_mode             | False                              |
+----------------------------------+------------------------------------+
[admin:avi-ctrl-1]: systemconfiguration>
[admin:avi-ctrl-1]: systemconfiguration> portal_configuration
[admin:avi-ctrl-1]: systemconfiguration:portal_configuration>

You can actually see the setting already in line 27
To change it to true, navigate to portal_configuration and enter allow_basic_authentication

[admin:avi-ctrl-1]: systemconfiguration> portal_configuration
[admin:avi-ctrl-1]: systemconfiguration:portal_configuration>
[admin:avi-ctrl-1]: systemconfiguration:portal_configuration> allow_basic_authentication
Overwriting the previously entered value for allow_basic_authentication
[admin:avi-ctrl-1]: systemconfiguration:portal_configuration>

Now you can exit out. And validate the setting.

[admin:avi-ctrl-1]: systemconfiguration:portal_configuration> exit
[admin:avi-ctrl-1]: systemconfiguration> exit
+----------------------------------+------------------------------------+
| Field                            | Value                              |
+----------------------------------+------------------------------------+
| uuid                             | default                            |
| dns_configuration                |                                    |
|   server_list[1]                 | 172.31.1.10                        |
|   search_domain                  | vraccoon.lab                       |
| ntp_configuration                |                                    |
|   ntp_servers[1]                 |                                    |
|     server                       | 0.us.pool.ntp.org                  |
|   ntp_servers[2]                 |                                    |
|     server                       | 1.us.pool.ntp.org                  |
|   ntp_servers[3]                 |                                    |
|     server                       | 2.us.pool.ntp.org                  |
|   ntp_servers[4]                 |                                    |
|     server                       | 3.us.pool.ntp.org                  |
| portal_configuration             |                                    |
|   enable_https                   | True                               |
|   redirect_to_https              | True                               |
|   enable_http                    | True                               |
|   sslkeyandcertificate_refs[1]   | avi-ctrl-crt                       |
|   use_uuid_from_input            | False                              |
|   sslprofile_ref                 | System-Standard-Portal             |
|   enable_clickjacking_protection | True                               |
|   allow_basic_authentication     | True                               |
|   password_strength_check        | True                               |
|   disable_remote_cli_shell       | False                              |
|   disable_swagger                | False                              |
|   api_force_timeout              | 24 hours                           |
|   minimum_password_length        | 8                                  |
| global_tenant_config             |                                    |
|   tenant_vrf                     | False                              |
|   se_in_provider_context         | True                               |
|   tenant_access_to_provider_se   | True                               |
| email_configuration              |                                    |
|   smtp_type                      | SMTP_NONE                          |
|   from_email                     | admin@avicontroller.net            |
|   mail_server_name               | localhost                          |
|   mail_server_port               | 25                                 |
|   disable_tls                    | False                              |
| admin_auth_configuration         |                                    |
|   auth_profile_ref               | vraccoon-AD                        |
|   mapping_rules[1]               |                                    |
|     index                        | 0                                  |
|     group_match                  |                                    |
|       criteria                   | AUTH_MATCH_CONTAINS                |
|       groups[1]                  | g_avi-admins                       |
|     is_superuser                 | True                               |
|   allow_local_user_login         | True                               |
| docker_mode                      | False                              |
| ssh_ciphers[1]                   | aes128-ctr                         |
| ssh_ciphers[2]                   | aes256-ctr                         |
| ssh_hmacs[1]                     | hmac-sha2-512-etm@openssh.com      |
| ssh_hmacs[2]                     | hmac-sha2-256-etm@openssh.com      |
| ssh_hmacs[3]                     | hmac-sha2-512                      |
| default_license_tier             | ENTERPRISE                         |
| secure_channel_configuration     |                                    |
|   sslkeyandcertificate_refs[1]   | System-Default-Secure-Channel-Cert |
| welcome_workflow_complete        | True                               |
| fips_mode                        | False                              |
| enable_cors                      | False                              |
| common_criteria_mode             | False                              |
+----------------------------------+------------------------------------+
[admin:avi-ctrl-1]: >

From here on, your SupervisorCluster installation should succeed. You might need to restart it though.

One Reply to “The control plane VM was unable to authenticate to the load balancer AVI”

Leave a Reply

Your email address will not be published. Required fields are marked *