While attempting to install the Azure Arc agent on a virtual machine (VM) hosted in the Vultr Cloud, I encountered the following error:
----
Installation failed: [1603]: Fatal error during installation. : See C:\Users\ADMINI~1\AppData\Local\Temp\2\installationlog.txt for additional details.
At line:13 char:5
+ & "$env:TEMP\install_windows_azcmagent.ps1";
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,install_windows_azcmagent.ps1
Details: MSI (s) (28:F0) [23:57:11:300]: Product: Azure Connected Machine Agent -- Error 1920. Service 'Guest Configuration Arc Service' (GCArcService) failed to start. Verify that you have sufficient privileges to start system services.
MSI (s) (28:F0) [23:57:13:215]: Product: Azure Connected Machine Agent -- Installation failed.
MSI (s) (28:F0) [23:57:13:215]: Windows Installer installed the product. Product Name: Azure Connected Machine Agent. Product Version: 1.33.02405.1188. Product Language: 1033. Manufacturer: Microsoft Corporation. Installation success or error status: 1603
-----
After spending a significant amount of time searching for solutions, I reviewed the hybrid agent's log file. You can locate the log file at the following path: C:\ProgramData\AzureConnectedMachineAgent\Log\himds.log
I found this interesting:
level=debug msg="Machine is not on AzSHCI: unable to communicate with IMDS endpoint: unable to communicate with IMDS endpoint: Get \"http://169.254.169.253:80/metadata/attested/document?api-version=2018-10-01\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
level=debug msg="Machine is not on AWS: received empty auth token"
level=debug msg="Machine is not on GCP: unable to communicate with IMDS endpoint: unable to communicate with IMDS endpoint: Get \"http://metadata.google.internal/computeMetadata/v1/?recursive=true\": dial tcp: lookup metadata.google.internal: no such host"
level=debug msg="Machine is not on OCI: received empty IMDS response"
level=debug msg="Machine is not on Azure: received empty IMDS response"
level=info msg="Received service control message: 1"
level=warning msg="Stop service request received"
level=info msg="Service stopped"
At that point, it occurred to me that the issue might be connected to the metadata – Vultr also utilizes metadata. Let's quickly recap what metadata is, drawing from Microsoft's documentation:
"The Azure Instance Metadata Service (IMDS) provides information about currently running virtual machine instances."
--
To resolve the issue, I created an outbound firewall rule that blocked
traffic to 169.254.169.0/24. After implementing this rule and attempting
to reinstall the agent, everything started working as expected, and I
successfully onboarded the VM to Azure Arc.
