How to Troubleshoot Azure Application Gateway

The health probe is the heartbeat of Azure Application Gateway. If it's misconfigured, every backend in the pool gets marked unhealthy, and your users get 502s or 503s even though your app is running perfectly. This is the single most common cause of AGW issues I troubleshoot.

In the Azure Portal, navigate to your Application Gateway → Settings → Health probes. Click on your probe to inspect it. Check these fields carefully:

• Protocol, must match what your backend actually listens on (HTTP vs HTTPS)
• Host, if your backend is a multi-tenant service like App Service, this must be the correct hostname, not blank
• Path, must return a 200 status code. If your app returns 301 redirects from /, change this to a dedicated health endpoint like /health or /ping
• Timeout (seconds), should be lower than the interval. I recommend 30 seconds timeout, 30 seconds interval to start
• Unhealthy threshold, set this to 3, not 1, to avoid flapping on transient failures

To update via PowerShell when you need to script changes across multiple gateways:

$gw = Get-AzApplicationGateway -Name "myAppGateway" -ResourceGroupName "myRG"
$probe = Get-AzApplicationGatewayProbeConfig -ApplicationGateway $gw -Name "myProbe"
Set-AzApplicationGatewayProbeConfig -ApplicationGateway $gw `
  -Name "myProbe" `
  -Protocol Http `
  -Path "/health" `
  -Interval 30 `
  -Timeout 30 `
  -UnhealthyThreshold 3 `
  -Host "myapp.azurewebsites.net" `
  -Match (New-AzApplicationGatewayProbeHealthResponseMatch -StatusCode "200-399")
Set-AzApplicationGateway -ApplicationGateway $gw

After saving, give the probe 2 to 3 minutes to cycle. Then check Backend health again. If backends flip to green, you found your problem.

A 502 from Application Gateway specifically means the gateway established a connection attempt to the backend but either got an invalid HTTP response or the connection was reset/timed out before a valid response arrived. I know how maddening this is, the gateway is running, your app looks fine, but users are hitting a wall.

First, check whether the 502 is consistent or intermittent. Consistent 502s usually point to a configuration problem. Intermittent 502s (especially under load) usually point to backend timeouts or connection pool exhaustion.

For consistent 502s, check your HTTP Settings in the portal under Settings → HTTP settings. Specifically look at:

• Request timeout, default is 20 seconds. If your backend takes longer to respond (e.g., a cold-start on App Service), increase this to 60 or 120 seconds
• Override with new host name, when targeting App Service, set this to "Pick host name from backend target" to avoid hostname mismatch errors
• Cookie-based affinity, if enabled but your backend doesn't support sticky sessions, this can cause session routing failures

For intermittent 502s, enable diagnostic logging and query the access log. Run this in Log Analytics:

AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS"
| where httpStatus_d == 502
| project TimeGenerated, requestUri_s, serverStatus_s, timeTaken_d, clientIP_s
| order by TimeGenerated desc
| take 100

The serverStatus_s field tells you what the backend actually returned (or if the connection was reset). If you see 0 in that field, the backend never responded, that's a timeout, not a bad response. Increase your request timeout in HTTP settings and consider scaling your backend.

If you see consistent backend response codes like 403 or 500, the problem is in your application layer, not the gateway itself.

This one catches people off guard constantly, and it's one of the first things I check when ALL backends go unhealthy at once after a network team makes changes. NSG rules silently drop health probe packets, and Application Gateway interprets that silence as backend failure.

Azure Application Gateway health probes originate from IP addresses in the gateway subnet. But the Azure infrastructure also uses the IP address 168.63.129.16 for management traffic. Your NSG on the backend subnet (or the gateway subnet itself) needs to allow inbound traffic from the gateway's IP range and from the GatewayManager service tag.

Navigate to your backend VM's or App Service's associated NSG. Go to Inbound security rules and verify these rules exist:

# Required inbound rule on backend subnet NSG
Source: Application Gateway subnet CIDR (e.g., 10.0.1.0/24)
Destination: Backend subnet CIDR
Port: Your backend app port (e.g., 80 or 443)
Action: Allow
Priority: Lower number than any Deny rules (e.g., 100)

# Required inbound rule for AGW management
Source: GatewayManager (service tag)
Destination: Any
Port: 65200-65535
Action: Allow

That second rule, ports 65200 through 65535, is specifically for Application Gateway V2 infrastructure communication. Without it, the gateway itself may fail to provision or update. This is a requirement Microsoft documents but buries deep, and I've seen enterprise teams accidentally block it when tightening security posture.

Also check your Application Gateway subnet's NSG. It needs an inbound rule from AzureLoadBalancer service tag to any port, Application Gateway V2 relies on Azure's internal load balancer for its own health.

After updating NSG rules, test backend health again. Changes to NSGs take effect almost immediately, within 30 to 60 seconds, so you don't need to wait long.

End-to-end SSL with Application Gateway is powerful for security, but it's one of the trickier configurations to get right. When you enable HTTPS on your HTTP settings (backend protocol = HTTPS), the gateway validates the backend's certificate. If that validation fails, backends go unhealthy and you get 502 errors, all while the app itself serves HTTPS just fine when accessed directly.

There are two common certificate-related failure patterns I see:

1. Self-signed or private CA certificate on the backend. Application Gateway V2 (Standard_v2 / WAF_v2 SKU) requires you to upload the trusted root certificate of your backend's certificate chain. Go to Settings → HTTP settings, open your HTTP setting, and under Use for App Service or Custom probe, find the CER certificate upload field. Export your backend server's root CA certificate in Base64-encoded X.509 (.CER) format and upload it here.

To export the root certificate from a Windows machine where you have the backend cert installed:

# Open PowerShell as admin
$cert = Get-ChildItem -Path Cert:\LocalMachine\My | Where-Object {$_.Subject -like "*yourbackend*"}
$rootCert = (New-Object System.Security.Cryptography.X509Certificates.X509Chain)
$rootCert.Build($cert) | Out-Null
$rootCA = $rootCert.ChainElements[-1].Certificate
[System.IO.File]::WriteAllBytes("C:\temp\backend-root.cer", $rootCA.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert))

2. Certificate CN/SAN mismatch. If your backend certificate's Common Name or Subject Alternative Name doesn't match the hostname the gateway uses to connect, the TLS handshake fails. Check the Host name override field in your HTTP settings, it must match exactly what's on the backend certificate. Wildcard certificates need the specific subdomain in the host override, not just *.domain.com.

After uploading the correct root certificate and saving HTTP settings, the gateway reprovisioned state typically takes 2 to 5 minutes to fully propagate. Check Backend health after that window.

If you're past the obvious fixes and still seeing intermittent failures, you need data. Gut feelings don't fix Azure Application Gateway, log data does. Enabling diagnostic logging is non-optional for any production AGW deployment, and I'll show you exactly how to get it set up and queried.

In the Azure Portal, go to your Application Gateway → Monitoring → Diagnostic settings. Click Add diagnostic setting. Name it something like agw-logs-la. Check these log categories:

• ApplicationGatewayAccessLog, every request/response with timing, status codes, client IPs, backend IPs
• ApplicationGatewayPerformanceLog, throughput, connection counts, failed requests per backend
• ApplicationGatewayFirewallLog, WAF rule matches and blocks (critical if WAF is enabled)

Send logs to a Log Analytics Workspace. Once data starts flowing (allow 5 to 10 minutes for initial ingestion), use these Kusto queries to dig in:

// Find all non-200 responses and what backend served them
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayAccess"
| where httpStatus_d != 200
| summarize Count = count() by httpStatus_d, serverRouted_s, requestUri_s
| order by Count desc

// Find slowest backend responses (potential timeout candidates)
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayAccess"
| where timeTaken_d > 5000  // milliseconds
| project TimeGenerated, requestUri_s, timeTaken_d, serverRouted_s, httpStatus_d
| order by timeTaken_d desc
| take 50

// WAF blocks, what's being blocked and why
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayFirewall"
| where action_s == "Blocked"
| project TimeGenerated, clientIP_s, requestUri_s, ruleId_s, ruleGroup_s, message_s
| order by TimeGenerated desc

The serverRouted_s field shows you the actual backend IP and port that handled (or failed to handle) each request. Cross-reference that with your backend pool IPs to identify if one specific backend instance is responsible for errors while others are healthy. That narrows your focus dramatically.