Error 502 (Bad Gateway) when sending a request with HttpWebRequest over SSL

Question

I have the following snippet in classic ASP, to send a command and retrieve the response over SSL:

Dim xmlHTTP
Set xmlHTTP = Server.CreateObject("Msxml2.ServerXMLHTTP.3.0")
xmlHTTP.open "POST", "https://www.example.com", False
xmlHTTP.setRequestHeader "Content-Type","application/x-www-form-urlencoded"
xmlHTTP.setRequestHeader "Content-Length", Len(postData)
xmlHTTP.Send postData
If xmlHTTP.status = 200 And Len(message) > 0 And Not Err Then
   Print xmlHTTP.responseText
End If

Then I used this code as a reference to reimplement the request in c#:

private static string SendRequest(string url, string postdata)
{
   WebRequest rqst = HttpWebRequest.Create(url);
   // We have a proxy on the domain, so authentication is required.
   WebProxy proxy = new WebProxy("myproxy.mydomain.com", 8080);
   proxy.Credentials = new NetworkCredential("username", "password", "mydomain");
   rqst.Proxy = proxy;
   rqst.Method = "POST";
   if (!String.IsNullOrEmpty(postdata))
   {
       rqst.ContentType = "application/x-www-form-urlencoded";

       byte[] byteData = Encoding.UTF8.GetBytes(postdata);
       rqst.ContentLength = byteData.Length;
       using (Stream postStream = rqst.GetRequestStream())
       {
           postStream.Write(byteData, 0, byteData.Length);
           postStream.Close();
       }
   }
   ((HttpWebRequest)rqst).KeepAlive = false;
   StreamReader rsps = new StreamReader(rqst.GetResponse().GetResponseStream());
   string strRsps = rsps.ReadToEnd();
   return strRsps;
}

The problem is, when calling GetRequestStream I keep getting a WebException with the message "The remote server returned an error: (502) Bad Gateway."

At first I thought it had to do with the SSL certificate verification. So I added this line:

ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

Where

public class AcceptAllCertificatePolicy : ICertificatePolicy
{
    public bool CheckValidationResult(ServicePoint srvPoint, 
                                      System.Security.Cryptography.X509Certificate certificate,
                                      WebRequest request, 
                                      int certificateProblem)
    {
        return true;
    }
}

And I keep getting the same 502 error. Any ideas?

Answer 1

With the help of this I got a more detailed description of the problem: The proxy was returning the message: "The user agent is not recognized." So I set it manually. Also, I changed the code to use GlobalProxySelection.GetEmptyWebProxy(), as described here. The final working code is included below.

private static string SendRequest(string url, string postdata)
{
    if (String.IsNullOrEmpty(postdata))
        return null;
    HttpWebRequest rqst = (HttpWebRequest)HttpWebRequest.Create(url);
    // No proxy details are required in the code.
    rqst.Proxy = GlobalProxySelection.GetEmptyWebProxy();
    rqst.Method = "POST";
    rqst.ContentType = "application/x-www-form-urlencoded";
    // In order to solve the problem with the proxy not recognising the user
    // agent, a default value is provided here.
    rqst.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)";
    byte[] byteData = Encoding.UTF8.GetBytes(postdata);
    rqst.ContentLength = byteData.Length;

    using (Stream postStream = rqst.GetRequestStream())
    {
        postStream.Write(byteData, 0, byteData.Length);
        postStream.Close();
    }
    StreamReader rsps = new StreamReader(rqst.GetResponse().GetResponseStream());
    string strRsps = rsps.ReadToEnd();
    return strRsps;
}

Answer 2

Read the entity body of the error response. It might have a hint as to what is happening.

The code to do that is as follows:

catch(WebException e)
{
if (e.Status == WebExceptionStatus.ProtocolError)
{
    WebResponse resp = e.Response;
    using(StreamReader sr = new StreamReader(resp.GetResponseStream()))
    {
         Response.Write(sr.ReadToEnd());
    }
}
}

That should show the full contents of the error response.

Answer 3

It is possible that the wsdl for the web service is "arguing" with the domain name and the SSL certificate. IIS will autogenerate a web service's WSDL using the IIS registered domain name (which by default is the machine name on the local domain, not necessarily your web domain). If the certificate domain doesn't match the domain in the SOAP12 Address you will receive communication errors.

Answer 4

UserAgent is missing

for example : request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)";

Answer 5

This was happening for me because a Java proxy on the remote machine was timing out requests if the Java application did not respond in time, rendering the .NET default timeouts kind of useless. The following code loops through all exceptions and writes out responses which helped me determine it was actually coming from the proxy:

static void WriteUnderlyingResponse(Exception exception)
{
    do
    {
        if (exception.GetType() == typeof(WebException))
        {
            var webException = (WebException)exception;
            using (var writer = new StreamReader(webException.Response.GetResponseStream()))
                Console.WriteLine(writer.ReadToEnd());
        }
        exception = exception?.InnerException;
    }
    while (exception != null);
}

The response body from the proxy looked something like this:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/xxx/xxx/xxx">POST&nbsp;/xxx/xxx/xxx</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>