RejectedSoftware Forums

Sign up

Issue with HTTPS client requests on 0.7.26

avatar
punkUser
Posted Tue, 10 Nov 2015 16:25:12 GMT
Reply

I'm having what appears to be a pretty basic issue with HTTPS requests on 0.7.26: 

requestHTTP("https://www.google.com/",
            (scope req) {
                req.method = HTTPMethod.GET;
            },
            (scope res) {
                logInfo("Response: %s", res.bodyReader.readAllUTF8());
            }
            );

Produces: 

object.Exception@C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\stream\openssl.d(148): Peer failed the certificate validation: 20
----------------
0x0044B21A in pure @safe void std.exception.bailOut!(Exception).bailOut(immutable(char)[], uint, const(char[])) at C:\D\dmd2\windows\bin\..\..\src\phobos\std\exception.d(400)
0x00428D6E in pure @safe bool std.exception.enforce!(Exception, bool).enforce(bool, lazy const(char)[], immutable(char)[], uint) at C:\D\dmd2\windows\bin\..\..\src\phobos\std\exception.d(352)
0x00470D6F in D4vibe6stream7openssl13OpenSSLStream6__ctorMFC4vibe4core6stream6StreamC4vibe6stream7openssl14Op0C6F1EDE6DF6FB6BD45A5E22DC7179F7 at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\stream\openssl.d(148)
0x004576A9 in vibe.stream.openssl.OpenSSLStream vibe.stream.openssl.OpenSSLContext.createStream(vibe.core.stream.Stream, vibe.stream.tls.TLSStreamState, immutable(char)[], vibe.core.net.NetworkAddress) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\stream\openssl.d(592)
0x0043B877 in vibe.stream.tls.TLSStream vibe.stream.tls.createTLSStream(vibe.core.stream.Stream, vibe.stream.tls.TLSContext, vibe.stream.tls.TLSStreamState, immutable(char)[], vibe.core.net.NetworkAddress) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\stream\tls.d(136)
0x0041C0E0 in bool vibe.http.client.HTTPClient.doRequest(scope void delegate(vibe.http.client.HTTPClientRequest), bool*, bool, std.datetime.SysTime) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\http\client.d(516)
0x0041BB82 in void vibe.http.client.HTTPClient.request(scope void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse)) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\http\client.d(404)
0x004235D8 in void vibe.http.client.requestHTTP(vibe.inet.url.URL, scope void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse), vibe.http.client.HTTPClientSettings) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\http\client.d(137)
0x0040C2B5 in void vibe.http.client.requestHTTP(immutable(char)[], scope void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse), vibe.http.client.HTTPClientSettings) at C:\Users\Andrew\AppData\Roaming\dub\packages\vibe-d-0.7.26\source\vibe\http\client.d(107)
... etc

Reverting to 0.7.25 solves the issue. This is reproducible with a simple program w/ custom main that only calls that one client request so I'm not sure what I'm doing wrong as I can't think of how to simplify this repro further...

Any ideas?

Re: Issue with HTTPS client requests on 0.7.26

avatar
punkUser
Posted Tue, 10 Nov 2015 16:40:37 GMT in reply to punkUser
Reply

I should add: happens both on Linux-x64 and Windows (x86) on different machines/networks. DMD 2.068.2 in both cases.

Re: Issue with HTTPS client requests on 0.7.26

avatar
punkUser
Posted Tue, 10 Nov 2015 22:30:47 GMT in reply to punkUser
Reply

Had a bit more time to debug further into this... seems some code in the "verify" function is the culprit... 

        int verify_callback(int valid, X509_STORE_CTX* ctx)
	{
		X509* err_cert = X509_STORE_CTX_get_current_cert(ctx);
		int err = X509_STORE_CTX_get_error(ctx); // <-- this returns err 20, i.e. X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY
                
                ...
			if (err != X509_V_OK) // <-- this triggers
				logDebug("SSL cert initial error: %s", X509_verify_cert_error_string(err).to!string);

			if (err == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) { // <-- this triggers as well
				logDebug("SSL certificate not accepted by remote.");
				return false;
			}

                        ...
					case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY:
                                                ...
                                                // Never gets to here, which would clear there error...
						if (!(vdata.validationMode & TLSPeerValidationMode.checkTrust)) {
							valid = true;
							err = X509_V_OK;
						}

So it seems like despite it being set to not check the certificate (due to it being a "client" request, and there doesn't seem to be an opportunity to provide a certificate store to requestHTTP), it errors out of that function due to OpenSSL complaining about it being an untrusted certificate.

Forgive my ignorance if I'm off track here as I'm not super-familiar with OpenSSL, but as I mentioned this started in the change from 0.7.25->0.7.26 for me. I tried using the Botan implementation instead but that just crashes outright for me even on simple requests...

Not sure what's up.

Re: Issue with HTTPS client requests on 0.7.26

avatar
Sönke Ludwig
Posted Wed, 11 Nov 2015 09:07:07 +0100 in reply to punkUser
Reply

Am 10.11.2015 um 23:30 schrieb punkUser:

Had a bit more time to debug further into this... seems some code in the "verify" function is the culprit... 

			if (err == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) { // <-- this triggers as well
				logDebug("SSL certificate not accepted by remote.");
				return false;
			}

Should be fixed now by removing the above if block. This was added in
an attempt to improve the error messages, but the semantics of the
checkCert validation mode got affected by this.

I'll tag a 0.7.27 alpha version once travis finishes its test run.

Re: Issue with HTTPS client requests on 0.7.26

avatar
punkUser
Posted Wed, 11 Nov 2015 14:28:06 GMT in reply to Sönke Ludwig
Reply

Should be fixed now by removing the above if block. This was added in
an attempt to improve the error messages, but the semantics of the
checkCert validation mode got affected by this.

I'll tag a 0.7.27 alpha version once travis finishes its test run.

Cheers, thanks! I guess I should do this more via GitHub issues than the forums these days, so sorry for the churn here :)