Client certificate validation issues - WCF

We encountered an issue using client certificate authentication with a client certificate issued by a CA in the Trusted Root CA store.

The client certificate chain is valid when opening it on the server, showing the whole chain. When attaching it to a request the request would be rejected with response code 403 - Forbidden. Looking at the System eventlog for SChannel errors showed nothing.

Eventually we switched on WCF tracing, and found the following in the trace:

<TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Warning">
    <TraceIdentifier>http://msdn.microsoft.com/en-ZA/library/System.ServiceModel.Channels.HttpsClientCertificateInvalid.aspx</TraceIdentifier>
    <Description>Client certificate is invalid with native error code 0x2013 (see http://go.microsoft.com/fwlink/?LinkId=187517 for details).</Description>
    <AppDomain>MyApp.exe</AppDomain>
    <Source>System.ServiceModel.Channels.HttpsChannelListener`1[System.ServiceModel.Channels.IReplyChannel]/30640645</Source>
    <ExtendedData xmlns="http://schemas.microsoft.com/2006/08/ServiceModel/HttpRequestTraceRecord">
        <Headers>
            <SOAPAction>"Authorize"</SOAPAction>
            <Connection>Keep-Alive</Connection>
            <Content-Length>1281</Content-Length>
            <Content-Type>text/xml;charset=UTF-8</Content-Type>
            <Accept-Encoding>gzip,deflate</Accept-Encoding>
            <Host>MyAppURL</Host>
            <User-Agent>Apache-HttpClient/4.1.1 (java 1.5)</User-Agent>
        </Headers>
        <Url>https://MyAppURL</Url>
    </ExtendedData>
</TraceRecord>

Searching on Google brought us no closer, eventually I found an old link I've used before under different but related circumstances.http://code-clarity.blogspot.com/2012/06/how-to-troubleshoot-ssltsl-or-x509.html. I enabled CAPI2 logging in the Event Viewer, and saw entries containing the following:

<RevocationInfo>
  <RevocationResult value="80092013">The revocation function was unable to check revocation because the revocation server was offline.</RevocationResult>
  </RevocationInfo>

 <TrustStatus>
  <ErrorStatus value="1000040" CERT_TRUST_REVOCATION_STATUS_UNKNOWN="true" CERT_TRUST_IS_OFFLINE_REVOCATION="true" />
  <InfoStatus value="102" CERT_TRUST_HAS_KEY_MATCH_ISSUER="true" CERT_TRUST_HAS_PREFERRED_ISSUER="true" />
  </TrustStatus>

<CertVerifyRevocation>
  <Certificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <Flags value="0" />
  <AdditionalParameters timeToUse="2015-08-12T07:58:21.234Z" currentTime="2015-08-12T07:58:21.234Z" urlRetrievalTimeout="PT15S" />
  <RevocationStatus index="0" error="80092013" reason="0" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="5" />
  <Result value="80092013">The revocation function was unable to check revocation because the revocation server was offline.</Result>
  </CertVerifyRevocation> 

To identify which URLs are not accessible, look for entries like these:

<CertRejectedRevocationInfo>
  <SubjectCertificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <URL scheme="http">http://ocsp.trustwave.com/MFEwTzBNMEswSTAJBgUrDgMCGgUABBQ1mI4Ww4R5LZiQ295pj4OF%2F44yyAQUyk7dWyc1Kdn27sPlU%2B%2BkwBmWHa8CEEssZkb0NHdbZ980oE3VBOY%3D</URL>
  <EarliestOnlineTime>2015-08-12T08:15:03.729Z</EarliestOnlineTime>
  <Action name="CanRetrieveFromNetwork" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="3" />
  </CertRejectedRevocationInfo> 

<CertRejectedRevocationInfo>
  <SubjectCertificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <URL scheme="http">http://crl.trustwave.com/STCA.crl</URL>
  <EarliestOnlineTime>2015-08-12T08:15:04.759Z</EarliestOnlineTime>
  <Action name="CanRetrieveFromNetwork" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="4" />
  </CertRejectedRevocationInfo> 

Make sure that the URL(s) in the log entries are accessible, this should sort out the issue.

Help! I've imported a certificate using the mmc snap-in and it is not associated with a private key!

Dont' fear, use the following command to fix the associating:

certutil -repairstore my "certificate serialnumber"

After this step refresh the view in mmc, a key should now be displayed as part of the certificate icon

SSL cert generation using OpenSSL

Create CA

Generate private key for CA

openssl genrsa -des3 -out keys/ca.key 1024

 Generate CA cert

openssl req -config openssl.conf -new -x509 -days 1001 -key keys/ca.key -out certs/ca.cer

 Generate server cert request

Generate private key for server cert

openssl genrsa -out keys/domain.com.key 2048

Generate server certificate request 

openssl req -new -key keys/servercert.key -out requests/certreq.txt

Sign certificate request and create certificate

openssl ca -policy policy_anything -config openssl.conf -cert certs/ca.cer -in requests/certreq.txt -keyfile keys/ca.key -days 360 -out certs/servercert.cer

Combine private key and certificate in pkcs12 format for importing into Windows

openssl pkcs12 -export -out servercert.p12 -in servercert.cer -inkey servercert.key

Credits:
 http://www.dylanbeattie.net/docs/openssl_iis_ssl_howto.html
http://www.rackspace.com/knowledge_center/article/generate-a-csr-with-openssl
http://www.cs.virginia.edu/~gsw2c/GridToolsDir/Documentation/ImportUserCertificate.htm

HTTP client, SSL and CRL check

We recently experienced a severe spike in latency of our one applications that uses HttpWebRequest to connect to a remote service over SSL under high load. After looking at it from a lot of angles, the thought of the certificate verification being blocked seemed like a possible cause, especially since the application is hosted in a locked-down DMZ and only the necessary minimum access granted to access the service URL. Once we granted access to be able to perform the OCSP and/or CRL checks, transactions started flowing immediately. I found this article at the time explaining the whole process. I also think a contributing factor was that only two active HTTP requests are allowed to a destination at once by default according to RFC 2616, my guess is that the CryptoAPI would adhere to this same limit, so all OCSP/CRL requests queued until the active ones timed out.