Client certificate validation issues - WCF

Posted on Wednesday, August 12, 2015 by Nicki de Wet

We encountered an issue using client certificate authentication with a client certificate issued by a CA in the Trusted Root CA store.

The client certificate chain is valid when opening it on the server, showing the whole chain. When attaching it to a request the request would be rejected with response code 403 - Forbidden. Looking at the System eventlog for SChannel errors showed nothing.

Eventually we switched on WCF tracing, and found the following in the trace:

<TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Warning">
    <TraceIdentifier>http://msdn.microsoft.com/en-ZA/library/System.ServiceModel.Channels.HttpsClientCertificateInvalid.aspx</TraceIdentifier>
    <Description>Client certificate is invalid with native error code 0x2013 (see http://go.microsoft.com/fwlink/?LinkId=187517 for details).</Description>
    <AppDomain>MyApp.exe</AppDomain>
    <Source>System.ServiceModel.Channels.HttpsChannelListener`1[System.ServiceModel.Channels.IReplyChannel]/30640645</Source>
    <ExtendedData xmlns="http://schemas.microsoft.com/2006/08/ServiceModel/HttpRequestTraceRecord">
        <Headers>
            <SOAPAction>"Authorize"</SOAPAction>
            <Connection>Keep-Alive</Connection>
            <Content-Length>1281</Content-Length>
            <Content-Type>text/xml;charset=UTF-8</Content-Type>
            <Accept-Encoding>gzip,deflate</Accept-Encoding>
            <Host>MyAppURL</Host>
            <User-Agent>Apache-HttpClient/4.1.1 (java 1.5)</User-Agent>
        </Headers>
        <Url>https://MyAppURL</Url>
    </ExtendedData>
</TraceRecord>
Searching on Google brought us no closer, eventually I found an old link I've used before under different but related circumstances.http://code-clarity.blogspot.com/2012/06/how-to-troubleshoot-ssltsl-or-x509.html. I enabled CAPI2 logging in the Event Viewer, and saw entries containing the following:

<RevocationInfo>
  <RevocationResult value="80092013">The revocation function was unable to check revocation because the revocation server was offline.</RevocationResult>
  </RevocationInfo>
 <TrustStatus>
  <ErrorStatus value="1000040" CERT_TRUST_REVOCATION_STATUS_UNKNOWN="true" CERT_TRUST_IS_OFFLINE_REVOCATION="true" />
  <InfoStatus value="102" CERT_TRUST_HAS_KEY_MATCH_ISSUER="true" CERT_TRUST_HAS_PREFERRED_ISSUER="true" />
  </TrustStatus>
<CertVerifyRevocation>
  <Certificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <Flags value="0" />
  <AdditionalParameters timeToUse="2015-08-12T07:58:21.234Z" currentTime="2015-08-12T07:58:21.234Z" urlRetrievalTimeout="PT15S" />
  <RevocationStatus index="0" error="80092013" reason="0" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="5" />
  <Result value="80092013">The revocation function was unable to check revocation because the revocation server was offline.</Result>
  </CertVerifyRevocation> 

To identify which URLs are not accessible, look for entries like these:
<CertRejectedRevocationInfo>
  <SubjectCertificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <URL scheme="http">http://ocsp.trustwave.com/MFEwTzBNMEswSTAJBgUrDgMCGgUABBQ1mI4Ww4R5LZiQ295pj4OF%2F44yyAQUyk7dWyc1Kdn27sPlU%2B%2BkwBmWHa8CEEssZkb0NHdbZ980oE3VBOY%3D</URL>
  <EarliestOnlineTime>2015-08-12T08:15:03.729Z</EarliestOnlineTime>
  <Action name="CanRetrieveFromNetwork" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="3" />
  </CertRejectedRevocationInfo> 
<CertRejectedRevocationInfo>
  <SubjectCertificate fileRef="C2CBFE64932BBB842F04874445416E34164E3FE5.cer" subjectName="Trustwave Organization Validation SHA256 CA, Level 1" />
  <IssuerCertificate fileRef="8782C6C304353BCFD29692D2593E7D44D934FF11.cer" subjectName="SecureTrust CA" />
  <URL scheme="http">http://crl.trustwave.com/STCA.crl</URL>
  <EarliestOnlineTime>2015-08-12T08:15:04.759Z</EarliestOnlineTime>
  <Action name="CanRetrieveFromNetwork" />
  <EventAuxInfo ProcessName="lsass.exe" />
  <CorrelationAuxInfo TaskId="{612311D8-4441-4C4B-91B1-0A049B93BFD7}" SeqNumber="4" />
  </CertRejectedRevocationInfo> 
Make sure that the URL(s) in the log entries are accessible, this should sort out the issue.

Help! I've imported a certificate using the mmc snap-in and it is not associated with a private key!

Posted on Thursday, June 4, 2015 by Nicki de Wet

Dont' fear, use the following command to fix the associating:

certutil -repairstore my "certificate serialnumber"

After this step refresh the view in mmc, a key should now be displayed as part of the certificate icon

SSL cert generation using OpenSSL

Posted on Thursday, September 18, 2014 by Nicki de Wet

Create CA

Generate private key for CA

openssl genrsa -des3 -out keys/ca.key 1024

 Generate CA cert

openssl req -config openssl.conf -new -x509 -days 1001 -key keys/ca.key -out certs/ca.cer

 Generate server cert request

Generate private key for server cert

openssl genrsa -out keys/domain.com.key 2048

Generate server certificate request 

openssl req -new -key keys/servercert.key -out requests/certreq.txt

Sign certificate request and create certificate

openssl ca -policy policy_anything -config openssl.conf -cert certs/ca.cer -in requests/certreq.txt -keyfile keys/ca.key -days 360 -out certs/servercert.cer

Combine private key and certificate in pkcs12 format for importing into Windows

openssl pkcs12 -export -out servercert.p12 -in servercert.cer -inkey servercert.key

Credits:
 http://www.dylanbeattie.net/docs/openssl_iis_ssl_howto.html
http://www.rackspace.com/knowledge_center/article/generate-a-csr-with-openssl
http://www.cs.virginia.edu/~gsw2c/GridToolsDir/Documentation/ImportUserCertificate.htm

Excel trick for creating Gantt chart

Posted on Tuesday, September 16, 2014 by Nicki de Wet

This is quite a useful trick to create a Gantt chart to chart latency of individual requests to some service over time.

http://www.brighthubpm.com/templates-forms/3418-using-excel-to-create-a-gantt-chart

Error installing Windows 8.1 Update for x64-based Systems (KB2919355) - error 80070005

Posted on Saturday, April 26, 2014 by Nicki de Wet

My computer is not able to install this latest update from Microsoft. It has already been upgraded to 8.1 without any issues.

I ran ProcessMonitor and found this:

Date & Time: 2014-04-26 05:30:50 PM
Event Class: File System
Operation: SetLinkInformationFile
Result: ACCESS DENIED
Path: C:\Windows\WinSxS\amd64_microsoft-windows-b..nager-efi.resources_31bf3856ad364e35_6.3.9600.16384_en-gb_680d99c9c1bb411e\bootmgr.efi.mui
TID: 3056
Duration: 0.0000421
ReplaceIfExists: True
FileName: \SystemRoot\WinSxS\amd64_microsoft-windows-b..nager-efi.resources_31bf3856ad364e35_6.3.9600.17031_en-gb_68408c0dc1958b90\bootmgr.efi.mui

The process details:
Description: Windows Modules Installer Worker
Company: Microsoft Corporation
Name: TiWorker.exe
Version: 6.3.9600.17031 (winblue_gdr.140221-1952)
Path: C:\WINDOWS\winsxs\amd64_microsoft-windows-servicingstack_31bf3856ad364e35_6.3.9600.17031_none_fa50b3979b1bcb4a\TiWorker.exe
Command Line: C:\WINDOWS\winsxs\amd64_microsoft-windows-servicingstack_31bf3856ad364e35_6.3.9600.17031_none_fa50b3979b1bcb4a\TiWorker.exe -Embedding
PID: 3316
Parent PID: 672
Session ID: 0
User: NT AUTHORITY\SYSTEM
Auth ID: 00000000:000003e7
Architecture: 64-bit
Virtualized: False
Integrity: System
Started: 2014-04-26 05:21:13 PM
Ended: (Running)
Modules:

 Having a look at the permissions on the bootmgr.efi.mui files in both referenced locations shows that SYSTEM can only read and execute these files, which is most likely why the update fails.

My machine originally had a local account, and when I installed the 8.1 upgrade it changed my local account to a live account against my wishes. This should not have anything to do with it, as Windows Update does not run as me, right?

Is it safe to change the permissions so that SYSTEM has write access as well?

EDIT: After all this, the solution found here is quite simple: just reboot and only open Windows Update and install the update. It seems some files might have been held by other processes.

SSL connection issues when using client certificate authentication

Posted on Friday, April 4, 2014 by Nicki de Wet

The SSL subsystem in Windows has a limitation as to the maximum size of the Trusted Issuers list the server sends to the client during the client certificate authentication process. This causes the list to be truncated, and if the issuer of your client certificate is not listed, causes the authentication to fail with the following message

HTTP/1.1 403 Forbidden
Content-Length: 0
Server: Microsoft-HTTPAPI/2.0
Date: Fri, 04 Apr 2014 10:00:13 GMT
This will be accompanied by an entry with Event ID 36885 in the Windows System Eventlog
When asking for client authentication, this server sends a list of trusted certificate authorities to the client. The client uses this list to choose a client certificate that is trusted by the server. Currently, this server trusts so many certificate authorities that the list has grown too long. This list has thus been truncated. The administrator of this machine should review the certificate authorities trusted for client authentication and remove those that do not really need to be trusted.

There are two approaches to fix this:

  • remove certificates from the list of Trusted Issuers (not always easy or safe, you might delete certificates required by the operating system or other installed software), or
  • create a registry setting that will cause the SSL subsystem to no longer send the list of Trusted Issuers to the client
I personally prefer the 2nd option, as there is less chance of disaster IMHO.

HTTP client, SSL and CRL check

Posted on Tuesday, November 26, 2013 by Nicki de Wet

We recently experienced a severe spike in latency of our one applications that uses HttpWebRequest to connect to a remote service over SSL under high load. After looking at it from a lot of angles, the thought of the certificate verification being blocked seemed like a possible cause, especially since the application is hosted in a locked-down DMZ and only the necessary minimum access granted to access the service URL. Once we granted access to be able to perform the OCSP and/or CRL checks, transactions started flowing immediately. I found this article at the time explaining the whole process. I also think a contributing factor was that only two active HTTP requests are allowed to a destination at once by default according to RFC 2616, my guess is that the CryptoAPI would adhere to this same limit, so all OCSP/CRL requests queued until the active ones timed out.