Displaying a remote SSL certificate details using CLI tools - Server Fault

Содержание

Convert from your local firefox installation (linux):
Curl command for ( ssl )
Curl with client certificate authentication
How to explicitly provide a ca certificate?
Php, curl – работа с сертификатами
The conversion script mk-ca-bundle:

Convert from your local firefox installation (linux):

You can also extract the CA certificate list from your local Firefox installation, if you have the certutil tool installed. You just need to run the firefox-db2pem.sh script (yes, this is a BASH script so you need to be Linux to get this to run).

Curl command for ( ssl )

if you’re using a self signed certificate on the server, you can use:

curl -k https://example.com:8443/cli/agentCLI -u username:password

but be aware that then it’s no better than using non SSL connection to the server, as your communication won’t be secure anymore, enabling all sorts of man in the middle attacks.

Though my advice to you is to download the .pem from the server:

using:

echo "HEAD / HTTP/1.0n Host: example.comnn EOTn" | openssl s_client -prexit -connect example.com:8443 > cert.pem

to your computer, keep only the part between BEGIN CERTIFICATE and END CERTIFICATE within the file (including the BEGIN/END lines) and give it as parameter to the --cacert option, you might also download it. Then you’ll get to authenticate your server each time you connect!

curl --cacert cert.pem https://example.com:8443/cli/agentCLI -u username:password

Testing on my own self-signed server, it’s working fine:

% openssl s_client -showcerts -connect example.com:443 </dev/null 2>/dev/null | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | grep -m1 -B-1 -- '-----END CERTIFICATE-----'  > cert.pem
% curl --cacert cert.pem https://example.com

for an example that should be working:

% openssl s_client -showcerts -connect git.cryptolib.org:443 </dev/null 2>/dev/null | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | grep -m1 -B-1 -- '-----END CERTIFICATE-----'  > cert.pem
% curl --cacert cert.pem https://git.cryptolib.org
curl: (51) SSL: certificate verification failed (result: 5)

but sadly it’s not.

I also tried to do, as suggested here:

% openssl x509 -inform PEM -in cert.pem -text -out certdata.pem
% curl --cacert certdata.pem https://git.cryptolib.org

Which is not working, because that site (git.cryptolib.org) I’m using for testing is not self-signed, but it’s from the CACert chain, which can be solved by using the CACert root certificates, following this FAQ.

a few resources to dig:

But no definitive answer so far :-s

Curl with client certificate authentication

We like to access a webserver using client certificate authentication instead of basic authentication.

Certificate is a PEM cert and the key file is a separate file.

The curl call looks like this:

curl -v --cert cert.crt --key key.key --pass foobar https://testserver/soap/request

From the debug log:

HTTP/1.1 401 Unauthorized
X-message-code: PWD_WRONG
WWW-Authenticate: Basic realm="Test Platform"
Transfer-Encoding: chunked
Date: Wed, 30 May 2021 10:26:51 GMT
Server: TEST
Set-Cookie: ...

I learned that WWW-Authenticate: Basic is a request from the server to the client to provide a Basic auth. However this may happen for an unauthenticated request, so it does not prove the server does not provide certificate authentication.

Reading https://medium.com/@sevcsik/authentication-using-https-client-certificates-3c9d270e8326
…as double-check I tried with a Firefox Browser by merging both files to a PKCS#12 format and importing that into the browser. It works, so the server supports certificate authentication.

What is wrong about the failing curl call?
Is there any test server supporting certificate auth? https://www.httpbin.org/ does not seem to provide that setup.

The same error occurs when using the PKCS#12 version with curl like this:

curl -v --cert cert.p12 --cert-type p12 --pass foobar
https://testserver/soap/request...

Complete debug log:

* Uses proxy env variable no_proxy == '10.0.0.0/8,127.0.0.1,172.16.0.0/12,192.168.20.0/24,'
* Uses proxy env variable https_proxy == 'http://gateway01:8080'
*   Trying 10.190.224.23...
* TCP_NODELAY set
* Connected to gateway01 (10.190.224.23) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to testserver:443
> CONNECT testserver:443 HTTP/1.1
> Host: testserver:443
> User-Agent: curl/7.60.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=DE; ST=Baden-Wurttemberg; L=...; O=...; OU=...; CN=...
*  start date: Aug 15 07:54:38 2021 GMT
*  expire date: Aug 15 07:54:38 2021 GMT
*  subjectAltName: host "testserver" matched cert's "*.testserver"
*  issuer: C=NL; L=Amsterdam; O=Verizon Enterprise Solutions; OU=Cybertrust; CN=Verizon Public SureServer CA G14-SHA2
*  SSL certificate verify ok.
> GET /soap/request HTTP/1.1
> Host: testserver
> User-Agent: curl/7.60.0
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< X-message-code: PWD_WRONG
< WWW-Authenticate: Basic realm="Test Platform"
< Transfer-Encoding: chunked
< Date: Wed, 06 Jun 2021 05:10:56 GMT
< Server: TEST
< Set-Cookie: ...; path=/; httponly; secure
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< 
* Connection #0 to host gateway01 left intact

As Firefox is able to authenticate, I analyzed both requests with Wireshark. I find the certificate data by filtering for “tcp contains ” and the known CN of my certificate in both network dumps, but Firefox and curl are placing the certs on different packets. Having a SOCKS5 proxy on localhost:8081 I capture loopback interface and filter on that tcp port 8081 which you can see in the dump:

Про сертификаты: Сертификат соответствия на волокно вискозное получение и регистрация в Центре сертификации продукции ФЦСМ ГОСТ СТАНДАРТ с 1999 года!

Firefox:

$ tshark -r firefox_request.pcapng
    1 0.000000000    127.0.0.1 → 127.0.0.1    TCP 74 51254 → 8081 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=1945282596 TSecr=0 WS=128
    2 0.000013339    127.0.0.1 → 127.0.0.1    TCP 74 8081 → 51254 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=1945282596 TSecr=1945282596 WS=128
    3 0.000025766    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=1945282597 TSecr=1945282596
    4 0.000052377    127.0.0.1 → 127.0.0.1    TCP 69 51254 → 8081 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=3 TSval=1945282597 TSecr=1945282596
    5 0.000058018    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51254 [ACK] Seq=1 Ack=4 Win=43776 Len=0 TSval=1945282597 TSecr=1945282597
    6 0.000110299    127.0.0.1 → 127.0.0.1    TCP 68 8081 → 51254 [PSH, ACK] Seq=1 Ack=4 Win=43776 Len=2 TSval=1945282597 TSecr=1945282597
    7 0.000127024    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=4 Ack=3 Win=43776 Len=0 TSval=1945282597 TSecr=1945282597
    8 0.000145980    127.0.0.1 → 127.0.0.1    TCP 112 51254 → 8081 [PSH, ACK] Seq=4 Ack=3 Win=43776 Len=46 TSval=1945282597 TSecr=1945282597 [TCP segment of a reassembled PDU]
    9 0.049958725    127.0.0.1 → 127.0.0.1    TCP 76 8081 → 51254 [PSH, ACK] Seq=3 Ack=50 Win=43776 Len=10 TSval=1945282646 TSecr=1945282597
   10 0.050239394    127.0.0.1 → 127.0.0.1    TCP 583 51254 → 8081 [PSH, ACK] Seq=50 Ack=13 Win=43776 Len=517 TSval=1945282647 TSecr=1945282646 [TCP segment of a reassembled PDU]
   11 0.100158230    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51254 [ACK] Seq=13 Ack=567 Win=44800 Len=0 TSval=1945282697 TSecr=1945282647
   12 0.107621822    127.0.0.1 → 127.0.0.1    TLSv1.2 2786 Server Hello
   13 0.107672270    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=567 Ack=2733 Win=174720 Len=0 TSval=1945282704 TSecr=1945282704
   14 0.107832838    127.0.0.1 → 127.0.0.1    TLSv1.2 1426 Certificate, Server Key Exchange
   15 0.108417953    127.0.0.1 → 127.0.0.1    TCP 1426 8081 → 51254 [PSH, ACK] Seq=4093 Ack=567 Win=44800 Len=1360 TSval=1945282705 TSecr=1945282704 [TCP segment of a reassembled PDU]
   16 0.108423407    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=567 Ack=5453 Win=436608 Len=0 TSval=1945282705 TSecr=1945282704
   17 0.109263897    127.0.0.1 → 127.0.0.1    TLSv1.2 2674 Certificate Request, Server Hello Done
   18 0.118830308    127.0.0.1 → 127.0.0.1    TCP 3613 51254 → 8081 [PSH, ACK] Seq=567 Ack=8061 Win=464896 Len=3547 TSval=1945282715 TSecr=1945282706 [TCP segment of a reassembled PDU]
   19 0.118851470    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51254 [ACK] Seq=8061 Ack=4114 Win=175744 Len=0 TSval=1945282715 TSecr=1945282715
   20 0.173415105    127.0.0.1 → 127.0.0.1    TLSv1.2 117 Change Cipher Spec, Client Hello[Malformed Packet]
   21 0.173932334    127.0.0.1 → 127.0.0.1    TCP 798 51254 → 8081 [PSH, ACK] Seq=4114 Ack=8112 Win=464896 Len=732 TSval=1945282770 TSecr=1945282770 [TCP segment of a reassembled PDU]
   22 0.173943923    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51254 [ACK] Seq=8112 Ack=4846 Win=182912 Len=0 TSval=1945282770 TSecr=1945282770
   23 0.250413758    127.0.0.1 → 127.0.0.1    TLSv1.2 844 Application Data
   24 0.300115138    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=4846 Ack=8890 Win=464896 Len=0 TSval=1945282897 TSecr=1945282847
   25 2.490166002    127.0.0.1 → 127.0.0.1    TCP 798 51254 → 8081 [PSH, ACK] Seq=4846 Ack=8890 Win=464896 Len=732 TSval=1945285087 TSecr=1945282847 [TCP segment of a reassembled PDU]
   26 2.490202168    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51254 [ACK] Seq=8890 Ack=5578 Win=184320 Len=0 TSval=1945285087 TSecr=1945285087
   27 2.554148656    127.0.0.1 → 127.0.0.1    TLSv1.2 844 Application Data
   28 2.554155780    127.0.0.1 → 127.0.0.1    TCP 66 51254 → 8081 [ACK] Seq=5578 Ack=9668 Win=464896 Len=0 TSval=1945285151 TSecr=1945285151
   29 12.610141935    127.0.0.1 → 127.0.0.1    TCP 66 [TCP Keep-Alive] 51254 → 8081 [ACK] Seq=5577 Ack=9668 Win=464896 Len=0 TSval=1945295207 TSecr=1945285151
   30 12.610162448    127.0.0.1 → 127.0.0.1    TCP 66 [TCP Keep-Alive ACK] 8081 → 51254 [ACK] Seq=9668 Ack=5578 Win=184320 Len=0 TSval=1945295207 TSecr=1945285151

Curl:

$ tshark -r curl_request.pcapng
    1 0.000000000    127.0.0.1 → 127.0.0.1    TCP 74 51196 → 8081 [SYN] Seq=0 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=1944382941 TSecr=0 WS=128
    2 0.000014517    127.0.0.1 → 127.0.0.1    TCP 74 8081 → 51196 [SYN, ACK] Seq=0 Ack=1 Win=43690 Len=0 MSS=65495 SACK_PERM=1 TSval=1944382941 TSecr=1944382941 WS=128
    3 0.000026917    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=1 Ack=1 Win=43776 Len=0 TSval=1944382941 TSecr=1944382941
    4 0.000068294    127.0.0.1 → 127.0.0.1    TCP 69 51196 → 8081 [PSH, ACK] Seq=1 Ack=1 Win=43776 Len=3 TSval=1944382941 TSecr=1944382941
    5 0.000076382    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=1 Ack=4 Win=43776 Len=0 TSval=1944382941 TSecr=1944382941
    6 0.000105767    127.0.0.1 → 127.0.0.1    TCP 68 8081 → 51196 [PSH, ACK] Seq=1 Ack=4 Win=43776 Len=2 TSval=1944382941 TSecr=1944382941
    7 0.000114276    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=4 Ack=3 Win=43776 Len=0 TSval=1944382941 TSecr=1944382941
    8 0.012753479    127.0.0.1 → 127.0.0.1    TCP 76 51196 → 8081 [PSH, ACK] Seq=4 Ack=3 Win=43776 Len=10 TSval=1944382954 TSecr=1944382941
    9 0.056073156    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=3 Ack=14 Win=43776 Len=0 TSval=1944382997 TSecr=1944382954
   10 0.060558849    127.0.0.1 → 127.0.0.1    TCP 76 8081 → 51196 [PSH, ACK] Seq=3 Ack=14 Win=43776 Len=10 TSval=1944383002 TSecr=1944382954
   11 0.106077184    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=14 Ack=13 Win=43776 Len=0 TSval=1944383047 TSecr=1944383002
   12 0.156827822    127.0.0.1 → 127.0.0.1    TLSv1 583 Client Hello
   13 0.156838865    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=13 Ack=531 Win=44800 Len=0 TSval=1944383098 TSecr=1944383098
   14 0.204829368    127.0.0.1 → 127.0.0.1    TLSv1.2 2786 Server Hello
   15 0.204837221    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=531 Ack=2733 Win=174720 Len=0 TSval=1944383146 TSecr=1944383146
   16 0.204942723    127.0.0.1 → 127.0.0.1    TLSv1.2 4146 Certificate, Server Key Exchange
   17 0.204950957    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=531 Ack=6813 Win=305664 Len=0 TSval=1944383146 TSecr=1944383146
   18 0.204981363    127.0.0.1 → 127.0.0.1    TLSv1.2 1314 Certificate Request, Server Hello Done
   19 0.204987399    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=531 Ack=8061 Win=436608 Len=0 TSval=1944383146 TSecr=1944383146
   20 0.208331980    127.0.0.1 → 127.0.0.1    TLSv1.2 2066 Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
   21 0.208341083    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=8061 Ack=2531 Win=175744 Len=0 TSval=1944383149 TSecr=1944383149
   22 0.255529146    127.0.0.1 → 127.0.0.1    TLSv1.2 117 Change Cipher Spec, Client Hello[Malformed Packet]
   23 0.255792021    127.0.0.1 → 127.0.0.1    TLSv1.2 220 Application Data
   24 0.255799639    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=8112 Ack=2685 Win=179840 Len=0 TSval=1944383197 TSecr=1944383197
   25 2.312629664    127.0.0.1 → 127.0.0.1    TLSv1.2 535 Application Data
   26 2.312809015    127.0.0.1 → 127.0.0.1    TLSv1.2 97 Encrypted Alert
   27 2.312821190    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=8581 Ack=2716 Win=179840 Len=0 TSval=1944385254 TSecr=1944385254
   28 2.313444679    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [FIN, ACK] Seq=2716 Ack=8581 Win=444800 Len=0 TSval=1944385255 TSecr=1944385254
   29 2.356079624    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [ACK] Seq=8581 Ack=2717 Win=179840 Len=0 TSval=1944385297 TSecr=1944385255
   30 2.357676736    127.0.0.1 → 127.0.0.1    TCP 66 8081 → 51196 [FIN, ACK] Seq=8581 Ack=2717 Win=179840 Len=0 TSval=1944385299 TSecr=1944385255
   31 2.357690888    127.0.0.1 → 127.0.0.1    TCP 66 51196 → 8081 [ACK] Seq=2717 Ack=8582 Win=444800 Len=0 TSval=1944385299 TSecr=1944385299

Filtering for the special CN of my local certificate I can find this in both dumps:

$ for file in {firefox,curl}_request.pcapng;do tshark -r "$file" -Y "tcp contains FOOBAR";done
   18 0.118830308    127.0.0.1 → 127.0.0.1    TCP 3613 51254 → 8081 [PSH, ACK] Seq=567 Ack=8061 Win=464896 Len=3547 TSval=1945282715 TSecr=1945282706 [TCP segment of a reassembled PDU]
   20 0.208331980    127.0.0.1 → 127.0.0.1    TLSv1.2 2066 Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message

However it is detected as part of the TLS handshake only for curl:

$ for file in {firefox,curl}_request.pcapng;do tshark -r "$file" -Y "ssl contains FOOBAR";done
   20 0.208331980    127.0.0.1 → 127.0.0.1    TLSv1.2 2066 Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message

What is different in the authentication of Firefox (successful) and Curl (failing)?

Про сертификаты: Опубликованы официальные разъяснения Минздрава к приказу об аккредитации

How to explicitly provide a ca certificate?

In some cases, you may need to use a different certificate chain than the one supplied with Curl. Certificate chains provide trust relationships between the certificates, where the CA certificate is at the beginning of the chain and the certificate of the site we want to navigate at the end of the chain. With the –cacert filename command line parameter, we can provide another CA, such as our company’s local CA.

Php, curl – работа с сертификатами

В процессе интеграции веб приложений с другими приложениями, зачастую, может возникнуть задача:

Идентифицировать удаленный сервер, используя сертификат .cer.
Идентифицировать себя для удаленного сервера, используя сертификат .pfx
Передать данные, используя curl

Для решения этой задачи средствами php – curl, нам требуется иметь сертификаты в формате: .crt и .pem

Воспользуемся программой openssl.
После скачивания и установки, переконвертируем сначала клиентский сертификат

V:sslbinopenssl.exe pkcs12 -in V:sslcert.pfx -out V:sslcert.key
V:sslbinopenssl.exe rsa -in V:sslcert.key -out V:sslcert.pem
V:sslbinopenssl.exe x509 -in V:sslcert.key >> V:sslcert.pem

V:ssl – путь где установлена программа openssl и там же лежат наши сертификаты.

После второй команды наc попросят указать наш текущий пароль и задать новый. Новый пароль, мы будем использовать далее.

А теперь серверный:

V:sslbinopenssl.exe x509 -in V:sslcert.cer -inform DER -out V:sslcert.crt -outform PEM

Все готово. Теперь, используя CURL соединение, подключимся к удаленному серверу.
Например:

curl_setopt($ch, CURLOPT_URL, ‘https://xxx.xxx.xxx.xxx/’);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

В настройках CURL-соединения следует указать:

curl_setopt($ch, CURLOPT_VERBOSE, ‘1’);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, ‘1’);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_CAINFO, ‘путь к .crt файлу’);
curl_setopt($ch, CURLOPT_SSLCERT, ‘путь к .pem файлу’);
curl_setopt($ch, CURLOPT_SSLCERTPASSWD, ‘новый пароль’);

Про сертификаты: Лабораторная диагностика, повышение квалификации для среднего медицинского персонала в АМО

The conversion script mk-ca-bundle:

The mk-ca-bundle tool converts Mozilla’s cert bundle to PEM format, suitable for (lib)curl and others. Writtten by Guenter Knauf.

Displaying a remote SSL certificate details using CLI tools – Server Fault