Scoring

Currently, CryptCheck gives note from G for the worst sites to A+ for the best ones.

Scoring is based on the fact that TLS handshake is not authenticated, and so an attacker can force to use whatever cipher he wants as soon as both client and server support it, with a downgrade attack as simple as modify TCP packets on the fly.

Such downgrade attack doesn't require heavy resources and can be made with standard computer or phone.
The only difficult part is to be in position to modify the traffic between the client and the server. This is the case if the attacker is connected on the same network as the client (hotspot, 3G…) with simple ARP spoofing, doable with tools like Droid Sheep.

As client support can't be guessed, CryptCheck considers the weakest suite supported server side. This way, a connection to the scored service can't lead to a negociated handshake with a worse score than the one given to the service, whatever your client supports and whatever an attacker is present or not.

Score Protection Weakness
Hard Medium Easy Future Weak Deprecated
A+ Seriously take security into account and invest a lot on it.
Whatever the cost, encryption safety is implemented. You can be proud!
A Seriously take security into account and invest a lot on it.
B+ Seriously take security into account and invest on it.
B Take security into account and invest on it.
C+ Take security into account and invest a little on it.
C Take security into account but don't spend too much for it.
D Take security into account. Minimaly.
This is the worst score a decent service must have today.
E Take security into account. A little. Or not.
F Just don't take security into account.
G Just don't take security into account at all.
What the fuck you do, dude?
0 No security at all. Just plain text.
Seriously, in 2018?
V Invalid certificate (wrong domain, expired…)
T Unstrusted certificate. Not issued by a trusted certificate authority.
X Error occurs during the analysis. Try again later?
For protection:
Fully implemented Partially implemented
Easy: simple to implement, small protection
Medium: quiet hard to implement, middle protection
Hard: hard to implement, strong protection
For weakness:
Not vulnerable Vulnerable
Future: known weakness, but no practical attack known
Weak: known weakness, pratical attack exist
Deprecated: known weakness, merely equivalent or equal to plain text

Note: Unlike HTTPS or XMPP, SMTP uses opportunistic encryption.
When you send an email, the server used to forward the mail (the MTA to the recipient has no way to guess in advance if recipient MTA supports or not encryption and which cipher suite will be available. To avoid your email returning to you in case of failure, the standard for email encryption (RFC 3207) requires to retry in plain text in case of encryption handshake failure.
So, for SMTP, there is a compromise to make between strong configuration, leading to plain text fallback for old or badly configured MTA, and compatibility with such MTA to use weak encryption better than plain text but allowing downgrade attack on stronger MTA.
Given email is a real nightmare for security, with multiple way to force a connection to fallback to plain text (STARTTLS stripping, MX lying…), CryptCheck scores SMTP as HTTPS or XMPP and doesn't care about compatibility trouble. This way, weak people are still weak, but strong people can (not too much) hope strong encryption under normal condition.
Be advice than strong score here for SMTP means compatibility troubles. Or fucked service which doesn't take care of your security. I don't know, you turn to judge.

Checks

Protection

Hard protection

Such protection are hard or risky to deploy but give strong protection against attack.
Deploying such protection can just break your service for a long time if you don't understand what you are doing.
Service using such protection really tries to protect you and really know what security is.


A HTTP Public Key Pinning (HPKP) HTTPS only, incoming feature

HPKP protection, specified in RFC 7469, consists of putting headers on HTTP response to specify which keys or certificats are allowe for the encryption. If pinning mismatches, for example because of a man-in-the-middle attack connection is rejected and no data at all is transfered.

	
$ curl -sI https://cryptcheck.fr/ | grep public-key-pins
public-key-pins: report-uri="https://aeris.report-uri.io/r/default/hpkp/enforce"; max-age=5184000;
		pin-sha256="wdkD38iQQzxE7g0RpmN8VoaIqX7YmPWwoueD9Iqawfg="; pin-sha256="EswdUzfH2N8sx6Nb4Vr9gamtNF5VWQxLWUG0gDIPVLw=";
	

HPKP is difficult to deploy because has a redemption period of some days (maximum allowed: 60). During this time, in case of misconfiguration, returning visitors will faced a TLS error page, even if the configuration was fixed.


Medium protection

Such protection is not so easy to deploy and can have hazardous side effects, but provides quiet good protection against some attacks.
Broken service is unexpected or could be fixed in a small time range.
Using such protection is a clear sign the service try to protect you.


A B HTTP Strict Transport Security (HSTS) HTTPS only

HSTS protection, specified in RFC 6797, consists of putting headers on HTTP response to specify the service supports HTTPS.
After the first connection (HSTS is "Trust On First Use" (TOFU)), the browser automatically rewrite http:// address to https://, avoiding a plain request (with potential data leak) to be asked by the service to redirect to the https:// address.

	
curl -sI https://cryptcheck.fr/ | grep strict-transport-security
strict-transport-security: max-age=31536000; includeSubDomains; preload;
	

To have full score on HSTS, you need to have a long max-age period, at least 1 year (31536000 seconds).
If you correctly configure your service with HSTS, you can also ask for browser preloading, avoiding the trouble of the first connection.


Easy protection

Such protection is easy to deploy and without .
Broken service is unexpected or could be fixed in a small time range.
Using such protection is a clear sign the service try to protect you.


C Authentificated Encryption with Authenticated Data (AEAD)

Since 2014, TLS (and SSL) suffers of PODDLE vulnerability on the way padding is done during TLS handshake. An attacker can play with this encrypted padding to guess underlying plain data.
Any cipher mode operation other than AEAD is vulnerable to this attack.

In practice, POODLE is a serious flaw for SSLv2/v3, which must be avoided in all cases, but also for TLSv1.0/1.1.
Service operators must support AEAD cipher suite as soon as possible, to avoid trouble when practical attack will be found on POODLE on TLS.
Such cipher suite is only available on TLSv1.2, so operators must disable TLSv1.0 now, and TLSv1.1 as soon as possible.


C TLS Fallback Signaling Cipher Suite Value (SCSV)

SCSV, specify in RFC 7507 is a TLS extension to allow a client to signal to the server a previous hanshake attempt with higher TLS version was done, but unsuccessfully.
This way, the server can detect a downgrade attack on the line, because supporting better than the current TLS version.
Without this signaling value, the server has no way to distinguish between a client supporting TLSv1.2 but downgraded to TLSv1.1 and a client TLSv1.1 only.
For example, this feature allows blocking of downgrade attack from TLSv1.2 (AEAD & PFS) to TLSv1.0 (nor AEAD nor PFS) to exploit POODLE vulnerability more easily.

To activate SCSV, you just need a decent OpenSSL version (1.0.1j+). LibreSSL currently doesn't have support for this.


Weaknesses

Future weakness

This kind of weakness is theorical vulnerability but without practical attack or with too much side effects to be able to patch it.


Current weakness

Such weakness knows practical attacks to break encryption. Using such features is hightly discourage, and operators must take quick actions to remove them.


F TLSv1.0

TLSv1.0 is vulnerable to


Deprecated feature


G SSLv2, SSLv3

SSLv2 and SSLv3 are deprecated SSL protocol version.
Pratical attacks exist to decrypt SSL encrypted traffic to plain text in some minutes with standard computer. For SSLv3, it's POODLE again. For SSLv2, it's was supposed to never be in production because too bad and broken cryptography under the hood. DROWN vulnerability allows an attacker to decrypt encrypted traffic (even TLSv1.2!) as soon as one of the servers used for the service supports SSLv2 with the same key.


E G SHA-1 incoming feature for HMAC

SHA-1 is a cryptographic hash function used in TLS cipher suite. It was broken in 2016.

SHA-1 is used in two parts of the handshake.
For HMAC, which protect each messages exchanged during handshake. Because lifetime of such HMAC is very short (TCP/IP round trip), SHA-1 collision is not a real trouble on this part.
For key exchange and authentication. Each certificate is signed by the issuer certificate using a digest. In this case, if SHA-1 digest is used and because certificate lifetime is long (years to decades), collision on digest could allow an attacker to forge a rogue certificate which match the real certificate digest, and so to impersonate the TLS service behind.

SHA-1 signed certificates must be banned.
SHA-1 HMAC is currently quite safe, but operators must take action to ensure SHA-2 compatibility with clients in case if SHA-1 must be revoked even for HMAC.


G MD-5, MD-4, MD-2, MDC-2

MD-5, MD-4, MD-2 and MDC-2 are completely broken hash function. Just don't use it.

G Compression incoming feature

With TLS compression activated, some oracle attacks allow to decrypt the content. For example the BREACH or CRIME attacks.

TLS compression must be disabled on the service.