I’ve been using certificate transparency with increasing frequency during my network pentests. What a great source of information! I’ve found it so useful that I wrote a short standalone script to search for domains in a transparency log and resolve them to IP addresses.

What’s certificate transparency?

There’s an actual site dedicated to describing Certificate Transparency (https://www.certificate-transparency.org/), which I recommend you check out. In a nutshell, CT is a mechanism that provides real-time monitoring and auditing of certificate information. If you’ve ever clicked on that little padlock next to the URL of a site you’ve visited in your browser, chances are that you’ve used CT.

There are several ways to use CT: inside your browser as described above, and by querying a CT log using a web interface.

Google maintains a site for querying domains at https://www.google.com/transparencyreport/https/ct/, while Comodo hosts a lookup tool at https://crt.sh/. I was particularly interested in crt.sh, because you can run queries with wildcards and retrieve results in RSS format.

Why should I use Certificate Transparency during my recon activities?

Defensive countermeasures like Intrusion Prevention Systems and tar pits have become commonplace. Crazy notion, right: it’s like they don’t want us to get in or something! In all seriousness, though, these are good defense mechanisms that make it more difficult for unskilled attackers to find vulnerable services.

However, these defenses are not a panacea: with good recon, one can find a whole lot of information about a target online – and without triggering any alerts on the target’s side.

Scouring Certificate Transparency logs is a great way of teasing out host names. Furthermore, because this is a security mechanism for auditing web certificates, you know there’s a really good chance that any host information you do find is likely to lead to a web application – so you know which ports you’re likely to find open.

One thing that I’ve noticed during my pentests is that some of these certificates are clearly meant for internal use only. This is an awkward (and known) drawback of using a Certificate Authority that uses CT: if you generate certificates for internal services, they will be visible in the CT logs. They may not be accessible from the outside, but it doesn’t change the fact that you’re exposing internal information unnecessarily. It makes things easier for attackers to plan out an incursion.

Manual Certificate Transparency recon

The first thing you can try during your recon is to check out the google and comodo sites manually. Here’s an example using google.com:

Notice how I hit the “Include subdomains” checkbox? That’s what returns the certificates for all the hosts, not just google.com. Here’s a sample result:

Instead of using a checkbox, the Comodo site uses wildcards. Careful, though! With a company the size of google, you may get timeouts if you try to retrieve all of the logs.

Automating the process

Using bash

If your target domain is small, you can get away with running your lookups manually, transcribing them to a text file, and then resolving each host name to an IP address via a bash script. Something like this will typically work:

for curentry in $(cat hosts.txt) do dig $curentry +short

Using recon-ng

There is also a module in the very excellent recon-ng framework (https://bitbucket.org/LaNMaSteR53/recon-ng) that performs certificate transparency searches, which was committed in August 2016. To use it, fire up recon-ng and use the Certificate Transparency module:

recon/domains-hosts/certificate_transparency

You’ll need some domains:

add domain testdomain.local

Finally, run your request:

run

Note that the version I tested (4.8.3) didn’t seem to work. Still trying to figure out why before I file a bug report, because chances are that the issue’s on my side ^.^

With a nifty little python script

I wrote a python script to automate my CT log searches, which I called ct_scan. I’ve uploaded it to my pentest script repository (https://github.com/Inf0Junki3/pentesty_goodness/tree/master/ct_scan):

The script basically runs a query on crt.sh, requesting a response in RSS format. It parses the results, filters out wildcards, and spits them out to stdout (which makes it easy to pipe to other scripts, or to save to file). Because I got sick of piping things over to dig, I integrated DNS lookups. It saves me a bit of typing and makes it really easy to integrate my results into pentest reports.

Final words

So there you go: Certificate Transparency as a means to do evil instead of good. Always intersting to see how even a mechanism that has been designed to make things safer can be used to cause insecurity.

If you notice anything incorrect in this article or the code, please let me know!