Cracking Passwords with Michael McIntyre

Posted on 2020-10-14 by Billy Cody in Tools of the Trade


I was watching the comedian Michael McIntyre’s most recent Netflix special “Showman” when he began a segment on the evolution of the online password. He described an algorithm that would’ve cracked most of my pre-teen online passwords. I decided to dig further and see how effective this algorithm is against some real world data.

Is Michael McIntyre really a master hacker?

Is he watching me right now?

How do I protect myself from him?

No. No. Read on!

The Comedy Special

The quotes I use throughout this article are taken from the Netflix special “Michael McIntyre: Showman”. Timestamps are included below if you’d like to check it out.

  • The Special Word: 41:06-41:20
  • The Capital Letter: 41:20-42:00
  • The Number: 42:00-42:34
  • The Special Character: 42:34-43:36
  • The Michael McIntyre Special: 43:36-43:53

McIntyre eerily describes a thought process many of us had when we first came online. He examines the history (at a high level) of how password security has evolved since we first started using the Internet. Let’s take a look at his breakdown.

The Special Word

“Can I have your password?”

“Yes, you can, that is my special word.”

The word we decided early on would be the basis of our password. We used it for everything, from banking, shopping, to just general browsing.

Password. Secret. Apple. Spring. Friday. September. Did I guess anyone’s special word?

So what’s wrong with these words? Flip open a dictionary and you’ll find it in there. Our entire online history just waiting to be unraveled by a book even a child has access to.

Cracking

And then, companies started getting quite rude. You would put your password in, and it would go, “Weak.” Who are you to judge my special word?

Let’s take a look at how easy it is to retrieve a password using just a dictionary. Below is an example of what I’d be running if I was able to retrieve a password hash – say through a compromised Active Directory environment. I’m gonna try and crack it using a simple English dictionary available online.

Below are two examples using hashcat and john, two open source security tools I commonly use for password cracking.

# Our NT hash to crack
$ cat your-password-hash.txt
Michael:1000:aad3b435b51404eeaad3b435b51404ee:e2e8dcbc9f11072f9a29d6b5c1c83302:::

# Our wordlist... over 466,000 words!
$ wc -l words.txt
466550 words.txt

# Examples of password cracking tools using a plain dictionary
$ hashcat -a 0 -m 1000 your-password-hash.txt words.txt
hashcat (v6.1.1) starting...

...snip...

Dictionary cache built:
* Filename..: words.txt
* Passwords.: 466550
* Bytes.....: 4862992
* Keyspace..: 466550
* Runtime...: 0 secs

e2e8dcbc9f11072f9a29d6b5c1c83302:apples

Session..........: hashcat
Status...........: Cracked
Hash.Name........: NTLM
Hash.Target......: e2e8dcbc9f11072f9a29d6b5c1c83302
Time.Started.....: Thu Sep 24 10:40:29 2020 (0 secs)
Time.Estimated...: Thu Sep 24 10:40:29 2020 (0 secs)
Guess.Base.......: File (words.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:   401.5 kH/s (0.18ms) @ Accel:1024 Loops:1 Thr:1 Vec:8
Recovered........: 1/1 (100.00%) Digests
Progress.........: 24576/466550 (5.27%)
Rejected.........: 0/24576 (0.00%)
Restore.Point....: 20480/466550 (4.39%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidates.#1....: apesthetize -> arrgt

Started: Thu Sep 24 10:40:27 2020
Stopped: Thu Sep 24 10:40:30 2020

$ john --format=NT your-password-hash.txt --wordlist=words.txt
Using default input encoding: UTF-8
Loaded 1 password hash (NT [MD4 256/256 AVX2 8x3])
Warning: no OpenMP support for this hash type, consider --fork=4
Press 'q' or Ctrl-C to abort, almost any other key for status
apples           (Michael)
1g 0:00:00:00 DONE (2020-09-24 10:37) 100.0g/s 2169Kp/s 2169Kc/s 2169KC/s appertise..appositive
Use the "--show --format=NT" options to display all of the cracked passwords reliably
Session completed

This hash was cracked almost instantly in both cases. But how likely is it that someone would actually use “apples” as their password? Let’s look at some real-world data.

Data Analysis

I’m going to be using two main sources for my analysis. The first is an infamous data breach of a company called “Rock You”, which revealed over 14 million unique user passwords. This wordlist is commonly used as a password cracking base.

The second is FlameOfIgnis’ Pwdb-Public, which contains curated wordlists with the most common passwords from over 6000 data breaches containing user credentials. Let’s take a look at the top 20 most common passwords that have been found with his analysis.

Password Count
123456 5,365,167
123456789 1,962,603
password 1,155,715
qwerty 869,933
12345678 702,094
12345 678,025
123123 455,957
111111 447,887
1234 442,902
1234567890 397,290
1234567 390,755
abc123 294,245
1q2w3e4r5t 276,112
q1w2e3r4t5y6 270,438
iloveyou 261,525
123 258,718
000000 250,001
123321 211,418
1q2w3e4r 204,915

Could you classify these as special words? I would. They are all simple patterns and words that would be easy for someone to remember. Coincidentally, these are easy for someone to compromise.

I’ll cover this in more depth in a later section, but here’s some stats that should be concerning.

Dictionary No. Words
Regular English Dictionary (words_alpha.txt) 370,103
Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 63,578 0.44%
ignis-1K.txt 1,000 497 49.70%
ignis-10K.txt 10,000 3,570 35.70%
ignis-100K.txt 100,000 14,025 14.03%
ignis-1M.txt 1,000,000 41,408 4.14%
ignis-10M.txt 10,000,000 79,843 0.79%

Passwords using only a special word comprised almost half of the top 1000 passwords. As we increase the size of our wordlist, we can see our success rate drop. As expected, rockyou performed the worst as it is not a curated list.

Based on almost no scientific knowledge whatsoever, I’d say people are actually following McIntyre’s formula. Let’s take a look at the other components.

The Capital Letter

“Sorry, but the internet has become very popular, we need to strengthen your password. We must have from you a capital letter. I’m sorry, we will not be accepting passwords anymore unless it contains at least one capital letter.”

And we all momentarily considered our options… before deciding to capitalise the first letter of our password.

Flash forward a few years later. How frustrating. My bank now insists on capitalising a letter of my passwords. Why would they force me to remember yet another password? As Michael suggested, let’s now capitalise the first letter.

Let’s take a look at how this affects our special word. I’ve now capitalised every word in my English dictionary and will use it alongside the one we previously used.

Dictionary No. Words
Capitalised Dictionary 740,206
Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 75,425 0.53%
ignis-1K.txt 1,000 500 50.00%
ignis-10K.txt 10,000 3,668 36.68%
ignis-100K.txt 100,000 15,341 15.34%
ignis-1M.txt 1,000,000 49,627 4.96%
ignis-10M.txt 10,000,000 104,390 1.04%

Look at that! We managed to get over 12,000 more matches in rockyou (although only 0.53% total) by simply capitalising the first letter! In the top 1000, we managed to crack another 3, getting us to exactly 50% of the top 1000 most common passwords.

This is all well and good for searching through plaintext files, but let’s see how I’d do it with our friends hashcat and john. I’ve changed my hash to include a capital letter at the start. I’ll need to use some rules to automatically mangle my dictionary to include a capital letter at the start.

# Updated hash file
$ cat your-password-hash-capitalised.txt
Michael:1000:aad3b435b51404eeaad3b435b51404ee:D2C457F00FF74FB54EB71033E0E6E37A:::

# Rule files to capitalise the first letter
$ cat hashcat-rules.txt
:
c
$ cat john-rules.txt
[List.Rules:Capitals]
c
$ cat john-rules.txt >> /etc/john/john.conf

# Crack using new rules
$ hashcat -a 0 -m 1000 your-password-hash-capitalised.txt words.txt -r hashcat-rules.txt
hashcat (v6.1.1) starting...

...snip...

Dictionary cache hit:
* Filename..: words.txt
* Passwords.: 466550
* Bytes.....: 4862992
* Keyspace..: 933100

d2c457f00ff74fb54eb71033e0e6e37a:Apples

Session..........: hashcat
Status...........: Cracked
Hash.Name........: NTLM
Hash.Target......: d2c457f00ff74fb54eb71033e0e6e37a
Time.Started.....: Thu Sep 24 13:55:56 2020 (0 secs)
Time.Estimated...: Thu Sep 24 13:55:56 2020 (0 secs)
Guess.Base.......: File (words.txt)
Guess.Mod........: Rules (hashcat-rules.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:  5189.2 kH/s (0.39ms) @ Accel:1024 Loops:2 Thr:1 Vec:8
Recovered........: 1/1 (100.00%) Digests
Progress.........: 49152/933100 (5.27%)
Rejected.........: 0/49152 (0.00%)
Restore.Point....: 20480/466550 (4.39%)
Restore.Sub.#1...: Salt:0 Amplifier:0-2 Iteration:0-2
Candidates.#1....: apesthetize -> Arrgt

Started: Thu Sep 24 13:55:55 2020
Stopped: Thu Sep 24 13:55:58 2020

$ john --format=NT your-password-hash-capitalised.txt --wordlist=words.txt --rules=Capitals
Using default input encoding: UTF-8
Loaded 1 password hash (NT [MD4 256/256 AVX2 8x3])
Warning: no OpenMP support for this hash type, consider --fork=4
Press 'q' or Ctrl-C to abort, almost any other key for status
Apples           (Michael)
1g 0:00:00:00 DONE (2020-09-24 14:04) 25.00g/s 542400p/s 542400c/s 542400C/s Appertise..Appositive
Use the "--show --format=NT" options to display all of the cracked passwords reliably
Session completed

Still almost instant. What else can we do?

The Number

But the internet became even more popular, and then businesses started saying, “I’m afraid you cannot join unless you have at least one capital letter and at least one number.”

Again, less than half a microsecond’s consideration before we collectively decided “You shall be getting the number one and that will be at the end of my now capitalised password.”

Now I’m gonna take my capitalised dictionary and my regular English dictionary and append 1 to every word. I’m also going to add the previous dictionaries as well.

Dictionary No. Words
One-Appended Dictionary 1,480,412
Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 108,947 0.76%
ignis-1K.txt 1,000 528 52.80%
ignis-10K.txt 10,000 4,333 43.33%
ignis-100K.txt 100,000 19,689 19.69%
ignis-1M.txt 1,000,000 69,121 6.91%
ignis-10M.txt 10,000,000 154,782 1.55%

Wowee, that was a massive jump in success rates. I think McIntyre is onto something.

Let’s replicate this with our tools.

# Updated hash file
$ cat your-password-hash-capitalised-with-one.txt
Michael:1000:aad3b435b51404eeaad3b435b51404ee:EEC9F50872C496F3998692E8EC2C948D:::

# Rule files to capitalise the first letter and append 1
$ cat hashcat-rules.txt
:
c
$1
c$1
$ cat john-rules.txt
[List.Rules:CapitalsOnes]
c
$1
c$1
$ cat john-rules.txt >> /etc/john/john.conf

# Cracking with new rules
$ hashcat -a 0 -m 1000 your-password-hash-capitalised-with-one.txt words.txt -r hashcat-rules.txt
hashcat (v6.1.1) starting...

...snip...

Dictionary cache hit:
* Filename..: words.txt
* Passwords.: 466550
* Bytes.....: 4862992
* Keyspace..: 1866200

eec9f50872c496f3998692e8ec2c948d:Apples1

Session..........: hashcat
Status...........: Cracked
Hash.Name........: NTLM
Hash.Target......: eec9f50872c496f3998692e8ec2c948d
Time.Started.....: Thu Sep 24 14:27:44 2020 (0 secs)
Time.Estimated...: Thu Sep 24 14:27:44 2020 (0 secs)
Guess.Base.......: File (words.txt)
Guess.Mod........: Rules (hashcat-rules.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:  8484.6 kH/s (0.68ms) @ Accel:1024 Loops:4 Thr:1 Vec:8
Recovered........: 1/1 (100.00%) Digests
Progress.........: 98304/1866200 (5.27%)
Rejected.........: 0/98304 (0.00%)
Restore.Point....: 20480/466550 (4.39%)
Restore.Sub.#1...: Salt:0 Amplifier:0-4 Iteration:0-4
Candidates.#1....: apesthetize -> Arrgt1

Started: Thu Sep 24 14:27:43 2020
Stopped: Thu Sep 24 14:27:46 2020

$ john --format=NT your-password-hash-capitalised-with-one.txt --wordlist=words.txt --rules=CapitalsOnes

The Special Character

And, for a period of time, this was acceptable until whole new unexpected and exciting dawn emerged. A world of special characters, we didn’t even know what they were!

And businesses would say, “We need a capital letter, we need a number, but we will also require a special character.”

“There they are. I had no idea these characters were so special.” Until all of our eyes stopped upon the exclamation mark

…which we then put at the end of our now capitalised password, just after the one.

The formula is starting to pay off.

Now I’m gonna take my capitalised dictionary and my regular English dictionary and my one-appended dictionary and append “!” to every word.

Dictionary No. Words
Exclamation-Appended Dictionary 2,960,824
Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 118,713 0.83%
ignis-1K.txt 1,000 528 52.80%
ignis-10K.txt 10,000 4,334 43.34%
ignis-100K.txt 100,000 19,701 19.70%
ignis-1M.txt 1,000,000 69,670 6.97%
ignis-10M.txt 10,000,000 160,012 1.60%

Surprisingly, this rule’s performance barely stands up to the previous ones in improving success rate. With no additional passwords found in the top 1000 and only one additional in the top 10000.

Let’s do it with our tools.

# Updated hash file
$ cat your-password-hash-capitalised-with-one-and-exclamation.txt
Michael:1000:aad3b435b51404eeaad3b435b51404ee:C2D0F7134D4557224E923A8538C4444F:::

# Rule files to capitalise the first letter, add 1 and add !
$ cat hashcat-rules.txt
:
c
$1
c$1
$!
c$!
$1$!
c$1$!

$ cat john-rules.txt
[List.Rules:McIntyre]
c
$1
c$1
$!
c$!
$1$!
c$1$!
$ cat john-rules.txt >> /etc/john/john.conf

# Cracking with new rules
$ hashcat -a 0 -m 1000 your-password-hash-capitalised-with-one-and-exclamation.txt words.txt -r hashcat-rules.txt
hashcat (v6.1.1) starting...

...snip...

Dictionary cache hit:
* Filename..: words.txt
* Passwords.: 466550
* Bytes.....: 4862992
* Keyspace..: 3732400

c2d0f7134d4557224e923a8538c4444f:Apples1!

Session..........: hashcat
Status...........: Cracked
Hash.Name........: NTLM
Hash.Target......: c2d0f7134d4557224e923a8538c4444f
Time.Started.....: Thu Sep 24 16:10:56 2020 (0 secs)
Time.Estimated...: Thu Sep 24 16:10:56 2020 (0 secs)
Guess.Base.......: File (words.txt)
Guess.Mod........: Rules (hashcat-rules.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........: 12572.3 kH/s (1.31ms) @ Accel:1024 Loops:8 Thr:1 Vec:8
Recovered........: 1/1 (100.00%) Digests
Progress.........: 196608/3732400 (5.27%)
Rejected.........: 0/196608 (0.00%)
Restore.Point....: 20480/466550 (4.39%)
Restore.Sub.#1...: Salt:0 Amplifier:0-8 Iteration:0-8
Candidates.#1....: apesthetize -> Arrgt1!

Started: Thu Sep 24 16:10:55 2020
Stopped: Thu Sep 24 16:10:58 2020

$ john --format=NT your-password-hash-capitalised-with-one-and-exclamation.txt --wordlist=words.txt --rules=McIntyre
Loaded 1 password hash (NT [MD4 256/256 AVX2 8x3])
Warning: no OpenMP support for this hash type, consider --fork=4
Press 'q' or Ctrl-C to abort, almost any other key for status
Apples1!         (Michael)
1g 0:00:00:00 DONE (2020-09-24 16:08) 4.761g/s 13432Kp/s 13432Kc/s 13432KC/s Appendectomy1!..Applier1!
Use the "--show --format=NT" options to display all of the cracked passwords reliably
Session completed

Even with the added complexity rules, my laptop was able to crack the password in under a second. Crazy, right?

The “McIntyre Special”

The table below is the stats from our original dictionary.

Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 63,578 0.44%
ignis-1K.txt 1,000 497 49.70%
ignis-10K.txt 10,000 3,570 35.70%
ignis-100K.txt 100,000 14,025 14.03%
ignis-1M.txt 1,000,000 41,408 4.14%
ignis-10M.txt 10,000,000 79,843 0.79%

The table below is the stats from our final dictionary. Let’s examine this further.

Wordlist No. Passwords No. Matches % Matched
rockyou.txt 14,343,856 118,713 0.83%
ignis-1K.txt 1,000 528 52.80%
ignis-10K.txt 10,000 4,334 43.34%
ignis-100K.txt 100,000 19,701 19.70%
ignis-1M.txt 1,000,000 69,670 6.97%
ignis-10M.txt 10,000,000 160,012 1.60%

The table below summarises the differences between the two. Perhaps not unsurprisingly, the “McIntyre Special” cracking algorithm works more effectively on larger sets of data. At best, we doubled the amount of cracked passwords we could have got with a dictionary alone.

Wordlist ∆ Matches ∆ % Matched Improvement
rockyou.txt 55,135 0.39% 86.72%
ignis-1K.txt 31 3.10% 6.24%
ignis-10K.txt 764 7.64% 21.40%
ignis-100K.txt 5,676 5.67% 40.47%
ignis-1M.txt 28,262 2.83% 68.25%
ignis-10M.txt 80,169 0.81% 100.41%

For such a simple algorithm, I for one am deeply impressed with how performant it is. But how does it compare to what a real hacker might use? The following table compares the effectiveness of the “McIntyre Special” vs “OneRuleToRuleThemAll”. This rulelist came from some great research by NotSoSecure, who include a fascinating writeup on how they developed the rulelist.

Each wordlist was converted to a list of MD5 hashes using the original passwords. This is due to the superior parallelism of GPUs and I couldn’t be bothered figuring how to do plaintext comparisons with a GPU in Python.

Wordlist No. Passwords No. Matches % Matched Time
rockyou-md5.txt 14,343,856 4,207,908 29.34% 18 mins 2 secs
ignis-1K-md5.txt 1,000 800 80.00% 7 secs
ignis-10K-md5.txt 10,000 8,022 80.22% 9 secs
ignis-100K-md5.txt 100,000 61,975 61.98% 23 secs
ignis-1M-md5.txt 1,000,000 576,398 57.63% 2 mins 25 secs
ignis-10M-md5.txt 10,000,000 3,655,647 36.56% 15 mins 31 secs

Let’s do a final comparison with the “McIntyre Special”.

Wordlist ∆ Matches ∆ % Matched Improvement
rockyou.txt 4,089,195 28.51% 3544.61%
ignis-1K.txt 272 27.20% 151.52%
ignis-10K.txt 3,688 36.88% 185.09%
ignis-100K.txt 42,274 42.28% 314.58%
ignis-1M.txt 506,728 50.66% 827.33%
ignis-10M.txt 3,495,635 34.96% 2284.61%

Unfortunately it looks like Michael McIntyre isn’t at the forefront of password cracking. Who knew?

Recommendations

Mr McIntyre unknowingly makes a point that’s been a point of contention in the security industry. Should we be enforcing complexity in passwords? All it seems to do is give an attacker a map to what the most likely user passwords are. Indeed, an organisation’s password complexity requirements are something I use as a guide to help me guess and crack passwords.

The National Institute of Standards and Technology (NIST) weighs in on this too. NIST’s latest publication on Digital Identity Guidelines have a great list of suggestions for improving password security in section 5.1.1.2 (Memorized Secret Verifiers). It’s my go-to when recommending improvements to an organisation’s security policy.

Their words?

Verifiers SHOULD NOT impose other composition rules (e.g., requiring mixtures of different character types or prohibiting consecutively repeated characters) for memorized secrets. Verifiers SHOULD NOT require memorized secrets to be changed arbitrarily (e.g., periodically). However, verifiers SHALL force a change if there is evidence of compromise of the authenticator.

Now, a lot of people suggest that this could lead to poorer security, as it allows people to set weak passwords. But let’s take a look at their other suggestions.

When processing requests to establish and change memorized secrets, verifiers SHALL compare the prospective secrets against a list that contains values known to be commonly-used, expected, or compromised. For example, the list MAY include, but is not limited to:

  • Passwords obtained from previous breach corpuses.
  • Dictionary words.
  • Repetitive or sequential characters (e.g. ‘aaaaaa’, ‘1234abcd’).
  • Context-specific words, such as the name of the service, the username, and derivatives thereof.

If the chosen secret is found in the list, the CSP or verifier SHALL advise the subscriber that they need to select a different secret, SHALL provide the reason for rejection, and SHALL require the subscriber to choose a different value.

They definitely thought of that. There are some challenges in implementing this, but it should be something organisations move towards. After all, how many people do you think have “facebook” in their Facebook password? In my super scientific opinion, at least 1.

Verifiers SHOULD offer guidance to the subscriber, such as a password-strength meter [Meters], to assist the user in choosing a strong memorized secret. This is particularly important following the rejection of a memorized secret on the above list as it discourages trivial modification of listed (and likely very weak) memorized secrets [Blacklists].

Oh look, they directly address the trivial modification of a weak password. Thanks NIST!

Recommendations For Individuals

And it’s at this moment that everybody at the London Palladium is thinking, “I should probably change my password. I’m probably gonna do that tomorrow.”

You should change it as soon as possible. And not just that one, all of the passwords you can remember. You likely use the same password for everything, or a minor variation on a single word. Maybe you don’t add 1 and !, maybe you prefer 123! or even something “complex” like 123$%^. If just one variation of your password is found, it’s not too hard to find the other possible variants.

Here’s some tools to help you out.

Passphrases

“Need to think of another special word.”

Try special words. One of my favourite webcomics, xkcd, addresses this.

"Password Strength" from xkcd.com

Now the issue is having a strong password/passphrase that you use across all your accounts. How can you protect yourself while still making it easy on yourself?

Password Managers

If you store passwords in your web browser, this is not considered a secure password manager. If you use Chrome and save passwords in it, I have an exercise for you.

  1. Click the three vertical dots in the top right-hand corner
  2. Click Settings
  3. Under the Auto-fill section, click Passwords
  4. Click the three vertical dots above the list of passwords and select “Export passwords”.
  5. Open the file you just saved

There’s your entire online persona in a simple text file. How easy was that? Now consider how easy it is to do that if someone gains access to your computer. Luckily, you’re one step closer to good security!

Password managers like LastPass, Dashlane, and 1Password all do one thing: protect your passwords. And they all allow you to import all your passwords from Chrome, using that file you just generated.

They’ll even tell you exactly what passwords you should change!

Once that file has been imported into your password manager, make sure to delete it! And make sure to remove those passwords from Chrome.

Now you only need to remember ONE passphrase: the one you use for your password manager! Let the password manager generate stupidly complex and long passwords that no-one could ever remember or guess.

Once you have a password manager, set aside 30 minutes to go through every account and change the password to whatever your password manager suggests. I personally set a 20 character length password with all character types enabled. Remember, you don’t need to remember these passwords anymore! The longer, the better.

Multifactor Authentication

You might have some online accounts that annoyingly send you an SMS before you can login. That’s multifactor authentication! Not only would an attacker need to know your password, they’d need access to your phone (or gain access to your Telco).

SMS multifactor authentication has been known to be vulnerable for years in various ways, but it’s still a great deal more secure than a password alone.

Enter soft-tokens: things like Google Authenticator and Microsoft Authenticator. These are apps on your phone that generate a string of six numbers that change every 30 seconds. This code is tied to your device, so you are at risk of losing access to your accounts if you lose your phone.

How do you get around this? Enter Authy! This app automatically backs up your multifactor codes so you migrate them to a new device.

There are other methods of multifactor that use hardware tokens; little USB sticks like YubiKeys. You need to plug these in when you try to authenticate. I’d recommend these for your most sensitive accounts.

Where possible, enable multifactor authentication on every account you have. The piece of mind is worth it.

have i been pwned

have i been pwned is a great and scary resource. You can put in any of your email addresses and it’ll quickly tell you of any known data breaches that email was found in.

You can even set up email alerts to let you know as soon as your email was found in a data breach!

Although I DO NOT RECOMMEND IT, you can input a password and it’ll check if it’s in any of their breach data. Just remember, even if your super special password isn’t in any of THEIR breach data, doesn’t mean that it isn’t in a breach they haven’t got access to.

Recommendations For Enterprise

What about enterprises? What can they do to protect themselves, their users, and their customers?

Credential Policies

NIST’s Digital Identity Guidelines are a fantastic resource for implementing credential policies that ensure high levels of security without requiring architecture that would make Fort Knox jealous.

The key takeaways include:

  • Enforce a minimum character limit, such as 8 characters. This should be increased as needed for sensitive applications.
  • Allow, but do not enforce, all character classes. This includes all the complexity requirements we know and love/hate, as well as spaces and even non-ASCII characters. Want to have a French or Spanish password? Why not?
  • Implement password blocklists. As a start, maybe include the McIntyre Special? Be sure to include other common passwords, such as keyboard patterns (QWERTY123) and the name of your enterprise.
  • Try to include a password strength meter in all areas where credentials can be rotated, such as account settings and forgot password workflows.
  • Allow password pasting! Let users use password managers. Requiring a user to type a password means they’re going to make it easy on themselves!
  • Security questions and password hints should have died out a long time ago. Move to multifactor authentication and self-service password resets!
  • Rate limit your authentication endpoints. No legitimate user will be submitting 10 login requests per second.
  • Salt and hash your passwords. If you still store user passwords in plaintext, encrypted, or hashed with MD5, you’re putting your users at risk. If you’re wondering why encrypted passwords are bad, think about what happens when your encryption key is compromised.

Password Managers

Most password managers include an Enterprise option, which includes advanced features like password and note sharing.

If it’s good enough for someone outside of work, why not implement it at work? Say goodbye to the sticky note on people’s monitors!

Multifactor Authentication

Multifactor providers like Duo and Auth0 are working hard to bring the benefits of multifactor authentication to your enterprise!

These plug in to your Active Directory environment, allowing you to secure user and administrator accounts.

Account Lockouts

Account lockouts are a point of contention. This is an area where usability and security butt heads. My recommendation is to enable permanent lockouts only for the most highly privileged accounts (such as your domain administrators) after multiple failed login attempts. If they are secured with multifactor authentication, consideration can be made for temporary account lockouts (such as two hours).

After all, if your administrators are using a password manager, why would they be getting the password wrong multiple times?

For all other user accounts, a sensible balance between usability and security is a 15 minute temporary lockout after 5 failed attempts. Logging and alerting on multiple lockouts should be a priority, as these can be indicative of an attacker attempting to password spray or brute force credentials in your environment.

PAM

Privileged access management is a relatively new area that’s entered the spotlight in information security. Solutions like CyberArk and Thycotic take management of highly privileged accounts out of your user’s (and potential attacker’s) hands.

These solutions automatically rotate passwords according to a policy you set, and broker access to any sensitive machines. For example, an administrator account no longer means domain administrator. That admin account is only used to login to the PAM solution, which GIVES access to certain accounts. Think of it like a password manager on steroids.

Conclusion

We’ve just taken a wild ride through password security at a fairly high level. Each of these topics could easily be talked about for hours, but hopefully you’ve gained insight into just how complicated password security can be.

I’d like to challenge anyone responsible for password security in their organisation to use the McIntyre Special wordlist against their own password databases (with permission, of course). You may be surprised how many passwords you end up cracking!

The full script used to generate the bulk of the statistics used in this post can be found below. It’ll automatically download all the relevant wordlists, print out the stats, and even create the McIntyre Special wordlist!

Have fun!

More Stats

The following are the raw outputs of hashcat when I gathered stats for this post. You may find them interesting. The next big block is using OneRuleToRuleThemAll on the dictionary to crack our password files.

* Device #3: AMD Radeon Pro 5500M Compute Engine, 4016/4080 MB (1020 MB allocatable), 24MCU

$ hashcat ignis-1K-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-1K-md5.txt
Time.Started.....: Sat Sep 26 11:30:38 2020 (7 secs)
Time.Estimated...: Sat Sep 26 11:30:45 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........:  3096.8 MH/s (6.37ms) @ Accel:256 Loops:64 Thr:64 Vec:1
Recovered........: 800/1000 (80.00%) Digests
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-64
Candidates.#3....: a -> zwitterinnic

Started: Sat Sep 26 11:30:38 2020
Stopped: Sat Sep 26 11:30:46 2020

$ hashcat ignis-10K-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-10K-md5.txt
Time.Started.....: Sat Sep 26 11:30:02 2020 (9 secs)
Time.Estimated...: Sat Sep 26 11:30:11 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........:  2202.3 MH/s (6.57ms) @ Accel:128 Loops:128 Thr:64 Vec:1
Recovered........: 8022/10000 (80.22%) Digests
Remaining........: 1978 (19.78%) Digests
Recovered/Time...: CUR:N/A,N/A,N/A AVG:54729,3283761,78810272 (Min,Hour,Day)
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-128
Candidates.#3....: naipkin -> zwitterinnic

Started: Sat Sep 26 11:30:01 2020
Stopped: Sat Sep 26 11:30:12 2020

$ hashcat ignis-100K-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-100K-md5.txt
Time.Started.....: Sat Sep 26 11:29:13 2020 (23 secs)
Time.Estimated...: Sat Sep 26 11:29:36 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........:   848.6 MH/s (7.09ms) @ Accel:128 Loops:128 Thr:64 Vec:1
Recovered........: 61975/100000 (61.98%) Digests
Remaining........: 38025 (38.02%) Digests
Recovered/Time...: CUR:N/A,N/A,N/A AVG:163578,9814680,235552335 (Min,Hour,Day)
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-128
Candidates.#3....: naipkin -> zwitterinnic

Started: Sat Sep 26 11:29:12 2020
Stopped: Sat Sep 26 11:29:36 2020

$ hashcat ignis-1M-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-1M-md5.txt
Time.Started.....: Sat Sep 26 11:26:00 2020 (2 mins, 25 secs)
Time.Estimated...: Sat Sep 26 11:28:25 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........:   133.0 MH/s (9.49ms) @ Accel:128 Loops:128 Thr:64 Vec:1
Recovered........: 576398/1000000 (57.64%) Digests
Remaining........: 423602 (42.36%) Digests
Recovered/Time...: CUR:214455,N/A,N/A AVG:239005,14340336,344168083 (Min,Hour,Day)
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-128
Candidates.#3....: naipkin -> zwitterinnic

Started: Sat Sep 26 11:25:57 2020
Stopped: Sat Sep 26 11:28:25 2020

$ hashcat ignis-10M-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-10M-md5.txt
Time.Started.....: Sat Sep 26 10:45:42 2020 (15 mins, 31 secs)
Time.Estimated...: Sat Sep 26 11:01:13 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........: 20670.9 kH/s (15.08ms) @ Accel:128 Loops:32 Thr:64 Vec:1
Recovered........: 3655647/10000000 (36.56%) Digests
Remaining........: 6344353 (63.44%) Digests
Recovered/Time...: CUR:217526,N/A,N/A AVG:235588,14135326,339247827 (Min,Hour,Day)
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-32
Candidates.#3....: naipkin -> zwitterinnic

Started: Sat Sep 26 10:45:07 2020
Stopped: Sat Sep 26 11:01:13 2020

$ hashcat rockyou-md5.txt words_alpha.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: rockyou-md5.txt
Time.Started.....: Thu Sep 24 17:44:32 2020 (18 mins, 2 secs)
Time.Estimated...: Thu Sep 24 18:02:34 2020 (0 secs)
Guess.Base.......: File (words_alpha.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........: 33207.9 kH/s (4.77ms) @ Accel:8 Loops:256 Thr:64 Vec:1
Recovered........: 4207908/14343922 (29.34%) Digests
Remaining........: 10136014 (70.66%) Digests
Recovered/Time...: CUR:193877,N/A,N/A AVG:235050,14103053,338473292 (Min,Hour,Day)
Progress.........: 19243505485/19243505485 (100.00%)
Rejected.........: 0/19243505485 (0.00%)
Restore.Point....: 370103/370103 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-256
Candidates.#3....: xiphiplass3on -> zwitterinnic

Started: Thu Sep 24 17:43:35 2020
Stopped: Thu Sep 24 18:02:34 2020

I thought it’d be interesting to compare a base dictionary to what I regularly use for password cracking – rockyou. When testing on the ignis-10M password list, the results are shocking: 36.56% with the base dictionary vs 80.06% with rockyou… I think I’ll keep using rockyou. 16 minutes vs 1 hour 28 minutes to crack 219% more passwords is a tradeoff I’ll take any day.

$ hashcat ignis-10M-md5.txt rockyou.txt -r OneRuleToRuleThemAll.rule -d 3 -O --potfile-disable
hashcat (v6.1.1) starting...

...snip...

Session..........: hashcat
Status...........: Exhausted
Hash.Name........: MD5
Hash.Target......: ignis-10M-md5.txt
Time.Started.....: Sat Sep 26 12:27:41 2020 (1 hour, 28 mins)
Time.Estimated...: Sat Sep 26 13:55:49 2020 (0 secs)
Guess.Base.......: File (rockyou.txt)
Guess.Mod........: Rules (OneRuleToRuleThemAll.rule)
Guess.Queue......: 1/1 (100.00%)
Speed.#3.........:   329.0 MH/s (16.38ms) @ Accel:128 Loops:32 Thr:64 Vec:1
Recovered........: 8005518/10000000 (80.06%) Digests
Remaining........: 1994482 (19.94%) Digests
Recovered/Time...: CUR:9109,2216333,N/A AVG:91793,5507596,132182308 (Min,Hour,Day)
Progress.........: 745836246080/745836246080 (100.00%)
Rejected.........: 160872530/745836246080 (0.02%)
Restore.Point....: 14344384/14344384 (100.00%)
Restore.Sub.#3...: Salt:0 Amplifier:51968-51995 Iteration:0-32
Candidates.#3....: $HEX[3032303938383533313333] -> $HEX[042a0337c2a156616c6c732103]

Started: Sat Sep 26 12:27:07 2020
Stopped: Sat Sep 26 13:55:51 2020

McIntyre Special Python Script

#!/usr/bin/python3
import requests
from os.path import exists

WORDS_URL = "https://raw.githubusercontent.com/dwyl/english-words/master/words_alpha.txt"
WORDS_FILENAME = "words_alpha.txt"
IGNIS_1K_URL = "https://github.com/FlameOfIgnis/Pwdb-Public/raw/master/wordlists/ignis-1K.txt"
IGNIS_1K_FILENAME = "ignis-1K.txt"
IGNIS_10K_URL = "https://github.com/FlameOfIgnis/Pwdb-Public/raw/master/wordlists/ignis-10K.txt"
IGNIS_10K_FILENAME = "ignis-10K.txt"
IGNIS_100K_URL = "https://github.com/FlameOfIgnis/Pwdb-Public/raw/master/wordlists/ignis-100K.txt"
IGNIS_100K_FILENAME = "ignis-100K.txt"
IGNIS_1M_URL = "https://raw.githubusercontent.com/FlameOfIgnis/Pwdb-Public/master/wordlists/ignis-1M.txt"
IGNIS_1M_FILENAME = "ignis-1M.txt"
IGNIS_10M_URL = "https://raw.githubusercontent.com/FlameOfIgnis/Pwdb-Public/master/wordlists/ignis-10M.txt"
IGNIS_10M_FILENAME = "ignis-10M.txt"

# Check if the dictionaries are downloaded, if not, download them
def get_dictionary(url, filename):
	if not exists(filename):
		open(filename,"wb").write(requests.get(url).content)
	else:
		print(filename + " already exists, skipping download...")

# Convert wordlist into dictionary
# Adapted from https://github.com/dwyl/english-words/blob/master/scripts/create_json.py
def wordlist_to_dictionary(wordlist_filename):
	wordlist_dict = {}
	with open(wordlist_filename, errors="ignore") as wordlist:
		words = wordlist.readlines()
		for word in words:
			wordlist_dict[word.rstrip()] = "1"

	return wordlist_dict

# Counts number of words in dictionary
def count_dictionary(json_data):
	return(len(json_data))

# Generate wordlist with first letter capitalised
def capital_letter_wordlist(wordlist_dict):
	capital_letter_dict = {}
	for word in wordlist_dict:
		word_list = list(word)
		word_list[0] = word_list[0].upper()
		new_word = "".join(word_list)
		capital_letter_dict[new_word] = 1

	return capital_letter_dict

# Generate wordlist with first letter capitalised and "1" appended
def add_one_wordlist(capital_letter_dict):
	add_one_dict = {}
	for word in capital_letter_dict:
		new_word = word + "1"
		add_one_dict[new_word] = 1

	return add_one_dict

# Generate wordlist with first letter capitalised and "1!" appended
def add_exclamation_wordlist(add_one_dict):
	add_exclamation_dict = {}
	for word in add_one_dict:
		new_word = word + "!"
		add_exclamation_dict[new_word] = 1

	return add_exclamation_dict

# Checks against rockyou.txt to compare percentage of owned
def wordlist_benchmark(wordlist_dict, benchmark_dict):
	matches = 0
	num_benchmark_lines = len(benchmark_dict)
	num_wordlist_lines = len(wordlist_dict)

	for word in wordlist_dict:
		if word in benchmark_dict:
			matches += 1

	results = {
		"matches": matches,
		"num_benchmark_lines": num_benchmark_lines,
		"num_wordlist_lines": num_wordlist_lines,
		"percentage": "{:.2%}".format(matches / num_benchmark_lines)
	}

	return results

# Get English dictionary
print("Downloading wordlists...")
get_dictionary(WORDS_URL, WORDS_FILENAME)
words = wordlist_to_dictionary(WORDS_FILENAME)

# Get ignis dictionaries
get_dictionary(IGNIS_1K_URL, IGNIS_1K_FILENAME)
get_dictionary(IGNIS_10K_URL, IGNIS_10K_FILENAME)
get_dictionary(IGNIS_100K_URL, IGNIS_100K_FILENAME)
get_dictionary(IGNIS_1M_URL, IGNIS_1M_FILENAME)
get_dictionary(IGNIS_10M_URL, IGNIS_10M_FILENAME)

print("Converting wordlists...")

# Convert rockyou.txt to dictionary
rockyou = wordlist_to_dictionary("rockyou.txt")

# Load ignis dictionaries
ignis_1k = wordlist_to_dictionary(IGNIS_1K_FILENAME)
ignis_10k = wordlist_to_dictionary(IGNIS_10K_FILENAME)
ignis_100k = wordlist_to_dictionary(IGNIS_100K_FILENAME)
ignis_1M = wordlist_to_dictionary(IGNIS_1M_FILENAME)
ignis_10M = wordlist_to_dictionary(IGNIS_10M_FILENAME)

print("Just the special word...")
print("rockyou.txt")
print(wordlist_benchmark(words, rockyou))
print("ignis-1K.txt")
print(wordlist_benchmark(words, ignis_1k))
print("ignis-10K.txt")
print(wordlist_benchmark(words, ignis_10k))
print("ignis-100K.txt")
print(wordlist_benchmark(words, ignis_100k))
print("ignis-1M.txt")
print(wordlist_benchmark(words, ignis_1M))
print("ignis-10M.txt")
print(wordlist_benchmark(words, ignis_10M))

print("Now the capital letter...")
capital_dictionary = capital_letter_wordlist(words)
capital_dictionary.update(words)
print("rockyou.txt")
print(wordlist_benchmark(capital_dictionary, rockyou))
print("ignis-1K.txt")
print(wordlist_benchmark(capital_dictionary, ignis_1k))
print("ignis-10K.txt")
print(wordlist_benchmark(capital_dictionary, ignis_10k))
print("ignis-100K.txt")
print(wordlist_benchmark(capital_dictionary, ignis_100k))
print("ignis-1M.txt")
print(wordlist_benchmark(capital_dictionary, ignis_1M))
print("ignis-10M.txt")
print(wordlist_benchmark(capital_dictionary, ignis_10M))

if not exists("capital_dictionary.txt"):
	with open("capital_dictionary.txt", "w") as f:
		for word in capital_dictionary.keys():
			f.write(word.strip()+"\n")

print("Now the number one...")
add_one_dictionary = add_one_wordlist(capital_dictionary)
add_one_dictionary.update(capital_dictionary)
print("rockyou.txt")
print(wordlist_benchmark(add_one_dictionary, rockyou))
print("ignis-1K.txt")
print(wordlist_benchmark(add_one_dictionary, ignis_1k))
print("ignis-10K.txt")
print(wordlist_benchmark(add_one_dictionary, ignis_10k))
print("ignis-100K.txt")
print(wordlist_benchmark(add_one_dictionary, ignis_100k))
print("ignis-1M.txt")
print(wordlist_benchmark(add_one_dictionary, ignis_1M))
print("ignis-10M.txt")
print(wordlist_benchmark(add_one_dictionary, ignis_10M))

if not exists("add_one_dictionary.txt"):
	with open("add_one_dictionary.txt", "w") as f:
		for word in add_one_dictionary.keys():
			f.write(word.strip()+"\n")

print("Now the exclamation point...")
add_exclamation_dictionary = add_exclamation_wordlist(add_one_dictionary)
add_exclamation_dictionary.update(add_one_dictionary)
print("rockyou.txt")
print(wordlist_benchmark(add_exclamation_dictionary, rockyou))
print("ignis-1K.txt")
print(wordlist_benchmark(add_exclamation_dictionary, ignis_1k))
print("ignis-10K.txt")
print(wordlist_benchmark(add_exclamation_dictionary, ignis_10k))
print("ignis-100K.txt")
print(wordlist_benchmark(add_exclamation_dictionary, ignis_100k))
print("ignis-1M.txt")
print(wordlist_benchmark(add_exclamation_dictionary, ignis_1M))
print("ignis-10M.txt")
print(wordlist_benchmark(add_exclamation_dictionary, ignis_10M))

if not exists("mcintyre_special_dictionary.txt"):
	with open("mcintype_special_dictionary.txt", "w") as f:
		for word in add_exclamation_dictionary.keys():
			f.write(word.strip()+"\n")

About the author

Billy Cody is a Senior Security Consultant at Volkis. He’s focused the last few years on breaking stuff as a penetration tester, but also has experience in security engineering and security reviews. He is a co-organiser of SecTalks Brisbane and is on LinkedIn.

Photo by Matthew Brodeur on Unsplash.

If you need help with your security, get in touch with Volkis.
Follow us on Twitter and LinkedIn