Organizing Leaked Database
by cl0ud - January 24, 2019 at 11:43 AM
#1
Hi everyone,

Most of you knows, last db leak(Collections). Im trying to organize these leaks by every mail. Example;


I got 3 line like;


now im organizing these like that;

1.create directory with first and second character of mail and create file with first 3 character of mail if file not exists else print line in file;

cat ./f/s/fir.txt
[email protected]:example1

cat ./s/e/sec.txt
[email protected]:example2

cat ./t/h/thi.txt
[email protected]:example3

2.For search a mail, algorithm is; get first 3 char of given target and use them. Example;
search_target.sh;
    -input_search_target:  [email protected]
    -get characters : "t","h","i"
    -grep "[email protected]" t/h/thi


this will gives amazing speed for search mail, but while organizing these mail with this algorithm(first part), it takes lots of time.
-First, i tried with python.(Too Slow as excepted)
-Second, tried with c++.(Very Fast but some mails raise error while handling.)
-Third, tried with bash scripting.(Fast but not enought.)

Example, with bash organizing about 200 million line takes about 10 hours. If we calculate all process;


-1 billion     =  50 hours(about 2 days)
-100 billion =  about 200 days :sick:

Is there any other solution for organizing big data or any other suggestion about these process?

Thanks.
Reply
#2
Why not import them into a database.. like sql and then index them.
Reply
#3
Think that would be best bet.
Reply
#4
I prefer using first 3 or 4 character as you did, it will make the search way better and faster than anything else.

But it will take a lot of time to split 'em as you wish.
In BreachCompilation these is tools written in BASH that do splitting job.
Take a look at it i hope it helps you.

Good luck with that idea.
Reply
#5
The reason of doing that. This method gives amazing speed. You can search a mail about 2-3 second. With that you can combine leaked databases with any other attack scripts. For example one of my scripts like that:

mailsearch.sh --> search db and get mails, like grep (input:xxxxxx    output:[email protected]:example123, [email protected]:example1, [email protected]:example)
attack.py   --> try to login target site( input:target_mail target_password)

combine_scripts.py --> get target mails and passwords from mailsearh.sh then combine passwords and use them for attack.py

Example for Combine_scripts.py

1) Get target username (input: xxxxxx )
2) Search username mailsearch.sh and get output ( output:[email protected]:example123, [email protected]:example1, [email protected]:example)
3) Fix passwords for target password format, One uppercase, number e.g. (Example123, Example1, Example1, Example12, Example123, Example123. ,Example123.. e.g)
4) Fix target mail (target_mail = [email protected] )
5) Use [email protected] and fixed_passwords  as input for attack.py
6) Print target_mail and correct_password if attack.py returns True.

With this algorithm you can test any mail_server which doesnt use captcha. But if you want search and attack scripts works sametime, search script have to be fast.


As i told before,i wrote same functions in python, c++ , bash. Python like a turtle as i expected, but if you use special functions with c++, giving incredible speed but file/directory read/write makes c++ slow. After that, i wrote with bash scripting. It works excellent but it takes about 200 day and amazing space on disk. And still not enought speed for me...
Reply
#6
Looks like a good process. For sure think implementing a DB is the move tho/
Reply
#7
I am half way into developing the same exact thing at the moment. I'll lyk when Its all done
Reply
#8
I done this process before on "BreachComp" and worked great. Wrote code one time with python and change only login request(easily can get with burp and "copy as request" plugin).
Reply
#9
What DB do you suggest in order to import big combos like Collection1-5??
Reply
#10
I've imported all of mine into ElasticSearch and that has been working well
Reply
#11
This is pretty interesting i use a custom db searcher
Reply
#12
This is only efficient for email:password or user:password databases or combolists.
Reply

Possibly Related Threads…
Thread Author Replies Views Last Post
Organizing large DBs travisci 1 629 April 28, 2018 at 06:26 PM
Last Post: Omnipotent

 Users browsing this thread: 1 Guest(s)