Organizing Leaked Database
by cl0ud - January 24, 2019 at 11:43 AM
Hi everyone,

Most of you knows, last db leak(Collections). Im trying to organize these leaks by every mail. Example;

I got 3 line like;

now im organizing these like that;

1.create directory with first and second character of mail and create file with first 3 character of mail if file not exists else print line in file;

cat ./f/s/fir.txt
[email protected]:example1

cat ./s/e/sec.txt
[email protected]:example2

cat ./t/h/thi.txt
[email protected]:example3

2.For search a mail, algorithm is; get first 3 char of given target and use them. Example;;
    -input_search_target:  [email protected]
    -get characters : "t","h","i"
    -grep "[email protected]" t/h/thi

this will gives amazing speed for search mail, but while organizing these mail with this algorithm(first part), it takes lots of time.
-First, i tried with python.(Too Slow as excepted)
-Second, tried with c++.(Very Fast but some mails raise error while handling.)
-Third, tried with bash scripting.(Fast but not enought.)

Example, with bash organizing about 200 million line takes about 10 hours. If we calculate all process;

-1 billion     =  50 hours(about 2 days)
-100 billion =  about 200 days :sick:

Is there any other solution for organizing big data or any other suggestion about these process?

Why not import them into a database.. like sql and then index them.
Think that would be best bet.
I prefer using first 3 or 4 character as you did, it will make the search way better and faster than anything else.

But it will take a lot of time to split 'em as you wish.
In BreachCompilation these is tools written in BASH that do splitting job.
Take a look at it i hope it helps you.

Good luck with that idea.
The reason of doing that. This method gives amazing speed. You can search a mail about 2-3 second. With that you can combine leaked databases with any other attack scripts. For example one of my scripts like that: --> search db and get mails, like grep (input:xxxxxx    output:[email protected]:example123, [email protected]:example1, [email protected]:example)   --> try to login target site( input:target_mail target_password) --> get target mails and passwords from then combine passwords and use them for

Example for

1) Get target username (input: xxxxxx )
2) Search username and get output ( output:[email protected]:example123, [email protected]:example1, [email protected]:example)
3) Fix passwords for target password format, One uppercase, number e.g. (Example123, Example1, Example1, Example12, Example123, Example123. ,Example123.. e.g)
4) Fix target mail (target_mail = [email protected] )
5) Use [email protected] and fixed_passwords  as input for
6) Print target_mail and correct_password if returns True.

With this algorithm you can test any mail_server which doesnt use captcha. But if you want search and attack scripts works sametime, search script have to be fast.

As i told before,i wrote same functions in python, c++ , bash. Python like a turtle as i expected, but if you use special functions with c++, giving incredible speed but file/directory read/write makes c++ slow. After that, i wrote with bash scripting. It works excellent but it takes about 200 day and amazing space on disk. And still not enought speed for me...
Looks like a good process. For sure think implementing a DB is the move tho/
I am half way into developing the same exact thing at the moment. I'll lyk when Its all done
I done this process before on "BreachComp" and worked great. Wrote code one time with python and change only login request(easily can get with burp and "copy as request" plugin).
What DB do you suggest in order to import big combos like Collection1-5??
I've imported all of mine into ElasticSearch and that has been working well
This is pretty interesting i use a custom db searcher
This is only efficient for email:password or user:password databases or combolists.

Possibly Related Threads…
Thread Author Replies Views Last Post
Organizing large DBs travisci 1 629 April 28, 2018 at 06:26 PM
Last Post: Omnipotent

 Users browsing this thread: 1 Guest(s)