Need Help Automating
by zaogord - November 19, 2019 at 01:31 PM
#1
I'm gonna be writing an exam in feb of 2020, so i bought this expensive course to help me with preparations. Cost about $150.
It has study materials, test papers, practice papers etc. 

I downloaded the study material by monitoring the network, when i open a topic on the website, it requests a pdf file from the server. I managed to download the whole study material and sold it to a couple people. Turns out that a lot of people are interested in the Test Papers.

I can do the same as i did with the study material and download it, but the test papers are in the form  of one question per page. There are about 900 questions and each question has a solution, which makes it into 1800 images, which i will have to download.

i want to automate this process.

What i do is open a test, go to developer tools and inspect the traffic.
When i go to the next question, the next question is requested from the server in this form cdn.example.com/question.png
I click on the request, which opens an image in the new tab and then i download it.

This process becomes hectic for 1800 images, how can i automate it?

It would be of great help, if anyone could help me with it.


Tagging a few people who i think can help me with this.

@ @Unspoken @Melatonine
#2
You can use the Data Miner addon for chrome to automate the scraping of images

Or you can crawl the whole site with something like Xenu Link Sleuth to get al the images links

Or if the download links obey to a scheme you can use it t download all pics

It is hard to be more precise without more information.
#3
woulsd have say same: Data Miner addon for chrome is really cool bro.
This forum account is currently banned. Ban Length: Permanent (N/A).
Ban Reason: 1. Shitposting in General. 2. Automation of posts to leech.
#4
(November 19, 2019 at 01:50 PM)Joker2400 Wrote: You can use the Data Miner addon for chrome to automate the scraping of images

Or you can crawl the whole site with something like Xenu Link Sleuth to get al the images links

Or if the download links obey to a scheme you can use it t download all pics

It is hard to be more precise without more information.

Sexy
#5
you can search python scripts in github.com, there are many that work very well for scraping tasks
#6
You can record the screen and after extract the frame of each question to a png file. After it's just a matter of running an OCR software to have it converted to text.

If it looks stupid but it works, it ain't stupid. Tongue
#7
Why’d you tag me bruh

Just use Python
#8
I don't know about the other addons mentioned, but I would use FlashGot as it's super easy.

Possibly Related Threads…
Thread Author Replies Views Last Post
[I need your help] RF 2021 AWARDS Ringer 12 627 October 23, 2021 at 01:57 PM
Last Post: Billy Mays
need little gmail help. $20 reward. niemajak900 6 565 August 08, 2021 at 06:25 PM
Last Post: s6de
I NEED HELP HOW TO GRAB EMAIL AND PASSWORD from SQL clayser900 3 463 June 21, 2021 at 01:42 PM
Last Post: Dredgen Sun

 Users browsing this thread: 1 Guest(s)