• Welcome to the world's largest Chinese hacker forum

    Welcome to the world's largest Chinese hacker forum, our forum registration is open! You can now register for technical communication with us, this is a free and open to the world of the BBS, we founded the purpose for the study of network security, please don't release business of black/grey, or on the BBS posts, to seek help hacker if violations, we will permanently frozen your IP and account, thank you for your cooperation. Hacker attack and defense cracking or network Security

    business please click here: Creation Security  From CNHACKTEAM

Proxy scrapping softwares?


Recommended Posts

Not sure if this will get you thousands of proxies but i wrote a wrapper for a certain proxy scraper and i made it so you can very easily sort on geolocation. Get it on your box like so:
 
Code:
wget -O proxies.sh https://gist.githubusercontent.com/NullArray/35e3d894fe896ee1d7d8088a3d8175b7/raw/4756ba136456896512c26ee79f23c247d299aee0/geo-sorter.sh

If you want to use it to gather a large amount of proxies use something like the shell script i wrote for you below.
 
Code:
#!/bin/bash
#____   ____             __                
#\   \ /   /____   _____/  |_  ___________
# \   Y   // __ \_/ ___\   __\/  _ \_  __ \
#  \     /\  ___/\  \___|  | (  <_> )  | \/
#   \___/  \___  >\___  >__|  \____/|__|  
#              \/     \/    
##############################################

# Coloring scheme for notifications
ESC="\x1b["
RESET=$ESC"39;49;00m"
CYAN=$ESC"33;36m"
RED=$ESC"31;01m"
GREEN=$ESC"32;01m"



# Warning
function warning(){
    echo -"\n$RED [!] $1 $RESET\n"
    }

# Green notification
function notification(){
    echo -"\n$GREEN [+] $1 $RESET\n"
    }

# Cyan notification
function notification_b(){
    echo -"\n$CYAN [-] $1 $RESET\n"
    }

function main(){
    notification_b "Enter your country of choice"
    read -"Country: " country
        
    clear && notification "Running scraper on loop. Press CTRL+C to stop"
    while i=0
    do
        python3 fetch.py --country='$country' --max-latency=4 --anonymity='elite|anonymous' --output=proxies.log
        cat proxies.log >> unsorted.log && rm proxies.log
        cat unsorted.log | sort -u >> unique_proxies.txt

        notification_b "Starting delay, 15 minutes remaining"
        sleep 300 && clear
        notification_b "10 Minutes remaining"
        sleep 300 && clear
        notification_b "5 Minutes remaining"
        sleep 300 && clear
        notification "New iteration initiated."
    done
    }

if [[ -"fetch-some-proxies" ]]; then
    cd /home/$USER/fetch-some-proxies/
    main
else
    warning "Proxy fetcher not found."
    printf "Cloning latest version from Github.\n" && sleep 1  
    
    git clone https://github.com/stamparm/fetch-some-proxies.git
    
    notification "Done." && sleep 1
    cd /home/$USER/fetch-some-proxies/
    main
fi

 



The reason i put in a 15 minute delay, is so that the remote resource has some time to reload a new batch. It will automatically sort out duplicates and will loop forever until you manually stop it.

Not sure if the Python script that does the heavy lifting allows you to enter an argument to get just SOCKS5 proxies, but configured as t is it will get SOCKS4 and SOCKS5. Not sure how essential the fact the proxies are SOCKS5 is to your operation.
 
 
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now