check status proxy address

Often we need to use proxy addresses in our web spiders / crawler. But most of the time the proxies don't work. So I made a little python program to test the proxy IPs. Let's look into the code:

import urllib2, socket

socket.setdefaulttimeout(180)

# read the list of proxy IPs in proxyList
proxyList = ['125.76.226.9:80', '213.55.87.162:6588'] # there are two sample proxy ip

for item in proxyList:
if is_bad_proxy(item):
print "Bad Proxy", item
else
print item, "is working"


def is_bad_proxy(pip):
try:
proxy_handler = urllib2.ProxyHandler({'http': pip})
opener = urllib2.build_opener(proxy_handler)
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib2.install_opener(opener)
req=urllib2.Request('http://www.your-domain.com') # change the url address here
sock=urllib2.urlopen(req)
except urllib2.HTTPError, e:
print 'Error code: ', e.code
return e.code
except Exception, detail:

print "ERROR:", detail
return 1
return 0


Hope the proxy checker will be useful to you!
You can use this list to test your script.

Comments

ronin1770 said…
any python based web crawler - you can suggest???
Tamim Shahriar said…
I have heard that 'harvester' is good. I didn't explore it though.
Mark Stone said…
This comment has been removed by the author.
nikhiljjoshi said…
hey nice post

i was trying to open a webpage with the good proxy but somehow could not manage to do so

i was using the webbrowser function

can u help me pls??
Ant said…
I was looking for a good proxy checker in python. This could easily be edited to run through a file!

Sweet dude, on my blog i post proxys lists :D
Anonymous said…
Man! I love you!!
Thanks a lot for this cool code!
Thanks!!!!!
Unknown said…
Your source enabled me to create a great proxy tester. Thanks.

http://sourceforge.net/apps/mediawiki/proxytest/index.php?title=Main_Page
GamesBook said…
What I need is way to extract the default set of proxies that are already installed/available on a specific machine (which I can then test with your code)... any ideas on that?

Popular posts from this blog

Strip HTML tags using Python

lambda magic to find prime numbers

Convert text to ASCII and ASCII to text - Python code