Proxies can be necessary when web scraping because some websites restrict the number of page downloads from each user. With proxies it looks like your requests come from multiple users so the chance of being blocked is reduced.

Most people seem to first try collecting their proxies from the various free lists such as this one and then get frustrated because the proxies stop working.
If this is more than a hobby then it would be a better use of your time to rent your proxies from a provider like packetflip, USA proxies, or proxybonanza.
These free lists are not reliable because so many people use them.

Each proxy will have the format login:password@IP:port
The login details and port are optional. Here are some examples:

The above script will download content through a random proxy from the given list. Here is a standalone version:

importurllib2importgzipimportrandomimportStringIOdeffetch(url,data=None,proxies=None,user_agent='Mozilla/5.0'):"""Download the content at this url and return the content
"""opener=urllib2.build_opener()ifproxies:# download through a random proxy from the list proxy=random.choice(proxies)ifurl.lower().startswith('https://'):opener.add_handler(urllib2.ProxyHandler({'https':proxy}))else:opener.add_handler(urllib2.ProxyHandler({'http':proxy}))# submit these headers with the request headers={'User-agent':user_agent,'Accept-encoding':'gzip','Referer':url}ifisinstance(data,dict):# need to post this data data=urllib.urlencode(data)try:response=opener.open(urllib2.Request(url,data,headers))content=response.read()ifresponse.headers.get('content-encoding')=='gzip':# data came back gzip-compressed so decompress it content=gzip.GzipFile(fileobj=StringIO.StringIO(content)).read()exceptException,e:# so many kinds of errors are possible here so just catch them all print'Error: %s %s'%(url,e)content=Nonereturncontent