How can we help you today?

Cannot deploy spider using urllib

b

bpecheux

started a topic
about 1 month ago

Hi trying to deploy a spider that uses the urllib python library (python 2.7). added the requirements.txt file to scrapinghub.yml and "urllib" to requirements.txt but I am getting this when I deploy. See below.

bpecheux

Tried to use requests.compat.unquote instead of urllib.unquote. works locally still no luck on scrapinghub...

b

bpecheux

said
about 1 month ago

Answer

okm looks like upgrading to the latest version of scrapy 1.6 and using requests.compat.unquote instead of urllib.unquote made it work.

adriana

said
27 days ago

Good to hear you were able to find a solution on your own. For the future though, please note that while you have an active subscription with us, you are able to open support tickets directly from your dashboard, under Help - Contact Support. We respond to support tickets within one business day. The forum is for customers who don't have an active subscription with us.