Python script for scraping a website log

I need a script that will access a website, and iterate through a number of log pages that are organized by date. For each such log type, it should save a file with all of the entries sequentially by date. In addition, it should save one combined file for all of the log types, where the log entries are merged across all of the different log types, and sorted by each entry's datetime. So each entry will appear twice: once in the merged log file, and once in the specific log type file. The script should accept as input the oldest date from which to capture the log entries, as well as a few keywords. The keywords will be used for first running a search per each log type and only scraping logs which contains at least 1 of the keywords.

The website requires authentication, so it may be a good idea to use something like [url removed, login to view] with Firefox where I can log in manually and then fire the script. But I'm open to other implementation approaches.

As part of this project, you'll need to thoroughly test your script. This will include setting up yourself some free account on this website and creating some activity that will generate such logs that you can use for verifying that the script works 100% as it should, before submitting it to me for acceptance testing. The code itself should be of good quality, easily readable, annotated, and making plenty of assertions with descriptive error messages so that I can easily tweak it in the future if the website makes some minor changes to how it works (e.g. if the id of some key DOM element there changes)

In order to help me filter spam bids, please answer the following question at the very top of your bid message:

What do you think about using Marionette client for this project? If you prefer not to use it, what alternative tool/framework/modules would you prefer?

Also, I need this done quite quickly so please indicate how soon you think you can finish it.

Décerné à :

Hey, to answer your question regarding the use of Marionette: I usually just use a python library called requests for projects like this. It includes authentication http://docs.python-requests.org/en/latest/user/authenPlus

166 $ USD en 3 jours

(11 Commentaires)

5.1

8 freelance ont fait une offre moyenne de 175 $ pour ce travail

I can use scrappy or selenium or browser control for this.
Hello, I am kalpataru is a freelance expert developer and have specialized in bot, scraper, automation software, web apps, desktop software, android apps and Plus

I am an expert web scrapper with an experience of more than 3 years in area of developing crawlers and scrappers in Python. I have scrapped websites like Yellowpages.com, Yell.com, Zopnow.com and many more to count.
Plus

I am an expert web scrapper with an experience of more than 3 years in area of developing crawlers and scrappers in Python. I have scrapped websites like Yellowpages.com, Yell.com, Zopnow.com and many more to count.
Plus

Hello
I am an experienced developer and would be able to help you in writing this python program I have attached a bid that is my time and cost includes testing and ensuring the project meets requirements If you are iPlus