Mar-17-2020, 02:21 PM
Hello All,
I am looking for recommendation for a good start in building my first Python scraper script.
Actually I am not even sure if I should use scrapy, beautiful soup or else.
I have watched a few dozen of tutorial of scrapy, beautiful soup and others, but none were showing what I am really looking for.
I am a big fan of travel, all inclusives deal and many countries.
I am looking at many suppliers but always need to go back every day, sometime every hour to refresh the search.
The idea is to create a python script to automate this search and push the data in a spreadsheet.
The main issue I am facing is the multiple options on each suppliers, drop down for example.
From: , TO: , destination: , Date:, duration (Length) , Hotels: , Star rating... Then I press search...
I was able to create a script for simple search where we have only 1 option with 1 URL which covers the search and push it to .csv file, but this is different with the drop down options and multiple choices/ criteria in the selection.
https://voyagesarabais.com is the place where I look normally, but there are other url where we have to select multiple options prior to get the price and list of location/resorts.
Any recommendations?
Module, package to use for this mini-project.
Thanks,
Sylvain
I am looking for recommendation for a good start in building my first Python scraper script.
Actually I am not even sure if I should use scrapy, beautiful soup or else.
I have watched a few dozen of tutorial of scrapy, beautiful soup and others, but none were showing what I am really looking for.
I am a big fan of travel, all inclusives deal and many countries.
I am looking at many suppliers but always need to go back every day, sometime every hour to refresh the search.
The idea is to create a python script to automate this search and push the data in a spreadsheet.
The main issue I am facing is the multiple options on each suppliers, drop down for example.
From: , TO: , destination: , Date:, duration (Length) , Hotels: , Star rating... Then I press search...
I was able to create a script for simple search where we have only 1 option with 1 URL which covers the search and push it to .csv file, but this is different with the drop down options and multiple choices/ criteria in the selection.
https://voyagesarabais.com is the place where I look normally, but there are other url where we have to select multiple options prior to get the price and list of location/resorts.
Any recommendations?
Module, package to use for this mini-project.
Thanks,
Sylvain