Suggestion request for scrapping html table - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Suggestion request for scrapping html table (/thread-35720.html) |
Suggestion request for scrapping html table - Vkkindia - Dec-06-2021 Hi, I am struggling to scrap a html table. When the date is selected if there is data, the table will be displayed, i am scrapping it using the xpath table I'd and no issues with this. But if the selected date has no data, no table will be displayed. Here my code stops and waiting for the table to appear. How to specify timeout and continue without exceptions. RE: Suggestion request for scrapping html table - Larz60+ - Dec-06-2021 Please show latest code (working or not). RE: Suggestion request for scrapping html table - Vkkindia - Dec-06-2021 ----snip----- driver.get("https://website/list.php"); num_rows = 0 num_cols = 0 num_rows = len(driver.find_elements(By.XPATH,'/html/body/form/div[4]/div/table/tbody/tr')) num_cols = len(driver.find_elements(By.XPATH, '//*[@id="gv_output"]/tbody/tr[2]/td')) print("Num rows is {0} , num_col is {1}" .format(num_rows, num_cols)) table_output_reading = [] if num_rows > 1: for i in range (2,num_rows): table_zone = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[3]").text table_product = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[4]").text table_price = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[5]").text table_output_reading.append(str(table_zone)+str(table_product)+str(table_price)) print("table lists is", table_output_reading) --snip----This works fine if there is a table listed. Suppose when i select another date, and on that date if there is no data no html table is displayed. and this script here waits endlessly for the table i tried giving num_rows >1 not working The table id is gv_output I want this script to run only if table is present thanks in advance RE: Suggestion request for scrapping html table - Larz60+ - Dec-06-2021 What do the error traceback look like? |