Python Forum
Suggestion request for scrapping html table - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Suggestion request for scrapping html table (/thread-35720.html)



Suggestion request for scrapping html table - Vkkindia - Dec-06-2021

Hi,

I am struggling to scrap a html table.
When the date is selected if there is data, the table will be displayed, i am scrapping it using the xpath
table I'd and no issues with this.

But if the selected date has no data, no table will be displayed. Here my code stops and waiting for the table to appear.

How to specify timeout and continue without exceptions.


RE: Suggestion request for scrapping html table - Larz60+ - Dec-06-2021

Please show latest code (working or not).


RE: Suggestion request for scrapping html table - Vkkindia - Dec-06-2021

----snip-----
driver.get("https://website/list.php");

num_rows = 0
num_cols = 0
num_rows = len(driver.find_elements(By.XPATH,'/html/body/form/div[4]/div/table/tbody/tr'))
num_cols = len(driver.find_elements(By.XPATH, '//*[@id="gv_output"]/tbody/tr[2]/td'))
print("Num rows is {0} , num_col is {1}" .format(num_rows, num_cols))
table_output_reading = []
if num_rows > 1:
    for i in range (2,num_rows):
        table_zone = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[3]").text
        table_product = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[4]").text
        table_price = driver.find_element(By.XPATH, "/html/body/form/div[4]/div/table/tbody/tr["+str(i)+"]/td[5]").text
        table_output_reading.append(str(table_zone)+str(table_product)+str(table_price))
        print("table lists is", table_output_reading)

--snip----
This works fine if there is a table listed.

Suppose when i select another date, and on that date if there is no data no html table is displayed.
and this script here waits endlessly for the table
i tried giving num_rows >1 not working
The table id is gv_output

I want this script to run only if table is present

thanks in advance


RE: Suggestion request for scrapping html table - Larz60+ - Dec-06-2021

What do the error traceback look like?