+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: requests-html + Beautifulsoup (/thread-36711.html)
requests-html + Beautifulsoup - klaarnou - Mar-21-2022
Hello,
I am trying to scrape a dynamic web page. The code I use is:
from requests_html import HTMLSession from bs4 import BeautifulSoup session = HTMLSession() url = 'someurlhere' r = session.get(url) r.html.render(keep_page=True, sleep=1,timeout=10,wait=2)Question 1: How do I save the rendered HTML to file?
Question 2: How do I feed the rendered HTML to Beautifulsoup?
Thank you!