Login to website - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Login to website (/thread-20074.html) |
Login to website - kapibara - Jul-26-2019 I want to use Python to login and later download a file. Could someone show how to login based on this code? <div class="p"> <div id="ctl00_cph1_lg1_ch1_d1" class="rc_bg_bl"><div id="ctl00_cph1_lg1_ch1_d2" class="rc_tl_bl"><div id="ctl00_cph1_lg1_ch1_d3" class="rc_tr_bl"><div id="d4" class="rc_c"> <table class="rc_t"> <tr><td><div id="ctl00_cph1_lg1_ch1_divText">MEMBER LOGIN</div></td><td><div id="ctl00_cph1_lg1_ch1_divLink" class="hlink"></div></td><td align="right"><div id="ctl00_cph1_lg1_ch1_divIcon" class="hicon"></div></td></tr> </table> </div></div></div></div> <div class="bbf" style="padding:6px;"> <div id="ctl00_cph1_lg1_pnlLogin" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'ctl00_cph1_lg1_btnLogin')"> <table cellpadding="4" cellspacing="0" width="100%" border="0" style="margin-bottom:10px;"> <tr> <td>Email/Username:</td> <td><input name="ctl00$cph1$lg1$txtEmail" type="text" id="ctl00_cph1_lg1_txtEmail" style="width:160px;" /></td> </tr> <tr> <td>Password:</td> <td><input name="ctl00$cph1$lg1$txtPassword" type="password" id="ctl00_cph1_lg1_txtPassword" style="width:160px;" /></td> </tr> <tr> <td></td> <td><input id="ctl00_cph1_lg1_chkRemember" type="checkbox" name="ctl00$cph1$lg1$chkRemember" /><label for="ctl00_cph1_lg1_chkRemember">Remember Me</label></td> </tr> <tr><td colspan="2"><span id="ctl00_cph1_lg1_lblMessage" style="color:Red;"></span></td></tr> <tr> <td></td> <td><input type="submit" name="ctl00$cph1$lg1$btnLogin" value="Login" id="ctl00_cph1_lg1_btnLogin" class="fancy" with="80px;" style="height:28px;" /></td> </tr> </table> <table cellpadding="4" cellspacing="0" width="100%" border="0"> <tr> <td><a id="ctl00_cph1_lg1_lnkRegister" href="register.aspx">Register</a></td> <td align="center"><a id="ctl00_cph1_lg1_btnResendEmail" href="javascript:__doPostBack('ctl00$cph1$lg1$btnResendEmail','')">Resend Confirmation</a></td> <td align="right"><a id="ctl00_cph1_lg1_lnkForgot" href="support/Forgot.aspx">Forgot</a></td> </tr> </table> </div> </div> </div> RE: Login to website - metulburr - Jul-26-2019 The fact that there is javascript here, the best way around that is Selenium Quote:<div id="ctl00_cph1_lg1_pnlLogin" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'ctl00_cph1_lg1_btnLogin')"> https://selenium-python.readthedocs.io/ https://python-forum.io/Thread-Web-Scraping-part-1 https://python-forum.io/Thread-Web-scraping-part-2 You can search for elements any number of ways after getting that html from whatever site you are getting it from... username = driver.find_element_by_id('ctl00_cph1_lg1_txtEmail') password = driver.find_element_by_id('ctl00_cph1_lg1_txtPassword') username.send_keys('USERNAME') password.send_keys('PASSWORD') submit = driver.find_element_by_id('ctl00_cph1_lg1_btnLogin') submit.click()And you may need a time sleep between things if your internet is slower (if it get that the element is not found error). RE: Login to website - kapibara - Jul-27-2019 Thanks! Now the login works. However, a javascript popup commercial prevents me from getting the file with a simulated click: driver.find_element_by_link_text('The File').click()Error message: Quote:ElementClickInterceptedException: Element <a href="/data/filedownload.aspx?e=myfile.dat"> is not clickable at point (342.1000061035156,592) because another element <div id="cboxOverlay"> obscures it Here's the javascript popup code, but I dont understand how to close the popup. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head><title> </title> <link rel="stylesheet" href="../styles/popup.css" type="text/css" /></head> <body> <form name="aspnetForm" method="post" action="./platinum.aspx" id="aspnetForm"> <div> <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="lKA6V5Ovaauh5Sv9Mzk7WQR1SbFsBnGo/sAgE2UzfWggOGxhSINv1ExdbN2YRs/0Bt70n3vHza06JiSSHpg6p2d4dGFbplZX1X+CD5N9eyo=" /> </div> <div> <input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="F129DEA2" /> </div> <div> <div style="width:500px;height:335px;background-image:url('../images/er.png')"> <div style="width:500px;background:#666;opacity:0.9;filter:alpha(opacity=90)"> <div style="padding-top:10px;width:400px;text-align:right;color:#fff;font-size:26px;font-weight:bold;"> UPGRADE TO PLATINUM </div> <div style="padding-bottom:10px;width:400px;text-align:right;color:#ddd;font-size:14px;font-weight:bold;"> Our most popular subscription level </div> </div> <div style="width:240px;height:215px;background:#ccc;padding: 10px 10px;margin:0px;font-size:16px;font-weight:bold;opacity:0.8;filter:alpha(opacity=80)"> <ul style="margin:0px;"> <li>All</li> <li>None</li> </ul> <input type="submit" value="CLICK HERE" onclick="parent.window.location = '/products/default.aspx'" style="height:40px;font-weight:bold;font-size:18px;color:#fff;background:#369;margin-left:60px;cursor:pointer;opacity:1.0;" /> </div> <div style="width:500px;background:#CBD9F3;opacity:0.9;filter:alpha(opacity=90)"> <div style="padding:10px 0px 10px;width:500px;text-align:center;color:#369;font-size:26px;font-weight:bold;"> HURRY UP! </div> </div> </div> </div> </form> </body> </html> RE: Login to website - metulburr - Jul-27-2019 You have to find the popup box and close it/and or wait for it to end (Whatever you actually do on the website to bypass it). The links i provided earlier will tell you how to search html with selenium. You could try driver.switch_to_alert()and then proceed to close it. It may or may not work. alrt = driver.switch_to_alert() alrt.accept() RE: Login to website - kapibara - Jul-27-2019 It didnt work to accept alert Quote:NoAlertPresentException: No modal dialog is currently open But I found the box: <div id="cadboxClose" class="">close</div>And now it closes: driver.find_element_by_id("cadboxClose").click() Almost there! Now Firefox shows the dialog to start downloading, and it has to be dismissed manually. I added the following code to prevent the dialog from coming up, but probably I give it the wrong content type. How do I find out what content type the downloaded file is? profile = webdriver.FirefoxProfile() profile.set_preference("browser.helperApps.neverAsk.openFile", "text/csv") profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv") profile.set_preference("browser.helperApps.alwaysAsk.force", False); profile.set_preference("browser.download.manager.showWhenStarting",False) profile.set_preference("browser.download.manager.useWindow", False); profile.set_preference("browser.download.manager.focusWhenStarting", False); profile.set_preference("browser.download.manager.alertOnEXEOpen", False); profile.set_preference("browser.download.manager.showAlertOnComplete", False); profile.set_preference("browser.download.manager.closeWhenDone", True); profile.set_preference("pdfjs.disabled", True); profile.set_preference("browser.download.dir", '/home/justme/downloads/') profile.set_preference("browser.download.folderList", 2) |