Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Get data from a webpage
#4
Thanks a lot. I am a slow learner, so this will take a while.

Each page displays as a table. Whether or not it is really a table, I can't say right now.

After a quick first read, I'm thinking I need to find <table> </table> and get everything in there for each page.

Does that sound like a reasonable approach?

I can practice on my own little web page first!

First success! Thanks a lot!

Quote:>>> import requests
>>> from bs4 import BeautifulSoup
>>> url = 'http://www.mylittlewebpage.com/18BE/18BEsWeek2.html'
>>> url_get = requests.get(url)
>>> soup = BeautifulSoup(url_get.content, 'html.parser')
>>> print(soup.find('table').text)

A. data projector B. flipchart C. personal statement D. reimburse E. travel expenses

>>>

Now got to figure out how to login to the other page!

So, the login page has a number which must be entered. How to do that from Python?

I logged in manually. The page source looks like this:

Quote:<html>
<head>
<title>管理中心</title>
<meta http-equiv=Content-Type content=text/html;charset=gb2312>
</head>
<frameset rows="64,*" frameborder="NO" border="0" framespacing="0">
<frame src="admin_top.asp" noresize="noresize" frameborder="NO" name="topFrame" scrolling="no" marginwidth="0" marginheight="0" target="main" />
<frameset cols="200,*" id="frame">
<frame src="left.asp" name="leftFrame" noresize="noresize" marginwidth="0" marginheight="0" frameborder="0" scrolling="no" target="main" />
<frame src="right.asp" name="main" marginwidth="0" marginheight="0" frameborder="0" scrolling="auto" target="_self" />
<frame src="UntitledFrame-1"><frame src="UntitledFrame-2"></frameset>
</frameset>
<noframes>
<body></body>
</noframes>
</html>

I can't see any table there!

Attached Files

Thumbnail(s)
   
Reply


Messages In This Thread
Get data from a webpage - by Pedroski55 - Mar-02-2019, 01:14 AM
RE: Get data from a webpage - by Larz60+ - Mar-02-2019, 01:28 AM
RE: Get data from a webpage - by snippsat - Mar-02-2019, 01:29 AM
RE: Get data from a webpage - by Pedroski55 - Mar-02-2019, 03:13 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Extract data from a webpage cycloneseb 5 2,997 Apr-04-2020, 10:17 AM
Last Post: alekson
  flask requests display data from api on webpage with javacript pascale 0 2,831 Oct-25-2018, 08:30 PM
Last Post: pascale
  Not able to fetch data from a webpage sumandas89 3 4,869 Dec-21-2017, 08:30 AM
Last Post: sumandas89

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020