Python Forum
Make a Web crawler without using the recursion method.
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Make a Web crawler without using the recursion method.
#7
Hmmm maybe like this?

#! /usr/bin/python3


def recursive_urls(links):
    if len(links) > 0:
        # remove first element of list:
        current = links.pop(0)
        # instead of the print statement,
        # make your request/do business:
        print(current +'\n')
        # recursively call the method again
        # and pass remaining links:
        recursive_urls(links)


if __name__ == '__main__':
    urls = [
        'http://test1.com',
        'https://test2.com',
        'https://test3.com',
        'https://test4.com'
    ]
    recursive_urls(urls)
EDIT: My apologies I thought you were trying to use the recursion method... not avoid it :)
Reply


Messages In This Thread
RE: Make a Web crawler without using the recursion method. - by rootVIII - Jul-08-2019, 11:39 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Web Crawler help Mr_Mafia 2 1,949 Apr-04-2020, 07:20 PM
Last Post: Mr_Mafia
  Web Crawler help takaa 39 27,752 Apr-26-2019, 12:14 PM
Last Post: stateitreal

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020