Python : WebScraping , Beautiful Soup 4

1

I am new to webscraping and python and I wanted to make a code that can scrape Wikipedia and make it quicker . In this you just have to type in the thing you wanna search for example “Clock” So it will got to https://en.wikipedia.org/wiki/Clock and scrape it from there.

So when I run the code I get these errors:

Traceback (most recent call last):
  File "main.py", line 26, in <module>
    wikibot(url)
  File "main.py", line 25, in wikibot
    print(soup,('p')[i].text)
IndexError: string index out of range

And after this I want it to write this data to a text file , which I think can be done with

write()

the same way as

print()

I am following a tutorial and when he showed a demo it ran flawlessly with no errors. Is my code wrong or my packages are erroring out ?

2

Hey there,

so how do you think we could help you?

3

I am new to web-scraping and python and I wanted to make a code that can scrape Wikipedia and make it quicker . In this you just have to type in the thing you wanna search for example “Clock” So it will got to https://en.wikipedia.org/wiki/Clock and scrape it from there. If there is something other than this wrong please tell me.

GitHub

So when I run the code I get these errors:

Traceback (most recent call last):
  File "main.py", line 26, in <module>
    wikibot(url)
  File "main.py", line 25, in wikibot
    print(soup,('p')[i].text)
IndexError: string index out of range

And after this I want it to write this data to a text file , which I think can be done with

write()

the same way as

print()

I am following a tutorial and when he showed a demo it ran flawlessly with no errors. Is my code wrong or my packages are erroring out ?

4

Hello there,

This line looks quite off to me:

print(soup,('p')[i].text)

This line is doing the following:

  1. Printing the value of soup to the console
  2. Printing ('p')[i].text to the console
  • ('p') looks like you are creating a tuple of the string 'p'
  • [i] is indexing into the tuple of the string. So, any number where i > 0 will not work.
  • .text is a method presumably on a string (which is it not)

I assume you want (spoiler):

print(soup('p')[i].text)

or

print(soup('p')[i].text())

I hope this helps