New to web scraping

Hi, My 1st foray into web scrapping - using Python 2.7. I’m interested in learning about financial investing, and since I’m a programmer - since the '80s - I’ve arrived here. I’m looking at: “finance dot yahoo dot com / quote / BARC.L / key-statistics?p=BARC.L”

Like to loop thru an array of companies eg. [“BARC.L”, “BP.L”, “BRBY.L”] and for each company loop thru an array of stats eg. ["Price/Book (mrq) ", "Enterprise Value/Revenue ", "Enterprise Value/EBITDA "] and display the value to the right of each string - like so:
Company: BARC.L
price/book (mrq): 44.56
Enterprise Value/Revenue: -3.56
Enterprise Value/ EBITDA: N/A
Or put this into an Excel table.
Sorry if this is not the right place to ask for help or if my question’s to long, but this is what I’m trying to do. I’ve been looking at BeautifulSoup but finding it difficult to understand how to navigate a page and extract info.
Any code snippets, pointers, tutorials much appreciated.
Thanks

Firstly, welcome to the forums.

While we are primarily here to help people with their Free Code Camp progress, we are open to people on other paths, too.

With your current questions, we don’t have enough context to know what you already know or don’t know, so it is impossible to guide you without just telling you the answer (which we won’t do).

It is pretty typical on here for people to share a codepen / repl.it / jsfiddle example of what they have tried so that anyone helping has more of an idea of what help is actually helpful.

Please provide some example of what you’ve tried and I’m sure you’ll get more help.

Happy coding :slight_smile:

I would strongly suggest not using Python 2.7. Python 3 has been out for several years and 2.7 is no longer supported.

I would definitely take this to heart, you should be using Python 3 and not 2.7 since it is deprecated. Many companies are moving away from it.

We use Python 3 and BeautifulSoup for some of the work that we have. I think that their home page will really give you some basic information about how the web scraper works as a whole. It will definitely be beneficial for you to know some HTML to figure out what type of elements you are looking for. But their initial code snippets on the home page should give you a basic understanding.
https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Thanks, will definitely change to Python 3. I have an understanding of html and css. My weakness is Python and BeautifulSoup, so I’ll check out the link.

1 Like

If I have Python 2.7 installed can I just install Python 3?
I would in future only be using Python 3, and will not be doing any development in or upgrading from Python 2.7.
Is it recommended to first remove Python 2.7 from the system?
Thanks for any help

I have both Python 2.7 and Python 3. You can set up how it will work with your path.

The only reason I have both is some applications that I’ve interacted with didn’t make the switch to 3. So I am forced to use 2.7. If you’re not reliant on 2.7 for a program I would just get rid of it.

1 Like

I removed Python and installed the latest version - Python 3.8.
When installing make sure to tick the options to automatically update system variables. Bypasses the problem of Windows Path being truncated at 1024 chars.

Awesome! This is super fantastic. I hope that you’re all set and ready to code with Python 3!