Categories
Mastering Development

Web_Scraping Issue with Yahoo Finance Data

My code is saved in the below location for reference:

https://github.com/Ianfm94/Web_Scraping/blob/master/Web_Scraping_2.py

So, I’m fairly new to web-scraping and I wanted to start out with some basic scraping of financial data from Yahoo’s website. Firstly, all I wanted to do was scrape the last closing price for Apple’s stock and eventually I want to scrape it’s closing price for the rest of the month.

Here’s a link to the page I’m looking to scrape the data from:

https://finance.yahoo.com/quote/AAPL/history?p=AAPL&guccounter=1

My issue is that my program runs fine, it just returns an empty list, i.e. no wonderfully scraped price data. I think there is two causes to this error but I’m not sure which/both is/are the root cause.

Anyway, so when I run my code, a new chrome browser window opens up as expected, with the below image included which is obviously not the Yahoo Apple page, just some cookie requirement. I’m wondering is this what is causing my issue? If so, how would I rectify this issue?

Edit: The below might also be wrong as well, however when I printed the soup, it’s scraping the Cookie page rather than the Apple page which means the cookie is causing an issue.

O̶r̶,̶ ̶w̶h̶i̶c̶h̶ ̶I̶ ̶t̶h̶i̶n̶k̶ ̶i̶s̶ ̶m̶o̶r̶e̶ ̶l̶i̶k̶e̶l̶y̶ ̶t̶o̶ ̶b̶e̶ ̶t̶h̶e̶ ̶r̶o̶o̶t̶ ̶c̶a̶u̶s̶e̶ ̶o̶f̶ ̶m̶y̶ ̶i̶s̶s̶u̶e̶,̶ ̶i̶s̶ ̶i̶t̶ ̶t̶h̶i̶s̶ ̶l̶i̶n̶e̶ ̶o̶f̶ ̶m̶y̶ ̶b̶a̶s̶i̶c̶ ̶c̶o̶d̶e̶ ̶w̶h̶i̶c̶h̶ ̶i̶s̶ ̶m̶y̶ ̶c̶u̶r̶r̶e̶n̶t̶ ̶i̶s̶s̶u̶e̶?̶ ̶ ̶

̶c̶l̶o̶s̶e̶_̶p̶r̶i̶c̶e̶_̶2̶ ̶=̶ ̶[̶e̶n̶t̶r̶y̶.̶t̶e̶x̶t̶ ̶f̶o̶r̶ ̶e̶n̶t̶r̶y̶ ̶i̶n̶ ̶s̶o̶u̶p̶.̶f̶i̶n̶d̶_̶a̶l̶l̶(̶’̶s̶p̶a̶n̶’̶,̶ ̶{̶’̶d̶a̶t̶a̶-̶r̶e̶a̶c̶t̶i̶d̶’̶:̶’̶6̶1̶’̶}̶)̶]̶̶ ̶ ̶

W̶h̶e̶r̶e̶ ̶I̶ ̶h̶a̶v̶e̶ ̶d̶e̶r̶i̶v̶e̶d̶ ̶t̶h̶e̶ ̶a̶b̶o̶v̶e̶ ̶l̶i̶n̶e̶ ̶i̶s̶ ̶f̶r̶o̶m̶ ̶t̶h̶e̶ ̶b̶e̶l̶o̶w̶ ̶i̶m̶a̶g̶e̶;̶ ̶T̶h̶i̶s̶ ̶l̶i̶n̶e̶ ̶o̶f̶ ̶c̶o̶d̶e̶ ̶i̶s̶ ̶w̶h̶e̶r̶e̶ ̶I̶’̶m̶ ̶n̶o̶t̶ ̶r̶e̶a̶l̶l̶y̶ ̶c̶e̶r̶t̶a̶i̶n̶ ̶w̶h̶a̶t̶ ̶I̶’̶m̶ ̶d̶o̶i̶n̶g̶ ̶s̶o̶ ̶a̶n̶y̶ ̶h̶e̶l̶p̶ ̶w̶h̶a̶t̶s̶o̶e̶v̶e̶r̶ ̶w̶o̶u̶l̶d̶ ̶b̶e̶ ̶a̶p̶p̶r̶e̶c̶i̶a̶t̶e̶d̶.̶ ̶ ̶[̶!̶[̶Y̶a̶h̶o̶o̶ ̶W̶e̶b̶s̶i̶t̶e̶ ̶C̶o̶d̶e̶]̶[̶2̶]̶]̶[̶2̶]̶

Thanks for your help.

Ian

Leave a Reply

Your email address will not be published. Required fields are marked *