Beautiful Soup 4 Tutorial #4 - Finding The Best GPU Prices

What a video 👏👏

Looks like newegg is a bit fed up with webscraping ... Just got a warning in my doc file

Quick question but why didnt you index by the attributes inside "><"? Would retrieve 3 results which would be cleaner than the ones in the procedure adopted in the video

Hi Tim thank you for this awesome video! Question How would you go on by sending this to a text file.
with open('GPU_prices_newegg.txt', 'w') as file:
file.write(str(item[0]))
file.write(str(f"${item[1]['price']}"))
file.write(str(item[1]['link']))
file.close()
This only gives me the last data it finds not all of it. Thank you

Awesome course ❤

Great content mate. Good explanations and I look forward to more videos like this...👌✊

Great series! I learned a lot today

Thanks for the video. I am getting an attribute error for "price". It says " 'none type' object doest not have an attribute 'find' ". Can I get a response from someone about this please.

how does this .parent part work I dont quit get it and I am unfamiliar with it. Help is much appreciated.

Can I use this in freelance work after I finish your series?

Thank you so much for this series, Tim! You're doing an amazing job!

i have a problem, i have the same code, but in line 31 it says "NoneType object has no attribute 'find'", i tryed averything but it didnt work

Not sure if i make a mistake or the webshop changed something, With search query i get on the site 8 pages, and with the script 100 pages. Also when i do it with an other search query

Adidas is better

Sorry im completely new to data science sort of topics.
What is that italicized f thats in front of the newegg url? He casually threw that in there, Im assuming is something fundamental that i missed.

Thanks! Great series! Enjoyed it a lot

Just finished all 4 videos inthis series. All were great, thank you. I would love to see more advanced videos on this topic from you in the future.

nvm figured it out gpu variable needed changed

Happy to let you know that the cheapest GPU is now $1500 less than it was at the end of this video :)

Hi Tim! You are doing a great job. I am following your videos about web scraping :)

One thing I don't fully understand is: why do you persist in using the attribute '.string' after you have used the '.find' method from the variable 'doc'? Why not just use the '.text' attribute?
I just try to understand why the '.string' attribute is better or why I shouldn't use the '.text' attribute?

My solution for the chapter "Querying multiple pages":
# code above is something you already typed
page_text = doc.find(class_='list-tool-pagination-text')
last_page = int(page_text.text[-1])
print(last_page)

Great tutorial! Keep it up!

Tim, how do can i make the price an integer? Ive tried to figure it out but i get a TypeError saying it must be a string. There must be a way. Can someone help me out?

sorted_items = sorted(items_found.items(), key=Lambda x: x[1]['price'])

Hello everyone, i keep getting an error saying that maybe i need a comma for the Lambda, any tips?

Hi,
I want to find the text of only given tag and not its child tag. Is there a way to get text of HTML tag without getting text of its child tag?

How would you edit your code to only output the top 5 best prices in your list? I'm kind of new to python and Im not sure how you would take the whole library, have it sort it from lowest price to highest price, then ouput the 5 lowest prices and provide the name, price, and link

Great Video! Thanks

Great video. I did notice that you had
for page in... as your iterator with an implied type of int (for counting the pages)

then you modified both the content and type of the variable 'page' immediately afterward when you set that variable to the text of the web page itself. I'm surprised the code continued to run afterwards. Might account for some of the strange results and workarounds you had to do afterwards. These things are insidious sometimes lol

This is what I get when I run

What product do you want to search for? 3080
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-12-8beb69a205b7> in <module>()
9 doc = BeautifulSoup(page, "html.parser")
10
---> 11 page_text = doc.find(class_="list-tool-pagination-text").strong
12 pages = int(str(page_text).split("/")[-2].split(">")[-1][:-1])
13

AttributeError: 'NoneType' object has no attribute 'strong'

I followed this whole video and did all things you did but instead of getting 3080's, I'm getting pre-built pcs with rtx 3080 in them! Any idea why's this happening?

I understand this is a bit of a shot in the dark, but I'm having a problem with some code.
I decided to try to go off on my own a bit in some of the code, but I keep having problems with the BeautifulSoup methods working on anything besides "doc".
For example, price = div.find_all("...") or current_price = price.find_all("...") doesn't work.
I don't know if I'm missing something very basic and easy, but regardless, if anyone has a solution I'd love to hear it.

that was awesome I actuall understood :O
you are a life saver ,thank you very much <3

ha! got banned by newegg while doing trials an error following the course

hey man having trouble with what you did in the command prompt if i could get some help

Is there any reason not to use regex? All the splitting makes the code unmaintainble especially when there's no comments explaining haha.

Hi Tim, can you make a fifth video where you scrape an infinite scroll site/blog?

Tim, watching this tutorial series has been a pleasure. Seeing your workflow and application of python techniques is very informative!

Can anyone recommend a resource that further explores the "rules" and "ethics" of web-scraping in the wild?

Thanks!

Thanks for this series. I have not watched them all yet but I might suggest you start your development by using a browser and SAVING the web page to a local file. Now do all your development by reading your local copy. This way you dont hit the web page dozens of times which might get you noticed or banned. Once you get all your development done - then hit the real URL going forward.

Welldone, you are amazing boy

Nice!

When you imported re I feared you're going to try and use it to find stuff in html without parsing it xD

Why not use print(int(page_text.contents[-1])) to get the last index?

Dataframe is nice use here

Cool!

Pls make a python and programming basics course.

Cool videos TIm i really enjoyed..... you should make some videos about sentiment analysis