Yahoo Slurp Spider

[adjustablejohnsons]

When you check who's online, you'll see Yahoo's bot is reading lots of threads. Does this slow the site down signficantly?

[mazdaslam]

I just saw the same thing.Like 300 of them:eek: ! Im SCARED:(

[notebooms]

The Yahoo Slurp Spider is what explores the internet, indexing data so that it will show up in Yahoo search.

This is why when you do searches on hot rod related subjects, the HAMB many times appears towards the top of the search results.

Thus, depending on how you look at it-- it's a good and a bad thing. It helps draw traffic to the HAMB.

That said, maybe you shouldnt trust me.... I work for Yahoo, thus i'm part of the machine. However, we're a fine machine born and made in the USA :-)

-scott noteboom

[mazdaslam]

But does it bog down the site???

[adjustablejohnsons]

The Yahoo Slurp Spider is what explores the internet, indexing data so that it will show up in Yahoo search.

This is why when you do searches on hot rod related subjects, the HAMB many times appears towards the top of the search results.

Thus, depending on how you look at it-- it's a good and a bad thing. It helps draw traffic to the HAMB.

That said, maybe you shouldnt trust me.... I work for Yahoo, thus i'm part of the machine. However, we're a fine machine born and made in the USA :-)

-scott noteboom

What I was wondering about is how much it affects the HAMB's bandwidth/throughput.

[notebooms]

Yes, it can affect site performance and bandwith if resources of the site or bandwidth are limited.

It is possible to turn off slurping or limit the number of slurp requests (insert a robot.txt command file in the site that tells Yahoo what not to slurp,) however that could negatively affect the HAMBs search engine results.

Ryan knows this way better than me, but my guess is that HAMB performance demands nowadays have more to do with huge user growth than the Yahoo slurp.

-scott noteboom