StatCounter User Forum  
StatCounter Free web tracker and counter

Go Back   StatCounter User Forum > StatCounter.com > Help

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 06-07-2012, 06:54 PM
teezone teezone is offline
Member
 
Join Date: Feb 2006
Posts: 67
Default Yahoo Slurp is a bot

I'm seeing a steady increase in traffic reported from "Yahoo! Slurp 3.0".. via Inktomi & Yahoo.

Shouldn't this bot be filtered by Statcounter..?
  #2  
Old 06-07-2012, 07:10 PM
Arne Arne is offline
Moderator
 
Join Date: Sep 2004
Location: Sweden
Posts: 5,463
Default

Quote:
Originally Posted by teezone View Post
I'm seeing a steady increase in traffic reported from "Yahoo! Slurp 3.0".. via Inktomi & Yahoo.

Shouldn't this bot be filtered by Statcounter..?
It is probably a image enabled bot (Inktomi is), as the Google image bot, and they are not filtered.
  #3  
Old 06-08-2012, 07:38 PM
teezone teezone is offline
Member
 
Join Date: Feb 2006
Posts: 67
Default

Not sure I understand..

I'm not seeing any instances of the Google Image Bot in my stats..
  #4  
Old 06-13-2012, 03:58 AM
webado's Avatar
webado webado is offline
Moderator
 
Join Date: Apr 2004
Location: Montreal, Quebec, Canada
Posts: 27,487
Default

If you see Google at all then it's the image enabled Googlebot.
__________________
Christina
>>Forum Moderator<<

Please do not PM me for support. The forum is here for that.
  #5  
Old 06-14-2012, 03:44 PM
teezone teezone is offline
Member
 
Join Date: Feb 2006
Posts: 67
Default

I should probably be more clear...

I've been using StatCounter for more than 3 years.. in the past 4 weeks, the Inktomi bot is appearing in stats as a regular visitor, which it's not.

By comparison, I see no instance of a Google bot in the same statcounter reports - from my dom logs, I know Google crawls more than 20k+ pages daily.

This is a new issue - is there any plan to filter it as a bot?
  #6  
Old 06-14-2012, 05:33 PM
Arne Arne is offline
Moderator
 
Join Date: Sep 2004
Location: Sweden
Posts: 5,463
Default

Quote:
Originally Posted by teezone View Post
I've been using StatCounter for more than 3 years.. in the past 4 weeks, the Inktomi bot is appearing in stats as a regular visitor, which it's not.
What details can you see about the Inktomi visits?
Regular visitors usually are Javascript and images enabled, and use a regular browser.
Image bots are only image enabled and use an "irregular browser", and can't be logged for several of the detailed stats where Javascript is required.

Because they are image enabled they are tracked as a "visitor", by the small (1x1 pixels) image from the Statcounter code.
The "regular" bots are not images nor Javascript enabled so they can't be tracked by any 3rd party tracker like Statcounter.
No filtering by Statcounter is needed to make their visits invisible.

Quote:
By comparison, I see no instance of a Google bot in the same statcounter reports - from my dom logs, I know Google crawls more than 20k+ pages daily.
So you have not get any visits by the Google Image bot? I can believe that, since it's not often visiting me either. But you will get it eventually.
But even more rarely is the Inktomi visiting me. can't remember when this happened last time.
  #7  
Old 06-14-2012, 07:16 PM
rotarysteve rotarysteve is offline
Master Member
 
Join Date: Aug 2005
Location: Ohio
Posts: 1,878
Default

Got me curious, so I checked out my server logs, and I'm getting crawled from all kinds of bots/spiders. Not seeing "Yahoo! Slurp 3.0".. via Inktomi & Yahoo, but yahoo, googlebot, msnbot, and perhaps a couple dozen more different spiders. Some just look at my robots file and that's it, others give it a good once-over. Interesting.
  #8  
Old 06-15-2012, 02:27 AM
webado's Avatar
webado webado is offline
Moderator
 
Join Date: Apr 2004
Location: Montreal, Quebec, Canada
Posts: 27,487
Default

I see a few Inktomi Slurp, a few Googlebot Image and a few others.

Since most of the images on the site I checked are indexed in Google, Googlebot Image must be crawling or has already crawled sufficiently. Might even be crawling under different guises at times.

Regular Googlebot at times may be image enabled without calling itself Googlebot Image.
__________________
Christina
>>Forum Moderator<<

Please do not PM me for support. The forum is here for that.
  #9  
Old 06-15-2012, 05:29 AM
teezone teezone is offline
Member
 
Join Date: Feb 2006
Posts: 67
Default

Quote:
even more rarely is the Inktomi visiting me. can't remember when this happened last time
I know.. it feels like I woke up in 2004!

Quote:
What details can you see about the Inktomi visits?
There are two new distinct crawlers appearing as follows (with a variety of IP addresses coming in):

IP Address: Inktomi Corporation (74.6.13.121)
Browser: Yahoo! Slurp 3.0
OS: Unknown
Sunnyvale, California, United States
Inktomi Corporation

IP Address: Yahoo! (98.137.72.226)
Browser: Yahoo! Slurp 3.0
OS: Unknown
Sunnyvale, California, United States
Yahoo!

I see Google Image bot in my domlogs.. but not in the Statcounter reports (which is what I would expect). In apache, it's easy to spot "Googlebot-Image/1.0".

Inktomi now represents about 5% of my "visitors" (90% of my 404s!) - the number has been climbing steadily over the past few weeks, which is why I started this thread.

As a longtime user of Statcounter, I'm pretty obsessed about stats on all my sites (tracking 5), and this is only a recent trend. It seems there is something new going on, the question is what?

Thanks!
  #10  
Old 07-12-2012, 12:58 PM
webmaster's Avatar
webmaster webmaster is offline
StatCounter CEO
StatCounter Team
 
Join Date: Mar 2003
Location: Dublin, Ireland
Posts: 3,234
Default

New feature added today.

Block bots from being recorded in your stats.

Check it out here: http://forum.statcounter.com/vb/showthread.php?t=41289
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 03:32 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.