Do you want to track Googlebot?

JdS · #1 27-Jan-2003, 03:37

I have just 'installed' a very simple Bot Tracking script on my website(s) and wonder if anyone else would like to help me 'beta' test it.

It's just one simple class written in PHP that you can include in all your existing php web pages. No MySQL required.

For the time being it will only track Googlebot but eventually, it should do every important Search Engine bot.

Please PM if you would like to test this script out...

JdS · #2 27-Jan-2003, 10:46

Here's a sample of the first report : GIDTrackbot 1.0 Beta [link removed]

jrobbio · #3 27-Jan-2003, 16:45

Interesting idea you have there JDS. I'm not really using PHP yet but I'm sure others will take you up on this idea.

JdS · #4 27-Jan-2003, 22:18

It's quite useful reports like this - already, I am confused;

Unless I wasn't smoking something again, I think the 64.68.82.xx Googlebot is the freshbot. If I am correct with this assumption, then this is weird behaviour.

jrobbio · #5 28-Jan-2003, 03:11

I'll take your word for that.
Did you make paid submission for inktomi and ask jeeves/teoma or did they just find you?
I've actually found that Teoma to be an awesome search engine to find particular things.
I might be tempted to move to PHP just for this. :-D

JdS · #6 28-Jan-2003, 05:54

Ask Jeeves was spidering my pages even when it was hosted at Tripod, believe it or not. Since October and my folly with session ids in the URL - I lost them too. Now they're back... and I am glad!

Inktomi, I don't know what happened there. They must have found these pages somehow and I KNOW that they're NOT too crazy about session ids too.

Anyway, no session ids appear anywhere on my websites for these few Search Engine bots anymore - and that's the important thing to remember. The rest is just luck...

Anyone know the User Agent for Altavista?

jrobbio · #7 28-Jan-2003, 05:57

Is this what you mean?:
Spider IP addresses
http://www.searchengineworld.com/spiders/spider_ips.htm
If I'm wrong then oh well you can just explain it to me

JdS · #8 28-Jan-2003, 06:01

Hey thanks Rob, that's what I wanted, to confirm that it was indeed 'Scooter'!

JdS · #9 28-Jan-2003, 08:36

1 interesting 'perk' of using this script is apparent already! It helps you find dead internal links!

I will go now to find that link to www.desilva.biz/mb/

and add the trailing slash to category pages e.g. /css/ instead of /css.

jrobbio · #10 28-Jan-2003, 08:52

I'm trying to think of the site that has something like that but if you have that I suppose you don't need it lol
I think it is www.master.com but there is a better one I know that. Oh well

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Having a problem	Chuckles	Computer Hardware Forum	19	13-Sep-2004 12:17
Keeping track of a private variable outside the class	viperv80	C++ Forum	1	03-Dec-2003 17:25
Googlebot has change its ip addresses	jrobbio	Search Engine Optimization Forum	1	06-Apr-2003 06:48
Googlebot and sessions	JdS	Search Engine Optimization Forum	0	26-Nov-2002 03:53
Tracking Googlebot	Div	Search Engine Optimization Forum	13	12-Jul-2002 13:49

Do you want to track Googlebot?

GIDTrackbot: Phase 1 of the reporting page

What is the 'user agent' value for AltaVista?