Have an email sent to you when Google crawls your PHP site

jrobbio · #1 24-Feb-2003, 11:05

I got this from http://www.phphacks.com/googlebot.php

Auto Email on Googlebot Detected Crawling Page
Simple script that you can insert in a .php page that will email you when Google is indexing your site. You will need to change the values in the script for your own site and contact details. Simply cut and paste from the following box. Dont forget the opening and closing < ? PHP and ? > tags (without the spaces)

This script is completely free for you to use and modify however you see fit, but if you make any cool changes, please share them with us :)

Simple Version:

PHP Code:


if(eregi("googlebot",$HTTP_USER_AGENT))
 {
 mail("you@youremail.com", "Googlebot detected on yourdomainname.com", "Google has crawled yourdomainname.com");
 }

Advanced Version:

This is a much better version that will automatically fill in the Domain, the actual Page (including any query strings), as well as tell you the Date and Time the page was crawled. Very useful if you want to add this script to many pages.

PHP Code:


if(eregi("googlebot",$HTTP_USER_AGENT))
    {
    if ($QUERY_STRING != "")
        {$url = "http://".$SERVER_NAME.$PHP_SELF.'?'.$QUERY_STRING;}
    else
        {$url = "http://".$SERVER_NAME.$PHP_SELF;}
    $today = date("F j, Y, g:i a");
    mail("you@youremail.com", "Googlebot detected on http://$SERVER_NAME", "$today - Google crawled $url");
    }

Testing:

If you want to test the above script to see if it works, then change the word googlebot to mozilla (will work for 90% of browsers) and then load your page - you should get an email... If you do - it works.. Switch it back to googlebot and your away :)

JdS · #2 25-Feb-2003, 06:16

For a small website this is probably the most motivating thing you can set up for yourself. There's no feeling quite like it, receiving the first email confirming that Googlebot has been around your website...

However, on this site alone (and it's not even big... yet) I have had >5,000 hits from Googlebot alone, to-date (this month). Are you certain you want that many emails in 24-25 days (even a meagre 10% worth of emails is going to put you off very quickly).

It's perhaps wiser to extend that little script to 'write' the data to a compressed text file, so you can read, view or download the file at your leisure.

jrobbio · #3 25-Feb-2003, 07:58

What is the php command for that?

JdS · #4 26-Feb-2003, 07:15

It's not as easy as inserting a command, it's more of an extension to the script you submitted in your post above.

First, a few modifications to the original script you pasted off that site.

PHP Code:


<?php

// DON'T use eregi() or preg_whatever() unless you REALLY have to
// DON'T use double quotes unless you REALLY have to

if( stristr($_SERVER['HTTP_USER_AGENT'], 'googlebot') ):
  $ts = date( 'd/M/Y H:i:s' ); // the timestamp
  $data = $_SERVER['REMOTE_ADDR'].'|'.$ts.'|'.$_SERVER['REQUEST_URI'].'|'
         .$_SERVER['REQUEST_METHOD'].'|'.$_SERVER['SERVER_PROTOCOL']
         .'|'.$_SERVER['HTTP_USER_AGENT']."\n"; // the necessary data
  // write it to the logfile
  $fp = gzopen( '/path/to/your/logfile.gz', 'a' );
  gzwrite( $fp, $data );
  gzclose( $fp );
endif;

?>

Note: this requires you to ensure that you have the correct permissions to read and write the logfile.gz.

I have no way of telling if this script is bug-free since I have not tested it out - I am sure you'll keep me informed

jrobbio · #5 26-Feb-2003, 15:14

Ok final questions before I try it somewhere, is this logfile something that I can make myself in a location I choose and is it the unix route from say e.g. /home ?

JdS · #6 26-Feb-2003, 22:38

Yes, it's that simple; e.g. path to your logifle.gz could actually be:

Code:

/home/jrobbio/public_html/logs/logfile.gz

If you want to extend the little script further, try to figure out the following suggestions and PHP commands:

Use the current YYYY and MM to name a logfile e.g. log200302.gz. You use php function date( 'Ym' ) to get it.
If the file above doesn't exists, create it with touch() and chmod it with chmod()

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Check your keyword position using Google API	jrobbio	Search Engine Optimization Forum	5	20-Jul-2006 15:29
PHP script to show last referrer url and top referrer to your site	jrobbio	MySQL / PHP Forum	6	22-May-2006 01:06
Search Engine Positioning 101 and 201 "How To" Tips...	000	Search Engine Optimization Forum	0	29-May-2003 10:34
Poodle Predictor - See your site like Google does	jrobbio	Advertising & Affiliates Forum	3	28-Apr-2003 16:59
Site Statistics in PHP	BigZ	Web Design Forum	5	10-May-2002 08:49

Have an email sent to you when Google crawls your PHP site

Updating a compressed logfile that tracks Googlebot