JasonMichael,
The Yahoo Slurp behaviour you describe (only slurping the home page) is quite common. Don't worry though it will eventually go for more as long as your robots.txt file isn't blocking it.

ps - as JdS stated, the REVISIT-AFTER meta tag is useless due to abuse from way back in the day.

Thanks for the tip Div. I was wondering about that revisit Meta tag also. I had a feeling that alot of them are being done away with, except for the Title and Description ones. I read that its important to ahve those Meta tags on your pages.

Heloo.... could you please tell me where can i get more information on robots.txt file.

The robots.txt file tells search engine spiders whether they can spider your website or not. If you do not have one of these files, then the search engine bots will spider all of your site. For more info, check http://www.robotstxt.org/wc/robots.html

Since you're just starting out with using a robots.txt file for your web site, I would definitely suggest you discuss the contents of the file BEFORE you upload it. You don't have to discuss it here but get a second opinion, you must!

Thank you for the link "Div". I'm will check it out.




Thank you for the suggestion. I'll surely do that.

Thanks Rob!

I'm embarrassed to note that I didn't know of that free submission link. I've added our site, but what about their "add to our directory" policy of charging $249 (for commercial sites) to just "consider" your site?!?

I can't help but get angry with their logic there (eventhough I'm realistic enough to know they get bombarded by people requesting links every hour of every day). It's not too hard to remember the days when they were grateful for any relevant links to sites and resources.

On a couple of sites I have noticed that Slurp! takes a very long time to move from just coming by and checking the robots and index file to actually spidering the whole site. If Yahoo wants to be in the same SE league as Google, it'll have to let its spider out more often.