Have a look at this when I enquired about ways of making my website TopFunWebsites search engine friendly.

http://www.web-app.org/modapp/cgi-bi...lay &num=1155
(sorry I copied the wrong link this one works now)

I've basically followed this. What do you think? Is it going to get me banned in the future or anything like that?

There's something wrong with the link, Rob.

Should be ok now

By 'meta tags' I do not understand what they mean precisely.

If I were you, the only html I would concern myself with is the :
and to a lesser extend, the 'description' meta tag. Everything else may be correct yet irrelevant to a present-day Search Engine spider.

Changing your file to .html will not make much of a difference if the mod will still serve pages like :

Some search engines, including the Googlebot, will index your pages with query strings - this is a fact.

The problem here is that it may choose NOT to spider all the available pages on your website. This is the problem!

The only solution (if you do not have Apache's mod rewrite support on your server) is for you to add a link to a special page (from your homepage; sort of like a site map) that lists the last 20 Latest Topics for example. These links can be in their original forms, i.e. with query strings.

At least Googlebot and Slurp will find and index the 20 pages at each crawl...

You're lucky, your site is still new, so it's best you add that page right away and build it up with links to every existing page on your site.

Is this the kind of thing you are meaning:
Latest 10
On top of that do you think I should make a map page so that it doesn't constantly change and the intended pages may get searched?

Yes, something like a Site Map should work... but a lot of people seem to believe that there is limit to the number of links Googlebot will follow off one page, you'd have to consider that too eventually.

Also, like we've previously observed, some spiders (like the one of Altavista and Teoma / Ask) will keep retrieving 'index.cgi' and that's all it will do.

So does that mean regardless of whether the search engine is tricked into believing that the index.cgi is the index.html , it will still just look at that page.
So does the mod-rewrite actually make the page end in html? Is that what we were aiming for?

yes, for the spiders that don't do the 'query string' links, like Scooter and Ask/Teoma etc.

If you did the '.html' mod, the 'teoma / ask' bot will just keep fetching index.html and nothing more...

Using mod-rewrite, you can make your pages whatever you want; even 'index.rob' and the bots will still fetch the page...

If you're on an Apache web server, you might want to think about the (Apache) environment variable - PATH_INFO.

So, for example one of your urls may look like this:

In the index.cgi file, you may be able to access this environment variable, you can ask the developers of the script if this is indeed possible.

With PHP, the value of this $_SERVER['PATH_INFO'] would be:

then, still in your script, you can use explode() (or something similar in perl) to break this up into variables to feed your script!

I'm really in the dark here. I asked them what you said, but so far not much. I found this http://www.wdvl.com/Authoring/SSI/Intro/cgi.html
is there something I am particularly looking for in the index.cgi or do I really have to ask the creators.
One guy says there is a bunch of $ENV{...} that can be used and referred me to this page:
http://hoohoo.ncsa.uiuc.edu/cgi/env.html

I'm really new to this so please try and explain what it is that is going on and what I am looking for.
Cheers