Page 1 of 1

Portal & Bots

Posted: 3. April 2009 09:06
by Sharky
Your Portal Version: 1.0.0RC3
Your phpBB Type: Standard phpBB3
MODs installed: Yes
Your knowledge: Basic Knowledge

What have you done before the problem was there?


What have you already tryed to solve the problem?


Description and Message
Could anyone suggest why search engine bots are not viewing my index.php? They always only search the portal.php page (the home page).

I have a basic robots.txt file, all bot permissions are working fine (I can see all the forums with Google [Bot] permissions), but the crawler just never searches the topics on index.php.

Everything looks fine in Google Webmaster Tools (no errors)

I'm running out of ideas...

Thanks.

Re: Portal & Bots

Posted: 3. April 2009 10:51
by Huor
Did you tried sitemaps?

They are a kind of routing list for crawlers...

Re: Portal & Bots

Posted: 3. April 2009 11:22
by Sharky
Yeah, I added a text sitemap to GWMT, pointing to /index.php. The bots are now crawling through all the topics (yahoo included) I'd say it wasn't the sitemap on WebMaster Tools that fixed it but rather letting it go for a few hours after some permission problems were sorted.