How does search engine crawl blog?

clearingthedots · 2012-06-01 21:44:50

How does a search engine crawl a new entry in a blog?

Does the search engine only go to the home page and when it sees a new post does it then "read" the content of the new post from the home page?

Or does the search engine go to the home page, see the new post, and then follow the title link to that post to "read" the content?

I ask because of the different information dotclear displays under meta "description" and "title" between the home page and post page.

In other words, does a search engine ever notice the meta description and meta title of a single post? Or does the search engine only read the home page meta information?

tbtip · 2012-06-03 05:25:01

Hello clearingthedots,

sorry for my bad English, but this text is created using an online translator.

First of all, anticipate, and get it the best sitemap plugin and activate it. If you have done this you can see the path to your sitemap.

Then create the best local a robot.txt file, if you have not already done so, and then write clean the following:

# All robots respond
User-agent: *

# URL to tell the XML Sitemap
Sitemap: http://www.domain.com/xxxxxsitemap.xml >> the path of your Sitemap

Download these via FTP into your root directory. The search engines like to look to whether there is a robots.txt file.

This helps the search engines ever created your pages are better.

Then you should if you've posted a new entry and publish this, in principle, the pre ping ping services. Thus you share with the new entries.

The Google Com Ping service is already established. There are other ping services that could be set up.

The search engines come in not only the home page, but also sub-sites.

I hope I could help a bit.

P.S. My Christmas Dotclear blog has reached the finals of 32 blogs on the Blog EM 2012 in Germany. I can use each vote to get to the next round. Vote for my Dotclear Christmas Blog please on the following page to 06/06/2012 - 23.59 Clock ( German Time ): Blog EM 16. Finale

Now it's Blog vs. Blog. My blog is in the first duel The name of my blog is " weihnachtsblog.weihnachts lichter.de ". It may as well vote for friends, acquaintances and colleagues, but only once per duel. Otherwise there is a warning or disqualification for me.

Dernière modification par tbtip (2012-06-03 05:26:32)

clearingthedots · 2012-06-04 01:46:21

Thanks

clearingthedots · 2012-06-05 02:22:43

Hi tptip,

I have been checking my robot.txt file usage in the statistics and it seems that most robots, at least in my case, ignore the file.

If I add the code to the home.html file like this:

<! # URL to tell the XML Sitemap >
<! Sitemap: http://www.domain.com/xxxxxsitemap.xml >> the path of your Sitemap >

will the search engines pick it up or simply discard it as a comment?

[SiMON] · 2012-06-05 03:15:53

 <link rel="sitemap" type="application/xml" title="Sitemap" href="sitemap.xml" />

Placed in the <head> block of your pages.

Dotclear

Forum Dotclear

Annonce

#1 2012-06-01 21:44:50

How does search engine crawl blog?

#2 2012-06-03 05:25:01

Re : How does search engine crawl blog?

#3 2012-06-04 01:46:21

Re : How does search engine crawl blog?

#4 2012-06-05 02:22:43

Re : How does search engine crawl blog?

#5 2012-06-05 03:15:53

Re : How does search engine crawl blog?

Pied de page des forums

Sites map