Google sitemap

[JWJ]

For a while now I have been using Google Sitemap. I do the full xml version because with only 40 pages it is not difficult to keep it up-to-date. I have been quite good at maintaining it (though I'm not sure how long the disciplines will last ) and add any new page as it is uploaded and amend the 'last updated' date for any page that gets modified. So far I've been a good little bunny .... HOWEVER ....

I can honestly say I have seen no benefit yet. In fact I presently have a situation that I thought would have been ideal for the sitemap application. I had some pages on my site for a short while and they were referred to from my home page. On Dec 28th I removed those pages, removed the reference from my home page and updated my sitemap to reflect the changes. Today, Googles cache of my homepage is dated Dec 25th with the old references shown, Googles cache of the removed pages is dated Dec 26th, but what is worse is that a search for '404 named anchor' returns me as number 1. This is driving a lot of traffic to pages that don't exist.

Because I don't like 404's and because I remove temporary projects once they have served their purpose, I use htaccess to direct traffic to a "Project Removed" page so it's not a problem. I am now getting quite a few hits on that page from people searching on anything to do with Named Anchors with php and/or 404's.

Am I expecting too much to think that Google should have at least re-indexed my home page? I can understand that the removed pages will probably remain cached forever (or a very long time) but surely the whole purpose behind the new Sitemap strategy is to ensure better and more up-to-date indexing. One recommendation we often see for SEO purposes is frequent updates and changes to our web pages but what's the point in daily changes if the SE's are only going to look once in a blue moon? It seriously has me wondering whether there is any benefit in trying to maintain a sitemap.

[webado]

You may want to exclude access to pages or folders that are only for testing purposes from your robots.txt file and certainly not put them into the sitemap. Of course, that's if you know in advance that those are just temporary files.

I can't say one way or another that I've derived a benefit from the sitemap for older sites. But for my latest site it has indeed made it finally get crawled and indexed. Yet I cannot tell if the sitemap helped or it would have happened just the same.

[alandwalker]

I also use the XML Google Sitemap, updated every time I do a significant update to the site. Google's site tells me it is using the Sitemap every 24 hours or so, and encountering no problems.

Yet still, after 9 months, the Index only shows very old pages that have been subject to .htaccess 301 redirects for several months.

Last Friday there was a flicker of hope - suddenly all my pages appeared in the Index, and the old ones had gone. Until Sunday, when the new ones disappeared and the old ones returned!

Go figure.

[webado]

Obviously a case of seeing a different Google index. There are many of them and they are not synchronized.

[rotarysteve]

I think that google is way too big for their own good, at least to keep webmasters happy. Google is supposed to be going to war with the competition, something about world domination......

I have the xml sitemap and google looks at it each day. Google seems to have crawled a few more pages than before, though like Chris says, that could of happened naturally. The irony is, if I use the links function through google sitemap, it shows a whole different set of pages than what I would see if I use the links function from the google home page. Obviously looking at two different datacenters.

Additionally, google sitemaps is still reporting two http errors for pages it is looking for "that no longer exist" Why google hasn't got the clue that it is not going to find the pages, unless it is giving me a grace period to fix/repair those pages, but those pages haven't existed for 6 months.

I still have pages that haven't been crawled since last June, even though the pages have been completely overhauled and those pages are on my xml sitemap. Very frustrating.

Also I don't have any pages on the sitemap that I do have the meta NOINDEX option. Thankfully, google isn't indexing them.

Is there a benefit? Haven't seen any yet, just more frustration, though I have only had the xml sitemap up for 8 days.

[jonra01]

This is another of those things that make me smile. As webmasters we expect google to cater to us when they are in business to serve the surfers. We also complain that they don't quickly find and add our few pages to their index when there are billions (US), perhaps trillions, of pages on the 'Net. We should be thankful that they get any of our pages indexed.

[China Tea]

I started doing one and gave up because I have so many pages of products and I keep adding or deleting accordingly to match demand and availability.
I was on the 10th URL when I gave up . . . Too lazy to get to 1000th URL and beyond . .

P.S.

I, however, submit my popular products to the G-Base which is I believe more appropriate for commerce sites (with lazy and amateurish webmasters).

[rotarysteve]

Originally Posted by jonra01

This is another of those things that make me smile. As webmasters we expect google to cater to us when they are in business to serve the surfers. We also complain that they don't quickly find and add our few pages to their index when there are billions (US), perhaps trillions, of pages on the 'Net. We should be thankful that they get any of our pages indexed.

To an extent I have to agree with you, but MSN seems to stay on top of everything I have uploaded. Yahoo seems to do well to. What is the difference with google. To big for their own good??

[JWJ]

I agree with you John, and it's not a problem for me, as webmaster, if Google keep sending traffic. My 'Project Removed' page has my header and menu and some visitors have stayed anyway. No, I was looking at this frm the surfers point of view. What good is it to them being directed to pages that are no longer available .... and who do they blame, Google or the site owner? Me thinks the site owner It is excellent, in theory, that Google should give us the facility to 'advise them' when we make our changes, and it 'ought' to enhance the surfing experience right across the 'net, but if Google can't, or won't, handle the information we provide expediently, then we shall loose interest and stop providing it.

[China Tea]

RotarySteve wrote:

MSN seems to stay on top of everything I have uploaded. Yahoo seems to do well too

I agree with you on the above . . . So far, my biggest REFERRER of visits that translate into SALES is MSN . . . and I almost always stay on their front search page as well . . .so I kind of like them for that.

Maybe in time, as I grow up, Google will notice me more. Right now, I am too busy to attract Google and/or strictly adhere to Google's requirements . . .