Tears are falling, hearts are breaking
How we need to hear from god
Youve been promised, weve been waiting
Welcome holy child
Welcome holy child

Hope that you dont mind our manger
How I wish we would have known
But long awaited holy stranger
Make yourself at home
Please make yourself at home

Bring your peace into our violence
Bid our hungry souls be filled
World now breaking heavens silence
Welcome to our world
Welcome to our world

Fragile finger sent to heal us
Tender brow prepared for thorn
Tiny heart whose blood will save us
Unto us is born
Unto us is born

So wrap our injured flesh around you
Breathe our air and walk our sod
Rob our sin and make us holy
Perfect son of god
Perfect son of god
Welcome to our world

I expected to get a gift from Google as I do every year. One year I received the coolest mood radio. The next year I received a USB kit with the Google logo on everything. Last year I received a Google digital frame.

Each gift made me feel I was part of the Google family, enjoying the relationship and showing it off at family gatherings as the “cool old-lady geek with a friend in Google”.

This year, I thought I was perhaps inadvertently left off the list. The expected package never arrived with the Google marking on the box. I was disappointed.

But today, it doesn’t seem to matter.

Google has found my pages and put them back in the index!

So I don’t really need a fuzzy Google hat to cover up the bald head from pulling my hair as I searched for missing pages.

Thank you: Chris, Carly, Matt Cutts, Dave L, Brent, Dylan, Logician, and the many others who emailed, commented and helped me with tidbits here and there!

And thank you Google. Being re-indexed is better than 2 gb memory storage card. (But please put me back on the list for next year!)

With the help of some helpful SEO folks and assorted online webmaster friends, I might be making progress.

I’ve been working on meta tags

I’ve been tweaking css to include h one, h two, h three tags in some important page headings/subheadings.

I’ve added more things to the robots.txt.

I’ve prayed.

A lot.

Yesterday I had 12,700 pages showing in the index in google.

This morning it was 13,100 (a miracle to break the 13,000 barrier)

and this evening, 13,300.

<Edited to add: 13,600 Saturday morning>

<Edited again: 14,000 Saturday evening.>

I haven’t seen numbers over 13,000 in a long time.

Perhaps this is progress! Or perhaps this is a fluke. Time will tell and I will remain hopeful that my efforts have been rewarded, even if slightly.

For my quest to discover the answer to my mystery, another question for Matt Cutts who is busy answering questions at PubCon.

Matt, you said your folks found “no definitive cause for having a small number of pages indexed [at hystersisters.com],” and I was wondering how I can accurately check for how many pages have been indexed in the main index (vs. the supplemental index) to test to see if the situation improves from the things we are doing to try to improve our site structure.

So I’ve jam-packed a great deal of work into the past few days in my quest to find the problem with my website’s Google indexing.

I have made progress thanks to Matt Cutts, Brent, Chris, Logician, Dylan, g1smd, and many others who stood by and cheered me on with helpful tidbits filling my SEO challenged fabulous fifty brain.

I don’t know if I have a complete solution yet but I feel very much closer to the end of the problem than before. So this is good news!

What I discovered on my end:

1. My thread titles needed to be tidied up.
2. My htaccess needs to be fixed to make sure my domain resolves at http://www.hystersisters.com./vb2/ and not http://www.hystersisters.com/vb2/index.php or other assorted similar urls. Indexing both constitutes duplicate content. Not something I did intentionally, in fact I thought I resolved the issue a year or more ago when system admin created a 301 rewrite in htaccess for the domain. But apparently it also needs to be applied to the forum directory, too.
3. My meta descriptions were pulling the same description from the built in templates. Fixed to make sure they were unique to each thread.
4. I double checked my robots.txt and found something more to add.
5. I resubmitted my sitemap, removing more content from the config level.
6. More memory has been alloted to php so as not to serve white pages when the google bot arrives on top of our already busy site.

My main question which sent me on this tizzified quest: Why isn’t Googlebot visiting my site? Was I penalized? Where have all the indexed pages gone? Long time passing. Where have all the indexed pages gone. Long time ago. (Apologies to Joan Baez)

We created a page to log Googlebot. To log the threads it indexes and to note the time it visits. I figured it would be a place to hang out and watch for any activity since I wasn’t seeing the googlebot on “who’s online”.

The page was created. A miracle occurred! The bot of Google was in the house!

We timestamped the log from Dec 1: Looking at my log page I see a list of threads and this message at the top:

Since Dec.1.2007 Google bot visited 25,417 threads!

Ah ha! Relief! The answer to my biggest mystery. I was asking the wrong question. The question isn’t “why hasn’t the bot been visiting?” rather, “why isn’t there evidence of the google’s bot work in the site: search function?”

Are my threads in Google’s index or not?

I do a simple site: search to find that 13,000 threads result.

But. BUT. BUT!!!!

If I do an advanced search, looking for keyword: “hystersisters com” in domain “hystersisters.com” I find 415,000 threads indexed!

It is possible I’ve been pulling out my hair for nothing much except a broken function on Google? Or not. It could be (based on my traffic stats) that some results are missing from SERP because this is where it all started. Searching for specific discussions I knew should be in the search index.

Dear Google-sweetie,
When you send my adsense gift for the year, will you send something wonderful? My favorite gift was the mood/light/radio from 2004. The USB kit wasn’t used much because my Mac OSX wasn’t happy compatible with some of it, but is so nifty I took it for “show and tell” at the family Christmas in 2005. I was deemed “cool” by the teen geeks in the crowd. Last year’s 2006 digital frame is still in the box because sadly, my Mac OSX couldn’t find a way to connect to it.

So this year I have a request. Please send a fuzzy hat. Put a Google logo on it. I’ll use it to cover my bald head from all the hair pulling I did this week making sure you hadn’t broken up with me.

Sincerely yours,

Kathy