Notes Views on the Web & Search Engine Indexing

HCL Domino Domino Forum
1

Anyone out there have an informed opinion about using Notes views on the Web and getting them spidered and indexed by Search Engines? I have several specific questions about using views, or embedded views, and getting them spidered by search engines.

I am trying to get some web pages, that are done as Notes docs indexed or at least spidered by Google and Yahoo. The general consensus in the Google newsgroups seems to be I need a site map linked right off my home page. That is, the closer to the home page a web page is, the better chance of it getting spidered and perhaps indexed. Question 1: does this seem to be true?

So I am setting up a Notes view to handle this.

My concern is should I offer a view for the site map that is collapsed for ease of use for the average web surfer or should I offer a view that is expanded to help with indexing by Google?

I am almost certainly going to offer the collapsed view because I think it will be easier for visitors to my site to use. Question 2: will this be collapsed view be sufficient for the Google and Yahoo indexing?

Question 3: should I also offer an expanded view just to make sure my pages get indexed?

Also I have a lot of pages in the site. I assume with a very long view, where docs are continued on subsequent pages and you get to them with the “Next” command, that the search engines will follow this link and get to all of my pages. After all they must in order to be able to spider and index really large sites, sites much larger than mine. Question 4: will search engine spiders follow the “Next” links found on expanded views?

Question 5, and the final question: is this all a waste of time? Should I just set up my site with a logical organization, offer a site map for ease of use by visitors, and the search engines will follow all of the links to the pages, assuming my pages are linked somewhere in the site?

Thanks for any thoughts.

2

Subject: Notes Views on the Web & Search Engine Indexing

  1. Unless you embed the view and call it using the page or form’s URL, it will be marked with a robots noindex meta tag. A view by itself is no better than a blank HTML page.

  2. Collapsed Domino views do not contain links, except to themselves. If you are doing this for client convenience, you’ll need to server the view expanded for indexing and use DHTML to control the UI.

  3. Tables do not expand/collapse reliably on the web in any user-agent, so you’ll need to create the view using “Treat contents as HTML” to create an alternate display schema.

  4. Yes, but again, if the view is collapsed and/or is called as a view (the URL is ?openview&something=orother), the buck stops there.

  5. No, but it will take more time than simply throwing a view on the web. Think carefully about how you create and arrange the links. Think people first, then robots. If this is a general-audience site, then a bog-standard Domino view is not going to be the ideal navigation tool. Once human navigation is handled, the robot thing will take care of itself.

By the way, if it’s Google you’re wooried about, then you really need to worry more about external links to your content than links inside your site. If no-one thinks your content is worth linking to, then you’re going to wind up on page 1189 of the most specific search you can come up with. Content is king, followed by usability. If those are right, then all else will fall more-or-less naturally into place.

3

Subject: RE: Notes Views on the Web & Search Engine Indexing

Thanks for your post. It was an eye opener. I did not realize that views carried a noindex code with them. No wonder many of my web pages are no being indexed.

Can I ask you how do I solve this?

I think I know the answer, but I am not sure how to implement it.

Let me explain. On one site I have my docs grouped into several levels. The first level has only 4 groupings and thus four pages. Here are the links to two of those pages:

http://www.improvingyourhome.com/homestore.nsf/webpagesiyh/homelinks.html

http://www.improvingyourhome.com/homestore.nsf/webpagesiyh/homeproducts.html

These are Notes doc named with the .html extension as part of the name, and they have embedded views on them. These pages are in the Google index. They do not have a noindex on them by the way.

Now on these pages are links to a second level of grouping which then point to the third level of actual content pages. Here are two links to my second level pages from one of the above pages:

http://www.improvingyourhome.com/homestore.nsf/webpagesiyh/HomeInfoDisplay.html!opendocument&Category=LinkHomeAir%20Conditioners&Title=Resources%20about:%20Air%20Conditioners

http://www.improvingyourhome.com/homestore.nsf/webpagesiyh/HomeInfoDisplay.html!opendocument&Category=LinkHomeEnergy%20Conservation&Title=Resources%20about:%20Energy%20Conservation

These second level of pages are embedded single category views which call up all of the actual content pages on a topic, like air conditioning and energy conservation. Neither of these are in the Google index. No wonder my third level pages are not in Google.

Why doesn’t Google index these second level pages? I had been told Goggle is smart enough or tolerant enough to allow for parameter passing.

Google must allow parameter passing, but why not on my site? Is it because the page name, i.e. HomeInfoDisplay.html, is the same even though I pass different parameters to it?

Does Google see my two second level pages as the same page and not index it?

Now oddly Google did index this page and others named like it:

http://www.improvingyourhome.com/homestore.nsf/0/5cb6198ecbc57b7b85256db10063b024?OpenDocument

I thought this format with the “?” was a no no. This is a page from an older view.

And where I am stuck. The obvious solution is to name all of my pages like the one that works. But I need two levels of organization. The first is a major gourping with minor subcategories. I do not see how to generate the two levels of views without using embedded views with single categories and get Google to index them.

Do you having any tips on this?

Thanks,

Al

4

Subject: RE: Notes Views on the Web & Search Engine Indexing

Hi- I am experiencing a similar problem to your last example where google finds a link with /0/ in it. Did you find out any more information? Thanks!

5

Subject: RE: Notes Views on the Web & Search Engine Indexing

Google doesn’t find “/0/” links, it creates them from the links it does find (it saves them quite a bit of space on their machines to use /0/, which works, rather than the view’s name or UNID).

6

Subject: UPDATE: Notes Views on the Web & Search Engine Indexing

Is there a proven way to prevent it from happening? Let me be more specific-

ex. search for “292 Law Firms in an Independent Survey” in Google. You will be returned to my site’s content entry portal version of the page because it has the /0/ in the URL. If you go to the main site, there are many ways of getting to this article - depending where you are coming from. ex. http://www.pillsburylaw.com/go/News.nsf/news-publications-all/5C1094284ADB25B785256FD2007260FC?OpenDocument or http://www.pillsburylaw.com/go/News.nsf/aboutus-awards&rankings-all/5C1094284ADB25B785256FD2007260FC?OpenDocument

I have tried using a default view and not setting a default view - both to no avail. Any ideas?

Thanks again!

7

Subject: RE: UPDATE: Notes Views on the Web & Search Engine Indexing

The biggest problem with the “/0/” links is that they don’t allow form formulas to override the Form field on the document. The solution to that is to ensure that the public display form name is stored in the Form field, and that form formulas are used to open the document for editing, etc.

As for navigation, I’d use a hidden CFD field that tries to compute to @ViewTitle. If the document is opened in the normal course of navigating your site, the correct view will be available for building nav links. If @ViewTitle doesn’t work, the visitor likely came from an external link, so you’d want to throw in an appropriate view name.

8

Subject: RE: UPDATE: Notes Views on the Web & Search Engine Indexing

Thanks for your response. It seems like the developer has to code for this and there is nothing that can prevent the search engines from using the “/0/”.

In our previous site, I had a field on the form that computed its path (complete with view titles) for our internal search feature. Then had a formula that checked to see if the user was logged in - if anonymous, would use the proper view. I will go ahead and use one of these methods.