"User Experience" vs. "Spam"

I have been busy recently working on making certain sites more accessible. One of the techniques we have employed is to use DIVs and CSS. After catching up on my RSS habit and reading this SEW thread it struck me what a tricky job Search Engines have on their hands.

How can a search engine determine the intent behind how a web site (or application) is presented?

When does "designing for the end user" become "cloaking" in the eyes of the search engines? Danny Sullivan reckons that cloaking is

Quote:
Cloaking is getting a search engine to record content for a URL that is different than what a searcher will ultimately see, often intentionally

This for me is too strict a definition. "A searcher"? We would need to define what we mean by this for a start. People and devices are going to arrive at your site with differing capabilities and operation methods. And that is when we are only talking about the "home page" - any specific URL could be very different when taking into account personalisation, web applications, localisation ..

If a site reorganised its content for, say, visitors who like to read right to left, this breaks the cloaking definition but the intent is to serve the user. If we throw away lots of purely aesthetic content that will only serve to confuse or break the visitors browser, this is also a case where the SE "sees" something different.

I expect the search engines own web applications, particularly those employing "ajax", will look quite different to the search engine spider than we see. Looking at the CSS/JS code is not going to help. Returning spider visits identifying as a different browser will also not help.

Some search engines put a lot of weight on on-page factors, presumably they will want to also put as much emphasis on detecting the cloaked pages also?

I am afraid I do not have a solution. Then again, I am not a PHD, SE PHD or otherwise ;O)

Comments

redirects are OK

IMO reduirect and cloaking are OK if they serve the user !, see the all discussion on http://www.searchenginesmarketingblog.com/2005/09/12/what-is-good-content/ , it is all about giving the user what he wants

Not since SEO was first

Not since SEO was first coined,(I've been bitching about using this misleading term to describe an entire industry for years), has a term been used more to illustrate less. It is not about cloaking. It is about content management.

Again, the engines themsleves try to dictate the use of technology and techniques that make their job easier, (hence more profitable), at the expense of the wishes of the human providing the content. Every webmaster should have the right, maybe even the obligation, to use whatever resources he has at his disposal to meet any legal objective he has.

The problem, (and the heated debates), comes in when we try to force all defintions into one term that was not really appropriate to the technique and purpose in the first place. Content delivery, (the uninformed may choose to refer to this as cloaking because it shows something different to a human than to a bot), can reduce costs, increase conversions, enhance a user's experience and overcome some of the weaknesses in search eigne results. AHHHHH, therein lies the problem and the point of the propoganda and promotion of the misleading term.

If I could wave a magic wand and convert just one misleading term to one that better described it's function in cyber world, (OK, it would have to be 2 because #1 would be to eliminate the acronym SEO to label search marketing services), it would be to convert cloaking in everyone's mind to content delivery.

It can be based on personalization, localization or any other zation that suits the webmaster's fancy, but bottom line, it should not only be accepted, it should be encouraged, discussed and taught. It should not be all right for only search engines and large tech sites to use the exact same technique as a promoter, yet the perception is that the engine is doing a good thing but the promoter has to be "dealt with".

But that is just it

It is not if the site is using "cloaking" it is what is the definition of "bad" cloaking versus "acceptible" cloaking - I visit Google.com in my browser and a little banner containing a text link asks me to change my default search engine. Googles own bot would not see that. Acceptible? What about if it was someguyssite.tld who did the exact same thing but the link only appears for googles bot? Bad? How does the algo work that out?

IP Based Content

MozillaBot

Ack! Don't talk to me about MozillaBot! I wish I could tell what goes on in its tiny little mind - I've got some JS redirection issues going on. Nothing "sneaky" - from MY point of view that is...

How can a search engine

Quote:
How can a search engine determine the intent behind how a web site (or application) is presented?

They don't have to understand the complexities of how the page is going to be rendered they can render the page for themselves and see.

Many moons ago I had a heated discussion at another forum some of you may know (I Know Bob knows it) called the Warrior Forum. The question that was asked was "Can a search engine tell of a redirect done using Flash?"

My answer was, the same as it is here, that they don't need to reverse engineer the code (Flash, JS, CSS etc.) they have browsers available to them and they can use that. To prove my point I quickly knocked up a proof of concept application to see if the URL entered remains the same after the browser has rendered all the code on the page.

If you fancy giving it a try GMail and BBC News are good example sites to test it on.

The same philosophy could be used here to see where content on a page ends up being placed when rendered by a browser. As to whether any of the engines are using it now or not, well only the engineers can say for sure but MozillaBot is a common site in a lot of logs nowadays.