Google Sends Warning to Australia

31 comments

Via the SEW Blog comes news that Google has warned Australia that proposed changes to its copyright law could push the nation back to the "pre-Internet era." From the AFP:

"Given the vast size of the Internet, it is impossible for a search engine to contact personally each owner of a web page to determine whether the owner desires its web page to be searched, indexed or cached," Google said Tuesday.

"If such advanced permission was required, the internet would promplty grind to a halt," Google's senior counsel and head of public policy Andrew McLaughlin told the Senate Legal and Constitutional Affairs Committee.

Comments

Knowledge is Power

That should suit the current government perfectly - the last thing they want the electorate to have is information. People who have all the information are harder to manipulate and the current government is big on manipulating public opinion.

For example the country is facing what could be it's most severe drought ever - but don't expect the Prime Minister to admit to that.

Global warming is going to destroy this country - but until a couple of weeks ago the Prime Minister was still telling Australians that it was nothing to worry about.

What, do they have a drama

What, do they have a drama queen department now?

Quote:
"Given the vast size of the Internet, it is impossible for a search engine to contact personally each owner of a web page to determine whether the owner desires its web page to be searched, indexed or cached," Google said Tuesday.

If the bots can read no-cache, they can read yes-cache.

Just how is an opt-in caching policy going to cause the internet to come to a grinding halt?

bunch of whiners.

Any friends from Down Under

Any friends from Down Under have background on exactly what the proposed changes state?

adding teeth to

adding teeth to enforcement.

Copyright Amendment Bill 2006

Quote:
The indictable offence provisions were introduced to assist in the application of the Proceeds of Crime Act 2003 to copyright infringers, and to help prevent copyright infringers from benefiting from the proceeds of crime.

The penalties are, with some limited exceptions:

* Indictable offence: $60,500 or imprisonment for not more than 5 years, or both
* Summary offence: $13,200 or imprisonment for 2 years, or both
* Strict liability offence: $6,600 penalty units.

The amendments also increase the relief available for large scale Internet infringement, for example making pirate copies of films available online. In deciding what level of relief to grant, a court can consider whether the activity was on a commercial scale and the probability of likely further infringements.

FTA

My understanding (as an Australian) is that these laws are all about complying with the provisions of the US-Australian Free Trade Agreement, which is why it's odd that these amendments have been bought up by Google given they are suppose to be similar provisions as per US law. The new laws have been in the press here, there's some good in the them: for the first time it will be legal to rip CD's for personal use, and for that matter even record a TV show (it's previously been illegal, however everyone does it, anyone for VHS and the 80's...all technically illegal here), the bad stuff: mod chips for gaming consoles were previously legal here, but thanks to the FTA changes they will soon be illegal.

Either way, I've never hosted a thing in Australia, and I never will. Online gambling: illegal, adult: illegal, images and video that haven't been classified (in theory a local version of YouTube) illegal. But given the US is heading in the same direction, can anyone recommend a good 3rd country for hosting? :-)

BULLFUCKINGSHIT!

It's called ROBOTS.TXT you festering corporate assholes and NOARCHIVE should be a global ROBOTS.TXT parameter and not some shit we have to add in every page.

Lack of a robots.txt file should mean just that, they don't know about robots so robots should STAY THE HELL OUT!

Good for Australia, does this mean the entitlement mentality to do with whatever you want with any data placed on the internet is over? HURRAY!

Scrapers should run for cover.

I feel better now.

Errr....

Now tell us all how you really feel.

Good for SEOs

Well, all SEOs would naturally opt in and put up a robots.txt and allow Google to index, and those who don't know about it would not, and the population of Australian sites (potential competitors) would go down, and all round SWEET!

PS. I don't think it will happen, and Google will keep everything it sees.

"If such advanced permission

"If such advanced permission was required, the internet would promplty grind to a halt"

I don't think Google=Internet just yet.

..

I agree with IncrediBILL,

Quote:
Lack of a robots.txt file should mean just that, they don't know about robots so robots should STAY THE HELL OUT!

But this will NEVER happen, if google/m$n/yahoo were to do this they would lose 90+% of their index overnight, google and the other search engines are NEVER gonna do that on their own, ever!

>> I don't think

>> I don't think Google=Internet just yet.

No, but Google are pointing out the problem - as it's been expressed here, the law would require express, maybe written permission to do anything with a sites content other than browse. Taking a screenshot could be breaking the law, and I pretty sure indexing any such content would be.

To protecet themselves, I think SE's would be forced to just exclude the entire .au ccTLD form any further activity, although they may be able to retain whatever datat they have at the time of the law coming into effect.

It would make AU effectively a "no-search" zone, which would have serious effects on Australian businesses

The law is written poorly

But I agree with that they say in principle, and permission should be granted electronically via robots.txt, not with written permission, that's the stupid part.

The problem is, the GOOD GUYS honor robots.txt already and they aren't the ones I have trouble with. If any law lets me take other crawlers to court, I'm all for it and can produce almost 10K+ unique IPs tomorrow that have had some crawling activity for a class action ;)

Maybe I should call my congressperson and see about wiping scrapers off the face of the earth real quick...

robots.txt should be inclusionary not exclusionary

I actually agree with Aussy on this.

In the real world (not online) you need permission to copy books, magazines, CD's, DVD's... however online it's reversed... you have to currently opt out of copyright infringement with your robots.txt

It's ass backwards.

Yeah, Allow

G has recognized the Allow directives for the past what, few months to a year. Y announced support for it a few days ago.

Now if we can get all respectful bots on the same page it sure would make all of our lives easier.

Clarification

They're not breaking any legalities by spidering, indexing, and providing snippets in the serps. Sure you can deny them access, but if you put your stuff on a public network, thems the breaks if they get in anyway - nothing wrong there.

What is breaking copyright is the cache. Remove the cache from the serps and I don't think site owners have a leg to stand on with this whole issue.

Not exactly...

They're not breaking any legalities by spidering, indexing, and providing snippets in the serps. Sure you can deny them access, but if you put your stuff on a public network, thems the breaks if they get in anyway - nothing wrong there.

Just because you're on a public network doesn't entitle everyone to abuse your servers resources and bandwidth, that kind of thinking is just wrong, and it's why robots.txt was built in the first place to attempt to control such activities.

Websites were built for PEOPLE, not for every loon with a bot that wants to crawl as thousands of bots that hit my site were pushing my bandwidth thru the ceiling until I cut them off at the knees and brought it back under control. However, how many average webmasters know how to control bandwidth usage and more importantly, why should putting a site online cause someone to have to spend an inordinate amount of time securing it from all the leeches and bottom feeders scraping content, spying on you and every other damn thing under the sun.

Spiders that ignore the robots.txt and/or spider so fast they knock your server offline can be deemed a DoS attack or worse yet, racking up so much bandwidth it costs the website owner excessive bandwidth fees might find themselves on the other end of a lawsuit, small claims at a minimum.

Hurray, Oz, for reversing the tables!

Ok, so I'm not overly optimistic it'll actually happen, but what this at least can do is make more people (including legislators) across the board aware of the bigtime copyright infringements search engines have routinely been monetizing on for years now at the content providers' expense.

There's really no logical reason (except for the search engines' profit making ambitions, obviously) why online content should be treated in any other manner than offline stuff in terms of legitimate usage and adequate compensation.

Let the search engines explicitly pay their due for what they want to make money off, rather than reducing creative content producers who constitute, after all, their very raison d'etre, to ranking-fixated beggars fighting over the breadcrumbs they're graciously being doled out for contaminating their sites with crappy ads ...

But what a loss of information.

Ask 100 random webmasters what a robots.txt file is. Now, consider that every website out there that the developer doesn't know what it is would disappear from the search engine results. Scary. There are a lot of people with good information that would "drop off the face of the internet" because they aren't pros... doctors offices, mom and pop stores, cousin Joey that has a blog about comics, etc.

It may be the best way to do it from a copyright point of view, but it will hurt the net. Furthermore, removal from Google won't stop anyone from stealing your graphics and putting them on their site.

Let's not flatly underestimate all webmasters

After all, they've coped with domain registrations, HTML coding (more or less), FTP uploads etc., none of which is "intuitive" by any standard, so why expect them not to cope with new inclusion standards should they ever be imposed?
And for those who prefer click-and-run solutions, they'll be set up via their respective hosting providers in no time by default. Really, this is a non-issue.

And no, removal from Google won't stop people from stealing graphics and any other content for that matter, but that's not the point here. It's about Google & co. actually paying people for content they're currently exploiting parasitically without offering anything in return while making piles of money with it.

..

Quote:
...Ask 100 random webmasters what a robots.txt file is. Now, consider that every website out there that the developer doesn't know what it is would disappear from the search engine results. Scary.

You can bet your bottom dollar that as soon as these un-educated webmasters start dropping out of google that they will figure out why and fix it real fast.

If you can operate a website, you can figure out how to use Robots.txt, I mean Robots.txt is not like rocket science...

Drop Off the Net

Hey, if a bunch of sites without robots.txt drop off the 'net so be it. You shouldn't be posting sites if you don't know what it takes and making a web page is a heck of a lot harder than making a robots.txt file.

Here's robots.txt for dummies:

User-agent: *
Disallow:

Should just be included in the web hosting account at creation time and you can block things later.

Which brings up an interesting point, why in the heck don't the control panels like Plesk, CPanel and Exim help people with robots.txt and .htaccess files, what a concept!

>> so why expect them not to

>> so why expect them not to cope with new inclusion standards should they ever be imposed?

Because it's not an immediately, directly obvious thing. Your site is created, or not; uploaded, or not. Easy to see, easy to correct (in theory)

So many webmasters haven't the faintest idea how search engines work. Most non-SEO built sites don't have a robots.txt, even an empty one as a way to save bandwidth

>> Let's not flatly underestimate all webmasters

No, let's. It's almost impossible to underestimate the stupidity of the public at large, and that includes most webmasters.

I definitely subscribe to the Scott Adams school on this

If it makes you feel more comfortable, sure,

go ahead and nurture your convictions.

it's not an immediately, directly obvious thing.

Neither are .htaccess, .htpassword, SSIs, cgi-bin stuff or, for that matter, externalized CSS and JavaScript files, etc. So what? Ok, a lot of people probably won't ever get it, but without hard empirical - verifiable - data to back it up I'd be loath to issue estimates on how many sites it may actually affect.

Personally, I'm on record as pointing out that this is the "age of the clueles webmaster". Still I'm inclined to side with lots0 on this one: The Internet being the hectic, hysterical grapevine it is, word will soon be out on what it takes to get indexed in the search engines, no matter how unsophisticated the actual implementation that follows suit may be. (Just look how frames have generally been demoted these days.)

But ok, that's merely another, as yet unverifiable subjective impression.

Generally, I'd opine that professional myopia isn't perhaps the smartest of guides when gauging your market ...

>> Neither are .htaccess,

>> Neither are .htaccess, .htpassword, SSIs, cgi-bin stuff or, for that matter, externalized CSS and JavaScript files, etc

True - and lots of webmasters don't know what .htaccess is either! External CSS and JS ARE imediately obvious, since you get a "monkey see, monkey do" relationship between the action and the effects - you export a JS to a .js, you see a code change.

>> word will soon be out on what it takes to get indexed in the search engines, no matter how unsophisticated the actual implementation that follows suit may be.

Yup, look how well link brokers do selling green pixels

..

?Well most Webmasters may not know how the search engines work, but all Webmasters know how to work the search engines, at least a little.

All the information any Webmaster needs about Robots.txt, .htaccess, .htpassword, SSIs or cgi-bin is only a .025 second search away.

***

What this boils down to, in my opinion, is that google and the other search engines do not want to give up the FREE data no matter how the data was acquired. As most of us here know, data = money and google is a money making machine right now.

I don’t know for sure but I would think that less than 5% of google's current data is opt in, so if the ‘law’ forced google and the others to go opt in only, they would lose most of ‘their data’ (also known as scraped and stolen content) meaning that the search engines would also lose most of their revenue/income.

The essence of the copyright laws world wide (even in Oz), is to prevent the use of someone’s property to make money without the permission or compensation of the property owner.

I think we all have to admit that a big part of googles business model is to take (steal, abscond, appropriate without compensation) everyone’s data and make it searchable. This business model is just not compatible with the existing (and new) copyright laws around the world. I think Oz just made this point very clear to google.

Google may have enough clout and money in the US to get the copyright laws ignored or thrown out here, but as this new law in Oz shows, not everywhere does google have so much clout.

If google were to announce that it was going to an opt in only basis in say 12 months and spend some time and money on educating webmasters on how to opt in over those 12 months, this would not be such a big deal I think... Most Webmasters will opt in if given the choice...

So again the ball is in googles court, are they gonna drop it... you bet they are!

playing devils advocate

>>The essence of the copyright laws world wide (even in Oz), is to prevent the use of someone’s property to make money without the permission or compensation of the property owner.

there is an argument that the owner is compensated by the traffic sent

Of course there are two problems with that - Google (especially) appears to be trying to keep customers on their own site rather than send them through, and if you're below page three are you getting adequate compensation?

Which brings up an

Which brings up an interesting point, why in the heck don't the control panels like Plesk, CPanel and Exim help people with robots.txt and .htaccess files, what a concept!

The ability to edit .htaccess is certainly there and accessible through the respective "File Manager" systems.

>> "File Manager" systems

Brian, you missed the point as using the File Manager is useless for novices that you don't know the syntax of the files or that these files even exist in the first place.

Hell, in theory you could just edit a firewall that way too but they provide a nice place just to type in IP addresses, so on and so forth.

These control panel tools stop short of a couple of WWW things that need to be simplified for the masses, that's all.

..

?

Quote:
there is an argument that the owner is compensated by the traffic sent

A couple of things wrong with that argument.

1. Most sites that are indexed by google do not receive traffic from google. I believe that it is less than 10% of sites indexed by google receive any regular traffic from google.

2. Compensation is only one part of how the copyright laws work. Permission to use the property from the property owner is also a required part of Copyright law.

So what about all those millions and millions of website owners that have their property (content) taken by google and do not receive traffic from google nor have they given their permission to google to take and use their property? As far as I can see, google has taken something of value from these people (their websites)and is using the property they took to make money for themselves without the permission or compensation of the property owners, a clear violation of the copyright laws worldwide.

In my opinion, Googles only real excuse for not going to an opt in model is greed, pure and simple lust for money/data...

Australia talks out both sides of their ass

At the same time they ponder this legislation they're also archiving Australian websites:
http://pandora.nla.gov.au/crawl.html

Think they asked permission?

LOL

Trust politicians to demonstrate what Applied Hypocrisy is whenever they're given half a chance ...

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.