Do Search Engines Need To Be Regulated?
from the speak-freely,-search-freely dept
We’ve seen way too many lawsuits from people who get upset that Google doesn’t rank them highly enough (or that Google has erased them from its index for gaming the search rankings). They often seem to think that it’s a natural right that Google must rank them and must rank them highly. Of course, some of us feel that Google is a private company, and has the right to rank sites however it wants. If those rankings aren’t very good, then that simply represents an opportunity for the other search engines to provide a better solution and steal away users. Law professor Eric Goldman, who tends to agree with us on that point, now points us to a new academic paper suggesting the opposite: that search engines should be regulated as their results represent a form of free speech. Specifically, the paper argues four key points should be regulated into place:
- Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.
- Search engines must reveal the basis of their ranking methodology and must continue to use the methodology they have made public
- Search engines cannot manipulate search results except if there is a clear example of abuse that needs to be changed
- Search engines should be required to clearly state which results are paid and which are organic
The first three points seem to be based on the idea that somehow some sites’ right to free speech is somehow blocked by search engines not ranking them “properly.” While the author admits that free speech issues revolve around gov’t, not private company, censorship, she presents a rather complex argument that the internet is a platform for free speech and any intermediaries that get in the way of a listener hearing any particular opinion is thus interfering with free speech. It seems like quite a stretch. In the meantime, we’ve yet to see why this is a problem that requires legal help. Are there really important sites that have been shut out of the public realm by bad search engine rankings? These days, if a search engine really blocked out such a site, it would almost immediately allow the site owners to get publicity for being “blocked” and generate much more attention. However, we’ve yet to hear any credible claims of any search engine blocking any legitimate site, other than for trying to manipulate the search engine, for which it seems perfectly reasonable to get banned. As for requiring search engines to clearly mark paid and organic search, it’s unclear why this isn’t already covered by the FTC concerning deceptive advertising. So, at this point, it seems hard to justify the need for additional regulation.
Filed Under: regulation, search bias, search engines
Companies: google, yahoo
Comments on “Do Search Engines Need To Be Regulated?”
it seems that in the US is different than the rest of the world for what we see on search – go ask microsoft.
go here -> http://www.noooxml.org/forum/t-22697/what-happened-with-the-ranking-of-no-ooxml-on-live-com
and you will notice each country has a very different ranking for the most accurate wording on search engines….its as if microsoft is regulating its own search engine in a rather unfair fashion, huh?
So unfortunately, I don’t know where I could stand on that….minimal amount of regulation would be desired but the result would be an unintended oversight or maximum regulation (we know how stupid politicians are), for sure. Not to mention the concerns of free speech and other things…if the government can just remove search results a lot of information could be disappearing from search engines that Bush & co don’t want there.
Search is free speach?
1. Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.
I agree with this on the principle no one, especially corporations, should censor the internet.
2. Search engines must reveal the basis of their ranking methodology and must continue to use the methodology they have made public.
Clearly the author has no idea how internet search works. The main reason Google has been successful is because it serves up relevant search results very quickly. They do this using proprietary indexing and ranking technology. No company serving the best interests of its shareholders gives away their trade secrets and the primary means of their competitive advantage.
3. Search engines cannot manipulate search results except if there is a clear example of abuse that needs to be changed.
See # 2
4. Search engines should be required to clearly state which results are paid and which are organic
Hmmm, Paid links appear on the right under the title ‘Sponsored Links’. Not too hard to figure out which results are paid and which are ‘organic’.
Re: Search is free speach?
You can like mike all you want (I agree, I like Mike as well), but some of what you said there makes me think you stopped using a computer 10 years ago.
1. I do agree with the first point, unless there is a very VALID reason to do so.
2. Everyone screams that MS should show there code, why not search engines, which now-a-days have just as big as an impact on IT as MS.
3. I can’t believe you think there is no code for indexing and ranking that doesn’t manipulate the data. I would love to live in your world of the blind.
4. If that is the only people you think paid for links, I refer to No. 3, the world sounds rosey and honest.
I am done.
Re: Search is free speach?
I like Mike wrote:
1. Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.
I agree with this on the principle no one, especially corporations, should censor the internet.
Not showing up on a search engine’s results isn’t censoring anything. If you know the URL or IP address of the website, you can still view the page(s) without a search engine.
Why don’t you see websites written in chinese/swahili/etc in your search results very often? There are sites written in those languages discussing the same topics you searched for, but the search engine obviously does some localization. Do you want half of the search results you see to be languages you can’t read? I know I don’t, but then again I’m also not claiming that excluding a site from search results is somehow construed as censorship.
Re: Re: Search is free speach?
Do you want half of the search results you see to be languages you can’t read?
No, but having the option to remove those from the list of results should be my choice, not be made for me.
I have a real problem with this:
Not showing up on a search engine’s results isn’t censoring anything. If you know the URL or IP address of the website, you can still view the page(s) without a search engine.
The vast majority of searches is to find websites/-pages that contain whatever you put in for your search criteria, not to find the website you sort of, more or less, remember the URL or IP address of.
So yes, not showing up on a search engine’s results will for the most part result in people not finding your page/site. Because how would you ever remember the URL or IP address if you can’t find the site through a search engine the first time around???
Re: Re: Re: Search is free speach?
Pay and advertise, stupid
Re: Search is free speach?
> 1. Search engines should not be allowed to
> remove any sites from a search index unless
> required to remove it by law.
> I agree with this on the principle no one,
> especially corporations, should censor the
> internet.
No one’s censoring the internet. If I create a search engine that doesn’t list your web site, I haven’t censored your web site. Your web site is still there, on the internet, untouched and uncensored. The fact that someone can’t find it by using my search engine doesn’t change that fact.
Re: Re: Search is free speach?
so if a library would keep certain books in some stuffy back room without lighting, a sign on the door that says “maintenance” and not include those books in their index system, would that be equally ok with you? after all, they’re not censoring the books, the books are there, untouched and uncensored…
Re: Re: Re: Search is free speach?
A library is a federal establishment, so the comparison is inaccurate. The equivalent would actually be that a bookstore has taken the book and put it in a backroom. Then, if you wanted it, you’d have to really dig around for it, or go to another shop that made the book easier to find.
Re: Re: Re: Search is free speach?
so if a library would keep certain books in some stuffy back room without lighting, a sign on the door that says “maintenance” and not include those books in their index system, would that be equally ok with you? after all, they’re not censoring the books, the books are there, untouched and uncensored…
Your library example is flawed.
Google is a private bookstore with the right to advertise or carry any book at their discretion. Most libraries outside of congregational collections do not and will not stock every book out there – undesirable and unused materials will be filed away in storage or disposed of entirely.
Telling Google what to link to and how strongly is like telling Mike what story he should write about and his opinion on it.. a suggestion with no obligation to do act or even listen.
Re: Re: Re: Search is free speach?
> so if a library would keep certain books in some stuffy back
> room without lighting, a sign on the door that says “maintenance”
> and not include those books in their index system, would that
> be equally ok with you?
Sure. Especially if the library was a private organization, they have every right to index their books however they want. Besides, that analogy fails on its face because while libraries have physical control of the books, search engines do not. They have no acutal control over the “books” (websites) they index.
It May Be Your Free Speech .....
but I have a right to not listen.
A search engine is a convenience, not a requirement for free speech. If I have a political website that I want to attract visitors to, there are many options to do that. Some of the options involve money, as in buying advertising. But, that’s the way it is. Speech is free, but no one should be compelled to listen. Nor should any third party (search engine) be required to assist you exercise your right.
search for free speech
“Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.”
Whose law? The law of the nation where the search engine is based ? The law in the country of the person who is claiming “censorship”. The law of the country with the most nukes?
It makes very little sense to me.
Free Speech, Free Market. Government should keep their nose out of it.
If a search engine starts to suck enough, people will quit using it and go to another (Yahoo’s a good example of that).
Who the hell uses Microsoft’s search engine anyway? lol
If Google gets sued because someone’s page didn’t come up where they want it to – too bad. Google should just exclude them altogether. People are proposing a bunch of close to unenforceable laws.
I’ve noticed Google is starting to suck more and more too – trying to regulate what comes up where. Time for a new competitor, IMO. I don’t think Google’s so ‘innocent’ anymore when you search for stuff, I’m sure there’s some money moving behind the scenes too.
Last thing we want or need is the Government telling search engines what they can and can’t put up. If you like that mentality, there’s already a country serving your needs… China – just use their search engine!
Absurd.
It is a private company offering searching services. They can sort, filter, and regulate their results however they want. If people feel that one is over regulating or under regulating, they will move to another. Nature will take it’s course and kill off those which the public do not like.
A Right to Be Heard?
In my opinion the effect of any regulation regarding how a search engine returns results would put that search engine in the position of being required to return search results that are acceptable to whatever party happens to want their viewpoint or product represented or face action under the legislation. Should any entity (individual, corporate, or government) be responsible for ensuring that all possible viewpoints are represented? I assume everyone would agree that censorship is not acceptable. Are we prepared to extend that to a notional Right to Be Heard?
short, simple, correct answer
NO!
understatement of the day:
“…any intermediaries that get in the way of a listener hearing any particular opinion is thus interfering with free speech. It seems like quite a stretch”
LOL! quite?! try ridiculous!
geez, don’t people have anything better to do with their time ? you don’t like google – then use another search engine! there are literally tons out there – some good some bad … I don’t use all of ’em, but I sure do use a lot more than just google …
besides, you can use something like tor -it’s an interesting experiment – I have found that that sometimes results change in google searches depending on where it *thinks* you’re coming from … (i.e. your tor exit point …)
Me thinks...
that this professor supplements her income as a SEO and thinks anyone that makes it more difficult for her to steal, um profit, from morons should be regulated into aiding her.
Why should they follow any of this?
Why can’t they remove sites from their index? I would RATHER NOT see link farms and sites that generate text SOLELY to bring people to click their ads. That’s just ONE example of the trash the internet is littered with. Google does a good job of filtering those out for me, so good for them for removing irrelevant data.
When Google starts filtering legitimate companies, I’m sure they will have a reason. Even if they don’t, it’s no one’s business to know what the reason is, EVEN IF you are who is directly affected by the filter. What google offers is a free (to the end-user) service, and if you don’t like it there are plenty of other free services out there.
I believe it’d be a restriction of free speech to NOT allow google to remove sites from its index. That’s completely letting alone privacy laws, and what the government can and cannot dictate on private property.
show their methods? are you crazy?
Remember in the 90’s when altavista only used meta tags to rank sites? The top result for any word was porn pills poker or auto loans.
If search engines give away their ranking methods, the same thing will happen again. Is this what we want?
The author of this paper is crazy.
As for the index.. it’s their index. they made it. They should be able to do whatever they want with it.
Why do Americans keep talking about freedom, yet keep trying to do away with the free market?
Don't we have enough "regulation" yet .. ??
Where does it end, pray tell?
Or is this just an opening for regulating the Internet, then what ads can appear from who, and why, and/and/and ..
And who would be the watchdog over the regulating?
And who watches them .. ??
Bill Burke
Wireless Speech Recognition
this is an OLD problem
It is a pity that a law professor does not realise the difference between free speech and “ability to reach an audience”. It is also not a new problem at all.
Elections are the perfect example showing the difference between having free speech and actually being able to be heard. Anyone can say whatever they want, but if they don’t have the money to spend on advertising, nobody will know about it.
On the other hand: what is freedom? Does it mean “you can do whatever you want”? Most of us think it doesn’t, because that would mean (some sick) people would “feel free” to harm others. Freedom is not something you receive, freedom is something you give. If we’d all stop worrying about our own freedom and starting thinking about granting others freedom, the world could be very different.
So, does the Internet need be regulated in order to provide freedom to everyone? Probably. But there is a difficult balance between maintaining freedom and becoming a police state (with the best of intentions sometimes). Also, not everyone realises what Google really is. Just like not everyone realises what wikipedia is. Freedom also requires knowledge about the world. Intelligent, well educated people are often freer than those who are not. So the best we can do is educate people about how the internet and search engines really work. But in the end, people are free to be stupid….
Am I missing something?
Correct me if I’m wrong but Google has nothing to do with these sites. It offers a free service to help the user find something online. Nobody is obliged to use its service, nor do they owe anything to who it finds (or choses not to list).
It is just a service to help you find things quicker. Freedom of speech has nothing to do with finding websites. It sure as heck doesn’t help me find my keys in the morning, so what does it have to do with Google’s free searching service???
Freedom of Choice
As much as I dislike the methodology Google uses, I don’t have a right to make them change it. I have the freedom to go over to Yahoo or any other search engine.
Analogy....
What about publishing dining or movie reviews? Are these not the same? Does a publisher have the right to include or exclude whatever restaurant they want to? Or review some movies and not others? Does a publisher have to provide a “basis” as to why a restaurant was provided 3 stars or 4 stars? Does a movie reviewer have to provide his methodology as to why they thought a movie sucked? Do printed publications do a good job between distinguishing what is editorial and what is advertising (is a little 10pt line at the top of the page stating “This is an advertisement” enough)? Or business demanding they get an expose written about them because they want equal time to practice their free speech?
Yahoo started with a hand edited directory of links. Then process was automated. Google perfected it. I don’t think this is a violation of any of the indexed sites free speech. Search rankings are not a public forum. Google chooses to pubish their search indexing. While it is an issue of free speech, it is the free speech rights of a private company or organization to publish whatever they want however they want. A search engine is not a public space, and thus they are not subject to providing the right to free speech to everyone.
Here’s an interesting wrinkle. What if I created an advertisement in Wikipedia and posted it in an article. Editors would of course take it down. Now couldn’t that be considered impinging on my right to free speech? Especially, considering that technically Wikipedia is a public forum?
spam
so spam is free speech?
what the hell
Sounds like the desired result is that every result should be number one? After all, that’s the only way it can be fair 😛
Sometimes the innocent suffer, but...
As someone who works for a legitimate website that was banished to “Google Hell,” I can say from first-hand experience that their policies do have collateral damage. We went from 13 million page views a month to 3 million, and the effect was immediately noticeable when they blacklisted us (down to the hour it happened in our traffic logs).
Our “crime” was most likely that we didn’t understand the rules Google uses to rank sites, and we were penalized for an innocent part of our site design that, without context that would be given by a human review, must have looked like something they try to stop. We’ve spent months and several SEO contracts reworking our site to play by their rules, but it’s damn hard when they won’t tell us what those rules are.
That said, in general Google’s goals are good. I wish they would offer more assistance to sites that have been wrongly blacklisted, but I don’t think legislation is the way to go. If you tell them they can’t legally blacklist anyone, the quality of overall search results would suffer.
Viruses, malware, trojans
“1. Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.”
If a site infects users with a virus, it’s not required by law to remove that site, but it’s clearly a good thing for users if the search engine removes that site.
This is completely insane. As many people have pointed out so far, Google is a private company and has no obligation to anyone to include their site in the index and results. Google could only bring up results from companies that pay them and that would be fine too. They are a service. If you don’t like them, go somewhere else.
As for rBase4u, there something fishy about what you are saying. “…and we were penalized for an innocent part of our site design that, without context that would be given by a human review, must have looked like something they try to stop” I can’t imagine what is on your site, but with how many times you talk about yourselves being innocent, I just can’t believe you.
Do you pay Google to include you in the search results? If the answer is no, then they owe you nothing. The only people that Google has an obligation to include in the results are people who are paying for the advertising. If your business model relies so heavily on Google traffic, then why aren’t you advertising with them? It seems to me that you are angry that Google no longer supplies over 75% (10 million of your 13 million monthly hits) of your web traffic for free. If you don’t like losing out on that much traffic, pay them or find a better business model.
They do a good job of including relevent avertising as well as non advertised sites, thats why people use them. If they only included paid referrals, no one would go to them.
dont like google or some other engine then dont use it… its not like they force your computer or you not to visit certain site or the other way around. it the same as telling an armani store to sell lime green glowing pants because some dude created it so now every store has to show his product.
Pretty soon its gonna be a crime to walk down a street without gloves on in the winter time… or something like that.
P.S. Jeremy is a good man
A thought
As to whether this should be regulated or not, I’m really on the fence…
it’s one thing to give certain results a very low rating, resulting in them always showing up at the bottom of the list (but, they are still there and available for anybody who would want them) it’s another to remove them from the search results altogether.
Although I personally also don’t enjoy link farms and sites that generate text SOLELY to bring people to click their ads, I still think instead of removing them, rank them so low they end up at the bottom of the listing, instead of removing them.
I guess what I have an issue with is: Google, or any search engine for that matter, even if they don’t say so explicitely, implicitely claims to search the entire web and that the returned results are all the matches on the (entire) web. So removing any results, no matter how noble the intention might be, amounts to misleading or even fraud in my book (regardless of whether it would be legal)
@ #27
That is a completely inaccurate analogy. A library is a publicly funded program. Google isn’t.
By your line of logic, a bookstore that wants to only sell children’s books could also be told that they HAVE to sell adult magazines because someone might be in their store and want to buy one. If they don’t sell them, then they are obviously censoring the adult magazines.
An accurate analogy is Walmart. Walmart originally didn’t sell CD’s with explicit lyrics (not sure if they still don’t). It was their decision not to sell them. If you wanted a CD with explicit lyrics, then you had to shop somewhere else. Is anyone arguing that Walmart should HAVE to sell CD’s with explicit lyrics? No.
Google is a private company and shouldn’t be told what they HAVE to include or not include. If you don’t like what they keep out of their index, then “shop” somewhere else.
author is 100% right!
right on, i agree with these rules.
when the government creates their own search engine, it definitely should follow these rules. then the 200 people who use that crappy engine can know that they have unfettered access to all the crap that those of us who use google count on being filtered out.
i also agree with the library analogy. i plan to march down to my local library tomorrow and insist that they stock every book ever made, those censoring bastards.
Analogy Tuesday
ok, I’ll bite.
If said library were taxpayer supported and the only one available I would certainly give pause but if there were many (privately owned) libraries to choose from that displayed books in the style I prefer then I couldn’t care less.
Bright eager little fascists.
The buggers are everywhere, with their well scrubbed faces, high IQs, advanced degrees and wish to make my life better by imposing more rules on me because I’m really not as smart and insightful as they are.
1. The first rule makes no sense since there is no practical difference between removing page from an index and pushing its ranking down by a thousand or so.
2. Ranking methodology is a trade secret and it would seriously hurt Google (through competition) and Google users (through gaming of the algorithm) if they revealed their method. I really do not want all of my searches to bring up Russian porn sites.
3. Point three is undefinable and unenforceable and hence discredits the whole proposal.
4. Sure, search engines should disclose paid rankings.
The problem people have with getting an audience has nothing to do with getting exposure and everything to do with getting noticed. If you have a compelling point to make you don’t need Google and if you’re spouting stuff that people have already heard or will reject out of hand you’re going nowhere even with a high ranking.
The author should go offer her services to the French who want to build a French competitor for Google. She and the French deserve each other.
This response should be move to the top of the pag
“”blocked by search engines not ranking them “properly.””
Who should be the judge if search engine are ranking web sites properly or not, if it is not going to be done by the search provider themselves?
Personally I think my web site should be top rank for all key words(English or not) and I should not have to pay for priority placement. I also think being charged by my ISP is also a form of censorship I should have free unlimited wireless bandwidth and server space. Anyone charging for me for my 100Tbit/s wireless internet hates the constitution! While we are at it why am I paying for computing power? I need the worlds fastest super computer to render my videos; this is a god given right for without it I don’t have free-dumb of speech.
Regulation of Search Engines - Publicity Argument
The possibility that the press will come in with publicity to offset losses resulting from unfair lower ranking by search engines is nonsense. The search engines have caused thousands or tens of thousands of websites to be subjected to unfair lower ranking, but only a handful of these receive any publicity, and such publicity is short-lived anyway. Losing hundreds of thousands of dollars of revenue per year by the rank loss can never be recovered through the publicity a website might be able to obtain with a decided effort, or even by paying a publicist. The argument is obviously thrown in to try to justify a practice without regard to the equities of the situation. If the government caused the harm, would you argue there is no relief? If a monopolizing search company caused the harm, knowing that no competitor is in a position to challenge the monopolist, isn’t that a difference to take into account?
Carl E. Person
Absolutely right.
If you don’t think we are controlled or manipulated you are an idiot. All search engines should have the same basic rules but there are no rules today. Google is in the forefront of controlling what is seen, who sees what, and what is posted. We as a population are totaling being controlled and fed propaganda. If you want Google’s twist on the world keep using Google. I will look elsewhere.