I agree. Its a difficult issue for sure. However it definitely emphasizes the concept of proper due diligence. Something seriously lacking in the business world.
oh - forgot to mention credit for the word for word transcript and my being alerted to this goes to an article from Barry Schwartz over at SearchEngineLand
Matt Cutts, head of Googles Search Spam unit stated on the record over on TWIT.TV their intent with the way they are going after spammers:
"If you want to stop spam, the most straight forward way to do it is to deny people money because they care about the money and that should be their end goal. But if you really want to stop spam, it is a little bit mean, but what you want to do, is sort of break their spirits. There are lots of Google algorithms specifically designed to frustrate spammers. Some of the things we do is give people a hint their site will drop and then a week or two later, their site actually does drop. So they get a little bit more frustrated. So hopefully, and we’ve seen this happen, people step away from the dark side and say, you know what, that was so much pain and anguish and frustration, let’s just stay on the high road from now on."
So my position that it IS punishment, in my opinion based on this statement,is correct. Punishment is designed to break people of a bad habit.
ah thanks for the nofollow factor - I totally failed to check to see if they were nofollowed. Proving my "forensic SEO audit" skills aren't always used when I comment on the web. :-)
Okay as an actual SEO professional (one who only advocates real, sustainable SEO, and not crappy spam tactics), I will add this:
1. It IS punishment. Spam has gotten so far out of hand over the years that previous efforts to discourage and otherwise eliminate otherwise undeserving results from the organic listings were not getting the message across. Spam just became a massive business.
So to really send the message across, Google is now much more SEVERELY penalizing sites that use spam tactics, one of which is crappy link techniques. The notion here being that when a site gets a manual penalty for crap links, it becomes a very daunting task to clean up now.
Couple that with most of those sites then needing to re-earn (or in actually earn for the first time in legitimate ways) rankings, and more sites are doing all they can to become good netizens.
2. Leaving spam comments up just to spite foolish site owners is NOT helpful to TechDirt. And it doesn't contribute to punishing those site owners because they'll just disavow the links if you leave them up.
In fact, where it CAN be a problem for TechDirt is if Google's system detects too many spam comments, this site WILL be penalized.
I doubt there are that many on TD, so it's highly unlikely that this scenario would happen (as compared to sites like Mashable or others that have free-for-all comment spam where those are more likely to see some sort of hit).
3. Charging site owners to remove their links is a possible revenue stream, however the overwhelming majority of site owners or link-clean-up providers who encounter a fee situation ignore it and just disavow those links.
And for those site owners who come to me for an audit after they've been penalized, that's exactly what I recommend to them. Along with noting in their tracking of their clean-up those sites that attempted to charge for the service. Because that's potentially subject to being viewed as an extortion scheme under some circumstances (not a TD scenario though either).
How many more sleazy lowlife lawsuits are you intending to file in your lifetime? I mean, isn’t the impending doom from the fallout of your shenanigans with John enough to get you to run for cover at this point? Do you believe you can continue to trash the American legal system for every penny you can squeeze out of unsuspecting American citizens who are tricked by your tactics?
Do you feel no remorse? Do you completely disregard human decency? Do you believe you and John are so bullet-proof to the long term legal process? Or do you have tickets out of the country sitting on your nightstand awaiting that fateful day in the near future when a warrant will be issued for your arrest?
I ask out of simple human fascination.
And of course, I state here, for the record, that my views are purely my personal opinion.
that's my "not a lawyer" understanding as well. Curious to see if he can get around that minor stumbling block. And can't wait for this to reach Carreon levels. It's on that trajectory!
Your honor, my client informs me they cannot, unfortunately produce such records. Apparently all records were turned over to Salt Marsh. [communicated as the smell of burning paper wafts across the offices of Steele, Hansmeier and Duffy...]
Just because something did not have an original intent to be used in a certain way does not mean it should not be used in a new way if that way is innovative and provides value to the world.
Anything other than that understanding is called myopic thinking.
it sure as hell can be both. While robots.txt is not by original nature related to search engines, a means of security, Google has the power and resources to respect it for the sake of security. If you don't grasp that, not my problem.
Because its an opportunity for Google to help improve the securing of private information on the web. Since they already take proactive steps in other areas to improve security online, why not here?
For example - they proactively block sites their system detects that have malware or viruses. They don't have to. Its the responsibility of site owners to ensure their sites don't have malware or viruses baked in. Yet Google has chosen to help.
No, Google is NOT saying that. Yet their system is more than capable of keeping URLs out of the system that are listed in the robots file so there's no excuse why they, as a supposed security advocate, shouldn't honor robots.txt instructions. "Disallow" is pretty clear in its definition.
I'm not relying on it. I'm a forensic SEO consultant with a fair amount of digital security experience. What I'm saying is sites need to get their security methods right. At the same time, Google claims to be a security backstop, yet they allow those URLs into their system.
A citation for it? yeah half the search marketing industry. As an SEO audit professional I routinely encounter it. They list URLs, but beneath them, where a description of the file would go is a statement
A description for this result is not available because of this site's robots.txt – learn more.
They do not show all URLs that are blocked in the robots file, however if their (extremely flawed) system sees enough "other indicators" to countermand the robots instruction, they ignore that instruction.
"Other indicators" is most often "a link to that file somewhere on the site itself or pointing to the URL from another site.
While Google won't crawl or index the content of pages blocked by robots.txt, we may still index the URLs if we find them on other pages on the web. As a result, the URL of the page and, potentially, other publicly available information such as anchor text in links to the site, or the title from the Open Directory Project (www.dmoz.org), can appear in Google search results.
Which is complete bullshit. because while they're not actually indexing the CONTENT of the page, they're indexing the URL.
So in the case of a URL that includes variable parameters labeled with "order" or "customerID" or some-such, that opens up the can of WTF for anyone savvy enough to go snooping.
While Google is not ultimately responsible for other the administration of other sites, they have chosen to take a stand against hacked sites and malware ridden sites, going so far as to block them from search results pages.
Google claims to be on the side of security, yet they ignore the robots.txt file's disallow instructions, and not only that, but publicly display links found on a site where those links were clearly delineated as "disallow" in that file. As such, they are implicit in the breach.
Companies that have asshats who don't give a crap about security are the norm - its inexcusable. The fact they can even file such a suit, or that the police state can bring charges against people who expose such crap is frightening.
Just this morning in a cursory review of a prospective audit client's online presence, I did a Google search and discovered over 1,000 PDFs of customer invoices they blocked via robots.txt file but since Google now includes URLs of robots blocked files and slaps a "description not available due to robots instruction" that shit is wide open to anyone on the web, no hacking needed.
Companies need to be held accountable for their massive security failings and Google needs to be held accountable as well, even though that shit should have been completely blocked and behind a secure firewall.
The fact that this situation involved a couple reporters gives me little comfort in the notion that asshat companies might eventually be held accountable for causing such massive failings.
We need a comprehensive overhaul of the system, one NOT determined by congress or lobbyists. One that severely penalizes the asshats that cause the problem and rewards the ones who expose it.
Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Heh
I agree. Its a difficult issue for sure. However it definitely emphasizes the concept of proper due diligence. Something seriously lacking in the business world.
Re: "We Want To Break Their Spirits"
oh - forgot to mention credit for the word for word transcript and my being alerted to this goes to an article from Barry Schwartz over at SearchEngineLand
"We Want To Break Their Spirits"
Just saw this:
Matt Cutts, head of Googles Search Spam unit stated on the record over on TWIT.TV their intent with the way they are going after spammers:
"If you want to stop spam, the most straight forward way to do it is to deny people money because they care about the money and that should be their end goal. But if you really want to stop spam, it is a little bit mean, but what you want to do, is sort of break their spirits. There are lots of Google algorithms specifically designed to frustrate spammers. Some of the things we do is give people a hint their site will drop and then a week or two later, their site actually does drop. So they get a little bit more frustrated. So hopefully, and we’ve seen this happen, people step away from the dark side and say, you know what, that was so much pain and anguish and frustration, let’s just stay on the high road from now on."
So my position that it IS punishment, in my opinion based on this statement,is correct. Punishment is designed to break people of a bad habit.
Here's the link to the full audio
Re: Re: SEO, Spam and Googles Logic
ah thanks for the nofollow factor - I totally failed to check to see if they were nofollowed. Proving my "forensic SEO audit" skills aren't always used when I comment on the web. :-)
SEO, Spam and Googles Logic
Okay as an actual SEO professional (one who only advocates real, sustainable SEO, and not crappy spam tactics), I will add this:
1. It IS punishment. Spam has gotten so far out of hand over the years that previous efforts to discourage and otherwise eliminate otherwise undeserving results from the organic listings were not getting the message across. Spam just became a massive business.
So to really send the message across, Google is now much more SEVERELY penalizing sites that use spam tactics, one of which is crappy link techniques. The notion here being that when a site gets a manual penalty for crap links, it becomes a very daunting task to clean up now.
Couple that with most of those sites then needing to re-earn (or in actually earn for the first time in legitimate ways) rankings, and more sites are doing all they can to become good netizens.
2. Leaving spam comments up just to spite foolish site owners is NOT helpful to TechDirt. And it doesn't contribute to punishing those site owners because they'll just disavow the links if you leave them up.
In fact, where it CAN be a problem for TechDirt is if Google's system detects too many spam comments, this site WILL be penalized.
I doubt there are that many on TD, so it's highly unlikely that this scenario would happen (as compared to sites like Mashable or others that have free-for-all comment spam where those are more likely to see some sort of hit).
3. Charging site owners to remove their links is a possible revenue stream, however the overwhelming majority of site owners or link-clean-up providers who encounter a fee situation ignore it and just disavow those links.
And for those site owners who come to me for an audit after they've been penalized, that's exactly what I recommend to them. Along with noting in their tracking of their clean-up those sites that attempted to charge for the service. Because that's potentially subject to being viewed as an extortion scheme under some circumstances (not a TD scenario though either).
Sent Email to Paul
Okay I couldn't resist. Just sent an email to Paul at his mail@classicjustice.org address.
Re: HAWKS WIN
Hockey trolls. Go figure.
hahahahaha :-) What's REALLY going to bake your noodle though? Would they have even made it to the finals if there had been a FULL season?
Re: Re: Umm...
If that's the case, we can expect his former attorney to be on record as "instructed to file *with* prejudice..."
Re: Umm...
that's my "not a lawyer" understanding as well. Curious to see if he can get around that minor stumbling block. And can't wait for this to reach Carreon levels. It's on that trajectory!
the smell of burning paper
Your honor, my client informs me they cannot, unfortunately produce such records. Apparently all records were turned over to Salt Marsh. [communicated as the smell of burning paper wafts across the offices of Steele, Hansmeier and Duffy...]
it's called innovation
Just because something did not have an original intent to be used in a certain way does not mean it should not be used in a new way if that way is innovative and provides value to the world.
Anything other than that understanding is called myopic thinking.
crticial thinking
it sure as hell can be both. While robots.txt is not by original nature related to search engines, a means of security, Google has the power and resources to respect it for the sake of security. If you don't grasp that, not my problem.
Re: Re: Re: Re: Re: Re: Re: Re: Re: Re:
Because its an opportunity for Google to help improve the securing of private information on the web. Since they already take proactive steps in other areas to improve security online, why not here?
For example - they proactively block sites their system detects that have malware or viruses. They don't have to. Its the responsibility of site owners to ensure their sites don't have malware or viruses baked in. Yet Google has chosen to help.
This is no different.
Re: Re: Re: Re: Re: Re: Re: Re:
No, Google is NOT saying that. Yet their system is more than capable of keeping URLs out of the system that are listed in the robots file so there's no excuse why they, as a supposed security advocate, shouldn't honor robots.txt instructions. "Disallow" is pretty clear in its definition.
Re: Re: Re: Re: Re: Re:
I'm not relying on it. I'm a forensic SEO consultant with a fair amount of digital security experience. What I'm saying is sites need to get their security methods right. At the same time, Google claims to be a security backstop, yet they allow those URLs into their system.
Re: Re: Re: Re:
A citation for it? yeah half the search marketing industry. As an SEO audit professional I routinely encounter it. They list URLs, but beneath them, where a description of the file would go is a statement
They do not show all URLs that are blocked in the robots file, however if their (extremely flawed) system sees enough "other indicators" to countermand the robots instruction, they ignore that instruction.
"Other indicators" is most often "a link to that file somewhere on the site itself or pointing to the URL from another site.
The "learn more" link points to this Google answer page where it states:
Which is complete bullshit. because while they're not actually indexing the CONTENT of the page, they're indexing the URL.
So in the case of a URL that includes variable parameters labeled with "order" or "customerID" or some-such, that opens up the can of WTF for anyone savvy enough to go snooping.
Re: Re: Re:
uh complicit? explicit? expletive deleted? #FastRantTypingStrikesAgain
Re: Re:
While Google is not ultimately responsible for other the administration of other sites, they have chosen to take a stand against hacked sites and malware ridden sites, going so far as to block them from search results pages.
Google claims to be on the side of security, yet they ignore the robots.txt file's disallow instructions, and not only that, but publicly display links found on a site where those links were clearly delineated as "disallow" in that file. As such, they are implicit in the breach.
Re: Don't go out of your way to write a scraper program!
oh crap there's what appears to be a bomb over there. Fuck I better hide my eyes and not say anything.
OOTB you are the personification of troll=stupidity
(untitled comment)
Companies that have asshats who don't give a crap about security are the norm - its inexcusable. The fact they can even file such a suit, or that the police state can bring charges against people who expose such crap is frightening.
Just this morning in a cursory review of a prospective audit client's online presence, I did a Google search and discovered over 1,000 PDFs of customer invoices they blocked via robots.txt file but since Google now includes URLs of robots blocked files and slaps a "description not available due to robots instruction" that shit is wide open to anyone on the web, no hacking needed.
Companies need to be held accountable for their massive security failings and Google needs to be held accountable as well, even though that shit should have been completely blocked and behind a secure firewall.
The fact that this situation involved a couple reporters gives me little comfort in the notion that asshat companies might eventually be held accountable for causing such massive failings.
We need a comprehensive overhaul of the system, one NOT determined by congress or lobbyists. One that severely penalizes the asshats that cause the problem and rewards the ones who expose it.