Fixed: Remove Any Site From Google (even if you don’t control it)

by James on July 19, 2011

UPDATE: It would seem Google is looking into this right now, which is great. The sole reason I posted this was to get the issue patched, I couldn’t find a method of contacting Google or reporting this directly and maybe naively thought this would generate the most gravitas.

UPDATE 2: This was fixed within 7 hours of reporting the problem. Great work by the team at Google to get it fixed and all the URL’s removed in this way should now be back in the index.

This is my first post so I thought I should make it a good one, I hope you enjoy reading it as much as I have writing it.

Yesterday I was busy removing thousands of URL’s from within Googles Webmaster Tools, it was pretty time consuming as there were so many, there had to be an easier way? I settled on quickly making myself a chrome extension that adds a link next to a result in a Google search, deep linked into webmaster tools. With that installed I was busy clicking away removing the URL’s in record time.

Then I made a little mistake and accidentally removed a URL of a website I have no relation to?!? I was stunned it could be that easy. Surely there was no way Google would actually remove the page, right?

I decided to dig a little deeper and do a few tests to see how powerful this could potentially be and how wrong was I! These are the tests I performed, some of which I do not have screenshots for as I really didn’t think it would actually work.

The Tests

  1. Remove a website I control (not in my webmaster tools account) on 18/07/2011 – Gone!
  2. Remove a URL on one of the world’s largest websites (the accident) on 18/07/2011 – Gone!
  3. Remove a friends blog (blank and with permission) on 18/07/2011 – Gone!

NB: On none of the tests were the pages blocked by robots or returned a 404 response (apparently a prerequisite for removal)

How To Do It (please don’t and hopefully Google will patch it soon)

Disclaimer: If you are going to test this please make sure you have permission from the site owner, otherwise although it is a loophole I am pretty sure it is illegal.

The process is actually very simple and just requires some minor modifications to a URL, followed by a form submission. Edit the following URL:

https://www.google.com/webmasters/tools/removals-request?hl=en&siteUrl=http://{YOUR_URL}/&urlt={URL_TO_BLOCK}

Replace in the URL above:

  • {YOUR_URL} = A URL you control within Webmaster Tools
  • {URL_TO_BLOCK} = The URL of the site you want to block:
    • You can request removal of the following:
      • Site – Provide top level domain (E.g. http://www.someurl.com/)
      • Section – Provide URL of the folder (E.g. http://www.someurl.com/somefolder/)
      • Page – Provide URL of the page (E.g. http://www.someurl.com/somefolder/somepage.html)

If you request the modified URL in your browser (make sure you are logged in to your Google account) you should see:

I didn’t actually remove the News Of The World by the way, News International have done quite a good job of that themselves. If you do actuallyΒ  click the Submit Request button, you should see the following:

It then gets inserted as a Pending request in the site owners Webmaster Tools account. If the request is not cancelled it usually leads to the removal of the site from Google’s index which is why I think this is probably the biggest vulnerability in Google today and why I am highlighting it here. I can’t believe I am the only person to figure this out and there are a number of things that could be happening right now if this information is already in the wrong hands.

Thankfully, there is a time delay from when the request is made to when it is actually processed. The only reason I am happy to highlight this here whilst it is still possible, is because it should be so easy for them to fix (and should have never been possible in the first place).

I have tried to forward this on to Google in the hope that they fix it, but if anyone can pass it to the correct person that would be great. If someone from Google could email me, I also want to let you know the URL of the site I accidentally removed so you can remove the removal request for that URL.

{ 49 comments… read them below or add one }

Ryan Jones July 19, 2011 at 4:11 pm

Hmm, sounds like it’d be easy for them to punish you since they know a URL you control. Of course, you could always set up a fake Google WMT account.

Chris July 19, 2011 at 4:17 pm

I can almost hear the FBI choppers above your house πŸ™‚

Best first post I’ve read. You can officially call yourself a Google hacker πŸ™‚

Richard Valentine July 19, 2011 at 4:29 pm

was the daomin haswalt.blogspot.com by any chance?

James July 19, 2011 at 4:31 pm

I was given permission to use that one as a test. It’s not mine and no longer in the index.

Tom July 19, 2011 at 4:38 pm

Brilliant first post – seems to be a pretty gaping hole in GWT..

I’ve always wondered what the SERPs would look like without Wikipedia, eBay and Amazon! πŸ˜‰

Dr. Pete July 19, 2011 at 4:38 pm

All I can say is: Holy crap! Hope they plug that hole soon.

imnotadoctor July 19, 2011 at 4:45 pm

And boom goes the dynamite!

Andrew Rodgers July 19, 2011 at 4:45 pm

Must be the greatest first blogpost ever written.
Was this find the reason you created the blog by any chance?

seosteven July 19, 2011 at 4:56 pm

This is an excellent post. First or not.

Subscribing right now!

atomvillanas July 19, 2011 at 4:58 pm

“Since Google indexes the web and doesn’t control the content on web pages, we generally can’t remove results from our index unless the webmaster has blocked or modified the content or removed the page.”

I think it wont’t work if the page doesn’t blocked by robots.txt or doesn’t provide 404 status code…

James July 19, 2011 at 5:01 pm

@atomvillanas they have removed from the index everything I tested. Not sure how long the removal will last, but none of them were blocked by robots.txt, robots meta or returned a 404.

Raymond Theakston July 19, 2011 at 5:11 pm

What if you remove Google itself will it implode?

y0z2a July 19, 2011 at 5:36 pm

James,

Really interesting first post.

I ask one question that I have been mulling over testing, with the indexation/removal of content.

Q. If you +1 this URI (NB. On page +1, not within SERP +1) – is the result included in the SERP when you search for the specific page? Just curious to see if +1 ingnores index exclusion or removal request?

/y0z

5ubliminal July 19, 2011 at 6:27 pm

Are you sure it’s not removed only from YOUR custom specially tailored results that Google loves to serve us?

Because, if you could remove a site that’s not yours, there’s so much FAIL it hurts. Google is supposed to check robots.txt and a 404 to confirm removals. IT SHOULD NEVER BE DONE JUST PER REQUEST. Requests should be technically backed up by the underlying robots/HTTP protocols and should be aimed only towards speeding up the slower exclusion process determined by a 404 or robots rule.

Thierry July 19, 2011 at 6:47 pm

Maybe it’s not true? Maybe it’s just a link bait?

If so, it’s a really good one.
And if not, it’s still a a really good one πŸ™‚

5ubliminal July 19, 2011 at 8:03 pm

@Thierry: I agree with the link-bait. In such case, Google should remove his blog for false allegations and employing deceitful tactics in milking link-juice from gullible SEMOs.

Michel July 19, 2011 at 8:11 pm

I think Google disabled the function, I get “Hm. Something isn’t right. We’re checking into it now.”

Mike Boudet July 19, 2011 at 8:14 pm

Amazing find! You must have felt like a deity there for a moment with the power to control Google’s index. Good job not cashing out on that and doing the right thing!

SEO Mofo July 19, 2011 at 8:16 pm

I immediately went to Google Webmaster Tools and tried exploiting the shit outta this bug, but it seems Google has already taken action…?

Crawler Access -> Remove URL

Pest Control Guy July 19, 2011 at 8:41 pm

@RyanJones… Good point. Pretty easy for Google to catch the hacks and give them a fatal webmaster score. I am guessing that would be hard to recover from. …Unless you were looking for a new job… in a completely different industry, not recommended.

James July 19, 2011 at 9:19 pm

@5ubliminal @Thierry Sorry totally legit! Couldn’t believe it myself.

Miranda July 19, 2011 at 9:48 pm

Wow, nicely done. Google should be sending you one helluva fruit basket.

jason July 19, 2011 at 9:51 pm

Raymond’s comment was the best. “What if you remove Google itself will it implode?”

HA!

Keith Brown July 19, 2011 at 10:05 pm

This is pure greatness. Should have posted this in Google+ πŸ™‚

Edy July 19, 2011 at 10:19 pm

You deserve a gift from Google. Thanks man!

Mark July 19, 2011 at 10:21 pm

Cool find. Now try topping this post πŸ˜‰

Yosuaf July 19, 2011 at 10:40 pm

Hmm great find. Must admit this is such a school boy from Google. TFS

Reputation Armor July 19, 2011 at 11:09 pm

Great find my man! The bug you found is a dangerous one for sure and needs to be patched which I know Google is already working on this…. I also want to mention that you could have made Millions of $$$$ with that bug by removing URLS for businesses that were negative (black hat rep management)… Unethical YES… But still could have been done. The fact that you exposed it show you have morals and ethics. And heck yes your first post is the best one I have ever seen in my life!!!! Way to go!

Arjun Sandhu July 19, 2011 at 11:09 pm

If someone removed BBC or Facebook, it would make global news! You have no idea of the magnitude of the favour you’ve done Google. Good stuff James.

Joe July 20, 2011 at 12:09 am

i just checked (didnt click submit) and GWT was allowing removal still using this method. i hope they get it fixed soon before all mayhem breaks loose.

p.s. google this is why companies NEED to have a customer service presence.

Charlie July 20, 2011 at 12:34 am

Are you sure you don’t want to work in Google Plex? πŸ˜›

Black Hat Domainer July 20, 2011 at 1:49 am

Thanks a lot! *not*

I mean, do you pretend to release many of these, or this was just to launch your blog?

How do you think the dude who let you inside the forum and teach you about this is felling right now?

If you were not a Black Hat, it was very bad of you to pretend to be one just to get this info just to boost your blog.

Mike July 20, 2011 at 2:40 am

I contacted my buddy who works at Google and he got right on it and had it patched. Nice find.

Somsi July 20, 2011 at 2:42 am

This is a great post. I hope your database continue to grow.

Adis July 20, 2011 at 2:49 am

Wow, that is some hard to believe stuff. Way out of my comfort zone but still pretty hard to believe all the billion$$ minds missed that one. As mentioned I wonder how the SERPS would look without the wikis,amazon and G itself. Good thing Bing guys did not Figure this out and outsourced the process overseas of removing their competition. Having 99.9999999% removed would suck.
Great share

Darius July 20, 2011 at 2:51 am

So G did get in touch with you? Could u share the email/msg.

Lumin July 20, 2011 at 2:52 am

Holy Smokes…How long did it take for Google to contact you after the post?

Profits July 20, 2011 at 3:19 am

This is an interesting post and scary at the same time haha .. I found this site just today for the first time, thanks!

Aman July 20, 2011 at 3:25 am

Interesting find. Posting in Google + now…

j0rd0n se4ttle July 20, 2011 at 3:35 am

nice… you should’ve went and applied for a job there then told them of this vuln πŸ˜› lol that would’ve been a interesting interview πŸ™‚

Jimmyrose July 20, 2011 at 3:54 am

Haha wow, that is one epic find. I wonder how many found it before you and were secretly destroying their competition

Est July 20, 2011 at 4:07 am

is fake kkakaak , guy try rank this blog .

in webmaster tools you need verified your sites kakaka

get out from this blog now !

NARKOZ July 20, 2011 at 5:59 am

I’ve tried it thousand times and it worked everytime. Don’t understand why are you reporting this publicly.

Jodo Kast July 20, 2011 at 6:00 am

Noob test 101: First post: Google hack. Nice job! Blogspot Advertisement.

Clark Nova July 20, 2011 at 7:28 am

Amazing post and seems to have been a confirmed working exploit. This smacks of sloppy coding and is another reason why we shouldn’t be so quick to push all of our delicate details ‘into the cloud’.

What blows my mind is that nobody spotted this sooner!

Tony July 20, 2011 at 7:35 am

Hahahaha !!!!! That is a big hick-up by Google. For a moment I was thinking that this really is not a big deal, but as I see it right now – they fixed it. That probably means that its all thanks to you !

Thats not a fruit basket, thats a “$100k towards anything on Google” gift card that they should send you..

And yes, I have to admit – If I got a hold of it – I would of messed the whole internet up. Thank god for people like you, that do the smart thing, instead of just going at it haha.

Asdf July 20, 2011 at 7:40 am

Maybe now people will think twice storing documents on Google Docs and posting their calendars on Google Calendars.

Matthew July 20, 2011 at 8:33 am

Excellent first post James. Even though it now appears fixed, I’m still baffled as to how this was even possible!

My hat goes off to you, sir.

Good work making it public and causing Google to take action to fix it as well πŸ™‚

Dave Hulbert July 20, 2011 at 9:10 am

Google has a disclosure policy outlined here: http://www.google.co.uk/corporate/security.html

Basically, email security@google.com

Leave a Comment

{ 1 trackback }

Next post: