How to submit a sitemap.xml to Google Webmaster when using Heroku hosting

You may be facing the need to notify Google of your sitemap file, but you don’t know how because you have a rails app running on Heroku.

Ok, here is my hint to sort it out.

My setting is a rails app where I use a gem called sitemap-generator that you can see here: https://github.com/kjvarga/sitemap_generator. This gem works great, by the way.

My app is hosted in Heroku. And as you may probably know, Heroku does not have space for files, so you need to host everything in another server. In my case I use cloud hosting in Amazon, using Amazon Web Services S3.

So my sitemap is stored in the following address:

https://s3-us-west-2.amazonaws.com/myapp-production/maps/sitemap.xml.gz

And this is the url you have to submit to Google Webmaster. In order to set up the Webmaster account, you cannot do it with the true domain, because then you cannot refer to the file. I mean, that if your domain is “myapp.com” then google will expect that you submit an xml file after myapp.com/…

So you have to create a new account starting with https://s3-us-west-2.amazonaws.com/myapp-production/ and then add your sitemap file to be submitted.

Also, in my application I am using multiple subdomains, since each user account has her own subdomain. In this case you have to make an additional configuration and it is to set up verification with DNS text record, as explained here http://googlewebmastercentral.blogspot.com.es/2010/03/dns-verification-ftw.html. If you don’t do this, you will be getting plenty of these error messages:

This url is not allowed for a Sitemap at this location.

Thank you to barmstrong in this forum for providing the links: https://productforums.google.com/forum/?hl=en#!msg/webmasters/w4gMRNFHMlA/24pUYqi69mkJ

If you need further help, please get in touch with me via comments.

Good luck!

Fix this: rake aborted! AWS::S3::Errors::RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.

Ok, I have been a couple of hours trying to find the solution to this:

rake aborted!
AWS::S3::Errors::RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.

I was trying to run a rake:seed command in heroku

$ heroku run rake db:seed

And I got the above mentioned error once and again. It was strange because the behaviour was erratic, sometimes it would upload one file and break, and the next it could do the following.

Finally, I decided to investigate what was happening in the Amazon Bucket. So I opened the bucket and saw that there were some files. I then deleted all the folders with the contained files. And… magic! the error disapeared.

So what is my conclusion? there was something corrupt, or maybe you cannot run the seeds two times. Honestly, I have not tried a second time. It was already quite hard to get here. Next time I reset the database (it is in a staging app), I will find out if files in AWS S3 have to be deleted each time to avoid this conflict.

Hope this info is useful anyway! good luck

Changing ownership AWS Amazon S3 bucket

After some research, as of today, it is not easy to change one S3 bucket from one user to another. I have found this solution:

http://stackoverflow.com/questions/5518205/move-files-directly-from-one-s3-account-to-another

and this other one using a proprietary solution http://support.bucketexplorer.com/topic1165.html

But could not implement it, since I don’t understand well how the IAM works.

So I went for the simplest way: delete the original bucket and create a new one with the same name

Warning: you will have to wait to have the name released.

If you try to create a new bucket with the same name right after you delete it, you will receive the warning:

A conflicting conditional operation is currently in progress against this resource. Please try again.