Monday, June 25, 2012: tos3: An extremely simple tool to sync local files to Amazon S3.

For some time blog has been running on a Ruby on Rails webapp hosted on Heroku, which makes it feels somewhat “slow”. So I decided to make my website totally static and host them somewhere.

I got the blog engine done and then wondered where and how to put these static files. I chose Amazon S3 and tried to find a nice way to put the files up. I ended up writing my own syncer in Java.

Why Java?

Well, because I love its File class and how it abstracts the file system, so I can use them consistently on all platform. I don’t have to worry about slashes and backslashes.

Then, because Amazon AWS has a Java SDK.

Another nice thing is that I can compile it to a single .jar file and then run it on all platforms.

Usage

After compiling, you can run it like this:

java -jar tos3.jar <directory> <bucket> <path-to-s3credentials.properties> preview

It will look at the <directory> and list of files in the <bucket>, and compares them to show what file needs to be added or removed.

The <path-to-s3credentials.properties> is path to your credentials file, which should look like this:

accessKey=YOUR_ACCESS_KEY
secretKey=YOUR_SECRET_KEY

After you’re sure about what you’re going to upload, you can remove the “preview” part, and then tos3 will actually upload and remove files on s3.

.noex.html

For pretty URLs, I prefer that many URLs don’t have file extension, but served like an HTML file. For instance, this blog post URL doesn’t have the file extension.

For dynamic sites this can be achieved easily, and for static sites this app handles it for you: any file whose name ends with .noex.(whatever) will have that part removed when uploading.

So actually on my local computer this blog post resides at “page/tos3.noex.html”, but when uploaded it will go to “page/tos3” instead.