On August 15, 2009 Amazon begin requiring all calls to their AWS servers to be cryptographically signed. This caused MynaStuff, our home inventory website, to gradually break and stop serving pages. This post is to help me remember details about what I did this weekend and to help anyone else out who might be in the same boat.

Because we use memcached on our servers the actual degradation of the service was quite slow and intermittent. However it soon became clear that we had a major problem and after a few seconds of investigation I knew what I had to do. We had built MynaStuff pretty quickly and had just thrown together a home grown connection to the AWS service. It was time to either add crypto signing to our code or use someone elses code. Never a big fan or build over buy I downloaded Ruby-AAWS and started hacking.

I can't say enough good things about Ruby-AAWS, it was easy to install and pretty easy to work with. Even though it wasn't designed to work with Ruby on Rails everything went pretty smoothly until I tried to get the data out of the memcache. The problem centered around how Ruby-AAWS builds classes on the fly to contain the data coming back from AWS. The way it works is really slick, however Ruby doesn't like the virtual classes too much so AAWS has to provide a factory method (of sorts). Just a simple call to Amazon::AWS::AWSObject.load() with the string that you retrieved from your memcache system and your good to go. Or so I thought.

It turns out that Ruby-AAWS depends on the marshal.load() method to throw an ArgumentError when it has a problem reading some data and Rails somehow changes that exception to something else. So, no matter what I tried it always failed. The good thing about Open Source code is the ability to find out exactly what is wrong and then fix it (for your system or for the whole world). After changing the aws.rb file to check for any exception (i.e. Exception) the code worked again. However, checking for any exception is like killing a fly with a sledgehammer and is not a good idea. I am currently looking for the reason that the exception thrown by marshal.load() changed and how I can fix me code to accept this and deal with it.

Anyway, here is the patch I submitted to the Ruby-AAWS author. Happy hunting!


*** aws.rb.ORIG 2009-08-22 14:44:39.000000000 -0400
--- aws.rb 2009-08-22 14:44:51.000000000 -0400
***************
*** 230,236 ****
        def AWSObject.load(io)
   begin
     Marshal.load( io )
!  rescue ArgumentError => ex
     m = ex.to_s.match( /Amazon::AWS::AWSObject::([^ ]+)/ )
     const_set( m[1], Class.new( AWSObject ) )
  
--- 230,236 ----
        def AWSObject.load(io)
   begin
     Marshal.load( io )
!  rescue Exception => ex
     m = ex.to_s.match( /Amazon::AWS::AWSObject::([^ ]+)/ )
     const_set( m[1], Class.new( AWSObject ) )