Upgrading your libraries and dependencies regularly is a must if you want to keep up with application security and performance. Sometimes though, things do not go as planned and part of your code breaks because of the upgrade.

This happened when we upgraded our Ruby version from 2.7 to 3.0. Suddenly, all of our requests to Amazon’s API started failing. As this was a critical part of the business, we had to investigate the root cause of the problem.

Standards are optional?

When we dug deeper as to why they were failing, we found that the API rejected our requests due to a signature mismatch. From experience, this is usually caused by a misconfiguration in the request headers. Adding in debuggers in the request, we found that one particular header looks like this:

X-Amz-Access-Token = 'someaccesstoken'

Now this looks good at a glance, but the request is still rejected. Amazon’s API only accepts headers in lowercase. This is contrary to the HTTP standards, HTTP/1.1 and HTTP/2.

Header field names are strings of ASCII characters that are compared in a case-insensitive fashion.

However, at the end of the day we should comply with whatever API endpoint we are going to use. So we need to adjust the header names to make sure that they are always sent in lowercase.

How do we do that?

A string that does not change

We start by checking where the request gets sent in the code. This may be done via an HTTP library, such as HTTParty. At its core though, these libraries often use Ruby’s standard library for HTTP, which is Net::HTTP.

Net::HTTP handles the request headers in such a way that:

• The header keys are stored internally and processed in lowercase (to conform to the standard)
• When generating the request headers, the key names are capitalized per word

Thus, Amazon’s API wants the header name to be x-amz-access-token, but the library (and thus the application) sends it as X-Amz-Access-Token.

Unfortunately, there is no way to directly customize the behavior of Net::HTTP when it comes to processing the request headers. One excellent way to achieve this however is to use a subclass of String that conforms to our desired behavior when modified.

class ImmutableString < String
  def capitalize
    self
  end

  def to_s
    self
  end
end

Using this subclass, we can now define a string that does not change even when capitalized, for example:

> ImmutableString.new('always lowercase').capitalize
'always lowercase'

Then, we define the request headers in the application using this “immutable” string, prior to passing it to the HTTP library. This was the existing implementation in the codebase to handle the quirks in Amazon’s handling of the request headers. However, when we upgraded to Ruby 3.0, this class no longer works. Why is that?

Inheritance in Ruby 3.0

There is an interesting behavior between Ruby 2.7 and 3.0 with regards to inheritance. This became apparent due to the fact that we are using a subclass of String. It can be illustrated by an example.

For Ruby 2.7:

> key = ImmutableString.new('what am i')
> key.capitalize.class
ImmutableString
> key.capitalize.downcase.class
ImmutableString 

For Ruby 3.0:

> key = ImmutableString.new('what am i')
> key.capitalize.class
ImmutableString
> key.capitalize.downcase.class
String

What happened here? In Ruby 3, if you call a method that is not in the subclass, it will revert back to the superclass. In the example above, downcase is not modified by the ImmutableString subclass, so the string went back to being a String. And when this happens, our overrides will no longer work, Net::HTTP then capitalized the request headers, and the request fails.

In previous versions of Ruby, the subclass is preserved even after calling a method that only exists in the superclass. This is the reason why this bug was not detected before.

Making ImmutableString work again

So, we need to make a change to the ImmutableString class for it to work as expected in Ruby 3.0. In order to do this, we need to know first how Net::HTTP modifies the request headers.

Since Ruby is open-source, and Net::HTTP is part of the standard library, we can check the source code in Github here:

https://github.com/ruby/ruby/tree/master/lib/net/http

For the purposes of this article however, we will use the links for the Ruby 3.0 branch, so that the links will still work even for future version releases.

The first thing we will notice while reading the source code is:

@header[key.downcase.to_s] = [value]

All keys are converted to lower case and saved in an internal variable.

https://github.com/ruby/ruby/blob/ruby_3_0/lib/net/http/header.rb#L25

On initialization, all header keys are converted to lower case in order for the headers to be accessible using case-insensitive keys. Also, notice that key.downcase.to_s is used very frequently within the class.

The first update then is to add the downcase method to the ImmutableString class so that the key is not converted back into a String (and thus losing all of the other overrides we made). We already have the to_s method covered, so even if these methods are chained, we keep the result as the subclass.

Digging further in the class, we arrive at the method that capitalizes the header keys prior to sending the request:

name.to_s.split(/-/).map {|s| s.capitalize }.join('-')

Keys are split, capitalized, and then joined again to generate the request header keys.

https://github.com/ruby/ruby/blob/ruby_3_0/lib/net/http/header.rb#L221

The problem is, once you split an ImmutableString, each individual substring becomes a String object again, and thus can be capitalized. The second update we need is to also override the split method and ensure that each substring is an instance of ImmutableString and not a String.

The array of substrings are also joined in the end. However, it is not necessary to override the join method as this is the last transform performed before sending out the request.

Applying these updates to our ImmutableString subclass, the final code then becomes:

class ImmutableString < String
  def capitalize
    self
  end

  def to_s
    self
  end

  def downcase
    self
  end

  def split(*)
    super.map { |s| self.class.new(s) }
  end 
end

With this, the request headers are now always kept in lowercase and the issue with Amazon’s API was resolved!