return to OCLUG Web Site
A Django site.
December 21, 2012

Michael P. Soulier
But I Digress
» Signatures in OAuth

Hi again. I already went into the basics of OAuth in a previous post, and alluded to the signatures being the hard part of the implementation. I'm using a python-oauth module for the client, but for the server, I decided to implement my own in Perl and Mojolicious, since I had problems with Net::OAuth that I deemed a protocol violation. Besides, I learn more this way.

There are multiple signature methods supported in OAuth 1.0, specifically three.

  2. HMAC-SHA1
  3. RSA-SHA1

Unsurprisingly, PLAINTEXT is the simplest. In this case, the signature is a simple combination of the consumer secret and the token secret, if there is any yet, with a slight twist. To ensure that the characters are treated properly, they must first be "normalized". This means decoding them as you would any percent-encoded string, and then...

  • convert them to UTF-8

  • escape them with percent-encoding from RFC 3986 like so

    • these characters are not encoded: alphanumerics, "-", ".", "_", "~"
    • all other characters must be encoded
    • any hexidecimal used in encoding must be upper-case

I found that hard to read, but I did find this bit of Perl to solve the problem in Net::OAuth, which I converted to a Mojolicious helper.

helper encode => sub {
    my $self = shift;
    my $str = shift;
    $str = "" unless defined $str;
        if ($str =~ /[\x80-\xFF]/ and !utf8::is_utf8($str)) {
            warn("your OAuth message appears to contain some "
                . "multi-byte characters that need to be decoded "
                . "via or a PerlIO layer first. "
                . "This may result in an incorrect signature.");

Decoding is even simpler.

helper decode => sub {
    my $self = shift;
    my $str = shift;
    return uri_unescape($str);

In Python, the same code looks like so.

import urllib

def escape(s):
    """Escape a URL including any /."""
    return urllib.quote(s, safe='~')

def _utf8_str(s):
    """Convert unicode to utf-8."""
    if isinstance(s, unicode):
        return s.encode("utf-8")
        return str(s)

escaped = escape(_utf8_str(s))

# Unquoting is just
unquoted = urllib.unquote(s)

So, this needs to be done to each of the parameters, in this case, the consumer key and the token secret, if any, and then they are combined with an ampersand. So, if your consumer key is, say, "myconsumerkey", and you don't have a token yet, then the initial PLAINTEXT signature is myconsumerkey&.

Now, this isn't too bad, but once you get into HMAC-SHA1 signatures, it gets a lot worse. The signature from the PLAINTEXT method becomes the key for the signature, and you'll already have the code for that now, but the raw input to the HMAC-SHA1 algorithm is a base string, that is rather difficult to construct. The input is the HTTP method, the request URI, both normalized like above and contatenated with an ampersand. Then, this will be contatenated with an ampersand to all of the input parameters in the request, constructed in a particular way.

  1. Take all input parameter names and values from all sources, and normalize them like above. (but skip the oauth_signature parameter)
  2. Sort all parameters by the normalized parameter name.
  3. Pair the names and values, contatenated with an "=".
  4. Concatenate all pairs with ampersands in the sorted order.
  5. Escape the entire string using the method above.

This is the base string to the HMAC-SHA1 algorithm, along with the key we mentioned. The final signature should match what the client generated. Oh, and if you're running your service on a nonstandard port (80 or 443), then you must include the port in the URI.


A call to http://localhost/initiate on port 80 or 443, a GET request, with the following params:

{'oauth_nonce': '21823552', 'oauth_timestamp': 1356129798,
 'oauth_consumer_key': 'Mitel test', 'oauth_signature_method': 'HMAC-SHA1',
 'oauth_version': '1.0', 'oauth_signature': 'pevzNqSnJ8QtqFUDWVlYhVRp8D0=',
 'oauth_callback': 'oob'}

The base string would look like this:


with a key of:


and a final signature of:


Oddly, if I used the b64encode method in Digest::HMAC_SHA1, a final "=" sign is missing on the final result, so I had to pull in MIME::Base64 and do this instead:

my $sig = encode_base64($hmac->digest);

The equivalent Python in the oauth library does this:

import binascii

# HMAC object.
    import hashlib # 2.5
    hashed =, raw, hashlib.sha1)
    import sha # Deprecated
    hashed =, raw, sha)

# Calculate the digest base 64.
return binascii.b2a_base64(hashed.digest())[:-1]

That just leaves RSA-SHA1, but that requires a pre-existing SSL relationship with the server, using SSL certificates. As such, I'm not worrying about it just yet. I don't think it'll be used much.

I'll need to do some interop testing with a few different clients, I'm hoping that they're not all snowflakes. The point of the rigid nature of the base string construction is that the final product is supposed to be reproducable.

The base string construction is definitely the hardest part, and I've read that the signatures were dropped in OAuth 2.0 because they were too hard to do. I'd rather not drop the added security, and while they're a pain, there are sample implementations to follow. I think that OAuth 1.0 is a better choice. And it's, like, finished.

December 20, 2012

Michael P. Soulier
But I Digress
» Understanding OAuth

We meet again. Thanks for stopping by. This post has a high geek factor, so if you're not one, feel free to move along, nothing to see here.

Still here? K, your funeral. At work, I'm trying to add a ReST Web Service to a product that could desperately use it. I want to do this to permit better remote monitoring of said product and remote control for one of our management services and future UIs that I may implement (think apps on Android and iOS).

This is no small task, and it cannot be taken lightly. It will likely be used by many, so it must be easy to debug and integrate with, hence ReST. I'll rant about the evils of SOAP later. It must be secure, which is easy, Apache is already on the system, with SSL support. It must be authenticated. This is tougher.

Since I expect SSL, I could require client certificates, but they're a pain to provision. You don't authenticate to with client certificates for this reason. Setting them up is about as much fun as visiting the passport office, and it hurts the wallet too. Why should anyone pay for the privilege of shopping in a store? Oh...I'll rant about CostCo later.

A popular method of authentication for web services is now OAuth. OAuth was designed to fix a particular problem. Example: I like Twitter. I hate Twitter's web interface. I prefer lightweight and simple. Plus, it's terrible to use from a smartphone. Fortunately, Twitter has a ReST web service that uses OAuth for authentication, so a vendor can release a Twitter app for my phone that I can connect to my account. "But," you ask, "surely you don't give this app your Twitter password!" No. I don't. It's a closed-source app and for all I know, they'd send it to themselves for nefarious purposes. I want the app to access my Twitter account, but I do not want them to have my password. How do we do this? Enter OAuth.

How does it work? The idea is simple. I'm going to go to my Twitter account, and tell Twitter to issue a random "token" in my name. If you think you don't know what a token is, think again. I suspect you carry keys, id badges, smartcards, that kind of thing. There isn't any practical difference, beyond keys being functional mechanically. This token I will then securely provide to the Twitter client on my phone, and when Twitter sees it, it will know that the client is mine, and grant access. The token will be transmitted over SSL to avoid spying and replay attacks, the hard part is securely getting the token into the client, and keeping it convenient. We do not want a repeat of SSL client certificates. Provisioning needs to be simple.

But, automating such things is problematic. Even if you implement a request queue for token requests, you don't want it to fill up with spammed garbage requests, and it will if we don't prevent it. Spammers should be pulled into the street and shot, but I digress. A way to prevent this is to at least have an existing relationship between the app vendor and the service. So, if I want to distribute an app, I need to go into Twitter and tell them that I want to be a "consumer" of their service. They'll ask me all kinds of details about who I am, and then issue me a consumer ID, and a shared secret. The shared secret is a random string that only they and I know. Now, when my app asks them for a token, it identifies itself with the consumer ID, and they validate it, which includes checking a signature built from the shared secret, so they know it's a valid request. BTW, this consumer ID can be used for billing later, as not all web services are free. Here comes SAAS.

Furthermore, it's not Twitter staff that need to approve this request, it's the owner of the resource in question, the Twitter account holder. This is good, as the account holder is the one who really cares here, and the Twitter staff would otherwise be overloaded with handling requests like these. Now, HTTP is a stateless protocol, so OAuth needs to add some state in the approval process. So, initially when we request a token, OAuth responds with a temporary one, so that the resource owner can be redirected to Twitter itself to approve access to their account. Assuming that you approve, a new token called a verifier will be issued. This verifier can be provided in the response, or simply displayed in the web page so that you can copy it into the app in an "out-of-band" manner (ie. cut and paste).

This verifier is then used to request the final set of tokens from the service. It proves to the OAuth subsystem attached to the service that:

  1. The client was written by an approved vendor. consumer id
  2. The owner of the account wishes to grant access to this client. token and verifier
  3. The request was not forged by some random ass-hat on the net. signature including shared secrets

Once done, the app has a set of tokens that it can continually use to make HTTP requests to Twitter's web service. The tokens issued can also be limited in permissions, perhaps providing read access but not write access, and perhaps to only specific areas of the API. The tokens can also be revoked, if you wish to pull said access in the future. This is a "good thing". I use this to fetch tweets, to make them, to manage follower lists, etc. All securely and authenticated over HTTPS. And I do it from the command-line, with my own client written in Python, which I dubbed Twit. I pull my data from Google in the same manner, for calendar events and contacts. Thus far it has worked well.

Now, for what I want to do, I'm implementing OAuth 1.0, even though the 2.0 "standard" is kinda coming. I phrase it this way because it's been coming for a while now, and as communities often do, it's already forked, dropped some security in the form of strong signatures, in the name of developer convenience. So, I'm sticking with 1.0. It's harder to implement, but it's done, and it's more secure. I've only just gotten PLAINTEXT and HMAC-SHA1 signatures working, I'll post on that soon. I'm doing the server in Perl and Mojolicious for legacy reasons, and implementing it myself because I've had issues with the Net::OAuth module for Perl. Maybe I'm using it wrong, but thus far I've gotten no response from the author. Plus, I learn much, much more this way.

This post is running a tad long, so I'll end it here, with more to come on my trip into OAuth. Specifically implementing support for the signatures. Funny thing is that the signing part is simple, but constructing the base string is freakin' hard. You'll get what I mean if you read the RFC, or just stay tuned.

See? Geeky. I warned you.