Easy user accounts management with couchdb

Akshat Jiwan Sharma, Sun Aug 10 2014

User authentication gets real boring real fast. It is fun the first couple of times you do it. I remember the exhilaration that I felt when I wrote the authentication for my first ever web application. But after doing this for 108 times I don't particularly enjoy writing for register-login-logout scenario anymore. In fact I think it is one of the slowest most boring parts of the web application design.

It is a thankless job. Do it right and no one pats on your back and says "Well done joe! you wrote beautiful authentication code. Boy I just love signing in and out of this website" But if, God forbid, you get it wrong you end up on hacker news where this "piece of news" stays on the front page for some 23 days giving the entire programming community ample time to revel in your woe and making sure that your reputation as a programmer is completely destroyed. Life is sad, a bitter cookie if you ask me. But there it is.

Wouldn't it be nice if we could "outsource" the user account management to some software so we can focus on more important stuff like working on the actual application rather than writing boiler plate code. Ah yes I can picture you nodding your head in approval. "It would be nice indeed!" But what's that? I detect a slight sarcasm in your voice. You don't trust me. You think it is a pipe dream. I don't blame you. Once bitten twice shy and all that.

Say hello to couchdb.

"Hello to couchdb?"

"Yes hello to couchdb"

"But isn't that-----"

"Yes it is"

"But doesn't it-----"

"No it doesn't"

"Wouldn't it be -----"

"No it wouldn't"

Relax.

So here is the kicker. We are going offload entire account management to couchdb. We will create users in couchdb. We will store their roles in couchdb. And will let couchdb authenticate them while we sit back and feel clever. But before we do that we must learn some basics.


Start of basics tutorial

Step 0: install and start couchdb. The default port that it listens to is 5984 and I am going to assume that you are running on 5984.

Step 1: Create a couchdb administrator

The administrators are defined in the couchdb configuration files. But thankfully we don't have to look for big ugly files in big ugly directories. We will just type a beautiful single line command from our majestic terminal and let couchdb take care of the REST.

curl -X PUT http://localhost:5984/_config/admins/ABBA -d '"dancing-queen"'

You should see the

""

in the response. I guess couchdb was too stunned by your magnificence to make a reply, whatever. Any way lets check our config file to see if we have an admin named ABBA.

curl -X GET http://localhost:5984/_config

{"error":"unauthorized","reason":"You are not a server admin."}

All right we will login as the administrator and see if that helps.

curl -X GET http://ABBA:dancing-queen@localhost:5984/_config

{
.....irrelevant objects we don't care about
admins: {
ABBA: "-pbkdf2-2c3adb095f2c91498e816e6873ac2297f2bba378,11661435b847e1d5ad526b8ee6d36a67,10"
},
..... irrelevant objects we don't care about
}

`

Ahh that's better. Notice how our "dancing-queen" password was automatically saved in a secure hash rather than in plain text. All of this without us having to move our fingers. I mean without us having to move our fingers much.

Step 2 : Add couchdb users

Now that we have an administrator we will add a few users. All the users in couchdb are stored by default in _users database. To store a user you just have to issue a PUT request to the _user database. Here's how you do it.

curl -X PUT http://ABBA:dancing-queen@localhost:5984/_users/org.couchdb.user:fernando \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{"name": "fernando", "password": "apple", "roles": [], "type": "user"}'

{"ok":true,"id":"org.couchdb.user:fernando","rev":"1-e0ebfb84005b920488fc7a8cc5470cc0"}

And that adds our friend fernando as a user.

At a minimum user document must contain name, password, roles and type field. The type will always be user. The _id field must be prefixed with org.couchdb.user (we don't have to include _id in the json body we can simply append it after the _users in the url). roles is an array that can contain anything and name and password are self explanatory. Besides these fields you are free to add any arbitrary number of fields to the user document.

Let's fetch this document and see how couchdb saves it internally.

curl -X GET http://ABBA:dancing-queen@localhost:5984/_users/org.couchdb.user:fernando/

{
_id: "org.couchdb.user:fernando",
_rev: "1-f8d04469cd45d99c9c7cdf0beda1cf99",
password_scheme: "pbkdf2",
iterations: 10,
name: "fernando",
roles: [ ],
type: "user",
derived_key: "1864805738e4515cb677b4b55e99beaa844c0998",
salt: "bba686808c217c316604c84f15f666b5"
}

Ah once more we see that passwords are not stored in plain text. This makes us :-) no?

End of basics tutorial


We are almost done! we still have to see how to manage sessions though. But first let me run quickly and get a refill of the custard while everyone is still sleeping.

How to manage sessions? It is as easy as eating custard.

You know how easy it is to eat custard right? Dip in a spoon inside the bowl, bring the spoon filled with custard to the brim right up to your lips. Part your lips to about the width of the spoon(it might be a difficult estimation if this is your first time but soon you will be doing it without giving it a second thought). Drive the spoon inside your mouth to about the tip your tongue. At this point your sweet receptors should sense the angelic taste of custard which will melt in your mouth as you slowly try to chew it but discover that you can't and regretfully gulp it in hoping the next serve would last longer. Life is a tease. But there it is.

How to manage sessions? Right ho! all you got to do is make a post request to _session endpoint

curl -X POST http://localhost:5984/_session -d 'name=fernando&password=apple'

HTTP/1.1 200 OK
**Set-Cookie: AuthSession=amFuOjUzQzE4MjY4Opq9wMW4hZALkEqoZagHgu_v3-rT; Version=1; Path=/; HttpOnly**
Server: CouchDB/1.6.0 (Erlang OTP/17)
Date: Sat, 12 Jul 2014 18:46:00 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 36
Cache-Control: must-revalidate

{"ok":true,"name":"fernando","roles":[]}

Ah yes we have the cookie

Let us use it to get session information

curl -X GET http://localhost:5984/_session  -H " Host: localhost:5984
Accept: application/json
Cookie: AuthSession=amFuOjUzQzE4MjY4Opq9wMW4hZALkEqoZagHgu_v3-rT"

{
ok: true,
userCtx: { name: "fernando", roles: [ ] },
info: {
authentication_db: "_users",
authentication_handlers: ["oauth", "cookie", "default"],
authenticated: "cookie"
}
}

Nice. usrCtx has the name and the roles. Just what we want. But there are still some adjustments that we need to do.

Some adjustments

I think it is safe to assume that most of us will be using a middleware in front of couchdb. So we must find a way to extract this cookie header from couchdb response and send it to the browser to save it. Thankfully it's not too tough. Let us take a look at the example using nginx and lua (open resty)

location /couch_session{

proxy_pass http://localhost:5984/_session;
}

location /login {
content_by_lua
'
ngx.req.set_header("Host","localhost:5984")
ngx.req.set_header("Content-Type","application/json") 
local res = ngx.location.capture(
"/couch_session",{method=ngx.HTTP_POST,body=cjson.encode({name="fernando",password="apple"})});
local cookie = res.header["Set-Cookie"]
ngx.header["Set-Cookie"] = cookie 
ngx.say("Logged in sir");

';
}

location /logout{
content_by_lua

'
local cookie = ngx.req.get_headers()["Cookie"]
ngx.req.set_header("Cookie",cookie)
ngx.req.set_header("Host","localhost:5984")
local res = ngx.location.capture("/couch_session",{method=ngx.HTTP_DELETE});
ngx.header["Set-Cookie"] =  res.header["Set-Cookie"]

ngx.say(res.body)

';

}

The code above defines two handlers :- login and logout. The approach is similar for both of them

For login

  1. Do a POST request to the couchdb _session api with the credentials. Make sure to set the Host to where your couchdb is running.
  2. Parse the response headers and extract the Set-Cookie header.
  3. Send the Set-Cookie with the response to the client.

For logout

  1. Parse the request headers and extract the cookie field.
  2. Do a DELETE request to the couchdb _session api with the cookie we extracted in the first step. Make sure to set the Host to where your couchdb is running.
  3. Send the Set-Cookie(which will be null) to the client.

And so in a couple of lines we have a fully functional session and user management code on our hands that is secure (remember that all passwords are hashed) and well tested.

Addressing a potential complaint

Now you might be wondering if it is not a bit of an overkill to install a database just to do user management. It depends upon your outlook.

First of all couchdb is very light weight. Unless you look very closely you would not even notice it running on your machine. As a test I added a hundred thousand users with roles:["vassal","mama-mia"] and the reported memory usage for couchdb was a meager 22MB.

If this does not convince you let me present some other useful features. Couchdb can cache authenticated users in memory to reduce disk lookups. This makes session management very fast.

The configuration parameter is auth_cache_size. By default it is configured to hold 50 documents. The actual size of the auth cache would depend upon the maximum number of logged in users that your application experiences at peak time. For this you will have to run tests. But I am going to be reckless here and store all of my 100K users in the cache. For that we will need to override the default configuration.

curl -X PUT http://ABBA:dancing-queen@localhost:5984/_config/couch_httpd_auth/auth_cache_size -d '100000'

Now I will have to make authentication requests to the _session (just like we did earlier) handler to actually put them in a cache. Please read this totally relevant other article while I do that.

28 minutes later

Ahh it's all done now. All of 100000 users are in the cache. Before we dive deeper into the cache it is time for some tests. Now I suppose you don't have a 100k users handy. No problem. One will do.

  1. First kill couchdb.
  2. Start couchdb with an interactive shell. Use sudo couchdb -i
  3. Make a request to couchdb _stats handler
curl http://localhost:5984/_stats

{
couchdb: {
auth_cache_misses: {
description: "number of authentication cache misses",
current: 100000,
sum: 100000,
mean: 8.148,
stddev: 20.337,
min: 0,
max: 62
},
.......irrelevant objects that we don't care about.
auth_cache_hits: {
description: "number of authentication cache hits",
current: 8,
sum: 8,
mean: 0.001,
stddev: 0.045,
min: 0,
max: 3
}
.... irrelevant objects that we don't care about.
}
}

Couch db will respond with some statistics that it diligently collects. But at this moment we only care about two of them. auth_cache_misses and auth_cache_hits. My auth cache misses were sum: 100000 that is none of the users were in cache by default. This number should be 0 for you as we have not tried getting session for a user at the moment. Disregard the auth_cache_hits for now.

Now we will test if the user is actually in the cache.

First the old way


  1. Get session for fernando like we did before-: by issuing a post request to the session handler
  2. Now issue a get request to the _session handler passing in the cookie returned in the first step just like we did before
  3. query the _stats handler. You should see non zero numbers in both cache hits and misses.

Explanation

At first fernando is not in the cache list since we restarted couchdb. Then after we make a request to _session couchdb creates a session for fernando and puts him in the cache and increments the cache miss counter. Finally when we request the credentials of fernando using the supplied cookie couchdb looks up in the cache and returns the results and increments the cache hit counter.


Using couchdb shell

I don't know about you but I am getting mighty tired of keeping track of cookies from my terminal. Therefore we will just use the couchdb shell to trick it into authenticating our user without the cookie. Simply do

couch_auth_cache:get_user_creds("frenando").

and press enter. You should see something like this

[{<<"_id">>,<<"org.couchdb.user:fernando">>},
{<<"_rev">>,<<"1-f8d04469cd45d99c9c7cdf0beda1cf99">>},
{<<"password_scheme">>,<<"pbkdf2">>},
{<<"iterations">>,10},
{<<"name">>,<<"fernando">>},
{<<"roles">>,[]},
{<<"type">>,<<"user">>},
{<<"derived_key">>,
<<"1864805738e4515cb677b4b55e99beaa844c0998">>},
{<<"salt">>,<<"bba686808c217c316604c84f15f666b5">>}]

Now run that _stats handler once more and you should see

auth_cache_hits: {
description: "number of authentication cache hits",
current: 8,
sum: 8,
mean: 0.001,
stddev: 0.045,
min: 0,
max: 3
}

a number greater than 0 for current and sum for auth_cache_hits while there will be no change at all for the auth_cache_miss. I loved authenticating from shell so much that I did it 8 times which is why you see the number 8 in my results. Anyway it's time to dig a little bit deeper.

|

|

|

|

(80 feet below)

|

|

(140 feet below)

How does couchdb cache the users?

Internally couchdb uses ETS (erlang term storage) to hold the users in a cache. ETS is like a mini redis built into erlang. Let's check it out.

  1. Go into the couchdb shell and type ets:i().. This will give you a list of all the ETS tables that couchdb uses. The list will be divided into 5 coloumns: id,name,type,size,mem and owner.
  2. We are only interested in the tables that are owned by couch_auth_cache and especially the table auth_by_user_ets.
  3. Now type the command ets:lookup(auth_by_user_ets,<<"fernando">>). You should see
ets:lookup(auth_by_user_ets,<<"fernando">>).          
[{<<"fernando">>,
{[{<<"_id">>,<<"org.couchdb.user:fernando">>},
{<<"_rev">>,<<"1-f8d04469cd45d99c9c7cdf0beda1cf99">>},
{<<"password_scheme">>,<<"pbkdf2">>},
{<<"iterations">>,10},
{<<"name">>,<<"fernando">>},
{<<"roles">>,[]},
{<<"type">>,<<"user">>},
{<<"derived_key">>,
<<"1864805738e4515cb677b4b55e99beaa844c0998">>},
{<<"salt">>,<<"bba686808c217c316604c84f15f666b5">>}],
{1405,341019,516682}}}]

in the result. Sweet isn't it? Everything nicely stored in memory for quick retrieval.

There is just one more small test that we need to do. That is we need to check if the ets table is actually holding the correct result. This is simple we ask the table to lookup for a user that is not there.

ets:lookup(auth_by_user_ets,<<"some-unknown-user">>).

[]

We get an empty list just like we expected.

How much space the tables use largely depends upon the amount of data that is stored within them. For me with a 100k users the auth_by_user_ets table took 103 MB. The memory usage of couchdb bumped up to 160 MB. This means that I can do authentication for 100k users right from the memory in under 200 MB of space. I'll take that deal.

Some other configuration options

You can also configure couchdb to persist your cookies. All you got to do is set a config property

curl -X PUT http://ABBA:dancing-queen@localhost:5984/_config/couch_httpd_auth/allow_persistent_cookies -d '"true"'

"false"

Now lets test this

curl -X POST http://localhost:5984/_session -d 'name=fernando&password=apple' -v

> POST /_session HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:5984
> Accept: */*
> Content-Length: 23
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 23 out of 23 bytes
< HTTP/1.1 200 OK
**< Set-Cookie: AuthSession=amFuOjUzQzJENTVGOtZa0E2aNe_CtdCnDs5SDAkva02U; Version=1; Expires=Sun, 13-Jul-2014 13:32:15 GMT; Max-Age=600; Path=/; HttpOnly**
* Server CouchDB/1.6.0 (Erlang OTP/17) is not blacklisted
< Server: CouchDB/1.6.0 (Erlang OTP/17)
< Date: Sun, 13 Jul 2014 18:52:15 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 36
< Cache-Control: must-revalidate
<

Ah there is somthing new here. An expires header. Now the browser won't delete the cookie as long as it is not expired.

You can also do other neat things like changing the hashing algorithm and the default _user database. For that you will have to dig through the _config reference

If cookie based authentication is not your style then you can configure couchdb to use OAUTH. But that will be a topic for another post.

Closing

This post turned out to be a lot longer than I had initially expected. The idea was to show how couchdb can be used to simplify the user management process for your application. I think that couchdb offers many advantages that are hard to overlook. It is light. It is secure by default, all the passwords are hashed, it offers an easily configurable environment with which you can enable many other features like session caching for which otherwise you will have to use a third party application anyway. Most importantly it takes just three requests to the http api to have a fully functional register-login-logut workflow in your application. It can't get any easier than that. Life is good. Have fun :)

You might also want to read the official couchdb blog and toptal's fantastic article on 10 Most Common Web Security Vulnerabilities, by Gergely Kalman.


comments powered by Disqus