Thoughts on indexed db

Akshat Jiwan Sharma, Fri Jan 24 2014

Topics discussed in this article

  1. Limitations of local storage.
  2. Object stores in indexed db
  3. Indices in indexed db
  4. Cursors.
  5. Shortcomings of indexed db.

If you want to learn how to code an application with it then check out these excellent guides on mdn and nettuts

While working on my latest project I got a chance to try my hand at indexed db. My previous experience of client side storage was limited to the local storage api. Local storage was good but it suffered from one serious shortcoming , you could only store strings in it. Maybe "shortcoming" is not the right word to use here and it could be that local storage was designed to be that way but this requirement essentially makes manipulating your data cumbersome. It has no efficient query mechanism. The only way that you can retrieve data is by it's key. No matter how cleverly you design your keys sooner or later you will encounter a situation where you will feel that it just falls short and reach the inevitable conclusion that local storage is good only for storing tiny data like say user preferences etc, which is a shame considering that it is widely implemented across all the browsers.

Indexed db, on the other hand is a lot more exciting. First it allows you to store any javascript object. There is no need to stringify before storing and parsing after retrieving your data. A simple feature like this makes working with client side storage a lot more pleasant.

Indexed db saves all your data in 'stores'. A store in indexed db is like a table in sql databases. There can be more than one store in a database. The data is still saved in the form of key value pairs just like in local storage. The value can have any structure. A key can either be generated by a key generator (automatically) , supplied manually or it can be supplied as a key path. A key path as the name suggests defines the path within the value object from where it is to be extracted. Any combination of the above two ways can be used as long as a key within a given store is unique. Let us look at an example.

Suppose you have a simple javascript object {name:"akshat", meta: {email:"akshat_fullmetal@yahoo.co.in"}} that you need to persist . There are a couple ways to do it in the indexed db

  • 1: {name:"akshat", meta: {email:"akshat_fullmetal@yahoo.co.in"}}
  • "hello" : {name:"akshat", meta: {email:"akshat_fullmetal@yahoo.co.in"}}
  • "akshat_fullmetal@yahoo.co.in" : {name:"akshat", meta: {email:"akshat_fullmetal@yahoo.co.in"}}

The first approach uses an auto generated key. To keep the values unique these keys are incremented automatically. Every following key will have a value greater than 1.

The second approach supplies a key manually. Here it becomes the responsibility of the programmer to ensure that no two keys are same within the store.

In the third approach we are using a key path meta.email to get the key from the value it stores. This can be a good strategy if you can ensure that one of the values in the object will always be unique.

The keys can be an empty string, an array, a date or even a javascript object. So the takeaway from this is that you can use any way to create keys within a store as long as it is ensured that they will be unique to that store.

For any object store you can create one or more indexes. Indexes allow you to search for your data by value. Time for another example. Suppose you have the following data in your database

{name:"akshat", meta: {profile:"gamer"}}
{name:"jiwan", meta: {profile:"writer"}}
{name:"sharma", meta: {profile:"gamer"}}
{name:"less", meta: {profile:"musician"}}
{name:"dez", meta: {profile:""}}

Suppose we wanted to find the people by their profile. To do this we create an index on the store with the key path as meta.profile. Within the index all of your data will be sorted by the key like:

"gamer": {name:"akshat", meta: {profile:"gamer"}}
"writer" : {name:"jiwan", meta: {profile:"writer"}}
"gamer": {name:"sharma", meta: {profile:"gamer"}}
"musician":{name:"less", meta: {profile:"musician"}}
" ": {name:"dez", meta: {profile:""}}

In an index a key can be empty or non unique. However if you want you can supply a unique constraint at the time of index creation.

Another cool thing about index db is that it is transactional in nature (all of the transactions are asynchronous) which ensures that integrity of the data is intact at all times. The api is very similar to the ajax api. You issue a request and the result is available in it's call back. You can query directly from the store to get the value back for the queried key. However to query from the index you need a cursor.

A cursor is simply an iterator that will walk through the elements of the index for as long as you want it to. Consider the case where we need to find the gamer "akshat" . Now we don't know with what key he has been stored in the database so we will just search him from the index. Here are the steps we need to take.

  1. Create a cursor on the index "profile"
  2. Check the value returned by the cursor.
  3. if(!value.name === "akshat") continue cursor; else return value

And that's it. The only thing to remember here is that just like everything else the code for cursor will run inside a callback.

The cursor has some other interesting properties as well. First you can specify the direction the cursor starts reading from. By default the direction of cursor is from the beginning and it starts reading ahead. 'prev' starts the cursor from the end of the index and reads behind. You can also skip the cursor by an integer value by using cursor.advance(int) which will make the cursor start from that value. It is important to note that cursor only skips ahead, that is in the forward direction. No point trying to skip it back (I lost a few hours trying to do that)

Indexed db is schema less. But the method to upgrade schema or add new stores is really painful. You have to rely on onupgradeneeded event and it can be a hassle to track what client needs update. So if you have shipped your application all the changes to the database will have to be made through the upgrade event. This can be a bit of a hassle since a part of your database definition is in one event and another part is in an entirely different event.

My overall experience with indexed db has been quite positive. It is a bit saddening that it is not supported fully in all the browsers (safari,android) but I am willing to bet on it. My new project is based heavily on indexed db so I hope that it gains better adoption soon. I will be sharing some tips like pagination with indexed db in the coming posts so stay tuned.


comments powered by Disqus