Redis storage structures

- [Instructor] Welcome to Redis Storage Structures,part of the course Build Complex Express Siteswith Redis and Socket.io.In the previous video,we looked at many of the commands for Redis.In this video, we're gonna take at lookat designing a data we'll put into Redis.This is important as it is a different paradigmthan a classic database.We will do this by discussing what is differentand then how that changes what we do.First off, let's make sure we are on the same page.

When discussing the classic database,we are talking about things like relationships and indexes.Now let's use the example we've been usingfor this entire section of setting up a user database.In the classic database, we would have a user tablewhich would have columnsthat hold user ID, username, password,when they signed up, et cetera.Then we would have a person tablethat would hold the user's name,date of birth, stuff like that.Then these would be joined by an ID.We would need to look up users by ID and username,so we would put an index on those fieldsso that any queries would execute quickly.

Now let's compare this to Redis.We could essentially mimic a table by using hash.This would allow us to add columns or fields in Redisto store similar information,but what about relationships and joints?There is nothing that matches that in Redis.Also, what about looking up a user by either ID or username?We cannot query a hash by field.Everything in Redis can only be looked up by its key.This can definitely create problemsif we are not designing our datato match how Redis stores data.

So let's talk about how to actually do this.First of all, using a hashto store information about an object,or user in our case here, is fine.We just need to make sure we use a good key.For example, we will use a user ID,so this would mean that a keywould be something like user:1, with one being the ID,and then as different users are stored,that ID would be diff.Remember that when looking up data in Redis,when you know the key,the lookups are pretty much instantaneous.

It is very, very fast when you know the key.So that kind of mimicswhat we're talking about with the databasein that we can have a row with a bunch of columns of data,but what do we need to doif we need to search by username?We can't take that hash and then look up inside of itand say, "I don't know the user ID"which we would have as a key."I just know the username."Well, we would do this by creating another keythat then would point back to the hash.So if for example the username was josh,we would create a key called user:joshand then we would set the value of that to user:1.

We would then look up the username keyand then use that to look up the user information.We could do the same thing if we had,for example, like a person hash that we'd look up.Now what's great about thisis this essentially is pretty scalable.Any time we wanna index any field in a hash,we just create another key namespace.We would just have to make surethat we name these key namespaces uniqueand be able to look them up later.And all we'd have to do for the valueis always just point back to the official hashor the official row of the data that we want.

So let's go ahead and fire up Redisand take a look at how to do this.We're at the console.Again, I want to highlight that we are using Dockerand all the Docker commandsare in a text file at the root of this section.So here to run it, we're gonna run our Docker commandthat connects up to our Redis server that we have set upand we can run Redis commands.Okay, so we're ready.And just like the Docker commands,all these commands that I'm gonna run noware going to be in a file called Redis Commandsthat you can use.

So let's set up a couple of those user.We will use hmset, which again is hash multi-field setup.So that's our key.This is saying this is the user namespace,user key namespace, if you will,and this is user:1.And I'm gonna do username joshand gender male.Okay. We will do a couple more.We will do user:2,set the same thing.

We will say username chris,and we will say gender female.And now we're going to do a similar thing.This time, we're gonna say user:3.We're gonna say username is chris2and that the gender is male.Okay.If we do hgetall,we'll do user:1,we can see everything is set.So here, we have a simple three-row user namespace.

If we know the user's ID, we can easily look it up.We just say hgetall user, and then the user ID.So how do we look up by username?Well, we're essentially gonna have to create some more keysto point back to these rows.So we will do username, and then josh.And then we will point it to the correct row.And then we will do this for our other users,so we have a chris as a username, and that is two.

And we have a chris2, and that is three.Now we have these indexes, so now we can run a getand then we can say username:josh,which returns user:1.Then we would run hgetall user:1and now we have the user's information.So for example, if someone was logging inand then we got their username passed back in from a formand now we have it in Node.js,we can now then say, "Well okay,"we know they typed josh in as their username.

"We can run get username:josh"and look up the key that will then point us back"to the user row."So hopefully this starts to make sense.If we ever want to index on something else,we just go ahead and add rows with that.And again, the key thing is making unique keysthat we can then replicate when we have to look up the data.So we have a username,so we put username, as a namespace essentially, colon josh.Let's do a little more complex lookup.

Let's say you want to look someone up by their first name.So we really have no real wayof adding a key for every name.Again, this is somethingwhere people can have the same name,so we can't have a key that says this is the chris.That is not gonna work.So what we need to do is use a set.The set would store the nameand then we would just keep adding the user keysof people that have the same name.So let's do that.

We will do the set addand we will create one called names:josh'cause we have someone named josh,and we would set the user key.Then we would do, again, set add,and we would say names, we would say chris,and here, we have user:2 and we have user:3, so there.So we can look at theseand if we needed to say who is everyone named chris,we just run it, and there we go,we have user:2 and we have user:3.

So let's get even more complex.Let's say we have a question that says we need to knowsomeone that is named chris and that is a male,well how would we look this up?Again, we can't just query the hash.So to do this, we're gonna have to add two more sets.Again, because multiple people can be maleand multiple people can be female,we need to use a setbecause we can't use just a key to tie that together.

So we will set gender, we'll say male,and user:1 is male and user:3 is male.So we will do the same this time,say two, and we wanna say female.So now we have our set set up.Remember when we talked about commandsthat when you have sets, you can run set theory,and what is a set theory?If we need someone that is both named chris and a male,well we're looking for an intersection.

So that's what we'll run.sinter, and then we say names:chrisintersected with gender:male.We run that, we get one user back,and that should be the one that matches.If we do an hgetall,user returned from that,okay.Now technically, we didn't set like a first name field herebut we know that username chris2is named chris and gender male.

So this is how we can even get even more complex queriesout of data that is sitting in Rediswithout having to set indexes on fields.Finally, we will look at one last use case.So let's say we need everyone that has signed upin the last week.Again, everything we have covered so farwould not be sufficient for tracking this.We can't set a key.We can't really create a setthat stores every weeks of sign-upsbecause this question is really arbitrary.

What if we wanted everyone in the last yearor everyone within the last day?Are we gonna set up sets for every one of those?No, we're not.So this is where we use a sorted set.We can use a timestamp.This is usually the amount of seconds or millisecondsthat have gone by since January 1st, 1970.Now this is a Node.js course,so we're gonna use JavaScript's implementation of that,and that is in milliseconds since that time.

And then once we've set up a sorted set,we can use the sorted set rangebyscoreand then figure out who has signed up in the last week.So let's look at an example of this.Now I've copied some of these commandsbecause they have specific timestamps.So this is real timestamp of time right around now.Remember zadd is sorted set add.

We're gonna create one called loginsand that time is roughly just a few minutes ago.So we're gonna say user:1 signed uproughly a few minutes ago.Then we're gonna run another oneand this says that they signed up roughly 15 days ago.Now again, to get these timestamps,we were using JavaScript.We can just say, "What's the time now?"Get that number and then use that as the score."And then we're gonna use threeand say that was roughly five days ago.

So at this point, we have three items in our set hereand if we say who signed up in the last week,user:1 and user:3 should return.So to look that up, again I have this copiedbecause it has specific timestamps in it to make this work.We are gonna say zrangebyscore,look in the set loginsand then the first time is roughly seven days ago.

And then the other timestampis roughly just a few minutes ago.So between these two, user:1 and user:3 should come back.So we ran itand we see that is correct.Now here, we were just using timestampsbut again, this could be anythingthat we could create a number for and rank people byand then once we have that numberand we can compute that number within a range,we now have complete control of how we look it up.

And again, we have user:3 and user:1,so we can now immediately go to the hashesand determine what that is.Again, knowing that when you have that key,the lookup is extremely fast.In summary, in this video, we discussed the differencebetween classic databases and Redis.Then we covered ways to correctly put data in Redisso that we can look it up later.Now, this is the last video of this section,so in summary of this section,we have laid a nice foundation for Redis.

We have discussed that Redisis an in-memory data structure store,and what that means.Then we discussed the data structures,then we moved on to the commands,and finally, we talked about how to design datato go into Redis in a usable way.In the next section,we will discuss another great feature of Redis,which is messaging,or as it is known in Redis as Pub/Sub, Publish-Subscribe.

This is a great way to send messagesbetween different servers and processes.

Resume Transcript Auto-Scroll

Author

Released

6/2/2017

Learn how to use the Express framework in conjunction with Redis and Socket.IO to make robust and dynamic sites. This course explains the purpose and use of commands, data types, and the Redis storage structure. See how to add basic messaging and perform installation and integration steps. Find out how to use Node.js compatibly with Socket.IO and how to implement a multi-server setup. By the end of this course, you will have the knowledge needed to be able to build—in real time—a full-featured web application that supports two-way communication.

Note: This course was created by Packt Publishing. We are pleased to host this training in our library.