The blog that aims at the largest possible balance of pleasure over pain

Main menu

Tag Archives: elasticache

Post navigation

After creating an a Elasticache instance for Redis and loading it with data dynamically, it became apparent that it would be useful to warm the cache with a large dataset to further improve performance. Redis recommends using their Luke protocol to accomplish this by creating a text file with the data and then use the redis-cli to import that into Redis directly. This approach is significantly faster than doing it programmatically. Here I’ll elaborate on the steps I took to do that.

The first step is to create a text file with the data you want to import. The basic structure is:

However, the devil is in the details. First, you must escape all the quotes in your JSON string. Also, you must be sure to have a carriage-return/line-feed after each line or your data won’t import. This is required even after the last line in your file.

Once you have your file, you then need to create (or use an existing) EC2 instance to call your redis instance and import the data. On your EC2 instance you first need to install gcc to build the redis-cli and then make the redis-cli code. Here are the steps:

That’s it! If you want to schedule this on interval, you can simply create a cron job to refresh your cache as needed or use an event driven architecture to update the cache based on changes from the source system.

Post navigation

Hi, I'm a Solutions Architect for a major Telecom company. I design and develop software solutions leveraging several Java and JavaScript based platforms including ExtJS, Node.js, Spring, etc. I am also experienced using relational and NoSQL database solutions like MongoDB. I help evangelize Rich Internet Applications and open source technologies in Web application development, Web services, search, and developer collaboration. I graduated from the University of Michigan with a M.S. in Computer Science.
Note these views are my personal views and do not necessarily reflect the views of my employer. All code on this site is licensed under BSD.