/* When a length of a string object stored on disk has the first two bits * set, the remaining two bits specify a special encoding for the object * accordingly to the following defines: */#define REDIS_RDB_ENC_INT8 0 /* 8 bit signed integer */#define REDIS_RDB_ENC_INT16 1 /* 16 bit signed integer */#define REDIS_RDB_ENC_INT32 2 /* 32 bit signed integer */#define REDIS_RDB_ENC_LZF 3 /* string compressed with FASTLZ */

Editorial Note

This articles was originally at wiki.asp.net but has now been given a new home on CodeProject. Editing rights for this article has been set at Bronze or above, so please go in and edit and update this article to keep it fresh and relevant.

Working with NoSQL Databases

How to get started with NoSQL?

Since 2009, NoSQL databases becomes more and more popular. But why?

No usage of SQL, that means

Less complexity

Better portability

Boundlessness

User-friendliness

Most databases are Open-Source

Performance

Scalability

Famous companies like Twitter, Facebook and Amazon are using NoSQL databases.
What sorts of NoSQL databases are used today?

Key-Value stores

Easy to implement

Only key-value-pairs can be stored

Difficult to build complex data structures

Column stores

Columns don’t have to be defined in advance.

A row can have different numbers of cells

Document stores

Like key-value stores, but allows nested values

Graph databases

Objects and relationships are modelled and persisted as nodes and edges of a graph

We need to use all the tools at our disposal to develop faster and more robust applications. One of the ways we can achieve this is by using caching. Previously, under the System.Web.Caching.Cache namespace, the new -4.0-System.Runtime.Caching.Memory is more mature and still as powerful as its ancestor. It’s really easy to use and I think that every asp.net programmers have already used it. So, I won’t speak too much about this here.

The default caching strategy is the cache-aside programming pattern: This means that if your data is not present in the cache, your application, and not something else, must reload data into the cache from the original data source.

This works very well for most common use cases, but there is a hidden trade-off: caching means working with stale data. Should I increase the cache duration? Should I keep short TTL value? It’s never easy to answer to these questions, because it simply depends on your context: number of clients, user load, database activity…

Another common problem is how to update/remove instantly something from the cache on all your cache clients, such as this typical request from Product Owner ” I want a parameter to be instantly updated on all our XXX servers, but do not fear, I will only change it a few times per year”

I will investigate a not so new concept in this post: setup local cache invalidation using Redis.

What is Local Cache invalidation ?

Cache invalidation is a process whereby entries in a cache are removed/deleted.
This concept is not specific to .net but to all technologies.
The idea here is to send invalidation message to each server and do not wait –as usual- item expiration.

Why Redis ?

Redis is an open-source, networked, in-memory, key-value data store with optional durability. (Note familiar? Read the great documentation here). It’s now the most popular NoSql database, and many internet leaders are using it (Instagram, Stackoverflow, Twitter, GitHub, Tumblr, Pinterest, …). For the joke, nothing will be stored in Redis, and we will only use the PubSub feature.

And yes, since I discovered Redis, I’m a big fan

Implementation

Requires : .net, Redis & Booksleeve (high perf C# redis client).

The concept is quite handy : Redis events (pub/sub) give us an easy way to broadcast something to all nodes (lossely coupled), so they can drop their local copy – meaning: next time it is needed we’ll pick up the new copy from the data tier.

So, The redis pub/sub events are used to invalidate the cache for a given key from one node (the one that knows the state has changed) immediately to all nodes. A node could be a front web site, a back app or any redis client.

Local cache invalidation is done with ChangeMonitor class : “Provides a base class for a derived custom type that monitors changes in the state of the data which a cache item depends on”. It’s the base class of FileChangeMonitor and SqlChangeMonitor.

A few details:

A single connection is opened to redis, as BookSleeve is a thread-safe multiplexer

As there is a single connection to redis and possibly many monitors, I use a topic-based observer pattern to manage notification inside the application

I send only invalidation message following this format : channel = “invalidate”, and message = “keytoremove”.

Introduction

In this article I will show how to run the Window version of Redis Server, or other executables, as a Windows service.

Here at CodeProject, we use Redis as a distributed cache. We store massive amounts of information such as Articles, Forum Messages and retrieve these items from the cache rather than the database.

Redis alllows us to store and retrieve full documents, rather than querying SQL for the various pieces and composing and formatting the document on each request. This is possible on our site because most of the information is read a lot more than it is written, and some information, such as number of views, can be a little stale.

While in production we run Redis on a Linux server, in out development environment we are running the Microsoft port of Redis on a Window 7 desktop. The problem is that Redis is not designed to be run as a Windows Service. This means that someone had to logon and run the Redis executable. This was fine when Chris was the only one logging into the server, but last week, I had to connect to install a copy of our Search Server for development purposes. Needless to say, this logged Chris off, and killed the Redis Server. I restarted it under my session.

Back on my machine, I am fixing a subtle caching bug, and when I start testing, the code is acting like there is a connection failure to the Redis Server. As the code I am changing is related to the detection of problems with the connection to Redis, I spin my wheels for half an hour or so before I realize that Chris has remoted into the server, killing my session and the Redis Server.

Both Chris and I tried a number of recommended methods for running Redis as a Windows Service, without any success. Having written several Windows Services to support various CodeProject processes, I decided to write a utility which would allow us to install and run an exe as a Windows Service. This utility is called Exe2Srvc.

Using Exe2Srvc

Exe2Srv is a program that can be run as either a console application or installed as a Windows Service. This application reads the path to an executable, and the command line arguments from it .config file and then starts a the executable in a new Process.

The simplest way to use the executable is to

copy the files in the binfiles download, or from bin/Release from the compiled source, into the directory containing the executable you wish to run as a Service.

Edit the “Cmd” and “CmdArgs” values in the Exe2Srvc.exe.config to contain the full path to the executable, and the command line arguments required.

Run the Install.bat file from a ‘Run as Administrator’ command shell.

Use the Service Manager to:

set the start mode to Automatic

set the Recovery options. I usually set them to “Restart Service”.

start the Service.

For my tests, I downloaded Redis from Nuget, and copied the files from the /packages/Redis-64.2.6.12.1/tools under the solution directory to C:/Redis. I then copied the files fromExe2Srvc\bin\Release to the same directory.

How Exe2Srvc works

Basically you create a console application, and change the Program class to derive from ServiceBase. In the Main, the Environment.UserInteractive property is used to determine whether the program is being run from the command line, or run as a Service.

The required command and commandline parameters are read from the LoadConfiguration method.

This is called from the OnStart method which is called from Main when started as a console application, or by theService infrastructure when run as a Service. OnStart runs the executable in a new Process as shown below.

///<summary>/// When implemented in a derived class, executes when a Start command is sent to the
/// service by the Service Control Manager (SCM) or when the operating system starts
/// (for a service that starts automatically).
/// Specifies actions to take when the service starts.
///</summary>///<paramname="args";>/// Data passed by the start command.
///</param>protectedoverridevoid OnStart(string[] args)
{
if (Environment.UserInteractive)
{
string message = String.Format"Starting {0} at {1}.", _serviceName, DateTime.Now);
Console.WriteLine(message);
}
// loading the configuration file info here allows the service to be stopped,
// the configuration modified, and the service restarted.
LoadConfiguration();
// Start the executable.
ProcessStartInfo procInfo = new ProcessStartInfo(_cmd);
procInfo.UseShellExecute = false;
if (!string.IsNullOrWhiteSpace(_cmdArgs))
procInfo.Arguments = _cmdArgs;
_process = Process.Start(procInfo);
}

When the Service is stopped, the OnStop method is called. This kills the Process, waits for it to terminate, and disposes of the Process. Make sure you wait for the Process to terminate, failure to do so will result in improperly stopped Services that are difficult to remove and fix.

///<summary>/// When implemented in a derived class, executes when a Stop command is sent to the service
/// by the Service Control Manager (SCM). Specifies actions to take when a service stops running.
///</summary>///<remarks>Stops the background tasks.</remarks>protectedoverridevoid OnStop()
{
if (Environment.UserInteractive)
{
string message = String.Format("Stopping {0} at {1}.", _serviceName, DateTime.Now);
Console.WriteLine(message);
}
// Kill the process
if (_process != null)
{
_process.Kill();
_process.WaitForExit();
_process.Dispose();
_process = null;
}
}

Points of Interest

I’ve attempted to make this as simple and flexible as possible, but it only meets my original requirement of being able to run the Windows version of Redis as a Windows Server. If you have additional requirement, feel free to modify the code to match your needs.

Introduction

In this article, I would like to describe my experience with installing and configuring Redis server in most compact way. Also, I would like to do a brief overview of usage Redis hashes and lists in .NET/C# client.

Redis Server Protection: Password, IP Filtering

The primary way to protect Redis server is to set IP filtering using Windows firewall or properties of the active network connection. Additional protection can be set using redis password. It needs to update the Redis config file (redis.conf) in the following way:

Redis Server Replication (master – slave configuration)

This technique allows creation copy of the server data into the synchronized copy, this means that each time when master is modified, slave server gets notification and is automatically synchronized. Mostly replication is used for read (but not write) scalability or data redundancy and for the server failover. Setup two instances of Redis (two services on the same or different servers), then configure one of them as slave. To make Redis server instance to be slave of another server, change the config file in this way:

Typed entity sets are more interesting and practical, because they operate with exact types of objects. In the code sample below, there are two classes defined Phone, and Person – owner of the phone. Each phone instance has a reference to the owner. This code demonstrates how we can add, remove or find items in the cache by criteria:

NOTE, that password is optional, based on the server authentication. It must be replaced with real value, or removed, if Redis server doesn’t require authentication. server attribute and port also have to be replaced according to concrete values (default port is 6379). Then in the project, you can use the session state:

Redis sets are useful when it needs to store associated sets of data and gather statistical information, for example answer -> queustion, votes for an answer or question. Let’s say that we have questions and answers, it needs to store them in the cache for better performance. Using Redis, we can do it this way:

Attached Sources Description

List of included packages is in the packages.config,Funq IoC configuration, registering types and current controller factory – are in the Global.asax (property dependency injection)
Usage of a simple client – in the home controllerUsage of IoC based cache in the Question and Answer controllers, and Global.asax application file. To see how it works, you can run the project, and open in the browser following URL:http://localhost:37447/Question/GetQuestions?tag=test .
You can play with tags, like test3, test11, test2, etc.Redis Cache configuration – in the web config (<system.web><sessionState> section) and in theRedisSessionStateProvider.cs file.
There are a lot of TODOs in the MVC project, so if you want to improve/continue please update it, and upload.

I would much appreciate if someone could help build MVC application with simple UI, using Redis (with Funq IoC) cache. Funq IoC is already configured, example of the usage is in the Question controller.

NOTE: Samples were partially taken from the “ServiceStack.Examples-master” solution

Conclusion. Optimization Caching in Application with Fast Local Cache

Since Redis doesn’t store data locally (no local replication), it might make sence to optimize performance by storing some light or user – dependent objects in the local cache (to skip serialization to string and client – server data transfering). For example, in the web application, it is better to use ‘System.Runtime.Caching.ObjectCache‘ for light objects, which are user dependent and used frequently by the application. Otherwise, when object has common usage, large size it must be saved in the distributed Redis cache. Example of the user dependent objects – profile information, personalization information. Common objects – localization data, information shared between different users, etc.

Introduction

What do you do when your Removal Device or USB gets affected by a virus and files are hidden? Most of the Antivirus scan the device and delete the virus only. So we can’t see the files in our USB device which is hidden by the virus. So here, the “Ultimate USB Reset Attribute” comes with a powerful option.

Idea Behind the Code

I just embed my code with the existing one which is already available in here (CodeProject). So my first sincere thanks to CodeProject, without this I could not do it.

Using the Code

Using this code, the number of USB or removal devices is listed in a dropdown which is currently connected to a computer.