after_fork and redis

bradford <fingermark at gmail.com> wrote:
> I'm using unicorn w/ a rails app. I have the following in my
> environment.rb $redis = MyApplication::Application.config.redis and in
> production.rb I have config.redis = Redis.new(host: "localhost").
>> I've read I'm supposed to $redis = Redis.new(host: "localhost") in
> after_fork when preload_app is true.
>> When I don't do this, each worker/pid seems to have their own redis
> instance. So, why is this needed? Here's the logs of me printing out
> $redis.client.inspect when both $redis = Redis.new in the after_fork
> and just $redis = Redis.new in the environment.rb.
(Disclaimer: I still haven't used Redis, but similar knowledge applies
to every TCP-based service)
If a Redis client opens a TCP (or any stream) socket connection before
forking, all the children that are forked will share that _same_ local
TCP socket. Sharing stream sockets across processes/threads is nearly
always a bad idea: data streams become interleaved and impossible to
separate in either direction.
> No after fork
>https://gist.github.com/bc2c2a3bda01c35730e2>> After fork
>https://gist.github.com/0c96550660d3926ffe16>> The only thing different I notice is Connection::TCPSocket:fd is
> always 13 w/ the after fork.
FD shouldn't really matter. FDs get recycled ASAP once they're closed
(and GC can automatically close them). What you want is a different
local port for the TCPSocket on every worker process, and the after_fork
hook will give you that.
Btw, you can demonstrate FD recycling with:
require 'socket'
loop do
c = TCPSocket.new("example.com", 80)
p [ :fileno, c.fileno ]
p [ :addr, c.addr ] # (local address)
c.close
sleep 1 # be nice to the remote server
end
You'll see the same FD recycled over and over, but c.addr will
have a different local port.
You can see if after_fork is working correctly by checking the output of
tools like `ss' or `netstat':
(something like: ss -tan | grep -F :$PORT_OF_REDIS)
You can also check connections on the Redis server/process itself
using "lsof -p $PID_OF_REDIS_DAEMON"