SPrabhu's Blog

Monday, March 05, 2018

Samba uses the tevents library to handle new incoming connections. The Samba server makes use of event handling to perform tasks such as creating a new process for each new client connection and to further handle new requests made by this client. Before the Samba server can handle these events, the events meant to be caught have to be registered along with a handler which handles these events.

Friday, July 15, 2016

I have been using an energy meter for a while to track the energy consumption at home. The system consists of a transmitter and a receiver. The transmitter is placed in a meter box with a clamp on the live feed going into the house and the receiver sits in my office. The transmitter transmits the energy consumption in watts(joules/sec) at certain intervals while the receiver converts these readings from the transmitter to energy consumed in KW-hr and tracks the consumption for a day, week, month and year.

While this system works and allows me to track consumption, I wanted a bit more flexibility in the way energy consumption was displayed. For example, one of the requirements was to break down the energy consumption for various times in the day.

There were multiple solutions available.
1) Use energy consumption monitors which also provide a cloud based service reporting on energy consumption. I rejected this as I am locked into using a particular vendor who may one day decide to shut shop.
2) There are some monitors available which allow you to download data off the the receiver which you can then use. This is a manual process which requires you to connect to the device with a USB cable to pull the data.
3) Find another method to capture the data sent by the transmitter directly and process it. This is the method I finally chose.

The energy consumption monitor I have is an Efergy Elite Classic(Bought second hand on ebay for GBP 21). The transmitter operates in the 433 MHz range and can be set to transmit data in 6/12/18 second intervals.
I also purchased a RTL-SDR USB dongle(GBP 11.99, again bought off ebay) which allows me to tune into various frequencies using software on a computer. You can also use the software(I use gqrx on linux) to scan and determine exactly which frequencies have data being transmitted on them.
An idle raspberry pi was requisitioned for this project and will happily use the RTL-SDR dongle.

Looking around for ideas, I came across the project rtl_433 which uses rtl-sdr libraries and has a large number of decoders already in place. It scans the 433 MHz frequency and automatically decodes data packets for any of the various supported devices. The utility already supports the Efergy energy consumption monitor which is a big plus.

Once the dongle is connected and rtl_433 built, we can simply decode calls over the wire with the command

rtl_433 -f 433550000 -R 36

By default, the rtl_433 utility tunes to 433920000 Hz. The Efergy transmitter transmits on 433550000 Hz I have to pass this frequency with the -f option. This is where I used gqrx to determine the frequency which Efergy transmitter transmits on.

The mains power in the UK is at 240 volts and it shows my consumption at that voltage to be 201.60 watts. The data in watts is then further processed using a combination of grep, sed and a python script and upload it to Thingspeak.

Thingspeak is aimed at IoT applications. Thingspeak provides you a REST api to record data. This data can be analysed and used as needed. As a bonus, there are a few Android apps available too which can read the data off the Thingspeak platform.

I first started recording the data in watts updating the Thingspeak channel every time the transmitter transmitted. However the resulting graph was difficult to read because of the huge spikes every time I turned the kettle or the microwave on. To give me a better understanding of my energy consumption, I converted the watts to KW-hr and update the Thingspeak channel every hour instead. This gives me consumption for an hour instead. I preferred looking at the readings in KW-hr as my energy company charges me per KW-hr.

Sunday, June 26, 2016

MQTT is a lightweight protocol on top of TCP/IP which is used to transfer data between nodes. It was built to be used by devices having low network bandwidth. The low overheads in using this protocol has made it a popular choice for use with IoT devices.

The protocol uses a subscribe/publish method wherein both the data source and the recipient subscribe to a topic on a broker. This model is similar to twitter where a tweet sent by a user can be read by multiple users who subscribe to the user.

A broker is a software run on a more capable hardware which acts as a data distributor. A topic is similar to a subject and is built in a tree structure with its components separated by a '/'. Wildcards are allowed when describing topics to subscribe to. Mosquitto(http://mosquitto.org/) is the most popular opensource broker at this moment.

Clients connect to the broker and subscribe to the topic. A client wishing to transmit its data(the sender) will publish data for the topic on the broker. Subscribers to the topic on the broker receive the data and process it further. MQTT is data agnostic and the data can take any format the sender wishes.

Some use cases for MQTT are
a) Sensors such as temperature sensors periodically sending information about its surroundings.
b) Sending commands to a device from various sources.

An example project which uses MQTT in an arduino setting is available on the Make website. Here the automation software - openhab is configured with MQTT bindings to send commands specifying the colour required to the broker(mosquitto) running on a raspberry pi. The ESP8266 connected to a LED strip subscribes to this broker reads the colour data off the broker and sets the light colour accordingly.

More information is on the protocol is available in the mqtt man page at
http://mosquitto.org/man/mqtt-7.html

Wednesday, June 01, 2016

The article is from my notes on setting up a RHEL 7 build environment on a RHEL 6 machine. This can be used to setup a build environment for any of my guest on the host machine.

I have a RHEL 6 based server grade machine which hosts vms belonging to RHEL 6, RHEL 7 and Fedora rawhide. In some instances, I need to be able to compile kernels based on the git tree for a certain guest machine. Trying to do this on a vm usually means I need to export the git tree over NFS and given the resources the vm has, takes a long time. The idea is to compile kernels on the more capable host machine and simply run "make install modules_install" on the guest system.

5) Switch over to user mock and initialise the mock environment using the command

$ mock --init

6) Switch over to the mock shell

$ mock shell

At this point, we are in a chroot shell as a root user in the mock EPEL 7 environment. The directory containing the git tree is available at the directory specified in the mock configuration file. In my case, this is /scratch.

7) On the guest machine, make sure you mount the exported NFS share and run

make menuconfig/oldconfig/localmodconfig

to setup the .config file.

8) Switch back over to the mock environment on the host machine and run

make; make modules

to build the kernel and the modules.

9) Once the build is done, on the guest vm, go back to the git tree and run

Thursday, August 20, 2015

The investigation is driven by a user request to reduce the time spent by the client waiting for a request to an unresponsive server.

The client awaits responses by the server in the kernel thread "cifsd" which runs the call path
cifs_demultiplex_thread()->cifs_read_from_socket()->cifs_readv_from_socket()->kernel_recvmsg()

We investigate 2 scenarios
a) The server is orderly shutdown ie. the server closes the socket.
b) The server is abruptly taken off the network ie. it has no chance to orderly shutdown the socket. This is simulated using iptables.

In both cases we assume that the client has a share mounted from the server.

A call is made to cifs_reconnect() where an attempt is made to continuously reconnect to the server in the "cifsd" thread every in the code path
cifs_demultiplex_thread()->cifs_read_from_socket()->cifs_readv_from_socket()->cifs_reconnect()->generic_ip_connect()

While the server is down, generic_ip_connect() continously returns -EHOSTDOWN(111). It will sleep for 3 seconds between each reconnect attempt.

While the attempts to reconnect are made in the cifsd thread, any attempt to perform an operation on the share results in the thread calling smb_init() which calls cifs_reconnect_tcon()
ex: when calling ls -l - CIFSSMBUnixQPathInfo() -> smb_init() -> cifs_reconnect_tcon()

In the while loop above, we return after 10 seconds in case of a soft mount or continously retry in case of hardmounts.

Conclusion:

In cases where the server is shutdown orderly, the client immediately attempts to reconnect. It continues doing this through the cifsd thread keeping an interval of 3 seconds between each attempt. Any attempt to perform an operation on this mount on a soft mounted share(which is the default) will fail after waiting for 10 seconds for the reconnect attempts made by the cifsd thread to succeed.

Server is abruptly shutdown

The server tcp connection is represented by struct TCP_Server_Info. This contains a variable "unsigned long lstrp" which contains the jiffies at which the client last got a response from the server.

It is this last response variable server->lstrep which is used to determine the responsiveness of the server.

Before we look at how it is used to determine the server responsiveness, we should also take a look at the echo service. This is the keepalive service used to "ping" the server at regular intervals in case of inactivity. This is done by setting up a work on the cifsiod_wq workqueue which is called ever SMB_ECHO_INTERVAL(60) seconds.

Note that we send an echo call every SMB_ECHO_INTERVAL. So in case of an inactive connections, we should receive a reply atleast after every SMB_ECHO_INTERVAL(60) seconds. The timeout for receives on the socket is set to 7 seconds when setting up the socket in generic_ip_connect(). If no replies have been received for 7 seconds, kernel_recvmsg() timesout and we redo the for loop which checks if the server is unresponsive again.

Any filesystem activity performing an operation on this share uses functions such as SendReceive(), SendReceive2() or SendReceiveBlockingLock() to send over commands to the remote server. They then wait for a response from the server by calling wait_for_response().

It continues waiting for a response until we finally timeout in the cifs_demultiplex_thread "cifsd" in the call path cifs_demultiplex_thread()->cifs_read_from_socket()->cifs_readv_from_socket()->server_unresponsive(). Once the function server_unresponsive() confirms that the server hasn't replied for atleast 2 * SMB_ECHO_INTERVAL, it calls cifs_reconnect() to mark the TCP_Server_Info as needing a reconnect.

Sunday, June 28, 2015

This recipe is from Shyamala Kini from Konkani Amchi food.
Boil
2 cups of water.
Once it comes to rolling boil ,add salt to taste,
little jeera, a pinch of hing, pinch of chilly pwd and 1 cup of rava/sooji.Switch off the cooker and mix well into a consistency of a dough (something like upma/kesari consistency.)Cool it till lukewarm.Then add 1 cup of rice flour and a spoon of butter and some sesame seeds.Knead/mix well.Put it into moulds and deep fry them on medium flame.Allow it to cool before tasting