Journalhttp://prokash.squarespace.com/journal/Thu, 11 Jan 2018 03:06:15 +0000en-USSquarespace V5 Site Server v5.13.492-285 (http://www.squarespace.com)Roller CoasterProkash SinhaThu, 11 Jan 2018 02:48:49 +0000http://prokash.squarespace.com/journal/2018/1/10/roller-coaster-1.html130723:1174877:36020226It's been almost 30 years, when I first heard "Only thing is constant in High Tech is Change".

I can't think of anything better than that to reflect our experience, hence the notion about the state of affairs of it.

So what are the Changes, that we should be worried, interested, or otherwise curious ? - It's just two things. First Deep Learning, next Quantum computing. In another 5 years, they would be the main stream of knowledge(s) one would be required to someone who would - For me only thing that is constant is Change.

I like those two area, just because I've some bias in Mathematics and Statistics. I think once I was quite well trend in those areas, but was not easy to have a job where I could use for fun and profit !

Quantum Computation and Quantum Information and Deep Learing are two books I'm about to embark on. Well, I have had to start, so I started now.

Why are they important? Well, they are being already exploited, and going to be the next stage of High Tech. Most people using wide variety of devices directly or indirectly already gets the benifits of them, jus they don't know that behind the scene those are cropping up to play major roles. In one word - AI.

Recently, I got engaged in Corporate Research for new and customer facing engaging product(s). Yeah, only thing constant is ... Also I really did not want to retire being called Senior engineer. I've had that title for quite some years, and I know very well when someone is retired, they get the title Senior and would be waiting for Tuesday of everyweek to get discount on almost everything, at least where I live...

]]>http://prokash.squarespace.com/journal/rss-comments-entry-36020226.xmlTid Bits of stackProkash SinhaTue, 23 May 2017 04:32:36 +0000http://prokash.squarespace.com/journal/2017/5/22/tid-bits-of-stack.html130723:1174877:35914708What is stack? And what it is used for?

Stack is really a way to handle modularity of software. In the long past stack was designed and crafted by hands when most softwares were built using Assembly language. First the idea is to breakdown large code to manageable functions, and being called by yet other functions. So there had to be a way to pass arguments at call time, and revert back to the state where it was among other things. The result is stack machine, in the sense that the underlying architecture have enough support to create standard template code for stack paradigm.

After that stack became a software pattern that is one of the Abstract Data Type. Queue, list, doubly ended queue are some of the others.

Now a days, whenever one function calls another function there is stack management work going on.

At the start of a function ( in most systems) the following is a standard prologue -

push %rbp

move %rsp %rbp

rbp is the frame pointer, and rsp is the stack pointer. So system pushes frame pointer on the stack. Now the stack pointer is free to be messed up by the callee. And at the end it can be retrieved from %rbp ( frame pointer).

How ??? When we define local variables to be used by a function, system reserve space to hold those local variables value. Followng code just do that ---

sub 0x48 %rsp;

You can see that %rsp now changed and pointing to lower addressed memory. Stack grows downward.

In this case the local variables taking 0x48 bytes to hold the local variables current values.

Now after the computation within this function, it has to put back the frame pointer, as well clean up the current stack built, using following ---

mov %rbp %rsp ; // rsp was untouched ( never used for anything ), so it gets back the stack pointer.

pop %rbp

%rbp being the frame pointer, and never suppose to be used as a target of any operation except the prologue and the epilog ( previous two lines is called epilog), where ever a local variable is used in the computation, it uses %rbp like a reference point. Like

mov -0x8(%rbp) %rbx.

add $0x1 %rbx

More ...

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35914708.xmlStack walk and why I'm here ?Prokash SinhaSat, 31 Dec 2016 02:43:00 +0000http://prokash.squarespace.com/journal/2016/12/30/stack-walk-and-why-im-here.html130723:1174877:35836153Lot of OS provides ready APIs for stack walk programmatically, that can give us a trace like stack snap-shot when something go wrong and your application crashes. Most popular OS provides debugger and trouble shooting features that can help determine where and why something went wrong.

But it is not a piece of cake when you think you have to craft something right into kernel, and watch how things are happening. This is deep probe in my belief. So here are some background work that needs to be considered first --

-- why do we need to do. In other words, why am I here ?

-- What are the known and unknows? It could be a wide range of knowledge in general computer science.

-- How can I get some control of the kernel ?

-- How best I can craft a stack walk ?

-- What to look after I know that stack walk is working somewhat to my satisfaction ?

I've been able to finish this project/investigation and can see the result first hand.

-pro

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35836153.xmlLooking forward to 2017 and beyond!Prokash SinhaThu, 08 Dec 2016 03:25:41 +0000http://prokash.squarespace.com/journal/2016/12/7/looking-forward-to-2017-and-beyond.html130723:1174877:35823859Time flies, and we are just about to enter another year. Time and again I bumped into this thing we call stack back trace. For last 10 or so years I always thought that it would be nice to get good grasp of this lethal weapon, compareable to a sharp knife and sharper brain of a surgeon. If one can understand this then lot of todays technology and their underlying thoughts and implementation becomes clear.

One can use it for lots of different things. Debugger uses it all the time, so it is not a new ground breaking technology. But its use is unbelievably wide in range. In this continuing series of notesf, I will try to emphasize one such implementation, that perhaps some of us could call "Beautiful code", and some of its use. It will cover an wide range of topics like: assembler, symbol constructions, application binary interfaces and standards, register sets and its purpose with respect to activation records.Finally its incarnation into the kernel.

More coming ...

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35823859.xml2016 year end reflection - Beautiful codeProkash SinhaSat, 05 Nov 2016 16:09:30 +0000http://prokash.squarespace.com/journal/2016/11/5/2016-year-end-reflection-beautiful-code.html130723:1174877:35804216We are nearing the end of 2016, and holiday season is around us. I've appreciated code that by looking at it, I can tell it is well written. Then are some of them in the category of Beautiful code !

When I read a small article or a short stories about anything, if it can hold me engaged and I understand the flow and theme without much effort, I call it beautifully written. I've always thought about my writing that I could call beautiful code even merginally!

I recently wrote something to tackle some xnu virtual file system code that can alter program behavior depending how nice or rogue that program. So there was a deep dive into the xnu kernel code. The idea is to take early control of program execution, and see if it make sense to let a foreign program to play on your backyard.

As it turned out that depending on core kernel changes, there could be few things that needs to be checked even if I can not decide if it is going to harm or not.

So the idea, that it will make sure it keep a tab on those undecidable situations and learn of them on the fly. It will learn some traits that will drive the decision to take control.

I call this beautiful code, since it is a passive observer of the traits of foreign programs without even knowing by anyone its presence. Since it is in the kernel, it must be fault proof.

Display elapsed time of processes

Turn ps into an realtime process viewer

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35650664.xmlEtiquette to ProtocolProkash SinhaMon, 28 Mar 2016 01:13:39 +0000http://prokash.squarespace.com/journal/2016/3/27/etiquette-to-protocol.html130723:1174877:35646609Being trained in Graph Algorithms, Netwrork optimizations, stochastic calculas including queuing theories, I've been drifting around networking area for almost all my working years. But my own networking ability is immensly poor !

Just about 25 yrs ago, I was dealing with general network design/model to charactrise different metrics to measure certain known user perceptible networking traits. As a result, we were even able to produce some nice parametric results that showed up in conference papers. In particular, back then the micro controller based memory was quite expensive, and a mega bytes costed bit of extra money. So we had to model to comeup with estimated range of memory needed for network components thruout a network.

Without getting into details, we already had a sliding window protocol in place for receiver to advertise what it is capable to take within then next epoch. And sender will adjust itself based on the receiver's indication to sender. At that time I used to talk about how we could make Etiquette being a protocol. Hence the name of this post.

But the wave of high-tech revolutions took me off the track quite a few times, as well as perhaps lack of imagination. By the time we were done, only few companies were actively involved dealing with networking protocols, where as lot of companies were interested in platform areas. So it was hard, if not possible to convince that some of the protocols does not have to be the way it is. The perception was that few companies tried their own protocols but tcp/ip prevailed, so why bother ?

Good question, right? Way too many times we hard the mouthful direction - Don't reinvent the wheel. Even a damn illiterate people would advice you the same !

Now proliferation of IoTs and mobile everywhere things are changing. Once again, lot of bright people thinking and tearing apart our beloved tcp/ip protocol stack to comeup with need based design and implementations. What it means to me ?

Trying to find a wonder drug, so that I can retrieve all the lost temoral-memories! But the good sign is that people are now realizing that a protocol designed in late 1970s was good enough then but now we need to deal with present state of affairs.

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35326373.xmlKernel Debugging - OSX (part III)Prokash SinhaFri, 01 May 2015 02:05:16 +0000http://prokash.squarespace.com/journal/2015/4/30/kernel-debugging-osx-part-iii.html130723:1174877:35326371Once you setup your remote debugging environment, it is your hard work ( always) to figure out how you debug, when you debug. Assuming that you that you are somewhat familiar with kernel debugging, which includes debugging core operating systems, boot code, code related to device or other system resource frameworks, you would be looking at some basic commands that would work reliably, for example -

*) Breaking into debugger, mapping symbols, looking at sources etc.

*) Run the kernel under your debugger, and probe, steps thru the code etc.

OSX started out with GDB debugger, and moved to LLDB. LLDB maps some of the GDB commands (if not all, but I don't know yet), but it is quite verbose to type most of the commands. I read it as Long Long Debugger.

GDB as such is quite old and popular, but when it comes to kernel debugging there are many local shop to shop customized gdb. But one general one is kgdb. It is not a true sense kernel debugger, though.

So what is a true kernel debugger?

A true kernel debugger is one which freez the time when you broke into debugger. So if you leave your debug envrionment overnight, you must see old time, date. Lot of kernel debugger in the open source are really not there. Hence, it is quite difficult to debug some of the hard problems like once in a blue-moon race etc.

Windows kernel debugger outshine in this case. Before it windows softIce was another one. But in GNU open source environment, I'm yet to see a "True kernel debugger".

More...

]]>http://prokash.squarespace.com/journal/rss-comments-entry-35326371.xmlKernel Debugging - OSX (part II)Prokash SinhaSat, 04 Apr 2015 04:25:16 +0000http://prokash.squarespace.com/journal/2015/4/3/kernel-debugging-osx-part-ii.html130723:1174877:35295300Last time we talked about how to get a kernel debug session between a natively running debugger machine (a.k.a. Host), and a OSX VM running under VMware Fusion. It was fun, right?

In fact, the fun part(s) is(are) yet to come!. First, my immidiate task is to get a XNU kernel built in a VM, and replace the off the shelf kernel ( named mach_kernel or kernel or kernel.debug ...). Then get a kernel debugging going between that VM and the Host. Building the kernel is fine. I can built the XNU package for 10.10 and 10.9 but running with or without kernel debug configuration is where I'm really challenged. With the same steps, as discussed in part I, I landed on Kernel waiting for Debugger connection, but the ip address as well as the mac address are all 0s with respective format. Clearly there is some problem!

On the otherhand, if I try to run the newly built kernel without debugger configuration, it hangs. And it does not matter if it is RELEASE or DEBUG built. Here comes the KDP, kernel debugging protocol. FreeBSD has basically the same idea. This protocol is based on TTCP over UDP. As far as I know this is not self contained when it comes to configuration side of the interfaces. This is Bridged interface of VMware virtualization feature. This I will tackle later. But what is really the problem ? AFAIK, KDP part initialized few ip related stuff to 0s, and there are outer layers that does not come with XNU source ( but some are in opensource.apple.com ) that keeps configurations in tact when it comes to full OSX buit.

One thing I did not mention in part I is that we need to map the syms and src when we need to debug kext. This is like external driver programming. For that, first thing to find out is what version of OSX you are running. It can be found from Apple icon. Also from tty, uname -a will give you bit more information. Once you know what version you are running, go to Apple site, and download respective "Kernel Debugging Kit" onto your Host. In most versions of Kernel Debugging Kits, installing the dmg file is nothing but mounting the package. Once mounted, you will see Readme xml file. That has information about how to invoke LLDB, and thats it. You have symbolic debugging. Just start the VM with kernel debugging configured as explained in part I, and invoke LLDB, and play with the instructions given in the Readme xml file.

Note that using Kernel Debugging Kit implies, you are still using off the shelf XNU kernel that comes with the OS image. This is a complete build of Mac OSX,not just the XNU kernel code.

Next, we will get to bit more deeper side of Kernel hacking. For now Happy hacking !