Detecting image update changes (pixel changes)? to send over TCP/IP

I have to continously update an image based on screen capture. So how can I memorize only the part of the image that changes from one picture to another (from one frame to another)(pixels?)... Think of a remote desktop. It's not a good thing to send a 200K desktop screenshot image to the client every time because you get a BIG delay and you get to low frames per second. Ideal is to send only the part of the image that changes.

I hope you understood what I an trying to do... So how can I detect the image changes between to frames? I can't loop through all pixel to see what pixel changed because it takes a BIG amount of time... I was wondering how , for example, TeamViewer has such a good refresh rate. I think it sends only the pixel information that changes from frame to frame. That's what I need so much.

It would be pretty hard to detect what has changed since the last update, since the mouse could have been moved somewhere else, or a window could have been opened/closed/resized.

But the concept of 'updating only what has changed', is a something that is often applied in movie codecs (I and P frames). So perhaps that's something you could look into (perhaps through a UDP stream?).

Another option is to use a heavy (lossy) compression technique (JPEG?) to get smaller files. It uses principles like run length encoding that instead of 'red red red red' writes '4 red' to file. But the purpose of monitoring is that you see what happens, so perhaps a lossless alternative is worth checking out (PNG?).

E: Something else, what if you 'partition' your screen (in 4 or 8, 16 smaller images), and have your software scan if the diagonals of each partition are identical to the ones captured before? And then only send the partitions that were changed + their x and y position, to recreate the image at the other side. One caveat however, is to always evaluate the partition where the mouse pointer is located, as that might be too small for the 'detection' system to notice.

Although I don't know if the diagonals are 'enough' to be responsive enough, perhaps other patterns are worth looking into.

Also, why the requirement of TCP/IP? That requires a 3 way handshake to set up, and while you have more reliability, UDP is faster, but some packets may never arrive. But if the update frequency is high enough, I wonder if it matters that much.

>> Dr_Gonzo: great post! I would argue the point of UDP being faster but regardless of the reason, it is the right choice for this job.

You want to use UDP for streaming content. Any time you don't care about data fidelity it is likely a candidate for UDP. Think of RDP -- if the connection drops out for a few seconds you don't want to replay all of the "video" you missed, you just want to see the current screen capture. That being said you should use UDP.

Regarding how to capture changes in the image -- I don't know this for certain but I bet you can hook in to the win32 API and catch portions of the screen that are repainted/invalidated, then update your in-memory copy with those regions. When you detect a region has been repainted send the position & size of the invalidated region, and update your client.

Like I said I don't know for certain but that is how I would approach it. Doing a pixel-by-pixel comparison to detect changes is computationally expensive and there has to be a better way.

Thanks for replies... I am weak in win32 API... I don't know if somebody can give me an example of approach to what Dr_Gonzo and sknake adviced me.. I would really appreciate it. I always enjoy learning more and more... I just need the headstart. Thanks.

Thanks for replies... I am weak in win32 API... I don't know if somebody can give me an example of approach to what Dr_Gonzo and sknake adviced me.. I would really appreciate it. I always enjoy learning more and more... I just need the headstart. Thanks.

I've done some more research, I think the keywords here are 'dirty rectangles' as explained here.

There they also refer to the function getUpdateRgn, which might be something that could get you started.

Thanks for guiding. Very usefull. I read all you adviced. Now I think I would need at least how to call Win32 API from C# in order to get the coordonates of the Rectangle that will be repainted. So I need to call in fact GetUpdateRect() function from C# but I don't know how. This is API reference: http://msdn.microsoft.com/en-us/library/aa923766.aspx

//1) when I import this, gives it says RECT type not found...
[DllImport("user32.dll")]
public static extern int GetUpdateRect(int hwnd, [MarshalAs(UnmanagedType.Struct)] ref RECT lpRect, int bErase);
//2. The second parameter would be the coordonates of the rectangle.. how can I get them?

Sorry for posting twice.. here's what I found (still have a problem):
I have a class usefull for user32 functions. I use that class to get the region... I should get it but I always get 0,0,0,0 rectangle coordonates. WHY? Where's the mistake?

So I guess it's bad :(.... can't hook into WM_PAINT messages... that's what I'm trying to do. And if I would... why he says is very expensive? Would I have a big time delay to process this? Can you help me writing the code for hooking? Maybe my code doesn't work properly because it's not hooked exactly in WM_PAINED message. I'm in c#, I guess inc++ would be simplier to get into the message but I need this in c#

Hi. so this is actually a continuation from another question of mine[Here](https://www.daniweb.com/programming/software-development/threads/506795/dynamically-add-values-into-datagridview-cell-from-listbox-vb2010) but i was advised to start a new thread as the original question …

I have a 2d matrix with dimension (3, n) called A, I want to calculate the normalization and cross product of two arrays (b,z) (see the code please) for each column (for the first column, then the second one and so on).
the function that I created to find the ...

Write a C program that should create a 10 element array of random integers (0 to 9). The program should total all of the numbers in the odd positions of the array and compare them with the total of the numbers in the even positions of the array and indicate ...