Summary

Contents

Introduction

In our previous paper, An Introduction to P/Invoke and Marshaling on the Microsoft .NET Compact Framework, we discussed how the Platform Invoke service of both the Microsoft .NET Compact Framework and the Microsoft .NET Framework allow managed code to invoke functions residing in unmanaged DLLs, allowing both custom and operating system (Windows CE) APIs to be accessible to applications written for either framework. Although many of the features of the service are identical between the two frameworks, the .NET Compact Framework is a subset of the full .NET Framework, and so there are several differences, some of which we explored in the previous paper. In this whitepaper, we'll focus on two particular issues that arise when marshaling structures, and how they can be handled in the .NET Compact Framework.

Marshaling Complex Types

As we mentioned in the previous paper, one of the major differences between the marshaler in the .NET Compact Framework and that in the full .NET Framework is that the more lightweight .NET Compact Framework marshaler cannot marshal complex objects (reference types) within structures or classes. This means that if any of the fields in a structure or class are defined with types that do not have a common representation between the .NET Compact Framework and unmanaged code (referred to as blittable types, and enumerated in our previous paper), the structure or class cannot be fully marshaled. For practical purposes, this means that structures or classes that contain either string pointers or fixed-length character buffers will not be properly marshaled.

As an example, consider the user notification API available on Windows CE. Using this API, an application can display a notify dialog or cause an application to execute at a specific time, or in response to an event, such as synchronization, or when a PC Card is changed. Since the .NET Compact Framework doesn't include a managed class that performs this functionality, developers who need it will need to P/Invoke to make the correct operating system calls.

To use the Windows CE notification API (CeSetUserNotificationEx), the structure used to define what event activates a notification, CE_NOTIFICATION_TRIGGER, would be declared in managed code, and translate directly in VB.NET as follows, where SYSTEMTIME is another structure composed entirely of blittable types, and NotificationTypes and EventTypes are enumerations that map to integers.

Unfortunately, the two string values used to specify the application to execute, and its command-line arguments, are defined in unmanaged code as pointers to null-terminated Unicode strings (WCHAR *). The .NET Compact Framework marshaler, therefore, does not marshal the structure correctly, since String is a reference type (System.String).

Note: As mentioned in our previous paper, System.String is a blittable type in the .NET Compact Framework, since all strings can be treated as Unicode. However, this only applies if the String is passed directly to the unmanaged function, not if it is used inside a structure or class.

In the full .NET Framework, the marshaler can handle this situation because it includes the MarshalAsAttribute. With this attribute, the structure could be rewritten as:

As a result, on the full .NET Framework, the strings referenced in the structure would be marshaled as null-terminated Unicode character strings, as the unmanaged function expects. Because the .NET Compact Framework does not include this behavior, you'll need to work around this issue.

Marshaling Strings in Structures

To allow string pointers in structures and classes to be marshaled correctly by the .NET Compact Framework, there are three primary solutions: invoking a thunking layer, using an unsafe block, and creating a custom class to handle the string pointer.

Using a Thunking Layer

The term thunking has historically applied to the code that translates arguments and return values from 16-bit to 32-bit representations, and vice versa. However, in its more generic usage, the term simply refers to creating some intermediate code that handles the translation of data. In reference to the .NET Compact Framework and P/Invoke, a thunking layer is an unmanaged function that accepts the arguments that make up the structure, creates the unmanaged structure, and calls the appropriate function. This is the technique for passing complex objects in structures presented in the Visual Studio .NET help, and on MSDN.

To use this technique in the notification example above, you could create an unmanaged DLL in eMbedded Visual C++ that exports a function that accepts all of the arguments of the CE_NOTIFICATION_TRIGGER structure. For example, you could create an unmanaged function in C that schedules an application to be run at a specific time, using notification services (the CeSetUserNotificationEx function), like so:

You'll note in the above listing that the unmanaged function defaults the dwType, dwEvent, and endTime members of the structure, and passes the appropriate arguments to the CeSetUserNotificationEx function.

From managed code, this function is declared like so in C#, using the DLLImportAttribute. Since the .NET Compact Framework can treat strings as blittable types when passed directly to an unmanaged function, your declaration can simply use strings:

Note that, in this case, you needn't even declare a CE_NOTIFICATION_TRIGGER structure in managed code, since the managed method can be written to accept all of the appropriate arguments, thereby more fully encapsulating the underlying operating system interaction. Alternatively, you can create a managed version of the structure, like shown previously, and then simply pass the structure to the method. If an error is countered (the returned handle is zero), a custom exception is created and populated with the error code retrieved from the operating system, as discussed in our previous paper.

Note: You'll notice that the code used to translate the DateTime variables in managed code to SYSTEMTIME structures is not shown, and is included in the custom method DateTimeToSystemTime. This method calls the FileTimeToLocalFileTime and FileTimeToSystemFileTime Windows CE API functions located in coredll.dll.

The obvious downside to this option is that developers must install and use eMbedded Visual C++, a daunting task to many Visual Basic and .NET Framework developers. For that reason, we'll also discuss two other ways of solving this same problem using only managed code in the .NET Compact Framework.

Using Unsafe String Pointers

The second option for passing strings in structures is to use the unsafe and fixed keywords in C# (there are no equivalents in VB). While this option allows you to write only managed code, it does so at the cost of disabling the code verification feature of the common language runtime, which verifies that the managed code only accesses allocated memory, and that all method calls conform to the method signature in both the number and type of arguments. This is the case since using unsafe code implies that you wish to do direct memory management using pointers.

Note: As mentioned in our previous paper, in addition to creating unverifiable code, unsafe code in the full .NET Framework can only be executed in a trusted environment. However, in version 1.0 of the .NET Compact Framework, code access security (CAS) is not included, so this is not (yet) an issue.

The fixed keyword is used in conjunction with the unsafe keyword, and is used to ensure that the common language runtime's garbage collector (GC) does not attempt to deallocate an object while it is being accessed by the unmanaged function.

Besides the downside of losing code verification, of course, this technique cannot be used directly from VB. You can, however, create an assembly in C# that uses unsafe code, and then call that assembly from VB.

To use the unsafe and fixed keywords, you need to mark both the structure declared with pointers, as well as the method that uses pointers, as unsafe. For example, creating the same RunApplication method using unsafe code is shown below.

In this example, you'll notice that the CeSetUserNotificationEx Windows CE API function is first declared using the DllImportAttribute. The CE_NOTIFICATION_TRIGGER is marked as a ref argument (ByRef in VB), since it is defined as a structure. This is required, since the .NET Compact Framework will only pass the address of a structure if it is declared as a ref argument. Had CE_NOTIFICATION_TRIGGER been declared as a class, the argument to CeSetUserNotificationEx could have been passed by value, since the .NET Compact Framework automatically passes the address of reference types. Although, in many cases, it doesn't matter if you declare the structure expected by the unmanaged function as a structure or a class in managed code, the SYSTEMTIME structure is an exception in this case. Here SYSTEMTIME must be declared as a structure, since the CeSetUserNotificationEx function expects the CE_NOTIFICATION_TRIGGER to contain the entire structure inline. If SYSTEMTIME were declared as a class, only the 4-byte pointer to the class would be marshaled.

Next, the CE_NOTIFICATION_TRIGGER structure is declared, with pointers to character arrays for the lpszApplication and lpszArguments members, and is therefore marked with the unsafe keyword.

Note: Keep in mind that, unlike the full .NET Framework, you don't need to decorate your structures with the StructLayoutAttribute in the .NET Compact Framework, since all structures are automatically LayoutKind.Sequential.

Finally, the RunApplication method is declared with the same signature as in the previous example, only this time the method creates an instance of CE_NOTIFICATION_TRIGGER and proceeds to populate its members. As in the previous example, several of the members are set to values required to execute the given application, and the SYSTEMTIME structure is populated with the custom DateTimeToSystemTime method.

To populate the string pointers embedded in the structure, the method first checks to ensure the application name is not null or an empty string. If either is the case, an ArgumentNullException is thrown. If not, the pointer to the character array is declared in the fixed statement, and populated using the ToCharArray method of the String class. Note that the character array will stay pinned in memory within the scope of the fixed block. The arguments pointer is then populated in the same fashion. The lpszApplication and lpszArguments members of the structure are then populated with the pointers, and the structure passed to the CeSetUserNotificationEx function.

Note: In this example, the declaration of the CeSetUserNotificationEx function has an IntPtr as the third argument, when, in reality, a structure of type CE_USER_NOTIFICATION is called for. This declaration is required so that the RunApplication method can pass IntPtr.Zero to the function for that argument. For other uses of CeSetUserNotificationEx, you would need to actually pass the structure (for example popping up a notification dialog). In these cases, you can make a second declaration of CeSetUserNotificationEx that overloads the first. The compiler will then choose the appropriate one, based on the arguments.

If an error occurs (the handle returned by the function is empty), the custom WinCeException is created and passed to an error handler.

Using a Managed String Pointer

The final technique that you can use to marshal strings inside of structure in the .NET Compact Framework is to create your own managed string pointer class. The benefits to this approach are that it can be used in both C# and VB, and can be leveraged in a variety of situations once the string pointer class has been created. It does, however, require a little interaction with the memory allocation APIs of the Windows CE operating system.

To begin with this technique, you can create a managed class that declares and calls the necessary memory management APIs. This class can expose shared methods to allocate a block of unmanaged memory, free that memory, resize a block of unmanaged memory, and copy a managed string to unmanaged memory. The VB version of the Memory class is shown here.

As shown in the listing, the LocalAlloc, LocalFree, and LocalRealloc unmanaged functions are declared with the DllImportAttribute, and then wrapped in the AllocHLocal, FreeHLocal, and ReAllocHLocal methods, respectively. Finally, the StringToHLocalUni method allocates an unmanaged block of memory of the appropriate size for the given string (2 bytes for each character, since the .NET Compact Framework only supports Unicode), and then copies the character array to the IntPtr that points to the unmanaged block.

Once the Memory class is in place, a simple managed string pointer structure can be created, as shown here.

As you'll notice, the StringPtr structure contains a private IntPtr used to track the pointer to the string in unmanaged memory. The pointer is then populated in the constructor by calling the StringToHLocalUni method of the Memory class passing in the managed string. The ToString method then overrides the System.ToString method to return a managed string given a pointer, while the Free method frees the unmanaged memory allocated for the string.

Both the Memory class and StringPtr structure can then be placed in an assembly and referenced from any smart device project that requires managed string pointers.

To use the StringPtr structure in the case of the CE_NOTIFICATION_TRIGGER structure, you can declare the structure using the StringPtr structure for the lpszApplication and lpszArguments members of the structure. In addition, since these members will need to be deallocated using the Free method of the StringPtr class, it makes good sense for CE_NOTIFICATION_TRIGGER to implement the IDisposable interface. The Dispose method can then be used to call the Free method of the two members. Along the way, you can also add a public constructor to the class, to not only create the StringPtr members, but also default other members required to set up the notification. The C# code for the structure is shown below.

Finally, the RunApplication method can simply create a new instance of the CE_NOTIFICATION_TRIGGER structure and call the CeSetUserNotificationEx function. Since the structure implements the IDisposable interface, in C# you can use the using statement to ensure that the compiler automatically calls the Dispose method after the unmanaged function is called.

Marshaling Fixed-Length Strings In Structures

The second situation that arises when using the P/Invoke service of the .NET Compact Framework involves marshaling fixed-length strings or character arrays inside a structure. For example, the Shell_NotifyIcon function that is used to place and remove application icons from the system tray accepts a pointer to a structure defined in the Windows CE SDK, as follows:

You'll notice that the final field in the structure, that holds the text of the tooltip to display when the cursor is over the icon, is defined as a 64-element array of characters. A straightforward translation of this structure in VB .NET would then look as follows:

Unfortunately, this simple translation won't work, since, even though System.Char is a blittable type, the array of characters will be marshaled as a 4-byte pointer to the array at runtime. As before, the full .NET Framework does support this situation through the MarshalAs attribute, in this case by decorating the array with the attribute and passing its constructor the ByValTStr value of the UnmangedType enumeration.

Because of this behavior, you have essentially two options. The first involves creating a byte array of the correct total size, and then copying the individual fields of the structure into and out of the byte array, as appropriate, using the methods of the Marshal class, or directly through pointers in unsafe code; essentially, marshaling by hand. A pointer can then be created to the byte array and passed to the unmanaged function. However, because this technique is more complex, the remainder of this paper will focus on a technique that combines using the .NET Compact Framework marshaler with a little custom marshaling.

Note: The key to this technique is that the szTip member of NOTIFYICONDATA is the final member of the structure. If this were not the case, then a more custom approach would be needed.

To use this technique, you must first make the appropriate unmanaged declarations in your class. As before, a best practice is to declare these as private functions in a class that then exposes static (Shared in VB) methods to perform the functionality. In this case, the DestroyIcon, RegisterWindowMessage, and Shell_NotifyIcon functions, along with the NOTIFYICONDATA structure and a few constants, would need to be declared, as shown here.

The key point to note here is that the szTip field is not included in the NOTIFYICONDATA structure, since it will need to be marshaled by hand. In addition, the RegisterWindowMessage function will be used to create a unique message identifier, used to populate the uCallbackMessage field and ultimately to notify the application when the notify icon is acted on by the user. The DestroyIcon function will be used to deallocate the memory used by the icon in the system tray. Finally, you'll notice that the Shell_NotifyIcon method is declared to accept an argument of type TrayConstants that, in part, defines what actions the function will take, such as adding or removing the icon, and an unmanaged pointer that will be populated with the pointer that includes the NOTIFYICONDATA structure.

The call to the Shell_NotifyIcon function can then be wrapped in a shared method called CreateNotifyIcon, as shown below.

You'll notice that this method accepts the window to notify when the icon is acted upon defined as a MessageWindow object from the Microsoft.WindowsCe.Forms namespace, an unmanaged pointer to the icon (actually the icon handle) to place in the system tray, and the text of the tooltip to display. The method then returns the Windows message handle that the MessageWindow can use to look for events, such as clicks that occur on the notify icon.

Note: One technique to create the pointer to the icon is to use the ExtractIconEx Windows CE function to extract the icons handle from an executable file or DLL.

First, this method declares an unmanaged pointer that will be used to hold the pointer to the structure, and creates a new instance of the NOTIFYICONDATA structure, and places it in the reference variable, nid. Next, the correct size of the structure expected by the unmanaged function is calculated by adding the size calculated by the marshaler (Marshal.SizeOf) to the size of the tooltip text (in this case, a 64-element array). In this particular case, had the szTip field been included in the structure, the SizeOf method would return 28 (4 bytes for each of the seven fields). However, by adding the size of the array manually, the correct total of 92 (24 bytes for the six other fields, plus 64 bytes for the character array) is calculated. The SystemDefaultCharSize is used to ensure that the correct character size for the system is taken into account.

Once the structure's correct size has been determined, the RegisterWindowMessage function is called to register a Windows message handle used by the application to determine when an event for the icon is generated. The remainder of the structure is then populated, including the size that was just calculated, the icon handle, the handle of the window that will be notified, the message handle just generated, an application defined identifier, and finally the flags that indicate which of the other fields contain valid data.

At this point, the structure can be copied into unmanaged memory, and a pointer returned. This can be done by using the AllocHLocal method of the Memory class shown in the first section of this paper. This method is passed the total size previously calculated for the structure. The returned IntPtr (structPtr in this case) can then be populated using the StructureToPtr method of the Marshal class. This code relies on the .NET Compact Framework marshaler to correctly marshal the blittable types in the NOTIFYICONDATA structure.

However, at this point, only the first 24 bytes of the memory pointed to by structPtr have been initialized. To populate the remaining 64 bytes, some custom marshaling is required. First, the tooltip argument passed into the method is copied to a character array. Next, a new pointer that points to the correct offset within the memory region pointed at by structPtr is created by adding the size of the NOTIFYICONDATA structure (24 bytes in this case) to the integer value of the pointer itself returned through the ToInt32 method. Finally, the Copy method of the Marshal class is used to copy the contents of the character array to the new pointer.

The Shell_NotifyIcon function can then be invoked to add the icon to the tray. If the function returns false, the GetLastWin32Error method of the Marshal class can be called (as we discussed in our previous paper), and a custom exception thrown. If the call succeeds, the message handle to hook is returned. In the event of an exception being thrown by the PInvoke service, our custom HandleCeError method (also discussed in the previous paper) is used to take the appropriate action.

It is important to note, however, that the Finally block contains a call to the FreeHLocal method of the Memory class, so that the memory allocated for the structure is always released.

Note: Those developers who will need to marshal reference types and character arrays like those shown in this paper may alternatively want to invest time in creating a custom MarshalAs attribute for the .NET Compact Framework. Using this approach allows you to place all of the memory and pointer manipulation into the attribute code, making the code to call the unmanaged function cleaner, while providing a centralized way of handling these situations. In fact, as of this writing, one of the developers in the .NET Compact Framework community has begun to develop just such a utility. For more information, see the Compact Framework public newsgroup (microsoft.public.dotnet.framework.compactframework).

For completeness, the associated DestroyNotifyIcon method is also shown here. It takes the same approach, but, of course, uses the NIM_DELETE value of the TrayConstants enumeration to remove the icon from the tray. It also calls the DestroyIcon unmanaged function to deallocate the icon handle.

Note: In order for the Shell_NotifyIcon function to return true when removing the icon, the application identifier (the uID member of the structure) must be populated to the same value as was passed in originally.

To use the CreateNotifyIcon and DestroyNotifyIcon methods then, the following Form and MessageWindows classes can be created.

As you can see, the Form includes private fields that reference the MessageWindow class that will process the events, the pointer to the icon, and the identifier of the message used for the notification. After creating the handle to the icon, the Form1_Load method then creates the instance of the EventWindow class (derived from MessageWindow), and passes both it and the icon pointer to the CreateNotifyIcon method. Note that the method is static and encapsulated in the Atomic.CeApi.Forms class to better organize the code. At this point, the icon will be added to the system tray and will be ready to receive notifications.

To receive those notifications, however, the method returns the message identifier, which is then placed in the msgId field of the EventWindow class. This allows the EventWindow class to filter the messages, and only raise an event for the message that pertains to the icon notification. Alternatively, the message identifier could have been hard-coded into both the CreateNotifyIcon method and the EventWindow class, making the call to RegisterWindowMessage unnecessary.

An event handler, the eventWnd_MsgProcessed method, is then created in Form1 to handle the MsgProcessedEvent event of the eventWindow class. You'll notice that the EventWindow class defines the corresponding delegate and event. The overridden WndProc method checks to see if the message identifier is the notification message, and, if so, raises the MsgProcessedEvent that is then caught by Form1. The form can then take some action, such as bringing the form to the foreground or displaying a context menu.

Summary

Although the .NET Compact Framework marshaler used by the P/Invoke service does not contain all of the functionality of the full .NET Framework marshaler, it can still be used effectively, even in fairly complex cases, using a variety of techniques. In this whitepaper, we showed how passing both string pointers and embedded character arrays inside structure can be accomplished using a thunking layer, pointers and unsafe code, creating your own string pointer class, and doing a little custom marshaling. Hopefully, you'll be able to use and build on these techniques as you create great .NET Compact Framework-based applications.

Comments and Discussions

I am working on migration project from vb to vb.net .I have to pass complex structure consisting of double,string,structure to an API as ByRef type. I am in need of your guidance to do this.ex:Public declare auto function "xyz" lib "lib1" (ByRef t1 as test) as integerpublic struct test public item1 as integer public item2 as struct2 public item3() as doublesub initialize() redim item3(45)end sub end structure--Pravin

AlexY wrote:It looks like this and other 3 articles by this author have been "stolen' from MSDN:

I assumed, given the name of the author that these were article by the Compact Framework team... probably trying to spread the word.

I may be wrong. You might want to check with the editor, as these aren't un-edited articles, they have gone through the submission process.

Michael

But you know when the truth is told,That you can get what you want or you can just get old,Your're going to kick off before you even get halfway through.When will you realise... Vienna waits for you? - "The Stranger," Billy Joel

juggler wrote:In other words, looking for free advertising rather than paying to get put in the product showcase section.

Oooh it makes me cross

If it is the Microsoft Compact Framework team, than I'm glad to see Microsoft being encouraged to share their knowledge outside of the MSDN library.

It's always nice to see Microsoft helping out other developers.

Michael

But you know when the truth is told,That you can get what you want or you can just get old,Your're going to kick off before you even get halfway through.When will you realise... Vienna waits for you? - "The Stranger," Billy Joel