Introduction

Function pointer is a pointer to function like pointer to variable. The user can call the function pointer after dereference with required parameters. The function pointer can be implemented using C or C++ language. Function pointer is useful for late binding, implements callbacks and passes a function as an argument to another function. This article explains the usage of function pointer and callback in Windows application programming Interface (API).

C++ supports the following two types of function pointers:

Static member function pointers

Non-static member functions pointers

C supports functions pointer like C++ Static member function. The application can define the variable name for address of the function and call using that variable name.

Calling Convention

Calling convention is used for passing the argument to function and cleaning the stack from memory. The functions pointer declaration should match with actual function calling convention. Microsoft C/C++ compiler supports the __cdecl, __clrcall, __stdcall, __fastcall and __thiscall calling convention. The __stdcall calling convention is used to call Win32 API functions. The __cdecl is default calling convention for C and C++ programs. The __cdecl calling convention creates larger executable than __stdcall. It requires each function to include clean up code.

The WinDef.h defines the calling convention macros. The CALLBACK, WINAPI, APIPRIVATE and PASCAL are __stdcall calling convention. The function and function pointer should use the same calling convention. If the developer doesn’t mention explicitly, the Microsoft C/C++ compiler takes __decl calling convention. But, if the developers use any win32 API, they should use __stdcall calling convention.

Callbacks

The callback function is a function pointer that is passed to another function and second function can call function through argument function pointer. The second function call directly to function pointer without knowing about the argument function. Every callback function has its own prototype. Because, the function name, return type and arguments are different for each function. All the C++ member functions have internal this pointer for access members of the class. So, the developer has no need to use this pointer for making use of callback function in C++. But, when we use Non-Member function and static function, we must use call back function explicitly. The reason these two would seem the same is because C does not have references, so you cannot send a reference.

Difference between Function Pointer and Callback Functions

No

Function Pointer

Callback

1

Function pointer is a pointer which points to the function.

Callback functions are function pointers passed as parameters of function, which can be called back when an event occurs.

2

Example vtable is array of function pointers

Callback function to be called when thread ends.

3

Function pointer is an address of function

Callback functions are passed function pointer as an argument and caller would callback if something happens.

Sample 1 explains C function pointer and callback functions. The BubbleSort sorts the array of elements and Display displays the array of integers. The BinarySearch and LinearSearch are two callback functions which are used for searching the element. The Binary search will work with only sorted elements and takes less time (better performance) compared to Linear search. The Linear search takes with sorted or unsorted array of elements.

The above function uses function pointer as an argument and uses function pointer for invoking the appropriate search function. The main function calls BinarySearch after sorting the unsorted input elements.

Secured CRT

The “Secured string handling” explains the basics about secured C Run time library string handling function. The secured CRT also provides a set of secured algorithm functions. The C run time algorithm uses callback functions for implementing compares elements.

No

function

Secured Version

Description

1

bsearch

bsearch_s

Binary search

2

_lfind

_lfind_s

Linear search

3

_lsearch

_lsearch_s

Linear search which is added to array if not found

4

qsort

qsort_s

Quick sort

The above sorting and searching functions compare function pointer will take three arguments. This C Run-Time Library functions use __cdecl calling convention. The Comparison function will take two parameters. The first parameter is pointer to the key for the search and the second parameter is a pointer to the array element to be compared with the key.

The secure version Comparison function will take three arguments. The argument compare is a pointer to a user-supplied routine that compares two array elements and returns a value specifying their relationship. The secured version compare will take Context pointer too. The context parameter makes it easier to perform thread-safe sorts. Instead of using static variables that must be synchronized to ensure thread safety, pass a different context parameter in each sort. The compare function may invoke one or more times during quick sort. MSDN has an example for the above secured version of CRT algorithms.

Visual Studio 2008 C Run time library run on Windows 2000, Windows XP, Server 2003 and Vista. Visual Studio 2010 C Run time library supports Windows XP with SP2, SP3, and Windows Server 2003 with SP1, Server 2003 with SP2, Vista, Windows Server 2008 and Windows 7. The Secured C Run time library comes along with Visual Studio. If the developer wants C Runtime library, they can download from Microsoft websites [12,13].

The WindProc is the function used in WNDCLASS structure. The WndProc is called for all windows messages and handles all the messages. If it doesn’t find any message in WndProc, it calls the default window procedure. The DefWindowProc function calls the default window procedure to provide default processing for any window messages that an application does not process.

The GetClassLong function is used to retrieve WNDCLASS information for a window class and The SetClassLong function replaces the specified 32-bit (long) value at the specified offset into the extra class memory or the WNDCLASSEX structure for the class to which the specified window belongs. The S/GetClassLong function nIndex GCL_WNDPROC replaces the address of the window procedure associated with the class.

The following Windows API functions use callback and function pointer. The following table discusses only the function and purpose for the function.

Enumerates all entries in the property list of a window by passing them, one by one, to the specified callback function.

EnumPropsEx

Enumerates all entries in the property list of a window by passing them, one by one, to the specified callback function.

PropEnumProc

Application-defined callback function used with the EnumProps function.

PropEnumProcEx

Application-defined callback function used with the EnumPropsEx function.

Windows

EnumChildProc

Application-defined callback function used with the EnumChildWindows function.It receives the child window handles and it is a placeholder for the application-defined function name.

EnumChildWindows

Enumerates the child windows that belong to the specified parent window by passing the handle to each child window, in turn, to an application-defined callback function.

EnumThreadWindows

Enumerates all nonchild windows associated with a thread by passing the handle to each window, in turn, to an application-defined callback function.

EnumThreadWndProc

Application-defined callback function used with the EnumThreadWindows function. It receives the window handles associated with a thread.

EnumWindows

Enumerates all top-level windows on the screen by passing the handle to each window, in turn, to an application-defined callback function.

EnumWindowsProc

Application-defined callback function used with the EnumWindows or EnumDesktopWindows function. It receives top-level window handles.

Dynamic Linked Libraries

Dynamic linked library is a windows executable file which contains a set of functions. DLL loads only once in memory and process can refer for executing the functions available in DLL. If no process refers DLL, it unloads from memory. DLL can be loaded implicitly or explicitly. The explicit DLL uses function pointer for loading the functions in memory. The implicit linking uses the library (.lib) file and loads the DLL when the executable starts loads in the memory. The explicit linking uses the DLL directly and loads the DLL when necessary using LoadLibrary (Ex) family functions.

The explicit linking has the following advantages over implicit linking:

Explicit linking is useful when the application does not know the name of a DLL that it will have to load until run time.

The application fails when implicit linking DLL fails to load during startup. But, explicit linking loads the DLL using LoadLibrary and it can handle and continue.

If the application contains many DLLs, the Implicit linking will take a lot of time for loading the application. But, implicit linking loads the DLL when necessary.

The processes call LoadLibrary/LoadLibraryEx for loading the DLL in memory. MFC application uses AfxLoadLibrary for loading the explicit linking DLL. If the MFC application uses AfxLoadLibrary function for loading the DLL, it should call AfxFreeLibrary for unloading an extension DLL. If the function is able to load the DLL successfully, it returns the HANDLE of the function. If the DLL is already loaded in memory, it increments the reference count and returns the HANDLE. If it is not able to find an entry point, it simply returns NULL. Once he/she gets the HANDLE of the DLL, the user can call GetProcAddress function for retrieving the address of an exported function and run the function or variable. It takes DLL module handle which returns from LoadLibrary and name of the function or function export ordinal.

The function pointer doesn’t have any compile time check. The typedef for function pointer is useful for check type-safety in function prototypes of the exported functions. If the DLL is built with a module definition (.def) file, the GetProcAddress can use ordinal number. It is slightly faster if the DLL contains many exported DLL functions.

Secured Pointer Encoding

We have discussed the function pointer and callback functions using function pointer as parameter. We are living in an insecure world. The buffer overrun vulnerability gives an option to the attacker to attack the code and change the normal flow. The attacker may even run some other application as part of our application function flow. When the program calls the function, it creates a new stack frame with function parameters, return address, Frame pointer, Exception Handler frame and local variables. The good coding practice recommends making use of a local variable or passing by reference instead of global variable. If it requires, make use of only immutable variables or pointers. The function pointer declares globally for access the functions. If the function pointer declares as global, the function pointer is alive till the application terminates.

Microsoft secured Lifecycle (5.0) recommends for encode long lived pointers. The insecure pointer leads to buffer overrun attacks. The global scope function pointer and global pointer are long lived pointers. The long lived pointers are insecure pointers. The developer or designer must identify long lived pointers. Microsoft Windows provides the EncodePointer() and DecodePointer() function for encrypt and decrypt pointers using a secret technique for the given process. If the attackers try to overwrite a pointer, the encoded pointer doesn’t allow for attacks. The EncodePointer()/DecodePointer() functions are available from Windows XP SP2 and above for client operating system and Windows Server 2003 SP1 for server Operating system. The EncodePointer()/DecodePointer() functions are windows functions and Microsoft uses those functions for internal to CRT and may use other APIs too for encoding the global pointers. Windows uses those functions for encode unhandled exception filter addresses, long lived function pointers, long running heap allocation memory, kernel to perform indirect calls, etc.

The encode function does XOR operation with random number. The EncodePointer/DecodePointer function use application’s process information block and EncodeSystemPointer/DecodeSystemPointer uses value for each system reboot. Michael Howard's Web Log explains about the algorithm for XOR encoding operation. The Michael Howard's[4, 5] Web Log explains more about algorithm uses for XOR Operations.

Assume, search is a function and FunctionPointer declared typedef for function pointer. It lives for a long time in memory and reclaims the memory when terminating the application. The search function may take a long time to execute and the function pointer is declared as global. The attacker may attack this function pointer using buffer overrun attacks. The attacker can change the return address and add to execute some shell code or some other application. The EncodePointer takes PVOID windows data type and returns the same pointer to Void (PVOID) data type. Once the pointer is encoded, the attacker cannot call or change the global function pointer address. The developer can encode and decode the function pointer like the following syntax:

The EncodePointer returns encoded pointer value. The attacker can’t attack the encoded pointer. But, the developer can decode the pointer using DecodePointer function with an encoded pointer. Michael Howard[4, 5] doesn’t recommend for use all global function pointers because it may affects the application performance.

The encode pointer used in crt\src\tidtable.c CRT file. This files use the OS mechanism for encoding pointers to prevent hijacking. The EncodePointer/ DecodePointer supports only for X86 machine. The tidtable.c gives _encode_pointer/_decode_pointer function for encode and decode CRT functions. But, those functions are undocumented and don’t export by CRT DLLs. The following _encode_pointer/_decode_pointer are defined in internal.h for internal helper functions for CRT.

Examples

Pointers internal to NT Heap

Vectored exception handler pointers

Long lived function pointers

Conclusion

The function pointer is an important concept in C and C++ programming. Windows application programming interface (Windows API) uses function pointer and callback function noisily. So, a Windows programmer should be aware of function pointer and callback function whether using or not using it in programming. The Microsoft Secured development Lifecycle (SDL) also recommends and provides EncodePointer/ EncodeSystemPointer and DecodePointer/ DecodeSystemPointer function for secure pointer operations. The pointer might be a function pointer or variable pointer. The insecure pointer leads to buffer overrun attacks. The C/C++ doesn’t support array bounds checking when initializing or moving data between functions. The buffer overrun attacks can override the function return address. When the function returns instead of jump back to calle, it jumps to attacker code. The stack overflow can use for change program flow or gain the privileges for operating system environment. The heap overrun involves for heap memory allocation functions which allocated memory by application.

Summary

Assign NULL for delete objects. It avoids "dangling" pointers.

Encode and Decode long lived static or global pointers. The encode pointer uses different one for each process. So, the attacker can’t predict the encode logic for buffer overrun attacks.

Windows messages and run time dynamic-link libraries use function pointer heavily. So, use Encode and Decode pointer for secure pointer access.

Share

About the Author

Selvam has worked on several technologies like Core Java, VC++, MFC, Windows API and Weblogic server. He takes a lot of interest in reading technical articles and enjoy writing them too. He has been awarded as a Microsoft Community Star in 2004, MVP in 2005-06, SCJP 5.0 in 2009, Microsoft Community Contributor(MCC) 2011.