Visual Basic 6 Win32 API Tutorial

Well, we've covered a lot of the major data type issues that we can run into when using
the Win32 calls. But there's still one more that we should address, and that's error
handling.

The majority of API calls will inform you in some way, shape, or form if an error
occurred (note, however, that this doesn't include memory exceptions). Most of the time,
it takes the form of a return value or some parameter that you pass into the procedure.
But virtually all of the calls set some internal OS information that you can obtain using
just two API calls. Let's review these calls, and then we'll create a VB function that we
can use within our main development application.

The GetLastError function simply returns a number that corresponds to the last error
that occurred within a Win32 API call. We can then use this value and pass it into
FormatMessage to obtain a message that may make more sense than error code 10594. As you
can see, that second parameter is already causing me some concern. We'd better look into
this further.

If you want, you can use the SetLastError API call to set the error code.I can't think
of a reason why you'd need to do this in VB, but it is possible to change this value.

The first parameter, dwFlags, is used to tell the call how it should be used. It's
cryptic, I know, but this function gets pretty flexible in a hurry when you start to read
the documentation on the call. The only value of dwFlags we're concerned about is
FORMAT_MESSAGE_FROM_SYSTEM (equal to 4096), which will tell the call to look up a
description from an internal resource. We can ignore the second parameter for our purposes
(thank goodness), since Microsoft's documentation tells us this argument is ignored when
we set dwFlags equal to 4096. We can pass in the return value from GetLastError to
dwMessageId. We don't care about language issues for now, so we'll set dwLanguageId equal
to 0. The lpBuffer is another case where we need to pass in a pre-allocated string to the
call - the length of the string is passed in through nSize. We can also ignore the
Arguments argument as well.

There's a lot more that you can use FormatMessage for, but we just want to use it to
obtain error information. If you're curious, hop onto Microsoft's web site and look up the
documentation on the call. For now, let's just use what we know about the call to create a
prototype API error call:

We'll improve upon the function in a moment, but I hope you see the point. Whenever you
get an error from a Win32 call, run this function to get an error message. It may help you
in debugging your applications.

Which DLL Error Code?

Notice that the GetWin32ErrorDescription uses an argument to get the error code. This
was done for a lot of reasons; such as letting us enter in any value to see what the
result would be (not the most exciting thing to do in the world, but it might be somewhat
educational). However, it was also done to illustrate the behavior of capturing the error
code from a DLL. Let's use a very simple but powerful example to illustrate this fact.

Create a new Standard EXE project in VB called MutexTest. Add one label to the form.
Here's some more specific information about the project:

Object

Name

Caption

Form

frmMutex

Mutex Tester

Label

lblMutex

(leave blank)

Your main form should look something like this:

As with the TestAPIStringReturn project, simplicity is the key with this project. We
want to focus in on the API calls and keep the UI to a minimum where possible.

Add the GetWin32ErrorDescription function that we just looked at to the code window for
the form. Now we'll need four API declarations in this project along with one form-level
variable and two constants. Add them to the form's Declarations section like this:

Note that we didn't use a module this time. For simple projects like this, we really
don't need a module to house the API declaration.

Now that we've got the project set up, let's back up for a second so I can explain what
this project will actually do!

You may run into a situation in your project development where you want to prevent the
user from opening up more than one instance of your application. The global App object
does have a property call PrevInstance, which you can use to determine if another instance
is running. But I was curious one day, and I started looking into ways to do this myself.
As it turns out, you can use a kernel object called a mutex (short for mutual exclusion)
to make this a very easy task.

Before we get into the details of the code, I should note that we're using a mutex in a
way that it probably wasn't intended (of course, that doesn't necessarily make it wrong,
either!). Mutexes are commonly used in multithreaded applications, and that's beyond the
scope of this book. However, we'll use one nice little feature of a mutex for our problem
at hand: it can be accessed from any process in Windows.

I'll come back to this function in a moment. For now, assume that this function will
return a True if the application is already running and False if it isn't. In the
Initialize event of the form, add this code:

Private Sub Form_Terminate()
If mlngHMutex <> 0 Then
' Close the mutex.
CloseHandle mlngHMutex
End If
End Sub

Now let's go through how this all works. The first thing we do is create a mutex using
the CreateMutex API call. The first argument is set to 0 since we don't care about the
security attributes of this mutex. The second argument is also set to 0, which means that
we dont "own" the mutex if we create it successfully. The last parameter
is the one that we're concerned about. If we didn't name the mutex, we'd have to identify
it by the return value, which is the handle to the mutex. However, a named mutex can be
used by the name given. Furthermore, as I hinted at before, mutexes can be used across
processes. Therefore, if another instance of our application has already created this
mutex, CreateMutex will let us know about it.

The way CreateMutex lets us know of this situation is a bit weird, though. The return
value is the handle to the mutex on success (i.e. a non-zero value), so we can't use the
return value to let us know if the mutex already exists. However, if the mutex already
exists, the GetLastError function would return a value equal to ERROR_ALREADY_EXISTS. In
our case, we catch that situation and return a True. Note, though, that we don't use
GetLastError - we use Err.LastDLLError. This will be important, as we'll see.

If we haven't created the mutex yet, the app will load up just fine. We should get a
window that looks like this:

Of course, we should get rid of the mutex when the application is done. This is why we
use CloseHandle on mlngHMutex in the Terminate event of the form.

So what does any of this have to do with GetLastError? Here's the problem. First,
compile the project into an executable (make sure the code is native code, not p-code! To
check go to Project | Properties and select the Compile tab.). Then run two instances of
the program. The first one should load up fine, but the second one will show the following
two message boxes:

That's the last you'll hear from the second instance.

Now go back into the code and replace this line from within IsPrevAppRunning:

lngVBRet = Err.LastDllError

to this:

lngVBRet = GetLastError

As before, compile the app and run it twice. Now the app shows up both times! What's
the deal?

Internally, VB is making Win32 calls all the time in your application. Sometimes, when
you make a Win32 call, this may trigger an event within your application that causes VB to
make other Win32 calls (we'll address window messages later on in the book). Well, what
happens if your initial call causes an error, but the internal Win32 call made by VB
clears out the value of the error? You get the situation that we just saw. GetLastError
returns a zero, but Err.LastDllError returns the correct error code.

In previous versions of VB, the LastDllError property didn't exist, and strange
situations like the one we just saw popped up again and again. Therefore, the VB designers
decided to add the LastDllError to the Err object. This property should be used when a
Win32 call is made that causes an error, because VB will track your Win32 call and make
sure that this property reflects any possible error conditions.

But you need to be quick. Grab the value of LastDllError after any Win32 calls in
question, even before you look at the return value! This is the only way to guarantee that
any error condition generated by the Win32 call you just made is in LastDllError. If you
start changing the size of the form, or adding text to a text box, all bets are off, and
you've lost the error code (unless you're really, really, REALLY lucky).

So what's the best situation to be in? Let's leave this discussion of capturing Win32
error codes in VB with this outline. It's no guarantee that we'll report the correct error
code all of the time, but this is about as close as we're going to get:

Grab the value of LastDllError immediately and store it in a variable (call it lngErrVB).

Grab the value of GetLastError and store it in a variable (call it lngErr32).

If lngErrVB is nonzero, then use it to report the error condition.

If it's zero, check lngErr32. Chances are it's probably zero as well, but if it's nonzero, use it to report the error.

The moral of the story? You should always use the value from LastDllError whenever
possible. VB is making a conscious effort to intervene on your behalf, so this is your
best bet for Win32 error codes.