Author
Topic: Testing in LPC (Read 326 times)

One of the (in my opinion) critical, but far too often overlooked aspects of software development lies in quality assurance - specifically, ensuring that that software you've written does what it's supposed to do and (even more importantly) when changes are made, things aren't inadvertently broken (ie: regression testing).

When I started rewriting the lib for RealmsMUD, one of the first things I did was implement a simple testing framework. Initially, I would run the tests manually whilst logged in, but it quickly became apparent that long-term, that was an untenable approach. I decided that it'd be nice to execute the tests outside of the running mud. Version 1 (which I'd suggest as a starting point for anyone who wants to try this) of this concept was to simply make use of preload_objects in the driver and call exit when this executed. Then, for any tests, I'd set up the init file with the list of everything I wanted to execute. The driver would load them (and in master.c, I changed the preload stuff to also call executeTests on anything that inherited my test fixture.). The one down side to this is that you will almost certainly have to give your driver's eval limit a hefty boost or it's going to puke if you have any meaningful number of tests in a single file.

I've since hacked the driver (well, my heavily hacked ldmud-3.2.17 driver) so that it can take a single file and evaluate it without having to fall to preload on master.c.

What I'm supplying should be looked at with the following caveat: I've only tested this on the ldmud 3.2.17 built in compat mode. Everything being done should be generic enough to work anywhere, but I make no promises.

Building tests is pretty straightforward, but since it's caused some confusion with other wizzes on Realms, I figured I'd give an explanation of what you should do.

Try to avoid testing more than one thing in any given method and make sure you name you test methods in a way that it's immediately obvious what you're doing. Good: QuestIsInProgressReturnsFalseWhenQuestHasBeenCompleted, Bad: ThingDoesStuff, Worst: X

Make use of setup methods to keep your actual tests succinct. When you make tests, you need to set up a bunch of objects so that you can properly exercise them. If you need to clone a player, give them a sword, clone a monster, and make them fight, don't "muddy" the test with that - put it in a separate method and call it from the test.

You should have system tests that encompass interactions between many objects. You should also have unit tests that test a single thing in a vacuum. These unit tests should use fake/facade items "around them" to limit what you're testing and to simplify the test.

Here's a couple example tests. This first one exercises the quest module in the player:

This second one exercises the quest state machine. Note the last couple tests (QuestSucceededReturnsTrueWhenQuestCompletesAsSuccess, for example) as they use ancillary methods and fake objects that transition the quest to its various states:

/////////////////////////////////////////////////////////////////////////////void SetUpQuestItem(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!");

QuestItem->testAddState("met the king", "I met King Tantor the Unclean of Thisplace. He seems to like me."); QuestItem->testAddTransition("meet the king", "met the king", "meetTheKing", "/lib/tests/support/quests/testKingObject.c");

QuestItem->testAddState("serve the king", "The king asked me - ME - to be his personal manservant. Yay me!"); QuestItem->testAddTransition("met the king", "serve the king", "serveTheKing");

QuestItem->testAddState("ignore the king", "I told the king to piss off. I have socks to fold."); QuestItem->testAddTransition("met the king", "ignore the king", "ignoreTheKing"); QuestItem->testAddEntryAction("ignore the king", "killTheKing"); QuestItem->testAddFinalState("ignore the king", "failure");

QuestItem->testAddState("save the king", "Earl the Grey tried to kill the king but I gutted him like a fish."); QuestItem->testAddTransition("serve the king", "save the king", "hailToTheKing"); QuestItem->testAddFinalState("save the king", "success");

QuestItem->testAddState("king is dead", "I must lay off the sauce - and the wenches. King Tantor is dead because of my night of debauchery."); QuestItem->testAddTransition("serve the king", "king is dead", "maybeNobodyWillNotice"); QuestItem->testAddFinalState("king is dead", "failure");

/////////////////////////////////////////////////////////////////////////////void AddStateSilentlySucceedsWhenStateIsValid(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!");}

/////////////////////////////////////////////////////////////////////////////void AddStateThrowsWhenAddingTheSameStateTwice(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); string err = catch (QuestItem->testAddState("meet the king", "I've been asked to meet the king!")); ExpectEq("*ERROR - stateMachine: the 'meet the king' state has already been added.", err);}

/////////////////////////////////////////////////////////////////////////////void AddStateSilentlySucceedsWhenEntryEventIsValid(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!", "someEvent");}

/////////////////////////////////////////////////////////////////////////////void AddStateThrowsWhenAddingAnInvalidFinalStateResult(){ string err = catch (QuestItem->testAddState("meet the king", "I've been asked to meet the king!", "killTheKing", "blah")); ExpectEq("*ERROR - stateMachine: the final state result must be 'success' or 'failure'.", err);}

/////////////////////////////////////////////////////////////////////////////void AddStateSilentlySucceedsWhenFinalStateResultIsSuccess(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!", "killTheKing", "success");}

/////////////////////////////////////////////////////////////////////////////void AddStateSilentlySucceedsWhenFinalStateResultIsFailure(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!", "killTheKing", "failure");}/////////////////////////////////////////////////////////////////////////////void InitialStateThrowsWhenStateNotPresent(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); string err = catch (QuestItem->testSetInitialState("blah")); ExpectEq("*ERROR - stateMachine: the initial state must have been added first.", err);}

/////////////////////////////////////////////////////////////////////////////void AddEntryActionThrowsWhenStateNotPresent(){ string err = catch (QuestItem->testAddEntryAction("blah", "things")); ExpectEq("*ERROR - stateMachine: an entry action can only be added if both the state exists and the method to call has been implemented on this object.", err);}

/////////////////////////////////////////////////////////////////////////////void AddEntryActionThrowsWhenAddingAnInvalidEntryAction(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!");

string err = catch (QuestItem->testAddEntryAction("meet the king", "badMethod")); ExpectEq("*ERROR - stateMachine: an entry action can only be added if both the state exists and the method to call has been implemented on this object.", err);}

/////////////////////////////////////////////////////////////////////////////void AddEntryActionSilentlySucceedsWhenEverythingValidates(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); QuestItem->testAddEntryAction("meet the king", "killTheKing");}

/////////////////////////////////////////////////////////////////////////////void AddExitActionThrowsWhenStateNotPresent(){ string err = catch (QuestItem->testAddExitAction("blah", "things")); ExpectEq("*ERROR - stateMachine: an exit action can only be added if both the state exists and the method to call has been implemented on this object.", err);}

/////////////////////////////////////////////////////////////////////////////void AddExitActionThrowsWhenAddingAnInvalidEntryAction(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!");

string err = catch (QuestItem->testAddExitAction("meet the king", "badMethod")); ExpectEq("*ERROR - stateMachine: an exit action can only be added if both the state exists and the method to call has been implemented on this object.", err);}

/////////////////////////////////////////////////////////////////////////////void GetStateDescriptionReturnsDescriptionWhenStateExists(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); ExpectEq("I've been asked to meet the king!", QuestItem->getStateDescription("meet the king"));}

/////////////////////////////////////////////////////////////////////////////void GetStateDescriptionReturnsNullWhenStateDoesNotExist(){ QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); ExpectFalse(QuestItem->getStateDescription("blah"));}

/////////////////////////////////////////////////////////////////////////////void AddExitActionSilentlySucceedsWhenEverythingValidates(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); QuestItem->testAddExitAction("meet the king", "killTheKing");}

/////////////////////////////////////////////////////////////////////////////void SetInitialStateSetsTheInitialState(){ // If anything were to go amiss, this would throw. QuestItem->testAddState("meet the king", "I've been asked to meet the king!"); QuestItem->testSetInitialState("meet the king"); ExpectEq("meet the king", QuestItem->initialState());}

/////////////////////////////////////////////////////////////////////////////void QuestStoryReturnsCorrectNarrativeForQuestStatesCompleted(){ SetUpQuestItem(); ExpectEq("[0;36mI've been asked to meet the king! I met King Tantor the Unclean of Thisplace. He seems to like me. The king asked me - ME - to be his personal manservant. Yay me! I told the king to piss off. I have socks to fold.[0m[0;31m [Failure][0m", QuestItem->questStory(({"meet the king", "met the king", "serve the king", "ignore the king"})));}

I think I could see this being used for mudlib code. For realms and domain code I think it might be a bit harder to teach content coders to do this properly but it's still a good idea. In the past I have noticed that a many content coders simply don't have much of a technical background so they might need quite a bit of help in implementing proper testing. Neat idea though.