MockItNow
MockItNow is a C++ mocking framework I started in 2007 to improve my productivity when writing unit tests. I’ve been using it for nearly a year now in my projects at home, so I thought it was about time I shared it. It’s a little rough around the edges, but it provides some really nice features that I haven’t seen in other C++ mocking frameworks.
Currently the only compiler/CPU architecture supported is Microsoft/x86, however I’ve seen it working under gcc and PowerPC, so it can be done! In order to stay clear of any legal issues, I can’t make those versions available at the moment.
If you’re interested in the Microsoft/x86 version, then you can download it for free here. You are free to use MockItNow for any application, but a thank you credit would be nice if you find it useful.
If you’re interested in contributing a new version to the project, then please email me.
Considerations
Apparently there’s no such thing as a free lunch, so there are some things you will need to consider when setting up your projects to be mockable. For most of these, you won’t have to worry about them once you’ve gotten them right. A couple of them require a tiny bit of extra code to be written for each new type you want to record, but the I’ve found that this cost is small compared to the gains from using MockItNow.
Function Interception
MockItNow works by intercepting function calls and redirecting them in order to record and replay. It does this via a special flag that Microsoft added to their compiler in order to support profiling. This means that you have to be a little careful with your project settings to make sure that you don’t optimize out those function calls.
In particular, you need to disable:
- Inlining
- Incremental linking
- COMDAT folding
If these settings aren’t disabled then you’ll see the mocker failing to mock certain functions, or incorrectly mocking other functions. The need for compile setting changes also means that you’re not going to be able to mock functions from code you haven’t compiled yourself (like third party compiled libraries).
Also, you need to make sure that the classes you use with the mocker have default constructors. This is to ensure that they can be returned during the record phase, and that virtual functions can be fixed up.
Performance
There is a performance hit you take when enabling mocking, since you now need to do two function calls for every one you had before. In practice I haven’t had any problems so far with performance, however, you should be aware that this could be an issue. The good news is that there are ways around this problem. For example, you don’t pay the performance hit (but can’t mock functions) if you don’t specify the /Gh flag for the compiler. This means that you could have different configurations depending on whether you want mocking (and hence the unit tests) enabled or not.
Argument Comparison
MockItNow needs to be able to compare function call arguments. It defaults to doing this by using operator==. This means that if you write a class and want to be able to stub a function which takes that class as a parameter, then you need to supply an operator== for that class type.
You can actually customize the comparer for specific types to use other methods (compare memory, compare by address), but I wouldn’t recommend it.
Error Reporting
MockItNow reports errors by throwing standard library exceptions. The first thing you need to do is to make sure that exceptions are enabled in your project settings. Beyond this, you need to make sure that whatever unit testing framework you use (I use UnitTest++) catches std::exception in all cases. I believe that the latest version of UnitTest++ already does this.
In order to show friendly error messages to the user, MockItNow uses ostream to print out the expected and actual arguments. This means that for new types that you want to use as recorded function parameters, you need to supply an implementation of operator<< for ostream.
Usage
Once you have addressed the considerations above, you should be ready to start using MockItNow. Using the mocker is basically broken down into three steps: Register, record and replay. Although you can use the Mocker class directly as the MockItNow unit tests do, I’ve provided some macros to help out with doing most of the common tasks. All you need to do is to inlcude MockItNow.h in your test file, create a Mocker class in your fixture, and you’re ready to go!
Register
The first thing you need to do is to register functions that you want to be able to record and replay. The reason for this step is to allow the compiler to generate replacement functions for the original which will do the actual recording and replaying of function arguments and return values. The following are some examples of using the register macro:
REGISTER(FreeFunction);
REGISTER(Foo::MemberFunction);
REGISTER(Foo::StaticFunction);
That’s all you need to do for most cases! If the function is overloaded, then you need to help the compiler out and tell it which overload you want to register by providing the function signature:
REGISTER_OVERLOADED(Foo::OverloadedMemberFunction, bool (Foo::*)(int, float));
Once a function is registered then it is allowed to be replayed, but there is no expectation that it will be called, so if you try to replay it without recording it first, then you will call the original function.
I found that sometimes this behavior is not what I want. In particular, I want some functions to always return a specific value whether they were recorded or not. For this reason I added another macro to tell the mocker to always stub out a particular function. If the function has a return value, then you must provide the value you want to use at register time:
REGISTER_STUB(Foo::FunctionThatReturnsVoid);
REGISTER_STUB(Foo::FunctionThatReturnsInt, 12345);
As soon as you put the mocker into replay mode during the test, the above functions will never be called, and instead the replacement functions will run, and any supplied return values will be returned. Note that using this macro does not set up an expectation inside the mocker, so if you don’t call these functions then it won’t complain.
Record
Once you have registered your functions, the next step is to set up some expectations. These are basically an ordered list of function calls you expect to see called when the code under test is run. For functions with a return value, you must supply that return value to the mocker at record time using a macro. For functions that return void, you can just place the function call inside the record block, or optionally use a macro.
The following example puts the mocker into record mode, sets some expectations, and finally puts the mocker into replay mode (this happens implicitly when the record scope is exited).
Foo foo;
RECORD
{
foo.FunctionThatReturnsVoid();
EXPECT_RETURN(foo.FunctionThatReturnsInt, 54321);
EXPECT(foo.FunctionThatReturnsVoid);
}
During the record phase, you can tag on some functions to the end of the EXPECT macros which instruct the mocker to change its behavior during the replay of that function:
- Ignore(X): This tells the mocker to ignore differences about specific aspects of the function during replay. These can be particular arguments (Argument1, Argument2 etc.), all arguments (AllArguments), or the object a member function is being called on (This).
CallOriginal(): This calls the original function. You’ll probably only need this if you’re trying to do something with recursive functions.
- FailWhenCalled(): This tells the mocker to expect that the function won’t be called, and to fail if it does get called.
- ReturnArgument(X, Y): You can use this to return values into arguments passed in as non-const references.
These can be chained together however you like:
Foo foo;
int result;
RECORD
{
EXPECT(Foo::FunctionWithOutParameter(result, 2, 3, 4)).Ignore(Argument1).ReturnArgument(Argument1, 123);
}
Sometimes you may want to always return a certain value from a particular function. You can use the RETURN_ALWAYS macro to do this. Unlike the other expectations, the order of this function call in the RECORD block doesn’t matter.
Replay
There’s nothing more to be done when you have finished recording other than to run your test code. During this phase, the mocker looks for functions to replay using the following rules:
- Is there an explicit expectation set using EXPECT or EXPECT_RETURN, and does the object (if it is a member function) match?
- Is there a RETURN_ALWAYS expectation set, and does the object (if it is a member function) match?
- Was the function registered using REGISTER_STUB?
If all three conditions fail, then the function is not replayed.
When the mocker is destroyed, it runs through the list of expected function calls, verifies that they were called, and that all of the arguments matched the recorded arguments.
Examples
I have provided a project illustrating how to use MockItNow on Google Code, you can het it here.
This file includes the Examples project, the latest version of MockItNow, and UnitTest++. The projects are provided in Visual Studio 2005 format, and should compile out of the box.
Please note that this is a slightly modified version of UnitTest++, since the current release doesn’t catch std::exception in all cases. If you already use UnitTest++, you may just want to look at the diffs to see where I needed to add catch blocks.
I’ve provided three configurations for the Examples project: Debug, Release and Final. Debug and Release configurations both run the unit tests, but Final has optimizations turned on, and so does not run the tests.
If you’re just interested in seeing what the the test code looks like when using MockItNow, you can browse the example file.
24 Comments so far
Leave a reply
Hey,
Just wanted to say thanks for this! It’s a fantastic tool. Really appreciate you putting the time in.
For others trying to use this, the only thing that bit me was that the example project defaults to the Final configuration which has optimizations turned on. The result is that most of the tests will fail with that config. This is noted on the post dedicated to the example project but not here so I was clueless for a while. Hope that helps someone.
Cheers!
-Tim
Thanks for the feedback Tim! It’s nice to know that someone else out there is finding MockItNow useful. I look back at the time when I was having to inherit from class in order to stub its functions and shudder!
I don’t think that I am able change the default build configuration for the solution, since I think that’s stored in the per-user settings.
Still, I’ve added the information from my post to the bottom of this page which should help.
Apologies for the confusion!
Rory
Hi,
Great stuff! Thanks for releasing this.
I have trouble to use it in Linux environment. Have you a success it with Linux and g++?
Thanks
cheol
Hi,
I’ve never actually used it in Linux before, but I have seen it working using gcc on the ps3, where we used the following flags when compiling:
-Wall -Werror -fexceptions -finstrument-functions -mbase-toc
The instrument functions flag tells the compiler to insert the hooks. To intercept the function calls, you’d just have to replace the current thunk file with something like this:
namespace MockItNow
{
int g_jumpAddress;
extern “C” void __cyg_profile_func_enter(void*, void *) __attribute__((no_instrument_function));
void __cyg_profile_func_enter(void* callingFunctionAddress, void*)
{
// Same implementation as before
}
extern “C” void __cyg_profile_func_exit(void*, void*) __attribute__((no_instrument_function));
void __cyg_profile_func_exit (void*, void*)
{
}
}
This is all just off the top of my head, so take it with a grain of salt. One thing in particular I’m not sure about is the location in the prologue of the jump to the profiling function. It is inserted at the very top of the prologue by the MS compiler, and if gcc does the same then it should just work. Otherwise you might need to change the prologue in the thunk to account for this.
I don’t have a linux box I’m afraid, so I can’t actually test all this out. If you get it working, then let me know and I’ll add you as a contributor to the project.
Rory
The example program runs great. However, we use CppUnit here. MockItNow seems to rely on UnitTest++ functionality, and I have been unsuccessful at getting it working in with CppUnit. Any suggestions would be helpful.
Larry
Hi,
I haven’t tried it with other unit testing frameworks, but the only changes I made to UT++ were to add catch blocks for std::exception. What kind problems are you seeing? If you have the test app set up with CppUnit, you can mail it to me (rory) at this domain, and I can take a look at it.
Rory
Hey, looks like this could be very useful, thanks.
I’ve been looking, but I can’t yet see a way to mock ‘out’-reference parameters. i.e. void getMultipleRefs( A &a, B& b, C&c);
where the values of a,b,c are something i want to have during my test method, but getMultipleRefs is pulling their values in from other systems - so I want to mock the method, and set the value of the refs. Of course, I could just refactor the method to return a class containing the values I want - but I will have to get buy-in on TDD first!
Any thoughts?
Cheers,
Lance
Actually, it seems I’m an idiot, or can’t decipher camelcasing - just saw NoReturnWithOutArgument while looking at another problem
Haha. That’s ok. I just replied on google code. But yeah, that’s how you do it, using ReturnArgument.
Are there any plans to introduce exception based expectations? For example:
ExceptionClass exception(”Some error message”);
ExceptionClass exception1(”Some other error message”);
REGISTER_THROW(SomeFunction, exception); // “Stubs” a function to always throw the specified exception
…
RECORD
{
EXPECT_EXCEPTION(foo.SomeOtherFunction(), exception1); // function throws the specified exception when called in replay mode (note calling this macro EXPECT_THROW would clash with those that use the Google testing framework)
}
…
This functionality would help a lot for mocking test cases to simulate errors and the like.
Well, I didn’t have any plans to add something like this because I don’t use exceptions at the moment. Luckily enough it was pretty easy to add, so I’ve updated the zip file and source on Google Code. I pretty much went with the example you gave, but I did use EXPECT_THROW because it’s a bit shorter. You can always rename the macro to whatever you want inside MockItNow.h
I added a couple of really simple tests in TestExamples.cpp. Please let me know if there are any issues with this new feature, since I don’t have a wide variety of tests for this. It’s pretty simple though: It just stores the exception value like it does for the return values, and throws it during the replay.
The function call arguments are still checked by default, so beware!
That’s great Rory, exactly what I was looking for. Thanks for the swift response.
Hi Rory,
first of all thanks for this nifty project! I worked on some quite similar library until I found your code…
Nearly everything is covered by your examples, but one wish left. Is it somehow possible to extend the “OutArguments” FIXTURE to handle pointers instead of references? For example
void NoReturnWithOutArgumentPointer(int *a)
{
*a = 1;
}
{
REGISTER(Foo::NoReturnWithOutArgumentPointer);
int b = 12345;
int a = 0;
RECORD
{
EXPECT(foo.NoReturnWithOutArgumentPointer(&a)).ReturnArgument(Argument1, &b);
}
foo.NoReturnWithOutArgumentPointer(&a);
CHECK_EQUAL(12345, a);
}
This would allow some nice mocking of leagacy C code where you often have pointers as parameters.
Emanuel
Hi Emanuel,
That sounds like a good idea. I don’t think it would be too hard. I’ll take a look at this sometime this week.
I would think that you would want to pass the actual value to return (rather than the pointer) to mockitnow, since the pointer may change as the test runs.
I may have to make a separate function for returning pointer arguments since the code will be a bit different. I’ll probably change it to ReturnPointerArgument/ReturnReferenceArgument, but leave the old ReturnArgument in there for a while so that I don’t break tests.
Rory
Hi Rory,
thanks for your fast answer. I will stay tuned…
I just updated MockItNow on Google Code. The examples should compile correctly now. Also, I added the ability to set return values for arguments passed by pointer. I added a couple of quick tests and it all appears to be working fine. Let me know if you have any problems.
Hi rory,
I played a little bit with your new pointer extension and have to say: nice work, that is exactly what I was looking for … except for one thing - char* !
If I try to mock a function with a char* parameter MockItNow.CopyArgument.Copy will only copy the first character during the replay. I think this is due to the default copy-constructor. Considering that much of my legacy code looks like this
int GetName(char *name, int len);
only the first character is not really sufficient
Maybe some kind of functor in the record stage as substitute for the missing copy-constructor could do the trick. I thought about something like this
void fillBuffer( char *buffer ){}
void myCopyConstructor( char *dst, const char *src)
{
strcpy(dst, src, strlen(*src));
}
…
RECORD
{
EXPECT(fillBuffer(buf)).ReturnArgument(Argument1, mockBuf, &myCopyConstructor); // default NULL if not provided
}
I know, really rough implementation but it should work. What do you think?
Cheers
Emanuel
Hi Emanuel,
You’re right that I didn’t think about that case. It’s actually easy to change the way any type is copied thanks to template specialization. I added the new copy function for char* and added a new test in the examples where it overrides the returned string. There’s a new zip file and new source uploaded to google code.
If you just want the fix, then copy the char* CopyArgument struct from this file: http://code.google.com/p/mockitnow/source/browse/trunk/Library/ReplayMockedFunction.h
There was one other small change by the way. I realised there was a bug in one of the error messages which caused a null pointer access.
I hope this does what you need.
Cheers,
Rory
This looks really good. I was hoping to use it to mock a 3rd party lib we’re using (open source), unfortunately some of those objects have non-public default constructors (they’re created with friend factories).
We’d have to conditionally compile some changes to the lib that gives us accessible default constructors, and that’s getting messy.
So close.
PS: Cheers from an ex-HMS developer!
Nice to hear from you Michael! I assume the issues you are getting are related to the Storage template class in MockItNow since that uses the default constructor. Is that right?
Looking at it, I can’t see a reason why it should have to do this. It could potentially store a pointer, and then use the copy constructor to new the object. Of course, you’re still screwed if the copy constructor is also private.
I guess the only couple of options you’d have if everything is private is to make it public (yuck), or to make the MockItNow Storage class and the test class friends. That’s also not very nice. It’s just another example of where really well encapsulated code makes things harder to test I suppose.
The error I was getting occurs in FixVirtualFunction, which has a local T (needing the default constructor). But the other spot you mentioned looks like it would also cause a problem.
As you say, I think my only option to use MockItNow would be to alter the 3rd party lib (OGRE, open source), at least for testable builds. Not very appealing.
For now I’m going down the path of adapting the library classes, and using mock subclasses of those adapters in tests. It’s a lot more work initially.
But I’d also like tests to run on Mac OSX, and I don’t know enough about that to reproduce the MS compiler flags. So I’ll probably stick with another mocker for now. Good work anyway!
Hi Rory,
thanks a lot for this great tool. I had a to make a few changes to get it compiling in my circumstances and to meet my needs, is there any way of feeding these back as they might be useful additions for other people?
ed
Ed, if you’d like to email me then I can add you as a ‘project committer’ to google code.
Likewise to anyone else reading this, I’d love to have other people help out with MockItNow. If anyone fancies porting it to work using gcc then there appear to be a lot of people who would be grateful.
As long as the changes don’t break anything that’s there now, then I’m fine with whatever you need to change. You can email me rory at rorydriscoll dot com.
Can this ever work with VC6? Seriously
Or will VC6’s inadequate template support will make this impossible?
I have compiled the sample code, and it has trouble with TypeTraits.h (”template class has already been defined as a non-template class”). Similar error I get trying to build other mock/unit test libraries.