beatworm.co.uk

There is a top level navigation menu at the foot of the page

Tracing Unit Tests with the XCode 3 Debugger

XCode has a nifty integrated debugger which is really a pretty wrapper around gdb. It lets you point and click, and drill down on things within the gui with ease, but still preserves access to the underlying raw gdb console and output. You can create breakpoints and watches, both literal and dynamic, step through your application as it runs, all the usual stuff.

I'm not the world's greatest user of debuggers. I'm more likely to trace through things until they make sense using some combination of logging, print statements, paper and pencil, or my absolute favourite, just explaining your mystery problem out loud to a nearby third party, embarrassing yourself by spotting the obvious bug mid-flow. That last one sometimes even works with the dog. Sometimes though, you're stumped, and you want to set some watchpoints, step through your program as it executes, or just generally prod things mid-run, and poke around under digital rocks.

Something I've been trying to practice recently is Test Driven Development. XCode 3 ships with support for the OCUnit testing framework built in. You can add a Testing target to your XCode project, and build up test case classes that use this framework, and the build tools know how to run these through the test harness. And so you progress, write a test for a feature, run the test harness, write code to pass the test harness, repeat. It's a great way of not only catching certain classes of bug before they happen, but perhaps more interestingly imposing a more minimal design focus on your application as you build it; you're automatically casting yourself more in the mind of a consumer of your application services, something I find really helps avoid over-design.

At some point though you are likely to run into some kind of hard to understand failure case within a unit test, and find yourself reaching for the debugger. And then finding that the debugger doesn't work. This is because the runtime of your unit testing target is actually the separate test harness framework, and not your application target. The test harness is a regular application that's dynamically loading your test classes and running them. In order to be able to use the IDE to debug your unit tests, you just need to do a little extra configuration within your XCode project, as follows.

The tool that runs the tests is called otest. You need to add this to your XCode Unit Test target as the executable. You can do this with the command 'New Custom Executable' in the Project menu. Add /Developer/Tools/otest.

Once it is added, set it as the active executable for the Unit Testing target, using Set active executable, in the Project menu. A green tick badge appears over the active exectuable in the xcode source list.

otest as active executable

The otest tool expects to be run with a certain environment, and arguments. There's a man page that describes them. You could run gdb against the otest executable from a shell in this fashion, but it means switching away from XCode. Alternatively, you can set up XCode to provide these when it runs your target by double clicking the otest executable in the source list to bring up it's inspector. The runtime settings you need to set are all on the Arguments tab.

Add two arguments -SenTest Self and the name of your Unit test bundle, which will be the name of the Unit Test target with a '.octest' suffix e.g. "My Unit Tests.octest". The quotes are important, if you have whitespace in your bundle name. Make sure that the order in the inspector list has '-SenTest Self' as the first element, and the bundle name the second, so that when they are concatenated to a command line, the switches come before the bundle name.

otest executable arguments pane

You also need to set two environment variables, in the lower pane of the arguments inspector, so that the dynamic linker can resolve your test components. The lower pane of the Arguments tab covers variables. Add two items to this list, DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH. Set both of these to be $(BUILT_PRODUCTS_DIR) which is the variable xcode build will populate with the correct destination of your compiled test cases object code.

otest exectuabel environment variables pane

With all of this set, you can just use the debugger within XCode. Click to set break points within the editor as you write your test cases, and the debugger will spring into action appropriately, whenever you build and run the test target.

This entry was posted on Thursday, February 7, 2008 at 15:56 in computers, programming.
You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

13 Responses to “Tracing Unit Tests with the XCode 3 Debugger”

  1. Quinn Slack Says:

    Thanks. This worked great for me. I also had to set the OBJ_DISABLE_GC environment variable to YES in the otest executable info window. That's something that iPhone developers will have to do (since the iPhone isn't GC'ed), so I wanted to make a note of it here.

  2. nicolas rolland Says:

    Pretty cool. I would never have done it so fast without those explanatios.
    Thanks mr Strickland. I also had to set OBJ_DISABLE_GC to YES, otherwise I get the error message below (So thanks mr Slack as well)

    GC capability mismatch

    The test bundle at UnitTests.octest could not be loaded because its Objective-C runtime information does not match the runtime information required by the test rig. This is likely because the test rig is being run with Objective-C garbage collection enabled, but the test bundle does not support Objective-C garbage collection. To disable Objective-C garbage collection for the test rig, run it in an environment with the OBJC_DISABLE_GC environment variable set to YES.

  3. Mark Says:

    I am having problems getting the test tow work. Here's the output – any ideas?:

    2009-04-10 12:28:28.643 otest[71012:10b] Error loading /Users/mark/code/iPhone/Grabs/build/Debug-iphonesimulator/NetworkTests.octest/NetworkTests: dlopen(/Users/mark/code/iPhone/Grabs/build/Debug-iphonesimulator/NetworkTests.octest/NetworkTests, 265): no suitable image found. Did find:
    /Users/mark/code/iPhone/Grabs/build/Debug-iphonesimulator/NetworkTests.octest/NetworkTests: mach-o, but wrong architecture
    2009-04-10 12:28:28.707 otest[71012:10b] The test bundle at NetworkTests.octest could not be loaded because it is built for a different architecture than the currently-running test rig (which is running as unknown).

    [Session started at 2009-04-10 12:28:28 -0700.]
    2009-04-10 12:28:28.726 otest[71013:203] *** NSTask: Task create for path '/Users/mark/code/iPhone/Grabs/build/Debug-iphonesimulator/NetworkTests.octest/NetworkTests' failed: 8, "Exec format error". Terminating temporary process.

  4. cms Says:

    @Mark – I'm not sure. It seems like the test bundle isn't loading because otest doesn't have an architecture specified (Unknown). You can set an environment variable ARCHPREFERENCE to specify a preferred binary architecture for your system – perhaps setting this in the environment section of the arguments tab for otest to an appropriate value for the architecture you're trying to debug ? Valid values are documented in the man page for the 'arch' command.

  5. cms Says:

    @Mark – grepping through the shell scripts in /Developer/Tools/ that are normally used to run otest for unit test targets might be illuminating. It looks to me like they use the arch command directly to invoke otest, picking up the architectures to test for from XCode via several environment variables, or falling back to the default system architecture, derived by running 'arch' with no arguments.

    I don't think you can use the 'arch' utility in the same way, because you need otest to be he executable so that you can debug it as it runs your classes. But setting the value of ARCHPREFERENCE in the otest environment to your default architecture might suffice.

  6. UIKIt requires garbage collection? Huh? - iPhone Dev SDK Forum Says:

    [...] the OBJ_DISABLE_GC environment variable to YES in the otest executable info window". beatworm.co.uk Blog Archive Tracing Unit Tests with the XCode 3 Debugger __________________ ~~ DO NOT EAT THE NEWBIES ~~ Intro to objective C and When to retain + [...]

  7. Max Hillaert Says:

    I'm getting this output:

    GC: forcing GC OFF because OBJC_DISABLE_GC is set

    Error loading /Users/myname/Projects/MyProject/build/Debug-iphonesimulator/Tests.octest/Tests: dlopen(/Users/myname/Projects/MyProject/build/Debug-iphonesimulator/Tests.octest/Tests, 265): Library not loaded: /System/Library/Frameworks/UIKit.framework/UIKit
    Referenced from: /Users/myname/Projects/MyProject/build/Debug-iphonesimulator/Tests.octest/Tests
    Reason: image not found

    The test bundle at Tests.octest could not be loaded because its Objective-C runtime information does not match the runtime information required by the test rig. This is likely because the test rig is being run with Objective-C garbage collection disabled, but the test bundle requires Objective-C garbage collection. To enable Objective-C garbage collection for the test rig, run it in an environment without the OBJC_DISABLE_GC environment variable.
    *** NSTask: Task create for path '/Users/maxhillaert/Projects/MyBookShelf/build/Debug-iphonesimulator/Tests.octest/Tests' failed: 8, "Exec format error". Terminating temporary process.
    2009-11-15 12:55:55.610 otest[13234:203] Usage: otest [-SenTest Self | All | None | ]
    2009-11-15 12:55:55.612 otest[13234:203] *** -[NSConcreteTask terminationStatus]: task not launched

  8. cms Says:

    @Max

    Have you tried explicitly setting the OBJC_DISABLE_GC for the otest executable in it's inspector ? ( see comments #1 and #2 from @Quinn and @nicolas )

  9. Max Hillaert Says:

    I removed UIKit framework from the 'other linker flags' setting in my test configuration. Now the problem is that even though otest will run my tests it will not break at my breakpoint. Are there other configuration settings I need to specify? How do I go about solving this problem?

  10. Max Hillaert Says:

    @cms: yes I have done that.

  11. Max Hillaert Says:

    Found it: Needed to turn off 'Lazy Loading' of symbols in the xcode preferences. Working in xcode for the past year, really makes me appreciate Visual Studio at my day job! ;)

  12. cms Says:

    @Max – cool ! I made a previous post about that problem too :-)

    http://beatworm.co.uk/blog/computers/smartypants-ides-considered-harmful/

    XCode is improving all the time, and it does have some nice features.

    I'm afraid I've not used Visual Studio ( Visual C++ even) since around, maybe 1996, I really hated it back then! Software sucks.

  13. Elise van Looij Says:

    @Max, your problem seems to stem from non-matching build settings. To match them, double-click on each target to bring up the Target Info and click on the Build tab. You can arrange the windows next to each other and scroll through them, comparing the settings. As far as garbage collection goes, "Supported" is the most flexible option.