Covergrind

An Source Code Coverage Tool

for Valgrind

Introduction

Covergrind is a code coverage tool for the excellent Valgrind framework. Rather than having to link against strange libraries or compile in various support, Covergrind uses run time instrumentation in order to work out which parts of your source code are traversed. There have been comments made about how trivial it would be to use the Valgrind framework to construct such a tool but I wasn't able to find one so here is mine to share.

How do I get it?

Covergrind is available as a "difference" file to the Valgrind Subversion trunk. It is maintained outside of the main Valgrind tool but you can get the current (Beta 2) patch from here.

How do I build it?

Once you have downloaded the latest Covergrind patch, you need to obtain a copy of the Valgrind source. Instructions for doing that are here. Assuming that you have checked out the Valgrind source into the directory 'valgrind', you can apply the Covergrind patch like this:

$> cd valgrind
$> gunzip -c /path_to/covergrind_patch_name.patch.gz | patch -p0

You can now follow the rest of the instructions here to build and install Valgrind.

Is there anything I need to do to my program before I run Covergrind on it?

For the absolute best results, your program should be compiled with "-O0 -g". Without debugging symbols, you won't get any output. If you don't turn off optimisation, you are likely to get garbage out as optimised code doesn't flow anything like your program would lead you to believe (ever tried to debug optimised code? If not, try it sometime and you will see what I mean).

Enough chat! How do I run it?

Covergrind has no runtime options of its own so you can normally run it like this:

$> valgrind -q --tool=covergrind <programname>

You might want to direct the output into a file though – see --log-file= (valgrind --help) for how to do that.

How long does it take to run things then?

It's a lot faster than Memcheck, mainly because it isn't doing anything like the same amount of work. I haven't tested against compiling in the standard basic block support and using gcov but it's on my “todo list”. If you beat me to it, please drop me a line.

Why have you done this?

At work, we need to get a better idea of how much of the code base our Factory Acceptance Test suite covers. Having introduced Memcheck to the company and subsequently written the Omega tool, making another plug-in for Valgrind seemed logical as it would be simple to swap tools in order to do coverage testing but without having to do anything to the programs themselves.

That patch has "Beta" in the name!

Covergrind is NOT perfect. I hope that it will get even better but it is perfectly usable NOW. It currently supports x86_64 (x86 support will be tested when I get the chance – x86_64 was the priority but it may well just work – give it a go and let me know!). I would welcome assistance to make this work on PPC32 and PPC64 but I have no hardware to test or build on so I would need patches rather than just helpful advice.

Beta 02 now outputs information in “kcachegrind” format (many thanks to Josef Weidendorfer for help and advice to get this working). The output is far from ideal and will be tweaked in Beta-03 but it should give you an idea of what is going on. Basically, if there is a number in the “lineTouched” column, that line was executed during your run.

I am more than happy to receive your comments, criticisms, bug reports and especially patches (although I make no promises to accept them). Any success stories would also be nice to hear about.

Who are you?

I am Bryan "Brain Murders" Meredith and you will find me on the valgrind-users and valgrind-developers mailing lists.

Covergrind is Copyright (c) 2006 Bryan "Brain Murders" Meredith and is released under the GNU GPL(v2).