On Fri, Dec 21, 2012 at 3:05 PM, Jeff Morriss <jeff.morriss.ws@xxxxxxxxx> wrote:
> Evan Huus wrote:
>>
>> On Fri, Dec 21, 2012 at 10:41 AM, Jeff Morriss
>> <jeff.morriss.ws@xxxxxxxxx> wrote:
>>>
>>> Evan Huus wrote:
>>>>
>>>> They've been on my to-do list for a while, as emem provides them.
>>>>
>>>> However, I've never personally used emem's canaries, and I've never
>>>> actually heard of or seen anyone else using them. Are they actually
>>>> useful anymore, or has Moore's law made valgrind the better tool in
>>>> all situations?
>>>
>>>
>>> Well, the canaries have helped us find (and fix) a *lot* of bugs over the
>>> years. I have this vague memory of a time when most of the fuzz failures
>>> complained of canary corruption but maybe that's an exaggeration.
>>> Hopefully
>>> the lack of canary corruption these days is a sign of improvement. :-)
>>>
>>> I think they're still useful for the automated fuzz testing because we
>>> get a
>>> fuzz failure when the fuzz-bot finds a corrupted canary. Valgrind is
>>> useful
>>> to let us humans *find* the memory corruption, but unless we're at a
>>> point
>>> where the fuzz-bot can run Valgrind instead of its normal testing, I
>>> don't
>>> think we should give up the canaries.
>>
>>
>> fuzz-test.sh has a -g flag that does exactly this. Is it possible to
>> enable that flag on the fuzz-bot or would that kill performance too
>> much?
>
>
> My experience with Valgrind is that it is *many* times slower. But I guess
> if it is more likely to detect every memory problem it might not matter.
Yes it's slow, but the fuzz-bot isn't an interactive thing, so as long
as it's not taking weeks to do a pass it shouldn't matter too much.
> But, at least in taking a quick look at that "-g" flag, the bot would also
> need a way to declare failure (so it would know when to raise a bug).
I believe lines 173-178 look for valgrind failures, but it's been a
while since I put them in so they may not work :)