The failures don't appear to be limited to the test steps. Earlier we had the following cluster of failures:
x86 builder:
Branch Build Step Start Stop
master 1680 ctest 08:38:18 09:02:45
master-3.2 365 ctest 08:27:02 08:55:58
master-3.0 133 WiX pkg 07:56:04 08:16:38
master-2.6 221 NSIS pkg 07:43:59 08:05:01
x64 builder:
Branch Build Step Start Stop
master 1604 ctest 08:22:58 08:47:45
master-3.2 369 ctest 08:24:09 08:48:15
master-3.0 322 NSIS pkg 07:36:14 07:57:08
master-2.6 212 WiX pkg 07:35:23 07:59:54
Each builder is an EC2 instance at AWS, and the AWS console shows that the CPU on each instance was 100% during that time. The obvious fix is more and/or faster Windows builders), but I also noticed that tshark does more work than it should in many cases. If I open Process Monitor and set the filters
- Operation contains "Process Create"
- Path contains "<my Wireshark development folder>"
and run test/test.py, procmon shows that we create 5573 process for 576 tests. For the decryption suite, procmon shows 970 processes for 89 tests. Most of the created decryption processes are extcaps, which seems odd for those particular tests. If I rename the extcap directory, procmon shows that 97 process are created and the suite runs much more quickly (50s vs 93s).
On 7/2/20 8:02 AM, Graham Bloice wrote:
> The tests are run all in parallel, and I'm not sure if the PD is a spot or reserved instance which might affect available CPU cycles.
>
> On Thu, 2 Jul 2020 at 15:58, Dario Lombardo <lomato@xxxxxxxxx <mailto:lomato@xxxxxxxxx>> wrote:
>
> You're right, it's not the test I pointed out. However it worked fine before, then I would check the host to see if it's suffering.
>
> On Thu, Jul 2, 2020 at 4:51 PM Graham Bloice <graham.bloice@xxxxxxxxxxxxx <mailto:graham.bloice@xxxxxxxxxxxxx>> wrote:
>
> The decryption test might need a bigger timeout, currently at 600 seconds:
>
> 9/34 Test #3: suite_decryption .......................***Timeout 630.47 sec
>
>
> On Thu, 2 Jul 2020 at 15:39, Anders Broman via Wireshark-dev <wireshark-dev@xxxxxxxxxxxxx <mailto:wireshark-dev@xxxxxxxxxxxxx>> wrote:
>
> Hi,____
>
> We have intermittent false petri dish failures. Can someone with insight into the tests have a look?____
>
> Best regards____
>
> Anders____
>
> Example Builder Windows Petri Dish x64 failed (failed ctest) - http://buildbot.wireshark.org/petri-dish/builders/Windows%20Petri%20Dish%20x64/builds/11913____
>
>
>
> --
>
> Naima is online.
>
>
>
>
> --
> Graham Bloice
>
> ___________________________________________________________________________
> Sent via: Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
> Archives: https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
> mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe
>