VBP (64-bit) doesn't terminate at end of build

[jbenson]

I've been trying to set up a new build VM (Windows 8.1 64-bit on VMware). Unfortunately, Visual Build Pro 8.7 (64-bit) has been very "flaky" and unreliable during testing. The VBP is launched as a Windows scheduled task, runs under an administrator logon and the "Run with highest privileges" option is checked. No user is logged into the VM when the VBP task is launched.

I've done a lot of tinkering with the build project and made improvements but it's still not good enough. Some days - infrequently - it runs the entire build without trouble. Most days, however, it will get "stuck" on some build step and won't proceed beyond that point. I am basically out of ideas how to get past this.

Lately, I've found a combination of project steps that reliably runs to completion (i.e., the important build steps are all successful). The only problem is VisBuildCmd.exe doesn't terminate at the end but instead remains running. The Windows scheduled task is defined to terminate after a certain number of hours but it still leaves the VBP executable in memory. This tells me that VBP still suffers from some problem and can't be trusted.

Things I've tried:

* VisBuildPro.exe vs. VisBuildCmd.exe (the latter is currently in use)

* /MTA command-line option (seemed to have no effect or even make matters worse)

* Replace "Copy Files" VBP steps with alternatives (build no longer sticks on file copy operations, so this is an improvement)

* Implemented or confirmed all the other suggestions in the "stuck builds" FAQ

* Disabled the "Send Email" build step at the end of the project since VBP almost always sticks on this step (nothing written in the build log). By contrast, a "Send Email" step I added at the start of the build always works.

This morning I found three instances of VisBuildCmd.exe running in Task Manager - one instance from today's scheduled task, the others left from two previous days' builds. I ran the Sysinternals utility, Process Monitor, in an attempt to find information about these processes. According to ProcMon, the most recent VisBuildCmd.exe process seems to still be "active". Every minute or so, VBP engages in a flurry of registry activity, some of it involving HKLM\Software\Licenses, along with a small amount of file activity (C:\ProgramData\Temp, C:\ProgramData\Licenses folders). The two older processes appear to be entirely idle, according to ProcMon.

Can anyone suggest reasons for the above behavior? Or offer any ideas for how to get VBP to behave better in our environment?

Thanks