VisBuildPro.macros occasionally locking

[jdavidi]

Greetings,

We have 2 build machines, "A" and "B", both of whom utilize/update (via network share) VisBuildPro.macros on "A" when running similar yet separate nightly builds.

A failure routine is set up to kill the build process and email the error; two or three times over the last month has thrown the following error:

Failed to create '\\BuildMachineA\VBP6_files\VisBuildPro.macros': The process cannot access the file because it is being used by another process.

My first thought to get around this was to have my failure step search %FAILSTEP_OUTPUT% for this error and then just issue the equivalent of a Rebuild (Shift + F7) from script but after searching through the forums this doesn't seem to be an option (at least in v6.2a).

Was wondering if anyone had any ideas with regard to how to best mimic a build script Rebuild or otherwise prevent the .macros file lock?

Thanks!

--jdavidi

[kinook]

It sounds like machines A and B tried to update the global macros file simultaneously. If they truly do need to share updates to that file, it would probably be best to prevent both build boxes from performing a build at the same time.

One way to do that would be to exit (or wait in a loop) at the beginning of the build if a signal macro existed, creating that global macro next if it doesn't exist, and delete it at the end of the build (on success or failure).

If the macro changes don't actually have to be shared, another option may be to copy the global macros file locally and load the local copy (so changes during the build are saved there) at the start of the build.

[jdavidi]

Thanks for the reply!

Yeah, the two machines are trying to update one of my global macros each time an interative script is run, and while the probability is low that they execute the global script (where the global macro is getting set) simultaneously, there's still a few seconds of exclusivity in there I guess.

For this weekend's build run, I'm going to try to split said global macro in 2 (one for each machine) and also assign a temporary macro (named after the original global) with the same value so each machine can work with its own copy. I still need both machines to use the same macros file, as version comparisons constantly take place across the 2 machines, but I'm hoping that now that the machines won't be going after the same EXACT macro that this will reduce the chance of the lock even further.

Is this plausible, or is the fact that 2 machines are trying to update the overall macros file (as opposed to the same macro in the file) still make the overall process prone to locks?

[kinook]

VBP rewrites the entire .macros file when a macro changes, so that probably won't help.

[jdavidi]

"One way to do that would be to exit (or wait in a loop) at the beginning of the build if a signal macro existed, creating that global macro next if it doesn't exist, and delete it at the end of the build (on success or failure)."

Hi again,

After what I last wrote failed with the same error (as we both pretty much expected), I'm attempting to implement the signal macro suggestion above (from your 2/16 post).

Here's what I tried:

Right before they need to update a global macro, the build scripts will :

1) check if global "signal" macro %UPDATING_GLOBAL_MACROS_FILE% is defined (this is a repeatable step with a child "Wait" step of 1 second while %UPDATING_GLOBAL_MACROS_FILE% is defined)
2) assuming that it's not defined, create global macro %UPDATING_GLOBAL_MACROS_FILE%
3) Set/Update all the other global macros I need
4) destroy global macro %UPDATING_GLOBAL_MACROS_FILE%

However, the problem I'm still having is the scripts will either step over the loop in step 1 (because the .macros file doesn't have %UPDATING_GLOBAL_MACROS_FILE% when the build script opens) or it will stay in the loop indefinitely (if the .macros file DOES have %UPDATING_GLOBAL_MACROS_FILE% at the time the build script opens). The problem is that depending upon the 2 machines' timing

You mention that the entire .macros file is rewritten whenever a change is written to it; do I have any way of "refreshing" the macros file that an in-process build script is using?

Otherwise I'm not sure how to test for the signal macro without closing and reopening the build script.

We're so close, I can feel it :)
--jdavidi

[jdavidi]

And then I found this thread:

http://www.kinook.com/Forum/showthread.php?s=&threadid=2049&highlight=reload

I will try and let you know!

[kinook]

Before a step is built, any changes to the global macros in the current VBP instance are saved to disk.

After a step is built, global macros are reloaded if the global macros file was modified externally (by another VBP instance, etc.).

http://www.visualbuild.com/Manual/globalmacros.htm

[jdavidi]

Awesome, I think I've got the desired behavior going!

I was having trouble--even after reloading the global macros file via script--getting the other build machine to "unlearn" the existence of the %UPDATING_GLOBAL_MACROS_FILE%--even though the macro no longer existed in the .macros file on disk, it was still acting as though it was still defined and thus wouldn't exit my loop (I guess it was still in memory). I got around this by making %UPDATING_GLOBAL_MACROS_FILE% a permanent global macro and now flip its state between True and False when one machine needs to update it.

That being said, I modified steps 1 through 4 in my last post so that the updates to my global macros are handled exclusively via calls to separate VBP build scripts.

I have 3 such external scripts: one to set %UPDATING_GLOBAL_MACROS_FILE% =True (in order to lock the file), one that various procedures call to update my "comparison" macros, and another to set %UPDATING_GLOBAL_MACROS_FILE% = False.

I broke it out the %UPDATING_GLOBAL_MACROS_FILE% state-setting scripts this way so that there's as little time as possible between the macro value update and when it gets saved to disk. I just finished running through the same debug scenarios that were tripping me up yesterday and can no longer "break" it--it's entering and exiting the "wait for %UPDATING_GLOBAL_MACROS_FILE% = False" loop appropriately.

Thanks kinook for this fantastic solution!

If anyone sees any potential holes in this latest attempt please let me know.

--jdavidi