Problem with LASTSTEP_OUTPUT and xml output

[mballou]

I'm using Vault 6.1 with the history command in my build script. Vault outputs the results in xml format which is perfect for my needs.

I'm writing a vbscript in Visual Build as the Stepdone event of the vault step, but it doesn't parse the xml correctly. I think I tracked the problem down to visual Build changing the xml text in the LASTSTEP_OUTPUT macro.

I use this line to get the text:
xmlText = vbld_AllMacros().Item("LASTSTEP_OUTPUT")

Instead of lines like this:
<item txid="225650" date="5/12/2006 8:57:35 AM" name="_Mainline/file.cpp" type="60" version="411" user="build" omment="Can't use single quote" actionString="Checked In" />

I get a line like this from the macro:
<item txid='225650' date='5/12/2006 8:57:35 AM' name='_Mainline/file.cpp' type='60' version='411' user='build' comment=Can't use single quote' actionString='Checked In' />

If you look closely, you will see that the double-quotes of the original output have been converted to single quotes. That by itself is still valid xml and can be loaded. My problem is that the output also contains single quotes which is not valid. the word "Can't" becomes invalid.

Is there some option that will let me get the real output of the command that hasn't been converted by Visual Build macro?

Thanks,
Mike Ballou

[kinook]

VBP converts double quotes in step output macros to single quotes so that if these macros are referenced in a script expression like "%LASTSTEP_OUTPUT%", double quotes within the value will not cause problems. But this can lead to the situation you're describing if the output actually contains XML attribute values with single quotes. We'll add an option in the next release to disable conversion of double to single quotes in the output macros. For now, you can work around it by converting back to double quotes in the script event before using:


' convert single quotes surrounding XML attribute values possibly
' containing single quotes back to double quotes
output = Application.ExpandMacrosAndScript("%LASTSTEP_OUTPUT%")
Set re = New RegExp
re.Pattern = "(\s*=\s*)'(([^']|'[^\s])*)'(\s)"
re.Global = True
output = re.Replace(output, "$1""$2""$4")
' do stuff with output

[mballou]

Perfect!

Thank you for the quick response. I look forward to the next version.
I tried your work around for converting the single quotes back to double quotes and it also works great.

Thanks,
Mike Ballou

[rmaines]

We are also experiencing the same problem. The workaround works great for any comments containing a single quote followed by an s at the end of a word, but we've already had several comments containing single quotes in other parts of the comment field, ex: comment="Comment 'containing' single quotes"

We are building continuously, so if a comment contains a single quote, no changes are found because the xml does not load. Is there a timeframe when a new version will be available?

[kinook]

The v6.2 beta (released today) has this capability. More details on the beta are here: http://www.kinook.com/Forum/showthread.php?threadid=1973

[rmaines]

I installed the beta and it fixed our problem. However, it introduces a new one....

The problem introduced is that in a step done event, I have a Builder.LogMessage statement, but the statement is not finding it's way into the log file.

When I uninstall the beta and reinstall 6.1, it's working fine, but I still have the single quote issue. Any ideas? let me know and i can send you the build project if needed.

[kinook]

Another change (actually a bug fix) in this build is to not fire the vbld_StepStarted/vbld_StepDone events for a step that is skipped. Is that coming into play here?

[rmaines]

Nope, this step is getting executed and the logic w/in the event is being processed, but the Builder.LogMessage output is no longer getting logged

[kinook]

Can you ZIP and post or send a .bld file that demonstrates the problem, and the info from Help | About | Install Info? Thanks.

[rmaines]

Attached are the bld files. Basically, the MediLinksCIC.bld fires the ChainMaster.bld. The ChainMaster fires a bunch that do basically the same thing. We can focus in on the MediLinks.bld one to keep it simple. This one is an example of the Builder.LogMessage not getting fired. You probably won't need the first 2 .bld's, but just so you can see our process in its entirety.

[kinook]

I can't find a Builder.LogMessage call in any vbld_StepDone event in the attached files. Please pare things down to a reproducible sample we can build here. Thanks.

[rmaines]

So sorry, i should have pointed out that the StepDone calls a Global Function which contains the Builder.LogMessage, which you probably don't get by me sending you the .bld files.

In the Build Solution step of MediLinks.bld, there is a call to CreateReportLink (the global function). It is in this function that the message does not get logged. Below is the global function note that it is also calling a URLEncode function which i've also provided below):

Sub CreateReportLink(LinkHeader, ReportDir, FileName)

Dim strHTML
Dim file

Builder.LogMessage LinkHeader

strHTML = "<HTML>"
strHTML = strHTML & "<BODY>"
strHTML = strHTML & "<a href='..\" & URLEncode(ReportDir & FileName) & "'>" & FileName & "</a>"
strHTML = strHTML & "</BODY>"
strHTML = strHTML & "</HTML>"

Builder.LogMessage(strHTML)

End Sub

Public Function URLEncode(sRawURL)
Const sValidChars = "1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmn opqrstuvwxyz:/.?=_-$(){}~&\"

If Len(sRawURL) > 0 Then
' Loop through each char
For iLoop = 1 To Len(sRawURL)
sTmp = Mid(sRawURL, iLoop, 1)

If InStr(1, sValidChars, sTmp, vbBinaryCompare) = 0 Then
' If not ValidChar, convert to HEX and prefix with %
sTmp = Hex(Asc(sTmp))

If sTmp = "20" Then
sTmp = "+"
ElseIf Len(sTmp) = 1 Then
sTmp = "%0" & sTmp
Else
sTmp = "%" & sTmp
End If
End If
sRtn = sRtn & sTmp
Next
URLEncode = sRtn
End If
End Function

[kinook]

LogMessage calls from script event code for a step with logging disabled were incorrectly suppressed. This is fixed in the latest beta download.

[rmaines]

Thank you very much. I don't see a new download though, just the same one (6.2). Will this be available soon?

[kinook]

It is, just download it again (it's still the v6.2 beta, only better). VisBuildBld.dll version (Help | About | Install Info) will be 6.1.9.37 after installing.

[rmaines]

Well, there's now yet another bug introduced resulting around the logging functionality, so i've had to revert back to 6.1 again. I think i will just wait until 6.2 is officially released so that it has been regression tested.

[kinook]

Can you describe the bug? Our testing is completed, so any info you can provide could help us identify and fix any remaining bugs before final release. Thanks.

[rmaines]

Well, for one, the logging doesn't work the way it used to when a build project is Aborted. Here's the short version:
We run our project from the command line. It runs continuously. Before, a person could close the window while the continuous integration build project was in "sleep mode", causing the build to end and the next build cycle would log just fine. W/the beta, whenever the build is stopped, the next build cycle's "Transform Log File" step would Fail because of missing xml elements, which is what the transform step is suppose to add. I didn't look too close at it though.

[kinook]

I haven't been able to reproduce that behavior. Using the attached sample, running it like

VisBuildCmd test.bld

from a Command Prompt and aborting by closing the Command Prompt or Ctrl+C correctly writes the closing XML tags to the log file, and running the build again and closing Notepad (so that it completes normally) successfully generates an HTML report from the XML log file.

Can you pare yours down to a reproducible test case? Thanks.

[rmaines]

I wish I had the time, but we're trying to get a release out...

But I can tell you the other thing I noticed which may be part of the problem. Our ..CIC.bld calls the ChainMaster.bld which subsequently fires other build projects. Before the upgrade, when running from the command line, I'd see logging output (in the dos box) for the individual chain build projects and their details. In the beta, the last thing shown is the step for executing the ChainMaster build. It doesn't show the output for the steps w/in the ChainMaster nor does it show the output for the build projects called from the ChainMaster like it did before.

[kinook]

The issue with echoing of chained program output from the console app has been fixed (actually, we added an app option [defaulting to the old behavior] to toggle echoing of chained console output on or off). I'm pretty sure that issue would not have any effect on file logging. I tried a stripped down version of your projects that would build here (attached; calling it like VisBuildCmd MediLinksCIC.bld) in the previous and updated beta builds but couldn't reproduce the issue with unclosed XML tags in the log file.

[rmaines]

the actual error is relating to 'too many top level elements' during the closing tags step (transform).