Tool to merge more VBP scripts?

[Seeker]

Hi there.

I am looking for a tool that can merge several VBP scripts into one. Our problem at hand is that we use a component based structure where different parts of the build process are located in smaller scripts which are in turned call from a main script.

This is great for delegating responsibility for the smaller aspects of the build process and integrates well into the local policy of both CI, nightly and release builds. But it does make the overview and documentation, overly complex and we are therefore searching for a tool (fx a simple Xpath util) which can merge the different components used in the main script into a consolidated main script which holds each step explicitly.

Anyone have any thoughts on this?

-- Mikael Jensen

[kinook]

http://support.microsoft.com/kb/q286817/

[Seeker]

Good resource if I wanted to create the merge from scratch, but I was hoping for an already existing tool, which already hold the logic needed to ensure the validation of VBP data and functionality.

Unless this consolidation method is supported and ensured by Kinook?

[kinook]

We don't have anything specific to VBP project files for merging via XSLT. You could also use the VBP object model to do this. Send your license info (Help | About | Install Info) to support@kinook.com and we can assist with that if desired.

[Seeker]

I was writing the requested mail yesterday, when it hit me, that most of what we want can be implemented as global subroutines and managed from central network "globals" repository.

The idea hasn't been completely thought through, but I think it will be an easier solution, then implementing the first mentioned process.

I'll try and explain my self:

The original thought was to make a utility/toolset which could handle merging of a collection of working production build scripts into a completely detailed and consolidated build script based on a main build script which holds references to first mentioned scripts as “VisBuildPro Project” call/project steps.

This was primarily for documentation and maintenance of the scripts. Because reading a script with 20 calls to other VBP scripts with no direct access to their content without doing a testrun, would become quite frustrating.

The setup at hand is several multisolution .Net products, build on several build machines, all by independently maintained buildscripts. This need to change to lower the administrative/maintenance overhead.

The project at hand is the streamlining of the build process and migration of these several large build scripts with very few VisBuildPro-calls to what ever structure provides the best solution for us.

The original thought was to create many smaller cross-product compatible scripts and a main-script pr product which only holds references to the smaller scripts by VisBuildPro-calls. These main-scripts would be fairly simple, as they aren’t to hold any other information then references, but also gave the problems mentioned with documentation and maintenance.

The project is still at and VERY early stage and I’m basically just collecting the initial information for the project specifications, but is it estimated that a “script-splitting” in some form or another, would improve consistency between applications, reduce the time need to implement new products and braches, improve the support for different build demands (nightly/release/continues integration) and make build process setup, test and responsibility mapping easier.
The original thought was to substitute each “VisBuildPro Project”-steps with a group step that have the name set with the name of the file called.

My question is now:
Does this sound like complete gibberish? As mentioned, this is still in early planning phase and I’m basically just throwing out ideas based on my very limited experience with VBP.

Is there another way that this kind of functionality is usually implemented

First and initial idea was the tool to merge several smaller script components in a main build script.
Would this provide the advantages we expect or is there and easier way around the problem that eliminates the need for a tool like that? The idea spawned from the apparent lack of an include function or something similar(?), which can include functionality from additional script and still make the information available from the GUI of the main script.

The second suggestion is where all functionality was imported as global subroutines/scripts/macros has been suggested, firstly this would require all build scripts to always include all subroutines (including a lot of unused functionality in even the most basic scripts) and the cost of this is unknown. Secondly, this requires all VBP installation to get the globals file locations changed, a feat I'm still not sure about how to do best. I was thinking along the line of implementing a prebuild step that loads a registry file holding the globals-path. This would require the machine to run the step at least once for the globals to be viewable from the GUI. I have seen the "Application.Steps.Load" function, but since (to the best of my knowledge) it is not possible to run code on load of the script, this would not provide any advantage over loading a registry file?

Can you recommend any changes or best praxis to the suggested strategies and your advise on what to choose?

I've send the Q pr mail as well, including license info, so you know I'm not a hack.

[kinook]

You can access a chained (child) build project from the parent by:

1) View | Shell | Launch on the menu (Ctrl+Shift+L)

or

2) Opening the step properties dialog (Enter/double-click), switching to the Project tab (if not already there), and clicking Shell -> Launch.
http://www.visualbuild.com/Manual/steppropertiesdialogbuttons.htm

So just leaving your structure as it is may be fine.

You could also convert some or all of your child projects to global and/or project subroutines and call them rather than child projects. While it is possible to automate this via the object model, it may be better to do the conversion manually one step at a time (you may be able to refactor some duplicate steps into a single subroutine, etc.).
http://www.visualbuild.com/Manual/objectmodel.htm
http://www.visualbuild.com/Manual/scriptsample.htm

As far as boot-strapping the loading of global subroutine steps, a few ways you could do that:

1) Use a Run Script step at the start of the project to load alternate/additional global subroutine steps, using VBScript code of

Application.Steps.Load "c:\path\to\filename.steps"

http://www.visualbuild.com/Manual/macrosloadmethod.htm

2) Create a build project that retrieves config data from source control (or a network path) to the default config file path (%VISBUILD_CONFIG_DIR%). Run this build once on each build box to prep for future builds.

3) As part of the initial setup of a build box, create a ConfigFilesPath registry key to point to an alternate location for configuration files.
http://www.visualbuild.com/Manual/configurationfiles.htm

4) Launch builds from the command-line, using either the /configpath (to load all config files from an alternate location) or /steps flags (to just load alternate global sub steps)
http://www.visualbuild.com/Manual/command_linegui.htm

[Seeker]

Launching from GUI:
Thanks. That helps a lot. Doesn't quite eliminate the problem, but sure makes it a lot easier to manage with quick access to referenced scripts.

Script restructuring:
Some of the scripts are already heavy users of subroutines, but project subs are not really a possibility here, since this exactly the problem we are facing now. To many independent scripts and to much maintenance. I think that in any case, global subroutines is a must and implementing smaller "component" script would simply increase the complexity and not give any noticeable advantage over subs (am I right here?).

I've made a simple .reg file that maps the file location/configuration path to a network share. It should be overcomable to ensure that the fil is registered on all build-related machines before they are used.

[kinook]

Global subroutines may be an improvement. If you want to ZIP and send your .bld files to support@kinook.com, we can look at them and see if the conversion could be automated and/or provide ideas.

[Seeker]

Thanks Kinook. Great offer, but for now I'll give it a wack my self (hopefully learn a thing or two in the process). The project is still in the planning phase and first order of business is to document the suggested changes for management consideration.

Though I might run (some part of) the project proposal by you, some time in near future for suggestion and comments.

[mvermeulen]

Had not seen this before but seems to be along the lines of what I have implemented at several locations. The basic premise of this is to maintained versioned templates for the build process including configuration, steps, actions, etc...

The versions are stored in a central source code repository (choose your favorite... VSS, SVN, ETC...)

The build process or Visual Build Project file uses a standard format and will perform the following basic steps on every project across the network.

1. PreLoad Step - Sets the version information for the current VBP application, project paths, config file paths (config, scripts, macros, steps) relative to the current server.

2. Performs a checkout/get from source control for the versions of the files that are used by the current build process.

3. Reloads all of the VBP files based upon what was retrieved from source control. (See example below)

Application.Options.Load "%configfilepath%\VisBuildPro.config"
Application.Macros(vbldMacroGlobal).Load "%configfilepath%\VisBuildPro.macros", True
Application.Scripts(vbldScriptGlobal).Load "%configfilepath%\VisBuildPro.Global.scripts"
Application.Project.Steps(vbldStepGlobalSubroutine ).Load "%configfilepath%\VisBuildPro.steps"

4. Calls a global subroutine step that loads/set macros and processes any other steps that are used throughout every build process. Project specific data is stored in an xml file and loaded during this phase.

Up to this point, every project is exactly the same in the way that it processes information. Only what it loads may be different.

5. Main section of build project executed. This may actually differ from project to project.

6. Cleanup phase - The remainder of the steps are stored in the global subroutines section and therefore identical from project to project. This may upload project info to a database, produce a build report, send mail, handle any error conditions, etc...

The main objective here is to not have to maintain VBP specific info on any machine but have the ability to download anything that is needed to any machine (given a default operating system and tool set) and roll out changes on a global basis as needed.