What is a Manifest?
If you're familiar with Microsoft's Visual Studio manifests and what they're for, you may just want to browse this Background section or skip it entirely and move down to the section titled The Visual Studio 2008 Twist
Once upon a time when a Microsoft operating system had to search for a DLL as part of running up an application's .exe file, it would follow a fairly simple search process and would look for a DLL with the name indicated. So if app1.exe indicated that it needed coolstuff.dll, then the operating system would search for coolstuff.dll.
This approach worked fine most of the time, but it could lead to a scenario like this:
- app1.exe is installed and brings with it version 1.0 of coolstuff.dll, which is a Microsoft system DLL that Microsoft allows to be redistributed with applications that use it.
- app2.exe is later installed and brings with it version 2.0 of coolstuff.dll.
- app1.exe now doesn't work because it can't use version 2.0 of coolstuff.dll.
- coolstuff.dll is backed off to version 1.0 so app1.exe will start working again.
- app2.exe now doesn't work because it can't use version 1.0 of coolstuff.dll.
Scenarios like this became known as "DLL Hell". Newer versions of DLLs like coolstuff.dll that broke existing software could be brought in with newer applications or with operating system upgrades.
To try to fix this issue, starting with Visual Studio 2005 Microsoft introduced the idea of manifests. In this sense a manifest is a short bit of XML that gets inserted at the beginning of a .exe or .dll file that is built by Visual Studio. The manifest contains information about DLLs upon which the containing .exe or .dll file is dependent. One of these pieces of information is a very precise version number that goes to four levels (major.minor.release.update or something similar).
Let's look at an example. We'll use the classic C "hello world" example:
If we build this code with Visual Studio 2005 (Professional Edition in my case) using the Release configuration of a project called HelloWorld, we'll get a .exe file called HelloWorld.exe in the release folder of the solution that contains the HelloWorld project. Note that if you have the Express edition of Visual Studio, your system may not be set up the same as what I've outlined below, and you may see different behavior.
But we also get something else. In the Release (note the capital R) folder of the HelloWorld project itself there is a file called HelloWorld.exe.intermediate.manifest. It's a text file, so we can look at it with a text editor. If we do, we'll see that it looks like this:
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
<assemblyIdentity type='win32' name='Microsoft.VC80.CRT' version='8.0.50727.4053'
processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b' />
The key piece is that line that starts with assemblyIdentity. An assembly in this case is a collection of DLLs that are grouped together under a common name. And that name is the name attribute of this tag. In this case it's Microsoft.VC80.CRT. VC80 means Visual C 8.0, which is the actual version number of Visual Studio 2005. CRT means the C run-time collection. The HelloWorld.c file contains a call to the printf() function, which is part of the C run-time library, so that means our little HelloWorld program has a dependency on the Microsoft C run-time.
Note that there's also a version attribute in this tag. This attribute specifies the exact version of the Microsoft C run-time assembly that Windows should look for when it activates our HelloWorld.exe image.
What's Done with this Manifest?
Before we go on, we should discuss a little bit about what's done with this manifest. The file that we see here isn't the end of the story. When we built HelloWorld.exe, one of the last lines printed to Visual Studio's Output window said "Embedding manifest...". Visual Studio takes the information in this .manifest file and embeds it into the HelloWorld.exe image.
There are a couple ways to examine the manifest that is embedded within a .exe or a .dll file produced by Visual Studio. One way is to start the Visual Studio IDE and choose File->Open->File... and then browse to the .exe or the .dll file we want to examine. Visual Studio will give us a tree-like presentation of the beginning of the .exe or .dll file, and this will include a node called RT_MANIFEST. If we expand that node, the next node just has a number (1 for a .exe or 2 for a .dll). If we right click on this node and choose either Open or Open Binary Data from the context menu, we'll see a view of the manifest that has hexadecimal digits on the left and ASCII characters on the right. It's a little hard to look at, but we can make out the contents without too much difficulty.
A second way to examine an embedded manifest is to use the mt command (mt stands for manifest tool). This command is specific to Visual Studio, and it isn't available from an ordinary command window. We must open up a Visual Studio command window to be able to use it. So for Visual Studio 2005 we would choose Start->All Programs->Microsoft Visual Studio 2005->Visual Studio Tools->Visual Studio 2005 Command prompt. This will open a command window that looks very much like an ordinary command window, but there are some PATH definitions pre-configured so programs specific to Visual Studio 2005 can be seen.
To view the manifest that's embedded within our HelloWorld.exe program using the mt command, we can extract it to a file by saying something like this:
mt -inputresource:HelloWorld.exe;#1 -out:HelloWorld.manifest
The #1 would become #2 if the file we were extracting the manifest from were a .dll instead of a .exe. The mt command can also be used to insert manifests into .exe or .dll files; this is essentially what Visual Studio does once it completes the link step. And that points to a concept that's worth noting. If we build software using makefiles that invoke either cl.exe or link.exe to create the .exe or .dll file, these commands will not embed the manifest. The .manifest file will get created, but it will not get embedded into the binary unless we explicitly invoke the mt.exe command after the link step is completed. But if we use the msbuild command to process a Visual Studio .sln or .vcproj (.vcxproj for Visual Studio 2010) file, then the manifest will get embedded, just like it does if we use the Visual Studio IDE.
So what will happen if we use cl to build this HelloWorld.exe file and forget to embed the manifest? Well, most likely if the C run-time DLL that the .exe depends upon isn't in the same folder as the .exe file or somewhere else in the PATH, we'll get an error message when we try to run HelloWorld saying that particular DLL can't be found. If we find it and copy it to the same folder as the .exe file, or if it's already somewhere where it can be found, then when we try to run the .exe, we'll get an error message saying the application is trying to invoke the C run-time incorrectly. The message doesn't say why, and in fact it doesn't really give us any clue that the problem is actually that the .exe doesn't have a manifest.
Where is the Software that the Manifest Indicates?
So we have a manifest embedded within HelloWorld.exe that specifies the exact version of the C run-time assembly that's needed. So what exactly is in this C run-time assembly? We can find out by looking at our Visual Studio 2005 install folders. Usually Visual Studio 2005 gets installed into a folder named Microsoft Visual Studio 8, and that folder in turn would be placed into either Program Files or Program Files (x86), depending on whether we have the 32-bit or the 64-bit flavor of Windows installed.
Under this Microsoft Visual Studio 8 folder there's a folder called VC, which contains files specific to the Visual C and C++ portions of Visual Studio. Under the VC folder there's a folder called redist, which contains files that Microsoft allows to be redistributed with software that uses them. Under the redist folder there's a folder called x86, which has 32-bit files. And under this x86 folder there's a folder called Microsoft.VC80.CRT. Now we're getting somewhere. So if you've been following along, we're now likely in folder C:Program Files (x86)Microsoft Visual Studio 8VCredistx86.
If we look in this Microsoft.VC80.CRT folder, we'll see three .dll files and another .manifest file. This .manifest file is slightly different from the one created for HelloWorld.exe. Instead of describing DLLs upon which the C run-time is dependent, it instead describes the DLLs that constitute the C run-time itself. There are three of them, the three DLLs that are in this folder: msvcm80.dll, msvcp80.dll, and msvcr80.dll. Together these three DLLs form the C run-time for Visual Studio 2005; aka Visual Studio 8.0.
We can also see that the tag for the C run-time assembly specifies a version attribute. The version specified in this attribute is the same as what was specified in the manifest created for HelloWorld.exe.
So we now know that HelloWorld.exe contains a manifest that specifies the exact version of the C run-time that it needs. So it seems that the DLL Hell problem is solved, right? Well, maybe.
Keep in mind that the place where we looked to find the files that constitute the C run-time assembly was part of the Visual Studio 2005 hierarchy. That's just where Microsoft locates them as part of the product. That's not where Windows looks for them when it activates HelloWorld.exe. Indeed HelloWorld.exe may be run on a system that doesn't have Visual Studio 2005 installed.
In order for HelloWorld.exe to run successfully, the version of the C run-time that its manifest specifies must be present on the system. Windows includes a large collection of multiple versions of system DLLs. On my Windows 7 workstation, this collection is in the folder c:windowswinsxs. The letters sxs mean side by side, which is Microsoft's name for this manifest and DLL mechanism. On my workstation the winsxs folder itself contains 11,083 subfolders. The folder names are cryptic, but they do contain enough recognizable information that we can figure out what they contain.
For example, when I build the HelloWorld.exe file on my workstation, the manifest specifies version 8.0.50727.4053 of the C run-time. In my winsxs folder there is a subfolder named x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4053_none_d08d7da0442a985d. Like I said, the names are cryptic. But we can see that it starts with x86, meaning it contains 32-bit files instead of 64-bit files (in which case it would start with amd64). We can also see the name microsoft.vc80.crt, which is the name of the assembly. And we can see the version number, 8.0.50727.4053.
If we look in this folder, we see the three DLLs that comprise the C run-time.
So as long as the version of an assembly or DLL that's specified in a manifest is present on the system, the program will run. But can we always be guaranteed that the version specified in the manifest will be there? Not necessarily. Most versions of Windows, particularly the newer ones like Vista and Windows 7, do have enough different versions of important system assemblies like the C run-time so that programs that need them will work.
But we can't be absolutely certain that this will be the case, particularly with Windows XP. What influences whether a specific version of the C run-time or other assemblies like it (e.g., Microsoft Common Controls) is present on a given system is anybody's guess. A specific version of an assembly might get put down when a piece of software that uses it is installed, or it might get put down with an update to Windows. Then again, removing a piece of software that needs that version of the assembly might remove the assembly itself, particularly if that software is the only software that used it.
And unfortunately, the error messages that are displayed when a piece of software specified in a manifest isn't present usually give us no clue as to what the problem is. For example, one scenario produces an error window that says the application is not configured correctly and that re-installing it might fix the problem. Rubbish! The only way to fix the problem is to install the software that is missing (or correct the manifest, see The Visual Studio 2008 Twist below). But the error messages produced in these situations tend to be so generic that they don't even tell us that the problem is missing software, let alone what software is actually missing.
The way to avoid this unpleasantness is to distribute the needed Microsoft assemblies with our software. In the case of the C run-time we can distribute the three DLLs and the Microsoft.VC80.CRT.manifest file with our application, in the same folder as the application's .exe file. This will ensure that Windows will find the version of the C run-time that the manifest embedded in our .exe file specifies.