.NET Framework

How Windows CRT Checks For Managed Module

While debugging an issue I came across an interesting piece of code in Windows CRT source. So far I have been using the code below to determine if an executable is managed or native. Basically I am just checking for the presence of IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR Data Directory entry in PE headers.

 
int CheckCLRHeader(PBYTE pbFile)
{
    PIMAGE_DOS_HEADER pDOSHeader;
    PIMAGE_NT_HEADERS pNTHeader;
    PIMAGE_OPTIONAL_HEADER32 pNTHeader32;
    PIMAGE_OPTIONAL_HEADER64 pNTHeader64;
    PIMAGE_DATA_DIRECTORY pDataDirectory;
 
	if(NULL == pbFile)
	{
		cout << "Invalid file" << endl;
		return 0;
	}
 
	pDOSHeader = reinterpret_cast
< PIMAGE_DOS_HEADER >(pbFile);
	if(IMAGE_DOS_SIGNATURE != pDOSHeader->e_magic)
	{
		cout << "Cannot find DOS header. Not an executable file" << endl;
		return 0;
	}
 
	pNTHeader = ImageNtHeader(pbFile);
	if(NULL == pNTHeader)
	{
		cout << "Cannot find PE header. Invalid or corrupt file" << endl;
		return 0;
	}
 
	if(IMAGE_NT_SIGNATURE != pNTHeader->Signature)
	{
		cout << "Cannot fine PE signature. Invalid or corrupt file" << endl;
		return 0;
	}
 
	switch(pNTHeader->OptionalHeader.Magic)
	{
		case IMAGE_NT_OPTIONAL_HDR32_MAGIC:
			pNTHeader32 = reinterpret_cast
< PIMAGE_OPTIONAL_HEADER32 >(&pNTHeader->OptionalHeader);
			if(pNTHeader32->NumberOfRvaAndSizes <= IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR)
			{
				cout << "No CLR Data Dictionary. Not a Managed Assembly" << endl;
				return 0;
			}
			pDataDirectory = pNTHeader32->DataDirectory;
			break;
 
		case IMAGE_NT_OPTIONAL_HDR64_MAGIC:
			pNTHeader64 = reinterpret_cast
< PIMAGE_OPTIONAL_HEADER64 >(&pNTHeader->OptionalHeader);
			if(pNTHeader64->NumberOfRvaAndSizes <= IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR)
			{
				cout << "No CLR Data Dictionary. Not a Managed Assembly" << endl;
				return 0;
			}
			pDataDirectory = pNTHeader64->DataDirectory;
			break;
 
		default:
			cout << "Invalid NT header. Invalid or corrupt file" << endl;
			return 0;
	}
 
	if(NULL == pDataDirectory)
	{
		cout << "Cannot find data directories. Invalid or corrupt file" << endl;
		return 0;
	}
 
	if(0 == pDataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR].VirtualAddress ||
		0 == pDataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR].Size)
	{
		cout << "COM Data Directory not found. Not a Managed Assembly" << endl;
		return 0;
	}
 
	cout << "Managed Assembly" << endl;
 
	return 1;
}
 

The code for check_managed_app function in crtexe.c is similar but slightly different in how it checks for the same IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR but only checks for presence of VirtualAddress.

 
/***
*check_managed_app() - Check for a managed executable
*
*Purpose:
*       Determine if the EXE the startup code is linked into is a managed app
*       by looking for the COM Runtime Descriptor in the Image Data Directory
*       of the PE or PE+ header.
*
*Entry:
*       None
*
*Exit:
*       1 if managed app, 0 if not.
*
*Exceptions:
*
*******************************************************************************/
 
static int __cdecl check_managed_app (
        void
        )
{
        PIMAGE_DOS_HEADER pDOSHeader;
        PIMAGE_NT_HEADERS pPEHeader;
        PIMAGE_OPTIONAL_HEADER32 pNTHeader32;
        PIMAGE_OPTIONAL_HEADER64 pNTHeader64;
 
        pDOSHeader = (PIMAGE_DOS_HEADER)&amp;__ImageBase;
        if ( pDOSHeader->e_magic != IMAGE_DOS_SIGNATURE )
            return 0;
 
        pPEHeader = (PIMAGE_NT_HEADERS)((char *)pDOSHeader +
                                        pDOSHeader->e_lfanew);
        if ( pPEHeader->Signature != IMAGE_NT_SIGNATURE )
            return 0;
 
        pNTHeader32 = (PIMAGE_OPTIONAL_HEADER32)&pPEHeader->OptionalHeader;
        switch ( pNTHeader32->Magic ) {
        case IMAGE_NT_OPTIONAL_HDR32_MAGIC:
            /* PE header */
            /* prefast assumes we are overflowing __ImageBase */
#pragma warning(push)
#pragma warning(disable:26000)
            if ( pNTHeader32->NumberOfRvaAndSizes <=
                    IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR )
                return 0;
#pragma warning(pop)
            return !! pNTHeader32 ->
                      DataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR] .
                      VirtualAddress;
        case IMAGE_NT_OPTIONAL_HDR64_MAGIC:
            /* PE+ header */
            pNTHeader64 = (PIMAGE_OPTIONAL_HEADER64)pNTHeader32;
            if ( pNTHeader64->NumberOfRvaAndSizes <=
                    IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR )
                return 0;
            return !! pNTHeader64 ->
                      DataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR] .
                      VirtualAddress;
        }
 
        /* Not PE or PE+, so not managed */
        return 0;
}
 

I came across this code while debugging past the main() function. I thought this might be a new piece of code in CRT source that comes with VS2008 only to be proven wrong when searching through the CRT source that comes with all versions of Visual Studio starting from VS2002. Another interesting discovery (at least for me) is that similar code is in various CRT source files. In VS2008 CRT source the files crt0dat.c, crt0.c and crtexe.c contain check_managed_app function in Win32 CRT and crt0dat.c in WinCE CRT. In VS2005 the same function is in crt0.c, crt0dat.c and crtexe.c. In VS2003 and VS2002 it is in crt0.c and crtexe.c. There are other gems of code in CRT source waiting to be discovered!

.NET Framework
VS2005
VS2008

Comments (0)

Permalink

How to determine event subscribers

While debugging a reasonably complex .NET application, many times you may come across a situation where you want to know which different methods have subscribed to your event. If you are using VS2005 or VS2008 then DebuggerTypeProxyAttribute, a relatively unknown feature of Visual Studio, comes to your rescue. For a long time Visual Studio has allowed developers to customize the information about the variable that is displayed in the IDE during debugging. Before VS2005 the customization was done using mcee_cs.dat (C#), mcee_mc.dat (MC++), and autoexp.dat (C++) files. Starting with VS2005, this customization is taken to next level. VS2005 introduces two new attributes, DebuggerDisplayAttribute and DebuggerTypeProxyAttribute, that allows you to change the information displayed in the Visual Studio debugger. Typically one would apply these attributes to the class in the code which the VS debugger will use to display information. However, there is a trick in which you can use these attributes outside of the original code. This means that you can change the information displayed about ANY class, even the classes that you did not write yourself. VS2005 and above use AutoExp.dll which provides display information about various types to the debugger. The code for AutoExp.dll is provided in AutoExp.cs file with the Visual Studio installation in Common7\Packages\Debugger\Visualizers\Original\ folder. You can add your own code to this file, compile the AutoExp.dll, and place it in either the Common7\Packages\Debugger\Visualizers folder or in the My Documents\Visual Studio 2005 (or Visual Studio 2008)\Visualizers folder. I learned about this trick a while back from John Robbins' excellent book Debugging .NET 2.0 Applications. It turns out that Visual Studio loads all the valid assemblies from the Visualizers folder, and if the assembly contains Visualizer or any debugger related objects then it will use it during the debugging session. Now back to the topic of this post; to easily display which methods have subscribed to an event I wrote a class called MulticastDelegateDebuggerProxy for MulticastDelegate class. Since, all the events are delegates that are derived from MulticastDelegate class, my MulticastDelegateDebuggerProxy class will be used for all the events. The DebuggerTypeProxyAttribute is used at the assembly level to tell Visual Studio debugger that MulticastDelegateDebuggerProxy class is the proxy for MulticastDelegate class. The code for MulticastDelegateDebuggerProxy is listed below.

 
using System;
using System.Diagnostics;
using System.Reflection;
using System.Text;
 
[assembly: DebuggerTypeProxy(typeof(MulticastDelegateDebuggerProxy),
		Target = typeof(MulticastDelegate))]
 
public class MulticastDelegateDebuggerProxy
{
	private string[] m_methods;
 
	public MulticastDelegateDebuggerProxy(System.MulticastDelegate del)
	{
		m_methods = GetMethodList(del);
	}
 
	[DebuggerDisplay(@"Subscribers:\{{Methods.Length}}")]
	public string[] Methods
	{
		get { return m_methods; }
	}
	private string[] GetMethodList(System.MulticastDelegate myEvent)
	{
		string retType = myEvent.Method.ReturnType.Name;
 
		Delegate[] delegates = myEvent.GetInvocationList();
		string[] methods = new string[delegates.Length];
 
		for (int i = 0; i &lt; delegates.Length; ++i)
		{
			Delegate del = delegates[i];
			MethodInfo m = del.Method;
			string accessModifier =
				m.IsPrivate ? "private" : (m.IsPublic ? "public" : (m.IsFamily ? "protected" : ""));
			StringBuilder sb = new StringBuilder();
			foreach (ParameterInfo param in m.GetParameters())
			{
				sb.Append(param.ParameterType.FullName).Append(" ").Append(param.Name).Append(",");
			}
			if (sb.Length &gt; 0)
			{
				sb.Remove(sb.Length - 1, 1);
			}
			string isStatic = m.IsStatic ? "static" : string.Empty;
			string isVirtual = m.IsVirtual ?
				(((m.Attributes &amp; MethodAttributes.NewSlot) == MethodAttributes.ReuseSlot)
				? "override" : "virtual") : string.Empty;
			methods[i] = string.Format("{0} {1} {2} {3} {4}.{5}({6})",
				accessModifier, isStatic, isVirtual, retType, m.ReflectedType.FullName, m.Name, sb.ToString());
		}
 
		return methods;
	}
}

The information displayed in VS2005 by default is shown below.
Default Event information in VS2005

The information displayed in VS2005 with MulticastDelegateDebuggerProxy is shown below. The MulticastDelegateDebuggerProxy displays fully qualified method name and the parameter names used in the method.

Event information in VS2005 with MulticastDelegateDebuggerProxy
In VS2008, the default information displayed is slightly better than VS2005 as it shows _invocationList by default which can be drilled down to the actual methods. However, the default display can get ambiguous as shown in the screenshot below. The MulticastDelegateDebuggerProxy does a much better job at displaying the same information.

Default event information in VS2008

.NET Framework
Debugging
VS2005
VS2008

Comments (0)

Permalink

Learning .NET debugging using WinDBG

If you wanted to learn advanced debugging of .NET application using WinDBG but didn't know where to start, look no further. Tess Ferrandez has started posting debugging labs to teach you how to debug tough problems. Her blog has invaluable tips about debugging real world problems. She has posted real world case studies and explained how to approach and solve the problems. In debugging labs, she posts a buggy application with instructions for troubleshooting the problem. A couple of days later, she posts a review of the lab where she walks you through troubleshooting the problem in great detail. So far she has posted 3 debugging labs (one, two, three) and 2 lab reviews (one, two). She will be posting a total of 10 labs. These labs are a great learning tool and best of all, they are all free! Thank you Tess!

.NET Framework
Debugging

Comments (0)

Permalink

DEP, NXCOMPAT Redux

I recently wrote about the impact of DEP on .NET applications due to changes in the .NET Framework 2.0 SP1. A couple of days later I came across Michael Howard's post about new APIs related to DEP. As Michael points out, in Windows Vista SP1, Windows XP SP3 and Windows Server 2008, it is possible for developers to enable DEP for an application at runtime instead of using a linker switch or being forced into setting NXCOMPAT bit by the C# compiler. The new function SetProcessDEPPolicy, allows the running process to change the DEP settings for the process calling it. I think this is a much more flexible option since it puts application developers in control of using DEP for their applications. Michael also explains the options available in SetProcessDEPPolicy API which allows the application that is using the old ATL code to enable DEP, and not throw the DEP exception at runtime. Raymond Chen has written about how in Windows XP, DEP will not throw an exception for what he calls "well-known DEP-voilating thunks". The new DEP API brings this feature to WinXP SP3, Vista, and Windows Server 2008. Now if only Microsoft would release a hotfix for the C# compiler to provide a /NXCOMPAT switch, the .NET development world would be a much better place.

.NET Framework

Comments (0)

Permalink

DEP, NXCOMPAT and changes in .NET Framework 2.0 SP1

This topic has been written about but not as much as it should have been. Hence, I am repeating it here again. It will also serve as a future reference for me.

DEP stands for Data Execution Prevention. It is a hardware based feature supported by Intel as well as AMD processors that disables execution of code that resides in memory areas marked as data pages. On AMD processors, this feature is implemented as NX (No eXecute) bit, and on Intel processors it is implemented as XD (eXecute Disable) bit. An OS can expose this feature and enable the application to take advantage of this protection. Once enabled, it becomes very difficult to inject and run malicious code using popular exploits such as buffer overflow which injects code on stack or heap memory and branches the instruction pointer (EIP) to execute the code. Microsoft has made this feature available in its operating systems starting with the Windows XP Service Pack 2. This feature can be enabled either system wide, or per application in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003. In Windows Vista this feature is enabled by default for all the system applications. Vista also has an Opt-out feature that allows you to enable DEP for all the (system as well as user) applications except those on the opt-out list. Robert Hensing explains this is great detail in his post. For VC++ developers there is one more way to take advantage of DEP. Visual Studio 2003 and above provide a linker switch called /NXCOMPAT. This switch sets a bit in the executable that indicates the OS loader that the application is DEP enabled. However, there is one catch. Setting this bit in the executable overrides all the other DEP settings - meaning even if DEP is disabled system wide or the application is opted out of DEP, the OS will still enable DEP for that application!!! This is where the problem starts for applications compiled using .NET 2.0 SP1. The C# compiler was updated to set this bit in the assemblies that it creates, and there is no way to unset it as in VC++ linker. If your application does not use native code, then you have nothing to worry about. If your application uses native then you may be in for a rude shock; your application might suddenly crash with IP_ON_HEAP errors. The old ATL framework generated code on the fly that ran on the stack or heap. If your .NET application uses a component or an ActiveX control that was built using the old ATL framework then your application will fail. The option is to recompile your native code using VS2003 or above. If this is a third party component that the vendor has not updated then you may be out of luck. Don't panic however! There is a ray of hope. You can disable the NXCOMPAT bit set by the C# compiler in the executable using the EditBin utility that is a part of Visual Studio installation. If your assembly is strongly named then you will have to resign it using SN utility. I strongly believe that Microsoft made a horrible mistake by deciding to set this bit by default on all the assemblies. It would have been much better if they had provided a switch just like the C++ linker. This new C# compiler feature is not well publicized by Microsoft. All the .NET developers that I have spoken to so far do not even know what DEP or NXCOMPAT is. Following the pure managed code mantra, most .NET developers do not care about native code any more even if they are using native code in their application. There is still lot of native code out there in the enterprises that is incompatible with DEP. There are lot of .NET applications that use such native code. This creates quite a problem when unknowing developers are faced with a situation when their application(s) suddenly stops working by just recompiling it in .NET 2.0 SP1. There are also other breaking changes as well as new properties, methods and Types added in SP1 as mentioned in a post by Scott Hanselman. This situation is made even worse by the release of Visual Studio 2008 as it installs .NET FX 2.0 SP1. Developers are keen to take advantage of the new features in C# 3.0 in .NET 2.0 applications using VS2008's multi-targeting feature. In an enterprise application development scenario, it is quite possible that developers will have .NET 2.0 SP1 on their development and test machines due to VS2008, but the user desktops will not have SP1. If the application uses new methods or types then it will fail on user desktops. Although one might be inclined to blame developers for developing and testing on environment that differs from the user's environment, such a scenario is highly likely to occur since the changes in .NET 2.0 SP1 are not well known. I have been able to find a list of bug fixes in SP1, but there is no knowledge base article that lists the differences between .NET 2.0 and SP1 other than the few scattered blog entries. It is a very bad idea to make such significant changes in a service pack level release. This is however a reality. . . so developers beware!

.NET Framework

Comments (7)

Permalink

Understanding VS2008 Multi-Targeting

Visual Studio 2008 has a new feature called Multi-Targeting which allows developers to create applications that use different versions of .NET Framework using the same IDE. This feature has caused quite a bit of confusion among developers. Let me explain what the reason is for this confusion. Up until .NET Framework 2.0 the version of the framework was synonymous to the version of CLR. In fact there was no distinction between the two. Meaning if you were developing for .NET Framework 1.0, then you are targeting CLR 1.0. The same is true for versions 1.1 and 2.0. Starting with version 3.0, Microsoft broke the relationship between the .NET FX version and the CLR version. To put it simply, Microsoft is now considering .NET FX and CLR as two separate entities. Version 3.0 of .NET FX contains new libraries that support new technologies such as WPF, WCF and WF. However, these features were built on top of the existing .NET FX 2.0. However, no changes where made to the CLR, which still remains version 2.0. In reality, the FX 3.0 is actually .NET FX/CLR 2.0 + new libraries. Why Microsoft branded the new set of libraries as a new .NET FX version is beyond me. The marketing groks at Microsoft apparently felt it was the right thing to do. Additionally, the .NET FX 3.5 takes this idiosyncrasy to the next level. The .NET FX 3.5 contains new features such as LINQ as well as some enhancements to the C# language. The C# language gets the new version 3.0. The new features of C# language such as Object Initializers, Collection Initializers, Anonymous types, Implicit Type Inference, and Automatic Properties are compiler features and require no additional supporting libraries. However, lambda expressions and LINQ which are also considered C# language features, do need supporting libraries which are part of .NET FX 3.5. The CLR version for .NET Framework 3.5 is still 2.o. The end result being, .NET FX 3.5 = .NET FX/CLR 2.0 + FX 3.0 (WPF, WCF, WF) + C# 3.0 and LINQ. The relationship between the various versions of .NET Framework, CLR, and Visual Studio is nicely explained by David Broman in his post, so I won't repeat it here. Though, I would like to make one point clear and that is whether you are using .NET FX 2.0, 3.0, or 3.5, you are still targeting the CLR version 2.0. In an interview in the February 2008 issue of DDJ, Microsoft Vice President of Developer Division  S. Somasegar mentions that one of the biggest feature requests that Microsoft received from its customers was the ability to target the older version of .NET Framework using the newer version of Visual Studio. He goes on to mention that using VS2008 you can build applications targeting .NET FX 2.0, 3.0, and 3.5. While this is true, Microsoft is missing the point. While Visual Studio 2005 was in beta (and even after its release), when customers asked for the feature for targeting the older version of the framework, they actually meant CLR 1.0, 1.1, and 2.0. What customers were asking for and what they actually received in VS2008 are two different things. The VS2008 Multi-Targeting feature is quite misleading IMHO. Whether you are using FX 2.0, 3.0, or 3.5, you are still using CLR 2.0. What multi-targeting really does is just provide an easy way to reference different sets of libraries while still using the same old CLR. Since Microsoft changed the meaning of the .NET Framework version and made the distinction between CLR version and the Framework version, I guess they are correct when they say VS2008 supports multiple .NET FX. However, Microsoft also has changed the meaning of the request that the customers originally made and supplied something quite different.  Additionally, they created quite a bit of confusion.

Note: I understand and I am not denying that .NET FX 3.5 is much more than C# 3.0 and LINQ. I am specifically discussing the VS2008 feature, its relationship with FX and CLR version, and the confusion that it has caused.

.NET Framework
VS2008

Comments (2)

Permalink

.NET Framework Source Released

For those living their lives in Reflector this is very welcome news. Microsoft has released source code of .NET Framework. Shawn Burke has detailed instructions on how to configure VS2008 to use symbols and be able to download source on the fly while debugging. Lot of people have blogged about this. However, some may still have problems making this work. The first time I tried, I was not able to download the source either. I had experimented a lot with different symbol server settings before the source was actually released. Due to this I already had symbols in my local folder which is the reason why the new symbols were not getting downloaded and hence, the source was not getting downloaded. I deleted the local symbol cache and lo and behold, everything started working. Turns out, this was already mentioned in Shawn's post. The key is to read the instructions properly and follow them. Hence I am posting key points below:

  1. Enable Symbol Server Support.
  2. Disable Just My Code.
  3. Configure Symbol URL correctly and provide local symbol folder.
  4. If already using Symbol server then delete the local symbol cache -this caused me some trouble for a bit.
  5. If the assembly shows grayed out in call stack window then right click and select Load Symbols.

John Robbins has also posted some additional tricks.

Happy Coding!

.NET Framework
VS2008

Comments (0)

Permalink