PDA

View Full Version : Debugging a custom action DLL with MS Studio .NET 2003



installerGuy
08-22-2005, 09:30 PM
Hi everyone,

I have written a standard DLL that I have added as a support file to a Basic MSI project. I tried to call a function both from installscript code (Using UseDLL and UnuseDLL) and from a custom action calling into the DLL directly. Both approaches work fine, and I am getting the correct behavior.

Now, I am trying to debug the C++ code inside my standard DLL, but VS .NET 2003 does not seem to load the DLL at runtime. I have tried various settings, but I still cannot put breakpoints in my C++ code and debug the DLL source code.

Does anybody know how to get this to work?

Best regards,

furrymcmonst
08-24-2005, 05:45 PM
I'm not really a hard core developer so there may be a better way than what I'm about to suggest.

What I do is put a message box at the start of my custom action. When it pops up, I choose "Debug", "Processes" and attach to the process. Once it is attached, you can set your breakpoints and then dismiss the dialog.

installerGuy
08-24-2005, 07:29 PM
Hi,

Thanks a lot for your answer. Actually it got me started, and I figured out a bit more after that. Seems like there is a process called IDriver.exe that is spawned by setup.exe. Using MSDEV, you can attach to IDriver.exe and you will be able to start debugging.

In case you have custom code that executes very early in the install process, you can also overwrite the OnBegin() function in InstallScript, and put an InstallScript breakpoint there. Once you hit the breakpoint, bring up MSDEV, and attach to IDriver.exe (it is already started at this time). This way you will be guaranteed to attach before your custom code fires.

Thanks a lot!

aviswanathan
08-25-2005, 10:32 AM
An easier approach would be to use:


#ifdef _DEBUG
DebugBreak();
#endif
Add this code at the beginning of the MSI function, so that it breaks automatically. You can choose your debugger when it breaks.

- Arun

marc.gibian
10-14-2005, 05:15 PM
I just tried this, and instead of dropping into the debugger or giving me a choice of actions, I just get the following dialog:

InstallShield DLL Custom Action

! The DLL being called encountered a problem.

(OK)

Well, of course it encountered a problem... I want to drop into the debugger. But, for some reason it did not?