PDA

View Full Version : LPSTR out paramters in DLL call



MHolmes
07-07-2005, 04:05 PM
I have a small helper DLL I have written to wrap some advanced Win32 API calls, and one of them needs to pass a string back to InstallShield. I figured the best way to do this would be to take an LPTSTR parameter in the DLL function and _tcsncpy the returned string over to the string IS passes in...no go, I get stack corruption.

Here is the C prototype:

BOOL __declspec(dllexport) GetServicePath(LPTSTR name, LPTSTR path, DWORD pathSize)

Here is the IS prototype:

prototype BOOL ServiceHelper.GetServicePath(LPSTR, LPSTR, LONG);

And here is the string copy:

_tcsncpy(path, buf->lpBinaryPathName, pathSize);

In IS, I call it like so:

STRING name;
STRING path[512];

name = "Some Service";
GetServicePath(&name, &path, 512);

When my C function gets to the string copy operation, I get stack corruption. If this is not the regular way to pass back string from DLL calls, what is?

aviswanathan
07-07-2005, 06:21 PM
I am assuming you are using a Win32 dll (and not a COM or a .Net dll)

Try the following prototype declaration.

prototype BOOL ServiceHelper.GetServicePath(BYREF STRING, BYREF STRING, LONG);

Then call UseDll(<PATH TO DLL>)

Then call your function.
GetServicePath( name, path, pathSize);

Then call UnUseDLL (<just the dll name>);

Hope this works!

There are many posts in the forum with similar questions and you may go through them to get a better idea. Also see the examples for individual functions used above in the help (though not much useful).

MHolmes
07-08-2005, 09:12 AM
Yes, it's a Win32 DLL, no COM or .NET involved :)

I am still getting the stack corruption at the _tcsncpy() call though. Even when prototyping them as BYREF STRING.

I will see if I can find some other posts on the subject (though I looked pretty extensivley and couldn't find anything, before I posted this).

MHolmes
07-08-2005, 09:21 AM
Hmm, could it be because I am throwing an exception during the call? Will InstallShield detect that as a stack corruption because C++ needs to unwind?

aviswanathan
07-08-2005, 09:28 AM
Hi,

Since you are getting stack corruption, did you try calling your wrapper function via a command line executable (for testing). If that throws an error too then the problem is not your installshield call.

Also check if the path size is valid.

Arun

MHolmes
07-08-2005, 11:43 AM
Hi,

Since you are getting stack corruption, did you try calling your wrapper function via a command line executable (for testing). If that throws an error too then the problem is not your installshield call.

Also check if the path size is valid.

Arun

I did in fact do this, and it works correctly. I am thinking it's because I am using C++ exception handling, and considering that alters the stack when an exception is thrown/caught, InstallShield sees this as a stack corruption.

aviswanathan
07-08-2005, 01:33 PM
I did in fact do this, and it works correctly. I am thinking it's because I am using C++ exception handling, and considering that alters the stack when an exception is thrown/caught, InstallShield sees this as a stack corruption.

Hmmmmm.... Are you by any chance passing NULL strings to the C++ function call from InstallShield? Try setting the strings to some value and test it.

MHolmes
07-08-2005, 01:34 PM
Okay, it's not C++ exception handling. Again, this works perfectly when I call these functions from native C/C++. Either something bad is happening, or InstallShield thinks it is, when it isn't. It's very frustrating though, that something as simple as passing back string data from a C call, seems to be so hard to nail down in IS.

MHolmes
07-08-2005, 01:39 PM
Hmmmmm.... Are you by any chance passing NULL strings to the C++ function call from InstallShield? Try setting the strings to some value and test it.

Let me try this. According to the C++ function, the pointers are not NULL...but they may be uninitialized.

Update: This isn't it either. Gave the recieving string variable a value in my InstallScript, still says the stack is being corrupted.

MHolmes
07-08-2005, 01:52 PM
Got it! I wasn't marking the functions __stdcall (WINAPI), I was allowing them to stay __cdecl. InstallShield doesn't appreciate this, as it expects the callee to clean up the stack. Obviously with __cdecl, the caller has to clean it up.

Problem solved :)

aviswanathan
07-08-2005, 02:00 PM
Whew ... this was a long one :rolleyes:
Glad that you fixed it.