Displaying Dialog Boxes from Within a DLL or XLL
To display a Win32 dialog box using, for example, the Windows SDK function DialogBox, you must first obtain the full 32-bit instance and main window handles for Excel. For more information, see How to: Access Excel Instance and Main Window Handles.
Assuming your project contains the dialog box resource, you must take several steps to set the message-handling routine to that of the newly displayed dialog box and to restore the Excel message handling routine when the dialog box is closed. The example command fShowDialog in the Generic project demonstrates the use of the Windows functions to do this correctly.
You can also display dialog boxes using the C API without having to use Windows SDK functions. However, the dialog box capabilities of the C API are very limited compared with those of Windows, Visual Basic for Applications (VBA), or the Microsoft Foundation Classes (MFC). (For example, C API dialog boxes are always modal).