A colleague came to me with a Delphi application that would not shut down, but just hang. The application in question had been refactored such that one module was extracted into a DLL to be reused in another application. When this extracted module was loaded into the application and the application was closed, it would hang. If the module was not loaded, the application would shut down normally.
Debugging the application was initially unsuccessful. Stepping through our code we verified that the shutdown logic executed normally with destructors running as expected. Interestingly, it was not possible to break into the application once it had become unresponsive. Trying to pause the hung program from within the IDE would simply cause the IDE to hang as well.
Thus we used Process Explorer instead to look at the application’s threads and their callstacks.
There we saw that there was one thread stuck on a call to WaitForSingleObject which originated in our DLL code. Higher up the callstack was ExitProcess. I looked at the documentation for ExitProcess to see look for ways in which it could deadlock. One sentence looked promising: “If one of the terminated threads in the process holds a lock and the DLL detach code in one of the loaded DLLs attempts to acquire the same lock, then calling ExitProcess results in a deadlock.” But since there was only one thread, this could not be it.
Looking next at what happens exactly inside ExitProcess, two other things jumped at me:
- All threads are terminated (except for the one calling ExitProcess).
- All DLLs are unloaded.
It turns out, the initial analysis that “the shutdown logic executed normally” was wrong. One of the shared units compiled into the DLL (through several layers of indirections), had a finalization section. In this finalization section, a background thread that had been created in the corresponding initialization section, was being destroyed. As part of the destructor code, the thread class was waiting for an event that was set when the thread had stopped executing.
This finalization section was running as part of the “all DLLs are unloaded” step by ExitProcess. Unfortunately, all threads (including the one created in the initialization) had already been terminated. I am not quite sure how that was accomplished, but it apparently circumvented the normal thread termination logic which set the event that the thread had stopped executing.
This is different for code in the main application, where finalization sections are run while the application is still in working order.
Instead of waiting for the thread to set its “stopped executing” event, I wait on the thread handle to check if the thread was even there to set the event. When run from the DLL’s finalization section, this detects the thread’s absence and just returns.