-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No feedback from debugger when program overflows stack #120
Comments
I haven't yet confirmed if we are able to detect the stack overflow in this case (VS also doesn't seem to, for example), but I did fix a serious bug that this program exposed, which may be related to some of the weird circumstances you were experiencing. I get similar results to other debuggers now, as long as I specify the Custom Entry Point for the target explicitly as |
I think I'm running into the same issue here, Here is my case
#include <windows.h>
struct structure {
int a;
};
void tt()
{
constexpr auto BUFFER_SIZE = 1024 * 1024 * 8;
char bigArray[BUFFER_SIZE] = {};
// This is handled by `ExceptionHandler`
// structure* struc = 0;
// struc->a = 3;
}
int RemoveCRTTest()
{
AddVectoredExceptionHandler(1, ExceptionHandler);
tt();
return 0;
} this is the generated code void tt(void) PROC ; tt
$LN3:
push rdi
sub rsp, 8388624 ; 00800010H ;; 00800000H for the array itself and 0x10H for BUFFER_SIZE variable
mov DWORD PTR BUFFER_SIZE$[rsp], 8388608 ; 00800000H
lea rax, QWORD PTR bigArray$[rsp]
mov rdi, rax
xor eax, eax
mov ecx, 8388608 ; 00800000H
rep stosb
add rsp, 8388624 ; 00800010H
pop rdi
ret 0
void tt(void) ENDP The debugger hangs on the SUB instruction that reserves stack space for the procedure, the weird thing is that it reports an access violation 0xc0000005 which I already can handle using the custom ExceptionHandler routine Note: I have to set the entry point name in the debugger myself to step into the program |
I don't know if there is any way that I can detect that this exception is caused by a stack overflow specifically (which seems like the same boat that Visual Studio is in), but as of 7496f3b, the debugger properly responds to this situation. This change is on |
I had an error in my program that caused the stack to overflow. When I tried stepping through my code in the RAD debugger, I would get strange behaviour, like it would start stepping through assembly instructions indefinitely, or it would stop stepping altogether, and the call stack panel would go blank. Sometimes I wouldn't be able to kill my program's process, so I'd have to quit and reopen the debugger. When I first noticed the issue, it seemed like the debugger was getting confused about where my WinMain entry point was.
This is in a non-CRT build, and I was mistakenly not setting a (large enough)
/STACK:,
size in the linker parameters.I've attached a minimal example of what I was doing.
I realize this is a niche and specific concern, and it's ultimately my fault for not setting the stack size. But if it's possible to offer any kind of error messaging that my program overflowed its stack, that would be awesome.
Thank you for great tool!
windows_main_bug.zip
The text was updated successfully, but these errors were encountered: