Allocation of more than 2GB in unreachable heap causes out of memory errror
Created by: SweetVishnya
How to reproduce
This reproduces on both Linux x86-32 and x86-64.
Just allocate more than 2GB from dr_client_main
with malloc
:
int *tmp = 0;
int cnt = 0;
for (int i = 0; i != 4 * 96086; ++i)
{
tmp = (int *) malloc(30000);
++cnt;
std::cout << cnt << std::endl;
for (int j = 0; j != 30000 / sizeof(int); ++j)
{
tmp[j] = i * 43;
}
}
std::cout << "Do not optimize out malloc " << tmp[20] << std::endl;
Run debug:
dynamorio/bin64/drrun -debug -c client.so -- /bin/ls
Get an error:
<Out of vmheap reservation - reserving 256KB. Falling back onto OS allocation>
<CURIOSITY : (0) && "Out of vmheap reservation" in file /home/vishnya/dynamorio/core/heap.c line 1614
<Out of memory - cannot reserve 256KB. Trying to recover.>
<Out of memory. Program aborted.>
<OOM Status: R 0x0000000000000001>
What goes wrong
When unreachable memory size reaches first 2GB it tries to ask more memory from system. But this code fails and leads to out of memory.