Memory leak while reading frames from preloaded video buffer
Created by: igor-kondratiev
This leak can easily be reproduced with this script
import io
import av
ITERATIONS = 1000
INPUT_SAMPLE_PATH = "/path/to/any/mp4"
def main():
with open(INPUT_SAMPLE_PATH, "rb") as f:
input_data = f.read()
for i in range(ITERATIONS):
print(f"Iteration {i + 1} started")
buffer = io.BytesIO(input_data)
container = av.open(buffer)
frame = next(container.decode(video=0))
input("Press enter to exit...")
if __name__ == '__main__':
main()
Increasing number of iterations will cause memory usage constant rising. I'm using version 0.3.3 but 0.4.0 is affected as well. Python version: 3.6.5
I've made some investigations and found actual leaking place. The problem is ContainerProxy::__dealloc__
method. In case of python file-like buffer there will be two extra objects allocated - self.iocontext
and self.buffer
(spoiler: they are both leaking). Next code - self.ptr.pb = self.iocontext
will force ffmpeg to set AVFMT_FLAG_CUSTOM_IO
(this happens here). Another important thing is that avformat_close_input
will not free buffer and iocontext if AVFMT_FLAG_CUSTOM_IO
flag is set - avformat_close_input code. As a result, both self.iocontext
and self.iocontext.buffer
will not be deallocated.
You could also use valgrind with provided script to verify described behavior. Important note is that valgrind will not point to the buffer that is being allocated in ContainerProxy::__init__
. libAV
can reallocate provided buffer for some reasons, so valgrind will show only the last reallocation call-stack.
I've made a fix for version 0.3.3 (here is link to commit) but there is no release branch for this version and I can't create a PR. Are there any chances to release a patch for version 0.3.3?
The code in ContainerProxy::__dealloc__
was changed in current develop version, but it will not be difficult to adapt this fix for current version. Do you have any plans about dates for the next PyAV release where this fix could be included?
Thank you in advance!