Fix zlib + lzma memory usage inefficiency
authorNick Terrell <terrelln@fb.com>
Wed, 12 Apr 2017 19:44:56 +0000 (12:44 -0700)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Wed, 12 Apr 2017 19:49:59 +0000 (12:49 -0700)
Summary:
Since IOBuf rounds to a good malloc size, zlib and lzma waste space.
For a 4 MiB `length`, about 1 MiB of the IOBuf is wasted.

Reviewed By: yfeldblum

Differential Revision: D4872830

fbshipit-source-id: 42fc83277b2dae22b75403e0e71c43e8f2b19f21

folly/io/Compression.cpp

index ae75b2a..4a9deaa 100644 (file)
@@ -840,7 +840,7 @@ std::unique_ptr<IOBuf> ZlibCodec::addOutputBuffer(z_stream* stream,
   CHECK_EQ(stream->avail_out, 0);
 
   auto buf = IOBuf::create(length);
-  buf->append(length);
+  buf->append(buf->capacity());
 
   stream->next_out = buf->writableData();
   stream->avail_out = buf->length();
@@ -1147,7 +1147,7 @@ std::unique_ptr<IOBuf> LZMA2Codec::addOutputBuffer(
   CHECK_EQ(stream->avail_out, 0);
 
   auto buf = IOBuf::create(length);
-  buf->append(length);
+  buf->append(buf->capacity());
 
   stream->next_out = buf->writableData();
   stream->avail_out = buf->length();