2 * Copyright 2013 Facebook, Inc.
4 * Licensed under the Apache License, Version 2.0 (the "License");
5 * you may not use this file except in compliance with the License.
6 * You may obtain a copy of the License at
8 * http://www.apache.org/licenses/LICENSE-2.0
10 * Unless required by applicable law or agreed to in writing, software
11 * distributed under the License is distributed on an "AS IS" BASIS,
12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 * See the License for the specific language governing permissions and
14 * limitations under the License.
17 #ifndef FOLLY_IO_IOBUF_H_
18 #define FOLLY_IO_IOBUF_H_
20 #include <glog/logging.h>
29 #include <type_traits>
31 #include <boost/iterator/iterator_facade.hpp>
33 #include "folly/FBString.h"
34 #include "folly/Range.h"
35 #include "folly/FBVector.h"
40 * An IOBuf is a pointer to a buffer of data.
42 * IOBuf objects are intended to be used primarily for networking code, and are
43 * modelled somewhat after FreeBSD's mbuf data structure, and Linux's sk_buff
46 * IOBuf objects facilitate zero-copy network programming, by allowing multiple
47 * IOBuf objects to point to the same underlying buffer of data, using a
48 * reference count to track when the buffer is no longer needed and can be
55 * The IOBuf itself is a small object containing a pointer to the buffer and
56 * information about which segment of the buffer contains valid data.
58 * The data layout looks like this:
66 * +------------+--------------------+-----------+
67 * | headroom | data | tailroom |
68 * +------------+--------------------+-----------+
70 * buffer() data() tail() bufferEnd()
72 * The length() method returns the length of the valid data; capacity()
73 * returns the entire capacity of the buffer (from buffer() to bufferEnd()).
74 * The headroom() and tailroom() methods return the amount of unused capacity
75 * available before and after the data.
81 * The buffer itself is reference counted, and multiple IOBuf objects may point
82 * to the same buffer. Each IOBuf may point to a different section of valid
83 * data within the underlying buffer. For example, if multiple protocol
84 * requests are read from the network into a single buffer, a separate IOBuf
85 * may be created for each request, all sharing the same underlying buffer.
87 * In other words, when multiple IOBufs share the same underlying buffer, the
88 * data() and tail() methods on each IOBuf may point to a different segment of
89 * the data. However, the buffer() and bufferEnd() methods will point to the
90 * same location for all IOBufs sharing the same underlying buffer.
92 * +-----------+ +---------+
93 * | IOBuf 1 | | IOBuf 2 |
94 * +-----------+ +---------+
96 * data | tail |/ data | tail
98 * +-------------------------------------+
100 * +-------------------------------------+
102 * If you only read data from an IOBuf, you don't need to worry about other
103 * IOBuf objects possibly sharing the same underlying buffer. However, if you
104 * ever write to the buffer you need to first ensure that no other IOBufs point
105 * to the same buffer. The unshare() method may be used to ensure that you
106 * have an unshared buffer.
112 * IOBuf objects also contain pointers to next and previous IOBuf objects.
113 * This can be used to represent a single logical piece of data that its stored
114 * in non-contiguous chunks in separate buffers.
116 * A single IOBuf object can only belong to one chain at a time.
118 * IOBuf chains are always circular. The "prev" pointer in the head of the
119 * chain points to the tail of the chain. However, it is up to the user to
120 * decide which IOBuf is the head. Internally the IOBuf code does not care
121 * which element is the head.
123 * The lifetime of all IOBufs in the chain are linked: when one element in the
124 * chain is deleted, all other chained elements are also deleted. Conceptually
125 * it is simplest to treat this as if the head of the chain owns all other
126 * IOBufs in the chain. When you delete the head of the chain, it will delete
127 * the other elements as well. For this reason, prependChain() and
128 * appendChain() take ownership of of the new elements being added to this
131 * When the coalesce() method is used to coalesce an entire IOBuf chain into a
132 * single IOBuf, all other IOBufs in the chain are eliminated and automatically
133 * deleted. The unshare() method may coalesce the chain; if it does it will
134 * similarly delete all IOBufs eliminated from the chain.
136 * As discussed in the following section, it is up to the user to maintain a
137 * lock around the entire IOBuf chain if multiple threads need to access the
138 * chain. IOBuf does not provide any internal locking.
144 * When used in multithread programs, a single IOBuf object should only be used
145 * in a single thread at a time. If a caller uses a single IOBuf across
146 * multiple threads the caller is responsible for using an external lock to
147 * synchronize access to the IOBuf.
149 * Two separate IOBuf objects may be accessed concurrently in separate threads
150 * without locking, even if they point to the same underlying buffer. The
151 * buffer reference count is always accessed atomically, and no other
152 * operations should affect other IOBufs that point to the same data segment.
153 * The caller is responsible for using unshare() to ensure that the data buffer
154 * is not shared by other IOBufs before writing to it, and this ensures that
155 * the data itself is not modified in one thread while also being accessed from
158 * For IOBuf chains, no two IOBufs in the same chain should be accessed
159 * simultaneously in separate threads. The caller must maintain a lock around
160 * the entire chain if the chain, or individual IOBufs in the chain, may be
161 * accessed by multiple threads.
164 * IOBuf Object Allocation/Sharing
165 * -------------------------------
167 * IOBuf objects themselves are always allocated on the heap. The IOBuf
168 * constructors are private, so IOBuf objects may not be created on the stack.
169 * In part this is done since some IOBuf objects use small-buffer optimization
170 * and contain the buffer data immediately after the IOBuf object itself. The
171 * coalesce() and unshare() methods also expect to be able to delete subsequent
172 * IOBuf objects in the chain if they are no longer needed due to coalescing.
174 * The IOBuf structure also does not provide room for an intrusive refcount on
175 * the IOBuf object itself, only the underlying data buffer is reference
176 * counted. If users want to share the same IOBuf object between multiple
177 * parts of the code, they are responsible for managing this sharing on their
178 * own. (For example, by using a shared_ptr. Alternatively, users always have
179 * the option of using clone() to create a second IOBuf that points to the same
180 * underlying buffer.)
182 * With jemalloc, allocating small objects like IOBuf objects should be
183 * relatively fast, and the cost of allocating IOBuf objects on the heap and
184 * cloning new IOBufs should be relatively cheap.
187 // Is T a unique_ptr<> to a standard-layout type?
188 template <class T, class Enable=void> struct IsUniquePtrToSL
189 : public std::false_type { };
190 template <class T, class D>
191 struct IsUniquePtrToSL<
192 std::unique_ptr<T, D>,
193 typename std::enable_if<std::is_standard_layout<T>::value>::type>
194 : public std::true_type { };
195 } // namespace detail
201 typedef ByteRange value_type;
202 typedef Iterator iterator;
203 typedef Iterator const_iterator;
205 typedef void (*FreeFunction)(void* buf, void* userData);
208 * Allocate a new IOBuf object with the requested capacity.
210 * Returns a new IOBuf object that must be (eventually) deleted by the
211 * caller. The returned IOBuf may actually have slightly more capacity than
214 * The data pointer will initially point to the start of the newly allocated
215 * buffer, and will have a data length of 0.
217 * Throws std::bad_alloc on error.
219 static std::unique_ptr<IOBuf> create(uint32_t capacity);
222 * Create a new IOBuf pointing to an existing data buffer.
224 * The new IOBuffer will assume ownership of the buffer, and free it by
225 * calling the specified FreeFunction when the last IOBuf pointing to this
226 * buffer is destroyed. The function will be called with a pointer to the
227 * buffer as the first argument, and the supplied userData value as the
228 * second argument. The free function must never throw exceptions.
230 * If no FreeFunction is specified, the buffer will be freed using free().
232 * The IOBuf data pointer will initially point to the start of the buffer,
234 * In the first version of this function, the length of data is unspecified
235 * and is initialized to the capacity of the buffer
237 * In the second version, the user specifies the valid length of data
240 * On error, std::bad_alloc will be thrown. If freeOnError is true (the
241 * default) the buffer will be freed before throwing the error.
243 static std::unique_ptr<IOBuf> takeOwnership(void* buf, uint32_t capacity,
244 FreeFunction freeFn = NULL,
245 void* userData = NULL,
246 bool freeOnError = true) {
247 return takeOwnership(buf, capacity, capacity, freeFn,
248 userData, freeOnError);
251 static std::unique_ptr<IOBuf> takeOwnership(void* buf, uint32_t capacity,
253 FreeFunction freeFn = NULL,
254 void* userData = NULL,
255 bool freeOnError = true);
258 * Create a new IOBuf pointing to an existing data buffer made up of
259 * count objects of a given standard-layout type.
261 * This is dangerous -- it is essentially equivalent to doing
262 * reinterpret_cast<unsigned char*> on your data -- but it's often useful
263 * for serialization / deserialization.
265 * The new IOBuffer will assume ownership of the buffer, and free it
266 * appropriately (by calling the UniquePtr's custom deleter, or by calling
267 * delete or delete[] appropriately if there is no custom deleter)
268 * when the buffer is destroyed. The custom deleter, if any, must never
271 * The IOBuf data pointer will initially point to the start of the buffer,
272 * and the length will be the full capacity of the buffer (count *
275 * On error, std::bad_alloc will be thrown, and the buffer will be freed
276 * before throwing the error.
278 template <class UniquePtr>
279 static typename std::enable_if<detail::IsUniquePtrToSL<UniquePtr>::value,
280 std::unique_ptr<IOBuf>>::type
281 takeOwnership(UniquePtr&& buf, size_t count=1);
284 * Create a new IOBuf object that points to an existing user-owned buffer.
286 * This should only be used when the caller knows the lifetime of the IOBuf
287 * object ahead of time and can ensure that all IOBuf objects that will point
288 * to this buffer will be destroyed before the buffer itself is destroyed.
290 * This buffer will not be freed automatically when the last IOBuf
291 * referencing it is destroyed. It is the caller's responsibility to free
292 * the buffer after the last IOBuf has been destroyed.
294 * The IOBuf data pointer will initially point to the start of the buffer,
295 * and the length will be the full capacity of the buffer.
297 * An IOBuf created using wrapBuffer() will always be reported as shared.
298 * unshare() may be used to create a writable copy of the buffer.
300 * On error, std::bad_alloc will be thrown.
302 static std::unique_ptr<IOBuf> wrapBuffer(const void* buf, uint32_t capacity);
305 * Convenience function to create a new IOBuf object that copies data from a
306 * user-supplied buffer, optionally allocating a given amount of
307 * headroom and tailroom.
309 static std::unique_ptr<IOBuf> copyBuffer(const void* buf, uint32_t size,
311 uint32_t minTailroom=0);
314 * Convenience function to create a new IOBuf object that copies data from a
315 * user-supplied string, optionally allocating a given amount of
316 * headroom and tailroom.
318 * Beware when attempting to invoke this function with a constant string
319 * literal and a headroom argument: you will likely end up invoking the
320 * version of copyBuffer() above. IOBuf::copyBuffer("hello", 3) will treat
321 * the first argument as a const void*, and will invoke the version of
322 * copyBuffer() above, with the size argument of 3.
324 static std::unique_ptr<IOBuf> copyBuffer(const std::string& buf,
326 uint32_t minTailroom=0);
329 * A version of copyBuffer() that returns a null pointer if the input string
332 static std::unique_ptr<IOBuf> maybeCopyBuffer(const std::string& buf,
334 uint32_t minTailroom=0);
337 * Convenience function to free a chain of IOBufs held by a unique_ptr.
339 static void destroy(std::unique_ptr<IOBuf>&& data) {
340 auto destroyer = std::move(data);
344 * Destroy this IOBuf.
346 * Deleting an IOBuf will automatically destroy all IOBufs in the chain.
347 * (See the comments above regarding the ownership model of IOBuf chains.
348 * All subsequent IOBufs in the chain are considered to be owned by the head
349 * of the chain. Users should only explicitly delete the head of a chain.)
351 * When each individual IOBuf is destroyed, it will release its reference
352 * count on the underlying buffer. If it was the last user of the buffer,
353 * the buffer will be freed.
358 * Check whether the chain is empty (i.e., whether the IOBufs in the
359 * chain have a total data length of zero).
361 * This method is semantically equivalent to
362 * i->computeChainDataLength()==0
363 * but may run faster because it can short-circuit as soon as it
364 * encounters a buffer with length()!=0
369 * Get the pointer to the start of the data.
371 const uint8_t* data() const {
376 * Get a writable pointer to the start of the data.
378 * The caller is responsible for calling unshare() first to ensure that it is
379 * actually safe to write to the buffer.
381 uint8_t* writableData() {
386 * Get the pointer to the end of the data.
388 const uint8_t* tail() const {
389 return data_ + length_;
393 * Get a writable pointer to the end of the data.
395 * The caller is responsible for calling unshare() first to ensure that it is
396 * actually safe to write to the buffer.
398 uint8_t* writableTail() {
399 return data_ + length_;
403 * Get the data length.
405 uint32_t length() const {
410 * Get the amount of head room.
412 * Returns the number of bytes in the buffer before the start of the data.
414 uint32_t headroom() const {
415 return data_ - buffer();
419 * Get the amount of tail room.
421 * Returns the number of bytes in the buffer after the end of the data.
423 uint32_t tailroom() const {
424 return bufferEnd() - tail();
428 * Get the pointer to the start of the buffer.
430 * Note that this is the pointer to the very beginning of the usable buffer,
431 * not the start of valid data within the buffer. Use the data() method to
432 * get a pointer to the start of the data within the buffer.
434 const uint8_t* buffer() const {
435 return (flags_ & kFlagExt) ? ext_.buf : int_.buf;
439 * Get a writable pointer to the start of the buffer.
441 * The caller is responsible for calling unshare() first to ensure that it is
442 * actually safe to write to the buffer.
444 uint8_t* writableBuffer() {
445 return (flags_ & kFlagExt) ? ext_.buf : int_.buf;
449 * Get the pointer to the end of the buffer.
451 * Note that this is the pointer to the very end of the usable buffer,
452 * not the end of valid data within the buffer. Use the tail() method to
453 * get a pointer to the end of the data within the buffer.
455 const uint8_t* bufferEnd() const {
456 return (flags_ & kFlagExt) ?
457 ext_.buf + ext_.capacity :
458 int_.buf + kMaxInternalDataSize;
462 * Get the total size of the buffer.
464 * This returns the total usable length of the buffer. Use the length()
465 * method to get the length of the actual valid data in this IOBuf.
467 uint32_t capacity() const {
468 return (flags_ & kFlagExt) ? ext_.capacity : kMaxInternalDataSize;
472 * Get a pointer to the next IOBuf in this chain.
477 const IOBuf* next() const {
482 * Get a pointer to the previous IOBuf in this chain.
487 const IOBuf* prev() const {
492 * Shift the data forwards in the buffer.
494 * This shifts the data pointer forwards in the buffer to increase the
495 * headroom. This is commonly used to increase the headroom in a newly
498 * The caller is responsible for ensuring that there is sufficient
499 * tailroom in the buffer before calling advance().
501 * If there is a non-zero data length, advance() will use memmove() to shift
502 * the data forwards in the buffer. In this case, the caller is responsible
503 * for making sure the buffer is unshared, so it will not affect other IOBufs
504 * that may be sharing the same underlying buffer.
506 void advance(uint32_t amount) {
507 // In debug builds, assert if there is a problem.
508 assert(amount <= tailroom());
511 memmove(data_ + amount, data_, length_);
517 * Shift the data backwards in the buffer.
519 * The caller is responsible for ensuring that there is sufficient headroom
520 * in the buffer before calling retreat().
522 * If there is a non-zero data length, retreat() will use memmove() to shift
523 * the data backwards in the buffer. In this case, the caller is responsible
524 * for making sure the buffer is unshared, so it will not affect other IOBufs
525 * that may be sharing the same underlying buffer.
527 void retreat(uint32_t amount) {
528 // In debug builds, assert if there is a problem.
529 assert(amount <= headroom());
532 memmove(data_ - amount, data_, length_);
538 * Adjust the data pointer to include more valid data at the beginning.
540 * This moves the data pointer backwards to include more of the available
541 * buffer. The caller is responsible for ensuring that there is sufficient
542 * headroom for the new data. The caller is also responsible for populating
543 * this section with valid data.
545 * This does not modify any actual data in the buffer.
547 void prepend(uint32_t amount) {
548 CHECK(amount <= headroom());
554 * Adjust the tail pointer to include more valid data at the end.
556 * This moves the tail pointer forwards to include more of the available
557 * buffer. The caller is responsible for ensuring that there is sufficient
558 * tailroom for the new data. The caller is also responsible for populating
559 * this section with valid data.
561 * This does not modify any actual data in the buffer.
563 void append(uint32_t amount) {
564 CHECK(amount <= tailroom());
569 * Adjust the data pointer forwards to include less valid data.
571 * This moves the data pointer forwards so that the first amount bytes are no
572 * longer considered valid data. The caller is responsible for ensuring that
573 * amount is less than or equal to the actual data length.
575 * This does not modify any actual data in the buffer.
577 void trimStart(uint32_t amount) {
578 CHECK(amount <= length_);
584 * Adjust the tail pointer backwards to include less valid data.
586 * This moves the tail pointer backwards so that the last amount bytes are no
587 * longer considered valid data. The caller is responsible for ensuring that
588 * amount is less than or equal to the actual data length.
590 * This does not modify any actual data in the buffer.
592 void trimEnd(uint32_t amount) {
593 CHECK(amount <= length_);
600 * Postcondition: headroom() == 0, length() == 0, tailroom() == capacity()
603 data_ = writableBuffer();
608 * Ensure that this buffer has at least minHeadroom headroom bytes and at
609 * least minTailroom tailroom bytes. The buffer must be writable
610 * (you must call unshare() before this, if necessary).
612 * Postcondition: headroom() >= minHeadroom, tailroom() >= minTailroom,
613 * the data (between data() and data() + length()) is preserved.
615 void reserve(uint32_t minHeadroom, uint32_t minTailroom) {
616 // Maybe we don't need to do anything.
617 if (headroom() >= minHeadroom && tailroom() >= minTailroom) {
620 // If the buffer is empty but we have enough total room (head + tail),
621 // move the data_ pointer around.
623 headroom() + tailroom() >= minHeadroom + minTailroom) {
624 data_ = writableBuffer() + minHeadroom;
627 // Bah, we have to do actual work.
628 reserveSlow(minHeadroom, minTailroom);
632 * Return true if this IOBuf is part of a chain of multiple IOBufs, or false
633 * if this is the only IOBuf in its chain.
635 bool isChained() const {
636 assert((next_ == this) == (prev_ == this));
637 return next_ != this;
641 * Get the number of IOBufs in this chain.
643 * Beware that this method has to walk the entire chain.
644 * Use isChained() if you just want to check if this IOBuf is part of a chain
647 uint32_t countChainElements() const;
650 * Get the length of all the data in this IOBuf chain.
652 * Beware that this method has to walk the entire chain.
654 uint64_t computeChainDataLength() const;
657 * Insert another IOBuf chain immediately before this IOBuf.
659 * For example, if there are two IOBuf chains (A, B, C) and (D, E, F),
660 * and B->prependChain(D) is called, the (D, E, F) chain will be subsumed
661 * and become part of the chain starting at A, which will now look like
664 * Note that since IOBuf chains are circular, head->prependChain(other) can
665 * be used to append the other chain at the very end of the chain pointed to
666 * by head. For example, if there are two IOBuf chains (A, B, C) and
667 * (D, E, F), and A->prependChain(D) is called, the chain starting at A will
668 * now consist of (A, B, C, D, E, F)
670 * The elements in the specified IOBuf chain will become part of this chain,
671 * and will be owned by the head of this chain. When this chain is
672 * destroyed, all elements in the supplied chain will also be destroyed.
674 * For this reason, appendChain() only accepts an rvalue-reference to a
675 * unique_ptr(), to make it clear that it is taking ownership of the supplied
676 * chain. If you have a raw pointer, you can pass in a new temporary
677 * unique_ptr around the raw pointer. If you have an existing,
678 * non-temporary unique_ptr, you must call std::move(ptr) to make it clear
679 * that you are destroying the original pointer.
681 void prependChain(std::unique_ptr<IOBuf>&& iobuf);
684 * Append another IOBuf chain immediately after this IOBuf.
686 * For example, if there are two IOBuf chains (A, B, C) and (D, E, F),
687 * and B->appendChain(D) is called, the (D, E, F) chain will be subsumed
688 * and become part of the chain starting at A, which will now look like
691 * The elements in the specified IOBuf chain will become part of this chain,
692 * and will be owned by the head of this chain. When this chain is
693 * destroyed, all elements in the supplied chain will also be destroyed.
695 * For this reason, appendChain() only accepts an rvalue-reference to a
696 * unique_ptr(), to make it clear that it is taking ownership of the supplied
697 * chain. If you have a raw pointer, you can pass in a new temporary
698 * unique_ptr around the raw pointer. If you have an existing,
699 * non-temporary unique_ptr, you must call std::move(ptr) to make it clear
700 * that you are destroying the original pointer.
702 void appendChain(std::unique_ptr<IOBuf>&& iobuf) {
703 // Just use prependChain() on the next element in our chain
704 next_->prependChain(std::move(iobuf));
708 * Remove this IOBuf from its current chain.
710 * Since ownership of all elements an IOBuf chain is normally maintained by
711 * the head of the chain, unlink() transfers ownership of this IOBuf from the
712 * chain and gives it to the caller. A new unique_ptr to the IOBuf is
713 * returned to the caller. The caller must store the returned unique_ptr (or
714 * call release() on it) to take ownership, otherwise the IOBuf will be
715 * immediately destroyed.
717 * Since unlink transfers ownership of the IOBuf to the caller, be careful
718 * not to call unlink() on the head of a chain if you already maintain
719 * ownership on the head of the chain via other means. The pop() method
720 * is a better choice for that situation.
722 std::unique_ptr<IOBuf> unlink() {
723 next_->prev_ = prev_;
724 prev_->next_ = next_;
727 return std::unique_ptr<IOBuf>(this);
731 * Remove this IOBuf from its current chain and return a unique_ptr to
732 * the IOBuf that formerly followed it in the chain.
734 std::unique_ptr<IOBuf> pop() {
736 next_->prev_ = prev_;
737 prev_->next_ = next_;
740 return std::unique_ptr<IOBuf>((next == this) ? NULL : next);
744 * Remove a subchain from this chain.
746 * Remove the subchain starting at head and ending at tail from this chain.
748 * Returns a unique_ptr pointing to head. (In other words, ownership of the
749 * head of the subchain is transferred to the caller.) If the caller ignores
750 * the return value and lets the unique_ptr be destroyed, the subchain will
751 * be immediately destroyed.
753 * The subchain referenced by the specified head and tail must be part of the
754 * same chain as the current IOBuf, but must not contain the current IOBuf.
755 * However, the specified head and tail may be equal to each other (i.e.,
756 * they may be a subchain of length 1).
758 std::unique_ptr<IOBuf> separateChain(IOBuf* head, IOBuf* tail) {
759 assert(head != this);
760 assert(tail != this);
762 head->prev_->next_ = tail->next_;
763 tail->next_->prev_ = head->prev_;
768 return std::unique_ptr<IOBuf>(head);
772 * Return true if at least one of the IOBufs in this chain are shared,
773 * or false if all of the IOBufs point to unique buffers.
775 * Use isSharedOne() to only check this IOBuf rather than the entire chain.
777 bool isShared() const {
778 const IOBuf* current = this;
780 if (current->isSharedOne()) {
783 current = current->next_;
784 if (current == this) {
791 * Return true if other IOBufs are also pointing to the buffer used by this
792 * IOBuf, and false otherwise.
794 * If this IOBuf points at a buffer owned by another (non-IOBuf) part of the
795 * code (i.e., if the IOBuf was created using wrapBuffer(), or was cloned
796 * from such an IOBuf), it is always considered shared.
798 * This only checks the current IOBuf, and not other IOBufs in the chain.
800 bool isSharedOne() const {
801 // If this is a user-owned buffer, it is always considered shared
802 if (flags_ & kFlagUserOwned) {
806 if (flags_ & kFlagExt) {
807 return ext_.sharedInfo->refcount.load(std::memory_order_acquire) > 1;
814 * Ensure that this IOBuf has a unique buffer that is not shared by other
817 * unshare() operates on an entire chain of IOBuf objects. If the chain is
818 * shared, it may also coalesce the chain when making it unique. If the
819 * chain is coalesced, subsequent IOBuf objects in the current chain will be
820 * automatically deleted.
822 * Note that buffers owned by other (non-IOBuf) users are automatically
825 * Throws std::bad_alloc on error. On error the IOBuf chain will be
828 * Currently unshare may also throw std::overflow_error if it tries to
829 * coalesce. (TODO: In the future it would be nice if unshare() were smart
830 * enough not to coalesce the entire buffer if the data is too large.
831 * However, in practice this seems unlikely to become an issue.)
842 * Ensure that this IOBuf has a unique buffer that is not shared by other
845 * unshareOne() operates on a single IOBuf object. This IOBuf will have a
846 * unique buffer after unshareOne() returns, but other IOBufs in the chain
847 * may still be shared after unshareOne() returns.
849 * Throws std::bad_alloc on error. On error the IOBuf will be unmodified.
858 * Coalesce this IOBuf chain into a single buffer.
860 * This method moves all of the data in this IOBuf chain into a single
861 * contiguous buffer, if it is not already in one buffer. After coalesce()
862 * returns, this IOBuf will be a chain of length one. Other IOBufs in the
863 * chain will be automatically deleted.
865 * After coalescing, the IOBuf will have at least as much headroom as the
866 * first IOBuf in the chain, and at least as much tailroom as the last IOBuf
869 * Throws std::bad_alloc on error. On error the IOBuf chain will be
870 * unmodified. Throws std::overflow_error if the length of the entire chain
871 * larger than can be described by a uint32_t capacity.
881 * Ensure that this chain has at least maxLength bytes available as a
882 * contiguous memory range.
884 * This method coalesces whole buffers in the chain into this buffer as
885 * necessary until this buffer's length() is at least maxLength.
887 * After coalescing, the IOBuf will have at least as much headroom as the
888 * first IOBuf in the chain, and at least as much tailroom as the last IOBuf
889 * that was coalesced.
891 * Throws std::bad_alloc on error. On error the IOBuf chain will be
892 * unmodified. Throws std::overflow_error if the length of the coalesced
893 * portion of the chain is larger than can be described by a uint32_t
894 * capacity. (Although maxLength is uint32_t, gather() doesn't split
895 * buffers, so coalescing whole buffers may result in a capacity that can't
896 * be described in uint32_t.
898 * Upon return, either enough of the chain was coalesced into a contiguous
899 * region, or the entire chain was coalesced. That is,
900 * length() >= maxLength || !isChained() is true.
902 void gather(uint32_t maxLength) {
903 if (!isChained() || length_ >= maxLength) {
906 coalesceSlow(maxLength);
910 * Return a new IOBuf chain sharing the same data as this chain.
912 * The new IOBuf chain will normally point to the same underlying data
913 * buffers as the original chain. (The one exception to this is if some of
914 * the IOBufs in this chain contain small internal data buffers which cannot
917 std::unique_ptr<IOBuf> clone() const;
920 * Return a new IOBuf with the same data as this IOBuf.
922 * The new IOBuf returned will not be part of a chain (even if this IOBuf is
923 * part of a larger chain).
925 std::unique_ptr<IOBuf> cloneOne() const;
928 * Return an iovector suitable for e.g. writev()
930 * auto iov = buf->getIov();
931 * auto xfer = writev(fd, iov.data(), iov.size());
933 * Naturally, the returned iovector is invalid if you modify the buffer
936 folly::fbvector<struct iovec> getIov() const;
938 // Overridden operator new and delete.
939 // These directly use malloc() and free() to allocate the space for IOBuf
940 // objects. This is needed since IOBuf::create() manually uses malloc when
941 // allocating IOBuf objects with an internal buffer.
942 void* operator new(size_t size);
943 void* operator new(size_t size, void* ptr);
944 void operator delete(void* ptr);
947 * Destructively convert this IOBuf to a fbstring efficiently.
948 * We rely on fbstring's AcquireMallocatedString constructor to
951 fbstring moveToFbString();
954 * Iteration support: a chain of IOBufs may be iterated through using
955 * STL-style iterators over const ByteRanges. Iterators are only invalidated
956 * if the IOBuf that they currently point to is removed.
958 Iterator cbegin() const;
959 Iterator cend() const;
960 Iterator begin() const;
961 Iterator end() const;
966 kFlagUserOwned = 0x2,
967 kFlagFreeSharedInfo = 0x4,
970 // Values for the ExternalBuf type field.
971 // We currently don't really use this for anything, other than to have it
972 // around for debugging purposes. We store it at the moment just because we
973 // have the 4 extra bytes in the ExternalBuf struct that would just be
974 // padding otherwise.
975 enum ExtBufTypeEnum {
977 kExtUserSupplied = 1,
983 SharedInfo(FreeFunction fn, void* arg);
985 // A pointer to a function to call to free the buffer when the refcount
986 // hits 0. If this is NULL, free() will be used instead.
989 std::atomic<uint32_t> refcount;
995 // SharedInfo may be NULL if kFlagUserOwned is set. It is non-NULL
996 // in all other cases.
997 SharedInfo* sharedInfo;
1000 uint8_t buf[] __attribute__((aligned));
1003 // The maximum size for an IOBuf object, including any internal data buffer
1004 static const uint32_t kMaxIOBufSize = 256;
1005 static const uint32_t kMaxInternalDataSize;
1007 // Forbidden copy constructor and assignment opererator
1008 IOBuf(IOBuf const &);
1009 IOBuf& operator=(IOBuf const &);
1012 * Create a new IOBuf with internal data.
1014 * end is a pointer to the end of the IOBuf's internal data buffer.
1016 explicit IOBuf(uint8_t* end);
1019 * Create a new IOBuf pointing to an external buffer.
1021 * The caller is responsible for holding a reference count for this new
1022 * IOBuf. The IOBuf constructor does not automatically increment the
1025 IOBuf(ExtBufTypeEnum type, uint32_t flags,
1026 uint8_t* buf, uint32_t capacity,
1027 uint8_t* data, uint32_t length,
1028 SharedInfo* sharedInfo);
1030 void unshareOneSlow();
1031 void unshareChained();
1032 void coalesceSlow(size_t maxLength=std::numeric_limits<size_t>::max());
1033 // newLength must be the entire length of the buffers between this and
1034 // end (no truncation)
1035 void coalesceAndReallocate(
1039 size_t newTailroom);
1040 void decrementRefcount();
1041 void reserveSlow(uint32_t minHeadroom, uint32_t minTailroom);
1043 static size_t goodExtBufferSize(uint32_t minCapacity);
1044 static void initExtBuffer(uint8_t* buf, size_t mallocSize,
1045 SharedInfo** infoReturn,
1046 uint32_t* capacityReturn);
1047 static void allocExtBuffer(uint32_t minCapacity,
1048 uint8_t** bufReturn,
1049 SharedInfo** infoReturn,
1050 uint32_t* capacityReturn);
1057 * Links to the next and the previous IOBuf in this chain.
1059 * The chain is circularly linked (the last element in the chain points back
1060 * at the head), and next_ and prev_ can never be NULL. If this IOBuf is the
1061 * only element in the chain, next_ and prev_ will both point to this.
1067 * A pointer to the start of the data referenced by this IOBuf, and the
1068 * length of the data.
1070 * This may refer to any subsection of the actual buffer capacity.
1081 struct DeleterBase {
1082 virtual ~DeleterBase() { }
1083 virtual void dispose(void* p) = 0;
1086 template <class UniquePtr>
1087 struct UniquePtrDeleter : public DeleterBase {
1088 typedef typename UniquePtr::pointer Pointer;
1089 typedef typename UniquePtr::deleter_type Deleter;
1091 explicit UniquePtrDeleter(Deleter deleter) : deleter_(std::move(deleter)){ }
1092 void dispose(void* p) {
1094 deleter_(static_cast<Pointer>(p));
1105 static void freeUniquePtrBuffer(void* ptr, void* userData) {
1106 static_cast<DeleterBase*>(userData)->dispose(ptr);
1110 template <class UniquePtr>
1111 typename std::enable_if<detail::IsUniquePtrToSL<UniquePtr>::value,
1112 std::unique_ptr<IOBuf>>::type
1113 IOBuf::takeOwnership(UniquePtr&& buf, size_t count) {
1114 size_t size = count * sizeof(typename UniquePtr::element_type);
1115 CHECK_LT(size, size_t(std::numeric_limits<uint32_t>::max()));
1116 auto deleter = new UniquePtrDeleter<UniquePtr>(buf.get_deleter());
1117 return takeOwnership(buf.release(),
1119 &IOBuf::freeUniquePtrBuffer,
1123 inline std::unique_ptr<IOBuf> IOBuf::copyBuffer(
1124 const void* data, uint32_t size, uint32_t headroom,
1125 uint32_t minTailroom) {
1126 uint32_t capacity = headroom + size + minTailroom;
1127 std::unique_ptr<IOBuf> buf = create(capacity);
1128 buf->advance(headroom);
1129 memcpy(buf->writableData(), data, size);
1134 inline std::unique_ptr<IOBuf> IOBuf::copyBuffer(const std::string& buf,
1136 uint32_t minTailroom) {
1137 return copyBuffer(buf.data(), buf.size(), headroom, minTailroom);
1140 inline std::unique_ptr<IOBuf> IOBuf::maybeCopyBuffer(const std::string& buf,
1142 uint32_t minTailroom) {
1146 return copyBuffer(buf.data(), buf.size(), headroom, minTailroom);
1149 class IOBuf::Iterator : public boost::iterator_facade<
1150 IOBuf::Iterator, // Derived
1151 const ByteRange, // Value
1152 boost::forward_traversal_tag // Category or traversal
1154 friend class boost::iterator_core_access;
1156 // Note that IOBufs are stored as a circular list without a guard node,
1157 // so pos == end is ambiguous (it may mean "begin" or "end"). To solve
1158 // the ambiguity (at the cost of one extra comparison in the "increment"
1159 // code path), we define end iterators as having pos_ == end_ == nullptr
1160 // and we only allow forward iteration.
1161 explicit Iterator(const IOBuf* pos, const IOBuf* end)
1164 // Sadly, we must return by const reference, not by value.
1172 val_ = ByteRange(pos_->data(), pos_->tail());
1175 void adjustForEnd() {
1177 pos_ = end_ = nullptr;
1184 const ByteRange& dereference() const {
1188 bool equal(const Iterator& other) const {
1189 // We must compare end_ in addition to pos_, because forward traversal
1190 // requires that if two iterators are equal (a == b) and dereferenceable,
1192 return pos_ == other.pos_ && end_ == other.end_;
1196 pos_ = pos_->next();
1205 inline IOBuf::Iterator IOBuf::begin() const { return cbegin(); }
1206 inline IOBuf::Iterator IOBuf::end() const { return cend(); }
1210 #endif // FOLLY_IO_IOBUF_H_