folly.git
4 years agoMove folly/Functional.h v2017.11.06.00
Yedidya Feldblum [Sun, 5 Nov 2017 04:34:13 +0000 (21:34 -0700)]
Move folly/Functional.h

Summary: [Folly] Move `folly/Functional.h` to `folly/lang/RValueReferenceWrapper.h`.

Reviewed By: Orvid

Differential Revision: D6242160

fbshipit-source-id: a8ec42106cfe0ea617c87d382694d5eb2f0f61a0

4 years agoconstexpr_clamp
Yedidya Feldblum [Sat, 4 Nov 2017 22:34:00 +0000 (15:34 -0700)]
constexpr_clamp

Summary:
[Folly] `constexpr_clamp`.

Like `std::clamp` (C++17).

Reviewed By: Orvid

Differential Revision: D6236825

fbshipit-source-id: 0f6c5dc9a955b148021ee6ed3e86201b53ae090c

4 years agoMake SemiFuture::via throw on nullptr executor.
Lee Howes [Sat, 4 Nov 2017 02:51:04 +0000 (19:51 -0700)]
Make SemiFuture::via throw on nullptr executor.

Summary: Make SemiFuture throw if no executor provided to via because in that situation the deferred work will never run.

Reviewed By: yfeldblum

Differential Revision: D6233233

fbshipit-source-id: 43b642c46cc0be80b1f13c03bdaf20b8038acec2

4 years agoMove folly/SafeAssert.h
Yedidya Feldblum [Fri, 3 Nov 2017 23:53:23 +0000 (16:53 -0700)]
Move folly/SafeAssert.h

Summary: [Folly] Move `folly/SafeAssert.h` to `folly/lang/`.

Reviewed By: Orvid

Differential Revision: D6230421

fbshipit-source-id: 0086cd6fedd4ce0e7a4d5302a41153ec1a502e74

4 years agoParameter order
Lee Howes [Fri, 3 Nov 2017 23:12:29 +0000 (16:12 -0700)]
Parameter order

Summary: Fix for function parameter order that shows up in opt build on later diff.

Reviewed By: andriigrynenko

Differential Revision: D6237125

fbshipit-source-id: fbb7be2c70b32203c658fc239cd74164e01fa1ca

4 years agoNitpick: fix typo in name
Louis Dionne [Fri, 3 Nov 2017 22:17:09 +0000 (15:17 -0700)]
Nitpick: fix typo in name

Summary: Closes https://github.com/facebook/folly/pull/705

Reviewed By: yfeldblum

Differential Revision: D6234351

Pulled By: Orvid

fbshipit-source-id: e71bb3882c783a47ace4d08f134b2e450aaabb6b

4 years agoExpose the zerocopy buf ID, change the AsyncSocket fd constructor to accept the id...
Dan Melnic [Fri, 3 Nov 2017 18:16:10 +0000 (11:16 -0700)]
Expose the zerocopy buf ID, change the AsyncSocket fd constructor to accept the id, Buff->Buf, make the cmsghdr methods private

Summary: Expose the zerocopy buf ID, change the AsyncScokey fd constructor to accept the id, Buff->Buf, make the cmsghdr methods private

Reviewed By: djwatson

Differential Revision: D6221324

fbshipit-source-id: d0fc4937adf6cf5790d11e406ffd3ec64c558b9c

4 years agoMissing Future/SemiFuture->Value conversion check
Lee Howes [Fri, 3 Nov 2017 17:16:04 +0000 (10:16 -0700)]
Missing Future/SemiFuture->Value conversion check

Summary: Conversion check was lost in an earlier refactor. This meant that SemiFuture could be accidentally converted to Future through the value constructor. This should be disabled.

Reviewed By: yfeldblum

Differential Revision: D6214526

fbshipit-source-id: 3fc2d026ec6062b38b9500c8adf3eee12c0f2693

4 years agoAlias std::launder when it is available
Yedidya Feldblum [Fri, 3 Nov 2017 02:44:08 +0000 (19:44 -0700)]
Alias std::launder when it is available

Summary: [Folly] Alias `std::launder` when it is available.

Reviewed By: Orvid

Differential Revision: D6221443

fbshipit-source-id: 33136a8744a39db01fb05513d5ed5476ea67559a

4 years agodon't try to run the poly tests on gcc-4.9. they will fail.
Eric Niebler [Thu, 2 Nov 2017 23:58:07 +0000 (16:58 -0700)]
don't try to run the poly tests on gcc-4.9. they will fail.

Summary: gcc-4.9 does not support features that Poly needs. Disable the tests on that platform.

Reviewed By: meyering

Differential Revision: D6211674

fbshipit-source-id: 289f029122a45b0f9ec740c62b1faaafb51dcab5

4 years agoMake Expected presume it has a value and not an Error
Phil Willoughby [Thu, 2 Nov 2017 22:38:42 +0000 (15:38 -0700)]
Make Expected presume it has a value and not an Error

Summary: This only affects instruction ordering in GCC-compatible compilers to make the value-having branch preferred.

Reviewed By: davidtgoldblatt, nbronson, yfeldblum

Differential Revision: D6223188

fbshipit-source-id: 57c69b88eda7ee769912874921c45b47ec7a38de

4 years agoChange kDefaultZeroCopyThreshold to 0 to avoid a regression and avoid a failure while...
Dan Melnic [Thu, 2 Nov 2017 20:50:33 +0000 (13:50 -0700)]
Change kDefaultZeroCopyThreshold to 0 to avoid a regression and avoid a failure while running as not root

Summary:
Change kDefaultZeroCopyThreshold to 0 to avoid a regression when using a buffer chain that exceeds 32K but each buffer is small.
Change the benchmark to set it's own threshold. Also use calloc vs malloc (in the benchmark only) to get around some weird kernel interaction on non zero copy enabled systems - 2 back to back tests report very different results.

Reviewed By: djwatson

Differential Revision: D6112299

fbshipit-source-id: 3895d3ece2925c4626284ff364495708293edc3e

4 years agoIntroduce non-throwing try* methods for IPAddress{,V4,V6}.
Andrey Ignatov [Thu, 2 Nov 2017 17:22:09 +0000 (10:22 -0700)]
Introduce non-throwing try* methods for IPAddress{,V4,V6}.

Summary:
Now there is no interface to create `IPAddress{,V4,V6}` from a string or
`ByteRange` that doesn't throw. All available static methods throw
`IPAddressFormatException`.

It has a few disadvantages:

== 1. It's unsafe ==

Caller is not forced to catch exception, it's very easy to write
`IPAddress(potentiallyInvalidString)` and discover that it can throw when it's
already in prod.

== 2. It's inconvenient ==

if caller is aware about exception, (s)he's forced to write `try {} catch` that
is inconvenient and leads to code like this:
  folly::IPAddress taskIp;
  try {
    taskIp = folly::IPAddress(kv.second.taskIp);
  } catch (const folly::IPAddressFormatException&) {
    // Error handling ..
  }
  // Use IP ..

== 3. It's expensive ==

Amended benchmark shows that `IPAddress` constructor is ~10 times slower when a
string with invalid IP is passed to it.

 ---

The diff introduces two non-throwing interfaces for all tree flavors of `IPAddress`:

`tryFromString()` tries to create IP address from string and returns either
corresponding IP address or `enum class IPAddressFormatError` using
`folly::Expected`.

`tryFromBinary()` does same thing but for `ByteRange` input.

The code can be as short as:
  if (auto maybeIp = IPAddress::tryFromString(ipStr)) {
    // Use maybeIp.value() ..
  }

The `try` prefix is chosen to be consistent with other interfaces in folly,
e.g. `to` and `tryTo`.

Reviewed By: yfeldblum

Differential Revision: D6211182

fbshipit-source-id: f27cf90997c100a5fd42138e66ff9bb172204c20

4 years agoRefactor is_simple_allocator and callers
Yedidya Feldblum [Thu, 2 Nov 2017 16:55:29 +0000 (09:55 -0700)]
Refactor is_simple_allocator and callers

Summary:
[Folly] Refactor `is_simple_allocator` and callers.

* Swap order of template parameters.
* Do decaying in the callers instead.
* Do a direct invocability test, rather than an indirect test of whether the allocator has a method `destroy` with the expected signature.

Reviewed By: ericniebler

Differential Revision: D6184062

fbshipit-source-id: aec32e6e323b8c6023b94c258ab2bcddd8c53e09

4 years agoFix folly::Function under C++17 exception specifier rules
Christopher Dykes [Thu, 2 Nov 2017 03:33:39 +0000 (20:33 -0700)]
Fix folly::Function under C++17 exception specifier rules

Summary: Under C++17's exception specifier rules (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0012r1.html) we need additional specializations to achieve the desired effect.

Reviewed By: yfeldblum

Differential Revision: D6214359

fbshipit-source-id: a007c2fcea1496bdfe4fdf923b39c1611c6ad9bc

4 years agoMake associative container out-of-range exception provide missing key
Ognjen Dragoljevic [Wed, 1 Nov 2017 22:54:38 +0000 (15:54 -0700)]
Make associative container out-of-range exception provide missing key

Summary:
When the key is missing, the standard associative containers throw an exception like `unordered_map::at: key not found`. That's nice, but it would be even better if we would actually know what key is missing. This is not an issue when one is accessing the container directly, but when that access happens several levels deep, such as here within folly formatting logic, we have no way of knowing what key was missing. This poses some difficulties in presenting error to the user.
This change makes folly format throw a subclass of `std::out_of_range` exception on a missing key when a string keyed associative container is used for providing parameters. That subclass stores the actual key used so it can be accessed in the exception handler. Existing callers can still catch `std::out_of_range` so they should be unaffected by this change.

Reviewed By: ot, yfeldblum

Differential Revision: D6202184

fbshipit-source-id: b8a6740aaccc5d8914ad7d099c8b901427f00083

4 years agoQuick comment on DE to clarify use cases and lack of thread safety.
Lee Howes [Wed, 1 Nov 2017 22:46:19 +0000 (15:46 -0700)]
Quick comment on DE to clarify use cases and lack of thread safety.

Summary: (Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D6212919

fbshipit-source-id: b066d99e13e97cbab489132bcb91872ed4407f81

4 years agoFix unsynchronized accesses in IOThreadPoolExecutor::getEventBase
Yedidya Feldblum [Wed, 1 Nov 2017 21:11:27 +0000 (14:11 -0700)]
Fix unsynchronized accesses in IOThreadPoolExecutor::getEventBase

Summary:
[Folly] Fix unsynchronized accesses in `IOThreadPoolExecutor::getEventBase`.

`getEventBase` may be invoked concurrently from two threads - RMWs to `nextThread_` must be synchronized with each other.

`getEventBase` may be invoked concurrently with `setNumThreads` - the former's reads of `threadList_.vec_` must be synchronized with the latter's writes to it.

Reviewed By: kennyyu

Differential Revision: D6206916

fbshipit-source-id: 8bfae158effb5896ab478d0c20310293b037c892

4 years agoadd a complete example to poly's docs, fix typos and think-o's
Eric Niebler [Wed, 1 Nov 2017 19:50:41 +0000 (12:50 -0700)]
add a complete example to poly's docs, fix typos and think-o's

Summary: Address doc criticisms

Reviewed By: Orvid

Differential Revision: D6210376

fbshipit-source-id: fee11cef1c407b092f891a97f94271a81d3718b8

4 years agoFix OSS's "make check"
Alisson Gusatti Azzolini [Wed, 1 Nov 2017 19:11:16 +0000 (12:11 -0700)]
Fix OSS's "make check"

Reviewed By: yfeldblum

Differential Revision: D6182541

fbshipit-source-id: 31d255819df1f97b13e475903c0627a1ac96b516

4 years agoAdd integrated reference counting
Maged Michael [Wed, 1 Nov 2017 14:47:41 +0000 (07:47 -0700)]
Add integrated reference counting

Summary:
Add support for reference counting integrated with the internal structures and operations of the hazard pointer library. The operations are wait-free.
The advantages of this approach over combining reference counting with hazard pointers externally are:
(1) A long list of linked objects that protected by one reference can all be reclaimed together instead of going through a potentially long series of alternating reclamation and calls to retire() for descendants.
(2) Support for iterative deletion as opposed to potential deep recursion of alternating calls to release reference count and object destructors.

Reviewed By: djwatson

Differential Revision: D6142066

fbshipit-source-id: 02bdfcbd5a2c2d5486d937bb2f9cfb6f192f5e1a

4 years agoClear frame cache when activating a fiber
Andrii Grynenko [Wed, 1 Nov 2017 06:06:22 +0000 (23:06 -0700)]
Clear frame cache when activating a fiber

Reviewed By: yfeldblum

Differential Revision: D6207160

fbshipit-source-id: 57468c9d05cdb3ee6e1d10a3a254a5d1bfddc36f

4 years agoA macro for creating member-invoke traits
Yedidya Feldblum [Wed, 1 Nov 2017 05:12:53 +0000 (22:12 -0700)]
A macro for creating member-invoke traits

Summary:
[Folly] A macro for creating member-invoke traits.

The macro creates a specialized traits container with member types and aliases mimicking `std::invoke_result` and the related traits types and aliases.

Reviewed By: aary

Differential Revision: D6195087

fbshipit-source-id: 07c2bbab6cccb04dc8ff12e20923351e8f38abfd

4 years agoAdding DeferredExecutor to support deferred execution of tasks on a future returned...
Lee Howes [Tue, 31 Oct 2017 21:20:08 +0000 (14:20 -0700)]
Adding DeferredExecutor to support deferred execution of tasks on a future returned from an interface.

Summary: This adds a DeferredExecutor type that is boostable, which means that it follows the expectation we expect for C++20 that .then and get will trigger boost-blocking behaviour and ensure work makes progress. Unlike discussions for C++ this adds boost blocking to folly only in the specific case of deferring work to run on the caller's executor, to avoid the necessity to pass an executor into a library purely to ensure that finalisation work and future completion occor on a well-defined exewcutor.

Reviewed By: yfeldblum

Differential Revision: D5828743

fbshipit-source-id: 9a4b69d7deaa33c3cecd6546651b99cc99f0c286

4 years agoRemove unused field or local var.
Martin Martin [Tue, 31 Oct 2017 21:09:59 +0000 (14:09 -0700)]
Remove unused field or local var.

Summary: Remove unused field or local var.

Reviewed By: terrelln

Differential Revision: D6199120

fbshipit-source-id: 616a2b2549c37bcb57d2f8c530b26089f24c2973

4 years agoImprove folly::RequestContext onSet and onUnset efficiency
Teng Qin [Tue, 31 Oct 2017 20:49:15 +0000 (13:49 -0700)]
Improve folly::RequestContext onSet and onUnset efficiency

Summary:
In previous discussions, it has been pointed out that `folly::RequestContext::setContext` consumes considerable amount of CPU cycles in production environments. After some investigation, we thought that might be caused by looping over all `RequestData` instances can calling the virtual `onSet` and `onUnset` callback of them. Both the iteration and invoking virtual methods are not cheap.

As you can see from this change, most of the derived classes of `RequestData` don't override the `onSet` and `onUnset` methods. Mostly likely they are only used for per-Request tracking. So the natural idea is to skip those instances when iterating and avoid the unnecessary virtual method invoke.

I have explored the solution to dynamically examine if the `RequestData` instance added has `onSet` and `onUnset` method overridden. That is possible with GCC's PMF extension, but not [[ http://lists.llvm.org/pipermail/llvm-bugs/2015-July/041164.html | for Clang ]] and probably many other compilers. This definitely won't be very good for `folly`'s probability, even if we gate it by compiler flags.

Therefore, this Diff adds the `hasCallback` method to `RequestData` class indicating whether the instance would have `onSet` and `onUnset` overridden. To make it clear to users that they need to correctly override it in order for their `onSet` and `onUnset` callback to work, making it abstract so that user must override it to something and would aware of that.

Also made some improvements on documentation.

Reviewed By: myreg

Differential Revision: D6144049

fbshipit-source-id: 4c9fd72e9efaeb6763d55f63760eaf582ee4839e

4 years agoImprove folly::RequestContext::get()
Teng Qin [Tue, 31 Oct 2017 20:49:14 +0000 (13:49 -0700)]
Improve folly::RequestContext::get()

Summary:
Since `folly::RequestContext::get()` returns raw pointer, it could directly use the reference returned by `getStaticContext()`
I don't expect this to make much of a difference, just tiny improvements

Reviewed By: yfeldblum

Differential Revision: D6153353

fbshipit-source-id: 1c41d4fc259aa5cb3e69e50ed24bed1ba9caf6c3

4 years agoAdd utility function for loading certificates from a buffer
Alex Guzman [Tue, 31 Oct 2017 06:38:41 +0000 (23:38 -0700)]
Add utility function for loading certificates from a buffer

Summary: Adds a function that reads certificates in from a buffer and returns them as a vector of X509 pointers.

Reviewed By: yfeldblum

Differential Revision: D6133332

fbshipit-source-id: eaaaffcbd4d03f37d9d5b4c99a52b0d968b163ba

4 years agoOutline most throw expressions in Expected
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:27 +0000 (22:00 -0700)]
Outline most throw expressions in Expected

Summary:
[Folly] Outline most `throw` expressions in `Expected`.

They are definitionally cold, but in-line `throw` statements can expand code size more than is desirable.

* Inline `throw` statement: https://godbolt.org/g/LPaf7V.
* Outline `throw` statement: https://godbolt.org/g/HZBXn6.

Reviewed By: Orvid

Differential Revision: D6183613

fbshipit-source-id: 28240bb4aa40790d99da783a3c368db81fded124

4 years agoRemove dep on Format.h from GenerateFingerprintTables.cpp
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:25 +0000 (22:00 -0700)]
Remove dep on Format.h from GenerateFingerprintTables.cpp

Summary:
[Folly] Remove dep on `Format.h` from `GenerateFingerprintTables.cpp`.

`GenerateFingerprintTables.cpp` is a tool used to generate other sources which get built as part of the main library when using the autotools build, so it must be as free of other Folly dependencies as possible.

Reviewed By: ot, Orvid

Differential Revision: D6183725

fbshipit-source-id: f12b18553c78e085599a5505ae57f12bc0cd44b0

4 years agoRemove folly/ContainerTraits.h
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:24 +0000 (22:00 -0700)]
Remove folly/ContainerTraits.h

Summary:
[Folly] Remove `folly/ContainerTraits.h`.

It has some handly helpers, but it is not a real abstraction or utility library. Within Folly, only `folly/Padded.h` uses it, so just rewrite the bit that needs it.

Reviewed By: LeeHowes

Differential Revision: D6183066

fbshipit-source-id: 24a223fe517d21ff531e0fa80172f15d4f963e51

4 years agoMove folly/experimental/AsymmetricMemoryBarrier.h
Yedidya Feldblum [Tue, 31 Oct 2017 00:57:54 +0000 (17:57 -0700)]
Move folly/experimental/AsymmetricMemoryBarrier.h

Summary: [Folly] Move `folly/experimental/AsymmetricMemoryBarrier.h` to `folly/synchronization/AsymmetricMemoryBarrier.h`.

Reviewed By: Orvid

Differential Revision: D6180676

fbshipit-source-id: f4833318cd365181e202d5f379815e728fba168b

4 years agoFix the CMake build
Christopher Dykes [Mon, 30 Oct 2017 23:01:31 +0000 (16:01 -0700)]
Fix the CMake build

Summary: By excluding the tools directory, excluding poly, and moving all the tests that have moved around to their new homes.

Reviewed By: yfeldblum

Differential Revision: D6191644

fbshipit-source-id: bdb39d01a796c1e52257200c0d411a4cb44116ce

4 years agoAllow to pass ObjC blocks into folly::Function
Felix Leupold [Mon, 30 Oct 2017 22:26:55 +0000 (15:26 -0700)]
Allow to pass ObjC blocks into folly::Function

Summary:
In iOS blocks are initially allocated on the stack and only lazily copied to the heap (e.g when they are assigned to a variable). This means, if you pass a block as an rvalue to a C++ method that keeps moving it around instead of copy assigning it at some point, the block remains on the stack and will get freed once the original method is done (leading to use after free if the block is executed later).

This was mitigated by deleting the conversion from ObjC functions to folly functions. Given that all we need is to make sure that the block is allocated on the heap (that is it is an instance of NSMallocBlock rather than NSStackBlock), it seems drastic to ban the conversion. ObjC developers tend to be more familiar with ObjC blocks and will find it convenient to use this conversion.

This diff insteads implements the constructor and assignment operator by wrapping the ObjC block in a c++ lambda and capturing it by copy. ARC keeps track of the reference count and automatically releases the block when the lambda is deallocated. Moreover, copy only increase the retain count (instead of doing an actual copy) if the block was already stored on the heap (https://www.cocoawithlove.com/2009/10/how-blocks-are-implemented-and.html section NSMallocBlock never actually copies).

Reviewed By: ericniebler

Differential Revision: D6109932

fbshipit-source-id: 48bb446d3a66f46affba774cfe1cfb8a60c661de

4 years agoSimplify type_t
Yedidya Feldblum [Mon, 30 Oct 2017 21:52:50 +0000 (14:52 -0700)]
Simplify type_t

Summary:
[Folly] Simplify `type_t` by lifting the type to be aliased into the structure template parameter list.

May also fix curious build failures in some compilers.

Reviewed By: akrieger

Differential Revision: D6188953

fbshipit-source-id: 96e1c3af9c11959c0899c092933158922efa7e60

4 years agoSplit SemiFuture and Future into separate types. Add BasicFuture shared between them.
Lee Howes [Mon, 30 Oct 2017 21:28:33 +0000 (14:28 -0700)]
Split SemiFuture and Future into separate types. Add BasicFuture shared between them.

Summary:
To avoid the risk of bugs caused by a Future being cast to a SemiFuture, and losing some of the properties in the process, this splits SemiFuture and Future into unrelated types, sharing a private superclass for code reuse.
 * Add BasicFuture in futures::detail
 * Make superclass privately inherited.
 * Unset executor when constructing SemiFuture from Future.

Reviewed By: yfeldblum

Differential Revision: D6177780

fbshipit-source-id: dea3116aeec0572bb973c2a561e17785199e86f2

4 years agoMention Windows (Vcpkg) build in README.md
Arkady Shapkin [Mon, 30 Oct 2017 19:15:00 +0000 (12:15 -0700)]
Mention Windows (Vcpkg) build in README.md

Summary: Closes https://github.com/facebook/folly/pull/697

Reviewed By: yfeldblum

Differential Revision: D6144274

Pulled By: Orvid

fbshipit-source-id: a79a2e36e8fcf271925e97ece2a6adbb3c074216

4 years agoFolly.Poly: a library for creating type-erasing polymorphic wrappers
Eric Niebler [Mon, 30 Oct 2017 16:26:13 +0000 (09:26 -0700)]
Folly.Poly: a library for creating type-erasing polymorphic wrappers

Summary:
`Poly` is a class template that makes it relatively easy to define a type-erasing polymorphic object wrapper.

== Type-erasure

`std::function` is one example of a type-erasing polymorphic object wrapper;
`folly::exception_wrapper` is another. Type-erasure is often used as an
alternative to dynamic polymorphism via inheritance-based virtual dispatch.
The distinguishing characteristic of type-erasing wrappers are:

* **Duck typing:** Types do not need to inherit from an abstract base
    class in order to be assignable to a type-erasing wrapper; they merely
    need to satisfy a particular interface.
* **Value semantics:** Type-erasing wrappers are objects that can be
    passed around _by value_. This is in contrast to abstract base classes
    which must be passed by reference or by pointer or else suffer from
    _slicing_, which causes them to lose their polymorphic behaviors.
    Reference semantics make it difficult to reason locally about code.
* **Automatic memory management:** When dealing with inheritance-based
    dynamic polymorphism, it is often necessary to allocate and manage
    objects on the heap. This leads to a proliferation of `shared_ptr`s and
    `unique_ptr`s in APIs, complicating their point-of-use. APIs that take
    type-erasing wrappers, on the other hand, can often store small objects
    in-situ, with no dynamic allocation. The memory management, if any, is
    handled for you, and leads to cleaner APIs: consumers of your API don't
    need to pass `shared_ptr<AbstractBase>`; they can simply pass any object
    that satisfies the interface you require. (`std::function` is a
    particularly compelling example of this benefit. Far worse would be an
    inheritance-based callable solution like
    `shared_ptr<ICallable<void(int)>>`. )

== Example: Defining a type-erasing function wrapper with `folly::Poly`

Defining a polymorphic wrapper with `Poly` is a matter of defining two
things:

* An *interface*, consisting of public member functions, and
* A *mapping* from a concrete type to a set of member function bindings.

Below is a (heavily commented) example of a simple implementation of a
`std::function`-like polymorphic wrapper. Its interface has only a single
member function: `operator()`

  lang=c++
  // An interface for a callable object of a particular signature, Fun
  // (most interfaces don't need to be templates, FWIW).
  template <class Fun>
  struct IFunction;

  template <class R, class... As>
  struct IFunction<R(As...)> {
    // An interface is defined as a nested class template called
    // Interface that takes a single template parameter, Base, from
    // which it inherits.
    template <class Base>
    struct Interface : Base {
      // The Interface has public member functions. These become the
      // public interface of the resulting Poly instantiation.
      // (Implementation note: Poly<IFunction<Sig>> will publicly
      // inherit from this struct, which is what gives it the right
      // member functions.)
      R operator()(As... as) const {
        // The definition of each member function in your interface will
        // always consist of a single line dispatching to folly::call<N>.
        // The "N" corresponds to the N-th member function in the
        // list of member function bindings, Members, defined below.
        // The first argument will always be *this, and the rest of the
        // arguments should simply forward (if necessary) the member
        // function's arguments.
        return static_cast<R>(
            folly::poly_call<0>(*this, std::forward<As>(as)...));
      }
    };
    // The "Members" alias template is a comma-separated list of bound
    // member functions for a given concrete type "T". The
    // "FOLLY_POLY_MEMBERS" macro accepts a comma-separated list, and the
    // (optional) "FOLLY_POLY_MEMBER" macro lets you disambiguate overloads
    // by explicitly specifying the function signature the target member
    // function should have. In this case, we require "T" to have a
    // function call operator with the signature `R(As...) const`.
    //
    // If you are using a C++17-compatible compiler, you can do away with
    // the macros and write this as:
    //
    //   template <class T>
    //   using Members =
    //       folly::PolyMembers<folly::sig<R(As...) const>(&T::operator())>;
    //
    // And since `folly::sig` is only needed for disambiguation in case of
    // overloads, if you are not concerned about objects with overloaded
    // function call operators, it could be further simplified to:
    //
    //   template <class T> using Members = folly::PolyMembers<&T::operator()>;
    //
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(
        FOLLY_POLY_MEMBER(R(As...) const, &T::operator()));
  };

  // Now that we have defined the interface, we can pass it to Poly to
  // create our type-erasing wrapper:
  template <class Fun>
  using Function = Poly<IFunction<Fun>>;

Given the above definition of `Function`, users can now initialize instances
of (say) `Function<int(int, int)>` with function objects like
`std::plus<int>` and `std::multiplies<int>`, as below:

  lang=c++
  Function<int(int, int)> fun = std::plus<int>{};
  assert(5 == fun(2, 3));
  fun = std::multiplies<int>{};
  assert(6 = fun(2, 3));

== Defining an interface with C++17

With C++17, defining an interface to be used with `Poly` is fairly
straightforward. As in the `Function` example above, there is a struct with
a nested `Interface` class template and a nested `Members` alias template.
No macros are needed with C++17.

Imagine we were defining something like a Java-style iterator. If we are
using a C++17 compiler, our interface would look something like this:

  lang=c++
  template <class Value>
  struct IJavaIterator {
    template <class Base>
    struct Interface : Base {
      bool Done() const { return folly::poly_call<0>(*this); }
      Value Current() const { return folly::poly_call<1>(*this); }
      void Next() { folly::poly_call<2>(*this); }
    };
    // NOTE: This works in C++17 only:
    template <class T>
    using Members = folly::PolyMembers<&T::Done, &T::Current, &T::Next>;
  };

  template <class Value>
  using JavaIterator = Poly<IJavaIterator>;

Given the above definition, `JavaIterator<int>` can be used to hold instances
of any type that has `Done`, `Current`, and `Next` member functions with the
correct (or compatible) signatures.

The presence of overloaded member functions complicates this picture. Often,
property members are faked in C++ with `const` and non-`const` member
function overloads, like in the interface specified below:

  lang=c++
  struct IIntProperty {
    template <class Base>
    struct Interface : Base {
      int Value() const { return folly::poly_call<0>(*this); }
      void Value(int i) { folly::poly_call<1>(*this, i); }
    };
    // NOTE: This works in C++17 only:
    template <class T>
    using Members = folly::PolyMembers<
      folly::sig<int() const>(&T::Value),
      folly::sig<void(int)>(&T::Value)>;
  };

  using IntProperty = Poly<IIntProperty>;

Now, any object that has `Value` members of compatible signatures can be
assigned to instances of `IntProperty` object. Note how `folly::sig` is used
to disambiguate the overloads of `&T::Value`.

== Defining an interface with C++14

In C++14, the nice syntax above doesn't work, so we have to resort to macros.
The two examples above would look like this:

  lang=c++
  template <class Value>
  struct IJavaIterator {
    template <class Base>
    struct Interface : Base {
      bool Done() const { return folly::poly_call<0>(*this); }
      Value Current() const { return folly::poly_call<1>(*this); }
      void Next() { folly::poly_call<2>(*this); }
    };
    // NOTE: This works in C++14 and C++17:
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(&T::Done, &T::Current, &T::Next);
  };

  template <class Value>
  using JavaIterator = Poly<IJavaIterator>;

and

  lang=c++
  struct IIntProperty {
    template <class Base>
    struct Interface : Base {
      int Value() const { return folly::poly_call<0>(*this); }
      void Value(int i) { return folly::poly_call<1>(*this, i); }
    };
    // NOTE: This works in C++14 and C++17:
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(
      FOLLY_POLY_MEMBER(int() const, &T::Value),
      FOLLY_POLY_MEMBER(void(int), &T::Value));
  };

  using IntProperty = Poly<IIntProperty>;

== Extending interfaces

One typical advantage of inheritance-based solutions to runtime polymorphism
is that one polymorphic interface could extend another through inheritance.
The same can be accomplished with type-erasing polymorphic wrappers. In
the `Poly` library, you can use `folly::PolyExtends` to say that one interface
extends another.

  lang=c++
  struct IFoo {
    template <class Base>
    struct Interface : Base {
      void Foo() const { return folly::poly_call<0>(*this); }
    };
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(&T::Foo);
  };

  // The IFooBar interface extends the IFoo interface
  struct IFooBar : PolyExtends<IFoo> {
    template <class Base>
    struct Interface : Base {
      void Bar() const { return folly::poly_call<0>(*this); }
    };
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(&T::Bar);
  };

  using FooBar = Poly<IFooBar>;

Given the above definition, instances of type `FooBar` have both `Foo()` and
`Bar()` member functions.

The sensible conversions exist between a wrapped derived type and a wrapped
base type. For instance, assuming `IDerived` extends `IBase` with `Extends`:

  lang=c++
  Poly<IDerived> derived = ...;
  Poly<IBase> base = derived; // This conversion is OK.

As you would expect, there is no conversion in the other direction, and at
present there is no `Poly` equivalent to `dynamic_cast`.

== Type-erasing polymorphic reference wrappers

Sometimes you don't need to own a copy of an object; a reference will do. For
that you can use `Poly` to capture a //reference// to an object satisfying an
interface rather than the whole object itself. The syntax is intuitive.

  lang=c++
  int i = 42;

  // Capture a mutable reference to an object of any IRegular type:
  Poly<IRegular &> intRef = i;

  assert(42 == folly::poly_cast<int>(intRef));
  // Assert that we captured the address of "i":
  assert(&i == &folly::poly_cast<int>(intRef));

A reference-like `Poly` has a different interface than a value-like `Poly`.
Rather than calling member functions with the `obj.fun()` syntax, you would
use the `obj->fun()` syntax. This is for the sake of `const`-correctness.
For example, consider the code below:

  lang=c++
  struct IFoo {
    template <class Base>
    struct Interface {
      void Foo() { folly::poly_call<0>(*this); }
    };
    template <class T>
    using Members = folly::PolyMembers<&T::Foo>;
  };

  struct SomeFoo {
    void Foo() { std::printf("SomeFoo::Foo\n"); }
  };

  SomeFoo foo;
  Poly<IFoo &> const anyFoo = foo;
  anyFoo->Foo(); // prints "SomeFoo::Foo"

Notice in the above code that the `Foo` member function is non-`const`.
Notice also that the `anyFoo` object is `const`. However, since it has
captured a non-`const` reference to the `foo` object, it should still be
possible to dispatch to the non-`const` `Foo` member function. When
instantiated with a reference type, `Poly` has an overloaded `operator->`
member that returns a pointer to the `IFoo` interface with the correct
`const`-ness, which makes this work.

The same mechanism also prevents users from calling non-`const` member
functions on `Poly` objects that have captured `const` references, which
would violate `const`-correctness.

Sensible conversions exist between non-reference and reference `Poly`s. For
instance:

  lang=c++
  Poly<IRegular> value = 42;
  Poly<IRegular &> mutable_ref = value;
  Poly<IRegular const &> const_ref = mutable_ref;

  assert(&poly_cast<int>(value) == &poly_cast<int>(mutable_ref));
  assert(&poly_cast<int>(value) == &poly_cast<int>(const_ref));

== Non-member functions (C++17)

If you wanted to write the interface `ILogicallyNegatable`, which captures
all types that can be negated with unary `operator!`, you could do it
as we've shown above, by binding `&T::operator!` in the nested `Members`
alias template, but that has the problem that it won't work for types that
have defined unary `operator!` as a free function. To handle this case,
the `Poly` library lets you use a free function instead of a member function
when creating a binding.

With C++17 you may use a lambda to create a binding, as shown in the example
below:

  lang=c++
  struct ILogicallyNegatable {
    template <class Base>
    struct Interface : Base {
      bool operator!() const { return folly::poly_call<0>(*this); }
    };
    template <class T>
    using Members = folly::PolyMembers<
      +[](T const& t) -> decltype(!t) { return !t; }>;
  };

This requires some explanation. The unary `operator+` in front of the lambda
is necessary! It causes the lambda to decay to a C-style function pointer,
which is one of the types that `folly::PolyMembers` accepts. The `decltype` in
the lambda return type is also necessary. Through the magic of SFINAE, it
will cause `Poly<ILogicallyNegatable>` to reject any types that don't support
unary `operator!`.

If you are using a free function to create a binding, the first parameter is
implicitly the `this` parameter. It will receive the type-erased object.

== Non-member functions (C++14)

If you are using a C++14 compiler, the definition of `ILogicallyNegatable`
above will fail because lambdas are not `constexpr`. We can get the same
effect by writing the lambda as a named free function, as show below:

  lang=c++
  struct ILogicallyNegatable {
    template <class Base>
    struct Interface : Base {
      bool operator!() const { return folly::poly_call<0>(*this); }
    };
    template <class T>
    static auto negate(T const& t) -> decltype(!t) { return !t; }
    template <class T>
    using Members = FOLLY_POLY_MEMBERS(&negate<T>);
  };

As with the example that uses the lambda in the preceding section, the first
parameter is implicitly the `this` parameter. It will receive the type-erased
object.

== Multi-dispatch

What if you want to create an `IAddable` interface for things that can be
added? Adding requires //two// objects, both of which are type-erased. This
interface requires dispatching on both objects, doing the addition only
if the types are the same. For this we make use of the `Self` template
alias to define an interface that takes more than one object of the the
erased type.

  lang=c++
  struct IAddable {
    template <class Base>
    struct Interface : Base {
      friend Self<Base>
      operator+(Self<Base> const& a, Self<Base> const& b) const {
        return folly::poly_call<0>(a, b);
      }
    };
    template <class T>
    using Members = folly::PolyMembers<
      +[](T const& a, T const& b) -> decltype(a + b) { return a + b; }>;
  };

Given the above defintion of `IAddable` we would be able to do the following:

  lang=c++
  Poly<IAddable> a = 2, b = 3;
  Poly<IAddable> c = a + b;
  assert(poly_cast<int>(c) == 5);

If `a` and `b` stored objects of different types, a `BadPolyCast` exception
would be thrown.

== Move-only types

If you want to store move-only types, then your interface should extend the
`IMoveOnly` interface.

== Implementation notes

`Poly` will store "small" objects in an internal buffer, avoiding the cost of
of dynamic allocations. At present, this size is not configurable; it is
pegged at the size of two `double`s.

`Poly` objects are always nothrow movable. If you store an object in one that
has a potentially throwing move contructor, the object will be stored on the
heap, even if it could fit in the internal storage of the `Poly` object.
(So be sure to give your objects nothrow move constructors!)

`Poly` implements type-erasure in a manner very similar to how the compiler
accomplishes virtual dispatch. Every `Poly` object contains a pointer to a
table of function pointers. Member function calls involve a double-
indirection: once through the v-pointer, and other indirect function call
through the function pointer.

Reviewed By: yfeldblum

Differential Revision: D4897112

fbshipit-source-id: ff1c1156316bfbdd8f2205c4f57932c0067cacac

4 years agoRemove a few memory allocations in ThreadWheelTimekeeper.after()
Alexander Pronchenkov [Mon, 30 Oct 2017 12:13:02 +0000 (05:13 -0700)]
Remove a few memory allocations in ThreadWheelTimekeeper.after()

Summary:
This diff reduces number of memory allocation in folly::ThreadWheelTimekeeper.after() method for a bit.

 * std::shared_ptr(new T) is replaced with std::make_shared<T>()
 * folly::Promise is stored by value

Reviewed By: yfeldblum

Differential Revision: D6172017

fbshipit-source-id: 41bf123f10570c76d64eaac1800b7e65fe381110

4 years agoOverride TemporaryFile's default move constructor v2017.10.30.00
Murali Vilayannur [Mon, 30 Oct 2017 05:07:46 +0000 (22:07 -0700)]
Override TemporaryFile's default move constructor

Summary:
A default move constructor/move assignment operator caused a bunch
of bugs in my test that was depending on the move constructor to either
not close the underlying file descriptor or to ensure that the TemporaryFile
object doesn't have a dangling reference to a file descriptor that has already
been closed. The current implementation caused both the moved' from and to
object to share the same underlying file descriptor and when the former object
is destroyed it ends up closing the file descriptor leaving the latter
with a dangling file descriptor which could be reused for some other
socket(s)/file descriptor by the kernel and ends up causing seg-faults
when the latter object is eventually destroyed due to it releasing
a file descriptor that now belongs to some other resource.

I changed the move constructor/move assignment operator
to have the former semantics to not close the underlying file descriptor of
the move'd from object (by releasing it and assigning it to the move'd to
object). I am not sure if anyone would ever want the alternative
semantics because the TemporaryFile object would not be usable
without a valid underlying file descriptor

Reviewed By: yfeldblum

Differential Revision: D6182075

fbshipit-source-id: bc809d704449c1c1182d76cdfa2f7d17b38a719a

4 years agoCodeMod: Replace includes of folly/Hash.h with folly/hash/Hash.h
Yedidya Feldblum [Sun, 29 Oct 2017 20:55:29 +0000 (13:55 -0700)]
CodeMod: Replace includes of folly/Hash.h with folly/hash/Hash.h

Summary: CodeMod: Replace includes of `folly/Hash.h` with `folly/hash/Hash.h`.

Reviewed By: luciang

Differential Revision: D6156195

fbshipit-source-id: 0941b3c9cf1d17d7cc62595111e506c06ee51236

4 years agoMake SysBufferDeleter::operator() inlineable
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:15 +0000 (03:33 -0700)]
Make SysBufferDeleter::operator() inlineable

Summary: [Folly] Make `SysBufferDeleter::operator()` inlineable, specifically, into `SysBufferUniquePtr`.

Reviewed By: luciang

Differential Revision: D6182999

fbshipit-source-id: e0409c0019f21ed44d7d4c531ebc11a239f25831

4 years agoMove folly/Launder.h
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:14 +0000 (03:33 -0700)]
Move folly/Launder.h

Summary: [Folly] Move `folly/Launder.h` to `folly/lang/`.

Reviewed By: luciang

Differential Revision: D6182882

fbshipit-source-id: 97e46bd4e4d6f212d8234209ee90a41e7850ecb9

4 years agoMove folly/Array.h
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:10 +0000 (03:33 -0700)]
Move folly/Array.h

Summary: [Folly] Move `folly/Array.h` to `folly/container/`.

Reviewed By: luciang

Differential Revision: D6182858

fbshipit-source-id: 59340b96058cc6d0c7a0289e316bbde98c15d724

4 years agoMove folly/Assume.h
Yedidya Feldblum [Sun, 29 Oct 2017 04:19:52 +0000 (21:19 -0700)]
Move folly/Assume.h

Summary: [Folly] Move `folly/Assume.h` to `folly/lang/`.

Reviewed By: luciang, ot

Differential Revision: D6181983

fbshipit-source-id: 25564bb07daa1a6765651cd919b4778efb931446

4 years agoBreak out BitIterator into its own header
Andrew Krieger [Sat, 28 Oct 2017 20:47:17 +0000 (13:47 -0700)]
Break out BitIterator into its own header

Summary:
BitIterator is a standalone construct that has a heavy boost
dependency, but not common usage. Breaking it out into its own header
will help a lot of transitive dependendencies, because it is included
in Hash.h which is included in a variety of other common headers.

This reduces the number of transitively included headers by 248 (!)

Reviewed By: yfeldblum, ot, luciang

Differential Revision: D6178564

fbshipit-source-id: 1380154b012615b7b8c73bc15ab0ac62f6b990d3

4 years agoMove folly/detail/AtomicUtils.h
Yedidya Feldblum [Sat, 28 Oct 2017 18:31:23 +0000 (11:31 -0700)]
Move folly/detail/AtomicUtils.h

Summary: [Folly] Move `folly/detail/AtomicUtils.h` to `folly/synchronization/detail/`.

Reviewed By: Orvid

Differential Revision: D6180482

fbshipit-source-id: 5671c149a59eea824db2935ffabcf85a2f78b690

4 years agoLOG_EVERY_N instead of every time
Scott Michelson [Sat, 28 Oct 2017 00:33:45 +0000 (17:33 -0700)]
LOG_EVERY_N instead of every time

Summary: when queues fill, this starts blowing up, affecting server performance and making logs useless. It's useful to know queues are full, but we don't need the log every time we try to append

Reviewed By: yfeldblum

Differential Revision: D6175784

fbshipit-source-id: b4e6966087c4a6f9fba51d7f9193b9f41e13b899

4 years agoFix FunctionScheduler::resetFunctionTimer concurrency bug
Yedidya Feldblum [Fri, 27 Oct 2017 04:24:48 +0000 (21:24 -0700)]
Fix FunctionScheduler::resetFunctionTimer concurrency bug

Summary:
[Folly] Fix `FunctionScheduler::resetFunctionTimer` concurrency bug.

The original code from the blamed diff code has a comment with this correct assessment of the bug:
> TODO: This moves out of RepeatFunc object while folly:Function can potentially be executed. This might be unsafe.

Namely, when the method is invoked with the id of a scheduled function which is concurrently being executed, the function object (including, say, lambda captures) will be moved while possibly being accessed. If the function object is small enough to be placed in-situ within `folly::Function` (48 bytes on x64), then that access to a moved-from object can happen. It might or might not, depending on the particular instance of the race in question. Or, worse, the access might be to a half-moved-from object!

The new test case for `resetFunctionTimer` passes after the fix, but is guaranteed to fail before the fix because we manage to control the concurrency enough to force the bad version of the race to happen. In the test, we just capture a `std::shared_ptr` (we could have capatured, e.g., a `std::unique_ptr` or a long-enough `std::string`) and check that it is not empty - if it is moved from, it will be empty, and the test expectation will fail.

Reviewed By: simpkins

Differential Revision: D6158722

fbshipit-source-id: 33a7ae699bb3b22089fddbebb6d922737668309d

4 years agotype_t, a generalization of void_t
Yedidya Feldblum [Fri, 27 Oct 2017 02:35:23 +0000 (19:35 -0700)]
type_t, a generalization of void_t

Summary: [Folly] `type_t`, a generalization of `void_t`.

Reviewed By: ericniebler

Differential Revision: D6082913

fbshipit-source-id: f9557b5da1f6684b12d570b6c1bd52c102cb0703

4 years agoJust use a volatile static in Malloc.h
Christopher Dykes [Thu, 26 Oct 2017 19:15:55 +0000 (12:15 -0700)]
Just use a volatile static in Malloc.h

Summary: As suggested late in the initial diff (D5840883), just mark it volatile, as it works under all supported platforms.

Reviewed By: davidtgoldblatt, yfeldblum

Differential Revision: D6155241

fbshipit-source-id: 00c07a11dc7fc2e33c2d1f9a45fd28006eeff6f9

4 years agoFix build with Windows SDK v10.0.16232
Christopher Dykes [Thu, 26 Oct 2017 19:14:23 +0000 (12:14 -0700)]
Fix build with Windows SDK v10.0.16232

Summary:
It defines `MSG_ERRQUEUE` which breaks things.

There's a github PR to do this in a different way, but it's faster to just do it myself.

Closes https://github.com/facebook/folly/pull/689

Reviewed By: yfeldblum

Differential Revision: D6155606

fbshipit-source-id: f1c6b247efc452b4005ad3b6d82fabfd5a92f49f

4 years agoDisable zerocopy if we're notified about deferred copies, add a isZeroCopyWriteInProg...
Dan Melnic [Thu, 26 Oct 2017 15:37:29 +0000 (08:37 -0700)]
Disable zerocopy if we're notified about deferred copies, add a isZeroCopyWriteInProgress method, replace pair with a proper struct

Summary: Add zeroWriteDone callback

Reviewed By: djwatson

Differential Revision: D6097129

fbshipit-source-id: b82a942557680c3a7a3be8f81ee6f2886e99e165

4 years agoExpected coroutines support
Yedidya Feldblum [Thu, 26 Oct 2017 03:27:41 +0000 (20:27 -0700)]
Expected coroutines support

Summary:
[Folly] `Expected` coroutines support.

Copied from `Optional` coroutines support.

Reviewed By: ericniebler, Orvid

Differential Revision: D5923792

fbshipit-source-id: 8661012c65762a0e540a4af2fd2fc237a8cb87a1

4 years agofolly: Fix data race in folly::Codel
Kenny Yu [Thu, 26 Oct 2017 01:11:02 +0000 (18:11 -0700)]
folly: Fix data race in folly::Codel

Summary:
Data race reported by TSAN:

```
WARNING: ThreadSanitizer: data race (pid=608219)
  Read of size 1 at 0x7b5800000c29 by thread T314:
    #0 0x60b3441 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ./folly/executors/Codel.cpp:76
    #1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    #2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    #3 0x619739d in __tsan_thread_start_func crtstuff.c:?

  Previous write of size 1 at 0x7b5800000c29 by thread T315:
    #0 0x60b33e4 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ??:?
    #1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    #2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    #3 0x619739d in __tsan_thread_start_func crtstuff.c:?

  Location is heap block of size 768 at 0x7b5800000c00 allocated by main thread:
    #0 0x616ab83 in operator new(unsigned long) ??:?
    #1 0x53cb92 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >, std::allocator<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > > >, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*)
    ...
```

It looks like there are multiple threads reading and writing the `overloaded_` bool. To fix it, wrap it in a `std::atomic`.

Reviewed By: yfeldblum, meyering

Differential Revision: D6149766

fbshipit-source-id: 605b29fa2f602d2ed2dfc22e46b739ef169f914e

4 years agorework reads/writes
Luca Niccolini [Wed, 25 Oct 2017 21:28:42 +0000 (14:28 -0700)]
rework reads/writes

Summary:
the current implementation had problems with scheduling reads and writes and
it would sometimes get stuck when transfering large chunks of data

here I am restructuring the code to look more like the one in HTTPSession

session flow control is not implemented yet really, it's coming next

Depends on: D6048238

Reviewed By: afrind

Differential Revision: D6048238

fbshipit-source-id: ae601e771154a7f1a669a58a6e05c9e3720e7017

4 years agoSupply an explicit default dtor impl
Marko Novakovic [Wed, 25 Oct 2017 17:15:40 +0000 (10:15 -0700)]
Supply an explicit default dtor impl

Reviewed By: yfeldblum

Differential Revision: D6142252

fbshipit-source-id: 9ac98585a92299ca5915982c65c7d2cfa68bf60f

4 years agoFlesh out Optional members swap, reset, emplace, has_value
Yedidya Feldblum [Wed, 25 Oct 2017 05:05:23 +0000 (22:05 -0700)]
Flesh out Optional members swap, reset, emplace, has_value

Summary:
[Folly] Flesh out `Optional` members `swap`, `reset`, `emplace`, `has_value`.

* `swap` as a member and deriving `noexcept`-ness to mimic `std::optional::swap`.
* `reset` v.s. `clear` to mimic `std::optional::reset`.
* `emplace` returning ref and overload taking initializer list to mimic `std::optional::emplace`.
* `has_value` v.s. `hasValue` to mimic `std::optional::has_value`.

Reviewed By: WillerZ

Differential Revision: D6132775

fbshipit-source-id: 34c58367b9dc63289e4b9721c5e79b1c41ba31e4

4 years agoSimplify IsUniquePtrToSL in IOBuf.h
Yedidya Feldblum [Wed, 25 Oct 2017 00:16:58 +0000 (17:16 -0700)]
Simplify IsUniquePtrToSL in IOBuf.h

Summary: [Folly] Simplify `IsUniquePtrToSL` in `IOBuf.h`.

Reviewed By: Orvid

Differential Revision: D6131231

fbshipit-source-id: b054ef7ef9f313943a3ac1022ca6a23874a464df

4 years agoMove folly/Hash.h to folly/hash/, leaving a shim
Yedidya Feldblum [Tue, 24 Oct 2017 21:12:03 +0000 (14:12 -0700)]
Move folly/Hash.h to folly/hash/, leaving a shim

Summary: [Folly] Move `folly/Hash.h` to `folly/hash/`, leaving a shim.

Reviewed By: Orvid

Differential Revision: D6132955

fbshipit-source-id: dc789e9c6daa28116be6a5d83c3cfbb40e247114

4 years agofix build with asan and static linking
Igor Sugak [Tue, 24 Oct 2017 17:55:58 +0000 (10:55 -0700)]
fix build with asan and static linking

Reviewed By: meyering

Differential Revision: D6117783

fbshipit-source-id: 048b056e119bf89ab88c33b1233297d197e8acb9

4 years agoPrefer bool literals rather than integers in boolean contexts
Christopher Dykes [Tue, 24 Oct 2017 02:26:29 +0000 (19:26 -0700)]
Prefer bool literals rather than integers in boolean contexts

Summary: Via clang-tidy's modernize-use-bool-literals

Reviewed By: yfeldblum

Differential Revision: D6130384

fbshipit-source-id: 359d5195897f04612c9b9042cf69383050a2ec7a

4 years agoConsistently have the namespace closing comment
Christopher Dykes [Tue, 24 Oct 2017 00:34:54 +0000 (17:34 -0700)]
Consistently have the namespace closing comment

Summary: It's done in a lot of places, but not all. clang-tidy gives us llvm-namespace-comment, which can force this consistently.

Reviewed By: yfeldblum

Differential Revision: D6108129

fbshipit-source-id: 1b44c5a26250364f9edf70f84173e9ba6389f06c

4 years agoModernize uses of std::make_shared
Christopher Dykes [Mon, 23 Oct 2017 23:52:19 +0000 (16:52 -0700)]
Modernize uses of std::make_shared

Summary: Via the modernize-make-shared from clang-tidy

Reviewed By: yfeldblum

Differential Revision: D6129464

fbshipit-source-id: 04f560c6beeb2b8631b819fd4e6a2d51b37eeb4b

4 years agoAdd a full drain for folly's ManualExecutor.
Lee Howes [Mon, 23 Oct 2017 20:36:48 +0000 (13:36 -0700)]
Add a full drain for folly's ManualExecutor.

Summary: ManualExecutor::run() is stable, which means that in cases where we want to fully drain the executor we have to loop over it. This adds ManualExecutor::drain() which does that internally.

Reviewed By: yfeldblum

Differential Revision: D6126840

fbshipit-source-id: e36cba5c373a57fe01de244977ec852636b58dbd

4 years agomove Iterator, Enumerate, EvictingCacheMap, Foreach, Merge, and
James Sedgwick [Mon, 23 Oct 2017 19:19:27 +0000 (12:19 -0700)]
move Iterator, Enumerate, EvictingCacheMap, Foreach, Merge, and

Summary: this is all non-hphp includes that are going in container/

Reviewed By: mzlee, yfeldblum

Differential Revision: D6121745

fbshipit-source-id: b024bde8835fc7f332686793d75eb8e71591c912

4 years agoMemoryIdler: use mallctl directly for tcache.flush
Qi Wang [Mon, 23 Oct 2017 19:09:40 +0000 (12:09 -0700)]
MemoryIdler: use mallctl directly for tcache.flush

Summary:
tcache.flush may fail if tcache is disabled.  Avoid using mallctlCall
which throws on error.

Reviewed By: davidtgoldblatt

Differential Revision: D6115419

fbshipit-source-id: 39411c80af08dc7c855efd43297809b749f935bf

4 years agoModernize use of std::make_unique
Christopher Dykes [Mon, 23 Oct 2017 18:49:39 +0000 (11:49 -0700)]
Modernize use of std::make_unique

Summary: Via the clang-tidy check modernize-make-unique.

Reviewed By: yfeldblum

Differential Revision: D6107790

fbshipit-source-id: 1cf186feae511cbd91f44893059737a85778b6cf

4 years agoRename unique_lock variables in `TimedMutex` in order to avoid shadowing
Jon Maltiel Swenson [Mon, 23 Oct 2017 17:16:53 +0000 (10:16 -0700)]
Rename unique_lock variables in `TimedMutex` in order to avoid shadowing

Summary: Rename `std::unique_lock` variables named `lock` to `ulock` in `folly::fibers::TimedMutex` member functions in order to avoid shadowing.

Reviewed By: andreazevedo

Differential Revision: D6123449

fbshipit-source-id: 5fa331bb1541ac995d9b69360ee09923c14f6698

4 years agoGate std::invoke_result et. al. to appropriate MSVC versions.
Andrew Krieger [Mon, 23 Oct 2017 16:36:28 +0000 (09:36 -0700)]
Gate std::invoke_result et. al. to appropriate MSVC versions.

Summary: Only available in >= 2017 15.3, which is 1911+.

Reviewed By: aary, Orvid

Differential Revision: D6117237

fbshipit-source-id: 255804af5bfd0c743fd225b8a4fddf3cfc9cfeaf

4 years agomove Arena, ThreadCachedArena, and Malloc to memory/
James Sedgwick [Mon, 23 Oct 2017 14:27:04 +0000 (07:27 -0700)]
move Arena, ThreadCachedArena, and Malloc to memory/

Summary: all memory/-destined components besides hphp includes

Reviewed By: yfeldblum, mzlee

Differential Revision: D6121822

fbshipit-source-id: 6c6214d84dcdefe4789ed5200399ae27203d6340

4 years agomove futures/test/ExecutorTest.cpp to executors/ v2017.10.23.00
James Sedgwick [Sun, 22 Oct 2017 20:18:56 +0000 (13:18 -0700)]
move futures/test/ExecutorTest.cpp to executors/

Summary: also gotta split it up/rename it, that's coming later

Reviewed By: yfeldblum

Differential Revision: D6121525

fbshipit-source-id: 9c6dbabd47323d94657508a0f75a0c6e7f988ace

4 years agomove InlineExecutor, ManualExecutor, and GlobalThreadPoolList to
James Sedgwick [Sat, 21 Oct 2017 22:28:58 +0000 (15:28 -0700)]
move InlineExecutor, ManualExecutor, and GlobalThreadPoolList to

Summary:
That's everything that's going in executors/ except for Executor.h
itself, which is included in hphp so will have to wait

Reviewed By: mzlee

Differential Revision: D6100274

fbshipit-source-id: 6be37892b1ad7f46828acfa6b2951e51b157a86a

4 years agomove MemoryMapping, Shell, ThreadId, ThreadName, and VersionCheck to system/
James Sedgwick [Sat, 21 Oct 2017 20:00:55 +0000 (13:00 -0700)]
move MemoryMapping, Shell, ThreadId, ThreadName, and VersionCheck to system/

Summary:
Everything that's going in system/ besides CpuId and Subprocess,
which are included in hphp

Reviewed By: mzlee

Differential Revision: D6102263

fbshipit-source-id: 564ef584c341a4ac79db14a9d58fe23ce51e78b3

4 years agomove io/Compression and io/compression/* to compression/
James Sedgwick [Sat, 21 Oct 2017 17:25:08 +0000 (10:25 -0700)]
move io/Compression and io/compression/* to compression/

Summary:
as above

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D6099826

fbshipit-source-id: 20152487135aa8eaf6d2e99369801b6dde4992aa

4 years agoUse folly/portability/GTest.h in folly/executurs/test/
Yedidya Feldblum [Sat, 21 Oct 2017 05:25:03 +0000 (22:25 -0700)]
Use folly/portability/GTest.h in folly/executurs/test/

Summary:
[Folly] Use `folly/portability/GTest.h` in `folly/executurs/test/`.

Applying the general rule for folly tests to some recently-introduced violations.

Reviewed By: Orvid

Differential Revision: D6113707

fbshipit-source-id: f9a2961ac845489b85b5a539595e4c82df03d922

4 years agomove executor task queues and thread factories into subdirectories
James Sedgwick [Fri, 20 Oct 2017 20:30:03 +0000 (13:30 -0700)]
move executor task queues and thread factories into subdirectories

Summary:
as title, see moves

(Note: this ignores all push blocking failures!)

Reviewed By: mzlee

Differential Revision: D6112001

fbshipit-source-id: 1eb10b44ae8ee1f90a10e05c29e48c99d824afa5

4 years agoDrop redundant void parameters from function declarations
Christopher Dykes [Fri, 20 Oct 2017 17:29:30 +0000 (10:29 -0700)]
Drop redundant void parameters from function declarations

Summary:
This is C++, not C, so let's keep things modern.
This is applying the modernize-redundant-void-arg clang-tidy check to Folly.

Reviewed By: yfeldblum

Differential Revision: D6106921

fbshipit-source-id: 06a56f036f59426df5dd475bf91a3f4a335266f5

4 years agoMatch commented argument names with actual names
Christopher Dykes [Fri, 20 Oct 2017 05:48:29 +0000 (22:48 -0700)]
Match commented argument names with actual names

Summary: Via the clang-tidy check misc-argument-comment.

Reviewed By: yfeldblum

Differential Revision: D6107482

fbshipit-source-id: a1fe6215c31fae472ad3b6e05abea974d706794e

4 years agoUse nullptr rather than 0 for a null pointer
Christopher Dykes [Thu, 19 Oct 2017 23:13:58 +0000 (16:13 -0700)]
Use nullptr rather than 0 for a null pointer

Summary: This time aided by clang-tidy's modernize-use-nullptr check.

Reviewed By: yfeldblum

Differential Revision: D6102739

fbshipit-source-id: aeb4bd0a8078d81cc88b766e0a034a37dd25fd1f

4 years agomove CallOnce to synchronization/
James Sedgwick [Thu, 19 Oct 2017 21:08:04 +0000 (14:08 -0700)]
move CallOnce to synchronization/

Summary: as above

Reviewed By: knekritz

Differential Revision: D6088687

fbshipit-source-id: 0efbb7f5fa33b5f553c0c2019658370fc6e8613f

4 years agoHandle timekeeperSingleton being nullptr in within()
Alex Yarmula [Thu, 19 Oct 2017 20:35:27 +0000 (13:35 -0700)]
Handle timekeeperSingleton being nullptr in within()

Summary: When timekeeper singleton no longer exists during shutdown and folly::Singleton::try_get() can return nullptr, make sure nullptr is handled gracefully.

Reviewed By: yfeldblum

Differential Revision: D6101311

fbshipit-source-id: fefeddfbd048d1a7632688bb3526db15b685dd72

4 years agoAdd InlineExecutor.cpp to Makefile.am
Yedidya Feldblum [Thu, 19 Oct 2017 19:33:58 +0000 (12:33 -0700)]
Add InlineExecutor.cpp to Makefile.am

Summary: [Folly] Add `InlineExecutor.cpp` to `Makefile.am`.

Reviewed By: Orvid

Differential Revision: D6101859

fbshipit-source-id: 4cd0f875dedb548189722fd719f4424d485e1b02

4 years agoFix fibers build on older boost
Yedidya Feldblum [Thu, 19 Oct 2017 18:37:54 +0000 (11:37 -0700)]
Fix fibers build on older boost

Summary:
[Folly] Fix `folly/fibers/` build on older boost.

Where `jump_fcontext` takes `intptr_t` rather than an actual pointer, and we warn about implicit conversions from pointers to integral types.

Reviewed By: Orvid

Differential Revision: D6098230

fbshipit-source-id: 61bd7bdff5de728d3febe8b35c97d024ab6f236a

4 years agoEnsure curly-braces around control-flow
Christopher Dykes [Thu, 19 Oct 2017 18:04:53 +0000 (11:04 -0700)]
Ensure curly-braces around control-flow

Summary: The style guidelines say control flow should always have curly braces, and we follow that, mostly. This just uses clang-tidy to clean up everywhere that we weren't.

Reviewed By: markisaa, luciang

Differential Revision: D6097377

fbshipit-source-id: bfe6766c37bd863ecf68851ef93265a200d4259d

4 years agomove ApplyTuple to functional/
James Sedgwick [Thu, 19 Oct 2017 14:45:51 +0000 (07:45 -0700)]
move ApplyTuple to functional/

Summary: as above

Reviewed By: yfeldblum

Differential Revision: D6086563

fbshipit-source-id: ab7f50ba46ebd1dbef6438f956258b2fbb13cb5c

4 years agoAdd ProducerConsumerQueue::capacity()
Peter Alexander [Thu, 19 Oct 2017 09:46:56 +0000 (02:46 -0700)]
Add ProducerConsumerQueue::capacity()

Summary: Simple addition. Easy to track externally, but might as well provide it in the class if it is readily available.

Reviewed By: yfeldblum

Differential Revision: D6093826

fbshipit-source-id: 9d8c02891b2cea9ce0d3f6ea78e1e0055b536eb8

4 years agoRevert D6050464: [Folly] Move folly/Hash.h to folly/hash/
Yedidya Feldblum [Thu, 19 Oct 2017 09:30:31 +0000 (02:30 -0700)]
Revert D6050464: [Folly] Move folly/Hash.h to folly/hash/

Summary:
This reverts commit 64eb65aac8e3e7cd0126e65ca3998bfe167e2d73

bypass-lint

Differential Revision: D6050464

fbshipit-source-id: 1ed63f30837dc11ae57b316f1f7cb233a210894a

4 years agoAdd window overload that takes an executor to prevent stack overflow
Walker Mills [Thu, 19 Oct 2017 07:57:03 +0000 (00:57 -0700)]
Add window overload that takes an executor to prevent stack overflow

Summary:
AIUI, if there is no executor available, then callbacks are executed inline. `folly::window` uses a recursive helper function (`spawn`) to handle chaining callbacks. So if `window` is used on a large enough collection of `Future`s without executors (e.g., created by `makeFuture`, or have otherwise already completed), you get a stack overflow. A minimal repro looks like:
```

int main(int argc, char** argv) {
  std::vector<int> v(100000);
  for(int i=0; i < v.size(); i++) {
    v[i] = i;
  }

  std::vector<folly::Future<folly::Unit>> f =
      folly::window(
          std::move(v),
          [](int /* unused */) { return folly::makeFuture(); },
          1);
  folly::collectAll(f).get();
}
```

This diff resolves the issue by adding an overload of `folly::window` which takes an executor as its first parameter. The executor-less `window` overload calls through to the new function using an `InlineExecutor` as the default executor.

Reviewed By: yfeldblum

Differential Revision: D6038733

fbshipit-source-id: 5dcab575592650efa2e106f12632ec06817a0009

4 years agoMove folly/Hash.h to folly/hash/
Yedidya Feldblum [Thu, 19 Oct 2017 07:37:48 +0000 (00:37 -0700)]
Move folly/Hash.h to folly/hash/

Summary: [Folly] Move `folly/Hash.h` to `folly/hash/`.

Reviewed By: jsedgwick

Differential Revision: D6050464

fbshipit-source-id: 64eb65aac8e3e7cd0126e65ca3998bfe167e2d73

4 years agoEnable bug workaround also for MSVC 2017.4
Arkady Shapkin [Thu, 19 Oct 2017 02:35:00 +0000 (19:35 -0700)]
Enable bug workaround also for MSVC 2017.4

Summary:
Workaround a bug in template instantiation in MSVC 2017 U3/4 with /permissive-

https://developercommunity.visualstudio.com/content/problem/81223/incorrect-error-c5037-with-permissive.html

/cc Orvid
Closes https://github.com/facebook/folly/pull/691

Reviewed By: yfeldblum

Differential Revision: D6085652

Pulled By: Orvid

fbshipit-source-id: f068dc3f8e474163815fc497b05410fe76834a52

4 years agoFix incorrect example usage in stop_watch
Boliu Xu [Thu, 19 Oct 2017 02:08:13 +0000 (19:08 -0700)]
Fix incorrect example usage in stop_watch

Summary: As titled.

Reviewed By: yfeldblum

Differential Revision: D6087099

fbshipit-source-id: 1c346fa6f65b5ea58e728759905560ae5ad9cc9d

4 years agoMove small fields to avoid padding in AsyncSocket.h and AsyncSSLSocket.h
Richard Fillman [Thu, 19 Oct 2017 00:19:59 +0000 (17:19 -0700)]
Move small fields to avoid padding in AsyncSocket.h and AsyncSSLSocket.h

Summary: Compiled mcrouter with the 'Wpadded' flag to try to find any wasted space. Mcrouter itself did not have anything, but there was a lot in AsyncSocket.h and AsyncSSLSocket.h so looked into those and removed a bit of wasted space

Reviewed By: yfeldblum, brianwatling

Differential Revision: D6033977

fbshipit-source-id: 2dc127208e09980be6a5db576b45d30ac6e044ff

4 years agomove futures/QueuedImmediateExecutor to executors/QueuedImmediateExecutor
James Sedgwick [Wed, 18 Oct 2017 23:32:35 +0000 (16:32 -0700)]
move futures/QueuedImmediateExecutor to executors/QueuedImmediateExecutor

Summary: as above

Reviewed By: yfeldblum

Differential Revision: D6076779

fbshipit-source-id: 4c223ab9fce3be8544f6f807781c3d0a99b61dad

4 years agoWorkaround MSVC bug with referencing template constructors before definition
Andrew Krieger [Wed, 18 Oct 2017 22:44:01 +0000 (15:44 -0700)]
Workaround MSVC bug with referencing template constructors before definition

Summary:
MSVC has a bug where it is unable to match an out-of-line constructor (and
possibly other kind of member) to the definition if it is implicitly referenced
before the definition is processed. The guilty method is
SemiFuture<Unit> makeSemiFuture(). Moving it after the constructor definitions
resolves the issue.

Reviewed By: yfeldblum

Differential Revision: D6042277

fbshipit-source-id: 97fe97c0edf3df3d9e3b808968b450c73959b600

4 years agomove futures/ScheduledExecutor to executors/ScheduledExecutor
James Sedgwick [Wed, 18 Oct 2017 22:01:33 +0000 (15:01 -0700)]
move futures/ScheduledExecutor to executors/ScheduledExecutor

Summary: see title

Reviewed By: yfeldblum

Differential Revision: D6062601

fbshipit-source-id: edd9a5e85f4ebecd1a6f1004a4d3b8b43b935c2b

4 years agoMove folly/MallctlHelper.h to folly/memory/
Yedidya Feldblum [Wed, 18 Oct 2017 21:18:47 +0000 (14:18 -0700)]
Move folly/MallctlHelper.h to folly/memory/

Summary: [Folly] Move `folly/MallctlHelper.h` to `folly/memory/`.

Reviewed By: aary

Differential Revision: D6087216

fbshipit-source-id: 4e0fa4aea976e2578127d3c340e0e9b559a224ca

4 years agomove Partial to functional/
James Sedgwick [Wed, 18 Oct 2017 18:49:02 +0000 (11:49 -0700)]
move Partial to functional/

Summary: as above

Reviewed By: ngoyal

Differential Revision: D6087941

fbshipit-source-id: 948ff4f2faa87dd34f87d14ea01c83335f850a27

4 years agoUse nullptr rather than 0 when initializing pointers
Christopher Dykes [Wed, 18 Oct 2017 16:49:45 +0000 (09:49 -0700)]
Use nullptr rather than 0 when initializing pointers

Summary:
Because we do this in a few places, and `nullptr` makes it far clearer what the intention is.
Note that with `-Wzero-as-null-pointer-constant` under GCC, doing `std::function<void()> f = {}` initializes `f` with a `0` rather than `nullptr`, triggering the warning, so I've enabled it there as well.
It is not currently possible to actually enable `-Wzero-as-null-pointer-constant`, because GCC 5 reports conversions resulting from a default value as occuring at the call-site rather than at the location where the parameter is defined, and the default allocator in libstdc++ is not clean for this particular warning -_-...

Reviewed By: yfeldblum

Differential Revision: D6046746

fbshipit-source-id: 6135bb20a503c861838575cf973324d74d75ca69

4 years agomove futures/DrivableExecutor to executors/DrivableExecutor
James Sedgwick [Wed, 18 Oct 2017 16:44:09 +0000 (09:44 -0700)]
move futures/DrivableExecutor to executors/DrivableExecutor

Summary: as title

Reviewed By: yfeldblum

Differential Revision: D6062437

fbshipit-source-id: 4f99e779e280fdb0b1f035013caff18764e86ab5