folly.git
4 years agoAdd SO_ZEROCOPY support v2017.10.09.00
Dan Melnic [Mon, 9 Oct 2017 01:27:54 +0000 (18:27 -0700)]
Add SO_ZEROCOPY support

Summary: Add SO_ZEROCOPY support

Reviewed By: djwatson

Differential Revision: D5851637

fbshipit-source-id: 5378b7e44ce9d888ae08527506218998974d4309

4 years agoLet EventBase::runInEventBaseThreadAndWait consume its argument
Yedidya Feldblum [Sat, 7 Oct 2017 01:04:56 +0000 (18:04 -0700)]
Let EventBase::runInEventBaseThreadAndWait consume its argument

Summary:
[Folly] Let `EventBase::runInEventBaseThreadAndWait` consume its argument.

Likewise `EventBase::runImmediatelyOrRunInEventBaseThreadAndWait`.

And enforce that the function is destructed before returning, so that, in the case of a wrapped lambda, all captured objects' destructors run before returning from the function.

Reviewed By: elsteveogrande

Differential Revision: D5994106

fbshipit-source-id: 816c9431a85a3d41e4fda321065614f4c18f0697

4 years agofuture.then() is identical to future.unit()
Stiopa Koltsov [Fri, 6 Oct 2017 20:07:02 +0000 (13:07 -0700)]
future.then() is identical to future.unit()

Summary:
Document that `future.then()` is identical to `future.unit()`.

Put functions next to each other to make it clear, and use identical
wording.

Reviewed By: yfeldblum

Differential Revision: D5955451

fbshipit-source-id: 1b55d5785dc0995d4d9364f48241e98ad01b31f4

4 years agoRemove Executor::addPtr
Yedidya Feldblum [Fri, 6 Oct 2017 18:45:18 +0000 (11:45 -0700)]
Remove Executor::addPtr

Summary:
[Folly] Remove `Executor::addPtr`.

It was there to support passing non-copyable callable objects wrapped as `std::shared_ptr`. It is no longer useful since we have `Function`, which is a non-copyable version of `std::function` which can support capturing non-copyable objects, and it is generally unused.

Reviewed By: spacedentist

Differential Revision: D5983801

fbshipit-source-id: b49a86f8dd7e5250a097b0e714a1bdf9ac362916

4 years agoWorkaround for a gcc crash
Jun Qu [Fri, 6 Oct 2017 16:54:59 +0000 (09:54 -0700)]
Workaround for a gcc crash

Summary: I recently got a gcc crash when using `ConcurrentHashMap`

Reviewed By: djwatson

Differential Revision: D5994587

fbshipit-source-id: 412d48aa6da12c428bb9ab5ed7a317d89c9580b9

4 years agoadd static makeFromPath to construct SocketAddress from a unix domain socket path
Valeriy Khromov [Fri, 6 Oct 2017 11:38:06 +0000 (04:38 -0700)]
add static makeFromPath to construct SocketAddress from a unix domain socket path

Summary:
Add `SocketAddress::makeFromPath(StringPiece)` static member function to
constructor `SocketAddress` from a path to Unix domain socket.

Reviewed By: yfeldblum

Differential Revision: D5974523

fbshipit-source-id: b5c1537e67d07d1ef401fea75e35753392eeaf6b

4 years agoUse fewer keep-alives in VirtualEventBase::keepAliveRelease
Yedidya Feldblum [Fri, 6 Oct 2017 00:02:00 +0000 (17:02 -0700)]
Use fewer keep-alives in VirtualEventBase::keepAliveRelease

Summary:
[Folly] Use fewer keep-alives in `VirtualEventBase::keepAliveRelease`.

{D5982132} mistakenly used `VirtualEventBase::add`, which holds a keep-alive on the master `EventBase`, instead of calling `EventBase::add` directly.

(Note: this ignores all push blocking failures!)

Reviewed By: andriigrynenko

Differential Revision: D5992403

fbshipit-source-id: f427025c06f01e6a93d6eae8a051c3553f58be20

4 years agoMake keepAliveRelease consistent between EventBase and VirtualEventBase
Yedidya Feldblum [Thu, 5 Oct 2017 22:52:28 +0000 (15:52 -0700)]
Make keepAliveRelease consistent between EventBase and VirtualEventBase

Summary: [Folly] Make `keepAliveRelease` consistent between `EventBase` and `VirtualEventBase`.

Reviewed By: elsteveogrande

Differential Revision: D5982132

fbshipit-source-id: 536d48e5672567e78786691bfb283c34d1f31960

4 years agoMove GlobalThreadPoolList.h into folly/concurrency/
Yedidya Feldblum [Thu, 5 Oct 2017 22:40:50 +0000 (15:40 -0700)]
Move GlobalThreadPoolList.h into folly/concurrency/

Summary:
[Folly] Move `GlobalThreadPoolList.h` into `folly/concurrency/`.

This seems like a better place than the top-level. It seems reasonable because this has *something* to do with concurrency. This is used by executors, but is not itself an executor, so it was not clear that it should go to `folly/executors/`, although that also seems like quite the reasonable choice.

Reviewed By: Orvid

Differential Revision: D5985142

fbshipit-source-id: 6117d133757af74023d6cffb838741da83d173bd

4 years agoAdd an overload to use backlashify with StringPiece
Andre Pinto [Thu, 5 Oct 2017 15:34:49 +0000 (08:34 -0700)]
Add an overload to use backlashify with StringPiece

Summary: Allows calling backlashify on a StringPiece.

Reviewed By: yfeldblum

Differential Revision: D5930015

fbshipit-source-id: ca14c78d9a90c45781da7229eb5d2f437792b2d1

4 years agoAdd SemiFuture class.
Lee Howes [Thu, 5 Oct 2017 02:59:14 +0000 (19:59 -0700)]
Add SemiFuture class.

Summary:
Offer a clean separation between future as a vocabulary type for crossing
library interfaces, and future as a type to manage continuations.

The principle is that if we want an API with clean separation of execution it
should both return and receive SemiFutures. The returned SemiFuture would
only be awaitable, but not continuable. If the caller wants to enqueue a
continuation then it is efficiently convertable into a folly::Future by
calling .via.

This means that an API a) Does not have to take an executor to ensure it
returns a valid future. That can be deferred to the caller or the caller's
caller/target for true separation. b) The API can ensure that its own executor
is not exposed to the caller, while still having a clean API.

Obviously if some API wants to allow returning of a continuable future, then it
can also provide an executor-taking API.

Reviewed By: yfeldblum

Differential Revision: D5769284

fbshipit-source-id: 46241d1274bf7b1698a7d28a47cff2a65a983740

4 years agogive folly future's exceptions default visibility
Eric Niebler [Wed, 4 Oct 2017 21:56:25 +0000 (14:56 -0700)]
give folly future's exceptions default visibility

Summary: Exception types should have default visibility, otherwise RTTI doesn't work right.

Reviewed By: elsteveogrande

Differential Revision: D5976415

fbshipit-source-id: 45dcfe3476b513aa49a6f78352318f31d381ada7

4 years agoMove keepalive-acquire code into Executor::keepAliveAcquire overrides
Yedidya Feldblum [Wed, 4 Oct 2017 18:56:32 +0000 (11:56 -0700)]
Move keepalive-acquire code into Executor::keepAliveAcquire overrides

Summary:
[Folly] Move keepalive-acquire code into `Executor::keepAliveAcquire` overrides.

This makes the API more symmetric and allows for more flexibility. Also serves as preparation for generalizing the keepalive-token interface.

Reviewed By: andriigrynenko

Differential Revision: D5974080

fbshipit-source-id: 26209e49a0f5834ba229d4bbfc9272c2e4ffb3fd

4 years agoBenchmark comparison
Tom Jackson [Wed, 4 Oct 2017 17:31:05 +0000 (10:31 -0700)]
Benchmark comparison

Summary:
I couldn't find anything that consumes `--json` output, so I made this. It prints out benchmark results from two runs, comparing each benchmark to its corresponding baseline. This should help with benchmarking changes across revisions.

Along the way:
- Two small transparent `struct`s replaces the use of `tuple`s everywhere.
- New output JSON format (additive), formatted as array of arrays so order is preserved.
- New comparison binary in `folly/tools/benchmark_compare`

Sample output:

{P58307694}

Reviewed By: yfeldblum

Differential Revision: D5908017

fbshipit-source-id: d7411e22b459db16bd897f656e48ea4e896cb1bf

4 years agoLet keep-alive tokens from VirtualEventBase be destroyed from any thread
Yedidya Feldblum [Wed, 4 Oct 2017 06:50:27 +0000 (23:50 -0700)]
Let keep-alive tokens from VirtualEventBase be destroyed from any thread

Summary:
[Folly] Let keep-alive tokens from `VirtualEventBase` be destroyed from any thread.

This case was missed in {D5951397} (facebook/folly@6f3b7616f3402355441c62ca60a36159435aa818).

Reviewed By: andriigrynenko

Differential Revision: D5970670

fbshipit-source-id: 19e2b00134516e68113234acd6111e21c3e23e8d

4 years agoDynamic MPMCQueue: Backout of last change as it may deadlock
Maged Michael [Tue, 3 Oct 2017 19:19:29 +0000 (12:19 -0700)]
Dynamic MPMCQueue: Backout of last change as it may deadlock

Summary: The previous change can lead to deadlock. Backing out.

Reviewed By: djwatson

Differential Revision: D5957084

fbshipit-source-id: 72ea1cb6236367912b4b087da7e4d57f8a2daed0

4 years agoFixes: prevent compiler reporting UB, hazptr_array move operator, empty array test
Maged Michael [Tue, 3 Oct 2017 14:56:57 +0000 (07:56 -0700)]
Fixes: prevent compiler reporting UB, hazptr_array move operator, empty array test

Summary:
Three fixes:
(1) The compiler reports UB in line 432 for the case M > HAZPTR_TC_SIZE even though it is executed only if M <= HAZPTR_TC_SIZE. Added a condition M <= HAZPTR_TC_SIZE to help the compiler determine that line 432 is not executed in that case.
(2) Added missing management of empty state in hazptr_array move operator
(3) Added nullptr argument to empty hazptr_array in Array test

Reviewed By: djwatson

Differential Revision: D5951283

fbshipit-source-id: cb8e61421ab06c7733f67bf2d2274d3311260ac4

4 years agoLet keep-alive tokens be destroyed from any thread
Yedidya Feldblum [Tue, 3 Oct 2017 02:45:55 +0000 (19:45 -0700)]
Let keep-alive tokens be destroyed from any thread

Summary: [Folly] Let keep-alive tokens be destroyed from any thread.

Reviewed By: andriigrynenko

Differential Revision: D5951397

fbshipit-source-id: 91e72588de4ef33a730ebef5770e77635d4e93ba

4 years agofix -Wvexing-parse
Igor Sugak [Mon, 2 Oct 2017 20:42:09 +0000 (13:42 -0700)]
fix -Wvexing-parse

Summary:
```lang=cpp
folly/test/stl_tests/StlVectorTest.cpp:1975:11: error: parentheses were disambiguated as redundant parentheses around declaration of variable named 'm' [-Werror,-Wvexing-parse]
    Vector(m);
          ^~~
folly/test/stl_tests/StlVectorTest.cpp:1975:11: note: add a variable name to declare a 'Vector' initialized with 'm'
    Vector(m);
          ^
           varname
folly/test/stl_tests/StlVectorTest.cpp:1975:5: note: add enclosing parentheses to perform a function-style cast
    Vector(m);
    ^
    (        )
folly/test/stl_tests/StlVectorTest.cpp:1975:11: note: remove parentheses to silence this warning
    Vector(m);
          ^ ~

1 error generated.
```

Reviewed By: Gownta

Differential Revision: D5955613

fbshipit-source-id: 3015570ce111603b3561914d07b55365a221c686

4 years agoMove Merge.h into folly/algorithm/ v2017.10.02.00
Yedidya Feldblum [Mon, 2 Oct 2017 05:11:27 +0000 (22:11 -0700)]
Move Merge.h into folly/algorithm/

Summary: [Folly] Move `Merge.h` into `folly/algorithm/`.

Reviewed By: Orvid

Differential Revision: D5951346

fbshipit-source-id: 487203e55e2a44888e599ca22849798154e5b386

4 years agoOptimize local and bulk management of hazptr_holder-s
Maged Michael [Sun, 1 Oct 2017 01:11:41 +0000 (18:11 -0700)]
Optimize local and bulk management of hazptr_holder-s

Summary:
Changes:
- Added hazptr_local<M> for optimized management of local hazptr_holder-s.
- Added hazptr_array<M> for optimized management of hazptr_holder-s
- Added benchmarks for hazptr_local and hazptr_array
- Added tests for hazptr_local and hazptr_array
- Changed SWMRList example to use hazptr_local<2> instead of two hazptr_holder-s.
- Updated benchmark performance results.

Reviewed By: davidtgoldblatt

Differential Revision: D5833721

fbshipit-source-id: 154811f67c38abac7342cecb71f829778ccf76b2

4 years agoShort-Circuit within() When Future Is Already Complete
Felix Handte [Fri, 29 Sep 2017 19:41:24 +0000 (12:41 -0700)]
Short-Circuit within() When Future Is Already Complete

Summary:
As title. No sense adding a timeout to the timeout manager when the future is
already complete.

Reviewed By: yfeldblum

Differential Revision: D5933144

fbshipit-source-id: 4d1bbd866c47ccee6bd0518cbe063afc1d34cbca

4 years agodelete deprecated SSLContext::setSSLLockTypes
Igor Sugak [Fri, 29 Sep 2017 06:35:24 +0000 (23:35 -0700)]
delete deprecated SSLContext::setSSLLockTypes

Summary: This is deprecated and not used.

Reviewed By: yfeldblum

Differential Revision: D5936286

fbshipit-source-id: 68f757be06ebe375e14db008d0e5828fad0c3aa5

4 years agoAdd AsyncSocketExceptionType for early data rejection.
Kyle Nekritz [Wed, 27 Sep 2017 20:11:31 +0000 (13:11 -0700)]
Add AsyncSocketExceptionType for early data rejection.

Summary: To be used to signal that data that was sent as TLS 1.3 early data was lost. This needs its own exception type as it requires special handling (ie it should usually be retried immediately).

Reviewed By: siyengar

Differential Revision: D5869914

fbshipit-source-id: ca0d8ef19cb991e9d6ffe0f8c99abfb03b748ce6

4 years agoFix unittest compile error on futures/test/RetryingTest.cpp.
Mohamed Amin JABRI [Wed, 27 Sep 2017 17:38:18 +0000 (10:38 -0700)]
Fix unittest compile error on futures/test/RetryingTest.cpp.

Summary:
On MacOS compiling unittests with "make chek" shows the following compile
error:
```
../futures/test/RetryingTest.cpp:168:26: error: no matching function for call to 'min'
 newMemLimit.rlim_cur = std::min(1UL << 30, oldMemLimit.rlim_max);
```
Closes https://github.com/facebook/folly/pull/680

Reviewed By: elsteveogrande

Differential Revision: D5921433

Pulled By: yfeldblum

fbshipit-source-id: 236d8336f852750a983c2e268db8811d1a4ed9ee

4 years agoDelete conversion from Objective-C block to folly::Function
Michael Lee [Tue, 26 Sep 2017 15:35:09 +0000 (08:35 -0700)]
Delete conversion from Objective-C block to folly::Function

Summary: Objective-C blocks are stack allocated, and unless there is a proper assignment it isn't retained and the memory is freed. Because folly::Function used to move, it would hold a reference, but after switch to a constructor by-value, it no longer does this and we see a use-after-free.

Reviewed By: yfeldblum, ericniebler

Differential Revision: D5888606

fbshipit-source-id: fe4cabb2f2ae289cce0e7429e0af3935ba314720

4 years agoLet ProcessReturnCode be publicly constructible v2017.09.25.00
Yedidya Feldblum [Sun, 24 Sep 2017 06:12:57 +0000 (23:12 -0700)]
Let ProcessReturnCode be publicly constructible

Summary:
[Folly] Let `ProcessReturnCode` be publicly constructible.

Via provided constructor functions, which limit how it may be constructed so that it is only constructible into a valid state.

Differential Revision: D5898739

fbshipit-source-id: 7490018adfc39408b4290248ef1220e8fd0238cb

4 years agoAvoid tautological compare in folly/experimental/symbolizer/
Yedidya Feldblum [Sat, 23 Sep 2017 04:19:34 +0000 (21:19 -0700)]
Avoid tautological compare in folly/experimental/symbolizer/

Summary:
[Folly] Avoid tautological compare in `folly/experimental/symbolizer/`.

`x < 0` when `x` is unsigned is tautological and there are warnings against such comparisons. The underlying type of an unscoped enumerations without a fixed underlying type is not specified, and whether it is signed is also not specified; it could be unsigned, and is unsigned in practice in common cases.

Reviewed By: Orvid, eduardo-elizondo

Differential Revision: D5897792

fbshipit-source-id: 24d84f9bf2c61c907585e1b675c2bbf11ef1720b

4 years agoFix deadlock in TimedMutex
Matthew Tolton [Sat, 23 Sep 2017 00:32:21 +0000 (17:32 -0700)]
Fix deadlock in TimedMutex

Summary:
If a lock is stolen from fiber f1, and then fiber f2 is notified before f1 one
wakes up and discovers the crime, then f1 will clear notifiedFiber_ so that f2
thinks the lock was stolen from it as well and hence recurses back into lock().

Reviewed By: andriigrynenko

Differential Revision: D5896323

fbshipit-source-id: 528ec1ed983175d3e08f3dc07b69bbc783a86cfb

4 years agofix ASAN stl_tests:stl_vector_test
Nathan Bronson [Fri, 22 Sep 2017 19:01:30 +0000 (12:01 -0700)]
fix ASAN stl_tests:stl_vector_test

Summary:
folly/test/stl_tests/StlVectorTest.cpp was too slow under
mode/dev-asan, resulting in timeouts, and the relinquish test allocated
with malloc and freed with operator delete.  This diff reduces the vector
size for testing under ASAN, and fixes the allocation mismatch.

Reviewed By: Orvid

Differential Revision: D5890969

fbshipit-source-id: 49a9498f6c0c4b3c7165906efd1262e518fea810

4 years agoUse unique_ptr rather than shared_ptr in MockReadCallback.
Kyle Nekritz [Thu, 21 Sep 2017 20:28:03 +0000 (13:28 -0700)]
Use unique_ptr rather than shared_ptr in MockReadCallback.

Summary: This is more convenient to use (for example with an IOBufEqual matcher).

Reviewed By: yfeldblum

Differential Revision: D5882029

fbshipit-source-id: 6aa12f80479f40bcc2af64dc270fb0a9382983b5

4 years agoAdd gdb deadlock detector script to new folly/experimental/gdb directory
Kenny Yu [Wed, 20 Sep 2017 03:31:08 +0000 (20:31 -0700)]
Add gdb deadlock detector script to new folly/experimental/gdb directory

Summary:
This adds a gdb deadlock detector script into a new directory in folly. I
chose to put it under the `experimental` directory and not the top-level
directory as we have only tested these scripts on x86_64 Linux and not other
types of platforms.

This diff includes:
- a README on the contents of this directory and how to use the scripts
- a script to detect deadlocks

gdb directory
---------------

This new directory will contain a collection of gdb scripts that we have
found helpful. These scripts use the [gdb extension Python API](https://sourceware.org/gdb/current/onlinedocs/gdb/Python.html#Python).

To run the scripts, fire up gdb and load a script with `source -v`. Example:

```
$ gdb -p 123456
(gdb) source -v ./folly/experimental/gdb/deadlock.py
Type "deadlock" to detect deadlocks.
(gdb) deadlock
Found deadlock!
...
```

deadlock detector script
----------------------------

Consider the following program that always deadlocks:

```
void deadlock3() {
  std::mutex m1, m2, m3;
  folly::Baton<> b1, b2, b3;

  auto t1 = std::thread([&m1, &m2, &b1, &b2] {
    std::lock_guard<std::mutex> g1(m1);
    b1.post();
    b2.wait();
    std::lock_guard<std::mutex> g2(m2);
  });

  auto t2 = std::thread([&m3, &m2, &b3, &b2] {
    std::lock_guard<std::mutex> g2(m2);
    b2.post();
    b3.wait();
    std::lock_guard<std::mutex> g3(m3);
  });

  auto t3 = std::thread([&m3, &m1, &b3, &b1] {
    std::lock_guard<std::mutex> g3(m3);
    b3.post();
    b1.wait();
    std::lock_guard<std::mutex> g1(m1);
  });

  t1.join();
  t2.join();
  t3.join();
}
```

Once the process has deadlocked, we can use gdb to find the threads and mutexes involved in the deadlock:

```
$ gdb -p 2174496
(gdb) source -v ./folly/experimental/gdb/deadlock.py
Type "deadlock" to detect deadlocks.
(gdb) deadlock
Found deadlock!
Thread 2 (LWP 2174497) is waiting on mutex (0x00007ffcff42a4c0) held by Thread 3 (LWP 2174498)
Thread 3 (LWP 2174498) is waiting on mutex (0x00007ffcff42a4f0) held by Thread 4 (LWP 2174499)
Thread 4 (LWP 2174499) is waiting on mutex (0x00007ffcff42a490) held by Thread 2 (LWP 2174497)
```

Reviewed By: yfeldblum

Differential Revision: D5860868

fbshipit-source-id: 020a32327a79bb066269fe08113695803ce06c7d

4 years agoBenchmark a realistic common use
Phil Willoughby [Tue, 19 Sep 2017 20:57:36 +0000 (13:57 -0700)]
Benchmark a realistic common use

Summary:
We didn't previously benchmark the performance of `get()`, which was a strange
omission.

Reviewed By: akrieger

Differential Revision: D5863567

fbshipit-source-id: 468b249da9120fcb84f3303ac5e2157761b6369d

4 years agoEnable -Wunused-variables
Christopher Dykes [Tue, 19 Sep 2017 05:54:48 +0000 (22:54 -0700)]
Enable -Wunused-variables

Summary:
Only for clang for the moment, as more work is needed for GCC.
MSVC builds have always had unused variable warnings enabled, I just hadn't gotten around to cleaning up all of the tests for it to be able to enable it.

Reviewed By: yfeldblum

Differential Revision: D5827840

fbshipit-source-id: ab503b5791fcc58d685b8327179b810880c5dea7

4 years agoFix hardware crc32 under MSVC
Christopher Dykes [Tue, 19 Sep 2017 02:03:12 +0000 (19:03 -0700)]
Fix hardware crc32 under MSVC

Summary: It was horribly broken in multiple ways. It didn't support input > 4gb on Windows, was truncating the entire table to chars, and simply couldn't be compiled because it was exceeding MSVC's max macro instantion depth of 255.

Reviewed By: yfeldblum

Differential Revision: D5827332

fbshipit-source-id: b08268c88db10c607b27231689b3ee3ec7553c1f

4 years agoSet names in NamedThreadFactory in the new thread
Christopher Dykes [Tue, 19 Sep 2017 02:02:08 +0000 (19:02 -0700)]
Set names in NamedThreadFactory in the new thread

Summary: This makes it now both work, and compile, on Windows.

Reviewed By: yfeldblum

Differential Revision: D5811100

fbshipit-source-id: 5d6bfc04ed8e60417615da15bd197769e0c79c11

4 years agofix strange recursive std::is_constructible instantiation involving the Function...
Eric Niebler [Mon, 18 Sep 2017 22:21:34 +0000 (15:21 -0700)]
fix strange recursive std::is_constructible instantiation involving the Function converting constructor

Summary: In rare and obscure cases, the fact that `folly::Function`'s perfectly-forwarding, implicit converting constructor was SFINAE-ing on the argument's constructibility was causing a recursive template instantiation of std::is_constructible. Switch to passing the argument by value to avoid the need to test for constructibility.

Reviewed By: yfeldblum

Differential Revision: D5847034

fbshipit-source-id: ce2ad1d2490206c6cae84c17544bd9fcd6ff47bb

4 years agoFix coroutine feature test macro on MSVC v2017.09.18.00
Eric Niebler [Fri, 15 Sep 2017 21:21:45 +0000 (14:21 -0700)]
Fix coroutine feature test macro on MSVC

Summary: Coroutines are only available on MSVC when the /await compiler switch was passed, in which case the `_RESUMABLE_FUNCTIONS_SUPPORTED` macro is defined.

Reviewed By: Orvid

Differential Revision: D5837303

fbshipit-source-id: af3349646e8875e3eac1bc3bbf47203f41f0bbde

4 years agoAllow awaiting on a folly::Optional that returns a move-only type
Dylan Yudaken [Fri, 15 Sep 2017 17:13:16 +0000 (10:13 -0700)]
Allow awaiting on a folly::Optional that returns a move-only type

Summary:
await_resume is only called once, so this allows it to move the value out.
At the same time remove a redundant clear (but keep the existing requirement that the promise type is an OptionalPromise), and clean up the tests.
Also add a test to make sure the coroutine is cleaned up

Reviewed By: ericniebler

Differential Revision: D5834861

fbshipit-source-id: 7ad487e818969cdf6fe27c9e82931aa247daf4e4

4 years agoAllow getAutoUncompressionCodec() to have 1 terminal decoder
Stella Lau [Fri, 15 Sep 2017 17:13:08 +0000 (10:13 -0700)]
Allow getAutoUncompressionCodec() to have 1 terminal decoder

Summary: getAutoUncompressionCodec() currently only allows unambiguous headers. Allow a single "terminal codec" to be called if all other codecs can't uncompress or throw.

Reviewed By: terrelln

Differential Revision: D5804833

fbshipit-source-id: 057cb6e13a48fea20508d5c028234afddf7435f6

4 years agofix usingJEMalloc() with LTO
Pádraig Brady [Fri, 15 Sep 2017 07:56:25 +0000 (00:56 -0700)]
fix usingJEMalloc() with LTO

Summary:
LTO has better optimization which results in the malloc() being optimized away in this function.

Note we don't use folly::dontOptimizeAway() because
that introduces a circular dependency between the
malloc and benchmark targets.

Reviewed By: marksantaniello, meyering, interwq

Differential Revision: D5840883

fbshipit-source-id: 5dd9b152f70b7a49f125b6d79b4dfa40f4cdac59

4 years agoNoHeap small_vector's no longer require a move constructor.
Martin Martin [Thu, 14 Sep 2017 20:04:20 +0000 (13:04 -0700)]
NoHeap small_vector's no longer require a move constructor.

Summary:
NoHeap small_vector's no longer require a move constructor.

Also changes const -> constexpr in a bunch of places.

Reviewed By: nbronson

Differential Revision: D5804361

fbshipit-source-id: 0f00fee6d318fa9296612409805d4ffcfbadb974

4 years agoWindow, mainly for futures
Tom Jackson [Thu, 14 Sep 2017 02:01:02 +0000 (19:01 -0700)]
Window, mainly for futures

Summary: Just a circular buffer in the middle of a pipeline.

Reviewed By: yfeldblum

Differential Revision: D5791551

fbshipit-source-id: 2808a53df9b8cd2a402da0678a6b4ed0e6ca6e00

4 years agoWorkaround a bug in template instation in MSVC 2017.3 with /permissive-
Christopher Dykes [Thu, 14 Sep 2017 01:59:20 +0000 (18:59 -0700)]
Workaround a bug in template instation in MSVC 2017.3 with /permissive-

Summary:
As described in the comment in `folly/stats/Histogram.h`, MSVC 2017.3 has issues with `/permissive-`.
The bug has been fixed upstream, it's just waiting for release.

These explicit template instantiations are never forward-declared anywhere, so, as long as `Histogram-defs.h` has been included, it's perfectly safe to just not do them.

https://developercommunity.visualstudio.com/content/problem/81223/incorrect-error-c5037-with-permissive.html

Reviewed By: yfeldblum

Differential Revision: D5828891

fbshipit-source-id: 4db8407c9d35aa5bb3f7b81aa24f611b5787fb6e

4 years agoFix build without zlib compression enabled
Christopher Dykes [Wed, 13 Sep 2017 21:41:27 +0000 (14:41 -0700)]
Fix build without zlib compression enabled

Summary: The namespace alias was trying to alias a namespace that doesn't exist unless Folly is being compiled with zlib support.

Reviewed By: terrelln

Differential Revision: D5810929

fbshipit-source-id: c659d8f775fc9b99e57cc95f5914418507e546fb

4 years agofixing namespacing issue for the dummy non sse crc32c_hw
Stephen Chen [Wed, 13 Sep 2017 21:19:14 +0000 (14:19 -0700)]
fixing namespacing issue for the dummy non sse crc32c_hw

Summary:
Because the way the #if was written, the dummy crc32c_hw function accidentally
ended in folly namespace instead of folly::details as expected. Fix this.

Reviewed By: yfeldblum

Differential Revision: D5826187

fbshipit-source-id: ba0f2207a00bc21eda7a727f3f1025e1106d55f9

4 years agoFix UB from signed integer overflow in RWSpinLock
Marcus Holland-Moritz [Wed, 13 Sep 2017 08:39:44 +0000 (01:39 -0700)]
Fix UB from signed integer overflow in RWSpinLock

Summary:
The signed counters used in a couple of places can in theory (and practice)
overflow, yielding UB. Use unsigned counters instead.

Reviewed By: luciang, philippv

Differential Revision: D5818606

fbshipit-source-id: c5928b779e76b309b00b7f6308a220e2bf6cbf79

4 years agoBacked out changeset ea9041ce2280
Jon Maltiel Swenson [Wed, 13 Sep 2017 04:30:08 +0000 (21:30 -0700)]
Backed out changeset ea9041ce2280

Summary: Revert D5787837. Breaks `onSet()`/`onUnset()` behavior.

Reviewed By: palmtenor

Differential Revision: D5817063

fbshipit-source-id: c7dea636fa60eb616d4ebe0a9d418bc96b3018ae

4 years agoInclude glog/logging.h in SmallLocksTest for DCHECK
Michael Lee [Tue, 12 Sep 2017 23:05:54 +0000 (16:05 -0700)]
Include glog/logging.h in SmallLocksTest for DCHECK

Summary: The test uses DCHECK, but does not include glog/logging.h.

Reviewed By: yfeldblum

Differential Revision: D5818647

fbshipit-source-id: fcb2630ac2b12acd1a7b84e228349b2887e976cd

4 years agoUpdate String.h
Alberto Schiabel [Tue, 12 Sep 2017 21:47:03 +0000 (14:47 -0700)]
Update String.h

Summary:
Fixed typo in doc comment related to `void toLowerAscii(char* str, size_t length)`.
This fixes the annoying xCode's warning `Parameter 'len' not found in the function declaration`.
Closes https://github.com/facebook/folly/pull/670

Reviewed By: yfeldblum

Differential Revision: D5815034

Pulled By: Orvid

fbshipit-source-id: 9580cd0bbd4924d4ed461a5435f525316e5c285a

4 years agoDon't use std::void_t
Christopher Dykes [Tue, 12 Sep 2017 20:54:49 +0000 (13:54 -0700)]
Don't use std::void_t

Summary: I was wrong, MSVC's implementation has the same bug as everything else.

Reviewed By: yfeldblum

Differential Revision: D5810646

fbshipit-source-id: 9caabbbc2115f57b7e836bb85c9b108588c94ad9

4 years agoconst -> constexpr in folly::small_vector
Martin Martin [Tue, 12 Sep 2017 18:51:07 +0000 (11:51 -0700)]
const -> constexpr in folly::small_vector

Summary: const -> constexpr in folly::small_vector

Reviewed By: yfeldblum

Differential Revision: D5814264

fbshipit-source-id: 4631bb7f3f04906636e5a188d4aa0d33ad796a3c

4 years agoRemove unneeded parts of build script
Philip Jameson [Tue, 12 Sep 2017 18:49:14 +0000 (11:49 -0700)]
Remove unneeded parts of build script

Summary: Remove an unneeded autoconf flag, as it causes the build to fail on missing boost-context

Reviewed By: yfeldblum

Differential Revision: D5815646

fbshipit-source-id: cb6544593de9fbf8248506b09c56412b4635b30c

4 years agoclang-format folly::small_vector. No functional change.
Martin Martin [Tue, 12 Sep 2017 18:34:55 +0000 (11:34 -0700)]
clang-format folly::small_vector. No functional change.

Summary: clang-format folly::small_vector.  No functional change.

Reviewed By: Orvid

Differential Revision: D5808357

fbshipit-source-id: d2ee831e25c60778c699214b875635c22854832d

4 years agoMake RequestContext provider overridable in order to save cost of setContext() on...
Jon Maltiel Swenson [Tue, 12 Sep 2017 16:09:42 +0000 (09:09 -0700)]
Make RequestContext provider overridable in order to save cost of setContext() on each fiber context switch

Summary:
Each fiber context switch currently involves the cost of saving/restoring `RequestContext`.  Certain
fibers-based applications such as mcrouter don't use `RequestContext` at all, so updating the current
thread-global `RequestContext` on each fiber context switch is unnecessary overhead.  Avoid this cost
by allowing `FiberManager` to override the `RequestContext` provider at the start and end of each fiber drain
loop.

Reviewed By: andriigrynenko

Differential Revision: D5787837

fbshipit-source-id: ea9041ce228063c8701165366fd1e34132868d22

4 years agoFix CMake build
Christopher Dykes [Tue, 12 Sep 2017 02:15:15 +0000 (19:15 -0700)]
Fix CMake build

Summary: Spookyhash got moved a long time ago, and a duplicate test name was added with the migration of executors from wangle to folly.

Reviewed By: yfeldblum

Differential Revision: D5811321

fbshipit-source-id: 789b4d3187a2e1e21c6cf8297f733bb4dffdbd60

4 years agoDead shift in ConcurrentHashMapSegment
Phil Willoughby [Mon, 11 Sep 2017 19:20:45 +0000 (12:20 -0700)]
Dead shift in ConcurrentHashMapSegment

Summary: Original problem detected by compiling with `-Werror,-Wunused-value`. On further inspection the only place which uses this detail class ensures that the `max_size` parameter is a power of two already, so we can discard the logic to manipulate `max_size` and put a `CHECK` clause in its place to guard against future changes to the caller that break this assumption.

Reviewed By: yfeldblum

Differential Revision: D5799110

fbshipit-source-id: d21ed9ff196d54ef91e38254df8b1b88bbf29275

4 years agoMove SSL_CTX sessionContext initialization into main SSL_CTX initialization path
Anirudh Ramachandran [Mon, 11 Sep 2017 11:54:57 +0000 (04:54 -0700)]
Move SSL_CTX sessionContext initialization into main SSL_CTX initialization path

Summary: This field was only previously set if setupSessionCache was called on the wangle::ServerSSLContext

Reviewed By: yfeldblum

Differential Revision: D5776057

fbshipit-source-id: ecdbe83816fee2ecbe7ea1b26a67b682a571309a

4 years agoAdd convenience std::string overload for toLowerAscii v2017.09.11.00
Giuseppe Ottaviano [Sun, 10 Sep 2017 16:59:56 +0000 (09:59 -0700)]
Add convenience std::string overload for toLowerAscii

Summary:
`toLowerAscii` is surprisingly hard to use with `std::string` because (until C++17) `data()` returns a `const char*`. In addition, it is not widely known that `s[0]` is legal even if the string is empty. This can lead to a variety of suboptimal code, from explicitly checking emptiness, to casting away the `const` (which causes UB).

Let's just have a simple overload for it.

Reviewed By: pixelb, philippv, yfeldblum

Differential Revision: D5801970

fbshipit-source-id: 407e2e9e66147a0662a5e09c75921255adff0702

4 years agoFix typo in async/README.md
Giuseppe Ottaviano [Sun, 10 Sep 2017 06:59:18 +0000 (23:59 -0700)]
Fix typo in async/README.md

Reviewed By: yfeldblum

Differential Revision: D5802013

fbshipit-source-id: 472b09da2be7234157a7886018004e2f4ed4f279

4 years agoOptional.h - make == operators constexprs
Benjamin Jaeger [Fri, 8 Sep 2017 20:59:20 +0000 (13:59 -0700)]
Optional.h - make == operators constexprs

Summary: This more closely matches std optional and allows folly Optional to be a drop in replacement.

Reviewed By: aary, yfeldblum

Differential Revision: D5775541

fbshipit-source-id: f754c006429fa3c5a866b6b5ffdfd8883ec2dd4f

4 years agosupport using co_await with folly::Optional when it is available
Eric Niebler [Fri, 8 Sep 2017 19:36:10 +0000 (12:36 -0700)]
support using co_await with folly::Optional when it is available

Summary:
Coroutines can be used to simulate the monadic "do" notion of Haskell. Use coroutines to enable monadic composition with folly::Optional.

```
Optional<int> f() {
  return 7;
}
Optional<int> g(int) {
  return folly::none;
}

void MonadicOptional() {
  Optional<int> r = []() -> Optional<int> {
    int x = co_await f();
    assert(x == 7);
    int y = co_await g(x);
    assert(false); // not executed
    co_return y;
  }();
  assert(!r.hasValue());
}
```

Reviewed By: yfeldblum

Differential Revision: D5763371

fbshipit-source-id: 9babc682244f38da7006d0b3a8444fd4efec1747

4 years agoGrammar fix: s/it's/its/
Michael Bolin [Fri, 8 Sep 2017 18:42:01 +0000 (11:42 -0700)]
Grammar fix: s/it's/its/

Summary:
I noticed this while reading the docs.

(Note: this ignores all push blocking failures!)

Reviewed By: lbrandy

Differential Revision: D5795548

fbshipit-source-id: 9ec5aa13a8913f217b3118e72714d02dc4a9407f

4 years agoFix data race in folly/executors/Codel.cpp exposed by TSAN
Kenny Yu [Thu, 7 Sep 2017 17:46:48 +0000 (10:46 -0700)]
Fix data race in folly/executors/Codel.cpp exposed by TSAN

Summary:
I found a data race with TSAN while attempting to run a sanitizer version of my service:

```
WARNING: ThreadSanitizer: data race (pid=266)
  Read of size 8 at 0x7b58000c0c08 by thread T370:
    @ folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) at ./folly/executors/Codel.cpp:44
    @ apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    @ apache::thrift::concurrency::PthreadThread::threadMain(void*) at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    @ __tsan_thread_start_func at crtstuff.c:?

  Previous write of size 8 at 0x7b58000c0c08 by thread T371:
    @ folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) at ./folly/executors/Codel.cpp:62
    @ apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
    @ apache::thrift::concurrency::PthreadThread::threadMain(void*) at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    @ __tsan_thread_start_func at crtstuff.c:?

  Location is heap block of size 744 at 0x7b58000c0c00 allocated by thread T314:
    @ operator new(unsigned long) at ??:?
    @ PriorityImplT at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:826
    @ void __gnu_cxx::new_allocator<apache::thrift::concurrency::PriorityThreadManager::PriorityImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > > >::construct<apache::thrift::concurrency::PriorityThreadManager::PriorityImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >, std::array<std::pair<std::shared_ptr<apache::thrift::concurrency::ThreadFactory>, unsigned long>, 5ul>&, bool&, unsigned long&>(apache::thrift::concurrency::PriorityThreadManager::PriorityImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >*, std::array<std::pair<std::shared_ptr<apache::thrift::concurrency::ThreadFactory>, unsigned long>, 5ul>&, bool&, unsigned long&)
    @ std::shared_ptr<apache::thrift::concurrency::PriorityThreadManager> apache::thrift::concurrency::PriorityThreadManager::newPriorityThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >(std::array<unsigned long, 5ul> const&, bool, unsigned long) at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:1090
    @ std::shared_ptr<apache::thrift::concurrency::PriorityThreadManager> apache::thrift::concurrency::PriorityThreadManager::newPriorityThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >(unsigned long, bool, unsigned long) at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:1100
    @ apache::thrift::ThriftServer::serve() at ./thrift/lib/cpp2/server/ThriftServer.cpp:475
    @ apache::thrift::server::TServer::run() at ./thrift/lib/cpp/server/TServer.h:186
    @ apache::thrift::concurrency::PthreadThread::threadMain(void*) at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    @ __tsan_thread_start_func at crtstuff.c:?

  Thread T370 (tid=638, running) created by thread T314 at:
    @ pthread_create at ??:?
    @ apache::thrift::concurrency::PthreadThread::start() at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:108
    @ apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::addWorker(unsigned long) at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:185
    @ apache::thrift::concurrency::PriorityThreadManager::PriorityImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::start() at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:840
    @ apache::thrift::ThriftServer::setup() at ./thrift/lib/cpp2/server/ThriftServer.cpp:347
    @ apache::thrift::ThriftServer::serve() at ./thrift/lib/cpp2/server/ThriftServer.cpp:475
    @ apache::thrift::server::TServer::run() at ./thrift/lib/cpp/server/TServer.h:186
    @ apache::thrift::concurrency::PthreadThread::threadMain(void*) at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    @ __tsan_thread_start_func at crtstuff.c:?

  Thread T371 (tid=639, running) created by thread T314 at:
    @ pthread_create at ??:?
    @ apache::thrift::concurrency::PthreadThread::start() at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:108
    @ apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::addWorker(unsigned long) at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:185
    @ apache::thrift::concurrency::PriorityThreadManager::PriorityImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::start() at ./thrift/lib/cpp/concurrency/ThreadManager.tcc:840
    @ apache::thrift::ThriftServer::setup() at ./thrift/lib/cpp2/server/ThriftServer.cpp:347
    @ apache::thrift::ThriftServer::serve() at ./thrift/lib/cpp2/server/ThriftServer.cpp:475
    @ apache::thrift::server::TServer::run() at ./thrift/lib/cpp/server/TServer.h:186
    @ apache::thrift::concurrency::PthreadThread::threadMain(void*) at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
    @ __tsan_thread_start_func at crtstuff.c:?

  Thread T314 (tid=582, running) created by main thread at:
    @ pthread_create at ??:?
    @ apache::thrift::concurrency::PthreadThread::start() at ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:108
    ...
```

Looks like there is a data race in how `codelMinDelay_` is used. I couldn't get `std::atomic` to compile with `std::chrono::nanoseconds`,
so I used `std::atomic<uint64_t>` and converted between `uint64_t` and time types appropriately.

Reviewed By: yfeldblum

Differential Revision: D5759588

fbshipit-source-id: 8213f3789808265ddfe5ab122f0f86490d0ea6ea

4 years agomove wangle/concurrent to folly/executors
James Sedgwick [Wed, 6 Sep 2017 06:09:06 +0000 (23:09 -0700)]
move wangle/concurrent to folly/executors

Summary:
For the initial cutover, just pull the copies in folly into the wangle namespace
The main problem with that approach is that forward declarations of wangle components no longer work, so I fixed those manually

ALSO, IMPORTANT: This is a great, once in a lifetime opportunity to rename/restructure these components. I have a few ideas that I'll noodle and share eventually, e.g. changing LifoSemMPMCQueue so it's just descriptive and not an enumeration of implementation details. Please chip in with yr ideas!

Reviewed By: yfeldblum

Differential Revision: D5694213

fbshipit-source-id: 4fc0ea9359d1216191676fc9729fb53a3f06339f

4 years agoCompress empty strings with underlying compressor
Stella Lau [Tue, 5 Sep 2017 22:15:13 +0000 (15:15 -0700)]
Compress empty strings with underlying compressor

Summary:
- Current behavior compresses empty strings to empty strings. This is undesirable as decompression using underlying decompressor (side-stepping the codec) will fail. This change passes empty strings to the underlying compressor
- Decompressing empty string -> empty string was kept for backwards compatibility
- Fix `getUncompressedLength` for zlib

Reviewed By: terrelln, yfeldblum

Differential Revision: D5740034

fbshipit-source-id: 5a747ea4963dad872103209aa4410197f6c605db

4 years agoFix constexpr_min after D5739715 v2017.09.04.00
Yedidya Feldblum [Sat, 2 Sep 2017 09:11:34 +0000 (02:11 -0700)]
Fix constexpr_min after D5739715

Summary:
[Folly] Fix `constexpr_min` after {D5739715} (facebook/folly@6d7c6d55f0f4b7b75607608ef9037db58083368f).

Add new unit-tests for `constexpr_min` and `constexpr_max` to avoid silly mistakes like this in the future.

It was defined recursively in terms of `constexpr_max` rather than, as intended, in terns of `constexpr_min`. HT Mandar12.

Reviewed By: Orvid

Differential Revision: D5761831

fbshipit-source-id: 2dad5833e05679232b3c529d33325a0205c2e4e4

4 years agoincrease stack size when TSAN is enabled
Igor Sugak [Fri, 1 Sep 2017 20:23:11 +0000 (13:23 -0700)]
increase stack size when TSAN is enabled

Reviewed By: meyering

Differential Revision: D5757469

fbshipit-source-id: ad70c47251993c79a502d30f312534e28c9241d4

4 years agoDeflake some AsyncSSLSocket tests.
Kyle Nekritz [Fri, 1 Sep 2017 16:13:58 +0000 (09:13 -0700)]
Deflake some AsyncSSLSocket tests.

Summary: Fulfilling starts destroying the evb in another thread which races with the socket closing on the evb.

Reviewed By: siyengar, ngoyal

Differential Revision: D5755271

fbshipit-source-id: ace37eee63e684c97ca0fe503293eee83514e0ac

4 years agoAdd a comment on File.h
Aravind Anbudurai [Thu, 31 Aug 2017 23:20:43 +0000 (16:20 -0700)]
Add a comment on File.h

Summary: title

Reviewed By: yfeldblum

Differential Revision: D5751299

fbshipit-source-id: d3967c2499da9a2c98c2862b9bdc9994c12edc76

4 years agoExtract non-portability constexpr math functions to new header
Yedidya Feldblum [Thu, 31 Aug 2017 02:26:32 +0000 (19:26 -0700)]
Extract non-portability constexpr math functions to new header

Summary:
[Folly] Extract non-portability `constexpr` math functions to new header.

This new header, `folly/ConstexprMath.h`, is specifically for compile-time-computable math functions. We start with `min`, `max`, `abs`, and `log2`.

Included substantive changes:
* Add new tests for `constexpr_strlen`, which remains in the portability header.
* Make `constexpr_min` and `constexpr_max` variadic.
* Make `constexpr_log2` tail-recursive, remove `const_log2` in `FixedString.h`, and move the related comment.

Reviewed By: Orvid

Differential Revision: D5739715

fbshipit-source-id: 29d3cc846ce98bb4bdddcc8b0fa80e4d32075fe0

4 years agoDon't use symbolizer on OSX
Philip Jameson [Thu, 31 Aug 2017 01:00:36 +0000 (18:00 -0700)]
Don't use symbolizer on OSX

Summary:
When trying to build with targets files on OSX, I couldn't use the symbolizer because it needs StackTrace, which requires libunwind and elf. This makes it so that we only build on linux for now. This also makes it so that we set FOLLY_USE_SYMBOLIZER in autoconf, since that wasn't set before.

Does a few things surrounding usage of the symbolizer library:
- Introduce FOLLY_USE_SYMBOLIZER in folly-config.h and USE_SYMBOLIZER as an AM definition
-- Filter some code out of init and some other random libs that optionally need the symbolizer
- Fix libdwarf detection. Previously on a fresh ubuntu container, we didn't find libdwarf/dwarf.h, so we stopped trying before looking at dwarf.h

Reviewed By: yfeldblum

Differential Revision: D5644352

fbshipit-source-id: f0a3580c41122e5e8fdfd17a9fdbb0921be21401

4 years agoInclude `<array>` for test
Michael Lee [Wed, 30 Aug 2017 23:16:27 +0000 (16:16 -0700)]
Include `<array>` for test

Summary: <array> is sometimes necessary for using `std::array`. Fix up

Reviewed By: aary

Differential Revision: D5740866

fbshipit-source-id: 13bafadd26fdd0f2eff3513115b43811682e7cda

4 years agoConditionally compiled for_each with constexpr
Aaryaman Sagar [Wed, 30 Aug 2017 21:36:25 +0000 (14:36 -0700)]
Conditionally compiled for_each with constexpr

Summary:
Code was breaking in versions of C++ where constexpr functions are
required to have one line bodies

Reviewed By: yfeldblum, mzlee

Differential Revision: D5739062

fbshipit-source-id: 6c509f1daf77751d33ce9c173a0d7f1d3bd2a006

4 years agoDynamic MPMCQueue: Eliminate cases of enqueue indefinite blocking and failure in...
Maged Michael [Wed, 30 Aug 2017 19:53:16 +0000 (12:53 -0700)]
Dynamic MPMCQueue: Eliminate cases of enqueue indefinite blocking and failure in the extensible version that impossible under the default pre-allocated version

Summary:
Currently under the extensible version (Dynamic == true), some enqueue operations may block indefinitely or fail (return false) even though such outcomes are impossible under the default (Dynamic == false) pre-allocated version.

This diff eliminates such cases by changing the algorithms for the extensible version. Some of the high-level changes:
- The offset formula for an expansion guarantees that no enqueue operation left behind in a closed array does not have an existing dequeue operation that unblocks it. The old formula was 1 + max(head, tail). The new formula is max(head, current offset) + current capacity.
- Conditional operations validate state after the success of CAS.

Reviewed By: djwatson

Differential Revision: D5701013

fbshipit-source-id: 4917c5b35b7e2a2fddfd2e11fb5aeb478502137c

4 years agoconstexpr_log2
Yedidya Feldblum [Wed, 30 Aug 2017 18:39:43 +0000 (11:39 -0700)]
constexpr_log2

Summary:
[Folly] `constexpr_log2`.

Useful for anything that needs to compute log2 at compile time.

Reviewed By: eduardo-elizondo

Differential Revision: D5734727

fbshipit-source-id: 8eab7991eea2104570eecd8e84ede6160bb0b549

4 years agoShrink MicroSpinLock.h transitive includes and inline methods
Yedidya Feldblum [Wed, 30 Aug 2017 03:28:59 +0000 (20:28 -0700)]
Shrink MicroSpinLock.h transitive includes and inline methods

Summary: [Folly] Shrink `MicroSpinLock.h` transitive includes and inline methods.

Reviewed By: meyering

Differential Revision: D5732693

fbshipit-source-id: 386816f0d97c145ff8a4180d41c8a682694aa6cb

4 years agorm old comment on HHWheelTimer::UniquePtr
Xiao Shi [Wed, 30 Aug 2017 03:05:38 +0000 (20:05 -0700)]
rm old comment on HHWheelTimer::UniquePtr

Summary: This comment is outdated.

Reviewed By: yfeldblum

Differential Revision: D5730049

fbshipit-source-id: f6088c4354210fafb019656cdf8246abf90fbc06

4 years agoUse weak_ptr to hold future context in timekeeper to allow clean up when future complete
Richard Meng [Tue, 29 Aug 2017 17:18:29 +0000 (10:18 -0700)]
Use weak_ptr to hold future context in timekeeper to allow clean up when future complete

Summary:
Before this change, future context will not be cleaned up until timekeeper times out. These objects has been occupying memory when more shorter future tasks are registered.
Switch to use weak ptr to hold context, so that context objects are deallocated as soon as the future completes (or times out)

Reviewed By: yfeldblum

Differential Revision: D5692040

fbshipit-source-id: b3b74a29b2ccafef6c4a06011699b069feb3a847

4 years agoAllow forcing value for FOLLY_FUTURE_USING_FIBER in folly-config.h
Andrew Krieger [Tue, 29 Aug 2017 05:36:16 +0000 (22:36 -0700)]
Allow forcing value for FOLLY_FUTURE_USING_FIBER in folly-config.h

Summary:
Let clients choose whether to use fibers or not, but default
to the same autodetected setting as prior logic.

Reviewed By: yfeldblum

Differential Revision: D5585605

fbshipit-source-id: 9a4bf9f9bc23b4e15601f30c40602f21798c9685

4 years agoFix potentially uninitialized struct used on MSVC
Andrew Krieger [Tue, 29 Aug 2017 05:34:42 +0000 (22:34 -0700)]
Fix potentially uninitialized struct used on MSVC

Summary:
MSVC warns this might be used but uninitialized. Force
the struct to be zero initialized.

Reviewed By: yfeldblum

Differential Revision: D5585320

fbshipit-source-id: 9454a2a4a66c7689f42b1eb211dc57f5d3b88fea

4 years agoFix portability opendir() behavior on paths without trailing slashes
Andrew Krieger [Tue, 29 Aug 2017 05:33:44 +0000 (22:33 -0700)]
Fix portability opendir() behavior on paths without trailing slashes

Summary:
Off-by-one error in DIR::open() would result in paths not
ending in a trailing separator to fail to open. Fix the arithmetic.

Reviewed By: Orvid

Differential Revision: D5579657

fbshipit-source-id: 79507bc398549033eb26b2ffa788d66241deb623

4 years agoImplement folly::launder for MSVC
Andrew Krieger [Tue, 29 Aug 2017 05:30:39 +0000 (22:30 -0700)]
Implement folly::launder for MSVC

Summary:
This implementation should suffice for MSVC. _ReadWriteBarrier()
generates no instructions but instructs the compiler not to reorder accesses
around it, and we have to rely on MSVC not currently having any other
optimizations for const members to worry about.

Reviewed By: yfeldblum

Differential Revision: D5723782

fbshipit-source-id: a68caa4673a5c410b326c1d2e3c0752bd945f4a6

4 years agoApply clang-format to folly/experimental/exception_tracer/
Yedidya Feldblum [Tue, 29 Aug 2017 02:59:43 +0000 (19:59 -0700)]
Apply clang-format to folly/experimental/exception_tracer/

Summary: [Folly] Apply `clang-format` to `folly/experimental/exception_tracer/`.

Reviewed By: Orvid

Differential Revision: D5722994

fbshipit-source-id: 2e52c920119ba58d123aaffd59491d7561c37c17

4 years agoRevert D5714883: [Folly] Shrink MicroSpinLock.h transitive includes and inline methods
Christopher Dykes [Tue, 29 Aug 2017 01:06:22 +0000 (18:06 -0700)]
Revert D5714883: [Folly] Shrink MicroSpinLock.h transitive includes and inline methods

Summary:
This reverts commit 1744685ff9fa8d3620aef2545c8fe3ebc481df06

bypass-lint

Differential Revision: D5714883

fbshipit-source-id: 0cdb5f2cac095a9202cb1310ff8e75e75b4afd74

4 years agoEnable auto-deps in all of Folly
Christopher Dykes [Mon, 28 Aug 2017 22:13:58 +0000 (15:13 -0700)]
Enable auto-deps in all of Folly

Summary: This enables the last of the magic internally.

Reviewed By: yfeldblum

Differential Revision: D5719795

fbshipit-source-id: fb59a0d7873e9a9f3b73f556b42b647091ca7e67

4 years agoShrink MicroSpinLock.h transitive includes and inline methods
Yedidya Feldblum [Mon, 28 Aug 2017 08:29:16 +0000 (01:29 -0700)]
Shrink MicroSpinLock.h transitive includes and inline methods

Summary: [Folly] Shrink `MicroSpinLock.h` transitive includes and inline methods.

Reviewed By: Orvid

Differential Revision: D5714883

fbshipit-source-id: 1744685ff9fa8d3620aef2545c8fe3ebc481df06

4 years agoMake cancelling and rescheduling of functions O(1) v2017.08.28.00
Patryk Zaryjewski [Mon, 28 Aug 2017 06:35:38 +0000 (23:35 -0700)]
Make cancelling and rescheduling of functions O(1)

Summary: Currently FunctionScheduler calls that cancel/restart timer for a function of particular id are O(n). By introducing hashmap that translate id to pointer of particular RepeatFunc, we make it O(1).

Reviewed By: simpkins

Differential Revision: D5668557

fbshipit-source-id: e5e8bf9bd75b6d5d42f0bfa398d476703e5801fa

4 years agogcc 6.2 was not accepting the current definition of void_t
Aaryaman Sagar [Fri, 25 Aug 2017 23:08:18 +0000 (16:08 -0700)]
gcc 6.2 was not accepting the current definition of void_t

Summary:
The current definition of void_t was leading to errors because unused
template parameters are ignored and SFINAE SFIAEs

Reviewed By: yfeldblum

Differential Revision: D5700825

fbshipit-source-id: d23336070c217e8594980d6db710cb417b014236

4 years agoAdded a flag FOLLY_FORCE_EXCEPTION_COUNT_USE_STD
Marko Novakovic [Fri, 25 Aug 2017 22:23:42 +0000 (15:23 -0700)]
Added a flag FOLLY_FORCE_EXCEPTION_COUNT_USE_STD

Summary:
The flag FOLLY_FORCE_EXCEPTION_COUNT_USE_STD forces
the usage of std:uncaught_exceptions() for folly's
implementation of uncaught_exceptions()

Reviewed By: yfeldblum

Differential Revision: D5542302

fbshipit-source-id: 2c2f507ab9dde916a160a9c370a267cdcfd7203a

4 years agoEnforce forward progress with StreamCodec
Stella Lau [Fri, 25 Aug 2017 16:23:46 +0000 (09:23 -0700)]
Enforce forward progress with StreamCodec

Summary:
- Throw exception if no forward progress was made with `StreamCodec.compress()` and `StreamCodec.uncompress()`
- Prevents infinite looping behavior when no forward progress was made
- Update tests

Reviewed By: terrelln

Differential Revision: D5685690

fbshipit-source-id: 969393896b74f51250f0e0ce3af0cd4fedcab49a

4 years agoFix typo in Function.h
Qinfan Wu [Fri, 25 Aug 2017 10:05:04 +0000 (03:05 -0700)]
Fix typo in Function.h

Summary: [Folly] Fix typo in `Function.h`.

Reviewed By: yfeldblum

Differential Revision: D5703550

fbshipit-source-id: 9dc09697fd09db6c65b1a4f2d8bdf44451b4aab2

4 years agoadd public kSlotSize to IndexedMemPool
Greg Nisbet [Fri, 25 Aug 2017 00:53:08 +0000 (17:53 -0700)]
add public kSlotSize to IndexedMemPool

Summary:
add public kSlotSize to IndexedMemPool,
needed to support getting the approximate memory footprint of the pool
(since a Slot has two atomic uint32_t's more than a bare Elem)

Reviewed By: yfeldblum

Differential Revision: D5690225

fbshipit-source-id: 667da6b67b339038b92b0e5acde17219fe1c85c5

4 years agoAdded rvalue and const rvalue overloads to Try
Aaryaman Sagar [Thu, 24 Aug 2017 21:46:17 +0000 (14:46 -0700)]
Added rvalue and const rvalue overloads to Try

Summary:
Try was missing some important-ish overloads that help it behave well
in rvalue contexts

Reviewed By: yfeldblum

Differential Revision: D5692021

fbshipit-source-id: c34627b56eb52dceaeb1f00ae930ee3bc6e00306

4 years agounwrapTryTuple only accepted rvalue tuple types
Aaryaman Sagar [Thu, 24 Aug 2017 21:42:53 +0000 (14:42 -0700)]
unwrapTryTuple only accepted rvalue tuple types

Summary:
unwrapTryTuple only accepted rvalue tuple types, this diff fixes that
to work with forwarding reference tuple types.

Also reduces the number of template instantiations

Reviewed By: yfeldblum

Differential Revision: D5682996

fbshipit-source-id: ee6dd2c20d8dfca33e769a98a6ca56fa96c73b72

4 years agoCreate generic method to extract mac address from EUI-64 constructed addresses
Andrii Kryzhyk [Thu, 24 Aug 2017 19:08:18 +0000 (12:08 -0700)]
Create generic method to extract mac address from EUI-64 constructed addresses

Summary: Link local is not the only type of address that is based on mac address, therefore create generic function to extract mac address from EUI-64 autoconfigured addresses using the same logic as for getMacAddressFromLinkLocal

Reviewed By: pallotron

Differential Revision: D5697781

fbshipit-source-id: 4d69085a1d8f08e06496b8a9b638ac7ff31c6c3a

4 years agoImprove formatting of scripts and test files
Peter DeLong [Thu, 24 Aug 2017 17:16:37 +0000 (10:16 -0700)]
Improve formatting of scripts and test files

Summary: Reformatted the scripts and test files to look better and also to avoid lint warnings in the future

Reviewed By: andriigrynenko

Differential Revision: D5673856

fbshipit-source-id: 5818625ec14d18ef3b9b804df53949afefc3b98f

4 years agoAdd LZMA streaming interface
Stella Lau [Thu, 24 Aug 2017 16:26:59 +0000 (09:26 -0700)]
Add LZMA streaming interface

Summary:
- Replace LZMA2Codec with LZMA2StreamCodec
- Update tests to reflect LZMA2_VARINT_SIZE requiring data length

Reviewed By: terrelln

Differential Revision: D5625388

fbshipit-source-id: 3303c6dda5d41f40615c87504a46923815b0b716

4 years agoTry::exception overload for Try&&
Yedidya Feldblum [Thu, 24 Aug 2017 07:42:16 +0000 (00:42 -0700)]
Try::exception overload for Try&&

Summary:
[Folly] `Try::exception` overload for `Try&&`.

For consistency with `Try::value`.

Reviewed By: WillerZ

Differential Revision: D5691758

fbshipit-source-id: 9904b2a5c90f4575a3c09dc012658d359d11fdd9

4 years agoRename FOLLY_A64 to FOLLY_AARCH64
Christopher Dykes [Thu, 24 Aug 2017 05:51:43 +0000 (22:51 -0700)]
Rename FOLLY_A64 to FOLLY_AARCH64

Summary:
`FOLLY_AARCH64` is more descriptive.
(searching for A64 tells you nothing, searching for aarch64 tells you a lot)

Reviewed By: yfeldblum, andrewjcg

Differential Revision: D5663075

fbshipit-source-id: 8f31fde4aa394f5452305929541af6d38e4d8a37