Add getVia and getTryVia to SemiFuture.

Summary: Add getVia and getTryVia to SemiFuture to allow driving chains of work conveniently in the current thread.

Reviewed By: yfeldblum

Differential Revision: D6631898

fbshipit-source-id: 324ef342a44d4ef502188b3cffde17103f0e6cb2

Move getTry to subclasses.

Summary:
Move getTry from FutureBase to Future and SemiFuture.

Make SemiFuture version move the result out for consistency with get.

Reviewed By: yfeldblum

Differential Revision: D6638561

fbshipit-source-id: 551c0f06ed52ef6d8976a5971a5e90b3ab793da0

Add continuation to semifuture return test.

Summary: Add more detail to testing return of SemiFuture from a continuation to ensure correct behaviour.

Reviewed By: yfeldblum

Differential Revision: D6637565

fbshipit-source-id: 096a7ca140e6bdef7baab6725e86d25cf79742d8

Add deprecation comments to folly::makeFuture.

Summary:
To ensure that we do not end up with continuable futures without attached executors we should deprecate folly::makeFuture. In most cases folly::makeSemiFuture is adequate here.

This diff only adds comments to dissuade future use.

Reviewed By: yfeldblum

Differential Revision: D6628800

fbshipit-source-id: c2b91df351cc5980c1bfb752f7536d320ef8168a

Add getSemiFuture to folly::SharedPromise

Summary: Adds getSemiFuture functionality to folly::SharedPromise. Implements getFuture in terms of this, using folly::InlineExecutor to ensure that there is no change of behaviour.

Reviewed By: yfeldblum

Differential Revision: D6628723

fbshipit-source-id: 0ce7c7773b9290998ce87f84fa5d82ba957f0313

Namespacing and comments in folly/Likely.h

Summary:
[Folly] Namespacing and comments in `folly/Likely.h`.

This adds `FOLLY_LIKELY` and `FOLLY_UNLIKELY`.

Reviewed By: Orvid

Differential Revision: D6636136

fbshipit-source-id: da93220201cabca91b4477ab98269a0febb735db

Tweaks to AtomicStruct

Summary: [Folly] Tweaks to `AtomicStruct`.

Reviewed By: Orvid

Differential Revision: D6636432

fbshipit-source-id: 274c118a732287219c569d7d3f4e170f275518f1

Ensure that returning a semifuture from a continuation works correctly.

Summary: Returning a SemiFuture from a continuation should work by correctly checking the types and returning a folly::Future on the same executor as the original future that .then was applied to.

Reviewed By: yfeldblum

Differential Revision: D6597273

fbshipit-source-id: cf2016a344d4b29f1d31c1da20c89df5b4cfe64e

Remove folly/detail/UncaughtExceptionCounter.h

Summary:
[Folly] Remove `folly/detail/UncaughtExceptionCounter.h`.

It's a thin and unnecessary shell around `folly/UncaughtExceptions.h`.

Reviewed By: Orvid

Differential Revision: D6636260

fbshipit-source-id: cdf6fa5fefc9fd69586c1c4c1a8443c5e8543b1c

constexpr_log2_ceil

Summary: [Folly] `constexpr_log2_ceil`, like `constexpr_log2` but rounding up.

Reviewed By: Orvid

Differential Revision: D6636433

fbshipit-source-id: a10f031cc9c91cfeba7b74bbf143895a311ca772

Move folly/detail/Sleeper.h to folly/synchronization/detail/

Summary: [Folly] Move `folly/detail/Sleeper.h` to `folly/synchronization/detail/`.

Reviewed By: Orvid

Differential Revision: D6636459

fbshipit-source-id: e0b37459fe721c96837b653e652c1bc6bfeb5dce

Fix link to window tests in folly/future/README.md

Summary:
Window tests were moved to a separate file. Update the README to the
new location.

Reviewed By: yfeldblum

Differential Revision: D6636178

fbshipit-source-id: 0313dfe80de7686e1e530ed9eb4bbcefb459633c

Format digits10

Summary: [Folly] Format `digits10`.

Reviewed By: Orvid

Differential Revision: D6636190

fbshipit-source-id: 6e6b834f6c9070d58f8e2b085b09df8b857fe878

Move folly/AtomicStruct.h to folly/synchronization/

Summary: [Folly] Move `folly/AtomicStruct.h` to `folly/synchronization/`.

Reviewed By: Orvid

Differential Revision: D6636072

fbshipit-source-id: 87098e25fc40d0d60b4a657ba395418388e170fc

Remove unused boost dep from Conv.h

Summary: [Folly] Remove unused `boost` dep from `Conv.h`.

Reviewed By: Orvid

Differential Revision: D6636040

fbshipit-source-id: 77dbfcf95f68c05346bce52d1a965e465c2efe3b

Update homebrew script to use gtest 1.8.0

Summary:
[Folly] Update homebrew script to use gtest 1.8.0, to be consistent with the docs.

Closes #722.

Reviewed By: Orvid

Differential Revision: D6633606

fbshipit-source-id: 5e9a5790100fe7a408be7a9a77c847bece91d405

Move expensive Singleton code to .cpp

Summary:
[Folly] Move expensive `Singleton` code to `.cpp`.

Including string manipulation, `ostream::operator<<` operations, `throw` statements, etc,

Reviewed By: ot

Differential Revision: D6632052

fbshipit-source-id: 82e0033d5042b02951cf3b05c55cf62c97cc2b6d

Add getSemiFuture to folly::Promise

Summary: Also deprecates folly::getFuture function.

Reviewed By: yfeldblum

Differential Revision: D6594299

fbshipit-source-id: 67435f35dca660da049cd8c505ee9a21424c0d2b

Namespace portability symbols better in Singleton.cpp

Summary: [Folly] Namespace portability symbols better in `Singleton.cpp`.

Reviewed By: Orvid

Differential Revision: D6631290

fbshipit-source-id: eb802ace6a9bf704e011eb44710922c997a9cd8e

Fix the assumption in the propagate_const test about local layout

Summary: [Folly] Fix the assumption in the `propagate_const` test about local layout.

Reviewed By: aary

Differential Revision: D6624962

fbshipit-source-id: 6871dc20f2b5ec2d1a86f520030c7cda8a225958

Returning reference to *this from propagate_const

Summary: As title

Reviewed By: yfeldblum

Differential Revision: D6620395

fbshipit-source-id: 477aae84b2bbde2e79d46ae93c285909b56f575e

Fix crashing on corrupted ELF binaries with invalid offsets in ELF header.

Summary:
There are cases where ELF binaries are running fine, but have slightly
corrupted ELF headers, e.g., with section headers offset pointing beyond boundaries
of file. I'm guessing this is due to running strip or objdump with either some
particular combination of flags or some due to buggy version of those tools. This
change prevents from crashing on such files.

Reviewed By: yfeldblum

Differential Revision: D6616027

fbshipit-source-id: 8cb3ac4475a51d1f1045c395977a6a77fbefffb2

test that the value remains alive even if the .then callback takes no arguments

Summary:
It was not clear to me, if a callback takes no arguments, the
underlying value is guaranteed to be alive during the execution of the
callback, so I wrote these tests.

Reviewed By: yfeldblum

Differential Revision: D6594921

fbshipit-source-id: 6a658afc1bf4d29eaa9c62269ddc21c7f897ad01

Kill FOLLY_ALIGNED etc

Summary:
[Folly] Kill `FOLLY_ALIGNED` etc.

`alignas` is standardized as of C++11. Let us just use that.

Replace:
* `FOLLY_ALIGNED` with `alignas`
* `FOLLY_ALIGNED_MAX` with `alignas(folly::max_align_v)`
* `FOLLY_ALIGN_TO_AVOID_FALSE_SHARING` with `alignas(folly::hardware_destructive_interference_size)`

Because where `alignas` may be placed is more restrictive than where attributes may be placed, we also need to move these directives in some cases on top of doing the replacement.

Reviewed By: Orvid

Differential Revision: D6555167

fbshipit-source-id: 4b05b570bace3f8c0fe810b6dd58781dd45757f4

Fix Build: propagate_const under GCC

Summary: [Folly] Fix Build: `propagate_const` under GCC.

Reviewed By: aary

Differential Revision: D6609722

fbshipit-source-id: 47d8b0f2d54e0e5834383ea64c93af30927c921d

Add readWithPriority to PriorityMPMCQueue

Summary: Add readWithPriority to PriorityMPMCQueue

Reviewed By: yfeldblum

Differential Revision: D6606604

fbshipit-source-id: 07382e3c6d0f53a8727c059410c0577cf0aac82c

Add hardware_destructive_interference_size

Summary:
[Folly] Add `hardware_destructive_interference_size` and `hardware_constructive_interference_size` to `folly/lang/Align.h`.

As backports from C++17. Which may require keeping, depending on how stanard libraries choose to implement these.

And replace `CacheLocality::kFalseSharingRange` with `hardware_destructive_interference_size`.

Reviewed By: ot

Differential Revision: D6554817

fbshipit-source-id: bff49f5ca8b01d38fa806076f99201355df76cd9

propagate_const

Summary: [Folly] `propagate_const`, backported from C++ Library Fundamentals TS v2.

Reviewed By: ericniebler

Differential Revision: D6589681

fbshipit-source-id: cdc8981d17938b99afe60e2baefff7deb5316612

Fix erase in Iterate (2)

Summary:
Previously D6579707.

Correctly advance to next item if we erase the current element.  Corner cases were slightly off if we were at the end of a hash chain.

Reviewed By: yfeldblum

Differential Revision: D6603518

fbshipit-source-id: acb959e5bcd5da1c3df642b75985d464fdd3b23d

Fix ThreadLocal races

Summary:
I misread the ThreadLocal docs, thread destruction functions do *not* grab the accessAllTHreads_ lock
unless you use *strict* mode, and even then, it is only a read lock.

Easy enough to make the thread-destruction global bits to be atomic / use folly::Synchronized.

Reviewed By: yfeldblum

Differential Revision: D6592905

fbshipit-source-id: 4ae600dff4c8c04751483a452ca7c07ef3f26380

Switch the Baton template params

Summary: [Folly] Switch the `Baton` template params for consistency with `SaturatingSemaphore`.

Reviewed By: davidtgoldblatt

Differential Revision: D6591060

fbshipit-source-id: 44d6243d3185d95364a27e497216cc02fb3bc2e8

Secure RNG utilities

Summary: Adding secure RNG utility functions (rand32, rand64, oneIn, randDouble01, randDouble).

Reviewed By: yfeldblum

Differential Revision: D6551975

fbshipit-source-id: 720d138de1329669b1a15eb3e9cb3fe91ce982a4

Baton::ready, a const variant of try_wait

Summary: [Folly] `Baton::ready`, a `const` variant of `try_wait`.

Reviewed By: djwatson

Differential Revision: D6508064

fbshipit-source-id: ba458577574ba58165408a93238da7eb09adf1e6

Make global executors shutdown-safe

Summary:
The `get*Executor()` APIs don't check whether the singletons
haven't been destroyed already. Add a check and allow to return
`nullptr` during shutdown.

Also do a general clean up of the code, there was no reason to use
three independent singletons (non-atomically destroyed) for each
executor.

Reviewed By: philippv, luciang

Differential Revision: D6589486

fbshipit-source-id: 20fb835db7e446bd811bbd6d5ddbc41db9e98b54

Remove the folly/Assume.h shim

Summary: The shim existed so that hhvm-third-party could be updated, and it has been, so it's time to remove the shim.

Reviewed By: yfeldblum

Differential Revision: D6590439

fbshipit-source-id: 3e1b039c1df6cf1e74102f7b7b26aa90fae31ec0

Test UnboundedBlockingQueue in ThreadPoolExecutorTest

Summary: Test UnboundedBlockingQueue for the case in D3527722

Reviewed By: ot

Differential Revision: D6587163

fbshipit-source-id: 1465991018187c5f841e6e3e7a11676390e2f8f2

UnboundedQueue: Change try_dequeue variants to fail only if the queue is empty

Summary:
The current semantics of try_dequeue variants can cause problems in cases where based on external dependencies the consumer is guaranteed that the queue is not empty and depends on the success of try_dequeue. See https://github.com/facebook/wangle/commit/abd07a06cae09efcd4347a9a019d59ae8a013c3f

This diff changes the semantics so that try_dequeue operations fail only if the queue is empty.

Reviewed By: yfeldblum

Differential Revision: D6586156

fbshipit-source-id: 25d6085e28d3e24034ecf6a8bafab51c95464b01

UnboundedQueue: Fix advanceHead

Summary:
Without the fix multiple consumers may update head concurrently and cause it to lag. If this persists until the destruction time of the queue, some segments may be incorrectly retired twice.

The fix is to wait for head to advance to the current segment first before advancing head to the next segment.

Reviewed By: djwatson

Differential Revision: D6588135

fbshipit-source-id: 3e916441bff5ad3f27de418601990c59a0b89bc2

Group the fields of UnboundedQueue

Summary:
[Folly] Group the fields of `UnboundedQueue`.

Into helper structures which are organized by, and aligned by, access.

This is all compiled away and has no runtime cost.

Reviewed By: magedm

Differential Revision: D6555031

fbshipit-source-id: 94424747bf76a0a9a2bd59b6a94749efb391d6a7

Fix comments in UnboundedQueue and DynamicBoundedQueue

Summary: Fix typos and wording in comments.

Reviewed By: yfeldblum

Differential Revision: D6584559

fbshipit-source-id: b4b72f6c25c9f9e6fb3f2fbee69eea4a39b5c004

Revert D6579707: [folly/ConcurrentHashMap] Fix erase in Iterate

Summary:
This reverts commit a45f47a53e106d22daa9cf57be6c40c4f6a430d9

bypass-lint

Differential Revision: D6579707

fbshipit-source-id: 6f6c35de5e05f8cf37491471e1ae1272290fe61b

Fix PicoSpinLock on aarch64

Summary:
Found that building folly on aarch64 gave errors with
picospinlock.  Looked and found that MSVC used casts
so I figured I'd try that

Reviewed By: yfeldblum

Differential Revision: D6504689

fbshipit-source-id: 9565bae5ffab485da407b8609be92ef7db10ab72

Fix erase in Iterate

Summary:
The iterator returned from erase api could skip nodes. The fix is to
initialize the returned iterator with right value.

Reviewed By: djwatson

Differential Revision: D6579707

fbshipit-source-id: a45f47a53e106d22daa9cf57be6c40c4f6a430d9

Fix folly::max_align_v for Clang on ARMv7

Summary:
[Folly] Fix `folly::max_align_v` for Clang on ARMv7.

There is some problem with computing the correct result of `alignof` for a compound `union` POD type, because why wouldn't there be. The result *should* just be the max of the `alignof` of the type of each field if each field is default aligned - but not on some platforms! So we must compute the max directly.

Reviewed By: mzlee, Orvid

Differential Revision: D6573548

fbshipit-source-id: 512a255fda64795104d71fde14372befa3bf41e4

Let Baton methods be noexcept

Summary: [Folly] Let `Baton` methods be `noexcept`.

Reviewed By: djwatson

Differential Revision: D6508057

fbshipit-source-id: 2138a228d8291e79e9368cb77b0f63f4412524dc

Fix zerocopy AsyncSocket memory leaks

Summary:
We currently leak ptr entries due to a missing map erase call in AsyncSocket::releaseZeroCopyBuf.
Also addZeroCopyBuf called when the bytesWritten == 0 will cause and extra id to be allocated. This will delay the deletion of the last IOBuf sent via zerocopy. The number of buffers can accumulate over time if the same bytesWritten == 0 situation is encountered.
Found when running the nbd-server zero downtime upgrade during heavy fio traffic.
Add an AsyncSocket::processZeroCopyWriteInProgress so we can drain the zerocopy msg error queue even after we set the ReadCallback to nullptr.

Reviewed By: djwatson

Differential Revision: D6552982

fbshipit-source-id: 3d2fdca84ec3b5fc46c3bed06c0c9ede66ed565a

Fix xlog docs

Summary: When I was investigating the difference between glog/xlog I encountered this.

Reviewed By: yfeldblum

Differential Revision: D6572117

fbshipit-source-id: 4130deb6cfed7cfb7d2433cdab292481beb2469f

Use explicit memory order in Baton::post

Summary: [Folly] Use explicit memory order in `Baton::post`.

Reviewed By: djwatson

Differential Revision: D6502268

fbshipit-source-id: a379ea711f165e3645ac2127a49824f46225b80a

Move security protocol and application protocol APIs to AsyncTransport.

Summary: They belong here rather than AsyncTransportWrapper.

Reviewed By: siyengar

Differential Revision: D6568209

fbshipit-source-id: 0856dba0b2d527ae68d33942b28e2c485a5b1f46

Dynamic Bounded Queue

Summary:
DynamicBoundedQueue supports:
- Dynamic memory usage that grows and shrink in proportion to the number of elements in the queue.
- Adjustable capacity that helps throttle pathological cases of producer-consumer imbalance that may lead to excessive memory usage.
- The adjustable capacity can also help prevent deadlock by allowing users to temporarily increase capacity substantially to guarantee accommodating producer requests that cannot wait.
- SPSC, SPMC, MPSC, MPMC variants.
- Blocking and spinning-only variants.
- Inter-operable non-waiting, timed until, timed for, and waiting variants of producer and consumer operations.
- Optional variable element weights.

Reviewed By: djwatson

Differential Revision: D6462123

fbshipit-source-id: 5c7a7d88c3fa273211702af9ff78f6302afd2276

UnboundedQueue: Use alignas instead of FOLLY_ALIGNED

Summary: Use alignas instead of FOLLY_ALIGNED

Reviewed By: Orvid

Differential Revision: D6554984

fbshipit-source-id: 5b11ac387faa137aa8dd0c35b6e9bb999cf0bf2d

Fix AsyncSocketTest.WriteErrorCallbackBytesWritten

Summary:
[Folly] Fix `AsyncSocketTest.WriteErrorCallbackBytesWritten`.

Thanks to congestion, especially when the tests are run concurrently, the expectations in the test were too restrictive.

If only 20KB are read, it is possible that only 20KB are acknowledged. The expectation was that if 20KB are read and the recv buffer and send buffer are both 8KB and 24KB are written then all 24KB are acknowledged, but congestion control disagrees.

It is possible that any number of bytes are written to the send buffer, from 28KB up to 40KB. And we have to explicitly wait for 28KB to be written even to know that (otherwise we only know that 20KB are written).

Differential Revision: D6550804

fbshipit-source-id: 100d086972c1526b909da0dbb6e609c144d7b17b

constexpr_ceil

Summary: [Folly] `constexpr_ceil`, an integral rounding-up util.

Reviewed By: Orvid

Differential Revision: D6558042

fbshipit-source-id: 6b42add9bf2e3605baf71391130c2a2c88cc4385

Move max_align_v and max_align_t to folly/lang/Align.h

Summary:
[Folly] Move `max_align_v` and `max_align_t` to `folly/lang/Align.h`.

And drop the macro `FOLLY_ALIGNED_MAX` - just use `alignas(max_align_v)` instead.

Reviewed By: Orvid

Differential Revision: D6554677

fbshipit-source-id: cacb335b141623506a41e95e0740613374279c6f

Remove unused #include <string.h> from folly/Portability.h

Summary: [Folly] Remove unused `#include <string.h>` from `folly/Portability.h`.

Reviewed By: Orvid

Differential Revision: D6557374

fbshipit-source-id: 369ef0b622dc057e882ed0f4ad25b47fa3f05100

Explicitly refer to the std::chrono namespace to avoid conflicts with the folly::chrono namespace

Summary:
This is a hard requirement due to an upcoming change in Folly.

The codemod was only run on files that referenced the `std::chrono` namespace with an unqualified `chrono` and also used both the `std` and `folly` namespaces.
```
fbgr -sl "[^:]chrono::" | xargs grep -l "using namespace std;" | xargs grep -l "using namespace folly;" | xargs sed -r -i 's/([^:])chrono::([a-z])/\1std::chrono::\2/g'
```

Reviewed By: yfeldblum

Differential Revision: D6547864

fbshipit-source-id: 4a8230d311edbaa173722a09b2773e8d053fae7e

typo in io/Cursor.h

Summary: Closes https://github.com/facebook/folly/pull/721

Differential Revision: D6558163

Pulled By: yfeldblum

fbshipit-source-id: ed03aa92b54c61f9980a4759da29910aa77d5db6

RCU

Summary: This adds an RCU implementation, matching http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0461r1.pdf as closely as pratical.  This implementation does not require thread registration or quiescence.

Reviewed By: magedm

Differential Revision: D6330631

fbshipit-source-id: 2c729f3a4c0f151cde5d9a599ecd2a2c20c7da55

logging: minor clean up in Logger.h

Summary:
Update the comment describing the Logger class, and remove a `clang-format off`
since recent versions of `clang-format` appear to do a good job formatting this
macro.

Reviewed By: yfeldblum

Differential Revision: D6525996

fbshipit-source-id: 6abbb3740afb53feb2418ca890cd78af1eb5f7a2

logging: add more documentation

Summary: Add several new documents describing the behavior of the folly logging library.

Reviewed By: yfeldblum

Differential Revision: D6525997

fbshipit-source-id: 8ba2ff99c70c7566987b3da321526a16755e72bf

Switch to the try_wait_for and try_wait_until Baton APIs

Summary: Switch to the `try_wait_for` and `try_wait_until` `Baton` APIs.

Reviewed By: davidtgoldblatt

Differential Revision: D6532103

fbshipit-source-id: aa3ce64152d167bb9c9cb1f266be0f9f8bd498f5

Store filename and provide detailed message on data access assertion failure.

Summary:
It looks like not having offset/size/filename information is way more
harmful, than storing filename just for the sake of this error message.

Reviewed By: yfeldblum

Differential Revision: D6536616

fbshipit-source-id: 469fbdf1deedd76ebd79cf98716c2c269cb10e4d

Assorted tweaks to folly/String.h

Summary:
[Folly] Assorted tweaks to `folly/String.h`.

* Use `void_t` in the definition of `IsConvertible`.
* Remove `AllConvertible`. Callers can use `StrictConjunction` instead.
* Move the `static_assert`s from the header to the source file.

Reviewed By: andrewjcg, ot

Differential Revision: D6501515

fbshipit-source-id: 472ecf23bf7f06be211480b0aceea95f7e60dc21

Fix case where ssl cert does not match key

Summary: In some cases, SSLContextManager seg faults if a cert and key do not match.  This guards against that case when strictSSL = false, and throws a more useful error in the cases when SSL is required.

Reviewed By: xybu

Differential Revision: D6513964

fbshipit-source-id: 8e63a22b346fd3f2a30d558a3659ab6794c7a105

folly::fibers::Baton API consistency with folly::Baton

Summary:
[Folly] `folly::fibers::Baton` API consistency with `folly::Baton`.

Specifically, the suite of `wait`, `try_wait`, `try_wait_for`, and `try_wait_until` members and member templates.

Hold onto the `timed_wait` members for now, but mark them deprecated.

Additionally, for consistency, offer main-context function params consistently for all `try_wait_for`, `try_wait_until`, and both variants of `timed_wait`.

Reviewed By: andriigrynenko

Differential Revision: D6531145

fbshipit-source-id: 960fba48716b12b0ef53262786eacab88d8b2375

Pass ZSTD_CONTENTSIZE_UNKNOWN

Summary:
Content size 0 as unknown is deprecated.
See code comments for details.

Reviewed By: felixhandte

Differential Revision: D6521582

fbshipit-source-id: 876b94cc0123c1020b456cc8f4cd86997afe0619

Remove multi-poster support from Baton

Summary:
[Folly] Remove multi-poster support from `Baton`.

Code which needs multi-poster support may use `SaturatingSemaphore` instead.

Reviewed By: magedm

Differential Revision: D6529661

fbshipit-source-id: d9dc053ca984ef3a404e9361910b0044817d4905

Fix some clang build failures on aarch64

Reviewed By: Orvid

Differential Revision: D6387412

fbshipit-source-id: 4c1424ed4fa49e69885de8cd72e78169314fc502

UnboundedBlockingQueue: Remove extra include

Summary: Remove include not needed after recent change

Reviewed By: yfeldblum

Differential Revision: D6529608

fbshipit-source-id: 904072084e180125d68bd1ef6e4dffb2939cd1eb

Fix signed integer overflow in StaticTracepointTest.cpp

Summary:
[Folly] Fix signed integer overflow in `StaticTracepointTest.cpp`.

It is undefined behavior and is caught by UBSAN in some test runs.

Reviewed By: Orvid

Differential Revision: D6528445

fbshipit-source-id: e4a199e9005b57bcd93c8c4b69f37ccb1e072bdf

Split Baton wait methods into fast and slow paths

Summary:
[Folly] Split `Baton` wait methods into fast and slow paths.

Force-inline the fast paths, and force-outline the slow paths.

Reviewed By: magedm

Differential Revision: D6501992

fbshipit-source-id: 611e26b3cfeef01eb8d3a3500ae3ebc26bee6e86

Fix UBSAN failure in RangeTest.cpp with libstdc++-v4.9

Summary:
[Folly] Fix UBSAN failure in `RangeTest.cpp` with `libstdc++-v4.9`.

The implementation of `std::array` in `libstdc++-v4.9` has undefined behavior in the case of the zero-length array - dereferencing (but immediately taking the address again) of `nullptr`. This is fixed in `libstdc++-v5`.

Reviewed By: ot

Differential Revision: D6528614

fbshipit-source-id: 26be6bdc28d9e6dc6e6293a4d4ecffd769c01413

replace std::dequeue with UMPMCQueue in UnboundedBlockingQueue

Summary: As above. Thanks magedm for giving us this beautiful piece of equipment!

Reviewed By: magedm

Differential Revision: D6488661

fbshipit-source-id: 95aa9646ca1ea937bb1d055e9baa037896c3161e

UnboundedQueue: Add LgAlign template parameter - Refactor code

Summary:
- Add a template parameter, LgAlign, to control memory usage. The parameter is used in DynamicBoundedQueue.
- Refactor code.

Reviewed By: yfeldblum

Differential Revision: D6508015

fbshipit-source-id: 6e17b1d8fd900595147dc4217e04d379a13fbdf8

Move folly/Bits.h to folly/lang/

Summary: [Folly] Move `folly/Bits.h` to `folly/lang/`.

Reviewed By: phoad, Orvid

Differential Revision: D6495547

fbshipit-source-id: a93159321df8277f8a4b4f10a5e4e0fc58cb6022

prevent folly::Function<T const&()> from being initialized with a function that returns a prvalue

Summary: If the folly::Function returns by const& and the wrapped function returns a prvalue, it is guaranteed to return a dangling reference. Prevent it.

Reviewed By: yfeldblum, aary

Differential Revision: D6487447

fbshipit-source-id: 61700b4688e29409eefa27f546b31ecac258cfdd

logging: use raw string literals in config tests

Summary:
Update string literals with escaped quotes and backslashes to use raw string
literal syntax instead.

Reviewed By: yfeldblum

Differential Revision: D6501426

fbshipit-source-id: 557366f86b360596b13148d03fd33c7df5c3f890

FlatCombining: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton

Summary: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton, in preparation for restoring Baton to being single-poster and with blocking wait().

Reviewed By: davidtgoldblatt

Differential Revision: D6510619

fbshipit-source-id: eed87beabc1b698080b8e41c746d570d30804ce7

Fix crash in exception_wrapper::get_exception<>

Summary:
[Folly] Fix crash in `exception_wrapper::get_exception<>`.

When the contained object is unthrown and does not inherit `std::exception`, `get_exception` templated over a type that does not inherit `std::exception` must throw and catch internally and may then return a pointer to the internally thrown object, which has since been deallocated.

Attempting to dereference that pointer is undefined behavior is correctly caught by ASAN as heap-use-after-free.

Fix it by storing objects not inheriting `std::exception` using only the `std::exception_ptr` representation.

As a downside, we no longer get the small-object optimization or the non-throwing optimization for objects which do not inherit `std::exception`. But this is not likely to be terribly concerning in practice.

Reviewed By: ericniebler

Differential Revision: D6504911

fbshipit-source-id: 0065de911733b5cab87be55e7e4e47f0a9c09140

Use simpler tags for ctor dispatch in exception_wrapper

Summary:
[Folly] Use simpler tags for ctor dispatch in `exception_wrapper`.

The tags do not need to be `std::integral_constant`, but can just be arbitrary empty types.

Also format the usage sites of the tags.

Reviewed By: ericniebler

Differential Revision: D6506417

fbshipit-source-id: 377359086d94fc1a8bf8eebf6f8b058ff544829f

Fix DCHECKs in IOBufQueue

Summary: glog actually tries to pull srtings from these pointers when the checks fail, which obfuscates the error

Reviewed By: yfeldblum

Differential Revision: D6506886

fbshipit-source-id: 0283e02aecaa248b84fca1433d3f29f92c37b2e6

Removed dead IsSomeVector

Summary: deadcode

Reviewed By: ddrcoder

Differential Revision: D6498754

fbshipit-source-id: 9dda0ade6928d648b8938a06b185e430753f2318

Fix SignalHandlerTest with ASAN

Reviewed By: yfeldblum

Differential Revision: D6503762

fbshipit-source-id: f82b88c401979761053004eadd2e4c6a74f05f9c

logging: allow partial updates to log handler settings

Summary:
This updates the LogHandlerConfig code to allow changing the settings on an
existing log handler without listing all of its existing options from scratch.

This also changes the syntax of the basic log handler configuration string to
use a colon to separate the log handler name+type from the options list.  In
other words, rather than specifying `default=stream,stream=stderr,async=true`
you now say `default=stream:stream=stderr,async=true`.

The primary motivation for this change is to make it easy for users to switch
the async setting for the default log handler on or off.  Callers can now
specify `default:async=true` to easily enable async mode on the default log
handler without having to completely re-list the full settings for the default
handler.

Reviewed By: yfeldblum

Differential Revision: D6494228

fbshipit-source-id: 52a296f800a5456f0c3aa10546298139c8db52fc

logging: split FileHandlerFactory into two classes

Summary:
Split FileHandlerFactory into separate FileHandlerFactory and
StreamHandlerFactory classes, where FileHandlerFactory only handles logging to
files by path name, and StreamHandlerFactory only supports logging to stdout or
stderr.

The primary motivation for this is to allow logging to stdout or stderr in some
cases without allowing arbitrary files to be opened by FileHandlerFactory.
This can be achieved now by registering StreamHandlerFactory but not
FileHandlerFactory.  This makes it safer to allow controlling logging
configuration via command line flags even in setuid binaries.

Reviewed By: yfeldblum

Differential Revision: D6494226

fbshipit-source-id: a3ec371ca4266424d07dff20be18e6e13c057593

logging: add a StandardLogHandlerFactory helper class

Summary:
This moves some of the FileHandlerFactory code out into a new
StandardLogHandlerFactory helper class.  This will make it easier in the future
to add new LogHandlerFactory implementations that create StandardLogHandler
objects.

In particular, I plan to use this soon to split FileHandlerFactory into two
separate classes: one for writing to files on disk and a separate class for
writing to stdout or stderr.

Reviewed By: yfeldblum

Differential Revision: D6494227

fbshipit-source-id: 52e24250d020d21a5395d2a68fa5bd40bb32fbd4

fix ubsan-detected UMR in TemporaryFile

Summary:
Move constructors are just like any other constructors. `this` is not in a valid state. You aren't supposed to destroy `this` before move-constructing it.

This was detected by ubsan stress runs.

UBSAN report:

```
[ RUN      ] TemporaryFile.moveCtor
folly/experimental/TestUtil.cpp:91:20: runtime error: load of value 3, which is not a valid value for type 'bool'
 [ failed: 1 ]
 ````

Reviewed By: yfeldblum

Differential Revision: D6501500

fbshipit-source-id: 83bd0525ac2c1f5951f70f5596034bb6cb0ca985

Rename Baton::timed_wait to try_wait_for and try_wait_until

Summary:
[Folly] Rename `Baton::timed_wait` to `try_wait_for` and `try_wait_until`.

For consistency with `std::timed_mutex`'s suite of `lock` member functions, as well as `SaturatingSemaphore` and others.

Reviewed By: davidtgoldblatt

Differential Revision: D6493714

fbshipit-source-id: 566c6d5dc0f6db4a0308cbd1972d53794f898383

Move folly/Baton.h to folly/synchronization/

Summary: [Folly] Move `folly/Baton.h` to `folly/synchronization/`.

Reviewed By: phoad, Orvid

Differential Revision: D6490282

fbshipit-source-id: 66e2d25ffe3275d576b97b81c1987709000f6649

Use boost::filesystem::current_path in ChangeToTempDir

Summary:
[Folly] Use `boost::filesystem::current_path` in `ChangeToTempDir`.

It will report failures noisily.

Reviewed By: pixelb

Differential Revision: D6493243

fbshipit-source-id: 423dc0e3a46781e9af42fee69060d31085f1a7c6

exclude Unix Domain Socket from enableTTLBANotifications

Reviewed By: yfeldblum

Differential Revision: D6482348

fbshipit-source-id: 5b2d3c984b761a54d5a9eba48953c556ed22cda6

Fix RequestContext held too long issue in EventBase

Summary:
Upon debugging for the attached task, it appears to me the problem are in two places:
. After a callback has been run, context_ wasn't reset
. After a callback has been canceled, context_ wasn't reset

In this diff:
. Fix these two places.
. Updating unit tests to cover these two cases.

Reviewed By: yfeldblum

Differential Revision: D6465788

fbshipit-source-id: 85b3b29dc80c9f3971c85f302385d41ded44fa0e

Split get_default() into two for deferred default construction and added forwarding to speed up default element construction with rvalues

Summary:
As it stood currently folly::get_default() would unnecessarily
construct a value into the third parameter, which was unnecessary in the code
path where the element was found in the map.  Also the default value can be
forwarded to the return type in the code path where the element is not found
and an rvalue parameter is passed as the default value

Reviewed By: yfeldblum

Differential Revision: D6390315

fbshipit-source-id: ef692b827d5a36751b4eb1e12042869e8fbba2e5

bitreverse

Summary:
Adds a bitreverse function to Bits.h

AFAIK there is no great way to do this on x86, takes ~4-5ns.
arm has a single instruction we can drop in in the future.

Reviewed By: yfeldblum

Differential Revision: D6459283

fbshipit-source-id: 129db196b2fac95386f601ae57843aa87523b915

exclude Unix Domain Socket from setErrMessageCB

Summary:
In the latest stable kernel 4.14.3 as of 2017-12-04, unix domain socket does not support MSG_ERRQUEUE. So recvmsg(MSG_ERRQUEUE) will read application data from unix doamin socket as error message, which breaks the message flow in application. This diff disable setErrMessageCB for Unix Domain Socket.

Both [[ https://github.com/torvalds/linux/blob/master/net/ipv4/tcp.c#L1782 | tcp_recvmsg ]] and [[ https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L1571 | udp_recvmsg ]] will check flag MSG_ERRQUEUE and read from error queue.  However, there is nothing about MSG_ERRQUEUE in [[ https://github.com/torvalds/linux/blob/master/net/unix/af_unix.c#L2249 | af_unix ]].

Reviewed By: yfeldblum

Differential Revision: D6479465

fbshipit-source-id: eba2c8650e96466f2b361a42ddf90053d65f19bd

Add missing check against SemiFuture in FutureBase ctor

Summary: [Folly] Add missing check against `SemiFuture` in `FutureBase` ctor.

Reviewed By: LeeHowes

Differential Revision: D6469021

fbshipit-source-id: 0ca31461afee4bf64725e065a8cb4d5e933d2013

better error handling in MemoryIdler for inside jails

Summary:
/proc may be unavailable in restricted environments, which can
cause pthread_getattr_np to fail.  This diff adds checking to the return
code from that function.

Reviewed By: yfeldblum

Differential Revision: D6473789

fbshipit-source-id: de016d32f29edf8410808dfb491987692f39e768

Improve QueueAppender/IOBufQueue performance

Summary:
Currently QueueAppender needs to follow a chain of 4 indirections (QueueAppender->IOBufQueue->IOBuf(head)->IOBuf(tail)->data).
This diff adds a cache of writable tail range in IOBufQueue and allows it to be placed externally.

Before this diff on hot path QueueAppender::write<signed char> was ~167 bytes of code (with majority being actually executed), after this diff it's down to ~30 bytes:
  0x0000000000419d10 <+0>:     mov    (%rdi),%rax
  0x0000000000419d13 <+3>:     cmp    %rax,0x8(%rdi)
  0x0000000000419d17 <+7>:     je     0x419d28 <folly::io::QueueAppender::write<signed char>(signed char)+24>
  0x0000000000419d19 <+9>:     mov    %sil,(%rax)
  0x0000000000419d1c <+12>:    addq   $0x1,(%rdi)
  0x0000000000419d20 <+16>:    retq
  0x0000000000419d21 <+17>:    nopl   0x0(%rax)
  0x0000000000419d28 <+24>:    movsbl %sil,%esi
  0x0000000000419d2c <+28>:    jmpq   0x419ca0 <folly::io::QueueAppender::writeSlow<signed char>(signed char)>

With this diff, Thrift serialization performance is improved up to 2x with production workloads (2x for compact, 3x for binary).

Thrift benchmark output:
Before:
  ============================================================================
  thrift/lib/cpp2/test/ProtocolBench.cpp          relative  time/iter  iters/s
  ============================================================================
  BinaryProtocol_write_Empty                                  58.05ns   17.23M
  BinaryProtocol_write_SmallInt                               75.17ns   13.30M
  BinaryProtocol_write_BigInt                                 74.60ns   13.41M
  BinaryProtocol_write_SmallString                            85.12ns   11.75M
  BinaryProtocol_write_BigString                             802.96ns    1.25M
  BinaryProtocol_write_BigBinary                             174.69ns    5.72M
  BinaryProtocol_write_LargeBinary                           171.81ns    5.82M
  BinaryProtocol_write_Mixed                                 130.97ns    7.64M
  BinaryProtocol_write_SmallListInt                          123.99ns    8.06M
  BinaryProtocol_write_BigListInt                             40.72us   24.56K
  BinaryProtocol_write_BigListMixed                          784.78us    1.27K
  BinaryProtocol_write_LargeListMixed                         98.84ms    10.12
  CompactProtocol_write_Empty                                 64.38ns   15.53M
  CompactProtocol_write_SmallInt                              76.74ns   13.03M
  CompactProtocol_write_BigInt                                83.62ns   11.96M
  CompactProtocol_write_SmallString                           86.05ns   11.62M
  CompactProtocol_write_BigString                            786.18ns    1.27M
  CompactProtocol_write_BigBinary                            184.91ns    5.41M
  CompactProtocol_write_LargeBinary                          182.12ns    5.49M
  CompactProtocol_write_Mixed                                120.89ns    8.27M
  CompactProtocol_write_SmallListInt                         119.74ns    8.35M
  CompactProtocol_write_BigListInt                            43.76us   22.85K
  CompactProtocol_write_BigListMixed                         595.90us    1.68K
  CompactProtocol_write_LargeListMixed                        72.80ms    13.74
  ============================================================================
After:
  ============================================================================
  thrift/lib/cpp2/test/ProtocolBench.cpp          relative  time/iter  iters/s
  ============================================================================
  BinaryProtocol_write_Empty                                  65.97ns   15.16M
  BinaryProtocol_write_SmallInt                               72.31ns   13.83M
  BinaryProtocol_write_BigInt                                 72.67ns   13.76M
  BinaryProtocol_write_SmallString                            77.56ns   12.89M
  BinaryProtocol_write_BigString                             782.07ns    1.28M
  BinaryProtocol_write_BigBinary                             179.69ns    5.57M
  BinaryProtocol_write_LargeBinary                           182.62ns    5.48M
  BinaryProtocol_write_Mixed                                  91.62ns   10.92M
  BinaryProtocol_write_SmallListInt                           96.22ns   10.39M
  BinaryProtocol_write_BigListInt                             19.65us   50.90K
  BinaryProtocol_write_BigListMixed                          245.69us    4.07K
  BinaryProtocol_write_LargeListMixed                         46.56ms    21.48
  CompactProtocol_write_Empty                                 74.44ns   13.43M
  CompactProtocol_write_SmallInt                              80.35ns   12.45M
  CompactProtocol_write_BigInt                                85.30ns   11.72M
  CompactProtocol_write_SmallString                           82.61ns   12.10M
  CompactProtocol_write_BigString                            784.77ns    1.27M
  CompactProtocol_write_BigBinary                            193.20ns    5.18M
  CompactProtocol_write_LargeBinary                          192.53ns    5.19M
  CompactProtocol_write_Mixed                                 99.78ns   10.02M
  CompactProtocol_write_SmallListInt                         104.77ns    9.54M
  CompactProtocol_write_BigListInt                            25.62us   39.03K
  CompactProtocol_write_BigListMixed                         272.42us    3.67K
  CompactProtocol_write_LargeListMixed                        38.32ms    26.09
  ============================================================================

QueueAppender Benchmark output (although not very representative due to a tight loop):
Before:
  ============================================================================
  folly/io/test/QueueAppenderBenchmark.cpp        relative  time/iter  iters/s
  ============================================================================
  write_uint8                                                 10.50us   95.20K
  write_uint16                                                 5.48us  182.49K
  write_uint32                                                 2.73us  366.22K
  push_64b                                                     9.77us  102.36K
  push_1024b                                                 112.87us    8.86K
  append                                                      64.21us   15.57K
  preallocate_postallocate_1b                                 16.34us   61.19K
  preallocate_postallocate_4b                                 15.56us   64.26K
  preallocate_postallocate_32b                                22.17us   45.11K
  preallocate_postallocate_256b                              149.55us    6.69K
  ============================================================================

After:
  ============================================================================
  folly/io/test/QueueAppenderBenchmark.cpp        relative  time/iter  iters/s
  ============================================================================
  write_uint8                                                  8.86us  112.81K
  write_uint16                                                 3.91us  255.68K
  write_uint32                                                 2.08us  481.78K
  push_64b                                                     8.24us  121.39K
  push_1024b                                                 115.50us    8.66K
  append                                                      67.52us   14.81K
  preallocate_postallocate_1b                                 13.86us   72.17K
  preallocate_postallocate_4b                                 11.67us   85.71K
  preallocate_postallocate_32b                                20.35us   49.14K
  preallocate_postallocate_256b                              148.57us    6.73K
  ============================================================================

Reviewed By: yfeldblum

Differential Revision: D6427749

fbshipit-source-id: 8495cc74b6106b15d201e37533ae4c0a1abc9d74

add AtFork::unregisterHandler

Summary: Adds an AdFork::unregisterHandler command, such that we can register and unregister fork handlers.

Reviewed By: yfeldblum

Differential Revision: D6439796

fbshipit-source-id: b710152f8cf98371c330b484cd5fe42de44e74ac

Unbounded queue

Summary:
UnboundedQueue supports:
- SPSC, MPSC, SCMP, MPMC
- Non-waiting, waiting, and timed consumer operations.
- Producers never wait or fail (unless out-of-memory).
- Memory usage grows and shrinks dynamically

```
/// UnboundedQueue supports a variety of options for unbounded
/// dynamically expanding an shrinking queues, including variations of:
/// - Single vs. multiple producers
/// - Single vs. multiple consumers
/// - Blocking vs. spin-waiting
/// - Non-waiting, timed, and waiting consumer operations.
/// Producer operations never wait or fail (unless out-of-memory).
///
/// Template parameters:
/// - T: element type
/// - SingleProducer: true if there can be only one producer at a
///   time.
/// - SingleConsumer: true if there can be only one consumer at a
///   time.
/// - MayBlock: true if consumers may block, false if they only
///   spins. A performance tuning parameter.
/// - LgSegmentSize (default 8): Log base 2 of number of elements per
///   segment. A performance tuning parameter. See below.
///
/// When to use UnboundedQueue:
/// - If a small bound may lead to deadlock or performance degradation
///   under bursty patterns.
/// - If there is no risk of the queue growing too much.
///
/// When not to use UnboundedQueue:
/// - If there is risk of the queue growing too much and a large bound
///   is acceptable, then use DynamicBoundedQueue.
/// - If the queue must not allocate on enqueue or it must have a
///   small bound, then use fixed-size MPMCQueue or (if non-blocking
///   SPSC) ProducerConsumerQueue.
///
/// Template Aliases:
///   USPSCQueue<T, MayBlock, LgSegmentSize>
///   UMPSCQueue<T, MayBlock, LgSegmentSize>
///   USPMCQueue<T, MayBlock, LgSegmentSize>
///   UMPMCQueue<T, MayBlock, LgSegmentSize>
///
/// Functions:
///   Producer operations never wait or fail (unless OOM)
///     void enqueue(const T&);
///     void enqueue(T&&);
///         Adds an element to the end of the queue.
///
///   Consumer operations:
///     void dequeue(T&);
///         Extracts an element from the front of the queue. Waits
///         until an element is available if needed.
///     bool try_dequeue(T&);
///         Tries to extracts an element from the front of the queue
///         if available. Returns true if successful, false otherwise.
///     bool try_dequeue_until(T&, time_point& deadline);
///         Tries to extracts an element from the front of the queue
///         if available until the specified deadline.  Returns true
///         if successful, false otherwise.
///     bool try_dequeue_for(T&, duration&);
///         Tries to extracts an element from the front of the queue
///         if available for for the specified duration.  Returns true
///         if successful, false otherwise.
///
///   Secondary functions:
///     size_t size();
///         Returns an estimate of the size of the queue.
///     bool empty();
///         Returns true only if the queue was empty during the call.
///     Note: size() and empty() are guaranteed to be accurate only if
```

Reviewed By: djwatson

Differential Revision: D6157613

fbshipit-source-id: db423f86d1d0604d22f6b9c71ea0ed08be32e2a1