folly.git
3 years agoStatically allocate futex array v2017.12.04.00
Alastair Daivis [Sun, 3 Dec 2017 01:35:33 +0000 (17:35 -0800)]
Statically allocate futex array

Summary:
AFAICT this is currently the only thing preventing the non-dynamic version of Folly's MPMCQueue from being allocation-free on enqueue and dequeue operations. Allocating this with a constant expression should move the allocation from runtime to link (possibly compile?) time, which will let us use it in allocation sensitive contexts.

Feel free to suggest other reviewers, I couldn't find an existing folly reviewer group.

Reviewed By: yfeldblum, agola11, nbronson

Differential Revision: D6447848

fbshipit-source-id: 86b84b19d62f1e1bcecdb9e757a6dfa90597b084

4 years agoallow fiber executor to override fiber options
David Goode [Sat, 2 Dec 2017 03:56:14 +0000 (19:56 -0800)]
allow fiber executor to override fiber options

Summary: As in title, for eg larger stack size

Reviewed By: yfeldblum

Differential Revision: D6466762

fbshipit-source-id: 303e8732c8bfb64fbd8619163779942fe5f8b5a8

4 years agoadd makeSystemError*() helper functions
Adam Simpkins [Sat, 2 Dec 2017 02:05:54 +0000 (18:05 -0800)]
add makeSystemError*() helper functions

Summary:
Add makeSystemError() helper functions that are similar to the existing
throwSystemError() functions but just return the exception rather than throwing
it.  This is helpful for callers using folly::Expected or folly::Future, where
they need to return an exception type rather than throwing it.

This also includes comments about the fact that this code is using the wrong
error category.  The C++ standard indicates that std::generic_category() for
POSIX errno values.  I am not fixing this as part of this diff, since this
change has the potential to break existing users, and would need a fair amount
of testing first.

Reviewed By: yfeldblum

Differential Revision: D6456771

fbshipit-source-id: 4724b51b8d4a7f513ae70ea1b0c4f0516cfc205f

4 years agologging: fix compilation error on older C++ compilers
Adam Simpkins [Sat, 2 Dec 2017 02:01:59 +0000 (18:01 -0800)]
logging: fix compilation error on older C++ compilers

Summary:
Add an explicit `std::move()` around the return value of
`logConfigToDynamic()`.  This explicit move is required pre-C++14,
but is not required after DR 1579:
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579

Newer versions of g++ and clang no longer require the move.  However gcc
versions earlier than 5.0 do require it.

Reviewed By: yfeldblum

Differential Revision: D6466447

fbshipit-source-id: 1b15934490d4966e9b3e5612e0e2ecbb43c979ca

4 years agologging: update initialization code to use the new LogConfig logic
Adam Simpkins [Sat, 2 Dec 2017 00:53:51 +0000 (16:53 -0800)]
logging: update initialization code to use the new LogConfig logic

Summary:
Replace the initLoggingGlogStyle() function with a more generic initLogging()
function that accepts a log config string to be parsed with parseLogConfig().

Reviewed By: bolinfest, yfeldblum

Differential Revision: D6342086

fbshipit-source-id: fb1bffd11f190b70e03e2ccbf2b30be08d655242

4 years agounify FOLLY_SANITIZE existing uses and add a few new
Igor Sugak [Fri, 1 Dec 2017 19:19:44 +0000 (11:19 -0800)]
unify FOLLY_SANITIZE existing uses and add a few new

Reviewed By: yfeldblum

Differential Revision: D6453972

fbshipit-source-id: fa083c0b782188f5f45a2a354045aee03e528829

4 years agoimplement chrono conversions for unusual duration types
Adam Simpkins [Fri, 1 Dec 2017 19:18:17 +0000 (11:18 -0800)]
implement chrono conversions for unusual duration types

Summary:
Implement conversions between std::chrono::duration and POSIX-style time
structures even when neither the numerator nor the denominator of the duration
ratio are 1.

Both of these are done by first converting to an intermediate type where the
numerator is 1, and then using the conversion routines for that case.

Reviewed By: yfeldblum

Differential Revision: D6366647

fbshipit-source-id: 8f9495fb4101cac6d8b4cf0353a679107007b298

4 years agoMove threadlocal_detail::Atfork to its own file
Dave Watson [Fri, 1 Dec 2017 15:09:43 +0000 (07:09 -0800)]
Move threadlocal_detail::Atfork to its own file

Summary: As title

Reviewed By: yfeldblum

Differential Revision: D6440723

fbshipit-source-id: 3168d7bb616ae0ff3fe42f7584c5a255c4953875

4 years agofix ASAN build
Bin Liu [Thu, 30 Nov 2017 23:52:54 +0000 (15:52 -0800)]
fix ASAN build

Summary:
Fix bug in non-jemalloc build.

(Note: this ignores all push blocking failures!)

Reviewed By: igorsugak

Differential Revision: D6451342

fbshipit-source-id: 44725d4e0c685a59325fa3d08877d0f62120c5c2

4 years agoLet SaturatingSemaphore::try_wait be non-const and add ready
Yedidya Feldblum [Thu, 30 Nov 2017 22:56:07 +0000 (14:56 -0800)]
Let SaturatingSemaphore::try_wait be non-const and add ready

Summary:
[Folly] Let `SaturatingSemaphore::try_wait` be non-`const` and add `ready`.

For internal API consistency.

Reviewed By: magedm

Differential Revision: D6450089

fbshipit-source-id: 65b9b723672521710a69719b192eb2922a27b778

4 years agofix build when sanitizers are enabled and jemalloc is disabled
Igor Sugak [Thu, 30 Nov 2017 22:48:24 +0000 (14:48 -0800)]
fix build when sanitizers are enabled and jemalloc is disabled

Summary: Add preprocessor checks to guard jemalloc headers if any sanitizer is enabled.

Reviewed By: yfeldblum

Differential Revision: D6441578

fbshipit-source-id: 85364fca5af33c95cd05f232d1e3f9cbac9a4120

4 years agoSome fixes for custom conversions of enums
Yedidya Feldblum [Thu, 30 Nov 2017 06:13:39 +0000 (22:13 -0800)]
Some fixes for custom conversions of enums

Summary:
[Folly] Some fixes for custom conversions of enums.

Of note, `to` was defined for enum -> all conversions, including enum -> string conversions, but we actually want enum -> string conversions to be done via ADL-discovered toAppend.

Reviewed By: ot

Differential Revision: D6411250

fbshipit-source-id: 852b64309e6adf1c68e5153635cb29632e2d86d4

4 years agoSaturating semaphore
Maged Michael [Thu, 30 Nov 2017 05:12:16 +0000 (21:12 -0800)]
Saturating semaphore

Summary:
SaturatingSemaphore is a flag that allows:
- multiple concurrent posters
- multiple concurrent waiters
- idempotent posting
- non-destructive waiting
- blocking and spinning
- pre-block spin time control

```
/// SaturatingSemaphore is a flag that allows concurrent posting by
/// multiple posters and concurrent non-destructive waiting by
/// multiple waiters.
///
/// A SaturatingSemaphore allows one or more waiter threads to check,
/// spin, or block, indefinitely or with timeout, for a flag to be set
/// by one or more poster threads. By setting the flag, posters
/// announce to waiters (that may be already waiting or will check
/// the flag in the future) that some condition is true. Posts to an
/// already set flag are idempotent.
```

Reviewed By: djwatson

Differential Revision: D6379704

fbshipit-source-id: 59aed76caa2d159639e75425a778a9c63f18f375

4 years agoadd FOLLY_SANITIZE macro
Igor Sugak [Thu, 30 Nov 2017 02:14:04 +0000 (18:14 -0800)]
add FOLLY_SANITIZE macro

Summary: Most of the time we want to guard some code when any of the sanitizers are enabled not just one.

Reviewed By: yfeldblum

Differential Revision: D6429720

fbshipit-source-id: 7157a0a13f2b2acabd36a19c61b9fdbf38ba96b8

4 years agologging: add a LogConfig::update() method
Adam Simpkins [Thu, 30 Nov 2017 01:35:21 +0000 (17:35 -0800)]
logging: add a LogConfig::update() method

Summary:
Add a method for merging the settings from two LogConfig objects.
This allows LogConfig objects to be merged before applying them to the
LoggerDB.  The effects are the same as two sequential LoggerDB::updateConfig()
calls, but without having to apply the intermediate state to the LoggerDB.

Reviewed By: bolinfest

Differential Revision: D6342085

fbshipit-source-id: 0f8a1b7d8d195a80bc74342444dd3152d331fcb6

4 years agologging: add an assert for the FLAG_INHERIT bit
Adam Simpkins [Thu, 30 Nov 2017 01:35:19 +0000 (17:35 -0800)]
logging: add an assert for the FLAG_INHERIT bit

Summary:
Add a static_assert() to check that the FLAG_INHERIT bit does not conflict with
valid LogLevel values.

Reviewed By: bolinfest

Differential Revision: D6341242

fbshipit-source-id: cec284623c8a612a3c54f1b593d169310790616e

4 years agologging: remove comment about LogLevel::ERROR
Adam Simpkins [Thu, 30 Nov 2017 01:35:17 +0000 (17:35 -0800)]
logging: remove comment about LogLevel::ERROR

Summary:
The logging code used to support a LogLevel::ERROR value on non-Windows
platforms.  I removed it in D5288600 to make all platforms consistent, but I
forgot to remove this comment describing it.

Reviewed By: bolinfest

Differential Revision: D6341244

fbshipit-source-id: 054427d342066f354c859b12611b907dc2d4fa35

4 years agologging: add LoggerDB::updateConfig() and resetConfig()
Adam Simpkins [Thu, 30 Nov 2017 01:35:16 +0000 (17:35 -0800)]
logging: add LoggerDB::updateConfig() and resetConfig()

Summary: Add methods for applying config changes from a LogConfig object to the LoggerDB.

Reviewed By: bolinfest

Differential Revision: D6200564

fbshipit-source-id: a25eb99e84b2885bf6853e2222db0d7432a6c37b

4 years agologging: add a LoggerDB::getConfig() method
Adam Simpkins [Thu, 30 Nov 2017 01:35:13 +0000 (17:35 -0800)]
logging: add a LoggerDB::getConfig() method

Summary: Add a method to get the current LogConfig state from the LoggerDB.

Reviewed By: bolinfest

Differential Revision: D6200596

fbshipit-source-id: 3bc57d498a5d25d19099d861376d71ea9f7e4039

4 years agologging: don't clamp the log level to DFATAL in debug builds
Adam Simpkins [Thu, 30 Nov 2017 01:35:11 +0000 (17:35 -0800)]
logging: don't clamp the log level to DFATAL in debug builds

Summary:
Remove the logic that clamps the log level to DFATAL in debug builds.

This behavior logically makes some sense, but results in subtle and potentially
confusing behavior changes.  It seems confusing that after calling
`setLevel(LogLevel::MAX_LEVEL)` on a log category, calling `getLevel()` on that
object would return `DFATAL` rather than `MAX_LEVEL`.  We also weren't clamping
the level to DFATAL consistently: `setLevel()` would clamp the value, but on
construction the initial level was still set to MAX_LEVEL rather than DFATAL.
This resulted in some issues when implementing `LoggerDB::getConfig()` since we
could not consistently detect if a log category was using the default level
settings.  Rather than fix this inconsistency it seems better to simply remove
this clamping behavior.

This means that it is possible for users to disable DFATAL log messages even in
debug builds if they really want to.  Previously this was only allowed in
release builds.  This protection doesn't really seem all that
valuable--presumably most developers won't really want to do this, and if they
really do request this configuration it doesn't seem all that bad to honor it.

Reviewed By: bolinfest, yfeldblum

Differential Revision: D6200569

fbshipit-source-id: 83ef8e2e4d3b61bc5b105038cbe3132979e9ac67

4 years agologging: add a LogHandler::getConfig() method
Adam Simpkins [Thu, 30 Nov 2017 01:35:09 +0000 (17:35 -0800)]
logging: add a LogHandler::getConfig() method

Summary:
Add a method to LogHandler to return its current configuration.  This will
make it possible to query the LoggerDB for its current configuration state.

Reviewed By: bolinfest

Differential Revision: D6200563

fbshipit-source-id: 2b8b9752bbeb26c8aac28d1a73b7e2312fd198c8

4 years agologging: add a LogHandler registry to LoggerDB
Adam Simpkins [Thu, 30 Nov 2017 01:35:08 +0000 (17:35 -0800)]
logging: add a LogHandler registry to LoggerDB

Summary:
Update the LoggerDB to track a list of LogHandlers by name, and
LogHandlerFactories by handler type.

This will be needed to support updating the LoggerDB configuration from a
LogConfig object.

Reviewed By: bolinfest

Differential Revision: D6200562

fbshipit-source-id: e365b4e0df65aa5aaa34e118eb3cee9c9c45cb05

4 years agoSynchronize coupled caches in folly::threadlocal_detail::StaticMeta
Yedidya Feldblum [Wed, 29 Nov 2017 20:12:01 +0000 (12:12 -0800)]
Synchronize coupled caches in folly::threadlocal_detail::StaticMeta

Summary:
[Folly] Synchronize coupled caches in `folly::threadlocal_detail::StaticMeta`.

The caches should be set together, and only together, because they are coupled. This prevents bugs where one function that sets one cache but not the other cache is inlined into the caller in one module, and another function that reads both caches is inlined into the caller in another module.

Reviewed By: djwatson

Differential Revision: D6435175

fbshipit-source-id: 846c4972b40e525f2c04da6e6609c2ad54f674c0

4 years agologging: add a FileHandlerFactory class
Adam Simpkins [Wed, 29 Nov 2017 01:44:55 +0000 (17:44 -0800)]
logging: add a FileHandlerFactory class

Summary:
Add a new LogHandlerFactory interface for creating LogHandler objects from a
LogHandlerConfig.

Also add an initial FileHandlerFactory implementation capable of creating
LogHandler objects that write to a file descriptor.

Reviewed By: bolinfest

Differential Revision: D6200567

fbshipit-source-id: 14b86fc14ad475223aa4b57d45c40638b48c7594

4 years agoAdd unit tests for opening non-ELF files.
Andrii Nakryiko [Wed, 29 Nov 2017 00:48:14 +0000 (16:48 -0800)]
Add unit tests for opening non-ELF files.

Summary:
Just testing that ElfFile returns expected error code and message for
non-ELF files.

Depends on D6410210

Reviewed By: yfeldblum

Differential Revision: D6418365

fbshipit-source-id: aaab3b9f3ca1a12d384ae98a1772b7f640115192

4 years agoFix incorrect usages of folly::Synchronized
Sarang Masti [Tue, 28 Nov 2017 23:51:10 +0000 (15:51 -0800)]
Fix incorrect usages of folly::Synchronized

Summary:
We need to keep LockedPtr returned by rlock/wlock alive for the entire duration
of the loop. Else we are working on a snapshot of the data structure, which
might not be what we want.

Reviewed By: aary, ot, luciang

Differential Revision: D6426613

fbshipit-source-id: 294c0a7b9d756f3116d5cdb3a7b3678f693e0754

4 years agologging: convert assert() checks to FOLLY_SAFE_DCHECK()
Adam Simpkins [Tue, 28 Nov 2017 20:02:02 +0000 (12:02 -0800)]
logging: convert assert() checks to FOLLY_SAFE_DCHECK()

Summary:
Replace all assert() checks in the folly logging code with
FOLLY_SAFE_DCHECK().

Reviewed By: yfeldblum

Differential Revision: D6422885

fbshipit-source-id: 5cb12dd59f2fe5d346f15b9d317abede8654a879

4 years agoUnbreak folly on x86-32
Alexander Kabaev [Tue, 28 Nov 2017 19:42:20 +0000 (11:42 -0800)]
Unbreak folly on x86-32

Summary:
_mm_cvtsi64_si128 and _mm_cvtsi128_si64 intrinsics are available on
x86_64 only, so do not enable SSE instructions when compiling for 32 bits.

Reviewed By: yfeldblum

Differential Revision: D6422271

fbshipit-source-id: a3c5cf3f2ee4b47d5a1edcf93cfff3d1f3fe907e

4 years agoRemove some custom test main()'s in folly/experimental/symbolizer/
Yedidya Feldblum [Tue, 28 Nov 2017 06:18:54 +0000 (22:18 -0800)]
Remove some custom test main()'s in folly/experimental/symbolizer/

Summary:
[Folly] Remove some custom test `main()`'s in `folly/experimental/symbolizer/`.

In one of the two cases, we need to do some extra work to give `ElfFile` knowledge of the program path.

Reviewed By: ot, Orvid, philippv, luciang

Differential Revision: D6420517

fbshipit-source-id: 11c694a8342abb6fff05c368f3cda2e6287ccb4b

4 years agoReset context shared_ptr in AsyncTimeout::cancelTimeout()
Pingjia Shan [Tue, 28 Nov 2017 06:10:59 +0000 (22:10 -0800)]
Reset context shared_ptr in AsyncTimeout::cancelTimeout()

Summary:
This seems to fix issue in the attached task.

`context_` is set in `scheduleTimeout()` and never gets reset. So when an AsyncTimeout object is held across requests by objects like `IOThreadPool0` in I/O Thread Pool used by wangle/acceptor/ConnectionManager, RequestContext object created for the last request is leaked until `IOThreadPool0` handles another request and overrides `context_`. In the issue described in attached task, unit test has single request, next request never comes in, RequestContext doesn't end until test service stops, logging feature relying on RequestContext dtor doesn't get called in time, and thus unit test fails.

Reviewed By: yfeldblum

Differential Revision: D6402268

fbshipit-source-id: 200c6d358dfa6d7d9aa68ab05f6f1c7f4117b0ec

4 years agologging: fix unused variable warning in non-debug builds
Adam Simpkins [Tue, 28 Nov 2017 05:30:58 +0000 (21:30 -0800)]
logging: fix unused variable warning in non-debug builds

Summary:
Use FOLLY_SAFE_DCHECK() rather than assert() to avoid compiler errors about
this variable being unused in builds where assert() statements are disabled.

Reviewed By: bolinfest, yfeldblum

Differential Revision: D6421978

fbshipit-source-id: ab1019fa1fa420856e1bf3bc39c441662c4efd75

4 years agoUse "auto" with GENERATOR instead of std::function
Tom Jackson [Tue, 28 Nov 2017 01:13:37 +0000 (17:13 -0800)]
Use "auto" with GENERATOR instead of std::function

Summary: Now that we're using C++14 more broadly, we should use it to improve generator performance. This speeds up a microbenchmark >5x by removing type erasure.

Reviewed By: philippv

Differential Revision: D6398730

fbshipit-source-id: 5809058a3b5ff0e66fd4b1e8954698944e1a7d09

4 years agoRevise API to load cert/key in SSLContext.
Xiangyu Bu [Mon, 27 Nov 2017 23:37:45 +0000 (15:37 -0800)]
Revise API to load cert/key in SSLContext.

Summary:
When loading cert/key pair, order matters:
(a) Wrong key will fail to load if a cert is loaded;
(b) Wrong cert will succeed to load even if a private key is loaded.

So this diff adds:
(1) SSLContext::checkPrivateKey() -- must call for case (b).
(2) SSLContext::loadCertKeyPairFromBufferPEM() -- use this if one loads both cert and key. Guaranteed to throw if cert/key mismatch.

Reviewed By: yfeldblum

Differential Revision: D6416280

fbshipit-source-id: 8ae370883d46e9b5afb69c506c09fbf7ba82b1b9

4 years agoFix ElfFile crashing when opening short (<64 bytes) files.
Andrii Nakryiko [Mon, 27 Nov 2017 19:31:18 +0000 (11:31 -0800)]
Fix ElfFile crashing when opening short (<64 bytes) files.

Summary:
ElfFile in init() assumes file is long enough to contain complete
ElfHeader, which is 64 bytes long. This is not true for valid non-ELF files like
short scripts. They shouldn't cause crash.

Reviewed By: myreg

Differential Revision: D6410210

fbshipit-source-id: 28fd017d8de17c431d7d006a1655ade8a95994bd

4 years agoUse hazptr_local and hazptr_array
Dave Watson [Mon, 27 Nov 2017 18:50:26 +0000 (10:50 -0800)]
Use hazptr_local and hazptr_array

Summary: Use newest hazptr hotness in concurrenthashmap.  Shaves ~10% off of the single-thread find performance.

Reviewed By: magedm

Differential Revision: D6259947

fbshipit-source-id: 7ecf99d38fdf8e311fca3313137e0fca5af3f165

4 years agomake `usingJEMalloc()` a constexpr when USE_JEMALLOC is defined v2017.11.27.00
Bin Liu [Mon, 27 Nov 2017 06:07:41 +0000 (22:07 -0800)]
make `usingJEMalloc()` a constexpr when USE_JEMALLOC is defined

Summary: This would save some function calls and reduce code size if we know we will use jemalloc.

Reviewed By: yfeldblum

Differential Revision: D6411810

fbshipit-source-id: 9db471678e473945f5f00354c0a1285ecdb91de6

4 years agoAdd folly::hasher support for floating point types
Giuseppe Ottaviano [Sun, 26 Nov 2017 22:43:06 +0000 (14:43 -0800)]
Add folly::hasher support for floating point types

Summary:
Move `folly::hasher` closer to feature parity with `std::hash`.
This is in order to replace some instances of `folly::hash::hash_combine(...)` with `folly::Hash()(...)` (`std::hash` is the identity for integers, which makes it an unsafe default for more sophisticated hash data structures, including open-addressing hash tables).

The implementation is similar to `libstdc++`'s implementation, in that we handle separately the `0` case, because `0` and `-0` have different binary representations but are equal according to `operator==`, and hash the bytes otherwise. It is probably a little faster than `libstdc++`'s implementation, that delegates a out-of-line Murmur hash routine for arbitrary buffers, while this uses a minimal inlineable machine word hashing routine.

Reviewed By: yfeldblum

Differential Revision: D6410713

fbshipit-source-id: 86d9e4ed8da04fffe283949825852e539ec7d5cf

4 years agoQueuedImmediateExecutor tweaks
Yedidya Feldblum [Thu, 23 Nov 2017 19:59:32 +0000 (11:59 -0800)]
QueuedImmediateExecutor tweaks

Summary:
[Folly] `QueuedImmediateExecutor` tweaks.

* Add a leaky meyers singleton instance.
* Make the thread-local queue an instance variable. Callers which want the singleton thread-local queue can use the singleton executor instance instance, for the same effect.
* Simplify the body of `add`, and perform the thread-local lookup only once per invocation.

Reviewed By: djwatson

Differential Revision: D6399067

fbshipit-source-id: 03904885a70c4b943141bd83868414d27232fd6a

4 years agoEnable EventBase switching for AsyncSocket even if it has registered events
Fuat Geleri [Wed, 22 Nov 2017 22:20:42 +0000 (14:20 -0800)]
Enable EventBase switching for AsyncSocket even if it has registered events

Summary:
AsyncSocket will return isDetachable() -> false if there is a read callback set.
So,
 - ignore event registration status
 - unregister the events when detaching and
 - register them back when attaching again.

Reviewed By: afrind, eduardo-elizondo

Differential Revision: D6331787

fbshipit-source-id: fca4e6101f2d5666edbafa09116aa50f34cb084d

4 years agoLink to unwind when using symbolizer
Guo Xiao [Wed, 22 Nov 2017 19:19:03 +0000 (11:19 -0800)]
Link to unwind when using symbolizer

Summary:
Fix #673
Closes https://github.com/facebook/folly/pull/715

Reviewed By: meyering

Differential Revision: D6394731

Pulled By: yfeldblum

fbshipit-source-id: fb92828d0c633278601125dbc64cadf0cdbaa7fe

4 years agoAdd a fast path to folly::ThreadLocal
Dave Watson [Wed, 22 Nov 2017 16:16:04 +0000 (08:16 -0800)]
Add a fast path to folly::ThreadLocal

Summary:
Currently folly::ThreadLocal[Ptr] is pretty heavy-weight for a get():

1) call instance(), take a static init guard, branch
2) call getThreadEntry, check if thread_local is not null, branch
3) check if id < threadEntry->capacity, branch
4) Finally, return threadEntry->elements[id]

If we have real thread_locals, we can do better by caching the capacity directly,
combining all three checks:

1) checkif id < threadLocalCapacityCheck, branch.  If not, do slow path.
2) return threadEntry->elements[id].  Threadentry is never null if capacity > 0, and
    instance() setup work is called during the first getThreadEntry call when threadlocalcapacity == 0.

Reviewed By: yfeldblum

Differential Revision: D6379878

fbshipit-source-id: 4fc7564bbb2f319d65875124026aef28d910ef06

4 years agoFix ASAN failure in FutureDAG test
Andrii Grynenko [Wed, 22 Nov 2017 04:58:05 +0000 (20:58 -0800)]
Fix ASAN failure in FutureDAG test

Summary: dogscience

Reviewed By: yfeldblum

Differential Revision: D6387604

fbshipit-source-id: 3e33da3e3c6801d6e2186637c8fb6adaf1eef6f6

4 years agoAdd -Werror to -Wshadow-local and -Wshadow-compatible-local warning check
Aaron Orenstein [Tue, 21 Nov 2017 22:02:18 +0000 (14:02 -0800)]
Add -Werror to -Wshadow-local and -Wshadow-compatible-local warning check

Summary:
Without -Werror Apple LLVM will print a warning about -Wshadow-local
and -Wshadow-compatible-local but won't fail.  This causes problems for
downstream code which uses -Werror.

Reviewed By: meyering

Differential Revision: D6375771

fbshipit-source-id: 4ac3a4ff46bdb76f1f07b0c8ba479a7d404c8c9b

4 years agologging: add a LogConfig class and parsing code
Adam Simpkins [Tue, 21 Nov 2017 18:44:02 +0000 (10:44 -0800)]
logging: add a LogConfig class and parsing code

Summary:
Add a new LogConfig class to represent the current configuration state of the
LoggerDB.  This also includes code to parse config strings in both JSON and a
simpler more human-writable format (intended primarily for use in command line
arguments).

I generally expect the human-writable format to be used mainly to configure log
levels.  It also supports configuring log handler details as well, but the
format for this data is slightly more cumbersome and will probably be harder
for people to easily remember.

The parsing code is intentionally kept as part of the 'init' library rather
than the core 'logging' library so that other libraries that simply wish to log
messages do not need to depend on it.  For instance, this would allow the folly
JSON library to use the logging library without causing a circular dependency.

Reviewed By: bolinfest

Differential Revision: D6200560

fbshipit-source-id: e4e3c7f941808251b6c7bcbbdac0210118675fb0

4 years agoimprove ThreadLocalBenchmark
Dave Watson [Tue, 21 Nov 2017 15:34:15 +0000 (07:34 -0800)]
improve ThreadLocalBenchmark

Summary:
Benchmark appears to be memory bound, and is affected by different cpus.
Add a new benchmark that does more work.

Reviewed By: yfeldblum

Differential Revision: D6380904

fbshipit-source-id: 5cfbaab70379aa4a2923d957e1147d8486deeff7

4 years agofix hazptr_array move
Dave Watson [Tue, 21 Nov 2017 15:33:09 +0000 (07:33 -0800)]
fix hazptr_array move

Summary: Broken, found via build failures in D6259947

Reviewed By: yfeldblum

Differential Revision: D6374215

fbshipit-source-id: 3a438b5416683c602966c2f029c6b7787acdccaa

4 years agoRevert D6366352: [folly] Split get_default() into two for deferred default construction
Giuseppe Ottaviano [Tue, 21 Nov 2017 07:10:34 +0000 (23:10 -0800)]
Revert D6366352: [folly] Split get_default() into two for deferred default construction

Summary:
This reverts commit db55b944ca63e565997094c11b90c4ebe98531ce

bypass-lint

Differential Revision: D6366352

fbshipit-source-id: e25906409186b077ef9117aa524cc7c86314ae12

4 years agoSplit get_default() into two for deferred default construction
Aaryaman Sagar [Mon, 20 Nov 2017 23:59:34 +0000 (15:59 -0800)]
Split get_default() into two for deferred default construction

Summary:
As it stood currently folly::get_default() would unnecessarily
construct a value into the third parameter, which was unnecessary in the fast
path where the element was found in the map

Reviewed By: yfeldblum

Differential Revision: D6366352

fbshipit-source-id: db55b944ca63e565997094c11b90c4ebe98531ce

4 years agocorrect usage of namespace std for coroutines_trait specialization
Dylan Yudaken [Mon, 20 Nov 2017 23:59:02 +0000 (15:59 -0800)]
correct usage of namespace std for coroutines_trait specialization

Summary: When specializing traits you should not use the libcpp inline std namespace, but rather the plain one.

Reviewed By: yfeldblum

Differential Revision: D6373787

fbshipit-source-id: d62ef9ccbf90c6f952db5f93f2377c920e68a809

4 years agoWork around a bug in MSVC name lookup within templated friend contexts
Christopher Dykes [Mon, 20 Nov 2017 21:44:02 +0000 (13:44 -0800)]
Work around a bug in MSVC name lookup within templated friend contexts

Summary: MSVC has a bug that causes it to think that `once_flag` in this context resolves to the `using` statement further up, rather than `detail::once_flag`.

Reviewed By: yfeldblum

Differential Revision: D6376501

fbshipit-source-id: 481dbd75ed21f3d519bd920258fa743f314668ad

4 years agoNo need for a wrapping structure for posixTimeToDuration
Yedidya Feldblum [Mon, 20 Nov 2017 21:29:48 +0000 (13:29 -0800)]
No need for a wrapping structure for posixTimeToDuration

Summary:
[Folly] No need for a wrapping structure for `posixTimeToDuration`.

We can just use a variant of tag dispatch. In this variant, we pass to `posixTimeToDuration` a default-initialized value of the desired return type and we write overload templates for each possible variant. The argument is used purely for overload resolution and return type deduction, not for its runtime value. It is slightly different from tag dispatch because we do not use separate types which are purely tag types.

Reviewed By: simpkins

Differential Revision: D6371572

fbshipit-source-id: 1987dee31fceec8733caa61495e96489dbf1ca39

4 years agoAdd free-function retire
Dave Watson [Mon, 20 Nov 2017 16:18:27 +0000 (08:18 -0800)]
Add free-function retire

Summary:
Adds a hazptr_retire and domain::retire methods to hazptr.  They both allocate memory instead of inheriting.

This will make implementation of google's cell proposal trivial, vs. D4754972 which felt overwraught.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0561r0.html

Reviewed By: magedm

Differential Revision: D6361162

fbshipit-source-id: 9f259f434139f960483b2ab7c5190d3807edcf52

4 years agofix SingletonTest v2017.11.20.00
Tianjiao Yin [Mon, 20 Nov 2017 02:51:19 +0000 (18:51 -0800)]
fix SingletonTest

Summary: There is no guarantee that this code will finish in 6 seconds (nor other threads will release singleton in 5 seconds), especially in ASAN mode. Though I don't have better idea, this diff will relax the condition in ASAN mode to make the unit-test less flaky.

Reviewed By: yfeldblum

Differential Revision: D6371692

fbshipit-source-id: 58dd15cc0b3273719314c8b323ba88ee47e8ff61

4 years agoimplement to() conversions for std::chrono to timespec/timeval
Adam Simpkins [Sun, 19 Nov 2017 23:14:20 +0000 (15:14 -0800)]
implement to() conversions for std::chrono to timespec/timeval

Summary:
Add folly::to() conversions to convert between std::chrono::duration or
std::chrono::time_point types and struct timespec or struct timeval types.

To conform to the behavior of the existing arithmetic-to-arithmetic
conversions, this code performs proper overflow checking and throws a
`ConversionError` on overflow.  This unfortunately does make the code rather
complicated compared to a non-checking implementation.

Conversions between some unusual duration types is not implemented yet, and
will fail at compile time if someone tries to use it.  This happens for
durations where neither the numerator nor the denominator of the ratio is 1.
For instance, 7/13ths of a second.

Reviewed By: yfeldblum

Differential Revision: D6356700

fbshipit-source-id: 9dce8ab8f32d8c18089f32c7176a8abf3c3f11f7

4 years agoget rid of redundant calls to RequestContext::saveContext()
Pingjia Shan [Sun, 19 Nov 2017 05:55:44 +0000 (21:55 -0800)]
get rid of redundant calls to RequestContext::saveContext()

Summary:
In the past, these calls were required.

Used to solve static destruction ordering issue.  Any static object that uses RequestContext must call this function in its constructor.

That is when we were using `static folly::ThreadLocal<std::shared_ptr<RequestContext>>`, which was non-leaky.

The problem being addressed is when we have some code of the form:

```lang=c++
void doWork() {
  static EventBase eb;
}
```

But now we are using `SingletonThreadLocal<std::shared_ptr<RequestContext>>`, which is leaky.

So the issue that these calls were there to address seems to have been resolved.

Reviewed By: yfeldblum

Differential Revision: D6332597

fbshipit-source-id: c6aba6620ef2fb3a344ea20f56c8b9c0cdf42c70

4 years agofolly: support FOLLY_FALLTHROUGH on GCC
Pádraig Brady [Sat, 18 Nov 2017 22:25:54 +0000 (14:25 -0800)]
folly: support FOLLY_FALLTHROUGH on GCC

Summary:
This is required to avoid new GCC 7 -Wimplict-fallthrough warnings.
We also update to use the C++17 [[fallthrough]] attribute if supported.

Reviewed By: yfeldblum

Differential Revision: D6367140

fbshipit-source-id: d5380983cb300f944df9c2885d0faa0155994be7

4 years agoreduce the number of iteration when test SharedMutexTest in ASAN mode
Tianjiao Yin [Sat, 18 Nov 2017 14:39:09 +0000 (06:39 -0800)]
reduce the number of iteration when test SharedMutexTest in ASAN mode

Summary: This unit-test takes too long to finish in ASAN mode from heavily loaded system (more than 10 minutes).

Reviewed By: yfeldblum

Differential Revision: D6362111

fbshipit-source-id: b097eff60f88ace4fb869132598806700804e267

4 years agoRemove unnecessary white line
Pingjia Shan [Fri, 17 Nov 2017 17:40:14 +0000 (09:40 -0800)]
Remove unnecessary white line

Summary:
Seems to be causing rendering error:

{F114885894}

Created from Diffusion's 'Open in Editor' feature.

Reviewed By: lskuff

Differential Revision: D6358188

fbshipit-source-id: 3c437007b425bcadb79a71807ad92d520a70e8cb

4 years agofolly/fibers/test/FibersTest.cpp: accommodate ASAN's detect_stack_use_after_return=1
Jim Meyering [Fri, 17 Nov 2017 04:52:38 +0000 (20:52 -0800)]
folly/fibers/test/FibersTest.cpp: accommodate ASAN's detect_stack_use_after_return=1

Summary:
With ASAN enabled (actually, only with ASAN *and* its detect_stack_use_after_return=1 option),
the addTaskFinally test would fail.  This adapts to accommodate the larger stack offsets.
Also, use EXPECT_GT and EXPECT_LT rather than EXPECT_TRUE.

Reviewed By: yfeldblum

Differential Revision: D6353666

fbshipit-source-id: 39e04caffa7b24cde97c749686c7e651a071dcec

4 years agoExpose the time remaining in HHWheelTimer::Callback
Jody Ho [Thu, 16 Nov 2017 07:23:02 +0000 (23:23 -0800)]
Expose the time remaining in HHWheelTimer::Callback

Summary:
We would like to know the time remaining for a scheduled timeout to decide
whether a new event should override the scheduled timeout.

Reviewed By: yfeldblum

Differential Revision: D6334067

fbshipit-source-id: f172d5cd7fc804db5fd53a42d06cadfddf857e22

4 years agoAdd unit test for timeout=0
Subodh Iyengar [Thu, 16 Nov 2017 01:59:20 +0000 (17:59 -0800)]
Add unit test for timeout=0

Summary:
Add unit test for immediate timeout since
we're using it in a few places.

Reviewed By: yfeldblum

Differential Revision: D6327012

fbshipit-source-id: ea80763d080b06e2a43277118d3147dc8016f348

4 years agoFix wrong source name in tests Makefile
Yedidya Feldblum [Wed, 15 Nov 2017 23:52:02 +0000 (15:52 -0800)]
Fix wrong source name in tests Makefile

Summary:
[Folly] Fix wrong source name in tests `Makefile.am`.

Closes #714.

Reviewed By: Orvid

Differential Revision: D6340209

fbshipit-source-id: 88da57fb4d663071eb578efec50c7e162b0c03d1

4 years agoallow small vector to be storage for sorted_vector_map
Benny Chen [Wed, 15 Nov 2017 22:03:27 +0000 (14:03 -0800)]
allow small vector to be storage for sorted_vector_map

Summary:
this is to allow small_vector to be a storage option for sorted_vector_map. Reas
on I want to do this is because in ads there are a lot of small maps where we wo
uld have to allocate separately.

Reviewed By: yfeldblum

Differential Revision: D6318811

fbshipit-source-id: b145d1bef2cbbeb946995aa66b55aaadeb6c54f5

4 years agoadd EXPECT_THROW_RE() and EXPECT_THROW_ERRNO() test macros
Adam Simpkins [Wed, 15 Nov 2017 20:48:56 +0000 (12:48 -0800)]
add EXPECT_THROW_RE() and EXPECT_THROW_ERRNO() test macros

Summary:
Add EXPECT_THROW_RE() and EXPECT_THROW_ERRNO() macros to folly/test/TestUtils.h
These allow more precise checks than the basic EXPECT_THROW() macro provided as
part of gtest.

These macros are being moved into folly from Facebook's eden repository
(https://github.com/facebookexperimental/eden)
This will allow us to use them in folly tests and in other projects that depend
on folly.

Reviewed By: yfeldblum

Differential Revision: D6301760

fbshipit-source-id: 1f434fb5bc9b7859f763171264fb0b2e1b4bda62

4 years agofix asan build
Igor Sugak [Wed, 15 Nov 2017 06:42:28 +0000 (22:42 -0800)]
fix asan build

Reviewed By: yfeldblum

Differential Revision: D6332075

fbshipit-source-id: 26a279b6ee4253a8be8f9f63e6900c2082a5486d

4 years agoExtra expectations for ADL toAppend
Yedidya Feldblum [Wed, 15 Nov 2017 06:39:18 +0000 (22:39 -0800)]
Extra expectations for ADL toAppend

Summary: [Folly] Extra expectations for ADL `toAppend`.

Reviewed By: Orvid

Differential Revision: D6330547

fbshipit-source-id: d1363280f097f860883ba84d6cfe7caa4e4cf9de

4 years agoMinor tweak to the wording of the vcpkg section
Christopher Dykes [Tue, 14 Nov 2017 22:01:08 +0000 (14:01 -0800)]
Minor tweak to the wording of the vcpkg section

Summary:
The commands as they were previously would attempt to install Folly as 32-bit, which would fail.
Also removes an extra `-` in the link's url.

Closes: https://github.com/facebook/folly/issues/713

Reviewed By: yfeldblum

Differential Revision: D6327981

fbshipit-source-id: ced8c86948b8be5c1dd88e14d6c3a77fc783aac9

4 years agoAdd ByteRange/StringPiece Conversion for Cython
Jason Fried [Mon, 13 Nov 2017 21:32:49 +0000 (13:32 -0800)]
Add ByteRange/StringPiece Conversion for Cython

Summary:
Moving the cython definition to folly/python.
Adding simple conversion helper to_bytes

This is to cut down on duplicate folly::range cython definitions

Reviewed By: yfeldblum

Differential Revision: D6291125

fbshipit-source-id: 314b732a1516a03fb5c9a57939552bbabd81970b

4 years agoFix an ICE in MSVC from functional/Invoke.h v2017.11.13.00
Andrew Krieger [Sun, 12 Nov 2017 22:31:02 +0000 (14:31 -0800)]
Fix an ICE in MSVC from functional/Invoke.h

Summary:
Similar to https://github.com/facebook/folly/commit/29ffcc50981fd50dd2ab1a69f8a262c4b7c27ad4,
(D6254219), this was manifesting as random ICEs that repro consistently in a given file on a given machine,
but in different files for different machines and sometimes not at all.

Reviewed By: Orvid, yfeldblum

Differential Revision: D6303186

fbshipit-source-id: d91ad633352fc5c28640e71fd1539f6a4ba8f70f

4 years agoFix some declared tests not being included in Makefile TESTS list
Yedidya Feldblum [Sun, 12 Nov 2017 00:16:59 +0000 (16:16 -0800)]
Fix some declared tests not being included in Makefile TESTS list

Summary: [Folly] Fix some declared tests not being included in `Makefile` `TESTS` list.

Reviewed By: pixelb

Differential Revision: D6307755

fbshipit-source-id: 8853d8fce25768ff17f9b1e53b1fbba0aa213691

4 years agoBackport C++17 container access functions: size, empty, data
Yedidya Feldblum [Sat, 11 Nov 2017 00:03:47 +0000 (16:03 -0800)]
Backport C++17 container access functions: size, empty, data

Summary: [Folly] Backport C++17 container access functions: `size`, `empty`, `data`.

Reviewed By: Orvid

Differential Revision: D6301986

fbshipit-source-id: 65c31df68b7743d5ed4a2cf2419586b862683c48

4 years agofix typo in Subprocess.h comments
Sergey Doroshenko [Fri, 10 Nov 2017 23:27:22 +0000 (15:27 -0800)]
fix typo in Subprocess.h comments

Summary: (Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D6299313

fbshipit-source-id: 95be2339f4845502e3c446698e54643eeac8055f

4 years agoMinor tweaks to the wording of a couple of errors in the CMake build
Christopher Dykes [Fri, 10 Nov 2017 20:28:17 +0000 (12:28 -0800)]
Minor tweaks to the wording of a couple of errors in the CMake build

Summary:
Tweak the wording of a couple error messages to make them clearer.
Also disable the 'digraphs not supported' warning, as none of our supported compilers actually interpret digraphs in any of our supported build configurations.

Closes: https://github.com/facebook/folly/issues/706

Reviewed By: yfeldblum

Differential Revision: D6299715

fbshipit-source-id: 7c847ac859e082aea711f6751f626b4b43886da4

4 years agofolly: fix sysMembarrier() with newer kernel headers
Pádraig Brady [Fri, 10 Nov 2017 03:55:39 +0000 (19:55 -0800)]
folly: fix sysMembarrier() with newer kernel headers

Summary:
When __NR_membarrier is defined (on newer kernels),
we don't define the constants, but also by default
don't include the membarrier header to define the necessary constants.
Therefore split the definition of the syscall value and
the constants used with it, to ensure both are defined.

Reviewed By: yfeldblum

Differential Revision: D6292178

fbshipit-source-id: 31ba9d4a698a4f5e14ae34de0acf8f851d75527d

4 years agofolly: avoid compile warning/failure due to lvalue-to-rvalue conversion
Pádraig Brady [Fri, 10 Nov 2017 03:26:16 +0000 (19:26 -0800)]
folly: avoid compile warning/failure due to lvalue-to-rvalue conversion

Summary:
With gcc 7.2 we get the warning:
  folly/io/async/DelayedDestructionBase.h:252:20:
  error: parameter ‘right’ set but not used [-Werror=unused-but-set-parameter]
     std::nullptr_t right) {
                    ^~~~~
I presume this is due to the implicit conversion, hence the named parameter is
never assigned.  Instead we use an explicit nullptr.

Reviewed By: yfeldblum

Differential Revision: D6279302

fbshipit-source-id: ed449601b0410c178777f20e82ed09d9097bd024

4 years agoadding a fibers compatible once flag
Shubhanshu Agrawal [Fri, 10 Nov 2017 01:35:30 +0000 (17:35 -0800)]
adding a fibers compatible once flag

Summary:
The current folly::once_flag is not compatible with folly fibers
and when using it with fibers is inefficient and also cause deadlocks.

This diff makes the once flag's mutex be a templatable paramter
and overrides it in fibers library with a fiber compatible mtuex.

Reviewed By: yfeldblum

Differential Revision: D6288508

fbshipit-source-id: 6f82e1794d1f417f8d267061f1702a26a7b4ff12

4 years agoAdd makeSemiFuture declarations to helpers.h
Yedidya Feldblum [Thu, 9 Nov 2017 21:34:35 +0000 (13:34 -0800)]
Add makeSemiFuture declarations to helpers.h

Summary:
[Folly] Add `makeSemiFuture` declarations to `helpers.h`.

For consistency with the `makeFuture` declarations that are also there. Definitions for both are found in `Future-inl.h`.

Reviewed By: LeeHowes

Differential Revision: D6281826

fbshipit-source-id: 4b22dd9086d05dbdebba358c6f569a772017949a

4 years agoMake ColdClassTest work on ancient compilers
Phil Willoughby [Thu, 9 Nov 2017 19:39:48 +0000 (11:39 -0800)]
Make ColdClassTest work on ancient compilers

Summary:
Some older versions of GCC/glibc/etc do not have the
std::is_trivially*_constructible or std::is_trivially*_assignable traits.

Reviewed By: yfeldblum

Differential Revision: D6285887

fbshipit-source-id: 1eb4ae4f899dc1f528321f9f087390291687aca3

4 years agoRemove the zerocopy write threshold support, add support for ENOBUFS
Dan Melnic [Thu, 9 Nov 2017 19:28:07 +0000 (11:28 -0800)]
Remove the zerocopy write threshold support, add support for ENOBUFS

Summary: Remove the zerocopy write threshold support since it is a little bit confusing

Reviewed By: djwatson

Differential Revision: D6256854

fbshipit-source-id: 1c992f93d7b04c4ede2fbefebde7a7ae89de3764

4 years agoAdd parsing for indirect functions
Teng Qin [Thu, 9 Nov 2017 06:11:33 +0000 (22:11 -0800)]
Add parsing for indirect functions

Summary:
Currently `folly::symbolizer`'s `getDefinitionByAddress` and `getSymbolByName` only parses `STT_OBJECT` and `STT_FUNC`. There are some standar library functions that uses the GNU indirect function feature that would have been missed:

  ==== For libpthread-2.23.so:
  ====== Symbol system Addr 119d0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol siglongjmp Addr 10700 Size 8 is a STT_GNU_IFUNC
  ====== Symbol longjmp Addr 10700 Size 8 is a STT_GNU_IFUNC
  ====== Symbol __vfork Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol vfork Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol system_ifunc Addr 119d0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol longjmp_ifunc Addr 10700 Size 8 is a STT_GNU_IFUNC
  ====== Symbol vfork_ifunc Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol siglongjmp_ifunc Addr 10700 Size 8 is a STT_GNU_IFUNC
  ====== Symbol __vfork_ifunc Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol __vfork@GLIBC_2.2.5 Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol siglongjmp@GLIBC_2.2.5 Addr 10700 Size 8 is a STT_GNU_IFUNC
  ====== Symbol vfork@GLIBC_2.2.5 Addr 10af0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol system@GLIBC_2.2.5 Addr 119d0 Size 8 is a STT_GNU_IFUNC
  ====== Symbol longjmp@GLIBC_2.2.5 Addr 10700 Size 8 is a STT_GNU_IFUNC

  ==== For libc-2.23.so:
  ====== Symbol __gettimeofday Addr c05e0 Size a8 is a STT_GNU_IFUNC
  ====== Symbol strcpy Addr 8e150 Size 35 is a STT_GNU_IFUNC
  ====== Symbol wmemcmp Addr afb50 Size 37 is a STT_GNU_IFUNC
  ====== Symbol strncmp Addr 8eb30 Size 41 is a STT_GNU_IFUNC
  ====== Symbol stpncpy Addr 929f0 Size 35 is a STT_GNU_IFUNC
  ====== Symbol __mempcpy_chk Addr 11cec0 Size 68 is a STT_GNU_IFUNC
  ====== Symbol strncpy Addr 903d0 Size 35 is a STT_GNU_IFUNC
  ====== Symbol time Addr c0500 Size a8 is a STT_GNU_IFUNC
  ====== Symbol strpbrk Addr 90700 Size 22 is a STT_GNU_IFUNC
  ====== Symbol strspn Addr 90a80 Size 22 is a STT_GNU_IFUNC
  ====== Symbol __stpncpy Addr 929f0 Size 35 is a STT_GNU_IFUNC
  ====== Symbol __strcasecmp Addr 92a80 Size 54 is a STT_GNU_IFUNC
  ====== Symbol memset Addr 92230 Size 41 is a STT_GNU_IFUNC
  ====== Symbol strstr Addr 916b0 Size 21 is a STT_GNU_IFUNC
  ====== Symbol strcspn Addr 8e270 Size 22 is a STT_GNU_IFUNC
  ====== Symbol memcmp Addr 91c40 Size 37 is a STT_GNU_IFUNC
  ====== Symbol mempcpy Addr 923b0 Size 68 is a STT_GNU_IFUNC
  And 80 more...
This Diff adds parsing for `STT_GNU_IFUNC` symbols as well

Reviewed By: yfeldblum

Differential Revision: D6282727

fbshipit-source-id: 71b7c44831e4ddfdccf1e794cb86e049e14227bc

4 years agoAdd a multi-type version of iterateSectionsWithType
Teng Qin [Thu, 9 Nov 2017 05:43:58 +0000 (21:43 -0800)]
Add a multi-type version of iterateSectionsWithType

Summary:
When using `folly::symbolizer`, it's very often that we want to use `iterateSectionsWithType` iterate through sections of a few types using the same callback. Current approach would require iterating the section header multiple times.

This Diff add `iterateSectionsWithTypes`, which is basically just `iterateSectionsWithType` but accepts multiple section types.

It is very similar to D6279651. However, in this Diff we did not change implementation of `getDefinitionByAddress` and `getSymbolByName`, since going through `.dynsym` separately would improve the efficiency of single-address or single-symbol lookup. However, for the use cases that we want to iterate through all symbols of an ELF file, this new interface would be useful.

Reviewed By: anakryiko, yfeldblum

Differential Revision: D6281449

fbshipit-source-id: f9afe0a0e95d9fafcd041014abad8ca86d1a882f

4 years agoFix folly/test/Makefile.am
Yedidya Feldblum [Thu, 9 Nov 2017 05:19:56 +0000 (21:19 -0800)]
Fix folly/test/Makefile.am

Summary:
[Folly] Fix `folly/test/Makefile.am`.

Closes #709.

Reviewed By: meyering

Differential Revision: D6282009

fbshipit-source-id: 0f2a992e92d4b94a535f29341920c9f2959819d7

4 years agoAdd a multi-type version of iterateSymbolsWithType
Teng Qin [Thu, 9 Nov 2017 04:04:18 +0000 (20:04 -0800)]
Add a multi-type version of iterateSymbolsWithType

Summary:
When using `folly::symbolizer`, it's very often that we want to use `iterateSymbolsWithType` iterate through symbols of a few types using the same callback. Current approach would require iterating the section multiple times.

This Diff adds `iterateSymbolsWithTypes`, which is basically just `iterateSymbolsWithType` but accepts symbol types.

This Diff also updated implementation of `getDefinitionByAddress` and `getSymbolByName` which currently does two iterations for `STT_OBJECT` and `STT_FUNC`.

Reviewed By: yfeldblum

Differential Revision: D6279651

fbshipit-source-id: a661dd15f18e4f2f63dbcca615f5a86d92e528ea

4 years agoIn SemiFuture members, fix DCHECK of pointer types
Yedidya Feldblum [Thu, 9 Nov 2017 02:55:00 +0000 (18:55 -0800)]
In SemiFuture members, fix DCHECK of pointer types

Summary:
[Folly] In `SemiFuture` members, fix `DCHECK` of pointer types.

Use `nullptr !=` to avoid ` error: ISO C++ forbids comparison between pointer and integer`.

Closes #708.

Reviewed By: Orvid

Differential Revision: D6277832

fbshipit-source-id: 8f65065d5347c6ac407b99cb780c38935e901362

4 years agoActually mark Unexpected as cold
Phil Willoughby [Thu, 9 Nov 2017 00:38:47 +0000 (16:38 -0800)]
Actually mark Unexpected as cold

Summary:
Testing indicates that GCC ignores the cold attribute when the function
is available for inlining. Because Unexpected is a template class we
can't make the constructors non-inline, but we can make it derive from a
class with a cold constructor, which has the effect of making all the
Unexpected constructors implicitly cold.

Reviewed By: yfeldblum

Differential Revision: D6261013

fbshipit-source-id: 482e49253d5b104742018133c53fb60279dd9f9b

4 years agoMark the base class of NoFutureInSplitter as public
Christopher Dykes [Thu, 9 Nov 2017 00:09:09 +0000 (16:09 -0800)]
Mark the base class of NoFutureInSplitter as public

Summary: Because every other exception is already marked as public.

Reviewed By: yfeldblum

Differential Revision: D6275414

fbshipit-source-id: 8e1fc4ceafedbdb44733ab57aecb6050c4160994

4 years agoFix static linking gflags and glog
Arkady Shapkin [Wed, 8 Nov 2017 19:30:38 +0000 (11:30 -0800)]
Fix static linking gflags and glog

Summary:
gflags and glog require some compile definitions when linking statically
/cc Orvid
Closes https://github.com/facebook/folly/pull/693

Reviewed By: yfeldblum

Differential Revision: D6264382

Pulled By: Orvid

fbshipit-source-id: 60c8af429f10e778d9c313f40227892238829f63

4 years agoHeterogeneous lookups for sorted_vector types
Yedidya Feldblum [Wed, 8 Nov 2017 17:22:17 +0000 (09:22 -0800)]
Heterogeneous lookups for sorted_vector types

Summary:
[Folly] Heterogeneous lookups for `sorted_vector` types.

When the `Compare` type has member type or alias `is_transparent`, enable template overloads of `count`, `find`, `lower_bound`, `upper_bound`, and `equal_range` on both `sorted_vector_set` and `sorted_vector_map`.

This is the protocol found in the equivalent `std::set` and `std::map` member functions.

> This overload only participates in overload resolution if the qualified-id `Compare::is_transparent` is valid and denotes a type. They allow calling this function without constructing an instance of `Key`.
>
> http://en.cppreference.com/w/cpp/container/set/count (same wording in all 10 cases)

Reviewed By: nbronson

Differential Revision: D6256989

fbshipit-source-id: a40a181453a019564e8f7674e1e07e241d5ab068

4 years agoAvoid implicitly dropping const modifier
Andre Pinto [Wed, 8 Nov 2017 16:30:36 +0000 (08:30 -0800)]
Avoid implicitly dropping const modifier

Summary: Use const_cast instead of implicitly dropping const modifier.

Reviewed By: reanimus

Differential Revision: D6269200

fbshipit-source-id: 61e1708c88a4139d3fdd9cf89f4ff778d0354bb2

4 years agoFix oss build
Andre Pinto [Wed, 8 Nov 2017 01:26:56 +0000 (17:26 -0800)]
Fix oss build

Summary:
Include folly/Portability.h in GenerateFingerprintTables, so that
gflags namespace gets defined.

Reviewed By: yfeldblum

Differential Revision: D6265226

fbshipit-source-id: d509dc163564420151de7007ad4336d7e5ef9625

4 years agofix -Wunused-variable
Igor Sugak [Tue, 7 Nov 2017 23:59:54 +0000 (15:59 -0800)]
fix -Wunused-variable

Summary:
Exposed by the latest clang:
```lang=bash
folly/test/FixedStringTest.cpp:52:36: error: unused variable 's' [-Werror,-Wunused-variable]
  constexpr folly::FixedString<42> s{"hello world"};
                                   ^
folly/gen/test/StringBenchmark.cpp:176:15: error: unused variable 'line' [-Werror,-Wunused-variable]
  StringPiece line(kLine);
              ^
```

Reviewed By: yfeldblum

Differential Revision: D6264853

fbshipit-source-id: 5cf4b0a8c99eaa31c2499746c70ddc49fdd73074

4 years agoWorkaround a bug in MSVC
Christopher Dykes [Tue, 7 Nov 2017 21:53:30 +0000 (13:53 -0800)]
Workaround a bug in MSVC

Summary:
It was broken, now it is fixed.
https://developercommunity.visualstudio.com/content/problem/145407/incorrect-c3520-within-noexcept-expression-in-alia.html

Reviewed By: yfeldblum

Differential Revision: D6254219

fbshipit-source-id: a03961db97d7ac211103655229c1ea703405826a

4 years agologging: set the thread name for the AsyncFileWriter thread
Adam Simpkins [Tue, 7 Nov 2017 20:11:23 +0000 (12:11 -0800)]
logging: set the thread name for the AsyncFileWriter thread

Summary:
AsyncFileWriter uses a separate thread to write messages to the file
descriptor.  This diff makes us call setThreadName() to set the name of this
thread to `log_writer`.

Reviewed By: bolinfest

Differential Revision: D6238229

fbshipit-source-id: 9c93d80e7ac989e03bc3160bb2f135d67e15c8be

4 years agoSupport movable keys
Dave Watson [Tue, 7 Nov 2017 18:27:36 +0000 (10:27 -0800)]
Support movable keys

Summary: Use the same trick as the values, so that non-copyable keys can be used in ConcurrentHashMap.

Reviewed By: yfeldblum

Differential Revision: D6252711

fbshipit-source-id: f0f168c4eb361d372bdfc3417f32222d66c11aaf

4 years agoRemove incorrect DCHECKS
Dave Watson [Tue, 7 Nov 2017 15:38:44 +0000 (07:38 -0800)]
Remove incorrect DCHECKS

Summary:
While the returned iterators are always 'valid' and can be read and iterated from,
they may still be concurrently erased from the map.  Current erase(iter) has DCHECKS that assert we can always
erase an iterator, but it may have already been removed.

Reviewed By: davidtgoldblatt

Differential Revision: D6221382

fbshipit-source-id: 70b21f53e2fc3daa126df4fb60bc5d3ecb253c71

4 years agoAllow to specify per-priority capacities in PriorityLifoSemMPMCQueue
Giuseppe Ottaviano [Tue, 7 Nov 2017 06:41:52 +0000 (22:41 -0800)]
Allow to specify per-priority capacities in PriorityLifoSemMPMCQueue

Summary: The `THROW` behavior of `LifoSemMPMCQueue` is unsafe when calling `join()`, because the queue may be full and `join()` will fail to enqueue the poisons. To work around this we can use `PriorityLifoSemMPMCQueue` and dedicate `LO_PRI` to the poisons, but there's no reason that the low priority queue should have the same size as the normal priority. Add a constructor to be able to specify different sizes.

Reviewed By: yfeldblum

Differential Revision: D6257017

fbshipit-source-id: c75f33c38fcdad646ba1499bcd434ab65711250c

4 years agocrange, and range const overloads
Yedidya Feldblum [Tue, 7 Nov 2017 01:29:12 +0000 (17:29 -0800)]
crange, and range const overloads

Summary:
[Folly] `crange`, and `range` `const` overloads.

Instead of using universal reference for `range` overloads, bifurcate explicitly between `&` and `const&` overloads. The `&` overloads return `Range<T*>` while the `const&` overloads return `Range<T const*>`.

Add `crange` overloads, which may accept non-`const` arguments but will return `Range<T const*>` results anyway.

Reviewed By: ot

Differential Revision: D6242038

fbshipit-source-id: bc373c3288ea88792f04b49a372262d12204b586

4 years agoMove folly/LifoSem.h
Yedidya Feldblum [Mon, 6 Nov 2017 23:22:31 +0000 (15:22 -0800)]
Move folly/LifoSem.h

Summary: [Folly] Move `folly/LifoSem.h` to `folly/synchronization/`.

Reviewed By: meyering

Differential Revision: D6245444

fbshipit-source-id: 14ffa012fa92b8c6aaf5900c930156894a492003

4 years agoValue-initialize the Data union member of folly::Function
Eric Niebler [Mon, 6 Nov 2017 19:42:48 +0000 (11:42 -0800)]
Value-initialize the Data union member of folly::Function

Summary: The Data field was not getting value-initialized, leading compilers to complain when value-initializing const folly::Function objects.

Reviewed By: yfeldblum, ot

Differential Revision: D6241712

fbshipit-source-id: 99ce7f6016f6e7d16b1cff7aa51b7bef53ec592d

4 years agoMark constructing an Unexpected as cold
Phil Willoughby [Mon, 6 Nov 2017 17:06:27 +0000 (09:06 -0800)]
Mark constructing an Unexpected as cold

Summary:
By marking the main constructors of `Unexpected` cold we are telling the compiler that we don't expect to execute them often. This can improve the locality and performance of the `Expected` branch, and reduce the code-size for the `Unexpected` branch.

The constructors we have marked cold here are only the ones used to originate an `Unexpected`. We don't mark the copy- move- or conversion-constructors as `cold` because they will typically be used in error-handling branches and it is not correct to say that they are colder than their surroundings in that case. Note that the entire error-handling branch is likely to be deduced to be cold because we told the compiler that it is likely that hasValue() is true and hasError() is false.

Because `makeUnexpected` unconditionally invokes one of these cold constructors it will inherit the property of being cold.

The GCC function attribute reference, which describes what `cold` means, is here: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes

Reviewed By: yfeldblum

Differential Revision: D6234305

fbshipit-source-id: 6876073e92ce54936ba7152175d76be653f5f463