Programming Languages Research Group: Git

copy wangle back into folly

Summary: copy everything but example/

Test Plan: fbconfig -r folly/wangle && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100811

Tasks: 5802833

Signature: t1:2100811:1432678173:6c336fe53aa223993f6f82de4ac91b3c19beacf1

move folly/wangle to wangle

Summary: this preserves history and will be followed by a copy diff

Test Plan: next diff will compile

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100784

Tasks: 5802833

Signature: t1:2100784:1432677219:838a336d84918145d19c6cfa4290dd58374821fd

removing non-existing file from the build

Summary: broken by D2096146

Test Plan: arc unit

Reviewed By: andrewcox@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2100773

Signature: t1:2100773:1432676073:a72e72fa5fb915b027fdfd31dd814d02e6606858

cleaning up RequestContext

Test Plan: built folly

Reviewed By: mshneer@fb.com

Subscribers: davejwatson, mshneer, trunkagent, alandau, bmatheny, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2096146

Tasks: 6337419

Signature: t1:2096146:1432658670:4ff00e4274412519dc3b02d2a849b7ff289ac979

Bump version to 40:0

remove spurious warning

Summary: see task

Test Plan: OK

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2099416

Tasks: 7166291

Signature: t1:2099416:1432658571:4ae632c218c4764f23e4b189167f6fdaff13dc2d

Some optimizations

Summary:
1. Eliminate some string -> StringPiece -> strings conversions
2. Mcrouter: eliminated unnecessary inlining by moving slow path logic into its own method.

Using a test setup with shadow sampling enabled and shadowing some requests,
(typical prod setup), this brings down the cost from ~1.4% cpu in standalone mcrouter to ~0.2%:

```
before:

+ 0.70%          3898  mcrouter_orig  mcrouter_orig  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.13%           864  mcrouter_orig  mcrouter_orig  [.] EventGroup<ScubaRow>::processExtraSamplers
+ 0.58%          3347  mcrouter_orig  mcrouter_orig  [.] DynamicScubaSampler::getSampler

~ 1.41% total

after:

+ 0.18%          1223  mcrouter_fix  mcrouter_fix  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.04%           205  mcrouter_fix  mcrouter_fix  [.] EventGroup<ScubaRow>::processSampler

~ 0.22% total
```

Fiber local optimization might have more of an effect.

Test Plan:
unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, alikhtarov, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2089133

Tasks: 5414865

Signature: t1:2089133:1432338487:4158dc6b720c04f43820193e73b98d4197afcffa

move FiberIOExecutor to wangle/concurrent

Summary: Need to cut this dep on wangle out of folly as we're moving wangle to its own repo

Test Plan: chimera unit

Reviewed By: rushix@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2092122

Signature: t1:2092122:1432239179:9261c1b3a3a346b886b15899d25a4d0530d4e890

Revert "Make strlcpy available in folly" (D2062632)

Summary:
Reverting diff to fix the build.

Test Plan: contbuild rule that failed to build is okay now

Reviewed By: ldbrandy@fb.com

Make strlcpy available in folly

Summary:
strncpy is bad. strlcpy is somewhat less bad. We already had
this function, but let's move it somewhere more reasonable.

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: trunkagent, lins, anca, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2062632

Signature: t1:2062632:1431969926:cc7f7283073d0242fe8f361efac2557aa0b0a481

remove unused variable

Test Plan: search, and fbmake

Reviewed By: delyank@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093523

Signature: t1:2093523:1432249208:84a27e2328636c2aafe43412f61a693b5645b895

folly/experimental/fibers/EventBaseLoopController.h: include <atomic>

Summary: Without <atomic>, this fails to compile with clang:dev and newer glibc.

Test Plan:
fbconfig --clang --with-project-version=clang:dev --with-project-version=llvm_compiler_rt:dev --platform-all=gcc-4.9-glibc-2.20 --sanitize=address mcrouter/lib/network:client-test-sync && fbmake dbg

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093709

Signature: t1:2093709:1432250378:2b78c142598bf0a7b0869089c752f5ccdae9da22

Refactor tests to verify that a single folly target can be used successfully

Summary:
This includes a change to Range to move operator<< into the
header, to achieve the goal. Specifically, see format_test,
dynamic_test, json_test, demangle_test

Test Plan: fbconfig -r buck && fbmake runtests

Reviewed By: njormrod@fb.com

Subscribers: darshan, tjackson, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2063698

Signature: t1:2063698:1431467309:069da6d74bb5c384e7a21e6be19a4b20466bdd92

folly refactorings to better support cross-platform

Summary:
I'm looking into pulling parts of folly (right now, dynamic,
json, and their dependencies) into fbandroid for use as part of xplat.
This diff includes a few kinds of changes: portability fixes for arm;
reduce the size of the code generated by moving non-templated
functions and methods into cpp files; refactoring header usages which
require extra compiler flags on android to cpp files; and slicing up
the libraries a bit differently to reduce dependencies. This should
all be backward-compatible, and do no harm to fbcode.

Test Plan: runtests, sandcastle

Reviewed By: njormrod@fb.com

Subscribers: darshan, davejwatson, tudorb, dancol, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2057797

Tasks: 7005344

Signature: t1:2057797:1432145435:fa10f129fc669e682da5b4b207fc96986ca035fc

fixup service filter interface

Summary:
Based on a more thourough reading of finagle's interface:
* adds close/isAvailable, which seem very close to thrift's interfaces
* ComposedServices are hardcoded to underlying services, to simplify the code (means extra allocs?)
* Made everything a shared_ptr
* Addd ServiceFactoryFilters

Test Plan: Updated the existing unittests and added some new ones

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037206

Signature: t1:2037206:1432147489:3464d4c12a9e434d4973febcabbf7b2b3a883257

tupleRange, tuplePrepend

Summary:
tupleRange<start, n>(tuple): return a tuple consisting of a range of elements
from the given tuple

tuplePrepend(x, tuple): return a tuple consisting of prepending x to the given
tuple.

For Lispies:

std::get<0>(tuple) is car.
tupleRange<1>(tuple) is cdr.
tuplePrepend(x, tuple) is cons.

Test Plan: test added

Reviewed By: lesha@fb.com

Subscribers: trunkagent, lesha, ilyam, folly-diffs@, yfeldblum, chalfant, jhj, alerer, kma, pamelavagata, tulloch

FB internal diff: D2087568

Signature: t1:2087568:1432164681:18795d0e8bb01f38ffc6949ac233f514ab098355

(Wangle) Fix typo

Summary:
This was supposed to be the `Result` type, since it's called on the
Future returned by the lambda.

Test Plan: Added tests for void and different types in vector/lambda.

Reviewed By: mhl@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2087819

Tasks: 7126300

Signature: t1:2087819:1432142435:72914fa64eff03454774b87a24c426379defab3b

Blame Revision: rFBCODEf229322bc273190a85b5e995dcd8209b1fbf0825

Bump version to 39:0

make AsyncSocket::WriteRequest an interface

Summary: This will allow a subsequent diff to implement file transfers as another type of write request

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant, fugalh, bmatheny

FB internal diff: D2080257

Signature: t1:2080257:1432044566:bcc0724d349879f46e3e58ee672aff7bf37fa5f6

Make AtomicHashMap shadow declaration clean

Summary: Some of AtomicHashMap's locals shadow member functions - rename the locals to fix the warnings

Test Plan: unit tests

Reviewed By: chaoc@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@

FB internal diff: D2086270

Signature: t1:2086270:1432083900:fae1be39e55e4c30b47fdc7a069bb13d75292b0a

Make Subprocess movable

Summary:
Subprocess doesn't have any non-movable members, and its implementation does not take addresses of the object, so I think it's safe. Move makes a bunch of code cleaner (you no longer have to wrap it in `std::unique_ptr` with associated clumsiness).

https://phabricator.fb.com/diffusion/FBCODE/browse/master/folly/Subprocess.h

Test Plan:
- unit test
- Searched for `this` in `Subprocess.{h,cpp}`.
- Inspected member variables: `pid_`, `returnCode_`, `pipes_`
- contbuild

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2079167

Signature: t1:2079167:1432048688:26f96e29310298f47a9a9a7abef22dc863f68942

Koans to the Future

Summary:
The first file of Koans. Some simple constructor and `makeFuture` stuff, but this diff is mostly about the framework (ie the `TARGETS` and `main.cpp` and `Koan.h`, and the layout of the Koan files).

Known Issues: I am not nearly enlightened enough to write these in a particularly zen style with lots of inside zen-jokes, so I'm not even trying.

Test Plan:
Work through the koans,
then `fbmake runtests`,
reach enlightenment

Reviewed By: davejwatson@fb.com

Subscribers: cgthayer, exa, folly-diffs@, jsedgwick, yfeldblum, bmatheny, chalfant

FB internal diff: D2082141

Tasks: 6973057

Signature: t1:2082141:1432057657:273708f566154cc54f726b85f05457388357ef4e

Add getTotalCount() to Histogram

Summary:
Add a getTotalCount() method to the Histogram class so that callers
can read out the number of values that were put into the histogram

Test Plan: Added a new test.

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, evanmao, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2078239

Tasks: 7097793

Signature: t1:2078239:1431739799:5de3a02df67ff535e06b4a1547690440cf594824

Add 'runnable' callback to ExecutionObserver

Summary: Add a callback when a fiber becomes runnable

Test Plan: unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2081306

Signature: t1:2081306:1432011152:0ee93cb2682eb2a289b99c403e91465e72dd4ee8

framing handler pipeline stage

Summary: last pipeline in the original diff (HeaderServer/Client channel can also become pipeline stages in the future...)

Test Plan:
fbconfig -r thrift/lib/cpp2; fbmake runtests

canary results will follow in a comment

Reviewed By: alandau@fb.com

Subscribers: fugalh, folly-diffs@, chalfant, trunkagent, doug, alandau, bmatheny, jsedgwick, yfeldblum

FB internal diff: D2033559

Signature: t1:2033559:1430417432:c6cf4ccbf9ef26d89e7d7c5955d103348205b365

RFC: FutureDAG

Summary:
See task. Set up a DAG of Future-returning tasks (optionally with executors) and eventually kick them off.
One big question is ownership. Currently the user would be responsible for ensuring that the FutureDAG outlives its own completion. This requirement could go away with shared_from_this magic maybe

Test Plan: unit. I didn't bother to test via() functionality because it's too much work for now - the functionality is trivial. Same for "true-async" dags...

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2073481

Signature: t1:2073481:1431961131:82a8898502d5308f6ab3cc8cc5b84b016d3998fe

Make folly::format no longer crash on invalid format strings

Summary:
Yes, ideally we'd detect this at compile time, but if we can't, causing a SEV1
is not the best way to do it.

format() now behaves like formatChecked(); the old API is maintained for
backwards compatibility, but deprecated.

Test Plan: format_test

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, dpittman, jdelong, sdmich, net-systems@, dmitri, folly-diffs@, yfeldblum, andrii, mssarang, chalfant

FB internal diff: D2075829

Tasks: 7095768

Signature: t1:2075829:1431713985:b3fdec4820104b4ddc4be0b6af999db174a692d9

Fix race issue in EventBase::runInEventBaseThreadAndWait (t6779663)

Summary: Huge thanks to @afrind for debugging this issue with me and found the root cause. As per the comment from @afrind for the diff D1823407, there is a tricky race issue. The main thread could have left and reset the condition_variable from its stack but the EventBase thread tries to access it afterwards due to race and could be blocked indefinitely. This caused the server-side IO threads not able to pick up the incoming connections for the proxygen case. The fix is to use a simpler struct barrier and get a hold of the shared_ptr instead of the same object in a safer way.

Test Plan:
The original issue reproes very easily in HDFS XDC encryption case. Servers easily enter into bad state and we got high volume of timeouts from the client. With the fix, this does not happen anymore after the fix being deployed at 11:15PM. Here is the Scuba log before and after the fix:
https://fburl.com/109969805
And here is the correspond Scuba diagram for successful calls in the same test:
https://fburl.com/109971729
The throughput improved a lot after the fix.

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant, afrind, brettp, dougw, fma

FB internal diff: D2071646

Signature: t1:2071646:1431709609:10fb033536f9e4fb428dea8ba68f6a9a051616c0

fix clang build

Summary: PipelineBase needs something virtual so we can dynamic_cast from it down to the real pipeline, as we do in Proxy

Test Plan: built proxy with clang

Reviewed By: hans@fb.com

Subscribers: fugalh, mathieubaudet, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2075855

Tasks: 7077419, 7077417

Signature: t1:2075855:1431708780:03ce8d8f40cdb56d24baf75f4dc29004e5ea4c7a

(Wangle) unorderedReduce

Summary:
Use this if you don't need the order of the input, e.g. summing up
values. This constructs a separate Future chain to do the reducing,
because we don't want to add locking while reducing. The only lock
necessary is when adding a new Future to the chain, which should be
really quick.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015326

Tasks: 6025252

Signature: t1:2015326:1431557191:9ea2edccb0162dedf067b5b3300de2fe72a1a4c9

(Wangle) chain -> thenMulti + thenMultiWithExecutor

Summary:
If we make `chain` a member function we can avoid the type issues and infer everything. I also added thenMulti for symmetry. Sadly the compiler doesn't like having a thenMulti with an optional `Executor*` as the first argument, it fails after some deductions. Hence `thenMultiWithExecutor`.

itssobeautiful

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2021000

Signature: t1:2021000:1431557618:169447dc9d747b23a8a1ba830e78c43713d09a96

(Wangle) window

Summary: `window` creates up to `n` Futures at a time and only starts new ones when previous ones complete. A sliding window.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: bmatheny, henryf, scottstraw, juliafu, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015310

Signature: t1:2015310:1431557556:1017006cc9c9c2562ebe2c3dabfc4dbf316ff408

via with priority

Summary:
I wish I could just have an add(Func, priority) but the damned overloaded virtual warnings become a nightmare, so it's addWithPriority.
I also switched priority to a uint8_t in the hopes of reducing Core size. Turns out std::atomic<uint8_t> is 8 bytes anyways :( I left it that way because come on you really shouldn't be using > 256 priorities.
Biggest problem is the data race with the two atomics executor_ and priority_. Should we just use a microspinlock to co-protect them? Could probably save some size from the atomics that way.

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: hannesr, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2039619

Tasks: 6928162

Signature: t1:2039619:1431551266:3b31ed2329301aaa9c32f0f41b6e61f3482d570e

transportActive and transportInactive

Summary:
These are equivalents to Netty's channelActive and channelInactive, but we've been calling channels transports so I'm staying consistent.
I skipped integrating this into TAsyncTransportHandler because thrift still does manual CB attachment/detachment and it's unclear how that fits into this model
If my suspicions are correct, it *should* be possible to make attachReadCallback and detachReadCallback private in AsyncSocketHandler, right? And perhaps get rid of the event base modifier methods? What's our use case for those?

Test Plan: unit, employ in telnet server

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2044520

Signature: t1:2044520:1431551998:af1de358b5dbefcca148814015d8e9f63f458d5d

more restrictive implicit Future construction enabling

Summary:
Decay so we don't try to instantiate this for attempts to copy Futures
See https://www.facebook.com/groups/499316706783616/permalink/863260220389261/

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: hannesr, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2062442

Signature: t1:2062442:1431551169:d1ba61537c998067ee7e6f4819f7e0817cc2e700

Disable stack recording with ASAN

Summary: ASAN crashes on std::fill and doesn't respect no_sanitize_address attribute for some reason.

Test Plan: run fibers-test with ASAN

Reviewed By: stepan@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069437

Signature: t1:2069437:1431547972:7d2c7a6547f8d76b309a76ef69fd19a1de4ce261

makeTryFunction -> makeTryWith codemod

Summary: This is more consistent with setWith, makeFutureWith, etc

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2064667

Signature: t1:2064667:1431541614:a0e3f23d5effde13a93ce58ca3e21c7c3575215c

folly: fix mismatched-tags

Summary:
I tried to fix this for default glibc but there are many mismatched-tags inside glibc itself. gcc-4.9-glibc-2.20 does not have such issue, using it.

```lang=bash
% fbconfig --clang --with-project-version clang:dev --extra-cxxflags=-Wmismatched-tags --extra-cxxflags=-ferror-limit=0 --platform-all=gcc-4.9-glibc-2.20-fb -r folly
% fbmake dev -j55 2> err
% perl -n -E 'm/\[-Werror,-Wmismatched-tags\]$/ && print' err | sort -u
./folly/experimental/fibers/TimeoutController.h:52:3: error: 'TimeoutHandle' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
folly/experimental/fibers/TimeoutController.h:52:3: error: 'TimeoutHandle' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
folly/experimental/JSONSchema.cpp:65:10: error: class 'ValidationContext' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/futures/detail/Core.h:76:1: error: 'Core' defined as a class template here but previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Future.h:392:10: error: class 'Promise' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/futures/Future.h:45:1: error: 'Future' defined as a class template here but previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:137:1: error: struct 'Timekeeper' was previously declared as a class [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:23:18: error: struct template 'Promise' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Future-pre.h:43:18: error: struct template 'Core' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Promise.h:26:20: error: class template 'Future' was previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/futures/Timekeeper.h:23:18: error: struct template 'Future' was previously declared as a class template [-Werror,-Wmismatched-tags]
./folly/futures/Timekeeper.h:44:1: error: 'Timekeeper' defined as a class here but previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/Singleton.h:378:10: error: class template 'SingletonHolder' was previously declared as a struct template [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLCacheOptions.h:17:1: error: 'SSLCacheOptions' defined as a struct here but previously declared as a class [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLContextManager.h:29:1: error: class 'SSLCacheOptions' was previously declared as a struct [-Werror,-Wmismatched-tags]
./folly/wangle/ssl/SSLContextManager.h:32:1: error: class 'TLSTicketKeySeeds' was previously declared as a struct [-Werror,-Wmismatched-tags]
% perl -n -E 'm/\[-Werror,-Wmismatched-tags\]$/ && print' err | sort -u | wc -l
16
```

Updated manually. In all cases preferred tag from definition.

Test Plan:
Compile with clang dev and gcc-4.9-glibc-2.20-fb and see fewer errors:
```lang=bash
% fbconfig --clang --with-project-version clang:dev --extra-cxxflags=-Wmismatched-tags --platform-all=gcc-4.9-glibc-2.20-fb -r folly
% fbmake dev -j55
```

Reviewed By: markisaa@fb.com, meyering@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2066327

Signature: t1:2066327:1431471232:c65c2827398ba29a4022cc6a5647fac2b3aad717

SharedPromise in OutputBufferingHandler

Summary:
as above. I'm torn on whether to sugar "*this = SharedPromise<T>" as SharedPromise<T>::reset()
If I see another use case I'll probably do it

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2064449

Signature: t1:2064449:1431476780:7113366b11feaf9e8a4ea1dc60fbafb36dd46ac5

SharedPromise

Summary: I tried two "smart" ways (deriving from Promise, encapsulating a Promise) and got nothing but trouble. The KISS principle is applied with gusto in this diff.

Test Plan: unit, integrating in 3+ places in separate diffs

Reviewed By: hans@fb.com

Subscribers: craffert, trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2035528

Signature: t1:2035528:1431393438:4e554cd30fa531d75b9267dccaade6dc516f2b15

expose FunctionScheduler::addFunction() with a custom distribution

Summary:
Make addFunctionInternal() publicly available, as an overloaded version of
addFunction(). This allows users to add functions with a specified poisson
distribution.

This allows us to deprecate our internal legacy version of FunctionScheduler,
and replace it with the folly version.

Test Plan: Confirmed all unit tests still pass.

Reviewed By: ldbrandy@fb.com

Subscribers: jwatzman, doug, net-systems@, exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051699

Signature: t1:2051699:1431379841:f3547d1ed371503b0bf91b509a4ef03e881aa991

close idle HTTPDownstreamSessions before load shedding

Summary:
when any system resource limit is reached, proxygen reduces the number of idle downstream sessions to accomodate new ones.

Test Plan:
canarying on edge241.01.ams3. Here is the number of idle connection closed during pre load shedding stage.

https://www.facebook.com/pxlcld/ml7J

Reviewed By: afrind@fb.com

Subscribers: alandau, noamler, fugalh, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant, xning, alexkr

FB internal diff: D2030988

Tasks: 5698711

Signature: t1:2030988:1431369559:ce7328d51c7fd0afa7e9e5c19b0c66736d01fee1

Bump version to 38:0

Increase fibers stack size if we're running with ASAN

Summary:
In most cases user is not aware of ASAN and fiber stack problem, thus if the stackSize is specified we're not detecting ASAN and probably will crash.
And thus it doesn't seem like a good idea to make it a user responsibility to detect ASAN.

Test Plan: tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2058741

Tasks: 7016680

Signature: t1:2058741:1431131082:9a41eb40d756c9c7af0632f7ecd55c17d10bb189

(Wangle) Implement collect* using mapSetCallback and shared_ptrs

Summary:
I figured it would make sense to implement all the collect* functions using a shared_ptr<Context>, instead of doing our manual reference counting and all that. Fulfilling the promise in the destructor seemed like the icing on the cake. Also saves some line of code.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015320

Signature: t1:2015320:1431106133:ac3001b3696fc75230afe70908ed349102b02a45

Add SpookyHashV2-based hash to StringPiece

Summary: Since the old one is weak and slow.

Test Plan: Unit tests

Reviewed By: ott@fb.com

Subscribers: trunkagent, maxime, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2052630

Tasks: 6998080

Signature: t1:2052630:1431015271:bc90ccf99941902cd4bd43a0980238c616e66abf

Factor out pipe methods, add takeOwnershipOfPipes

Summary:
In order to create an EventBase'd Suprocess class, I'd like to be able to manage the lifetime of pipes independently of the lifetime of the process. To this effect, I factored out basic Pipe handling, and provided a function that detaches the pipe vector from the Subprocess object.

#6996492 a push-blocking test is broken in trunk

Test Plan: added a unit test, fbconfig -r folly && fbmake runtests && fbmake runtests_opt

Reviewed By: dancol@fb.com

Subscribers: yfeldblum, chalfant, dancol, wez, anarayanan, trunkagent, net-systems@, njormrod, folly-diffs@

FB internal diff: D1699969

Signature: t1:1699969:1430975299:30d291ab7fcc555edddf098b33095a5b29500e76

Destructor DCHECKs for EBADF

Summary: This almost always indicates a double-close bug, or a similarly nasty logic error. I don't dare make it a check, we may have some code running in production which would be broken by the CHECK, but this should give us early warning of any such bugs.

Test Plan:
```
fbconfig folly/test:file_test folly/test:file_util_test
fbmake runtests && fbmake runtests_opt
```

This test used to pass until @tudorb made me remove it due to worries about death tests messing with (currently absent) multi-threading:

```
+
+void testDoubleClose() {
+  File f("/dev/null");
+  checkUnixError(close(f.fd()));  // This feels so... wrong!
+  // The destructor will now try to double-close.
+}
+
+TEST(File, DCHECKDoubleClose) {
+#ifndef NDEBUG
+  // This test makes no sense otherwise, since this is a DCHECK.
+  EXPECT_DEATH(testDoubleClose(), "double-close-FD");
+#else
+  // That sound you hear is millions of lemmings falling to their doom.
+  testDoubleClose();
+#endif
+}
```

Reviewed By: tudorb@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2055610

Signature: t1:2055610:1431048270:a469d5c1f8182ffb74700908faa022e9613ed383

update FunctionScheduler to use std::chrono::steady_clock

Summary:
Use std::chrono::steady_clock instead of clock_gettime(CLOCK_MONOTONIC).
In particular this fixes the build on Mac OS X, which doesn't have
CLOCK_MONOTONIC.

This also updates the code to use steady_clock::time_point correctly, instead
of using a raw milliseconds value for time since the epoch.

Test Plan:
Included unit tests, which were copied over from the legacy internal Facebook
(non-folly) version of this code.

Reviewed By: ldbrandy@fb.com

Subscribers: jwatzman, doug, fbcode-common-diffs@, net-systems@, exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051557

Signature: t1:2051557:1431019654:ee76cfcf8318cc3d8a8d1522b3fc97f08831ecf4

Tweak explicit std::max instantiation

Summary: Per discussion on D2040509 this is better.

Test Plan: Still builds.

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2051099

Signature: t1:2051099:1430949575:cc167b57f2d6ff42a73dee4e65d22d04932bb279

Blame Revision: rFBCODE9bdb427be1ef80b612e4f364db7809c6351cfe1c

OS X doesn't support constructor priorities

Summary:
OS X doesn't support constructor init priorities, at all. AIUI,
it's a limitation of their actual binary format and loader, not just a
tooling/compiler limitation.

This particular usage appears to just be for for logging/bug-finding
purposes, so it looks like just removing the priority on OS X isn't the
end of the world?

Test Plan: g++-4.9 on OS X compiles this file now.

Reviewed By: njormrod@fb.com

Subscribers: ldbrandy, jdelong, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2040557

Signature: t1:2040557:1430975025:73f817b5d19a18dca6b19ba783dbea99192cbc41

Add missing #include <gflags/gflags.h>

Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2042952

Tasks: 6783581

fix namespace v2

Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: yfeldblum@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2043151

Tasks: 6783581

Explicitly template instantiate `std::min/max` in a couple places

Summary:
This is for OS X; there is apparently some difference between
its types and Linux's typical types, causing incomopatibilities between
a `long int` and a `long long int`. These two explicit template
instantiations fix the issue.

Test Plan: g++-4.9 on OS X compiles these files now.

Reviewed By: lucian@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2040509

Signature: t1:2040509:1430516624:9db8146b7824c0d09bac418a10a5f54451cdd5db

Move FOLLY_DISABLE_ADDRESS_SANITIZER from Fiber::init()'s definition to declaration

Summary:
I just had topaggr built from trunk crash with https://phabricator.fb.com/P19825084. It seems that
placing `FOLLY_DISABLE_ADDRESS_SANITIZER` next to the function definition has no effect. Though
what's strange, https://phabricator.fb.com/D2036280 test plan says the false-positive crashes
stopped after that diff. Have we run the tao script on the latest revisions of that diff? Anyway,
clang/gcc docs clearly say the attributes belong to declarations, not definitions, so let's do that.

Test Plan:
Compiled, unit tests.

Reviewed By: aap@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, soren

FB internal diff: D2042149

Signature: t1:2042149:1430681824:aea21b9a1fcddfd5585180ba7f526c3a6ee0c8bf

Telnet client

Summary:
A client example to match telnet server.

Required a couple additions:
* future result when socket actually connects, similar to netty
* clients support IOThreadPoolExecutor groups
* a pipeline stage to make sure everything runs in the right eventbase thread.

Test Plan:
fbconfig follg/wangle/example/telnet && fbmake dbg
telnet_server
telnet_client

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2010289

Signature: t1:2010289:1430766232:65c6f946e454000f6ea9f41b49197ddbeea5ba3f

telnet server

Summary: similar to https://github.com/netty/netty/tree/master/example/src/main/java/io/netty/example/telnet

Test Plan:
fbconfig folly/wangle/example/telnet; fbmake dbg
_bin/folly/wangle/example/telnet_server --port=8080
telnet localhost 8080

Still a couple sharp edges:
* No easy way to wait for ServerBootstrap termination.
* Pipelines always have to call attachReadCallback
* a bunch of missing methods in pipeline still, like channelActive

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959172

Signature: t1:1959172:1427993978:463f237036996451187e3ef3983cf2b4e89685ef

Split HandlerContext and Pipeline into inl headers

Summary:
Leave the important headers visible. Opens the door for decent inline docs.

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036799

Signature: t1:2036799:1430749004:db5e58655c27b96826549849722fe962b9ae3985

explicit instantiation of common Future types

Summary:
Compiling folly/futures:futures-test, this saves 15% (dbg) and 7% (opt)
Compiling all of folly/futures, it's 7% ish for each

Main blocker right now is that this generates a spew of deprecated warnings from calls to core_->(de)activate() from Future::(de)activate(). Can the deprecations be moved up to the Future methods instead?

Also had to fix willEqual for Future<void> which was borked

Test Plan: compiles

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2021028

Signature: t1:2021028:1430749114:1dd78af47ea91aa5e67929a5884b66ca0c8ae2d8

Updated string.h

Summary:
@markisaa, @ldbrandy: stdarg.h is for ellipses arguments, ##...##, which this file does contain.
The pull request indicates that this header is required for windows. Given that adding it doesn't hurt, and should be there anyways, I'd say go on this pull request.

Fixed a fatal error when compiling folly for HHVM

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, ldbrandy, markisaa

FB internal diff: D1999557

Tasks: 6783581

Signature: t1:1999557:1429220813:0580882b8a55d86f51cdd602d2d152920d368b75

Disable ASAN for Fiber::init()

Summary:
Fiber::init() method modifies all the bytes in the fiber's stack in
order to produce a high watermark of stack memory usage. This behavior
generates ASAN errors, as it modifies stack red zones. For that reason,
ASAN is disabled for that particular method.

Test Plan:
Unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2036280

Tasks: 6861962

Signature: t1:2036280:1430439550:e156e0e06fae3615d9d9204e99acaf3072bbfafd

(Wangle) Fix Executor problem

Summary:
None of these functions should be templated with `class Executor`.
Except `then(Executor, Args...)` because otherwise the compiler gets
confused. This was the combination that worked for both Clang and GCC,
don't ask me why. I'm assuming this puts it on a low priority...

I think this is also OK, because `setExecutor` takes an actual
`folly::Executor`, so even `then(Executor, Args...)` won't just work for any
`Executor`.

Test Plan: Run all the tests.

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036912

Tasks: 6838553

Signature: t1:2036912:1430493088:44f2ffe146298c3f978ac27a45b9b2e33b2b0422

Lazily initialize AsyncSSLSocket EorBioMethod

Summary:
Address another Android initialization crash by delaying
initialization of the EorBio method until first AsyncSSLSocket
construction.

Test Plan: existing tests

Reviewed By: pgriess@fb.com

Subscribers: net-systems@, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, #csti

FB internal diff: D2036329

Tasks: 6925575, 6925570

Signature: t1:2036329:1430444665:a3201f90860a34808a3cf3b42d530608c8a619a8

(Wangle) Chaining reduce

Summary:
Oh, this is best of both worlds: D2015316

If your reduce is really cheap, just use `collect/CollectAll->reduce`.
Otherwise just use `reduce`. Great?

This one doesn't support returning Futures from `func`, but if you want
to do that, just use `reduce`.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2017238

Tasks: 6025252

Signature: t1:2017238:1430422971:c3471afa53239681f835624becf3c4a238204b70

(Wangle) Simplify reduce, use the same implementation for func returning T/Future<T>

Summary:
I wanted to use `collectAll` for `reduce` if `func` does not return a
Future, because the overhead seemed smaller, but it has been suggested
that running the callback as soon as possible might be better. Not sure
which is. Note that this also makes n copies of the lambda and moves the
value at least n times.

I also plan to add a `streamingReduce` which calls `func` as soon as
results come in (but out of order).

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015316

Tasks: 6025252

Signature: t1:2015316:1430349611:4f62a540ded85299a22670dd2add27cf1458e8f8

IOBuf::reserve would return less tailroom than requested under certain circumstances

Test Plan: test added

Reviewed By: lxiong@fb.com

Subscribers: lxiong, net-systems@, folly-diffs@, yfeldblum, chalfant, pamelavagata, kma

FB internal diff: D2036967

Tasks: 6925950

Signature: t1:2036967:1430431606:3e115f7ed76b207572db26d352bebefe7a3d306d

Fix race condition in collect(..)

Summary: This is a temporary fix (until D2015320 is checked in) for race condition(s) in collect(..) method.

Test Plan:
Run unit tests
Run buffalo_aggregator canary

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037406

Tasks: 6894157

Signature: t1:2037406:1430435227:ed9612d016cdbd708e2deba02dc4fe0b59632f5a

Unit::Lift<T>

Summary: Lift void into the unit monad and pass other types through unscathed.

Test Plan: new unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029785

Signature: t1:2029785:1430333928:ef2fbb2e3d94518a732f6818a06c32481120bd4f

Do not construct a Future<T> from a Future<Something> value

Summary: The value constructor can be nice. But when it matches on Future<Something> it just confuses everybody.

Test Plan:
building and running tests
contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036455

Tasks: 6925951

Signature: t1:2036455:1430423503:73906f748318c4ebec13f45ad3761f104e2ef888

Convenience functions to wrap a lambda as an AsyncTimeout.

Summary: see title

Test Plan: unit tests added

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, andrewcox

FB internal diff: D2000579

Signature: t1:2000579:1430345677:1d7a78f94bcd8b0912423ca4987a4048c103241c

fix return type for map

Summary:
facepalm

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036523

Tasks: 6925951

Signature: t1:2036523:1430423049:b69ace0256eeacdd43490465dc1c862f054a0098

Move a little test

Summary: truffleshuffle

Test Plan: runtests

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029715

Signature: t1:2029715:1430285509:165edabb1bdbb2a4766e7619c67e7e568626b903

Future<Unit>::Future()

Summary: Allow `makeFuture()`-like default ctor for `Future<Unit>`

Test Plan: new unit test

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029677

Signature: t1:2029677:1430417794:5ec7fca839294316957803229f4783f2ee875027

Tag dispatch for enqueue/dequeue implementation

Summary:
clang (>=3.6?) reports potential object slicing bug when MPMCQueue is used for
polymorphic class as the queue item, e.g. as in P19814469. This can be false
positive however, since the choice is based on the type trait already. This
diff uses tag dispatch to selectively compile the overload that will be
executed, therefore if there is no-throw move ctor supplied clang will not
examine the simulated relocation code.

This doesn't avoid object slicing bug however if the client insists to use
MPMCQueue to hold base class while enqueue and dequeue with subclassed item.

Test Plan: compile with --clang

Reviewed By: tudorb@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2029949

Signature: t1:2029949:1430264357:af479117adf90bc1915c071e7376a30aacb72f46

unidirectional pipelines

Summary:
Cleans up bootstrap a bit at the expense of a more complex Pipeline interface
This doesn't have to go in, lmk either way as I want to move on to reorganizing this code into inl headers etc

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2034634

Signature: t1:2034634:1430414670:c91712fb26353987cb471e35a54f55c869ae7cf1

use inbound/outbound handlers in a few more places

Summary: Also, instead of throwing on finalize() if there's no inbound/outbound handler in the pipeline, log a warning and throw when the operations are attempted. This was necessary for CodecTest which doesn't use outbound handlers.

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2028176

Tasks: 6836580

Signature: t1:2028176:1430346333:fdd645a535e8158d780cfd1119e27803995b663f

inbound/outbound handlers

Summary:
Much less copypasta this time around. I wonder if the getters and setters for write flags and read buffer settings are necessary in the new handler types, or even if they belong in the bidirectional handler

I'm all ears for more suggestions on reducing copypasta

I'm going to reorg the code (inl headers etc) in a subsequent diff once this is in - easier to review this way

Test Plan: existing unit, thinking about tests for these changes

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2026522

Tasks: 6836580

Signature: t1:2026522:1430346145:bd7f7770eddce0470e2ac72440fc001cf128df08

moar CRTP to minimize copypasta for inbound/outbound handlers

Summary:
I think this is the best I can do to minimize whjat must be copied for InboundHandler, InboundHandlerContext, InboundContextImpl, outbound equivalents, etc

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2025612

Tasks: 6836580

Signature: t1:2025612:1430346200:bc06162711bbc52b17574297a3569736f7973a7c

struct/class consistency

Summary:
This diff is based on https://github.com/facebook/folly/pull/95
by gitbug user mikekap.

It changes the struct/class naming of forward declarations to match
their definitions.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2032995

Tasks: 6783581

Signature: t1:2032995:1430339337:79193fd8409b73e8a9155133f326b7cee534244d

Bump version to 37:0

fix unittest break

Summary: It looks like the ubuntu version of gtest complains about this?

Test Plan:
Will watch jenkins fbthrift and proxygen builds

previous jenkins break:
https://ci-builds.fb.com/job/folly/2178/console

Reviewed By: bmatheny@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2032761

Tasks: 6913742

Signature: t1:2032761:1430337624:ec65b26cf8da7d0d8c7e449f235de7147d4f17d6

Blame Revision: D2029802

Move Singleton out of folly/experimental into folly/

Test Plan: contbuild

Reviewed By: chip@fb.com

Subscribers: wormhole-diffs@, trunkagent, hphp-diffs@, jan, simpkins, configerator-diffs@, fbcode-common-diffs@, chaoyc, bill, search-fbcode-diffs@, agallagher, nli, marcelo, ckwalsh, mcduff, hitesh, mshneer, unicorn-diffs@, vighnesh, fugalh, andreib, bmatheny, tw-eng@, tanmoyc, zhuohuang, rvm4, antonl, acampi, alikhtarov, hdoshi, rsethi, panin, folly-diffs@, lins, kennyyu, hannesr, jsedgwick, dominik, yfeldblum, songhao, raghavpi, labrams, lyang, chalfant, #preselection, macsyz, nimishshah

FB internal diff: D2012267

Tasks: 5676394

Signature: t1:2012267:1430334667:eaad0262b35ffbfae86df5bdb45bf057ac62c51b

Shave off 1.3s of THeader.h compile time

Summary:
THeader.h went from 1.8s to 0.5s.
Total build of sample project doing thrift client and server stuff went
down 1.5-3% from

```
(run 1)
real    5m25.059s
user    130m30.853s
sys     10m18.642s

(run 2)
real    5m29.483s
user    129m12.209s
sys     10m14.762s
```

to
```
(run 1)
real    5m23.292s
user    125m44.318s
sys     10m7.103s

(run 2)
real    5m20.999s
user    127m4.969s
sys     10m10.118s
```

Test Plan:
fbmake --distcc off --ccache off dbg
Will also watch contbuild, since some files might be missing includes

Reviewed By: haijunz@fb.com

Subscribers: jmarg, anca, reachfrequency-eng@, jhunt, yanli, jgeller, chaoyc, search-fbcode-diffs@, net-systems@, zeus-diffs@, vikas, jcoens, unicorn-diffs@, ldbrandy, jteller, atlas2-eng@, everstore-dev@, leis, benj, laser-diffs@, zhguo, jying, wanghuan, jeremyfein, dbolcioni, jacekm, maxwellsayles, osmith, pallotron, fbcode-common-diffs@, davejwatson, andrewcox, marcelo, ckwalsh, mcduff, hitesh, mshneer, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, haijunz, chalfant

FB internal diff: D2029911

Signature: t1:2029911:1430261056:9ae688c50585a6454479f810d80b2c1c79b60e04

Move this `reduce` to `helpers.h`

Summary: just a lowly non-erroring semantic merge conflict

Test Plan: builds

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029593

Signature: t1:2029593:1430319785:2cd96927b9080fe18b168ab95ad201afc4f00857

get_or_throw and get_optional

Summary:
Adds get_or_throw map helper (get a value, or throw with the specified
exception type) and get_optional (get an Optional<Value>). This is a folly
backport of some util helpers in experimental.

Test Plan: Unit tests

Reviewed By: wez@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2029802

Tasks: 4332480

Signature: t1:2029802:1430280249:3efbb2fb9394e31b3fbe6d5bd209d12ebb7ed587

An IOExecutor that executes under mapped fiber context.

Summary:
This should help executing sync thrift handlers on fibers. This allows the handler methods to achieve concurrency by awaiting.

Test Plan: Chimera unit tests (which also accesses running FacebookBase methods in fiber context!)

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, hannesr, andrii, folly-diffs@, jsedgwick, haijunz, yfeldblum, alandau, chalfant

FB internal diff: D2015886

Signature: t1:2015886:1430156870:a84499e0e781f5e6e77f48f1f3063f53d0968698

Dynmaic load shedding

Summary:
add a free system memory limit to Load Shed Configuration for dynamic shedding

Test Plan:
tested on edge241.01.ams3, works as expected. Once the free memory is less than 23G (picked on purpose for testing), new connections are shedded. But the shedding rate is totally wrong. I will explain and address it in another diff.

https://fburl.com/103620501
https://fburl.com/103620701

Reviewed By: afrind@fb.com

Subscribers: trunkagent, fugalh, bmatheny, nimishshah, folly-diffs@, jsedgwick, yfeldblum, chalfant, xning, alexkr

FB internal diff: D2026477

Tasks: 4604893

Signature: t1:2026477:1430176828:65eadd82efa3189a1bebcb8518efaf56cd36beed

Promise::setValue() for Unit

Summary: Unit is a bit special because it's just something special to use instead of `Promise<void>`, so let's offer the same sugar that `Promise<void>` has (`p.setValue()` instead of `p.setValue(Unit())`)

Test Plan: New unit tests. Look, a pun!

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, davejwatson, hannesr

FB internal diff: D2014139

Tasks: 6847876

Signature: t1:2014139:1430159950:1484ee420c6d7f0f794a546b78ef1601c2eec45c

Fix default arguments for orderBy()

Summary: Calling `folly::gen::orderBy()` without arguments causes a compilation error because the first template argument cannot be deduced. This diff fixes it.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2025789

Signature: t1:2025789:1430167404:02fde7287b015d9dcbf398e8dc84cde7d74b4a5b

BenchmarkSuspender::dismissing.

Summary:
[Folly] BenchmarkSuspender::dismissing.

Pass a lambda to it, and the lambda will be executed while the benchmark-suspender is dismissed. Just a bit of sugar around `BenchmarkSuspender::dismiss` and `BenchmarkSuspender::rehire`.

BENCHMARK(name_void, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
braces.dismissing([&] {
# benchmark timer is running
doSomething();
});
# benchmark timer is suspended
}

BENCHMARK(name_value, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
auto value = braces.dismissing([&] {
# benchmark timer is running
return doSomething();
});
# benchmark timer is suspended
}

Test Plan:
Unit tests:
* `folly/test/BenchmarkTest.cpp` (actually a benchmark)

Reviewed By: njormrod@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2024166

Signature: t1:2024166:1430163281:24df0ac98cbe36372f780372ee8f7dd3722b7868

Handler::getContext() when possible

Summary:
Only allow this if the handler is only ever attached to a single pipeline once. i.e. only ever associated with one Context

Test Plan: unit, thrift unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2024007

Tasks: 6836580

Signature: t1:2024007:1430157264:efcf70ca3531c10eec5d458c9e9d6cda60c507c3

fix detachPipeline/attachPipeline ordering

Summary:
detachPipeline always goes bottom to top
attachPipeline always goes top to bottom
now we can attachReadCallback in AsyncSocketHandler::attachPipeline()

not sure of the implications for TAsyncTransportHandler... looks like Cpp2Channel still wants to attach/detach cb manually

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2023982

Tasks: 6836580

Signature: t1:2023982:1430157500:e09a4103550a3e5721effaa1b28ac5bed071fa67

Bump version to 36:0

README: Fixed typo, formatted code, and linked to docs

Summary:
(from @njormrod) This changes the READMEs to be .md,
appropriate for github. I viewed the docs on github, and they looked
WAY better as .md files.

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: view on github

Reviewed By: markisaa@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1999531

Tasks: 6783581

Signature: t1:1999531:1429220432:68c6d8dc70806d993c83cacce6369cb7e6e964c8

Manage handlers with shared_ptrs, introduce StaticPipeline

Summary:
A few things:
- Eliminate HandlerPtr by managing all handlers with shared_ptrs instead of keeping them inline in the pipeline
- Kill recursively templated ChannelPipeline accordingly
- Introduce StaticPipeline to retain the flexibility of zero-alloc pipelines
- Introduce notion of an "owning handler" to avoid destruction order issues

Test Plan: unit (will add more), thrift unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2023976

Tasks: 6836580

Signature: t1:2023976:1430159578:e50e8a149e549a40670d093fb65987a4843cdd8d

rearrange Future.h

Summary: Move boring preamble stuff to `Future-pre.h` and `folly::futures` and `makeFuture` and pals to `helpers.h`.

Test Plan:
tests still build and pass

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, hannesr, davejwatson

FB internal diff: D2014330

Signature: t1:2014330:1429941589:1e2c336136f3375f9b96e5df8c06ca5820ba6aeb

strip Channel from all class names

Summary: as above. Only got a little messy when components within folly::wangle typedefed things to Pipeline

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

Subscribers: wormhole-diffs@, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2022181

Tasks: 6836580

Signature: t1:2022181:1430157032:df0bdfb9ca0d76b86d52c55c4ad41ea953a18cb4

Fix oss build

Summary: Fixing oss build

Test Plan: Visual analysis.

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2023889

Signature: t1:2023889:1430144784:e58fe9e1f1193d71527ba5f5f2c2efb60bce7a2d

Blame Revision: D2022859

Introducing Unit!

Summary: title

Test Plan: tests

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2022859

Signature: t1:2022859:1429920863:0d3afce9779a790baf246de97f98812e23e1574d

IPAddressV6::kToFullyQualifiedSize.

Summary:
[Folly] IPAddressV6::kToFullyQualifiedSize.

The scenario: One has a string known to be a fully-qualified IPv4 or fully-qualified IPv6 address, but one does not know which of the two it is.

The dirty but fast solution: just check its size.

Test Plan: Build. Unit tests.

Reviewed By: mshneer@fb.com

Subscribers: folly-diffs@, dougw, yfeldblum, chalfant, brettp

FB internal diff: D2016634

Signature: t1:2016634:1429831808:08ecb28eab3060988f9613748ed8dabf3724bf13