Needless temporary gone

Summary: We might be doing:
1) Create a temporary
2) Copy/Move out of it
3) Destroy that temporary.
Which isn't needed in many places.
And copy/move elision doesn't work for a temporary
bound to a reference.

We can forward arguments, directly.
To get the work done three constructors were added.

Closes #222

Reviewed By: @JoelMarcey, @yfeldblum

Differential Revision: D2151731

Pulled By: @sgolemon

Added fiber supported mutex to folly

Summary: To be able to use fiber lock in thrift

Reviewed By: @brianwatling, @yfeldblum

Differential Revision: D2149936

Add EOS checking to 'parseLeadingNumber' function.

Summary: parseLeadingNumber does not contain EOS validation and raise an exception when '/sys/devices/system/cpu/cpu0/index0/shared_cpu_list' contains only one number like '0'.
Closes #207

Reviewed By: @yfeldblum

Differential Revision: D2148015

Pulled By: @sgolemon

apple-gate ThreadLocal in fibers code

Summary: Some internal malloc shims broke on using folly::ThreadLocal, so only do that for `__APPLE__`.

Reviewed By: @brianwatling

Differential Revision: D2151318

Suppress clang memcpy warnings

Summary: Clang warns when types with vtables are memcpy'd. If the type
has declared itself to be relocatable, then this is the desired
behavior. If the type has not declared itself to be relocatable, then
the memcpy codepath is dead. However, the dead codepath is still
instantiated (it's inside an if block with a static check, but c++
doesn't have static-if), so the compiler spits out a nasty warning
anyways.

Each memcpy reference inside of fbvector has been void-ified. I have
looked at all the codepaths leading to the memcpys, and see that they
have isRelocatable or isTriviallyCopyable checks.

Reviewed By: @markisaa

Differential Revision: D2148286

Temporarily disable performWriteIovec

Summary: As in title

Reviewed By: @djwatson

Differential Revision: D2151092

Recycle heap on assignment

Summary: For standard containers, assignment tries to reuse heap space.
Dynamic assignment does not - it calls destroy(), and then reconstructs
a new dynamic. In the case that the old and new types are the same (eg
assigning a dynamic-vector to a dynamic-vector) we can call through to
the underlying type's assignment operator.

Reviewed By: @jdelong

Differential Revision: D2148093

opt proxygen with new SSL_write_iovec function

Summary: this taks advantage of the SSL_write_iovec function to improve CPU.  According our testing, it saves 10+% CPU.

Reviewed By: @djwatson

Differential Revision: D2119223

fix for Boost >= 1.56

Summary: Folly still continue to build and work on earlier boost versions(boost < 1.56).

**boost::context::make_fcontext() change history:**
* __1.55:__    fcontext_t * make_fcontext( void * sp, std::size_t size, void(* fn)(intptr_t) );
* __1.56:__    fcontext_t make_fcontext(void* sp,std::size_t size,void(*fn)(intptr_t));

**boost doc:**
http://www.boost.org/doc/libs/1_55_0/libs/context/doc/html/context/context/boost_fcontext.html
http://www.boost.org/doc/libs/1_56_0/libs/context/doc/html/context/context/boost_fcontext.html
Closes #205
Closes #210

Reviewed By: @jwatzman

Differential Revision: D2148069

Pulled By: @sgolemon

Check for python in configure

Summary: Some OSs have python named according to a version ID
e.g. `python2.7` without a symlink from plain `python`

Use autotool's AM_PATH_PYTHON to search for a suitable interpreter
and use that to generate the *Tables.cpp files.

Closes #109

Reviewed By: @yfeldblum

Differential Revision: D2147977

Keep Makefile.am from double-installing detail/Clock.h

Summary: On non-Linux systems, the autotools-based build system conditionally appends `detail/Clock.h` to `libfollybase_la_SOURCES` – even though that file is in there statically no matter what (as per [line 46](https://github.com/facebook/folly/blob/master/folly/Makefile.am#L46)), which causes the `install` command to error out trying to install the file twice, during `make install`… this patch takes out the conditional append.
Closes #201

Reviewed By: @yfeldblum

Differential Revision: D2148004

Pulled By: @sgolemon

Replace thread_local with FOLLY_TLS

Summary: OSX doesn't support C++11's `thread_local`. :-/ (yet? http://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c11-thread-local-when-official-clang-supports/29929949#29929949)
We have `FOLLY_TLS` in `Portability.h` https://github.com/facebook/folly/blob/master/folly/Portability.h#L163-L175

So replace the usages of `thread_local` with `FOLLY_TLS`. But `__thread` is not drop-in compatible with `thread_local`, specifically the former requires global storage and needs a constexpr initializer, and a trivial destructor.

`futures/QueuedImmediateExecutor` now uses `folly::ThreadLocal` (because `std::queue` has a non-trivial destructor). It might end up in one more alloc, per thread. I also rearranged the code to clarify the fact that although there may be many `QueuedImmediateExecutor`s, they all share the same queue per thread.

I didn't use `folly::ThreadLocal` for fibers, because I thought it might be too expensive. But now I'm not so sure. I had to do the "check for default and then initialize" thing because of the requirement for constexpr initialization for `__thread` (and therefore `FOLLY_TLS`).

Reviewed By: @jsedgwick

Differential Revision: D2140369

disable ThreadExecutor in FutureTest.cpp, on OSX

Summary: Something's broken here, will track fixing in #7372389

Reviewed By: @jsedgwick

Differential Revision: D2144077

Build and run futures tests on OSX

Summary: They don't pass, but they build!

Reviewed By: @jsedgwick

Differential Revision: D2141717

Future::unit()

Summary: Discard a result but propagate an exception. cf https://www.facebook.com/groups/715931878455430/permalink/879624552086161/

Reviewed By: @​sammerat

Differential Revision: D2144567

refactor the interpretation of SSL errors into a function

Summary: This fucntion will be re-used for different SSL write functions in future.

Reviewed By: @djwatson

Differential Revision: D2117529

use LifoSem instead of sem_t for ManualExecutor

Summary: OSX apparently knows better than all of us plebes, and got rid of anonymous semaphores. Rather than try to work around that directly, I just switched to folly::LifoSem, which seems like a not-terrible idea anyway.

Reviewed By: @jsedgwick

Differential Revision: D2143566

folly::via(Executor*, Func)

Summary: ((not yet) more performant) sugar for `via(&x).then(func)`

Reviewed By: @​hannesr

Differential Revision: D2131246

Make IOBuf copyable

Summary: Give IOBuf a copy constructor, which clones the whole chain. (IOBufs have
shared pointer semantics). You could argue for cloning one single IOBuf,
but cloning the whole chain has the nice side effect of making Thrift
structures with IOBufs in them copyable, so there's that.

Reviewed By: @djwatson

Differential Revision: D2127209

RequestEventBaseCache -> IOObjectCache

Summary: Next step towards killing RequestEventBase hackery.
Use EBs from global IO executor instead

Reviewed By: @fugalh

Differential Revision: D2123142

wangle::getEventBase()

Summary: this is going to be a really common pattern, so I'm providing a shortcut

Reviewed By: @fugalh

Differential Revision: D2146008

optimize makeFuture and Future<T>::Future()

Summary: No reason to go through the whole Promise rigamarole. Add an appropriate Core ctor and use that to make a completed Future with just the core alloc

Note the big win in the `constantFuture` benchmark.

```
Before:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             120.50ns    8.30M
promiseAndFuture                                  91.99%   130.98ns    7.63M
withThen                                          28.17%   427.77ns    2.34M
----------------------------------------------------------------------------
oneThen                                                    430.48ns    2.32M
twoThens                                          58.03%   741.86ns    1.35M
fourThens                                         31.85%     1.35us  739.97K
hundredThens                                       1.61%    26.80us   37.32K
----------------------------------------------------------------------------
no_contention                                                4.58ms   218.48
contention                                        83.70%     5.47ms   182.86
----------------------------------------------------------------------------
throwAndCatch                                                8.09us  123.55K
throwAndCatchWrapped                              94.43%     8.57us  116.67K
throwWrappedAndCatch                             154.69%     5.23us  191.12K
throwWrappedAndCatchWrapped                      614.06%     1.32us  758.70K
----------------------------------------------------------------------------
throwAndCatchContended                                     967.54ms     1.03
throwAndCatchWrappedContended                    103.48%   935.04ms     1.07
throwWrappedAndCatchContended                    148.24%   652.70ms     1.53
throwWrappedAndCatchWrappedContended            14313.28%     6.76ms   147.94
============================================================================

After:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                              69.11ns   14.47M
promiseAndFuture                                  55.12%   125.37ns    7.98M
withThen                                          16.49%   419.18ns    2.39M
----------------------------------------------------------------------------
oneThen                                                    370.39ns    2.70M
twoThens                                          55.11%   672.05ns    1.49M
fourThens                                         29.00%     1.28us  782.89K
hundredThens                                       1.23%    30.22us   33.09K
----------------------------------------------------------------------------
no_contention                                                4.56ms   219.46
contention                                        82.82%     5.50ms   181.77
----------------------------------------------------------------------------
throwAndCatch                                                8.30us  120.42K
throwAndCatchWrapped                              96.40%     8.61us  116.08K
throwWrappedAndCatch                             162.66%     5.11us  195.89K
throwWrappedAndCatchWrapped                      680.39%     1.22us  819.36K
----------------------------------------------------------------------------
throwAndCatchContended                                     979.17ms     1.02
throwAndCatchWrappedContended                    103.09%   949.84ms     1.05
throwWrappedAndCatchContended                    153.55%   637.71ms     1.57
throwWrappedAndCatchWrappedContended            10468.47%     9.35ms   106.91
============================================================================
```

Reviewed By: @fugalh, @​hannesr

Differential Revision: D2144664

Fix libc++ use

Summary: Use of std::max from libc++ resulted in error.
std::max found type of its arguments diffrent
whereas it expected them to be same.
This diff fix the error.
Closes #223

Reviewed By: @yfeldblum

Differential Revision: D2144639

Pulled By: @sgolemon

Fix capitalization error in README.md

Summary: Closes #211

Reviewed By: @Gownta, @lbrandy

Differential Revision: D2144702

Pulled By: @sgolemon

Fixing typos in README.md

Summary: Closes #202

Reviewed By: @paulbiss

Differential Revision: D2144850

Pulled By: @sgolemon

Update FBVector.md

Summary: Minor typo when referencing `std::allocator`
Closes #84

Reviewed By: @yfeldblum

Differential Revision: D2144830

Pulled By: @sgolemon

Update github.com/floitsch/double-conversion to new location

Summary: Closes #212

Reviewed By: @JoelMarcey, @paulbiss

Differential Revision: D2144749

Pulled By: @sgolemon

fix Baton.h typo

Summary: checked the sense
Closes #224

Reviewed By: @paulbiss

Differential Revision: D2144508

Pulled By: @sgolemon

(Wangle) Re-add race test

Summary: I figured this test would be useless since it tests a check added before
everything was converted to FSM. But since it appears to fail on Mac OS
X, it might be useful to have after all.

Reviewed By: @fugalh

Differential Revision: D2143500

Bump version to 45:0

fix typo in SharedPromise comment

Summary: ^

Test Plan: OK

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2140212

Signature: t1:2140212:1433877717:86b7bf372091b455cdb25a377451e1ed4e21f50d

Bump version to 44:0

AsyncSSLSocket StartTLS

Summary:
Adds a StartTLS mode to AsyncSSLSocket.  Previously I could only find anyone doing something like this by using AsyncSocket, calling detachFd, then creating a new AsyncSSLSocket, and calling sslConn/sslAccept.

That had a couple downsides: 1) All pointers to the previous AsyncSocket become invalid and similarly 2) have to be super careful reads/writes happen on the correct socket, are flushed before changing socket types, etc.

This makes it super easy to just use the same AsyncSSLSocket for everything:
a) Create AsyncSSLSocket in StartTLS mode
b) send/recv anything
c) Call sslAccept/sslConn.  Existing writes are still flushed in the correct order, any additional writes are buffered until handshake completes
d) Start receiving encrypted data.

I made it a new mode (vs. the default), since it seems bad to unintentionally send unencrypted data.

Use case is easy secure thrift upgrade (similar to how current kerberos does it)

Test Plan: New unittest

Reviewed By: afrind@fb.com

Subscribers: doug, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, haijunz, andrewcox, alandau, alikhtarov, jsedgwick, simpkins

FB internal diff: D2120114

Signature: t1:2120114:1433798448:caeddc8feb6cc10fb34200ba97ea323bcaf09f7a

fix osx bootstrap for new double-conversion directory structure

Summary:
Because https://github.com/google/double-conversion/commit/64702b40a2499702f9157ef015f0da6e006114bf

If this breaks an existing checkout, it is because you have an old version of double-conversion now. Go ahead and `rm -rf` it, or start with a clean folly clone.

Test Plan: Clean repo, `./bootstr<tab>`.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2136836

Signature: t1:2136836:1433874889:6f6c6a7bd337e052c088aabf89955e9b21b1d8e1

Add a guard page to a limited number of stacks

Summary:
GuardedPageAllocator now maintains a process-wide cache
of preallocated stacks with guard pages.  We can't create too many of these, since
each stack has the overhead of two memory mappings.  Once we run out of preallocated stacks
we simply fallback on the default allocator.

Test Plan:
unit tests

perflab
TAO unit tests

Reviewed By: pavlo@fb.com

Subscribers: zhuohuang, trunkagent, sameen, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129510

Tasks: 7319041

Signature: t1:2129510:1433542031:324942af3c9813344c1b331ee2c9b66c4bfa3f03

(Wangle) variadic collect

Summary:
For D2099047 (matthieu) and also for symmetry. Can re-use most of the
code, also refactored it a bit (using an empty base case).

Test Plan:
Run all the tests.

Will add some more before committing.

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, matthieu

FB internal diff: D2131515

Signature: t1:2131515:1433776852:544166fbfdfabf6760fd78f87821290e17e6e4a3

remove Singleton from io/async deps

Summary: unused, caused circular dep issues

Test Plan: compiles

Reviewed By: mzlee@fb.com

Subscribers: subodh, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2135780

Signature: t1:2135780:1433786607:015cfa9bb14472f55f80467f2cfeb7426be9f7ea

fix the compling error caused by tcpinfo of low version

Summary: my D2097198 break the project relying on old glibc

Test Plan: compling

Reviewed By: wez@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2135297

Tasks: 7283522

Signature: t1:2135297:1433791259:8b82b8d9b16da32be54c0dff1214fa20c94840e2

Add internal mode for HHWheelTimer

Summary:
This allows an EventBase to exit loop() even if there are pending timeouts in the timer (make sure to cancelAll before deleting!)

Test Plan: Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2111942

Signature: t1:2111942:1432934212:7ac1973bdbbfdbda714699ab373618399f29ef76

Fix haystack threading crashes

Summary:
proxygen/facebook/httpserver/filter/StatsFilter expects the threadlocal stats_ variable to be initialized before any requests, and destroyed on the same thread.

There were two issues:
1) addObserver was being called after bind(), so it was possible to get some requests before the stats_ variable was initialized.  Moved up the observer, should fix it (although I don't have a repro for this)

2) on server stop(), sometimes the observer onServerStop() callback was called, but the eventBase had already stopped.  The IOThreadPoolExecutor eventually deleted the eventBase, causing consumeUntilDrained to be called, but in the main thread, and not in the eventBase's thread.  I'm not as sure about this fix, hopefully @jsedgwick can chime in: I explicitly deleted the eventBase on the eventBase thread before join.  Here's the stack for this issue:

P19866015

Reviewed By: shiva@fb.com

Subscribers: doug, fugalh, bmatheny, folly-diffs@, yfeldblum, chalfant, shiva, jsedgwick

FB internal diff: D2132208

Signature: t1:2132208:1433537787:4379ba50a908ca6f19924c2339876afaf663364c

FileServer

Summary: Based on new FileRegion interface

Test Plan: ran it, streamed lots of files. I have found an obscure way to break it but I don't think it's a FileRegion issue, rather a pipeline/bootstrap issue. Still digging

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2088094

Signature: t1:2088094:1432758904:0f1e5765dbedea5ddbe047db50e7b632cfb0c053

AsyncSocket::writeRequest() and its first user wangle::FileRegion

Summary: similar to D2050808, but move the functionality into AsyncSocket itself so that you have a consistent interface and contiguous writes for a single file

Test Plan: added unit, will hook this up to a file server example next

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, net-systems@, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2084452

Signature: t1:2084452:1433181933:175158618966706db00bf6620cc86ae145d04ecf

Conv.h: fix a type error caught by clang

Summary: Need to be more explicit about types; it was trying to add types deduced to be `unsigned long` + `long` which didn't agree and could be an overflow.  Be explicit anyway about the types involved.  Appease Clang, and also make the code more obvious to the reader.

Test Plan: Tried with gcc 4.9, clang 3.5, clang 3.6.

Reviewed By: yfeldblum@fb.com

Subscribers: mathieubaudet, maoy, folly-diffs@, yzhan, yfeldblum, chalfant

FB internal diff: D2134814

Tasks: 7337462

Signature: t1:2134814:1433726862:5dd80b198187c610f793e26160919922863a22a2

Blame Revision: D1934777

decimal conversion: digits10 using bit-counting instructions on x86-64

Summary: To estimate length of a number's base-10 length in digits, use insn `bsrq` (Bit Scan Reverse) to count significant bits.  From that, approximate base-10 length.  Tries to avoid branchiness, expensive math, and loops.

Test Plan:
1) Tested correctness by comparing results with `snprintf` and ensuring same string lengths.  Tested at each boundary condition (2^k)-1, 2^k, (2^k+1); and similar for base 10.
2) Benchmarked with gcc 4.9 and clang 3.5.

Before/after values are millions operations / sec
GCC 4.9                             Clang 3.5
1       111.09  111.7   1.005       1       115.36  393.81  3.414
2       115.36  111.7   0.968       2       115.36  393.89  3.414
3       114.91  111.34  0.969       3       111.09  393.56  3.543
4       114.91  111.34  0.969       4       111.09  393.86  3.545
5       115.36  111.36  0.965       5       111.09  392.18  3.530
6        99.99  104.32  1.043       6       103.43  393.74  3.807
7        83.31   84.71  1.017       7        81.06  268.39  3.311
8        76.9    78.23  1.017       8        76.91  268.26  3.488
9        62.48   68.26  1.093       9        65.56  190     2.898
10        59.98   63.65  1.061      10        61.17  190.54  3.115
11        50.6    55.87  1.104      11        54.54  148.03  2.714
12        47.19   51.7   1.096      12        50.84  148.57  2.922
13        40.53   46.99  1.159      13        43.33  115.91  2.675
14        40.48   43.42  1.073      14        41.5   115.97  2.794
15        34.92   40.21  1.151      15        37.27   94.89  2.546
16        33.49   37.51  1.120      16        35.77   94.88  2.653
17        29.89   35.02  1.172      17        31.7    80.67  2.545
18        29.11   32.98  1.133      18        30.76   80.63  2.621
19        26.58   31.05  1.168      19        28.22   70.15  2.486
20        25.64   29.38  1.146      20        27.96   70.16  2.509

Reviewed By: ldbrandy@fb.com

Subscribers: dancol, trunkagent, marcelo, chalfant, maoy, folly-diffs@, yzhan, yfeldblum

FB internal diff: D1934777

Signature: t1:1934777:1433523486:3acbe7ed9c9560c44194f854754529041eaa289d

(Wangle) Clean up tests

Summary:
This always bothered me. Now:

1) Separate tests for each feature
2) `XTest.cpp` contains `TEST(X, lowerCaseCamelCase)`

Also cleaned up some header files and using statements.

Shamelessly trying to increase the number of lines that blame to me.

Test Plan:
Run all the tests.

`git grep 'TEST' -- 'folly/futures/test/*Test.cpp' | cut -d ',' -f 1 | cut -d '/' -f 4 | sort -u`

`diff <(git ls-files -- 'folly/futures/test/*Test.cpp' | cut -d '/' -f 4 | sort) <(git grep 'Test.cpp' -- folly/futures/TARGETS | cut -d '/' -f 4 | cut -d "'" -f 1 | sort)`

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2127359

Tasks: 6019442

Signature: t1:2127359:1433459532:54a91ae83d7fb2d0b3f3769f673fefea20f35435

Bump version to 43:0

disable stack madvise optimization on non-Linux platforms

Summary:
This diff strengthens the preconditions for attempting to use
optimize by madvising away idle stacks, because it seemed to be causing
problems on OS X (https://github.com/facebook/proxygen/issues/3).

Test Plan: unit tests

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129100

Signature: t1:2129100:1433458268:0b6b3696dde6c2bd13b89bc7ec58f0b1898be458

Add missing include

Summary:
`std::runtime_error` is defined in this header. Most build
systems seem to end up including it transitively anyways, but one version
of Clang on my Mac does not.

Test Plan:
`fbconfig -r folly && fbmake dbg` still works. Builds on my
Mac with MacPorts clang 3.6.

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2127268

Signature: t1:2127268:1433437985:546d0de71d8973d10f8ae3fdb6c61efae59444b9

merge dex and readme

Summary:
This is still manual as in the previous DEX.md revision, but I incorporated README.md into dex and redid the manual export.

Test Plan: reading_intensifies

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2124555

Tasks: 6047407

Signature: t1:2124555:1433370669:91f8bb55c1e6de4dbeccbb85556f54ae49464c7a

Reduce footprint of EliasFanoReader

Summary:
`EliasFanoReader` has a copy of `EliasFanoCompressedList` as member, but it only needs few of its members. With this diff, it only copies the members it needs.
Also, `progress_` is a duplicate of `upper_.position()`, so it was removed.
Microbenchmarks do not indicate any significant change in performance.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2125956

Signature: t1:2125956:1433381848:2a333ce7a741bec5d059e9e771309463d6018ea2

Add cloning constructor to folly::IOBuf

Summary:
``` lang=cpp
folly::IOBuf buf(folly::IOBuf::CLONE, other);
```

is more convenient (and expression-friendly) than

``` lang=cpp
folly::IOBuf buf;
other.cloneInto(buf);
```

Test Plan: used it

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2124605

Signature: t1:2124605:1433372767:c167a36ffdc6d20c740aa57895532e9445766fce

remove useless self-assignment in folly::iobuf

Summary:
I'm pretty sure this doesn't do anything.

Based on a cursory review, I also don't think this is a bug (ie, it should have been assigned to something else).

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: tudorb@fb.com

Subscribers: net-systems@, enis, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2124497

Tasks: 7306976

Signature: t1:2124497:1433368083:1097dc22734dacad59b30d4b721be111d021bb43

Bump version to 42:0

(Futures) manually-generated and manually-sanitized DEX.md

Summary:

Test Plan: I copied this to a test github project and see that it renders well enough.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2120950

Tasks: 7295194

Signature: t1:2120950:1433289846:8fbf5123e71d5594742305009fadb516206e2ba6

More flexible constructors for Elias-Fano lists

Summary:
Implement constructors for EliasFanoCompressedList to read a list from a contiguous byte range given either size and upper bound, or size, lower bits width, and upper bits size.

Refactor the rest accordingly.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: trunkagent, chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2105658

Tasks: 5474196

Signature: t1:2105658:1433270469:9948b159504e08c1b00eeb4cbe327752364ec300

fix mem leak

Summary:
- use folly::ThreadLocal to work around GCC bug 57914 (with the benefit of accessAllThreads)
- clean up corresponding thread-local and global cache entries before eventbase gets destructed since there was a possible memory leak for short-term living eventbase.

Test Plan:
tests

Reviewed By: andrii@fb.com

Subscribers: smarlow, rushix, ilyam, trunkagent, folly-diffs@, yfeldblum, chalfant, jinfu

FB internal diff: D2116216

Tasks: 7291028, 7279391

Signature: t1:2116216:1433212893:e57a7df90b15b89ccd9471469e669c6e7dc477bf

Blame Revision: D1941662

(Wangle) Don't use std::bind in Core

Summary:
Seems like an unnecessary indirection that might make it harder to for
the compiler to inline everything (if that's even possible with
`std::bind`)?

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2115032

Signature: t1:2115032:1433182233:3938d0498cad50df32c8b6ee4536de14ae5fd006

Minor fix

Summary:
`LockFreeRingBuffer` was using `std::unique_ptr` but not importing `memory`.
Also, Android NDK's libstdc++ doesn't provide `std::llround`, use straight
`llround`, we don't need the constexpr-ness anyway.

Test Plan: `fbmake runtests`

Reviewed By: jmkaldor@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116923

Signature: t1:2116923:1433210832:9ec0ed7f558c271a51d528b8f88fbce9b466ba3a

AsyncServerSocket: more tries for binding to a port

Summary: User reports of intermittent failures at 5.  Setting up a server is expensive.  Binding to a socket is pretty cheap.  Why not 25 tries?  Why not 100?

Test Plan: hope

Reviewed By: alandau@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116187

Signature: t1:2116187:1433199889:09d6916eddf8521f120894ce361c3e1a96905c90

folly: adding folly::ltrimWhitespace / folly::rtrimWhitespace

Summary:
folly: adding folly::trimWhitespace

extending the idea of folly::skipWhitespace.
the reason for adding it, we want to be able to do:

auto s = skipWhitespace(trimWhitespace(" aaaa "))

very similar to python's strip()

Test Plan:
adding unit tests:

# make ; _bin/folly/test/string_test --gtest_filter="String.whitespace"
# fbmake --ccache off --distcc off dbg -j 16
fbmake dbg -j 16
Fbmake run id: G62i4cDP42U
Note: Google Test filter = String.whitespace
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from String
[ RUN      ] String.whitespace
[       OK ] String.whitespace (0 ms)
[----------] 1 test from String (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[  PASSED  ] 1 test.

Reviewed By: ldbrandy@fb.com

Subscribers: ldbrandy, folly-diffs@, yfeldblum, chalfant, azhavnerchik

FB internal diff: D2109364

Signature: t1:2109364:1433192444:862e237bba1928fcb94be1f95c57a68d457939e9

enrich network-quality HTTPHeader field with retransmission rate

Summary:
estimating tcp retransmission rate of the socket

Test Plan: unit tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, njormrod, bmatheny, trunkagent, chalfant, yfeldblum, jsedgwick

FB internal diff: D2097198

Tasks: 4888253

Signature: t1:2097198:1433196365:16db26dfd721514481497eddfc7820a453618d33

folly: exception_tracer: add throw() to __cxa_begin_catch

Summary:
http://libcxxabi.llvm.org/spec.html

void* __cxa_begin_catch(void* exceptionObject) throw();

```
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:145:7: error: '__cxa_begin_catch' is missing exception specification 'throw()' [-Werror]
void* __cxa_begin_catch(void *excObj) {
^
throw()
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:35:7: note: previous declaration is here
void* __cxa_begin_catch(void* excObj);
^
1 error generated.
11.44s error: folly/experimental/exception_tracer/ExceptionTracerLib.cpp
fbmake: *** [_build/opt/folly/experimental/exception_tracer/b187621/ExceptionTracerLib.o] Error 1
```

Test Plan: fbmake

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2115305

Tasks: 5908365

Signature: t1:2115305:1433186210:3652d81ab10effe1be6da86705322ce090fb6cd4

Blame Revision: https://phabricator.fb.com/D2114085

folly: -Wsign-compare

Test Plan: runtests

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, tudort, chalfant

FB internal diff: D2114063

Signature: t1:2114063:1433111051:6686508d78de39ea4ea5dc283189c472e23f8f42

Comparing against tlsExthostname_

Summary:
Instead of comparing tlsExthostname_ with NULL, we are comparing it
against tlsExthostname_.

Test Plan: Automated tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, moa

FB internal diff: D2106257

Tasks: 4751985

Signature: t1:2106257:1432932687:78c6244392d7842e844a3c1e654ffc4dc0e760a2

update wangle/rx/README.md

Summary:
https://github.com/facebook/wangle/pull/1

Test Plan: reads better

Reviewed By: jsedgwick@fb.com

Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2111498

Signature: t1:2111498:1432927058:1f14b38faa324d0bdef9421ae5592244a6b78928

Fix asan test crashes

Summary: We need to prevent the socket from being deleted while in the loopCallback.

Test Plan: --  fbconfig '--sanitize' 'address' 'folly/io/async/test' && fbmake runtests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2109031

Tasks: 7250307

Signature: t1:2109031:1432918801:77f0352254925d3f33f1e004c8bba89cde63e2cf

Blame Revision: D2102601

Chain executor in timeout functions

Summary: within, onTimeout, etc use the executor of the future if it is set.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2104770

Tasks: 6958727

Signature: t1:2104770:1432773599:a71c00827071bef46e5700de04c7125142e4eb17

Make TurnSequencer non-x86-safe

Summary:
`TurnSequencer` uses inline assembly, which obviously won't work on ARM.
Guard said assembly to x86{,_64} targets.

Test Plan:
Added an always-false `static_assert` to the `ifdef` block, confirmed it gets
triggered (i.e., we're not removing the `pause` on x86_64).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069320

Signature: t1:2069320:1431663264:3b3fb4282cf2b755f9b26c76b608b91ff665e226

Make detail/Futex.cpp bionic-safe

Summary:
The headers that the Android NDK exports are all sorts of screwed up.
In particular, they expose a subset of the `futex.h`, so, even though the
kernel supports newer options, we can't reference them from the NDK headers.

This diff ensures Futex.cpp redefines all the things it uses, if the headers
are lacking.

It also transitions away from `SYS_` to `__NR_` which is apparently the newer
convention.

Lastly, the duration usage is made explicitly `time_t`-safe for use on 32-bit
platforms where `time_t` is `long int` but `std::chrono:seconds` is stored as
`long long int`.

Test Plan:
Existing tests, sample app compiled and ran cleanly on Android as well
(tests not verified on Android due to folly largely being unported to Android yet).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069306

Signature: t1:2069306:1431721711:36c77b1afe8dd9259c1050f11a87511dcf7dd25f

LockFreeRingBuffer

Summary:
This introduces a lock-free ring buffer with the following expected semantics:

1. Writers can't block on readers
2. Writers are mostly lock-free
3. Readers can detect if they're being too slow
4. Be usable on Android (TBD but should work as-is with the armeabi-v7a ABI; armeabi (ARMv5) support is explicitly a non-goal)

Non-goals:
1. Match MPMCQueue in level of optimization. There's no need for that yet.

Test Plan: iloveunittests

Reviewed By: ngbronson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2037718

Signature: t1:2037718:1432850250:c57963510d8cda58edc006f4c3260f5ac34d4996

Extract TurnSequencer to detail/TurnSequencer

Summary:
A completely mechanical transformation that moves TurnSequencer into its own
header, to be used by other collections.

Test Plan: Existing tests.

Reviewed By: jmkaldor@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2065108

Signature: t1:2065108:1431474613:a3a9d063ebd2bedb31abb37be5fd33f0fb3eca6a

Fix toString() for authority-less URIs

Summary: Uri("foo:bar").str() would incorrectly return "foo://bar"

Test Plan: test added

Reviewed By: savasp@fb.com, markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2107530

Tasks: 7248055

Signature: t1:2107530:1432837143:c100f148c07b5b141cc036b1b39e6c8317e9bbd6

Chain interrupt handlers

Summary: Chain interrupt handlers in then().  User can always choose to set a new one later.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2091569

Signature: t1:2091569:1432760579:ea7afa9b4e62e4b04b4d5f17950b9b7356a57f09

Fix AsyncSocket::handleRead

Summary:
If openssl has buffered data read from the socket, we might not get a
read event on the socket. So, we must schedule a readCallback to ensure
before exiting from AsyncSocket::handleRead if we have exhausted the
maxReadsPerEvent_ limit.

Test Plan: -- modifying existing test to test this corner case.

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2102601

Tasks: 7168699

Signature: t1:2102601:1432837605:82e72a2a1875c08c9c1e8e831796c9c90df927fa

folly: MemoryMapping: madvise: round the end to lower bound

Test Plan: n/a

Reviewed By: ott@fb.com

Subscribers: ott, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2100535

Tasks: 4421175

Signature: t1:2100535:1432674713:6f5f40a8462851b2b8972c68d34ae23aaf1e9340

drop V0 of EliasFanoEncoder

Summary: Cleanup. Drop support for V0 in favor of V1.

Test Plan: unit tests

Reviewed By: lucian@fb.com

Subscribers: fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2105967

Signature: t1:2105967:1432781247:e420d8b4b8c69d28dfc229e8a2af6df8a580f979

Bump version to 41:0

Fix cancel in ThreadWheelTimeKeeper

Summary: This is actually a bug, future.cancel() doesn't work with the current THreadWheelTimekeeper, because cancel() only works from the eventBase thread.

Test Plan: added unittest.  Crashes before, passes now

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2091531

Signature: t1:2091531:1432224024:4aa5dd71de15b1344034a414d47c97ffaba68949

folly::Unit docs and use lift in is_void_or_unit

Summary:
I'm not sure about the name Lift now. We are lifting, yes, but are we lifting into Unit, or into "can't be void because void is unit"? But LiftIntoNonVoid is a bit verbose. I'm totally open to other names or arrangements. We could also rename `is_void_or_unit`, but to what?

I reimplemented `is_void_or_unit` in terms of `Unit::Lift` because it's kinda cool but also to provide a little motivational example to the reader for why Lift exists in the first place.

Test Plan: Still builds and passes tests. Nothing significant depends on this yet.

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2102147

Tasks: 6847876

Signature: t1:2102147:1432742966:a03973a45882d3e9f6fa7158ef393b148cbe16fc

Instrument EventHandlers' execution

Summary:
Add the option to instrument every EventHandler that runs in a
EventBase.

Test Plan:
Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2044546

Tasks: 6958000

Signature: t1:2044546:1432221358:6500d6c1a6631d7a06a18e874d051f2ff3108e10

Promise::isFulfilled()

Summary: See task

Test Plan: runtests

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2101549

Tasks: 7225286

Signature: t1:2101549:1432688294:5fb9d7370c38c1392429a09ba48b131cac16647d

copy wangle back into folly

Summary: copy everything but example/

Test Plan: fbconfig -r folly/wangle && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100811

Tasks: 5802833

Signature: t1:2100811:1432678173:6c336fe53aa223993f6f82de4ac91b3c19beacf1

move folly/wangle to wangle

Summary: this preserves history and will be followed by a copy diff

Test Plan: next diff will compile

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100784

Tasks: 5802833

Signature: t1:2100784:1432677219:838a336d84918145d19c6cfa4290dd58374821fd

removing non-existing file from the build

Summary: broken by D2096146

Test Plan: arc unit

Reviewed By: andrewcox@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2100773

Signature: t1:2100773:1432676073:a72e72fa5fb915b027fdfd31dd814d02e6606858

cleaning up RequestContext

Test Plan: built folly

Reviewed By: mshneer@fb.com

Subscribers: davejwatson, mshneer, trunkagent, alandau, bmatheny, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2096146

Tasks: 6337419

Signature: t1:2096146:1432658670:4ff00e4274412519dc3b02d2a849b7ff289ac979

Bump version to 40:0

remove spurious warning

Summary: see task

Test Plan: OK

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2099416

Tasks: 7166291

Signature: t1:2099416:1432658571:4ae632c218c4764f23e4b189167f6fdaff13dc2d

Some optimizations

Summary:
1. Eliminate some string -> StringPiece -> strings conversions
2. Mcrouter: eliminated unnecessary inlining by moving slow path logic into its own method.

Using a test setup with shadow sampling enabled and shadowing some requests,
(typical prod setup), this brings down the cost from ~1.4% cpu in standalone mcrouter to ~0.2%:

```
before:

+ 0.70%          3898  mcrouter_orig  mcrouter_orig  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.13%           864  mcrouter_orig  mcrouter_orig  [.] EventGroup<ScubaRow>::processExtraSamplers
+ 0.58%          3347  mcrouter_orig  mcrouter_orig  [.] DynamicScubaSampler::getSampler

~ 1.41% total

after:

+ 0.18%          1223  mcrouter_fix  mcrouter_fix  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.04%           205  mcrouter_fix  mcrouter_fix  [.] EventGroup<ScubaRow>::processSampler

~ 0.22% total
```

Fiber local optimization might have more of an effect.

Test Plan:
unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, alikhtarov, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2089133

Tasks: 5414865

Signature: t1:2089133:1432338487:4158dc6b720c04f43820193e73b98d4197afcffa

move FiberIOExecutor to wangle/concurrent

Summary: Need to cut this dep on wangle out of folly as we're moving wangle to its own repo

Test Plan: chimera unit

Reviewed By: rushix@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2092122

Signature: t1:2092122:1432239179:9261c1b3a3a346b886b15899d25a4d0530d4e890

Revert "Make strlcpy available in folly" (D2062632)

Summary:
Reverting diff to fix the build.

Test Plan: contbuild rule that failed to build is okay now

Reviewed By: ldbrandy@fb.com

Make strlcpy available in folly

Summary:
strncpy is bad.  strlcpy is somewhat less bad.  We already had
this function, but let's move it somewhere more reasonable.

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: trunkagent, lins, anca, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2062632

Signature: t1:2062632:1431969926:cc7f7283073d0242fe8f361efac2557aa0b0a481

remove unused variable

Test Plan: search, and fbmake

Reviewed By: delyank@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093523

Signature: t1:2093523:1432249208:84a27e2328636c2aafe43412f61a693b5645b895

folly/experimental/fibers/EventBaseLoopController.h: include <atomic>

Summary: Without <atomic>, this fails to compile with clang:dev and newer glibc.

Test Plan:
fbconfig --clang --with-project-version=clang:dev --with-project-version=llvm_compiler_rt:dev --platform-all=gcc-4.9-glibc-2.20 --sanitize=address mcrouter/lib/network:client-test-sync && fbmake dbg

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093709

Signature: t1:2093709:1432250378:2b78c142598bf0a7b0869089c752f5ccdae9da22

Refactor tests to verify that a single folly target can be used successfully

Summary:
This includes a change to Range to move operator<< into the
header, to achieve the goal.  Specifically, see format_test,
dynamic_test, json_test, demangle_test

Test Plan: fbconfig -r buck && fbmake runtests

Reviewed By: njormrod@fb.com

Subscribers: darshan, tjackson, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2063698

Signature: t1:2063698:1431467309:069da6d74bb5c384e7a21e6be19a4b20466bdd92

folly refactorings to better support cross-platform

Summary:
I'm looking into pulling parts of folly (right now, dynamic,
json, and their dependencies) into fbandroid for use as part of xplat.
This diff includes a few kinds of changes: portability fixes for arm;
reduce the size of the code generated by moving non-templated
functions and methods into cpp files; refactoring header usages which
require extra compiler flags on android to cpp files; and slicing up
the libraries a bit differently to reduce dependencies.  This should
all be backward-compatible, and do no harm to fbcode.

Test Plan: runtests, sandcastle

Reviewed By: njormrod@fb.com

Subscribers: darshan, davejwatson, tudorb, dancol, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2057797

Tasks: 7005344

Signature: t1:2057797:1432145435:fa10f129fc669e682da5b4b207fc96986ca035fc

fixup service filter interface

Summary:
Based on a more thourough reading of finagle's interface:
* adds close/isAvailable, which seem very close to thrift's interfaces
* ComposedServices are hardcoded to underlying services, to simplify the code (means extra allocs?)
* Made everything a shared_ptr
* Addd ServiceFactoryFilters

Test Plan: Updated the existing unittests and added some new ones

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037206

Signature: t1:2037206:1432147489:3464d4c12a9e434d4973febcabbf7b2b3a883257

tupleRange, tuplePrepend

Summary:
tupleRange<start, n>(tuple): return a tuple consisting of a range of elements
from the given tuple

tuplePrepend(x, tuple): return a tuple consisting of prepending x to the given
tuple.

For Lispies:

std::get<0>(tuple) is car.
tupleRange<1>(tuple) is cdr.
tuplePrepend(x, tuple) is cons.

Test Plan: test added

Reviewed By: lesha@fb.com

Subscribers: trunkagent, lesha, ilyam, folly-diffs@, yfeldblum, chalfant, jhj, alerer, kma, pamelavagata, tulloch

FB internal diff: D2087568

Signature: t1:2087568:1432164681:18795d0e8bb01f38ffc6949ac233f514ab098355

(Wangle) Fix typo

Summary:
This was supposed to be the `Result` type, since it's called on the
Future returned by the lambda.

Test Plan: Added tests for void and different types in vector/lambda.

Reviewed By: mhl@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2087819

Tasks: 7126300

Signature: t1:2087819:1432142435:72914fa64eff03454774b87a24c426379defab3b

Blame Revision: rFBCODEf229322bc273190a85b5e995dcd8209b1fbf0825

Bump version to 39:0

make AsyncSocket::WriteRequest an interface

Summary: This will allow a subsequent diff to implement file transfers as another type of write request

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant, fugalh, bmatheny

FB internal diff: D2080257

Signature: t1:2080257:1432044566:bcc0724d349879f46e3e58ee672aff7bf37fa5f6