folly.git
9 years agoGet access to EventBaseManager object
Yuri Putivsky [Tue, 31 Mar 2015 06:39:58 +0000 (23:39 -0700)]
Get access to EventBaseManager object

Summary: as title

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1955074

9 years agoMPMCQueue - FPE to invalid_argument for 0 capacity
Daniel Pittman [Tue, 31 Mar 2015 20:33:48 +0000 (13:33 -0700)]
MPMCQueue - FPE to invalid_argument for 0 capacity

Summary:
If an MPMCQueue was initialized with an explicit zero capacity, it would
try to compute stride for zero and trigger SIGFPE which pretty fatally
killed the process -- but in a way that required gdb to figure out the
root cause.

This replaces that with a std::invalid_argument exception, which
includes a description of the problem and should make fixing this user
error much faster.

Test Plan:
GTEST unit test added to verify behaviour; if this didn't work it would
SIGFPE and abort the test script, which is probably a good indicator
that something went wrong. :)

fbconfig -r folly && fbmake runtests

Reviewed By: ngbronson@fb.com

Subscribers: chalfant, folly-diffs@, yfeldblum, rkomorn

FB internal diff: D1930978

Signature: t1:1930978:1427740315:cc06a8b9f3c314b956ae41f813b2f904d3e979c9

9 years agoDefault accept function in Acceptor
Dave Watson [Tue, 31 Mar 2015 16:53:51 +0000 (09:53 -0700)]
Default accept function in Acceptor

Summary: Since Acceptor can be used for TCP or UDP, make an empty default TCP accept function so subclasses don't have to.

Test Plan:
built D1942242, no build errors

contbuild

Reviewed By: yfeldblum@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948940

Signature: t1:1948940:1427498977:26b7bb23e3367ff6695dad86ebf0e60637815a7b

9 years agoEventBase::runImediatelyOrRunInEventBaseThreadAndWait.
Yedidya Feldblum [Tue, 31 Mar 2015 01:17:31 +0000 (18:17 -0700)]
EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

Summary:
[Folly] EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

When you have code that you need to run in the event loop and wait for it to finish, and you yourself might be in the event loop but might be in another thread instead.

Test Plan:
Unit tests:
* `folly/io/async/test/EventBaseTest.cpp`.

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, agallagher, njormrod, ldbrandy, brettp, dougw

FB internal diff: D1953110

Signature: t1:1953110:1427751076:ad2c9f4d5b65ef952cba94b06be887672ee44583

9 years agoFixing predicate inlining
Yuri Putivsky [Thu, 26 Mar 2015 19:04:03 +0000 (12:04 -0700)]
Fixing predicate inlining

Summary:
as title.

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, ridge@

FB internal diff: D1946589

9 years agoExtend IO pool with external optional event_base_manager
Yuri Putivsky [Sat, 28 Mar 2015 01:20:44 +0000 (18:20 -0700)]
Extend IO pool with external optional event_base_manager

Summary: as title

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1950610

9 years agoasyncudpserversocket reuse port
Dave Watson [Mon, 30 Mar 2015 17:17:12 +0000 (10:17 -0700)]
asyncudpserversocket reuse port

Summary:
AsyncUDPSocket supports reuse port, also make it available in AsyncUDPServerSocket.
Has to happen before bind

Test Plan:
?? it builds?

Mutliple binds already work with reuseaddr, but load balancing doesn't work as well.  See http://lwn.net/Articles/542629/ for details

Reviewed By: naizhi@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948932

Signature: t1:1948932:1427473013:8c114458a0746bb2b6957db33c1834179d860c00

9 years agoFuture::filter(A => bool)
Hans Fugal [Mon, 30 Mar 2015 15:44:48 +0000 (08:44 -0700)]
Future::filter(A => bool)

Test Plan: new unit tests

Reviewed By: davejwatson@fb.com

Subscribers: chalfant, bmatheny, trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1828251

Tasks: 6166893

Signature: t1:1828251:1427475565:512463d5728482a40c1da2548a5bed91e7f92d60

9 years agoAdd CaptureFD for log testing (and some glog patterns)
Alexey Spiridonov [Thu, 26 Mar 2015 23:26:09 +0000 (16:26 -0700)]
Add CaptureFD for log testing (and some glog patterns)

Summary:
Without a gadget like this, it's really hard to test logging output. With it, it's really easy. I found about 50 callsites to the various functions across several projects, so folly seems appropriate.

PS The patterns are functions for two reasons:
- Static variables are a pain.
- This leaves the option of adding an optional argument, so you can grep for a particular kind of error string.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933439

Signature: t1:1933439:1427345479:5b3d1c6566a026fdbccb16b382211688e327ea1a

9 years agoAdd EXPECT_{NO_,}_PCRE_MATCH macros
Alexey Spiridonov [Thu, 26 Mar 2015 23:16:15 +0000 (16:16 -0700)]
Add EXPECT_{NO_,}_PCRE_MATCH macros

Summary: There are about 40 callsites to these across a couple of projects, and they seem generally useful.

Test Plan: ```fbconfig folly/experimental/test && fbmake runtests```

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933415

Signature: t1:1933415:1427345307:6587eea3dac74c5d841950ace3c3501d6e3dbe4a

9 years agoTemporaryFile and TemporaryDirectory
Adam Simpkins [Thu, 26 Mar 2015 19:40:56 +0000 (12:40 -0700)]
TemporaryFile and TemporaryDirectory

Summary:
TemporaryFile, TemporaryDirectory, and ChangeToTempDir should all be moveable
objects, but not copiable.  Define default move constructors and move
assignment operators for these classes.  This will prevent copy constructor and
copy assignment operators from being implicitly defined.

Test Plan:
Used this in a new test to write a helper function which created and returned
a new TemporaryFile object using the move constructor.

Reviewed By: yfeldblum@fb.com

Subscribers: doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1945134

Signature: t1:1945134:1427342944:3428327e797ce4b3d362f9a2d2276de6d8b96137

9 years ago(Wangle) Reduce
Hannes Roth [Wed, 25 Mar 2015 22:39:32 +0000 (15:39 -0700)]
(Wangle) Reduce

Summary:
1] The lambda should ble able to return a `Try<T>`. Maybe? Can a `then`
return a `Try<T>` actually? Can fix this with `resultOf`. (Doubling the number of functions to 4.)

2] `initial` and `func` have to be copyable.

Test Plan: Added tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1870996

Tasks: 6025252

Signature: t1:1870996:1427318511:2ae5894b79022da88990835b26d35c4520fdbd29

9 years agoBump version to 32:0 v0.32.0
Noam Lerner [Wed, 25 Mar 2015 22:36:48 +0000 (15:36 -0700)]
Bump version to 32:0

9 years agoAdd a pretty macro for deprecation
Ben Maurer [Wed, 25 Mar 2015 21:33:30 +0000 (14:33 -0700)]
Add a pretty macro for deprecation

Summary: Make it easy for people to mark code as deprecated

Test Plan: Chose a random file and added the FOLLY_DEPRECATED macro to it

Reviewed By: yfeldblum@fb.com

Subscribers: ps, folly-diffs@, yfeldblum

FB internal diff: D1943248

Signature: t1:1943248:1427316689:b3305d134f374697e93d36bafbf0961eec1ef68a

9 years agoRelax CHECK condition in stringAppendfImpl
Sarang Masti [Sat, 14 Mar 2015 17:24:15 +0000 (10:24 -0700)]
Relax CHECK condition in stringAppendfImpl

Summary:
vsnprintf can return fewer bytes and bytes_used if
we print a string containing '\0' using a width
specifier.

Test Plan: -- ran all tests

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1915035

Signature: t1:1915035:1426799341:4aaea928c4bdde1998bf66cf9e2732a53572c6e3

9 years agohttpserver on serverbootstrap (2)
Dave Watson [Tue, 24 Mar 2015 16:40:29 +0000 (09:40 -0700)]
httpserver on serverbootstrap (2)

Summary:
Original diff D1800100, reverted in D1895181.  Issue seems to have been caused by a copy constructor issue.

Fixes: Remove copy constructor, move contructor only. Separate stop() and join(), since join() can happen only after all stop()s have completed when reusing the same IOThreadPoolExecutors.
Test Plan: build and unit tests of relevant product

Reviewed By: praveenr@fb.com

Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1901718

Tasks: 6431975

Signature: t1:1901718:1426885577:46b63bfe117df207be3d273b953c100249c1a7da

Blame Revision: D1895181

9 years agoAdd ChangeToTempDir to TestUtil
Alexey Spiridonov [Sat, 21 Mar 2015 04:58:14 +0000 (21:58 -0700)]
Add ChangeToTempDir to TestUtil

Summary: It's convenient to be able to write tempfiles in the current directory. This provides a one-liner way of doing that in tests.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: simpkins, folly-diffs@, yfeldblum

FB internal diff: D1931918

Signature: t1:1931918:1426913634:f988fb3c5061f5909e309dcaf42742d9b2ed18c6

9 years agoMake folly::Singleton DFATAL as late as possible
Andrii Grynenko [Fri, 20 Mar 2015 19:17:17 +0000 (12:17 -0700)]
Make folly::Singleton DFATAL as late as possible

Summary: We want to print all destructor stack-traces if Singleton destruction happens eventually (even after destroyInstances() was done).

Test Plan:
unit test

Reviewed By: chip@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1929881

Signature: t1:1929881:1426878091:5993d544c0f84bdbf8bcdd420ef01fb7bf88ac3f

9 years agoUdp Acceptor
Dave Watson [Fri, 20 Mar 2015 16:54:08 +0000 (09:54 -0700)]
Udp Acceptor

Summary:
major changes:

1) ServerSocketFactory and AsyncSocketBase to abstract the differences between UDP and TCP async socket.  Could possibly push some of this to the sockets themselves eventually
2) pipeline() is a pipeline between accept/receive of a UDP message, and before sending it to workers.  Default impl for TCP is to fan out to worker threads.  This is the same as Netty.  Since we don't know if the data is a TCP socket or a UDP message, it's a void*, which sucks (netty uses Object msg, so it isn't any different).

Test Plan: Added lots of new tests.  Doesn't test any data passing yet though, just connects/simple receipt of UDP message.

Reviewed By: hans@fb.com

Subscribers: alandau, bmatheny, mshneer, jsedgwick, yfeldblum, trunkagent, doug, fugalh, folly-diffs@

FB internal diff: D1736670

Tasks: 5788116

Signature: t1:1736670:1424372992:e109450604ed905004bd40dfbb508b5808332c15

9 years agofix service memory leak
Dave Watson [Thu, 19 Mar 2015 19:57:13 +0000 (12:57 -0700)]
fix service memory leak

Summary:
This code is only used in a test so far.

Basically it looks like Pipeline has some dependency issues - any addBack() pipelines have to be deleted *after* the main pipeline is destroyed.  Ideally if the pipeline is already closed, the destruction order wouldn't matter.  There is currently no removeBack() call either.   This is a quick fix for the test to just delete stuff in the right order, we can discuss a better solution when @jsedgwick returns

Test Plan: fbconfig --sanitize=address --with-project-version=clang:3.5 --clang folly/wangle/service && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, fugalh, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1903200

Tasks: 6415578

Signature: t1:1903200:1426528298:e109dcc0ec586a505a26cd95a6f20434d22cbd37

9 years agoAdd comment about djb2 hash
Alexey Spiridonov [Thu, 19 Mar 2015 00:12:09 +0000 (17:12 -0700)]
Add comment about djb2 hash

Summary:
In my tests, using djb2 was much worse for any sizable workload. Just use SpookyHashV2.

Test Plan: comment only

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1906439

Signature: t1:1906439:1426117575:f88b7b28b578092c1716433e21150d423d4e94a7

9 years agoSocketAddress::anyAddress should be a method
Sean Cannella [Wed, 18 Mar 2015 22:57:53 +0000 (15:57 -0700)]
SocketAddress::anyAddress should be a method

Summary:
Calling getaddrinfo from a static constructor is not the most
reliable thing to do. Stop doing it.

Test Plan: existing tests

Reviewed By: davejwatson@fb.com

Subscribers: dancol, mzlee, shikong, kmdent, fma, bmatheny, benyluo, ranjeeth, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1924578

Tasks: 6534662

Signature: t1:1924578:1426718586:d73bc2f001095e66fa77fde9c027af050cc26d2a

9 years ago(Wangle) Clean up move constructors
Hannes Roth [Wed, 18 Mar 2015 17:29:19 +0000 (10:29 -0700)]
(Wangle) Clean up move constructors

Summary: I thought this might fix #6120972. But it doesn't. Still a bit of a cleanup in my opinion.

Test Plan: Run all the tests?

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907259

Signature: t1:1907259:1426613567:9e33fe7e9e8a36ba006d4aee604086a56f128893

9 years agoSupport 'min ms to log'
Andrey Goder [Wed, 18 Mar 2015 16:34:09 +0000 (09:34 -0700)]
Support 'min ms to log'

Summary:
Sometimes we only want to log if enough time has passed, e.g. when logging
inside of a loop doing work to detect if things are slow. Add an option for
that.

Test Plan:
build and unit tests of folly/experimental

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1909055

Signature: t1:1909055:1426627731:6d3ad94fd71967ec89ffebc90cd2907dd1631d36

9 years agoFix logFormat
Alexey Spiridonov [Wed, 18 Mar 2015 06:27:55 +0000 (23:27 -0700)]
Fix logFormat

Summary: Ooops during port ;)

Test Plan: unit test

Reviewed By: agoder@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905639

Signature: t1:1905639:1426171227:a6e415046a85f098ac090f069476204fb0367c42

9 years agoBump version to 31:0 v0.31.0
Andre Azevedo [Wed, 18 Mar 2015 03:09:26 +0000 (20:09 -0700)]
Bump version to 31:0

9 years agoFixed README download link to get the correct GTest version
Nicholas Ormrod [Wed, 11 Mar 2015 18:11:12 +0000 (11:11 -0700)]
Fixed README download link to get the correct GTest version

Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan: Clicked link, gtest started downloading

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1901849

Tasks: 6255850

Signature: t1:1901849:1426097071:e887ed8749c70d5e15877082ac0175ce3b56647e

9 years agoBackoffTimeout disallow copy, move, default ctor
Praveen Kumar [Mon, 9 Mar 2015 15:32:53 +0000 (21:02 +0530)]
BackoffTimeout disallow copy, move, default ctor

Summary:
BackoffTimeout disallow copy construction, copy assignment, move
construction, move assignment, and default construction.

Closes https://github.com/facebook/folly/pull/146

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912521

Signature: t1:1912521:1426268413:6edb2d8203d1efd1e825816f03be115ac46e55ca

9 years agoWe might avoid some temporaries
Praveen Kumar [Fri, 13 Mar 2015 04:55:34 +0000 (10:25 +0530)]
We might avoid some temporaries

Summary:
It seems we might avoid temporaries.  To do so we pass arguments
directly to constructors with the help of emplace_back member.

Test Plan:
all folly/tests, make check for 37 tests, passed.

Closes https://github.com/facebook/folly/pull/150

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912505

Signature: t1:1912505:1426268158:086882bb53f3d79c3f3b7b2c362318f09b5ee3be

9 years agoAsyncServerSocket disallow copy, move, and default ctor
Praveen Kumar [Fri, 13 Mar 2015 03:41:03 +0000 (09:11 +0530)]
AsyncServerSocket disallow copy, move, and default ctor

Summary:
AsyncServerSocket disallow copy construction, copy assignment,
move construction, move assignment, and default
construction.

Closes https://github.com/facebook/folly/pull/148

Test Plan: all folly/tests, make check for 37 tests, passed.

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912488

Signature: t1:1912488:1426267982:a9690ce9429016f89630f0ded64b7be9ff80c39d

9 years agofix typo: add backslash to Makefile.am
Alecs King [Tue, 17 Mar 2015 20:38:53 +0000 (13:38 -0700)]
fix typo: add backslash to Makefile.am

Summary: backslash

Test Plan: tests

Reviewed By: aap@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1920872

Signature: t1:1920872:1426624366:c984fecb5bd0c292d5a8309fb320a48872fff6db

Blame Revision: D1890511

9 years agoAdd StringKeyedUnorderedMap to Makefile
Alecs King [Tue, 17 Mar 2015 18:57:37 +0000 (11:57 -0700)]
Add StringKeyedUnorderedMap to Makefile

Test Plan:
unit tests

Reviewed By: bwatling@fb.com

Subscribers: stepan, folly-diffs@, yfeldblum, trunkagent, vikas, alikhtarov

FB internal diff: D1890511

Tasks: 6311387

Signature: t1:1890511:1425924980:71af7c12a09a86b280dc3c9a9ef0756bf56bf639

9 years ago(folly osx) test for the zip, not the dir
Hans Fugal [Mon, 16 Mar 2015 18:02:03 +0000 (11:02 -0700)]
(folly osx) test for the zip, not the dir

Summary: Apparently `configure` creates this subdirectory and then the curl and unzip commands don't run, and then `make check` fails. Instead, do it if the zip file doesn't exist.

Test Plan:
git clone https://github.com/facebook/folly.git
cd folly/folly
/path/to/this/bootstrap-osx-homebrew.sh
make
make check

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, davejwatson

FB internal diff: D1882472

Signature: t1:1882472:1425920470:bdab25967d3b8e52c690d5627168077c65829b92

9 years agoexceptionStr(exception_wrapper)
James Sedgwick [Mon, 16 Mar 2015 16:02:24 +0000 (09:02 -0700)]
exceptionStr(exception_wrapper)

Summary: for consistency with exception and exception_ptr functions provided in String.h

Test Plan: unit

Reviewed By: vloh@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905087

Tasks: 5961362

Signature: t1:1905087:1426101740:670ceab5140250bbecdd1247025d3d70b7774690

9 years agoextend future api with Future<T>::poll
Jeroen Vaelen [Fri, 13 Mar 2015 21:10:55 +0000 (14:10 -0700)]
extend future api with Future<T>::poll

Summary:
Added folly::Future<T>::poll() function with the following semantics:
poll() checks if the promise has been fulfilled, if so it will extract the value of the resulting Try<T> to construct a new Try<T> that is wrapped in an Optional<Try<T>>. If the promise has not been fulfilled an empty Optional is returned.

Test Plan: Added two tests to SugarTest.cpp: one where the promise has been fulfilled and one where it hasn't.

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, arturg, yfeldblum

FB internal diff: D1909480

Tasks: 6166875

Signature: t1:1909480:1426280247:6b239c0f1a15c25ce7664afb4bfb3899993c5775

9 years agoAdd ctor to CPUThreadPoolExecutor to enable custom queue sizes
Matt Dordal [Fri, 13 Mar 2015 17:44:08 +0000 (10:44 -0700)]
Add ctor to CPUThreadPoolExecutor to enable custom queue sizes

Summary:
Seems useful to do without passing in a totally custom queue. Makes
the client code a bit simpler.

Test Plan: used it, saw low memory usage and no crash

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, iaroslav, fugalh, folly-diffs@, jsedgwick, yfeldblum, slarsen

FB internal diff: D1907874

Signature: t1:1907874:1426268010:3159ae339b51f5dbc7fe034644bbc968b92b072c

9 years agoAdd support for iovec for UDP send in folly::AsyncUDPSocket
Naizhi Li [Fri, 13 Mar 2015 17:32:19 +0000 (10:32 -0700)]
Add support for iovec for UDP send in folly::AsyncUDPSocket

Summary:
I will be using multi-buf UDP send soon, so adding the
support in folly::AsyncUDPSocket

Test Plan: Unit tests and turn server

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1907189

Signature: t1:1907189:1426266951:046198e0a009fef085ac7eb44f054c67dfb16ba3

9 years agoadd Cursor::isAtEnd()
Adam Simpkins [Thu, 12 Mar 2015 23:00:31 +0000 (16:00 -0700)]
add Cursor::isAtEnd()

Summary:
Add a helper method to efficiently check if the cursor is at the end of the
IOBuf chain.  This is equivalent to (cursor.totalLength() > 0), but it doesn't
need to walk the entire chain just to tell if it is at the end or not.

Test Plan: Updated the unit tests to contain some checks for isAtEnd().

Reviewed By: jasmeetbagga@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1875345

Signature: t1:1875345:1425006512:49ac246fd0ac7937fdcd6cf1359a841f048c444e

9 years agoProtect unprotected
Stepan Palamarchuk [Wed, 11 Mar 2015 22:43:42 +0000 (15:43 -0700)]
Protect unprotected

Summary: This used to be protected before.

Test Plan: folly/futures tests

Reviewed By: andrii@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907063

Signature: t1:1907063:1426116657:d7430304d5f72a048396e83f8dc6b5a92198fb29

9 years agoCPUThreadPoolExecutor's default queue is quite big
Matt Dordal [Wed, 11 Mar 2015 20:52:39 +0000 (13:52 -0700)]
CPUThreadPoolExecutor's default queue is quite big

Summary:
By default, CPUThreadPoolExecutor uses 100 MB of rss for the CPU queue. This is
substantially higher than expected.

Reduce the default queue size so that it uses much less memory upfront, down
to about 6 MB going by the heap profile.

Reviewed By: davejwatson@fb.com

Subscribers: mwa, ott, iaroslav, jrsherwood, aflock, slarsen, davejwatson, trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1900539

Tasks: 6453427

Signature: t1:1900539:1426103805:34fb7b081058fbde89370b5d9ec24ea572c4b0e2

9 years agofix Makefile.am formatting
Nathan Bronson [Wed, 11 Mar 2015 20:41:31 +0000 (13:41 -0700)]
fix Makefile.am formatting

Summary: D1798929 broke the automake folly build

Test Plan:
1. careful visual inspection
2. build

Reviewed By: alikhtarov@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1906281

Signature: t1:1906281:1426106340:621f1e7d2fb18ae2ec5dc604dea0d8ad4353486e

9 years agoSharedMutex - a small fast scalable reader-writer lock
Nathan Bronson [Wed, 11 Mar 2015 05:21:54 +0000 (22:21 -0700)]
SharedMutex - a small fast scalable reader-writer lock

Summary:
SharedMutex is a reader-writer lock.  It is small, very fast, scalable
on multi-core, and suitable for use when readers or writers may block.
Unlike most other reader-writer locks, its throughput with concurrent
readers scales linearly; it is able to acquire and release the lock
in shared mode without cache line ping-ponging.  It is suitable for a
wide range of lock hold times because it starts with spinning, proceeds
to using sched_yield with a preemption heuristic, and then waits using
futex and precise wakeups.

SharedMutex provides all of the methods of folly::RWSpinLock,
boost::shared_mutex, boost::upgrade_mutex, and C++14's
std::shared_timed_mutex.  All operations that can block are available in
try, try-for, and try-until (system_clock or steady_clock) versions.
Both reader-priority and writer-priority versions are provided.
Writer-priority is the default at the moment.

In my tests SharedMutex is as good or better than the other reader-writer
locks in use at Facebook for almost all use cases, sometimes by a wide
margin.  (If it is rare that there are actually concurrent readers
then RWSpinLock can be a few nanoseconds faster.)  I compared it to
folly::RWSpinLock, folly::RWTicketSpinLock64, boost::shared_mutex,
pthread_rwlock_t, and an internal RWLock that uses spinlocks to guard
its state and pthread_mutex_t+pthread_cont_t to perform blocking.
(The other ReadWriteMutex-s I found use pthread_rwlock_t underneath.)
It is generally as good or better than the rest when evaluating size,
speed, scalability, or latency outliers.  In the corner cases where
it is not the fastest (such as single-threaded use or heavy write
contention) it is never very much worse than the best.  See the bottom
of SharedMutexTest.cpp for lots of microbenchmark results.

Test Plan:
1. new unit tests
2. new microbenchmarks included here
3. uncommitted microbenchmark from bmaurer's RWSleepLock
4. replace admarket's RWSpinLock and RWLock with SharedMutex, observe
neutral adindexer perf in canary
5. replace multifeed's thrift ReadWriteLock with SharedMutex, observe
neutral perf in canary

Reviewed By: hans@fb.com

Subscribers: fbcode-common-diffs@, tnovak, march, davejwatson, trunkagent, philipp, folly-diffs@, yfeldblum, bwatling, bmaurer, bol, marccelani, adri, strager, macsyz, dzhulgakov, zamsden

FB internal diff: D1798929

Signature: t1:1798929:1425575976:1c9221317eaa47628a2b8c374f90c7a2d4e3f0f9

9 years agoAdd null check to avoid crash in unit tests that use mock singletons.
Adrian Hamza [Mon, 9 Mar 2015 21:04:47 +0000 (14:04 -0700)]
Add null check to avoid crash in unit tests that use mock singletons.

Summary: Some unit tests owned by my team are failing due to segmentation fault in SingletonHolder<T>::registerSingletonMock -> SingletonHolder<T>::destroyInstance.

Test Plan: Run folly unit tests and my unit tests.

Reviewed By: henryf@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1873889

Signature: t1:1873889:1425925156:29d54092939d7e9debea3fd55f7105fd320e987e

Blame Revision: 91f4942e

9 years agoAdd JSON Schema Validator
Andrey Goder [Sat, 7 Mar 2015 00:06:18 +0000 (16:06 -0800)]
Add JSON Schema Validator

Summary:
This is a validator for  JSON schema (http://json-schema.org/) that works on
folly::dynamic. Apparently there are no good open source ones for C++,
especially not if you want to use folly::dynamic. I am going to use this to
validate JSON configs.

It supports basically everything from the standard, except for fetching schemas
via http, and using id refs. It supports enough to check schemas against the
metaschema.

Currently you can define a schema that will crash on validation, if it's
infinitely self-recursive. See the unit test case that reproduces this.
Fixing this seems hard though, so I didn't bother. It would also probably
be slower for normal usage.

Test Plan: unit test

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1847657

Signature: t1:1847657:1425605163:635dc523aeda1b588c3634d0dc1a48d50a53db79

9 years agoBump version to 30:0 v0.30.0
Viswanath Sivakumar [Fri, 6 Mar 2015 23:32:28 +0000 (15:32 -0800)]
Bump version to 30:0

9 years agoRemove unnecessary includes from FBVector
Mark Isaacson [Fri, 6 Mar 2015 01:37:46 +0000 (17:37 -0800)]
Remove unnecessary includes from FBVector

Summary:
Remove unnecessary includes from FBVector

Gonna see just how cleaned-up sandcastle really is :P

Test Plan: Fix errors shown in contbuild

Reviewed By: njormrod@fb.com

Subscribers: cold-storage-diffs@, jcoens, ldbrandy, trunkagent, oleksandr, netego-diffs@, mmandal, weiyan, ws_dev_diffs, jmkaldor, jhunt, fbcode-common-diffs@, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1873145

Signature: t1:1873145:1425505591:9cb345053833afa70f543e2f7acd310ab80b8676

9 years agoBump version to 29:0 v0.29.0
Nicholas Ormrod [Thu, 5 Mar 2015 22:44:26 +0000 (14:44 -0800)]
Bump version to 29:0

9 years agohttpserver on serverbootstrap
Dave Watson [Thu, 5 Mar 2015 18:25:54 +0000 (10:25 -0800)]
httpserver on serverbootstrap

Summary:
Cleans up the httpserver startup code nicely.  The only major change to ServerBootstrap was a check if bind failed to throw an exception.

(depends on D1732895)

Test Plan:
fbconfig -r folly/wangle/bootstrap proxygen/httpserver; fbmake runtests

fbconfig -r dfsrouter; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: yfeldblum, cgheorghe, trunkagent, doug, fugalh, bmatheny, folly-diffs@, jsedgwick

FB internal diff: D1800100

Signature: t1:1800100:1424733970:67a61a22d2affadea16d2fd725003915326077b2

9 years agoBump version to 28:0 v0.28.0
Sara Golemon [Thu, 5 Mar 2015 18:22:59 +0000 (10:22 -0800)]
Bump version to 28:0

9 years agoBugfix uriEscapeTable generate.
Ram Kumar Rengaswamy [Sat, 14 Feb 2015 20:45:07 +0000 (15:45 -0500)]
Bugfix uriEscapeTable generate.

Summary:
generate_escape_tables.py uses python range function which does not include the end of the range.

Closes #131

Test Plan: See attached test case.

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1871692

Signature: t1:1871692:1425076132:2438ab7554fe87bdef17c82ff27713811a270d7c

9 years agoAdd FunctionScheduler to Makefile.am
Nicholas Ormrod [Thu, 5 Mar 2015 03:50:50 +0000 (19:50 -0800)]
Add FunctionScheduler to Makefile.am

Summary: D1845525 was missing some makefile changes.

Test Plan: Examine similar change in D1732895

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1890197

Signature: t1:1890197:1425526255:e9a23fd96a89d4463cbdfda1385d4fd5f5781e8e

9 years agoMoveWrapper::move()
Hans Fugal [Wed, 4 Mar 2015 20:42:31 +0000 (12:42 -0800)]
MoveWrapper::move()

Summary: sugar providing `foo.move()` instead of `std::move(*foo)`.

Test Plan: Inspection. Using it in another diff.

Reviewed By: yfeldblum@fb.com, hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum

FB internal diff: D1882208

Signature: t1:1882208:1425331389:a9c09ad2739838c829e2afdad64e985810154226

9 years agoBump version to 27:0 v0.27.0
Alecs King [Tue, 3 Mar 2015 03:31:37 +0000 (19:31 -0800)]
Bump version to 27:0

9 years agoAdd test for running after terminate
Subodh Iyengar [Mon, 2 Mar 2015 22:46:46 +0000 (14:46 -0800)]
Add test for running after terminate

Summary:
Add a test to check what will happen if
someone tries to run a job on evb after
termination.

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: seanc, folly-diffs@, yfeldblum

FB internal diff: D1882308

Signature: t1:1882308:1425333248:7ab6692eb0b866fcc9685048eb385bacd90023d1

9 years agomakeMoveWrapper(lvalue)
Hans Fugal [Mon, 2 Mar 2015 21:27:49 +0000 (13:27 -0800)]
makeMoveWrapper(lvalue)

Summary: Because why not?

Test Plan: new unit tests

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum, marccelani

FB internal diff: D1866123

Signature: t1:1866123:1425331395:742369597757456e5925af55c5d4198b607126f1

9 years agoMacOS doesn't have malloc_usable_size()
Sara Golemon [Mon, 2 Mar 2015 05:57:08 +0000 (21:57 -0800)]
MacOS doesn't have malloc_usable_size()

Summary:
Use malloc_size() instead

Test Plan: HHVM build on MacOS

Reviewed By: joelm@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1875364

Signature: t1:1875364:1425061479:7d8cec757c186fa8c30b6500fca6b2fe4c1ed137

9 years agoCreating a flatten() method to get the inner Future<T> value out of a Future<Future...
Dor Gross [Sat, 28 Feb 2015 01:17:18 +0000 (17:17 -0800)]
Creating a flatten() method to get the inner Future<T> value out of a Future<Future<T>>.

Summary: Method will only be applicable for types of the form Future<Future<T>>, calling flatten() on a Future<T> where <T> != Future<R> for any <R> will result with a compile time error.

Test Plan: Added tests in FutureTest.cpp.

Reviewed By: hans@fb.com

Subscribers: ldbrandy, trunkagent, folly-diffs@, jsedgwick, yfeldblum, icanadi

FB internal diff: D1843581

Tasks: 6166903

Signature: t1:1843581:1424399517:7146924db3c238b77dd309f1d5916e44a7e81968

9 years agoAdd const to the variable: needle
zhaokai [Fri, 27 Feb 2015 07:42:32 +0000 (15:42 +0800)]
Add const to the variable: needle

Summary:
Make it clear that the variable is const

Closes #139

Test Plan:
authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878361

Signature: t1:1878361:1425071448:6871577ddb34f4a8dc30ea37842f9950c3825dde

9 years agoUpdate UnitTest of MapUtilTest.cpp
zhaokai [Fri, 27 Feb 2015 07:20:35 +0000 (15:20 +0800)]
Update UnitTest of MapUtilTest.cpp

Summary:
1. Split the original simple test
2. Add unit test of function: get_ref_default

Closes #138

Test Plan: authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878603

Signature: t1:1878603:1425073330:914a2365fa648142d11b41b387423d78e52f2e08

9 years agoFix comment of Benchmark.h
zhaokai [Fri, 27 Feb 2015 07:51:45 +0000 (15:51 +0800)]
Fix comment of Benchmark.h

Summary:
Remove one of the "one" in the comment of BENCHMARK

Closes: #140
Test Plan: Inspection.

Reviewed By: markisaa@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1877997

Signature: t1:1877997:1425063475:c53c09d105809de9a9070f464fb0571a4dd1e5d1

9 years agoattempt at putting thrift2 on ChannelPipeline
Dave Watson [Fri, 27 Feb 2015 18:14:46 +0000 (10:14 -0800)]
attempt at putting thrift2 on ChannelPipeline

Summary:
Basically a ripoff of Dave's D1483148 but with ChannelPipeline instead of rx.
There's going to be a long tail of unit tests here, this is still pretty broken...

Not convinced that this is the way to go but I wanted to get a feel for what it would be like to plug it into an IRL system.
I think the simplicity/clarity of the handler implementations and to some extent the pipeline spec is an improvement over the rx way
but I'm less sure about everything else.

Test Plan: compiles and kind of works some of the time but not really

Reviewed By: alandau@fb.com

Subscribers: wormhole-diffs@, mcduff, hitesh, doug, yfeldblum, folly-diffs@, jsedgwick, subodh, andrewcox, njormrod, trunkagent, fugalh, alandau, bmatheny, mshneer

FB internal diff: D1618704

Tasks: 5981284

Signature: t1:1618704:1424980854:83c6193b1156be2ec65cf79f9f2ad1af670da7f8

9 years agoAdd willEqual to C++ Futures Code
Monica Lee [Fri, 27 Feb 2015 17:37:25 +0000 (09:37 -0800)]
Add willEqual to C++ Futures Code

Summary: Added willEqual function and wrote unit tests for it.

Test Plan: fbconfig --clang folly/futures && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: hannesr, trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1859840

Tasks: 6166911

Signature: t1:1859840:1424967149:865ee96ab4d3f5dbf17eb371b2ac3ccb5066ac87

9 years agosupport IndexedMemPool for types without default constructor
Nathan Bronson [Thu, 26 Feb 2015 22:54:59 +0000 (14:54 -0800)]
support IndexedMemPool for types without default constructor

Summary:
This diff gives IndexedMemPool<T> emplace-like semantics when T
is not trivial.

Test Plan:
1. new unit tests
2. LifoSem benchmark

Reviewed By: march@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1874941

Signature: t1:1874941:1424987308:61bbe7b7e5e6df625a6208cd873c65e523a79fa0

9 years agoCopyright 2014->2015
Nicholas Ormrod [Thu, 26 Feb 2015 21:00:59 +0000 (13:00 -0800)]
Copyright 2014->2015

Summary:
Update copyright notices in folly to 2015

find folly/ -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'find . -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: ldbrandy, shilin, sdwilsh, fugalh, njormrod, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1869183

Signature: t1:1869183:1424823514:a2dd33741a86bf099d5c9e3d8702e6cca6dd672f

9 years agoEventBase::runAfterDelay to throw an exception
Haim Grosman [Wed, 4 Feb 2015 17:15:10 +0000 (09:15 -0800)]
EventBase::runAfterDelay to throw an exception

Summary:
EventBase::runAfterDelay to throw an exception in case it fails to
schedule a proper execution of the given callback
(instead of silently returning false)

it appears to be the right thing to do, since:
@davejwatson: Digging through the layers of code, it appears this can only fail if
epoll_ctl() with EPOLL_CTL_ADD fails. Ignoring libevent bugs, it looks
like the only relevant errors could be ENOMEM or ENOSPC. So
nonrecoverable

Test Plan: Unit tests

Reviewed By: anca@fb.com

Subscribers: yzhan, haijunz, simpkins, net-systems@, varunk, zeus-diffs@, nli, dfechete, fugalh, atlas2-eng@, alandau, bmatheny, everstore-dev@, zhuohuang, wormhole-diffs@, anca, mwa, jgehring, oujin, alikhtarov, fuegen, mshneer, wch, bil, sanketh, zippydb, maxwellsayles, jsedgwick, trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, andrewcox, unicorn-diffs@, tw-eng@, xie, kennyyu, yfeldblum, folly-diffs@, davejwatson

FB internal diff: D1805125

Signature: t1:1805125:1424927912:8bebb4c3b9f1fa189c0ce97b12cdb8f95dba97ae

9 years agoUpdating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'
John Ehrhardt [Thu, 26 Feb 2015 04:52:21 +0000 (20:52 -0800)]
Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'

Summary:
Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'.
Updated Decimal Integers 'd' use of separators to directly use commas ',' and grouping of 3 digits.
Updated Numbers use of separators and grouping to follow locale settings.

Test Plan:
Wrote unit tests to validate the insertion of thousandsSeparators works.
Note that the numbers unit test does not modify the locale since the test cases are not thread safe.

Reviewed By: lesha@fb.com

Subscribers: tudorb, trunkagent, folly-diffs@, yfeldblum, aric

FB internal diff: D1821156

Tasks: 6087521

Signature: t1:1821156:1424923837:d0fb383a07fd733375b72b1905e6112afa141265

9 years agoReduce AsyncSocket code size
Sean Cannella [Wed, 25 Feb 2015 18:34:25 +0000 (10:34 -0800)]
Reduce AsyncSocket code size

Summary:
Manual inspection of AsyncSocket.o shows the constructors are generating ugly
(and large) code. Call the smaller constructors instead of duplicating the code. This
reduces the debug .o size from ~1.9MB to ~1.6MB.

Test Plan: existing tests

Reviewed By: benyluo@fb.com

Subscribers: mzlee, folly-diffs@, yfeldblum, mathieubaudet, subodh, pgriess, benyluo

FB internal diff: D1870896

Signature: t1:1870896:1424885638:d37fc79c0f88d04109c8bb73e632dc506b9f773b

9 years agoFuture::ensure
Hans Fugal [Wed, 25 Feb 2015 17:54:53 +0000 (09:54 -0800)]
Future::ensure

Summary: Unconditionally execute the action, passthrough semantics.

Test Plan: new unit test

Reviewed By: bmatheny@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1868837

Tasks: 6166860

Signature: t1:1868837:1424820181:0e83f54b59d7091dac60ab65feb387992e8ae89c

9 years agofolly/FileUtil.h: fix compiler warning signed/unsigned comparison
Sven Over [Wed, 25 Feb 2015 14:54:21 +0000 (06:54 -0800)]
folly/FileUtil.h: fix compiler warning signed/unsigned comparison

Summary:
writeFull() returns ssize_t and without proper casting, comparing
it with data.size() triggers a compiler warning (which is
treated as an error) in the gcc-4.9-glibc-2.20 toolchain.

Test Plan: fbmake runtests

Reviewed By: mhx@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1870710

Signature: t1:1870710:1424874592:f51026c35196d763ad4b192d43c8ccee0255b41d

9 years agoImprovement fbvector.computePushBackCapacity
Nicholas Ormrod [Wed, 25 Feb 2015 00:51:24 +0000 (16:51 -0800)]
Improvement fbvector.computePushBackCapacity

Summary:
See the github pull request at https://github.com/facebook/folly/issues/135

Important optimizations: remove one of the branches and change ##empty()## to ##capacity()==0##

Test Plan:
Ran extended fbvector test suite (re-enabled in experimental/njormrod)

Also

fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1869112

Tasks: 6338531

Signature: t1:1869112:1424823901:d2d7331aef82edad1e8c159005cc1c7185550d0c

9 years agoRemoving call to google::setThreadName from FunctionScheduler
Andrei Alexandrescu [Tue, 24 Feb 2015 21:09:36 +0000 (13:09 -0800)]
Removing call to google::setThreadName from FunctionScheduler

Summary: Funnily, google::setThreadName is Facebook specific: it's introduced only in the FB build of glog. This diff removes the call because folly should not depend on FB's glog.

Test Plan: made sure the line was deleted

Reviewed By: ldbrandy@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum

FB internal diff: D1868249

Tasks: 6345935

Signature: t1:1868249:1424812595:2a084754d614ab9b7b6168454f33638882e3f6db

9 years agoEmulatedFutexAtomic doesn't copy or move
Praveen Kumar [Wed, 11 Feb 2015 14:29:30 +0000 (19:59 +0530)]
EmulatedFutexAtomic doesn't copy or move

Summary:
EmulatedFutexAtomic concisely says here that it doesn't copy construct,
move construct, copy assign, or move assign.

Closes #130

Test Plan:
Inspection and all unit tests.

```
Summary (total time 66.97s):
PASS: 2029
FAIL: 0
SKIP: 0
FATAL: 0
TIMEOUT: 0
```

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1868033

Signature: t1:1868033:1424808823:d9dd618981b29b5949c00190d69d670d96e60e07

9 years agoadd preprocessor to hide unused functions
Beny Luo [Mon, 23 Feb 2015 15:12:29 +0000 (07:12 -0800)]
add preprocessor to hide unused functions

Summary:
encodeVarintToIOBuf and decodeVarintFromCursor are not used when
FOLLY_HAVE_LIBLZMA and FOLLY_HAVE_LIBLZ4 are false, which will trigger
-Werror,-Wunused-function on compile on mobile.

Test Plan: unit test

Reviewed By: seanc@fb.com

Subscribers: meisner, trunkagent, seanc, folly-diffs@, yfeldblum

FB internal diff: D1864016

Signature: t1:1864016:1424702813:75535ea92ac16385ce5c272d93bba507ccbb67a1

9 years agoFix folly::Singleton DFATAL
Andrii Grynenko [Sat, 21 Feb 2015 01:22:54 +0000 (17:22 -0800)]
Fix folly::Singleton DFATAL

Summary:
Previously leak-fatal could never happen because we were incorrectly checking singleton state.
Sending this diff to see how many tests will actually fail and potentially fix worst offenders.

Test Plan: unit tests

Reviewed By: stepan@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1838405

Signature: t1:1838405:1424478983:94cda86ed57f38b0cf626b74804fbc168d182c66

9 years agoOptimize perf of EventBase with new option.
Andre Pinto [Fri, 20 Feb 2015 22:51:39 +0000 (14:51 -0800)]
Optimize perf of EventBase with new option.

Summary:
Add option to disable time measurement in EventBase::loopBody()
to improve performances when time measurement is not used.

Test Plan:
Unit tests, benchmarking and manual tests.

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1846020

Tasks: 6086197

Signature: t1:1846020:1424459703:9686a84027e1137ee43eb0458b0baa4fee3a8abf

9 years agoAdd AutoTimer to folly
Andrey Goder [Fri, 20 Feb 2015 21:48:36 +0000 (13:48 -0800)]
Add AutoTimer to folly

Summary:
This 'moves' bistro::AutoTimer into folly. I actually changed it a bit, so I'm
not migrating any existing uses in this diff, but I can do that as a followup.
We're using it in various places now, so it doesn't make sense to have it in
bistro.

Basically this class lets you conveniently time a series of things while
printing log messages, e.g.:
{
AutoTimer t("Done");
processA();
t.log("processA() complete");
}

would print
"processA() complete in XXX seconds"
"Done in YYY seconds"

I changed some things from bistro::AutoTimer:
- Use std::chrono::time_point<Clock> to store the start time.
- I removed the gflags. It seems weird to rely on those to control the timer,
since they would affect all uses of this. If a specific user of the timer
wants to actually control timing at runtime, I think it's better to use a
custom logger.
- I removed the 'min log duration'. As implemented it actually made the
constructor conflict: AutoTimer("foo", 1.2) is ambiguous. Also, it should
probably be set on the logger, so this likewise feels like a place where you
could make a custom logger.

I'm not sure that the way I use the default logger as a template argument is
best, so please suggest improvements to that.

Test Plan: unit test

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1860794

Signature: t1:1860794:1424458219:a2dd0d428aedf063a888b2937b1443c7142554e8

9 years ago(folly/futures) Fix get() bug
Hans Fugal [Fri, 20 Feb 2015 21:15:15 +0000 (13:15 -0800)]
(folly/futures) Fix get() bug

Summary: I thought this was a race, but I think now it was something to do with using a value that had been moved out or something. Anyway, this refactor is cleaner and consolidates a few methods so it's all kinds of fuzzy feelings.

Test Plan: unit tests

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1861376

Tasks: 6298004

Signature: t1:1861376:1424465861:736353ab3174656fec9e036e0ebd964970da38d0

9 years agorevert D1471958. ParseError is not intended to be caught
Ranjeeth Dasineni [Wed, 18 Feb 2015 09:24:34 +0000 (01:24 -0800)]
revert D1471958. ParseError is not intended to be caught

Summary:
a couple of files were catching this exception and included the .cpp file :O. So we moved
to .h before realizing this was not intended to be caught. reverting the move to .h. codemod will follow.

Test Plan: fbmake -r folly && fbmake && fbmake runtests

Reviewed By: delong.j@fb.com

Subscribers: trunkagent, doug, seanc, folly-diffs@, yfeldblum

FB internal diff: D1854301

Tasks: 6245781

Signature: t1:1854301:1424375745:7e226196315fac99835752896cc5e549e91df2da

9 years ago(futures) Make executors sticky
Hans Fugal [Fri, 20 Feb 2015 17:03:14 +0000 (09:03 -0800)]
(futures) Make executors sticky

Summary:
Instead of returning a deactivated future, have `via` just set the executor. Propagate the executor from `then`. This fixes the `via().get()` problem, and has semantics similar to before for `via().then().then()`.

However, the semantics are now slightly different - each `then` goes back through the executor. This adds some overhead and tweaks the semantics (e.g. if the executor is a threadpool it might execute subsequent `then`s in another thread). However, with `futures::chain` recently introduced, and any other convenience methods that you can dream up and make a case for, we can reasonably get the old once-through-the-executor behavior when performance or other concerns demand it. e.g. `via().then(futures::chain(a, b, c))`.

Test Plan: unit tests

Reviewed By: hannesr@fb.com

Subscribers: zeus-diffs@, mmandal, steveo, rituraj, trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick, davejwatson

FB internal diff: D1839691

Tasks: 6048744

Signature: t1:1839691:1424397180:ca0b0ea7b3867769ab8abd254a510059df67011e

9 years agoRevert: (Wangle) Clean up move constructors
Hannes Roth [Thu, 19 Feb 2015 23:10:49 +0000 (15:10 -0800)]
Revert: (Wangle) Clean up move constructors

Summary:
This reverts commit 1e407b48d379a41f32e7a980285dbdf4dadb2e33.

Test Plan: revert-hammer

Reviewed By: yitingli@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1858994

Signature: t1:1858994:1424386588:5c4608ecbe1f9ab1108ac12e506b157b54d0078b

9 years ago(Wangle) Clean up move constructors
Hannes Roth [Thu, 19 Feb 2015 21:10:26 +0000 (13:10 -0800)]
(Wangle) Clean up move constructors

Summary:
These were doing more assignments than necessary.
1) We don't need to set `core_` to `nullptr` and then immediately
overwrite it with `other.core_`.
2) Don't need to swap the booleans (this was actually an unitialized
memory access).

Test Plan: Ran all the tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1852386

Signature: t1:1852386:1424375154:25997209e76ccd16169125597e868932a5143ffb

9 years agoRemove @/folly dependency from folly/io:compression
Michael Lee [Thu, 19 Feb 2015 18:44:16 +0000 (10:44 -0800)]
Remove @/folly dependency from folly/io:compression

Summary: This diff should let us include folly/io:compression into iOS and Android projects without pulling in a lot of dependencies and should allow the use of zlib.

Test Plan: fbconfig -r folly/io/tests && fbmake runtests

Reviewed By: seanc@fb.com

Subscribers: trunkagent, bmatheny, folly-diffs@, yfeldblum, seanc, benyluo

FB internal diff: D1853058

Tasks: 6245912

Signature: t1:1853058:1424370881:2ea243d93e6041502e5e356fa430c2483f652b40

9 years agouse ServerBootstrap
Dave Watson [Thu, 19 Feb 2015 18:06:27 +0000 (10:06 -0800)]
use ServerBootstrap

Summary:
Use server bootstrap.

* code reuse
* Allows multiple accept threads easily.

Accept now doesn't happen in the server's eventBase, but a new thread.

I think I hit all the corner cases involving existingSocket, duplex, stopListening(), etc.

There are a lot of minor changes here to support all the cases, let me know if I should split anything to separate diffs

Test Plan: fbconfig -r thrift/lib/cpp2; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: haijunz, yfeldblum, ruibalp, fbcode-common-diffs@, davejwatson, jsedgwick, trunkagent, doug, alandau, bmatheny, mshneer, folly-diffs@, mcduff, andrewcox, fugalh

FB internal diff: D1732895

Tasks: 5788102

Signature: t1:1732895:1423087631:379bbb131c35ce2221801bc7cec226f87ba0b1d9

9 years agosome moar unittests
Dave Watson [Wed, 18 Feb 2015 17:28:09 +0000 (09:28 -0800)]
some moar unittests

Summary: from discussion

Test Plan: unit tests

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1829282

Signature: t1:1829282:1423180907:3630dac1378750b05f316c672fbbd71138d2bc0a

9 years agoUse folly's async udp socket
Dave Watson [Wed, 18 Feb 2015 16:12:06 +0000 (08:12 -0800)]
Use folly's async udp socket

Summary: They are functionaly equivalent, no need to have more than one

Test Plan: fbconfig -r realtime/voip; fbmake runtests

Reviewed By: naizhi@fb.com

Subscribers: trunkagent, doug, ps, bmatheny, folly-diffs@, yfeldblum

FB internal diff: D1828044

Tasks: 6154007

Signature: t1:1828044:1423165354:f71d2fd28ca76a8f67a597c747f8578d2909823c

9 years agoKill get_fast/get_weak_fast Singletonn API
Andrii Grynenko [Wed, 18 Feb 2015 01:03:58 +0000 (17:03 -0800)]
Kill get_fast/get_weak_fast Singletonn API

Summary: After D1827390 regular get and get_weak is comparable to Meyers and static singletons, so there's no need to keey _fast APIs.

Test Plan: benchmark && fbmake

Reviewed By: mshneer@fb.com

Subscribers: trunkagent, configerator-diffs@, fbcode-common-diffs@, mcduff, hitesh, mshneer, fugalh, acampi, alikhtarov, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1843219

Signature: t1:1843219:1424216566:f2f182a1c86bb5f0fb1f978d8c6b7a4388198f5f

9 years agoMigrate FunctionScheduler from common/concurrency/ to folly/experimental/
Andrei Alexandrescu [Thu, 12 Feb 2015 21:17:31 +0000 (13:17 -0800)]
Migrate FunctionScheduler from common/concurrency/ to folly/experimental/

Summary: This are the open-source-related additions. A separate diff will replace the current implementation in common/concurrency/ to use this one.

Test Plan: fbmake runtests

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, net-systems@, folly-diffs@, yfeldblum

FB internal diff: D1845525

Signature: t1:1845525:1424207291:d30e3c5e85222527c2aff39c1250aa1e41b9a2cf

9 years agoReformat comment for readability, 80-col rule, and gcc cleanliness
Sara Golemon [Tue, 17 Feb 2015 19:45:21 +0000 (11:45 -0800)]
Reformat comment for readability, 80-col rule, and gcc cleanliness

Summary:
The continuation at the end of the line with the while
was causing gcc to complain about it because it's a
single line comment.

Reformat it into a multiline comment and wrap it to 80 chars.

Test Plan: It's just a comment...

Reviewed By: paulbiss@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1850876

Signature: t1:1850876:1424124973:b2afbc2a8177ac6d11a21bf5a52093bcfd9cbb3e

9 years ago(Wangle) Shrink Core to 128 bytes
Hannes Roth [Tue, 17 Feb 2015 19:15:36 +0000 (11:15 -0800)]
(Wangle) Shrink Core to 128 bytes

Summary: I think this makes sense, especially if `interrupt_` is really rare.

Test Plan:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             101.25ns    9.88M
promiseAndFuture                                  92.86%   109.03ns    9.17M
withThen                                          28.96%   349.65ns    2.86M
----------------------------------------------------------------------------
oneThen                                                    337.33ns    2.96M
twoThens                                          58.46%   577.06ns    1.73M
fourThens                                         32.42%     1.04us  961.23K
hundredThens                                       1.45%    23.24us   43.04K
----------------------------------------------------------------------------
no_contention                                                4.45ms   224.48
contention                                        72.89%     6.11ms   163.61
----------------------------------------------------------------------------
throwAndCatch                                                6.34us  157.77K
throwAndCatchWrapped                              95.65%     6.63us  150.91K
throwWrappedAndCatch                             142.58%     4.45us  224.95K
throwWrappedAndCatchWrapped                      488.86%     1.30us  771.27K
----------------------------------------------------------------------------
throwAndCatchContended                                        1.01s  987.01m
throwAndCatchWrappedContended                    100.03%      1.01s  987.27m
throwWrappedAndCatchContended                    148.37%   682.86ms     1.46
throwWrappedAndCatchWrappedContended            18765.19%     5.40ms   185.21
============================================================================

Compare to D1843129#31 (seems to be an improvement?)

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1847843

Signature: t1:1847843:1423855151:3602d49df245eda3f85325ab5efd63514213ca44

9 years agoExceptionWrapperTest.cpp vs clang's new warning about side effects in typeid argument...
Jim Meyering [Sat, 14 Feb 2015 01:20:06 +0000 (17:20 -0800)]
ExceptionWrapperTest.cpp vs clang's new warning about side effects in typeid argument list

Summary:
Disable the -Wunevaluated-expression warning for this test:

EXPECT_EQ(typeid(ie), typeid(IntException));

Otherwise, clang warns about the unevaluated expression because
the expansion of EXPECT_EQ applies sizeof to an expression
with side effects:

folly/test/ExceptionWrapperTest.cpp:179:122: error: expression with side effects has no effect in an unevaluated context [-Werror,-Wunevaluated-expression]
switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal:: EqHelper<(sizeof(::testing::internal::IsNullLiteralHelper(typeid(ie))) == 1)>::Compare("typeid(ie)", "typeid(IntException)", typeid(ie), typeid(IntException)))) ; else ::testing::internal::AssertHelper(::testing::TestPartResult::kNonFatalFailure, "folly/test/ExceptionWrapperTest.cpp", 179, gtest_ar.failure_message()) = ::testing::Message();

Test Plan:
Ensure this compiles with clang-3.4, 3.5 and clang:dev.
I.e., ensure that this prints PASS at the end.

for i in '' 3.5 dev; do
test -n "$i" && i=--with-project-version=clang:$i
fbconfig -r --clang $i folly/test:exception_wrapper_test && fbmake dbgo
done && echo PASS

Reviewed By: mpawlowski@fb.com

Subscribers: mathieubaudet, folly-diffs@, yfeldblum

FB internal diff: D1850778

Tasks: 6244745

Signature: t1:1850778:1424111029:136478e9a3cc3a219047547d501de4c579a1a181

9 years agowangle: make new pragma use portable to older clang
Jim Meyering [Sat, 14 Feb 2015 00:37:37 +0000 (16:37 -0800)]
wangle: make new pragma use portable to older clang

Summary:
* folly/wangle/channel/test/MockChannelHandler.h:
Avoiding a new warning that rendered this file uncompilable
by clang-3.5 and older.  Add cpp conditionals to protect
against that.

Test Plan:
Ensure it works with clang-3.4, 3.5 and clang:dev.
I.e., ensure that this prints PASS at the end.

for i in '' 3.5 dev; do
test -n "$i" && i=--with-project-version=clang:$i
fbconfig -r --clang $i folly/wangle/channel/test && fbmake runtests_dbgo
done && echo PASS

Reviewed By: meyering@fb.com

Subscribers: mpawlowski, fugalh, mathieubaudet, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1849474

Tasks: 6244745

Blame Revision: D1848330
@build-break

9 years agofolly: OptionalTest.cpp: suppress a -Wself-move warning
Jim Meyering [Fri, 13 Feb 2015 22:37:55 +0000 (14:37 -0800)]
folly: OptionalTest.cpp: suppress a -Wself-move warning

Summary:
Clang's new -Wself-move warning dings this code:

b = std::move(b);

But in this case, that is a deliberate test of
this capability, so we suppress that particular
warning for the duration of this small test function.

Test Plan:
Run this and note that it now compiles: (it passes the tests, too)
fbconfig --clang --with-project-version=clang:dev -r folly/test:optional_test && fbmake dbgo

Reviewed By: watashi@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1849689

Tasks: 6244745

Signature: t1:1849689:1423877932:4f479d682bd8ba97dcabed7fc29ce0a22c013969

@build-break (using this to work around bogus push failure: https://phabricator.fb.com/P19730606)

9 years agofolly/ExceptionWrapper.h: avoid warning about typeid operand side effects
Jim Meyering [Fri, 13 Feb 2015 17:23:08 +0000 (09:23 -0800)]
folly/ExceptionWrapper.h: avoid warning about typeid operand side effects

Summary:
* folly/ExceptionWrapper.h (class_name): Clang is not yet smart enough to
see that there is no harm in dereferencing an "item_" that we've just
verified is non-NULL.  Accommodate it, to avoid this:
./folly/ExceptionWrapper.h:199:30: error: expression with side effects will be evaluated despite being used as an operand to 'typeid' [-Werror,-Wpotentially-evaluated-expression]
return demangle(typeid(*item_));
^

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: mhorowitz@fb.com

Subscribers: trunkagent, yfeldblum, folly-diffs@

FB internal diff: D1848327

Tasks: 6244745

Signature: t1:1848327:1423869597:a58c9e9e3671befb78ae07fbd274a13d08ffb2a7

Blame Revision:

9 years agoclang:dev rejects vector<atomic<T>> in EventBaseTest.cpp.
Yedidya Feldblum [Fri, 13 Feb 2015 22:35:04 +0000 (14:35 -0800)]
clang:dev rejects vector<atomic<T>> in EventBaseTest.cpp.

Summary: [Folly] clang:dev rejects vector<atomic<T>> in EventBaseTest.cpp.

Test Plan: Build and run `folly/io/async/test/EventBaseTest.cpp` using the current `clang`; try building with `clang:dev` and verifying that at least the related failures are gone.

Reviewed By: meyering@fb.com

Subscribers: mathieubaudet, folly-diffs@, yfeldblum, dougw, brettp

FB internal diff: D1848749

Tasks: 6244720

Signature: t1:1848749:1423866420:7b7354d4568b6a6d6a824236ae8d271f6855d90b

9 years agofolly: move side effects out of EXPECT_EQ args
Jim Meyering [Fri, 13 Feb 2015 19:06:37 +0000 (11:06 -0800)]
folly: move side effects out of EXPECT_EQ args

Summary:
This avoids warnings about e.g., sizeof(k++) that result
from macro expansion.
* folly/futures/test/ExecutorTest.cpp: Move increment out
of macro argument list.
* folly/test/FBVectorTestBenchmarks.cpp.h: Likewise.
* folly/test/LazyTest.cpp: Likewise.
Otherwise, we'd get errors like this:

folly/test/LazyTest.cpp:49:118: error: expression with side effects has no effect in an unevaluated context [-Werror,-Wunevaluated-expression]
switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal:: EqHelper<(sizeof(::testing::internal::IsNullLiteralHelper(++globalCount())) == 1)>::Compare("++globalCount()", "1", ++globalCount(), 1))) ; else ::testing::internal::AssertHelper(::testing::TestPartResult::kNonFatalFailure, "folly/test/LazyTest.cpp", 49, gtest_ar.failure_message()) = ::testing::Message();
^

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: njormrod@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1848324

Tasks: 6244745

Signature: t1:1848324:1423860890:bce44c5e0895804a21957893ae6b78e76dfbc4d3

9 years agofolly: avoid new warnings from -Winconsistent-missing-override
Jim Meyering [Fri, 13 Feb 2015 18:36:27 +0000 (10:36 -0800)]
folly: avoid new warnings from -Winconsistent-missing-override

Summary:
Upgrading to clang:dev (clang >3.6) brought in some new warnings.
This change address all of the issues exposed by the new
-Winconsistent-missing-override, usually by simply adding the
missing "override" keyword.  However, in
folly/wangle/channel/test/MockChannelHandler.h, I chose
to ignore those warnings for the mocked functions.
* folly/futures/test/ViaTest.cpp: Add missing "override"(s).
* folly/io/async/AsyncSSLSocket.h: Likewise.
* folly/io/async/AsyncSocket.h: Likewise.
* folly/io/async/EventBase.h: Likewise.
* folly/test/ExceptionWrapperTest.cpp: Likewise.
* folly/wangle/channel/AsyncSocketHandler.h: Likewise.
* folly/wangle/channel/test/MockChannelHandler.h: Ignore
the new warning for these functions.

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: hans@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1848330

Tasks: 6244745

Signature: t1:1848330:1423858909:b167ca220d6c9fe036d3adca08cf3053a7a9de16

9 years ago(Wangle) Have Core own an FSM instead of inheriting
Hannes Roth [Fri, 13 Feb 2015 21:14:22 +0000 (13:14 -0800)]
(Wangle) Have Core own an FSM instead of inheriting

Summary:
This seems cleaner. I also moved a Core member around and changed state to be a smaller size to save us 16 bytes. Might be neat to try to get this down to 128?

Fixed compilation with GCC 4.9, too.

Test Plan: Ran all the tests. Will also benchmark.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1843129

Signature: t1:1843129:1423789502:60e56d1af553994991195fcc350d37d644a556ca

9 years agoBe able to access proxy client IP (including Lua)
Jonathan Frank [Fri, 13 Feb 2015 17:59:56 +0000 (09:59 -0800)]
Be able to access proxy client IP (including Lua)

Summary:
Pass TransportInfo object through connection setup.  This allows us to
pass along additional info from the setup if necessary.

Test Plan:
Run proxygen on my devserver listening on ipv4 and ipv6 for ports 80 and 443.
Configure it to run a lua request rule with the following lines:

io.stderr:write("jonlog: proxy " .. conn:getClientAddrOriginal() .. "\n")
io.stderr:write("jonlog: client " .. request:getClientIPStr() .. "\n")

Run haproxy also on my devserver, listening on ports 8555-8558, with each
forwarding to one of the four pairs (ipv4-localhost/ipv6-localhost, 80/443)

From my laptop, run curls against each of these four endpoints and make
sure that we print out "127.0.0.1" as the proxy IP, and my laptop's IP
as the client IP.

Reviewed By: cgheorghe@fb.com

Subscribers: jsedgwick, yfeldblum, trunkagent, ruibalp, bmatheny, folly-diffs@, dsp, nbm

FB internal diff: D1746590

Tasks: 5688127

Signature: t1:1746590:1423695880:f647964d95636a69a00304e144aef71ee0213d28

9 years agoFix EliasFanoReader position() when past-the-end
Giuseppe Ottaviano [Fri, 13 Feb 2015 01:46:14 +0000 (17:46 -0800)]
Fix EliasFanoReader position() when past-the-end

Summary:
`EliasFanoReader::position()` used to return `size() - 1` both when
the reader is positioned on the last element, and after `next()` is
called after that (and it return `false`). Now in the latter case
`position()` returns `size()` (consistently with the usual behaviour
of past-the-end iterators).

Also fix the return type of `jumpTo`.

Test Plan:
fbconfig folly/experimental/test:eliasfano_test && fbmake runtests_opt

Reviewed By: philipp@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1846275

Signature: t1:1846275:1423790264:151f5d2e1e09d4e24dfb758473dfb9cd52c070bd

9 years agostd::atomicize MicroSpinLock.
Keith Adams [Thu, 12 Feb 2015 23:52:39 +0000 (15:52 -0800)]
std::atomicize MicroSpinLock.

Summary:
A colleague at another company started making fun of
MicroSpinLock for its x86 assembly and ad hoc compiler memory
barriers. Use C++11 (which wasn't really a thing at the time I wrote
this).

Test Plan: folly's runtests. What else would we like?

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1841563

Signature: t1:1841563:1423780458:a447c081fbd72e3420b23e95dcf26575c9a06798