folly.git
9 years agoSharedMutex potential lost wakeup with exactly 3 or 4 contending writers
Nathan Bronson [Fri, 10 Apr 2015 02:45:58 +0000 (19:45 -0700)]
SharedMutex potential lost wakeup with exactly 3 or 4 contending writers

Summary:
SharedMutex used a saturating counter that records the number of
waiting lock() calls, but an ABA problem on futexWait could lead to a lost
wakeup when there was exactly 3 or 4 threads contending on the RW lock
in W mode.  This diff changes the kWaitingE count to be heuristic (it is
possible that the count says 1 but there are two waiters), saturates at
2 instead of 3 (because there is no benefit from differentiating those
two), and doesn't decrement the count on a successful wakeup.

Also, I noticed while debugging this that boost::noncopyable was causing
SharedMutex to be 8 bytes when it should only be 4.

One way the wakeup could be lost in the old code:

1. A calls lock()
2. A updates state <- kHasE
3. A returns
4. B calls lock()
5. B spins
6. B updates state <- kHasE + 1 * kIncrWaitingE
7. A calls unlock()
8. A updates state <- 0
9. A calls futexWake(), which returns 0
10. A calls lock()
11. A updates state <- kHasE
12. A returns
13. C calls lock()
14. C spins
15. C updates state <- kHasE + 1 * kIncrWaitingE
16. C calls futexWait, expecting kHasE + 1 * kIncrWaitingE
17. B calls futexWait, expecting kHasE + 1 * kIncrWaitingE
18. A calls unlock()
19. A updates state <- 0
20. A calls futexWake(), which returns 1
21. C receives the wakeup
22. C updates state <- kHasE
23. C returns
24. C calls unlock()
25. C updates state <- 0

B missed the wakeup that was intended for it (sent at step 9, wait
started at step 17), but went to sleep anyway because it saw the write
state at step 17. Now there are two waiters but only 1 recorded in the
SharedMutex, at which point failure is inevitable.

Test Plan:
1. DeterministicSchedule test using uniformSubset that can repro the problem
2. Test in production scenario that produced occasional deadlocks under high stress

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1980210

Tasks: 6720328

Signature: t1:1980210:1428623932:ef1c00c3f88154578b2b253ac0cfdbadf9f31d8c

9 years agoMove StringHandler to its own file
Dave Watson [Thu, 9 Apr 2015 16:59:36 +0000 (09:59 -0700)]
Move StringHandler to its own file

Summary: Move this from a test to its own file in codec folder

Test Plan:
already has a unittest
fbconfig -r folly/wangle; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959152

Signature: t1:1959152:1427935371:9d1338986df97fd94f98ad2bf08c731d229bc2b1

9 years agoLineBasedFrameDecoder
Dave Watson [Thu, 9 Apr 2015 17:00:48 +0000 (10:00 -0700)]
LineBasedFrameDecoder

Summary: Copy of netty's line based decoder.

Test Plan:
unittests
fbconfig folly/wangle/codec; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959155

Signature: t1:1959155:1427935150:e11280c5567df9ad9964dbb656aa090267856f57

9 years agoPipeline OutputBufferingHandler
Dave Watson [Wed, 8 Apr 2015 20:54:59 +0000 (13:54 -0700)]
Pipeline OutputBufferingHandler

Summary: Part 2 of pipeline.  Use OutputBufferingHandler, which is exactly the same output buffering thrift does currently.

Test Plan:
A couple canaries, unittests.
Specific issues of previous pipeline diffs I think are fixed already and should be fine with this one:
* Previously there were missing DestructorGuards.  I don't think anything in OutputBUfferHandler needs a DG
* previously broke http GET handling.  There is a unittest for it now.

Canary

Reviewed By: alandau@fb.com

Subscribers: doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, vloh, andreib

FB internal diff: D1970949

Signature: t1:1970949:1428360621:a0db142032894525f69c2b144cf946d63f790fe1

9 years agofolly: MemoryMapping: add offset/size params to advise
Lucian Grijincu [Wed, 8 Apr 2015 18:14:28 +0000 (11:14 -0700)]
folly: MemoryMapping: add offset/size params to advise

Test Plan: ran it

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1974936

Signature: t1:1974936:1428453023:1a83ed8336f75b745bffc633b8471231423b0fb0

9 years agofolly: MemoryMapping: respect writable when opening the file
Lucian Grijincu [Wed, 8 Apr 2015 18:14:17 +0000 (11:14 -0700)]
folly: MemoryMapping: respect writable when opening the file

Summary: the file needs to be opened with write-permissions before it can be mmaped with PROT_WRITE.

Test Plan: n/a

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1974895

Signature: t1:1974895:1428451664:1106019abf02b9de659ce0d305e3d788ae7d60bc

9 years agoAsyncSocketHandler cacheChainLength
Dave Watson [Wed, 8 Apr 2015 17:45:08 +0000 (10:45 -0700)]
AsyncSocketHandler cacheChainLength

Summary:
Is there any reason we shouldn't just cache the chain length everywhere?

Currently the pipeline does a lot of passing of buffers between IOBufQueues, I wonder if we can get even smarter about passing the length around

Test Plan: used in telnet server diff

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959167

Signature: t1:1959167:1427934568:eb9a49514b2bb7d157879bb8efa99f12092d5e95

9 years agoRemove FiberManager move-constructor
Andrii Grynenko [Tue, 7 Apr 2015 18:19:46 +0000 (11:19 -0700)]
Remove FiberManager move-constructor

Summary:
LoopController and Fibers keep references to FiberManager, so implementing move-constructor for it is non-trivial.
The only purpose of move constructor was to have a create() static method, replaced it with a constructor accepting a type tag.

Test Plan: unit test + tao build

Reviewed By: stepan@fb.com, pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1971881

Signature: t1:1971881:1428374073:945d913e69eaa6f957dace981c23835105d91935

9 years agoAsyncUDPServerSocket passes socket in callback
Dave Watson [Tue, 7 Apr 2015 16:53:44 +0000 (09:53 -0700)]
AsyncUDPServerSocket passes socket in callback

Summary: AsyncUDPServerSocket doesn't make it easy to write to the same socket you read from.  Add the socket as a callback param, similar to AsyncServerSocket

Test Plan:
fbconfig -r folly; fbmake dbg

Will fixup any other spots contbuild finds

Reviewed By: hans@fb.com

Subscribers: bmatheny, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948936

Signature: t1:1948936:1427841651:20d13d73c06d31c75056624f051a6fd35b9701fb

9 years agoAdd ability to yield execution of the currently running fiber
Brian Watling [Tue, 7 Apr 2015 12:34:06 +0000 (05:34 -0700)]
Add ability to yield execution of the currently running fiber

Summary: This diff allows fibers to explicitly yield execution to other fibers and the event loop

Test Plan: unit tests

Reviewed By: andrii@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@

FB internal diff: D1965297

Signature: t1:1965297:1428082686:e524e9dd21b3fb951e1d3556e4cb3eedc3e6511a

9 years agoFix open-source Makefiles after fibers were moved to folly
Andrii Grynenko [Tue, 7 Apr 2015 02:32:01 +0000 (19:32 -0700)]
Fix open-source Makefiles after fibers were moved to folly

Test Plan: contbuild

Reviewed By: pavlo@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1966698

Signature: t1:1966698:1428367351:62f2218c23d4911c15055917c0fbae31a2420903

9 years agoGlobalExecutor.h include Executor.h and IOExecutor.h
Hans Fugal [Tue, 7 Apr 2015 01:20:00 +0000 (18:20 -0700)]
GlobalExecutor.h include Executor.h and IOExecutor.h

Summary: We were forward-declaring `Executor` and `IOExecutor` for great compilation speed. However, this leaves users of this header to include at least one of those headers in their cpp file to actually use it, and that's kind of annoying. `Executor` and `IOExecutor` are not templated and are quite small, so it seems like the compilation speed benefit is minimal compared to the convenience factor, so this diff just includes the two headers. (Probably including `Executor.h` is redundant since `IOExecutor.h` probably includes it, but for completeness and explicitness, and in case `IOExecutor` also just forward-declares `Executor` now or in the future.)

Test Plan: stuff still builds

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1970179

Signature: t1:1970179:1428351076:e9c3700a868688035e7ff50486b8cead751d3c02

9 years agoAdd FiberManager local type
Andrii Grynenko [Mon, 6 Apr 2015 19:06:58 +0000 (12:06 -0700)]
Add FiberManager local type

Summary: This adds local type for each FiberManager. Only local of given type can be created on fibers of this FiberManager. Locals of other types will be just treated as regualar thread-locals.

Test Plan: unit test

Reviewed By: bwatling@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1969739

Signature: t1:1969739:1428345931:aff5deb526c179158e5881b29330ff3f6698149a

9 years agoOption to record precise stack on every N fiber
Pavlo Kushnir [Mon, 6 Apr 2015 02:51:13 +0000 (19:51 -0700)]
Option to record precise stack on every N fiber

Summary: title + some logging around

Test Plan: mcrouter unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1965097

Signature: t1:1965097:1428103637:341ff705c9d11c730197828256ec0b790ee7afaa

9 years agoshared ptr vector sockets
Dave Watson [Fri, 3 Apr 2015 20:07:13 +0000 (13:07 -0700)]
shared ptr vector sockets

Summary: promote the sockets vector to a shared_ptr, since both ServerWorkerPool and ServerBootstrap use it.  Otherwise there are destruction order issues between ServerBootstrap and any IOThreadPoolExecutor you use

Test Plan: Saw use after free in D1942242, gone after this.

Reviewed By: yfeldblum@fb.com

Subscribers: chalfant, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1947553

Signature: t1:1947553:1427484417:5b78f5c9c70d244d3f52a6f71b6d1fab7b29d106

9 years agoUse Fiber locals for TAO Fiber-locals
Andrii Grynenko [Fri, 3 Apr 2015 00:29:35 +0000 (17:29 -0700)]
Use Fiber locals for TAO Fiber-locals

Summary:
This simplifies TAO fibers locals, using FiberManager based fiber-local storage.
This depends on D1958135.

Test Plan: integration tests

Reviewed By: bwatling@fb.com

Subscribers: alikhtarov

FB internal diff: D1962214

Signature: t1:1962214:1427997755:b546b9039382a7600f234b0a2a60cc96da34e662

9 years agoFiber-local context
Andrii Grynenko [Fri, 3 Apr 2015 00:17:46 +0000 (17:17 -0700)]
Fiber-local context

Summary: This adds fiber-local context, which behaves more like static objects+fork rather than thread-locals.

Test Plan: unit test

Reviewed By: pavlo@fb.com, bwatling@fb.com

Subscribers: rushix, alikhtarov, bwatling

FB internal diff: D1958135

Signature: t1:1958135:1427999426:8e4b89f4af53a1a119b2e5a765fb549dd8442c50

9 years agoAdd static method to skip SSL init
Joel Goguen [Thu, 2 Apr 2015 23:37:50 +0000 (16:37 -0700)]
Add static method to skip SSL init

Summary:
Some things like gSOAP, initialize SSL only to have SSLContext stomp over their
work. Add a method to allow flagging SSLContext as "initialized" to use the
existing global state.

Test Plan:
Tested running mcrouter and paymentnegined to verify existing
functionality appears normal.

Reviewed By: subodh@fb.com

Subscribers: ssl-diffs@, folly-diffs@, yfeldblum, chalfant, dihde

FB internal diff: D1959936

Tasks: 6655251

Signature: t1:1959936:1427994292:8b2be1742ef17a5ec2c7b3ad37787d2cf4d142a6

9 years agoBump version to 34:0 v0.34.0
afrind [Thu, 2 Apr 2015 20:49:51 +0000 (13:49 -0700)]
Bump version to 34:0

9 years agoUpdate Makefile.am for AsyncSignalHandler
Alan Frindell [Thu, 2 Apr 2015 20:36:51 +0000 (13:36 -0700)]
Update Makefile.am for AsyncSignalHandler

Summary: facepalm

Test Plan: Built proxygen OSS that depends on this symbol

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1963679

Signature: t1:1963679:1428006035:2c4ceffc046b995431cde2fbcef757fa7ea1ea0b

9 years agoMove mcrouter/lib/fibers to folly/experimental/fibers
Rushi Desai [Thu, 2 Apr 2015 20:09:03 +0000 (13:09 -0700)]
Move mcrouter/lib/fibers to folly/experimental/fibers

Summary:
The mcrouter fibers library is quite general purpose and reusable by other projects. Hence I'm moving it into folly.

Test Plan:
fbconfig -r folly/experimental/fibers && fbmake runtests
fbconfig -r mcrouter && fbmake runtests
fbconfig -r tao && fbmake
fbconfig -r ti && fbmake

Reviewed By: andrii@fb.com

Subscribers: vikas, zhuohuang, jmkaldor, jhunt, pavlo, int, aap, trunkagent, fredemmott, alikhtarov, folly-diffs@, jsedgwick, yfeldblum, chalfant, chip

FB internal diff: D1958061

Signature: t1:1958061:1428005194:b57bfecfe9678e81c48526f57e6197270e2b5a27

9 years agoBump version to 33:0 v0.33.0
afrind [Thu, 2 Apr 2015 19:03:15 +0000 (12:03 -0700)]
Bump version to 33:0

9 years agoUDP bind via port or address
Dave Watson [Thu, 2 Apr 2015 18:48:01 +0000 (11:48 -0700)]
UDP bind via port or address

Summary: Bind via port or address, similar to tcp

Test Plan: I wrote a unittest that tested v4 vs v6 binding - can't check it in because we have v6 only clusters though.  Not sure how eles to write a unittest without picking specific ports

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948938

Signature: t1:1948938:1427841721:621031320981bf366a8252a0ea1d4491b694ea24

9 years agoMove TAsyncSignalHandler into folly
Alan Frindell [Thu, 2 Apr 2015 17:43:38 +0000 (10:43 -0700)]
Move TAsyncSignalHandler into folly

Summary:
TODO item, trying to remove unecessary dependencies on thrift

Test Plan: Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, fbcode-common-diffs@, davejwatson, andrewcox, alandau, bmatheny, anca, darshan, mshneer, folly-diffs@, bil, yfeldblum, haijunz, chalfant

FB internal diff: D1960215

Signature: t1:1960215:1427920934:8abd7e94c50676b05bf7ff79800df0db1bd04266

9 years agoAdd socket mocks
Alan Frindell [Thu, 2 Apr 2015 17:26:53 +0000 (10:26 -0700)]
Add socket mocks

Summary: These parallel the MockTAsync* from thrift/lib/cpp/test.  Once all the users of the thrift versions are gone they can be deleted

Test Plan: Unit tests

Reviewed By: yfeldblum@fb.com, alandau@fb.com

Subscribers: doug, net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1959979

Signature: t1:1959979:1427917909:19af219f88dd6847a064da986dd30765e29bdc99

9 years agoMove AsyncSocket tests from thrift to folly
Alan Frindell [Thu, 2 Apr 2015 17:20:49 +0000 (10:20 -0700)]
Move AsyncSocket tests from thrift to folly

Summary: These tests belong with the code that they test.  The old tests had a couple dependencies on TSocket/TSSLSocket, so I wrote a BlockingSocket wrapper for AsyncSocket/AsyncSSLSocket

Test Plan: Ran the tests

Reviewed By: alandau@fb.com

Subscribers: doug, net-systems@, alandau, bmatheny, mshneer, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1959955

Signature: t1:1959955:1427917833:73d334846cf248f8bb215f3eb5b596df7f7cee4f

9 years agoMove AtomicLinkedList to folly
Rushi Desai [Thu, 2 Apr 2015 04:51:08 +0000 (21:51 -0700)]
Move AtomicLinkedList to folly

Summary:
Adding new atomic data structure from the mcrouter project.

Test Plan: fbconfig -r mcrouter && fbmake runtests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant, achao, jpearce

FB internal diff: D1960219

Signature: t1:1960219:1427921204:78e5313ea916b8a249d32e31cc7ba1aa9e890d95

9 years ago(Wangle) Swap order of Try<T> and T matching
Hannes Roth [Thu, 2 Apr 2015 04:31:29 +0000 (21:31 -0700)]
(Wangle) Swap order of Try<T> and T matching

Summary:
See the test case and D1958860.

I don't really know why I chose one over the other. This fixes a bug. So
it's better?

Test Plan: Run all the tests?

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, over

FB internal diff: D1960740

Signature: t1:1960740:1427927644:25093b049a1519d7c869ee7043f3caced4cc971e

9 years agoAdd support for writev for AsyncUDPSocket
Naizhi Li [Thu, 2 Apr 2015 00:52:47 +0000 (17:52 -0700)]
Add support for writev for AsyncUDPSocket

Summary:
I have 2 local buffers and instead of allocating iobufs
on the heap, I can simply use iovec on the stack to send.

Test Plan: Unit tests and turn server.

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, ehrhardt, yfeldblum, chalfant, paramr

FB internal diff: D1956201

Signature: t1:1956201:1427918649:09f00399fd9e8ed3fe62be43b19adbf0a5a3f3b0

9 years agoMinor fixes from documentation
Tom Jackson [Wed, 1 Apr 2015 22:22:31 +0000 (15:22 -0700)]
Minor fixes from documentation

Test Plan: unittests

Reviewed By: jbrewer@fb.com, ashishsharma@fb.com

Subscribers: maxime, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1956101

Signature: t1:1956101:1427828321:2d828e6f2c06c6dcd6e7baa3ed5e72ee59d9b77f

9 years agoimprove split() docs
Mark McDuff [Wed, 1 Apr 2015 22:17:20 +0000 (15:17 -0700)]
improve split() docs

Summary:
From reading the docs, I assumed that casting exceptions would be caught.  Let's clarify this.

Test Plan: readingiskey

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1960186

Signature: t1:1960186:1427920471:b7aa2af38a90a986418b98294fc0f69e678f0d1e

9 years ago(Wangle) Allocate lambda space inside Core instead of inside std::function
Hannes Roth [Wed, 1 Apr 2015 21:35:55 +0000 (14:35 -0700)]
(Wangle) Allocate lambda space inside Core instead of inside std::function

Summary:
Taking this trick that is used in the fibers library. We can keep 64
bytes of space inside `Core` to allocate the callback lambda into,
instead of having `std::function` do another `malloc`. This seems to
greatly improve the synthethic benchmark, and hopefully production
workloads, too, by reducing the number of mallocs.

64 bytes were picked because it fits all the lambdas in the futures
tests. We might want to adjust this based on production data...?

https://fb.intern.facebook.com/groups/715931878455430/permalink/837898842925399/

Test Plan: Run all the tests for all platforms, compilers, and Windtunnel.

Reviewed By: hans@fb.com

Subscribers: chalfant, meisner, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1931620

Signature: t1:1931620:1427841595:6ec667b58980be232dfb116bc316148bb67de4fc

9 years agostop in same thread
Dave Watson [Wed, 1 Apr 2015 16:28:06 +0000 (09:28 -0700)]
stop in same thread

Summary: In a couple places in ServerBootstrap there is code that needs to run in the EB, but if we are already in the EB, it's fine to run it inline.  Maybe this should be a method on EB directly?  There is runInEventBaseThreadAndWait(), but it explicitly disallows this usage.

Test Plan: used in D1942242, deadlocks without this, since stop() was called in the same eb

Reviewed By: yfeldblum@fb.com

Subscribers: chalfant, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1947581

Signature: t1:1947581:1427831021:d1d08ff9a4a00501d6be60670709fcb17af04134

9 years agoMove OpenSSL uninitialization to a separate function
Joel Goguen [Wed, 1 Apr 2015 15:11:44 +0000 (08:11 -0700)]
Move OpenSSL uninitialization to a separate function

Summary:
The current behaviour of SSLContext is subject to a bug where an instance going
out of scope at the wrong time can decrement the global refcount to 0, causing
the destructor to destroy the SSL context. Instead, this requires callers to
explicitly uninitialize the SSLContext instance and assume the risk of wrongly
destroying the global SSL context.

Test Plan:
Tested with mcrouter

Reviewed By: subodh@fb.com

Subscribers: afrind, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, dihde, melitam

FB internal diff: D1949649

Tasks: 6358211

Signature: t1:1949649:1427854689:aea2dc801f63256ff64188b0f7a15121dcecee69

9 years agoGet access to EventBaseManager object
Yuri Putivsky [Tue, 31 Mar 2015 06:39:58 +0000 (23:39 -0700)]
Get access to EventBaseManager object

Summary: as title

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1955074

9 years agoMPMCQueue - FPE to invalid_argument for 0 capacity
Daniel Pittman [Tue, 31 Mar 2015 20:33:48 +0000 (13:33 -0700)]
MPMCQueue - FPE to invalid_argument for 0 capacity

Summary:
If an MPMCQueue was initialized with an explicit zero capacity, it would
try to compute stride for zero and trigger SIGFPE which pretty fatally
killed the process -- but in a way that required gdb to figure out the
root cause.

This replaces that with a std::invalid_argument exception, which
includes a description of the problem and should make fixing this user
error much faster.

Test Plan:
GTEST unit test added to verify behaviour; if this didn't work it would
SIGFPE and abort the test script, which is probably a good indicator
that something went wrong. :)

fbconfig -r folly && fbmake runtests

Reviewed By: ngbronson@fb.com

Subscribers: chalfant, folly-diffs@, yfeldblum, rkomorn

FB internal diff: D1930978

Signature: t1:1930978:1427740315:cc06a8b9f3c314b956ae41f813b2f904d3e979c9

9 years agoDefault accept function in Acceptor
Dave Watson [Tue, 31 Mar 2015 16:53:51 +0000 (09:53 -0700)]
Default accept function in Acceptor

Summary: Since Acceptor can be used for TCP or UDP, make an empty default TCP accept function so subclasses don't have to.

Test Plan:
built D1942242, no build errors

contbuild

Reviewed By: yfeldblum@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948940

Signature: t1:1948940:1427498977:26b7bb23e3367ff6695dad86ebf0e60637815a7b

9 years agoEventBase::runImediatelyOrRunInEventBaseThreadAndWait.
Yedidya Feldblum [Tue, 31 Mar 2015 01:17:31 +0000 (18:17 -0700)]
EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

Summary:
[Folly] EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

When you have code that you need to run in the event loop and wait for it to finish, and you yourself might be in the event loop but might be in another thread instead.

Test Plan:
Unit tests:
* `folly/io/async/test/EventBaseTest.cpp`.

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, agallagher, njormrod, ldbrandy, brettp, dougw

FB internal diff: D1953110

Signature: t1:1953110:1427751076:ad2c9f4d5b65ef952cba94b06be887672ee44583

9 years agoFixing predicate inlining
Yuri Putivsky [Thu, 26 Mar 2015 19:04:03 +0000 (12:04 -0700)]
Fixing predicate inlining

Summary:
as title.

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, ridge@

FB internal diff: D1946589

9 years agoExtend IO pool with external optional event_base_manager
Yuri Putivsky [Sat, 28 Mar 2015 01:20:44 +0000 (18:20 -0700)]
Extend IO pool with external optional event_base_manager

Summary: as title

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1950610

9 years agoasyncudpserversocket reuse port
Dave Watson [Mon, 30 Mar 2015 17:17:12 +0000 (10:17 -0700)]
asyncudpserversocket reuse port

Summary:
AsyncUDPSocket supports reuse port, also make it available in AsyncUDPServerSocket.
Has to happen before bind

Test Plan:
?? it builds?

Mutliple binds already work with reuseaddr, but load balancing doesn't work as well.  See http://lwn.net/Articles/542629/ for details

Reviewed By: naizhi@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948932

Signature: t1:1948932:1427473013:8c114458a0746bb2b6957db33c1834179d860c00

9 years agoFuture::filter(A => bool)
Hans Fugal [Mon, 30 Mar 2015 15:44:48 +0000 (08:44 -0700)]
Future::filter(A => bool)

Test Plan: new unit tests

Reviewed By: davejwatson@fb.com

Subscribers: chalfant, bmatheny, trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1828251

Tasks: 6166893

Signature: t1:1828251:1427475565:512463d5728482a40c1da2548a5bed91e7f92d60

9 years agoAdd CaptureFD for log testing (and some glog patterns)
Alexey Spiridonov [Thu, 26 Mar 2015 23:26:09 +0000 (16:26 -0700)]
Add CaptureFD for log testing (and some glog patterns)

Summary:
Without a gadget like this, it's really hard to test logging output. With it, it's really easy. I found about 50 callsites to the various functions across several projects, so folly seems appropriate.

PS The patterns are functions for two reasons:
- Static variables are a pain.
- This leaves the option of adding an optional argument, so you can grep for a particular kind of error string.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933439

Signature: t1:1933439:1427345479:5b3d1c6566a026fdbccb16b382211688e327ea1a

9 years agoAdd EXPECT_{NO_,}_PCRE_MATCH macros
Alexey Spiridonov [Thu, 26 Mar 2015 23:16:15 +0000 (16:16 -0700)]
Add EXPECT_{NO_,}_PCRE_MATCH macros

Summary: There are about 40 callsites to these across a couple of projects, and they seem generally useful.

Test Plan: ```fbconfig folly/experimental/test && fbmake runtests```

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933415

Signature: t1:1933415:1427345307:6587eea3dac74c5d841950ace3c3501d6e3dbe4a

9 years agoTemporaryFile and TemporaryDirectory
Adam Simpkins [Thu, 26 Mar 2015 19:40:56 +0000 (12:40 -0700)]
TemporaryFile and TemporaryDirectory

Summary:
TemporaryFile, TemporaryDirectory, and ChangeToTempDir should all be moveable
objects, but not copiable.  Define default move constructors and move
assignment operators for these classes.  This will prevent copy constructor and
copy assignment operators from being implicitly defined.

Test Plan:
Used this in a new test to write a helper function which created and returned
a new TemporaryFile object using the move constructor.

Reviewed By: yfeldblum@fb.com

Subscribers: doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1945134

Signature: t1:1945134:1427342944:3428327e797ce4b3d362f9a2d2276de6d8b96137

9 years ago(Wangle) Reduce
Hannes Roth [Wed, 25 Mar 2015 22:39:32 +0000 (15:39 -0700)]
(Wangle) Reduce

Summary:
1] The lambda should ble able to return a `Try<T>`. Maybe? Can a `then`
return a `Try<T>` actually? Can fix this with `resultOf`. (Doubling the number of functions to 4.)

2] `initial` and `func` have to be copyable.

Test Plan: Added tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1870996

Tasks: 6025252

Signature: t1:1870996:1427318511:2ae5894b79022da88990835b26d35c4520fdbd29

9 years agoBump version to 32:0 v0.32.0
Noam Lerner [Wed, 25 Mar 2015 22:36:48 +0000 (15:36 -0700)]
Bump version to 32:0

9 years agoAdd a pretty macro for deprecation
Ben Maurer [Wed, 25 Mar 2015 21:33:30 +0000 (14:33 -0700)]
Add a pretty macro for deprecation

Summary: Make it easy for people to mark code as deprecated

Test Plan: Chose a random file and added the FOLLY_DEPRECATED macro to it

Reviewed By: yfeldblum@fb.com

Subscribers: ps, folly-diffs@, yfeldblum

FB internal diff: D1943248

Signature: t1:1943248:1427316689:b3305d134f374697e93d36bafbf0961eec1ef68a

9 years agoRelax CHECK condition in stringAppendfImpl
Sarang Masti [Sat, 14 Mar 2015 17:24:15 +0000 (10:24 -0700)]
Relax CHECK condition in stringAppendfImpl

Summary:
vsnprintf can return fewer bytes and bytes_used if
we print a string containing '\0' using a width
specifier.

Test Plan: -- ran all tests

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1915035

Signature: t1:1915035:1426799341:4aaea928c4bdde1998bf66cf9e2732a53572c6e3

9 years agohttpserver on serverbootstrap (2)
Dave Watson [Tue, 24 Mar 2015 16:40:29 +0000 (09:40 -0700)]
httpserver on serverbootstrap (2)

Summary:
Original diff D1800100, reverted in D1895181.  Issue seems to have been caused by a copy constructor issue.

Fixes: Remove copy constructor, move contructor only. Separate stop() and join(), since join() can happen only after all stop()s have completed when reusing the same IOThreadPoolExecutors.
Test Plan: build and unit tests of relevant product

Reviewed By: praveenr@fb.com

Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1901718

Tasks: 6431975

Signature: t1:1901718:1426885577:46b63bfe117df207be3d273b953c100249c1a7da

Blame Revision: D1895181

9 years agoAdd ChangeToTempDir to TestUtil
Alexey Spiridonov [Sat, 21 Mar 2015 04:58:14 +0000 (21:58 -0700)]
Add ChangeToTempDir to TestUtil

Summary: It's convenient to be able to write tempfiles in the current directory. This provides a one-liner way of doing that in tests.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: simpkins, folly-diffs@, yfeldblum

FB internal diff: D1931918

Signature: t1:1931918:1426913634:f988fb3c5061f5909e309dcaf42742d9b2ed18c6

9 years agoMake folly::Singleton DFATAL as late as possible
Andrii Grynenko [Fri, 20 Mar 2015 19:17:17 +0000 (12:17 -0700)]
Make folly::Singleton DFATAL as late as possible

Summary: We want to print all destructor stack-traces if Singleton destruction happens eventually (even after destroyInstances() was done).

Test Plan:
unit test

Reviewed By: chip@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1929881

Signature: t1:1929881:1426878091:5993d544c0f84bdbf8bcdd420ef01fb7bf88ac3f

9 years agoUdp Acceptor
Dave Watson [Fri, 20 Mar 2015 16:54:08 +0000 (09:54 -0700)]
Udp Acceptor

Summary:
major changes:

1) ServerSocketFactory and AsyncSocketBase to abstract the differences between UDP and TCP async socket.  Could possibly push some of this to the sockets themselves eventually
2) pipeline() is a pipeline between accept/receive of a UDP message, and before sending it to workers.  Default impl for TCP is to fan out to worker threads.  This is the same as Netty.  Since we don't know if the data is a TCP socket or a UDP message, it's a void*, which sucks (netty uses Object msg, so it isn't any different).

Test Plan: Added lots of new tests.  Doesn't test any data passing yet though, just connects/simple receipt of UDP message.

Reviewed By: hans@fb.com

Subscribers: alandau, bmatheny, mshneer, jsedgwick, yfeldblum, trunkagent, doug, fugalh, folly-diffs@

FB internal diff: D1736670

Tasks: 5788116

Signature: t1:1736670:1424372992:e109450604ed905004bd40dfbb508b5808332c15

9 years agofix service memory leak
Dave Watson [Thu, 19 Mar 2015 19:57:13 +0000 (12:57 -0700)]
fix service memory leak

Summary:
This code is only used in a test so far.

Basically it looks like Pipeline has some dependency issues - any addBack() pipelines have to be deleted *after* the main pipeline is destroyed.  Ideally if the pipeline is already closed, the destruction order wouldn't matter.  There is currently no removeBack() call either.   This is a quick fix for the test to just delete stuff in the right order, we can discuss a better solution when @jsedgwick returns

Test Plan: fbconfig --sanitize=address --with-project-version=clang:3.5 --clang folly/wangle/service && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, fugalh, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1903200

Tasks: 6415578

Signature: t1:1903200:1426528298:e109dcc0ec586a505a26cd95a6f20434d22cbd37

9 years agoAdd comment about djb2 hash
Alexey Spiridonov [Thu, 19 Mar 2015 00:12:09 +0000 (17:12 -0700)]
Add comment about djb2 hash

Summary:
In my tests, using djb2 was much worse for any sizable workload. Just use SpookyHashV2.

Test Plan: comment only

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1906439

Signature: t1:1906439:1426117575:f88b7b28b578092c1716433e21150d423d4e94a7

9 years agoSocketAddress::anyAddress should be a method
Sean Cannella [Wed, 18 Mar 2015 22:57:53 +0000 (15:57 -0700)]
SocketAddress::anyAddress should be a method

Summary:
Calling getaddrinfo from a static constructor is not the most
reliable thing to do. Stop doing it.

Test Plan: existing tests

Reviewed By: davejwatson@fb.com

Subscribers: dancol, mzlee, shikong, kmdent, fma, bmatheny, benyluo, ranjeeth, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1924578

Tasks: 6534662

Signature: t1:1924578:1426718586:d73bc2f001095e66fa77fde9c027af050cc26d2a

9 years ago(Wangle) Clean up move constructors
Hannes Roth [Wed, 18 Mar 2015 17:29:19 +0000 (10:29 -0700)]
(Wangle) Clean up move constructors

Summary: I thought this might fix #6120972. But it doesn't. Still a bit of a cleanup in my opinion.

Test Plan: Run all the tests?

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907259

Signature: t1:1907259:1426613567:9e33fe7e9e8a36ba006d4aee604086a56f128893

9 years agoSupport 'min ms to log'
Andrey Goder [Wed, 18 Mar 2015 16:34:09 +0000 (09:34 -0700)]
Support 'min ms to log'

Summary:
Sometimes we only want to log if enough time has passed, e.g. when logging
inside of a loop doing work to detect if things are slow. Add an option for
that.

Test Plan:
build and unit tests of folly/experimental

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1909055

Signature: t1:1909055:1426627731:6d3ad94fd71967ec89ffebc90cd2907dd1631d36

9 years agoFix logFormat
Alexey Spiridonov [Wed, 18 Mar 2015 06:27:55 +0000 (23:27 -0700)]
Fix logFormat

Summary: Ooops during port ;)

Test Plan: unit test

Reviewed By: agoder@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905639

Signature: t1:1905639:1426171227:a6e415046a85f098ac090f069476204fb0367c42

9 years agoBump version to 31:0 v0.31.0
Andre Azevedo [Wed, 18 Mar 2015 03:09:26 +0000 (20:09 -0700)]
Bump version to 31:0

9 years agoFixed README download link to get the correct GTest version
Nicholas Ormrod [Wed, 11 Mar 2015 18:11:12 +0000 (11:11 -0700)]
Fixed README download link to get the correct GTest version

Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan: Clicked link, gtest started downloading

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1901849

Tasks: 6255850

Signature: t1:1901849:1426097071:e887ed8749c70d5e15877082ac0175ce3b56647e

9 years agoBackoffTimeout disallow copy, move, default ctor
Praveen Kumar [Mon, 9 Mar 2015 15:32:53 +0000 (21:02 +0530)]
BackoffTimeout disallow copy, move, default ctor

Summary:
BackoffTimeout disallow copy construction, copy assignment, move
construction, move assignment, and default construction.

Closes https://github.com/facebook/folly/pull/146

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912521

Signature: t1:1912521:1426268413:6edb2d8203d1efd1e825816f03be115ac46e55ca

9 years agoWe might avoid some temporaries
Praveen Kumar [Fri, 13 Mar 2015 04:55:34 +0000 (10:25 +0530)]
We might avoid some temporaries

Summary:
It seems we might avoid temporaries.  To do so we pass arguments
directly to constructors with the help of emplace_back member.

Test Plan:
all folly/tests, make check for 37 tests, passed.

Closes https://github.com/facebook/folly/pull/150

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912505

Signature: t1:1912505:1426268158:086882bb53f3d79c3f3b7b2c362318f09b5ee3be

9 years agoAsyncServerSocket disallow copy, move, and default ctor
Praveen Kumar [Fri, 13 Mar 2015 03:41:03 +0000 (09:11 +0530)]
AsyncServerSocket disallow copy, move, and default ctor

Summary:
AsyncServerSocket disallow copy construction, copy assignment,
move construction, move assignment, and default
construction.

Closes https://github.com/facebook/folly/pull/148

Test Plan: all folly/tests, make check for 37 tests, passed.

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912488

Signature: t1:1912488:1426267982:a9690ce9429016f89630f0ded64b7be9ff80c39d

9 years agofix typo: add backslash to Makefile.am
Alecs King [Tue, 17 Mar 2015 20:38:53 +0000 (13:38 -0700)]
fix typo: add backslash to Makefile.am

Summary: backslash

Test Plan: tests

Reviewed By: aap@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1920872

Signature: t1:1920872:1426624366:c984fecb5bd0c292d5a8309fb320a48872fff6db

Blame Revision: D1890511

9 years agoAdd StringKeyedUnorderedMap to Makefile
Alecs King [Tue, 17 Mar 2015 18:57:37 +0000 (11:57 -0700)]
Add StringKeyedUnorderedMap to Makefile

Test Plan:
unit tests

Reviewed By: bwatling@fb.com

Subscribers: stepan, folly-diffs@, yfeldblum, trunkagent, vikas, alikhtarov

FB internal diff: D1890511

Tasks: 6311387

Signature: t1:1890511:1425924980:71af7c12a09a86b280dc3c9a9ef0756bf56bf639

9 years ago(folly osx) test for the zip, not the dir
Hans Fugal [Mon, 16 Mar 2015 18:02:03 +0000 (11:02 -0700)]
(folly osx) test for the zip, not the dir

Summary: Apparently `configure` creates this subdirectory and then the curl and unzip commands don't run, and then `make check` fails. Instead, do it if the zip file doesn't exist.

Test Plan:
git clone https://github.com/facebook/folly.git
cd folly/folly
/path/to/this/bootstrap-osx-homebrew.sh
make
make check

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, davejwatson

FB internal diff: D1882472

Signature: t1:1882472:1425920470:bdab25967d3b8e52c690d5627168077c65829b92

9 years agoexceptionStr(exception_wrapper)
James Sedgwick [Mon, 16 Mar 2015 16:02:24 +0000 (09:02 -0700)]
exceptionStr(exception_wrapper)

Summary: for consistency with exception and exception_ptr functions provided in String.h

Test Plan: unit

Reviewed By: vloh@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905087

Tasks: 5961362

Signature: t1:1905087:1426101740:670ceab5140250bbecdd1247025d3d70b7774690

9 years agoextend future api with Future<T>::poll
Jeroen Vaelen [Fri, 13 Mar 2015 21:10:55 +0000 (14:10 -0700)]
extend future api with Future<T>::poll

Summary:
Added folly::Future<T>::poll() function with the following semantics:
poll() checks if the promise has been fulfilled, if so it will extract the value of the resulting Try<T> to construct a new Try<T> that is wrapped in an Optional<Try<T>>. If the promise has not been fulfilled an empty Optional is returned.

Test Plan: Added two tests to SugarTest.cpp: one where the promise has been fulfilled and one where it hasn't.

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, arturg, yfeldblum

FB internal diff: D1909480

Tasks: 6166875

Signature: t1:1909480:1426280247:6b239c0f1a15c25ce7664afb4bfb3899993c5775

9 years agoAdd ctor to CPUThreadPoolExecutor to enable custom queue sizes
Matt Dordal [Fri, 13 Mar 2015 17:44:08 +0000 (10:44 -0700)]
Add ctor to CPUThreadPoolExecutor to enable custom queue sizes

Summary:
Seems useful to do without passing in a totally custom queue. Makes
the client code a bit simpler.

Test Plan: used it, saw low memory usage and no crash

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, iaroslav, fugalh, folly-diffs@, jsedgwick, yfeldblum, slarsen

FB internal diff: D1907874

Signature: t1:1907874:1426268010:3159ae339b51f5dbc7fe034644bbc968b92b072c

9 years agoAdd support for iovec for UDP send in folly::AsyncUDPSocket
Naizhi Li [Fri, 13 Mar 2015 17:32:19 +0000 (10:32 -0700)]
Add support for iovec for UDP send in folly::AsyncUDPSocket

Summary:
I will be using multi-buf UDP send soon, so adding the
support in folly::AsyncUDPSocket

Test Plan: Unit tests and turn server

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1907189

Signature: t1:1907189:1426266951:046198e0a009fef085ac7eb44f054c67dfb16ba3

9 years agoadd Cursor::isAtEnd()
Adam Simpkins [Thu, 12 Mar 2015 23:00:31 +0000 (16:00 -0700)]
add Cursor::isAtEnd()

Summary:
Add a helper method to efficiently check if the cursor is at the end of the
IOBuf chain.  This is equivalent to (cursor.totalLength() > 0), but it doesn't
need to walk the entire chain just to tell if it is at the end or not.

Test Plan: Updated the unit tests to contain some checks for isAtEnd().

Reviewed By: jasmeetbagga@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1875345

Signature: t1:1875345:1425006512:49ac246fd0ac7937fdcd6cf1359a841f048c444e

9 years agoProtect unprotected
Stepan Palamarchuk [Wed, 11 Mar 2015 22:43:42 +0000 (15:43 -0700)]
Protect unprotected

Summary: This used to be protected before.

Test Plan: folly/futures tests

Reviewed By: andrii@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907063

Signature: t1:1907063:1426116657:d7430304d5f72a048396e83f8dc6b5a92198fb29

9 years agoCPUThreadPoolExecutor's default queue is quite big
Matt Dordal [Wed, 11 Mar 2015 20:52:39 +0000 (13:52 -0700)]
CPUThreadPoolExecutor's default queue is quite big

Summary:
By default, CPUThreadPoolExecutor uses 100 MB of rss for the CPU queue. This is
substantially higher than expected.

Reduce the default queue size so that it uses much less memory upfront, down
to about 6 MB going by the heap profile.

Reviewed By: davejwatson@fb.com

Subscribers: mwa, ott, iaroslav, jrsherwood, aflock, slarsen, davejwatson, trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1900539

Tasks: 6453427

Signature: t1:1900539:1426103805:34fb7b081058fbde89370b5d9ec24ea572c4b0e2

9 years agofix Makefile.am formatting
Nathan Bronson [Wed, 11 Mar 2015 20:41:31 +0000 (13:41 -0700)]
fix Makefile.am formatting

Summary: D1798929 broke the automake folly build

Test Plan:
1. careful visual inspection
2. build

Reviewed By: alikhtarov@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1906281

Signature: t1:1906281:1426106340:621f1e7d2fb18ae2ec5dc604dea0d8ad4353486e

9 years agoSharedMutex - a small fast scalable reader-writer lock
Nathan Bronson [Wed, 11 Mar 2015 05:21:54 +0000 (22:21 -0700)]
SharedMutex - a small fast scalable reader-writer lock

Summary:
SharedMutex is a reader-writer lock.  It is small, very fast, scalable
on multi-core, and suitable for use when readers or writers may block.
Unlike most other reader-writer locks, its throughput with concurrent
readers scales linearly; it is able to acquire and release the lock
in shared mode without cache line ping-ponging.  It is suitable for a
wide range of lock hold times because it starts with spinning, proceeds
to using sched_yield with a preemption heuristic, and then waits using
futex and precise wakeups.

SharedMutex provides all of the methods of folly::RWSpinLock,
boost::shared_mutex, boost::upgrade_mutex, and C++14's
std::shared_timed_mutex.  All operations that can block are available in
try, try-for, and try-until (system_clock or steady_clock) versions.
Both reader-priority and writer-priority versions are provided.
Writer-priority is the default at the moment.

In my tests SharedMutex is as good or better than the other reader-writer
locks in use at Facebook for almost all use cases, sometimes by a wide
margin.  (If it is rare that there are actually concurrent readers
then RWSpinLock can be a few nanoseconds faster.)  I compared it to
folly::RWSpinLock, folly::RWTicketSpinLock64, boost::shared_mutex,
pthread_rwlock_t, and an internal RWLock that uses spinlocks to guard
its state and pthread_mutex_t+pthread_cont_t to perform blocking.
(The other ReadWriteMutex-s I found use pthread_rwlock_t underneath.)
It is generally as good or better than the rest when evaluating size,
speed, scalability, or latency outliers.  In the corner cases where
it is not the fastest (such as single-threaded use or heavy write
contention) it is never very much worse than the best.  See the bottom
of SharedMutexTest.cpp for lots of microbenchmark results.

Test Plan:
1. new unit tests
2. new microbenchmarks included here
3. uncommitted microbenchmark from bmaurer's RWSleepLock
4. replace admarket's RWSpinLock and RWLock with SharedMutex, observe
neutral adindexer perf in canary
5. replace multifeed's thrift ReadWriteLock with SharedMutex, observe
neutral perf in canary

Reviewed By: hans@fb.com

Subscribers: fbcode-common-diffs@, tnovak, march, davejwatson, trunkagent, philipp, folly-diffs@, yfeldblum, bwatling, bmaurer, bol, marccelani, adri, strager, macsyz, dzhulgakov, zamsden

FB internal diff: D1798929

Signature: t1:1798929:1425575976:1c9221317eaa47628a2b8c374f90c7a2d4e3f0f9

9 years agoAdd null check to avoid crash in unit tests that use mock singletons.
Adrian Hamza [Mon, 9 Mar 2015 21:04:47 +0000 (14:04 -0700)]
Add null check to avoid crash in unit tests that use mock singletons.

Summary: Some unit tests owned by my team are failing due to segmentation fault in SingletonHolder<T>::registerSingletonMock -> SingletonHolder<T>::destroyInstance.

Test Plan: Run folly unit tests and my unit tests.

Reviewed By: henryf@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1873889

Signature: t1:1873889:1425925156:29d54092939d7e9debea3fd55f7105fd320e987e

Blame Revision: 91f4942e

9 years agoAdd JSON Schema Validator
Andrey Goder [Sat, 7 Mar 2015 00:06:18 +0000 (16:06 -0800)]
Add JSON Schema Validator

Summary:
This is a validator for  JSON schema (http://json-schema.org/) that works on
folly::dynamic. Apparently there are no good open source ones for C++,
especially not if you want to use folly::dynamic. I am going to use this to
validate JSON configs.

It supports basically everything from the standard, except for fetching schemas
via http, and using id refs. It supports enough to check schemas against the
metaschema.

Currently you can define a schema that will crash on validation, if it's
infinitely self-recursive. See the unit test case that reproduces this.
Fixing this seems hard though, so I didn't bother. It would also probably
be slower for normal usage.

Test Plan: unit test

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1847657

Signature: t1:1847657:1425605163:635dc523aeda1b588c3634d0dc1a48d50a53db79

9 years agoBump version to 30:0 v0.30.0
Viswanath Sivakumar [Fri, 6 Mar 2015 23:32:28 +0000 (15:32 -0800)]
Bump version to 30:0

9 years agoRemove unnecessary includes from FBVector
Mark Isaacson [Fri, 6 Mar 2015 01:37:46 +0000 (17:37 -0800)]
Remove unnecessary includes from FBVector

Summary:
Remove unnecessary includes from FBVector

Gonna see just how cleaned-up sandcastle really is :P

Test Plan: Fix errors shown in contbuild

Reviewed By: njormrod@fb.com

Subscribers: cold-storage-diffs@, jcoens, ldbrandy, trunkagent, oleksandr, netego-diffs@, mmandal, weiyan, ws_dev_diffs, jmkaldor, jhunt, fbcode-common-diffs@, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1873145

Signature: t1:1873145:1425505591:9cb345053833afa70f543e2f7acd310ab80b8676

9 years agoBump version to 29:0 v0.29.0
Nicholas Ormrod [Thu, 5 Mar 2015 22:44:26 +0000 (14:44 -0800)]
Bump version to 29:0

9 years agohttpserver on serverbootstrap
Dave Watson [Thu, 5 Mar 2015 18:25:54 +0000 (10:25 -0800)]
httpserver on serverbootstrap

Summary:
Cleans up the httpserver startup code nicely.  The only major change to ServerBootstrap was a check if bind failed to throw an exception.

(depends on D1732895)

Test Plan:
fbconfig -r folly/wangle/bootstrap proxygen/httpserver; fbmake runtests

fbconfig -r dfsrouter; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: yfeldblum, cgheorghe, trunkagent, doug, fugalh, bmatheny, folly-diffs@, jsedgwick

FB internal diff: D1800100

Signature: t1:1800100:1424733970:67a61a22d2affadea16d2fd725003915326077b2

9 years agoBump version to 28:0 v0.28.0
Sara Golemon [Thu, 5 Mar 2015 18:22:59 +0000 (10:22 -0800)]
Bump version to 28:0

9 years agoBugfix uriEscapeTable generate.
Ram Kumar Rengaswamy [Sat, 14 Feb 2015 20:45:07 +0000 (15:45 -0500)]
Bugfix uriEscapeTable generate.

Summary:
generate_escape_tables.py uses python range function which does not include the end of the range.

Closes #131

Test Plan: See attached test case.

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1871692

Signature: t1:1871692:1425076132:2438ab7554fe87bdef17c82ff27713811a270d7c

9 years agoAdd FunctionScheduler to Makefile.am
Nicholas Ormrod [Thu, 5 Mar 2015 03:50:50 +0000 (19:50 -0800)]
Add FunctionScheduler to Makefile.am

Summary: D1845525 was missing some makefile changes.

Test Plan: Examine similar change in D1732895

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1890197

Signature: t1:1890197:1425526255:e9a23fd96a89d4463cbdfda1385d4fd5f5781e8e

9 years agoMoveWrapper::move()
Hans Fugal [Wed, 4 Mar 2015 20:42:31 +0000 (12:42 -0800)]
MoveWrapper::move()

Summary: sugar providing `foo.move()` instead of `std::move(*foo)`.

Test Plan: Inspection. Using it in another diff.

Reviewed By: yfeldblum@fb.com, hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum

FB internal diff: D1882208

Signature: t1:1882208:1425331389:a9c09ad2739838c829e2afdad64e985810154226

9 years agoBump version to 27:0 v0.27.0
Alecs King [Tue, 3 Mar 2015 03:31:37 +0000 (19:31 -0800)]
Bump version to 27:0

9 years agoAdd test for running after terminate
Subodh Iyengar [Mon, 2 Mar 2015 22:46:46 +0000 (14:46 -0800)]
Add test for running after terminate

Summary:
Add a test to check what will happen if
someone tries to run a job on evb after
termination.

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: seanc, folly-diffs@, yfeldblum

FB internal diff: D1882308

Signature: t1:1882308:1425333248:7ab6692eb0b866fcc9685048eb385bacd90023d1

9 years agomakeMoveWrapper(lvalue)
Hans Fugal [Mon, 2 Mar 2015 21:27:49 +0000 (13:27 -0800)]
makeMoveWrapper(lvalue)

Summary: Because why not?

Test Plan: new unit tests

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum, marccelani

FB internal diff: D1866123

Signature: t1:1866123:1425331395:742369597757456e5925af55c5d4198b607126f1

9 years agoMacOS doesn't have malloc_usable_size()
Sara Golemon [Mon, 2 Mar 2015 05:57:08 +0000 (21:57 -0800)]
MacOS doesn't have malloc_usable_size()

Summary:
Use malloc_size() instead

Test Plan: HHVM build on MacOS

Reviewed By: joelm@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1875364

Signature: t1:1875364:1425061479:7d8cec757c186fa8c30b6500fca6b2fe4c1ed137

9 years agoCreating a flatten() method to get the inner Future<T> value out of a Future<Future...
Dor Gross [Sat, 28 Feb 2015 01:17:18 +0000 (17:17 -0800)]
Creating a flatten() method to get the inner Future<T> value out of a Future<Future<T>>.

Summary: Method will only be applicable for types of the form Future<Future<T>>, calling flatten() on a Future<T> where <T> != Future<R> for any <R> will result with a compile time error.

Test Plan: Added tests in FutureTest.cpp.

Reviewed By: hans@fb.com

Subscribers: ldbrandy, trunkagent, folly-diffs@, jsedgwick, yfeldblum, icanadi

FB internal diff: D1843581

Tasks: 6166903

Signature: t1:1843581:1424399517:7146924db3c238b77dd309f1d5916e44a7e81968

9 years agoAdd const to the variable: needle
zhaokai [Fri, 27 Feb 2015 07:42:32 +0000 (15:42 +0800)]
Add const to the variable: needle

Summary:
Make it clear that the variable is const

Closes #139

Test Plan:
authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878361

Signature: t1:1878361:1425071448:6871577ddb34f4a8dc30ea37842f9950c3825dde

9 years agoUpdate UnitTest of MapUtilTest.cpp
zhaokai [Fri, 27 Feb 2015 07:20:35 +0000 (15:20 +0800)]
Update UnitTest of MapUtilTest.cpp

Summary:
1. Split the original simple test
2. Add unit test of function: get_ref_default

Closes #138

Test Plan: authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878603

Signature: t1:1878603:1425073330:914a2365fa648142d11b41b387423d78e52f2e08

9 years agoFix comment of Benchmark.h
zhaokai [Fri, 27 Feb 2015 07:51:45 +0000 (15:51 +0800)]
Fix comment of Benchmark.h

Summary:
Remove one of the "one" in the comment of BENCHMARK

Closes: #140
Test Plan: Inspection.

Reviewed By: markisaa@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1877997

Signature: t1:1877997:1425063475:c53c09d105809de9a9070f464fb0571a4dd1e5d1

9 years agoattempt at putting thrift2 on ChannelPipeline
Dave Watson [Fri, 27 Feb 2015 18:14:46 +0000 (10:14 -0800)]
attempt at putting thrift2 on ChannelPipeline

Summary:
Basically a ripoff of Dave's D1483148 but with ChannelPipeline instead of rx.
There's going to be a long tail of unit tests here, this is still pretty broken...

Not convinced that this is the way to go but I wanted to get a feel for what it would be like to plug it into an IRL system.
I think the simplicity/clarity of the handler implementations and to some extent the pipeline spec is an improvement over the rx way
but I'm less sure about everything else.

Test Plan: compiles and kind of works some of the time but not really

Reviewed By: alandau@fb.com

Subscribers: wormhole-diffs@, mcduff, hitesh, doug, yfeldblum, folly-diffs@, jsedgwick, subodh, andrewcox, njormrod, trunkagent, fugalh, alandau, bmatheny, mshneer

FB internal diff: D1618704

Tasks: 5981284

Signature: t1:1618704:1424980854:83c6193b1156be2ec65cf79f9f2ad1af670da7f8

9 years agoAdd willEqual to C++ Futures Code
Monica Lee [Fri, 27 Feb 2015 17:37:25 +0000 (09:37 -0800)]
Add willEqual to C++ Futures Code

Summary: Added willEqual function and wrote unit tests for it.

Test Plan: fbconfig --clang folly/futures && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: hannesr, trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1859840

Tasks: 6166911

Signature: t1:1859840:1424967149:865ee96ab4d3f5dbf17eb371b2ac3ccb5066ac87

9 years agosupport IndexedMemPool for types without default constructor
Nathan Bronson [Thu, 26 Feb 2015 22:54:59 +0000 (14:54 -0800)]
support IndexedMemPool for types without default constructor

Summary:
This diff gives IndexedMemPool<T> emplace-like semantics when T
is not trivial.

Test Plan:
1. new unit tests
2. LifoSem benchmark

Reviewed By: march@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1874941

Signature: t1:1874941:1424987308:61bbe7b7e5e6df625a6208cd873c65e523a79fa0

9 years agoCopyright 2014->2015
Nicholas Ormrod [Thu, 26 Feb 2015 21:00:59 +0000 (13:00 -0800)]
Copyright 2014->2015

Summary:
Update copyright notices in folly to 2015

find folly/ -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'find . -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: ldbrandy, shilin, sdwilsh, fugalh, njormrod, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1869183

Signature: t1:1869183:1424823514:a2dd33741a86bf099d5c9e3d8702e6cca6dd672f

9 years agoEventBase::runAfterDelay to throw an exception
Haim Grosman [Wed, 4 Feb 2015 17:15:10 +0000 (09:15 -0800)]
EventBase::runAfterDelay to throw an exception

Summary:
EventBase::runAfterDelay to throw an exception in case it fails to
schedule a proper execution of the given callback
(instead of silently returning false)

it appears to be the right thing to do, since:
@davejwatson: Digging through the layers of code, it appears this can only fail if
epoll_ctl() with EPOLL_CTL_ADD fails. Ignoring libevent bugs, it looks
like the only relevant errors could be ENOMEM or ENOSPC. So
nonrecoverable

Test Plan: Unit tests

Reviewed By: anca@fb.com

Subscribers: yzhan, haijunz, simpkins, net-systems@, varunk, zeus-diffs@, nli, dfechete, fugalh, atlas2-eng@, alandau, bmatheny, everstore-dev@, zhuohuang, wormhole-diffs@, anca, mwa, jgehring, oujin, alikhtarov, fuegen, mshneer, wch, bil, sanketh, zippydb, maxwellsayles, jsedgwick, trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, andrewcox, unicorn-diffs@, tw-eng@, xie, kennyyu, yfeldblum, folly-diffs@, davejwatson

FB internal diff: D1805125

Signature: t1:1805125:1424927912:8bebb4c3b9f1fa189c0ce97b12cdb8f95dba97ae

9 years agoUpdating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'
John Ehrhardt [Thu, 26 Feb 2015 04:52:21 +0000 (20:52 -0800)]
Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'

Summary:
Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'.
Updated Decimal Integers 'd' use of separators to directly use commas ',' and grouping of 3 digits.
Updated Numbers use of separators and grouping to follow locale settings.

Test Plan:
Wrote unit tests to validate the insertion of thousandsSeparators works.
Note that the numbers unit test does not modify the locale since the test cases are not thread safe.

Reviewed By: lesha@fb.com

Subscribers: tudorb, trunkagent, folly-diffs@, yfeldblum, aric

FB internal diff: D1821156

Tasks: 6087521

Signature: t1:1821156:1424923837:d0fb383a07fd733375b72b1905e6112afa141265