folly.git
9 years agoIOBuf::reserve would return less tailroom than requested under certain circumstances
Tudor Bosman [Fri, 1 May 2015 00:17:13 +0000 (17:17 -0700)]
IOBuf::reserve would return less tailroom than requested under certain circumstances

Test Plan: test added

Reviewed By: lxiong@fb.com

Subscribers: lxiong, net-systems@, folly-diffs@, yfeldblum, chalfant, pamelavagata, kma

FB internal diff: D2036967

Tasks: 6925950

Signature: t1:2036967:1430431606:3e115f7ed76b207572db26d352bebefe7a3d306d

9 years agoFix race condition in collect(..)
Vladislav Isenbaev [Thu, 30 Apr 2015 22:39:27 +0000 (15:39 -0700)]
Fix race condition in collect(..)

Summary: This is a temporary fix (until D2015320 is checked in) for race condition(s) in collect(..) method.

Test Plan:
Run unit tests
Run buffalo_aggregator canary

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037406

Tasks: 6894157

Signature: t1:2037406:1430435227:ed9612d016cdbd708e2deba02dc4fe0b59632f5a

9 years agoUnit::Lift<T>
Hans Fugal [Thu, 30 Apr 2015 22:18:15 +0000 (15:18 -0700)]
Unit::Lift<T>

Summary: Lift void into the unit monad and pass other types through unscathed.

Test Plan: new unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029785

Signature: t1:2029785:1430333928:ef2fbb2e3d94518a732f6818a06c32481120bd4f

9 years agoDo not construct a Future<T> from a Future<Something> value
Hans Fugal [Thu, 30 Apr 2015 22:09:05 +0000 (15:09 -0700)]
Do not construct a Future<T> from a Future<Something> value

Summary: The value constructor can be nice. But when it matches on Future<Something> it just confuses everybody.

Test Plan:
building and running tests
contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036455

Tasks: 6925951

Signature: t1:2036455:1430423503:73906f748318c4ebec13f45ad3761f104e2ef888

9 years agoConvenience functions to wrap a lambda as an AsyncTimeout.
Marcelo Juchem [Thu, 30 Apr 2015 21:03:16 +0000 (14:03 -0700)]
Convenience functions to wrap a lambda as an AsyncTimeout.

Summary: see title

Test Plan: unit tests added

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, andrewcox

FB internal diff: D2000579

Signature: t1:2000579:1430345677:1d7a78f94bcd8b0912423ca4987a4048c103241c

9 years agofix return type for map
James Sedgwick [Thu, 30 Apr 2015 20:13:55 +0000 (13:13 -0700)]
fix return type for map

Summary:
facepalm

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2036523

Tasks: 6925951

Signature: t1:2036523:1430423049:b69ace0256eeacdd43490465dc1c862f054a0098

9 years agoMove a little test
Hans Fugal [Thu, 30 Apr 2015 19:36:26 +0000 (12:36 -0700)]
Move a little test

Summary: truffleshuffle

Test Plan: runtests

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029715

Signature: t1:2029715:1430285509:165edabb1bdbb2a4766e7619c67e7e568626b903

9 years agoFuture<Unit>::Future()
Hans Fugal [Thu, 30 Apr 2015 18:59:36 +0000 (11:59 -0700)]
Future<Unit>::Future()

Summary: Allow `makeFuture()`-like default ctor for `Future<Unit>`

Test Plan: new unit test

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029677

Signature: t1:2029677:1430417794:5ec7fca839294316957803229f4783f2ee875027

9 years agoTag dispatch for enqueue/dequeue implementation
Chao Yang [Thu, 30 Apr 2015 18:26:23 +0000 (11:26 -0700)]
Tag dispatch for enqueue/dequeue implementation

Summary:
clang (>=3.6?) reports potential object slicing bug when MPMCQueue is used for
polymorphic class as the queue item, e.g. as in P19814469. This can be false
positive however, since the choice is based on the type trait already.  This
diff uses tag dispatch to selectively compile the overload that will be
executed, therefore if there is no-throw move ctor supplied clang will not
examine the simulated relocation code.

This doesn't avoid object slicing bug however if the client insists to use
MPMCQueue to hold base class while enqueue and dequeue with subclassed item.

Test Plan: compile with --clang

Reviewed By: tudorb@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2029949

Signature: t1:2029949:1430264357:af479117adf90bc1915c071e7376a30aacb72f46

9 years agounidirectional pipelines
James Sedgwick [Thu, 30 Apr 2015 18:09:50 +0000 (11:09 -0700)]
unidirectional pipelines

Summary:
Cleans up bootstrap a bit at the expense of a more complex Pipeline interface
This doesn't have to go in, lmk either way as I want to move on to reorganizing this code into inl headers etc

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2034634

Signature: t1:2034634:1430414670:c91712fb26353987cb471e35a54f55c869ae7cf1

9 years agouse inbound/outbound handlers in a few more places
James Sedgwick [Thu, 30 Apr 2015 01:20:45 +0000 (18:20 -0700)]
use inbound/outbound handlers in a few more places

Summary: Also, instead of throwing on finalize() if there's no inbound/outbound handler in the pipeline, log a warning and throw when the operations are attempted. This was necessary for CodecTest which doesn't use outbound handlers.

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2028176

Tasks: 6836580

Signature: t1:2028176:1430346333:fdd645a535e8158d780cfd1119e27803995b663f

9 years agoinbound/outbound handlers
James Sedgwick [Thu, 30 Apr 2015 01:04:41 +0000 (18:04 -0700)]
inbound/outbound handlers

Summary:
Much less copypasta this time around. I wonder if the getters and setters for write flags and read buffer settings are necessary in the new handler types, or even if they belong in the bidirectional handler

I'm all ears for more suggestions on reducing copypasta

I'm going to reorg the code (inl headers etc) in a subsequent diff once this is in - easier to review this way

Test Plan: existing unit, thinking about tests for these changes

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2026522

Tasks: 6836580

Signature: t1:2026522:1430346145:bd7f7770eddce0470e2ac72440fc001cf128df08

9 years agomoar CRTP to minimize copypasta for inbound/outbound handlers
James Sedgwick [Thu, 30 Apr 2015 00:56:02 +0000 (17:56 -0700)]
moar CRTP to minimize copypasta for inbound/outbound handlers

Summary:
I think this is the best I can do to minimize whjat must be copied for InboundHandler, InboundHandlerContext, InboundContextImpl, outbound equivalents, etc

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2025612

Tasks: 6836580

Signature: t1:2025612:1430346200:bc06162711bbc52b17574297a3569736f7973a7c

9 years agostruct/class consistency
Nicholas Ormrod [Thu, 30 Apr 2015 00:43:19 +0000 (17:43 -0700)]
struct/class consistency

Summary:
This diff is based on https://github.com/facebook/folly/pull/95
by gitbug user mikekap.

It changes the struct/class naming of forward declarations to match
their definitions.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2032995

Tasks: 6783581

Signature: t1:2032995:1430339337:79193fd8409b73e8a9155133f326b7cee534244d

9 years agoBump version to 37:0 v0.37.0
Andrii Grynenko [Wed, 29 Apr 2015 22:57:48 +0000 (15:57 -0700)]
Bump version to 37:0

9 years agofix unittest break
Dave Watson [Wed, 29 Apr 2015 22:16:28 +0000 (15:16 -0700)]
fix unittest break

Summary: It looks like the ubuntu version of gtest complains about this?

Test Plan:
Will watch jenkins fbthrift and proxygen builds

previous jenkins break:
https://ci-builds.fb.com/job/folly/2178/console

Reviewed By: bmatheny@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2032761

Tasks: 6913742

Signature: t1:2032761:1430337624:ec65b26cf8da7d0d8c7e449f235de7147d4f17d6

Blame Revision: D2029802

9 years agoMove Singleton out of folly/experimental into folly/
Andrii Grynenko [Wed, 22 Apr 2015 02:37:08 +0000 (19:37 -0700)]
Move Singleton out of folly/experimental into folly/

Test Plan: contbuild

Reviewed By: chip@fb.com

Subscribers: wormhole-diffs@, trunkagent, hphp-diffs@, jan, simpkins, configerator-diffs@, fbcode-common-diffs@, chaoyc, bill, search-fbcode-diffs@, agallagher, nli, marcelo, ckwalsh, mcduff, hitesh, mshneer, unicorn-diffs@, vighnesh, fugalh, andreib, bmatheny, tw-eng@, tanmoyc, zhuohuang, rvm4, antonl, acampi, alikhtarov, hdoshi, rsethi, panin, folly-diffs@, lins, kennyyu, hannesr, jsedgwick, dominik, yfeldblum, songhao, raghavpi, labrams, lyang, chalfant, #preselection, macsyz, nimishshah

FB internal diff: D2012267

Tasks: 5676394

Signature: t1:2012267:1430334667:eaad0262b35ffbfae86df5bdb45bf057ac62c51b

9 years agoShave off 1.3s of THeader.h compile time
Alex Landau [Tue, 28 Apr 2015 22:32:13 +0000 (15:32 -0700)]
Shave off 1.3s of THeader.h compile time

Summary:
THeader.h went from 1.8s to 0.5s.
Total build of sample project doing thrift client and server stuff went
down 1.5-3% from

```
(run 1)
real    5m25.059s
user    130m30.853s
sys     10m18.642s

(run 2)
real    5m29.483s
user    129m12.209s
sys     10m14.762s
```

to
```
(run 1)
real    5m23.292s
user    125m44.318s
sys     10m7.103s

(run 2)
real    5m20.999s
user    127m4.969s
sys     10m10.118s
```

Test Plan:
fbmake --distcc off --ccache off dbg
Will also watch contbuild, since some files might be missing includes

Reviewed By: haijunz@fb.com

Subscribers: jmarg, anca, reachfrequency-eng@, jhunt, yanli, jgeller, chaoyc, search-fbcode-diffs@, net-systems@, zeus-diffs@, vikas, jcoens, unicorn-diffs@, ldbrandy, jteller, atlas2-eng@, everstore-dev@, leis, benj, laser-diffs@, zhguo, jying, wanghuan, jeremyfein, dbolcioni, jacekm, maxwellsayles, osmith, pallotron, fbcode-common-diffs@, davejwatson, andrewcox, marcelo, ckwalsh, mcduff, hitesh, mshneer, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, haijunz, chalfant

FB internal diff: D2029911

Signature: t1:2029911:1430261056:9ae688c50585a6454479f810d80b2c1c79b60e04

9 years agoMove this `reduce` to `helpers.h`
Hans Fugal [Wed, 29 Apr 2015 16:46:26 +0000 (09:46 -0700)]
Move this `reduce` to `helpers.h`

Summary: just a lowly non-erroring semantic merge conflict

Test Plan: builds

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2029593

Signature: t1:2029593:1430319785:2cd96927b9080fe18b168ab95ad201afc4f00857

9 years agoget_or_throw and get_optional
Blake Matheny [Wed, 29 Apr 2015 15:35:30 +0000 (08:35 -0700)]
get_or_throw and get_optional

Summary:
Adds get_or_throw map helper (get a value, or throw with the specified
exception type) and get_optional (get an Optional<Value>). This is a folly
backport of some util helpers in experimental.

Test Plan: Unit tests

Reviewed By: wez@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2029802

Tasks: 4332480

Signature: t1:2029802:1430280249:3efbb2fb9394e31b3fbe6d5bd209d12ebb7ed587

9 years agoAn IOExecutor that executes under mapped fiber context.
Rushi Desai [Wed, 29 Apr 2015 06:17:06 +0000 (23:17 -0700)]
An IOExecutor that executes under mapped fiber context.

Summary:
This should help executing sync thrift handlers on fibers. This allows the handler methods to achieve concurrency by awaiting.

Test Plan: Chimera unit tests (which also accesses running FacebookBase methods in fiber context!)

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, hannesr, andrii, folly-diffs@, jsedgwick, haijunz, yfeldblum, alandau, chalfant

FB internal diff: D2015886

Signature: t1:2015886:1430156870:a84499e0e781f5e6e77f48f1f3063f53d0968698

9 years agoDynmaic load shedding
Woo Xie [Tue, 28 Apr 2015 22:25:31 +0000 (15:25 -0700)]
Dynmaic load shedding

Summary:
add a free system memory limit to Load Shed Configuration for dynamic shedding

Test Plan:
tested on edge241.01.ams3, works as expected. Once the free memory is less than 23G (picked on purpose for testing),  new connections are shedded.   But the shedding rate is totally wrong.  I will explain and address it in another diff.

https://fburl.com/103620501
https://fburl.com/103620701

Reviewed By: afrind@fb.com

Subscribers: trunkagent, fugalh, bmatheny, nimishshah, folly-diffs@, jsedgwick, yfeldblum, chalfant, xning, alexkr

FB internal diff: D2026477

Tasks: 4604893

Signature: t1:2026477:1430176828:65eadd82efa3189a1bebcb8518efaf56cd36beed

9 years agoPromise::setValue() for Unit
Hans Fugal [Mon, 27 Apr 2015 21:40:09 +0000 (14:40 -0700)]
Promise::setValue() for Unit

Summary: Unit is a bit special because it's just something special to use instead of `Promise<void>`, so let's offer the same sugar that `Promise<void>` has (`p.setValue()` instead of `p.setValue(Unit())`)

Test Plan: New unit tests. Look, a pun!

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, davejwatson, hannesr

FB internal diff: D2014139

Tasks: 6847876

Signature: t1:2014139:1430159950:1484ee420c6d7f0f794a546b78ef1601c2eec45c

9 years agoFix default arguments for orderBy()
Giuseppe Ottaviano [Mon, 27 Apr 2015 20:45:25 +0000 (13:45 -0700)]
Fix default arguments for orderBy()

Summary: Calling `folly::gen::orderBy()` without arguments causes a compilation error because the first template argument cannot be deduced. This diff fixes it.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2025789

Signature: t1:2025789:1430167404:02fde7287b015d9dcbf398e8dc84cde7d74b4a5b

9 years agoBenchmarkSuspender::dismissing.
Yedidya Feldblum [Mon, 27 Apr 2015 19:49:35 +0000 (12:49 -0700)]
BenchmarkSuspender::dismissing.

Summary:
[Folly] BenchmarkSuspender::dismissing.

Pass a lambda to it, and the lambda will be executed while the benchmark-suspender is dismissed. Just a bit of sugar around `BenchmarkSuspender::dismiss` and `BenchmarkSuspender::rehire`.

BENCHMARK(name_void, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
braces.dismissing([&] {
# benchmark timer is running
doSomething();
});
# benchmark timer is suspended
}

BENCHMARK(name_value, iters) {
BenchmarkSuspender braces;
# benchmark timer is suspended
auto value = braces.dismissing([&] {
# benchmark timer is running
return doSomething();
});
# benchmark timer is suspended
}

Test Plan:
Unit tests:
* `folly/test/BenchmarkTest.cpp` (actually a benchmark)

Reviewed By: njormrod@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2024166

Signature: t1:2024166:1430163281:24df0ac98cbe36372f780372ee8f7dd3722b7868

9 years agoHandler::getContext() when possible
James Sedgwick [Mon, 27 Apr 2015 19:24:25 +0000 (12:24 -0700)]
Handler::getContext() when possible

Summary:
Only allow this if the handler is only ever attached to a single pipeline once. i.e. only ever associated with one Context

Test Plan: unit, thrift unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2024007

Tasks: 6836580

Signature: t1:2024007:1430157264:efcf70ca3531c10eec5d458c9e9d6cda60c507c3

9 years agofix detachPipeline/attachPipeline ordering
James Sedgwick [Mon, 27 Apr 2015 19:12:53 +0000 (12:12 -0700)]
fix detachPipeline/attachPipeline ordering

Summary:
detachPipeline always goes bottom to top
attachPipeline always goes top to bottom
now we can attachReadCallback in AsyncSocketHandler::attachPipeline()

not sure of the implications for TAsyncTransportHandler... looks like Cpp2Channel still wants to attach/detach cb manually

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2023982

Tasks: 6836580

Signature: t1:2023982:1430157500:e09a4103550a3e5721effaa1b28ac5bed071fa67

9 years agoBump version to 36:0 v0.36.0
Alecs King [Mon, 27 Apr 2015 23:56:01 +0000 (16:56 -0700)]
Bump version to 36:0

9 years agoREADME: Fixed typo, formatted code, and linked to docs
Nicholas Ormrod [Fri, 17 Apr 2015 01:56:01 +0000 (18:56 -0700)]
README: Fixed typo, formatted code, and linked to docs

Summary:
(from @njormrod) This changes the READMEs to be .md,
appropriate for github. I viewed the docs on github, and they looked
WAY better as .md files.

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: view on github

Reviewed By: markisaa@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1999531

Tasks: 6783581

Signature: t1:1999531:1429220432:68c6d8dc70806d993c83cacce6369cb7e6e964c8

9 years agoManage handlers with shared_ptrs, introduce StaticPipeline
James Sedgwick [Mon, 27 Apr 2015 18:44:31 +0000 (11:44 -0700)]
Manage handlers with shared_ptrs, introduce StaticPipeline

Summary:
A few things:
- Eliminate HandlerPtr by managing all handlers with shared_ptrs instead of keeping them inline in the pipeline
- Kill recursively templated ChannelPipeline accordingly
- Introduce StaticPipeline to retain the flexibility of zero-alloc pipelines
- Introduce notion of an "owning handler" to avoid destruction order issues

Test Plan: unit (will add more), thrift unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2023976

Tasks: 6836580

Signature: t1:2023976:1430159578:e50e8a149e549a40670d093fb65987a4843cdd8d

9 years agorearrange Future.h
Hans Fugal [Mon, 27 Apr 2015 18:20:04 +0000 (11:20 -0700)]
rearrange Future.h

Summary: Move boring preamble stuff to `Future-pre.h` and `folly::futures` and `makeFuture` and pals to `helpers.h`.

Test Plan:
tests still build and pass

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant, hannesr, davejwatson

FB internal diff: D2014330

Signature: t1:2014330:1429941589:1e2c336136f3375f9b96e5df8c06ca5820ba6aeb

9 years agostrip Channel from all class names
James Sedgwick [Mon, 27 Apr 2015 18:19:18 +0000 (11:19 -0700)]
strip Channel from all class names

Summary: as above. Only got a little messy when components within folly::wangle typedefed things to Pipeline

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

Subscribers: wormhole-diffs@, fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2022181

Tasks: 6836580

Signature: t1:2022181:1430157032:df0bdfb9ca0d76b86d52c55c4ad41ea953a18cb4

9 years agoFix oss build
Andre Pinto [Mon, 27 Apr 2015 16:34:34 +0000 (09:34 -0700)]
Fix oss build

Summary: Fixing oss build

Test Plan: Visual analysis.

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2023889

Signature: t1:2023889:1430144784:e58fe9e1f1193d71527ba5f5f2c2efb60bce7a2d

Blame Revision: D2022859

9 years agoIntroducing Unit!
Hans Fugal [Sat, 25 Apr 2015 01:10:49 +0000 (18:10 -0700)]
Introducing Unit!

Summary: title

Test Plan: tests

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2022859

Signature: t1:2022859:1429920863:0d3afce9779a790baf246de97f98812e23e1574d

9 years agoIPAddressV6::kToFullyQualifiedSize.
Yedidya Feldblum [Fri, 24 Apr 2015 00:02:55 +0000 (17:02 -0700)]
IPAddressV6::kToFullyQualifiedSize.

Summary:
[Folly] IPAddressV6::kToFullyQualifiedSize.

The scenario: One has a string known to be a fully-qualified IPv4 or fully-qualified IPv6 address, but one does not know which of the two it is.

The dirty but fast solution: just check its size.

Test Plan: Build. Unit tests.

Reviewed By: mshneer@fb.com

Subscribers: folly-diffs@, dougw, yfeldblum, chalfant, brettp

FB internal diff: D2016634

Signature: t1:2016634:1429831808:08ecb28eab3060988f9613748ed8dabf3724bf13

9 years agomakeFutureTry -> makeFutureWith
James Sedgwick [Thu, 23 Apr 2015 17:44:12 +0000 (10:44 -0700)]
makeFutureTry -> makeFutureWith

Summary: Similar to Promise::fulfil -> setWith change, this name is a lot clearer

Test Plan: tests

Reviewed By: hans@fb.com

Subscribers: netego-diffs@, fugalh, mwa, jgehring, fuegen, folly-diffs@, tingy, jsedgwick, yfeldblum, chalfant

FB internal diff: D2013124

Tasks: 6837405

Signature: t1:2013124:1429735106:e8861925dfeb6d7f0662c1057cbcf2ad8dcf008c

9 years agoChange ThreadManager interface to Executor
Dave Watson [Thu, 23 Apr 2015 17:03:51 +0000 (10:03 -0700)]
Change ThreadManager interface to Executor

Summary:
Change ThreadManager interface to more generic Executor interface.  Specific example of supporting fibers.  This diff would replace D1967655

Test Plan: added several unittests

Reviewed By: alandau@fb.com

Subscribers: doug, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, alikhtarov, andrii, yitingli

FB internal diff: D1994663

Signature: t1:1994663:1429640509:d729ce6f020563b680d1d549f9aa273c739eb925

9 years agoCleanup Future(Value) ctor
Hans Fugal [Wed, 22 Apr 2015 23:32:22 +0000 (16:32 -0700)]
Cleanup Future(Value) ctor

Summary:
We don't need to check for void after all, and with perfect forwarding we don't need separate const& and && versions.

Test Plan: tests still pass

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2014264

Tasks: 6847876

Signature: t1:2014264:1429735036:01ac166399ef8d0f2f34adb51e965809022c2b64

9 years agotake collections by reference
James Sedgwick [Wed, 22 Apr 2015 21:27:36 +0000 (14:27 -0700)]
take collections by reference

Summary: This covers the ubiquitous case. If the approach is approved (the task mentioned changing impl to use 'auto for' so this is a bit different) i'll codemod callsites

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2013573

Tasks: 6779710

Signature: t1:2013573:1429735631:cfa0d3f6672a8966afc0ea18308307e2153793ce

9 years agoMake MIN_WRITE_SIZE configurable for AsyncSSLSocket.
Jun Li [Wed, 22 Apr 2015 21:09:07 +0000 (14:09 -0700)]
Make MIN_WRITE_SIZE configurable for AsyncSSLSocket.

Summary:
MIN_WRITE_SIZE is used to avoid small write calls to SSL_write. If there are
consecutive small buffers to write, then will be combined together(by being
copied to a local buffer) until total size exceeds MIN_WRITE_SIZE. This reduces
number of calls SSL_write, improving performance, and avoiding overhead in OpenSSL.

Currently, MIN_WRITE_SIZE is hard coded to be 1500 bytes.

Wormhole could benefit from this, as our average message size is several
hundreds of bytes. We could get even better throughput with larger
MIN_WRITE_SIZE.

As discussed with Adam and Alan, there is a good reason to make it
configurable, though default value is still 1500.

Test Plan: unit tests

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, thomasf

FB internal diff: D1996570

Tasks: 6784543

Signature: t1:1996570:1429667035:a661ef30a715dafec3e134a7f6af6f56ada2e8e0

9 years agoAdd support for creating SocketAddress directly from IPAddress object
Naizhi Li [Wed, 22 Apr 2015 20:38:24 +0000 (13:38 -0700)]
Add support for creating SocketAddress directly from IPAddress object

Summary:
Today it's hard to create SocketAddress from IPAddress
without converting to strings and back. This change adds this capability.

Test Plan: build

Reviewed By: yfeldblum@fb.com

Subscribers: ps, bmatheny, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2011879

Signature: t1:2011879:1429733811:c05fc2d5ef5e9fdbbbb54ad26b4d1e3c0ad06dfa

9 years agofuture / fiber integration
Dave Watson [Wed, 22 Apr 2015 16:02:38 +0000 (09:02 -0700)]
future / fiber integration

Summary:
make future::wait() use fiber's baton, so wait works in threads or in fibers.

Much cleaner than making a new FiberRequest type in thrift

Test Plan: tests

Reviewed By: andrii@fb.com

Subscribers: doug, alandau, bmatheny, mshneer, andrii, folly-diffs@, yitingli, yfeldblum, jsedgwick, chalfant

FB internal diff: D1996283

Signature: t1:1996283:1429144165:da5dc6b1f2a053a45efd39877e79169e3fba810c

9 years agowhen* -> collect*
James Sedgwick [Wed, 22 Apr 2015 14:35:31 +0000 (07:35 -0700)]
when* -> collect*

Summary: title

Test Plan: tests

Reviewed By: hans@fb.com

Subscribers: laser-diffs@, trunkagent, mhl, rhe, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, hero-diffs@, zeus-diffs@, vikas, cold-storage-diffs@, rtgw-diffs@, unicorn-diffs@, targeting-diff-backend@, netego-diffs@, apollo-diffs@, everstore-dev@, zhuohuang, zhguo, jying, darshan, apodsiadlo, alikhtarov, folly-diffs@, wch, lins, tingy, jsedgwick, thom, yfeldblum, petchean, iaroslav, qhuang, gus, san, tomasz, pknowles, lyang, chalfant, paggarw, stevenkim

FB internal diff: D2003300

Tasks: 6025255

Signature: t1:2003300:1429659170:e18999cba45e8aa9019aa94f1f29732076a274ad

9 years agofix collect() for move-only types
James Sedgwick [Tue, 21 Apr 2015 23:52:09 +0000 (16:52 -0700)]
fix collect() for move-only types

Summary:
as above. it never ends.

Test Plan: added unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2011569

Signature: t1:2011569:1429660210:930cb17682d5c86a11881a23efe0a91f4c6a36b1

9 years agothen-with-Executor
Hans Fugal [Tue, 21 Apr 2015 23:52:06 +0000 (16:52 -0700)]
then-with-Executor

Summary:
Pass an Executor to `then`, which applies only for that callback. This is on
the one hand just a convenience method, but it's a major convenience when it's
needed, because grabbing, storing, and restoring the old Executor in the
middle of a chain is very inconvenient indeed.

Test Plan:
new unit
docblock comments

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, davejwatson, chalfant, yfeldblum, nkgupta, jsedgwick, exa, robbert

FB internal diff: D2011542

Tasks: 67715896838553

Signature: t1:2011542:1429660204:f5959b1e0b3b36dfb8c3c7091302d19101dde93b

9 years agofix collect for non-default-constructible types, for real this time
James Sedgwick [Tue, 21 Apr 2015 21:49:05 +0000 (14:49 -0700)]
fix collect for non-default-constructible types, for real this time

Summary:
this was a fun one. Add a specialized implementation that builds up the results in a map with their indices and aggregates them into a vector at the end

Test Plan: unit tests

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2002444

Signature: t1:2002444:1429642589:ee5aa5e8c461db97a28642b9887b3158df317813

9 years agomap()
James Sedgwick [Tue, 21 Apr 2015 21:08:44 +0000 (14:08 -0700)]
map()

Summary:
simple sugary syrup, best used in conjunction with collect/whenAll

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant, cgthayer

FB internal diff: D1999047

Tasks: 6045623

Signature: t1:1999047:1429570631:62361bb43dc5489fe3d4eb31af404faf8a765402

9 years agoAssume exception when Executor::add throws
Hans Fugal [Tue, 21 Apr 2015 18:19:04 +0000 (11:19 -0700)]
Assume exception when Executor::add throws

Summary:
Rather than crashing spectacularly, if `Executor::add` throws (e.g. because the queue is full), then discard the result we got and assume the exception the executor threw instead.

Alternatively, we could pass this exceptional Try to the callback (without an executor, as it is here), but not perturb `result_`. This would mean two different world views in these two code snippets:

auto f1 = makeFuture(42).via(&crappyExecutor);
f1.value(); // 42 (no callback happened)
f1.then(...); // would see the executor's exception. Would also be ill-advised to do this after value()

auto f2 = makeFuture(42).via(&crappyExecutor)
.then([](int x) { return x * 2; }); // skipped
f2.value(); // throws executor's exception

It feels rude to throw away the result, but it feels too potentially dangerous to allow this split view of the world.

Test Plan: modified unit

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2007729

Tasks: 5306911

Signature: t1:2007729:1429627114:b627ce758ce9231298f1b28e203ccc1ee415ed9a

9 years agorevert D1985475, clang still borked
James Sedgwick [Mon, 20 Apr 2015 19:20:01 +0000 (12:20 -0700)]
revert D1985475, clang still borked

Summary:
This reverts commit bb08e2405dc68e6dab1f0066b15faa2821ad3dc7.

Test Plan: revert-hammer

Reviewed By: hans@fb.com

Subscribers: mathieubaudet, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2006283

Signature: t1:2006283:1429556243:41e6b3189ce2da5a0f1a32a663ff8761628ca574

9 years agotest failure conditions in LengthFieldBasedFrameDecoder
Dave Watson [Mon, 20 Apr 2015 17:34:01 +0000 (10:34 -0700)]
test failure conditions in LengthFieldBasedFrameDecoder

Summary: Moar unittests.

Test Plan:
unittests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959161

Signature: t1:1959161:1429292487:b7d10be35c2cf1d0bc1b399f4f523392a138a217

9 years agoRevert "Revert "Deprecating folly::is_complete""
Nicholas Ormrod [Sat, 18 Apr 2015 17:57:56 +0000 (10:57 -0700)]
Revert "Revert "Deprecating folly::is_complete""

Summary:
This reverts commit 95a80fd4cb5a4b31b41b34fd24d591e0fc1b0650.

Test Plan: n/a

Reviewed By: andrewjcg@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2004342

Tasks: 6804947

Signature: t1:2004342:1429332777:4dd1c45e22ee4da69e5e84d3c5f67bada263f307

Blame Revision: D2002345

9 years agoRevert "Deprecating folly::is_complete"
Nicholas Ormrod [Fri, 17 Apr 2015 18:54:36 +0000 (11:54 -0700)]
Revert "Deprecating folly::is_complete"

Summary: This reverts commit 42add531d4bdb1f95c40e41981f908a1b70865fb.

Test Plan: n/a

Reviewed By: andrewjcg@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2002345

Tasks: 6804947

Signature: t1:2002345:1429296810:7db71fe4748d5c71f0f0486751ee426c2cdff653

9 years agothen-with-Executor
Hans Fugal [Fri, 17 Apr 2015 17:41:45 +0000 (10:41 -0700)]
then-with-Executor

Summary:
Pass an Executor to `then`, which applies only for that callback. This is on
the one hand just a convenience method, but it's a major convenience when it's
needed, because grabbing, storing, and restoring the old Executor in the
middle of a chain is very inconvenient indeed.

Test Plan:
new unit

Reviewed By: jsedgwick@fb.com

Subscribers: robbert, exa, folly-diffs@, jsedgwick, nkgupta, yfeldblum, chalfant, davejwatson

FB internal diff: D1985475

Signature: t1:1985475:1429148056:0450120263ba8110e8825420cbefe3b1887f7306

9 years agofix collect for non-default-constructible types
James Sedgwick [Fri, 17 Apr 2015 15:45:58 +0000 (08:45 -0700)]
fix collect for non-default-constructible types

Summary: as above

Test Plan: compile

Reviewed By: agoder@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1999218

Signature: t1:1999218:1429217794:0278955bb471fe73a14ccedb59eae80b98acdbf3

9 years agoREADME: Fixed typo, formatted code, and linked to docs
Nicholas Ormrod [Fri, 17 Apr 2015 01:56:01 +0000 (18:56 -0700)]
README: Fixed typo, formatted code, and linked to docs

Summary:
(from @njormrod) This changes the READMEs to be .md,
appropriate for github. I viewed the docs on github, and they looked
WAY better as .md files.

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: view on github

Reviewed By: markisaa@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1999531

Tasks: 6783581

Signature: t1:1999531:1429220432:68c6d8dc70806d993c83cacce6369cb7e6e964c8

9 years agoRemove unused variables
Nicholas Ormrod [Fri, 17 Apr 2015 00:32:30 +0000 (17:32 -0700)]
Remove unused variables

Summary:
This broke the HHVM (on ubuntu at least)

Test Plan:
compiled it

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1999341

Tasks: 6783581

Signature: t1:1999341:1429219016:235beef7522c4ada020eedf4f0a2ae200e94b7c8

9 years agoAdds x64 arch-specific test
Nicholas Ormrod [Fri, 17 Apr 2015 00:35:29 +0000 (17:35 -0700)]
Adds x64 arch-specific test

Summary:
Adds x64 arch-specific test inside the
if HAVE_X86_64

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: tests

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1999428

Tasks: 6783581

Signature: t1:1999428:1429220350:4287342a5a99823c9e4ffe20d0e47039f9fdbc0b

9 years agoUpdate Copyright 2014 -> 2015
Nicholas Ormrod [Fri, 17 Apr 2015 00:29:37 +0000 (17:29 -0700)]
Update Copyright 2014 -> 2015

Summary:
Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Test Plan: Comments only

Reviewed By: markisaa@fb.com, yfeldblum@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1999373

Tasks: 6783581

Signature: t1:1999373:1429219027:225446bb418f3f8d65fb195682e4e7fe315ed41a

9 years agoDeprecating folly::is_complete
Marcelo Juchem [Thu, 16 Apr 2015 22:52:29 +0000 (15:52 -0700)]
Deprecating folly::is_complete

Summary: Deprecating folly::is_complete. Replacement is fatal::is_complete.

Test Plan: unit tests

Reviewed By: njormrod@fb.com

Subscribers: njormrod, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1979124

Tasks: 66974546697600

Signature: t1:1979124:1429153237:dc5736ed0f546dd4dd57bda9176e3c3c13b1235d

9 years agoWe might avoid some temporaries in putMessagesImpl
Nicholas Ormrod [Thu, 16 Apr 2015 20:42:45 +0000 (13:42 -0700)]
We might avoid some temporaries in putMessagesImpl

Summary:
This is folly github pull request https://github.com/facebook/folly/pull/184

It seems we might avoid some temporaries in putMessagesImpl
function. To do so we pass arguments directly to constructors
with the help of emplace_back member.

Test Plan:
All folly/tests, make check for 37 tests, passed.

Signed-off-by: Nicholas Ormrod <njormrod@fb.com>
Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1998515

Tasks: 6783581

Signature: t1:1998515:1429207683:056db129405bed212f50a50821f095c8d3694ec8

9 years agofix build
James Sedgwick [Thu, 16 Apr 2015 19:18:42 +0000 (12:18 -0700)]
fix build

Summary:
^

Test Plan: compiles

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1998822

Signature: t1:1998822:1429211074:3ff9b0827a8ce029efa06a4a78ecea2b07eae3a8

9 years agorunImmediatly fix
Dave Watson [Thu, 16 Apr 2015 18:15:19 +0000 (11:15 -0700)]
runImmediatly fix

Summary: Should also work if eventbase isn't looping

Test Plan: added new unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1984259

Signature: t1:1984259:1428691788:edb89424a48b8071a776d828300b7c1094f1c48f

9 years agocollect()
James Sedgwick [Thu, 16 Apr 2015 01:32:11 +0000 (18:32 -0700)]
collect()

Summary: title

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: fbcode-common-diffs@, targeting-diff-backend@, zhuohuang, thom, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1992144

Tasks: 6025255

Signature: t1:1992144:1429120337:7678f790dd0f383295c036e6627bdf417ae43fc7

9 years agofulfil -> setWith, fulfilTry -> setTry
James Sedgwick [Thu, 16 Apr 2015 01:23:32 +0000 (18:23 -0700)]
fulfil -> setWith, fulfilTry -> setTry

Summary: title

Test Plan: tests

Reviewed By: hans@fb.com

Subscribers: cgist, cold-storage-diffs@, fugalh, atlas2-eng@, zhuohuang, folly-diffs@, jsedgwick, yfeldblum, chalfant, andrii

FB internal diff: D1994472

Tasks: 6768508

Signature: t1:1994472:1429117362:218c4fac3c88fcc8d37dc22ff8fe4135a73ec5d5

9 years agoInstrument fibers execution
Andre Pinto [Thu, 16 Apr 2015 00:56:57 +0000 (17:56 -0700)]
Instrument fibers execution

Summary: Add option to setup fibers' instrumentation.

Test Plan: Unit tests

Reviewed By: andrii@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1991638

Tasks: 6347990

Signature: t1:1991638:1429144487:3c3586b9943b4cc6bbef0297e48b8bffaf586f0d

9 years agoMake sure we can't access LocalData when destroying it
Andrii Grynenko [Wed, 15 Apr 2015 22:26:35 +0000 (15:26 -0700)]
Make sure we can't access LocalData when destroying it

Summary: We can access LocalData while currentFiber is set. We should make sure it's set to null when LocalData::reset is called.

Test Plan: unit test

Reviewed By: alikhtarov@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, bwatling

FB internal diff: D1996071

Tasks: 6725667

Signature: t1:1996071:1429135408:d549d577e140ce2867aff4130e73be3884dbd2ed

9 years agoonError(exception_wrapper)
James Sedgwick [Wed, 15 Apr 2015 17:11:47 +0000 (10:11 -0700)]
onError(exception_wrapper)

Summary: title

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, hannesr, vloh

FB internal diff: D1984864

Tasks: 6045789

Signature: t1:1984864:1429116418:b4a9cdbb88f605a09b5753eea41dd970c96b9d4e

9 years agoremove unnecessary & deprecated activate() call
James Sedgwick [Wed, 15 Apr 2015 14:24:02 +0000 (07:24 -0700)]
remove unnecessary & deprecated activate() call

Summary: let's not do this to ourselves :)

Test Plan: unit

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1993491

Signature: t1:1993491:1429063112:b8664b365ad12fa4b51e9bd7fde9e4d197dbc648

9 years agoEnsure the loop callback is scheduled when the ready queue is not empty
Brian Watling [Wed, 15 Apr 2015 06:04:34 +0000 (23:04 -0700)]
Ensure the loop callback is scheduled when the ready queue is not empty

Summary: Previously we'd call ensureLoopScheduled() but it'd be a no-op since the loop was already scheduled. Delaying the call to ensureLoopScheduled() fixes the issue

Test Plan: unit tests (FiberManager.yieldTest fails without the changes to FiberManager-inl.h)

Reviewed By: andrii@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1993686

Signature: t1:1993686:1429070253:af933abbbbb33868a402f1d643e4e6f5fef1be83

9 years agofix race in ~ThreadWheelTimekeeper
Louis Brandy [Sat, 11 Apr 2015 00:17:37 +0000 (17:17 -0700)]
fix race in ~ThreadWheelTimekeeper

Summary:
During destruction, we enqueue a callback to `eventBase_` that references `wheelTimer_`.

So long as that callback exists, we have a circular dependency as both reference the other and we -cannot- safely destroy this object (and its members). The fix is to break this dependency someway. In this case, I've chosen to simply `AndWait` until the eventBase has consumed the message referencing the timer. From here normal destruction of members can proceed safely (destorying HHWheelTimer first, EventBase second).

The problem in the current code is that the Eventbase will attempt to consume all messages during it's own destruction. If `wheelTimer_->cancelAll()` is still enqueued, it will attempt to use the now destroyed `wheelTimer_`.

Test Plan:
Running this test repeatedly was able to repro:

./folly/futures/futures-test --gtest_filter=Timekeeper.futureWithinVoidSpecialization

Reviewed By: yfeldblum@fb.com

Subscribers: enis, folly-diffs@, jsedgwick, yfeldblum, darshan, chalfant

FB internal diff: D1985967

Tasks: 63327296741095

Signature: t1:1985967:1428726270:997ec277c6a73554e54b8cf673acd36ff62976e6

9 years agoparse flags in unit tests
Louis Brandy [Sat, 11 Apr 2015 00:36:56 +0000 (17:36 -0700)]
parse flags in unit tests

Summary: This code has useful logging when turned up high enough (e.g. --v=11). But we need to parse the flags to see it.

Test Plan: trial & error

Reviewed By: yfeldblum@fb.com

Subscribers: enis, folly-diffs@, jsedgwick, yfeldblum, darshan, chalfant

FB internal diff: D1985965

Signature: t1:1985965:1428713065:4607610ec438416fffb5fced1867efa13544eef0

9 years ago(rx) Include types.h first
Hans Fugal [Fri, 10 Apr 2015 18:17:34 +0000 (11:17 -0700)]
(rx) Include types.h first

Summary: We need to do forward declarations first, and types.h already does declarations, so just make sure it gets included first in each file.

Test Plan:
tests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, exa, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1861040

Tasks: 6258585

Signature: t1:1861040:1428604059:de6393e3c87e0694a6b04ad6d440cc9635a1008d

9 years agoBump version to 35:0 v0.35.0
Viswanath Sivakumar [Fri, 10 Apr 2015 03:35:37 +0000 (20:35 -0700)]
Bump version to 35:0

9 years agoSharedMutex potential lost wakeup with exactly 3 or 4 contending writers
Nathan Bronson [Fri, 10 Apr 2015 02:45:58 +0000 (19:45 -0700)]
SharedMutex potential lost wakeup with exactly 3 or 4 contending writers

Summary:
SharedMutex used a saturating counter that records the number of
waiting lock() calls, but an ABA problem on futexWait could lead to a lost
wakeup when there was exactly 3 or 4 threads contending on the RW lock
in W mode.  This diff changes the kWaitingE count to be heuristic (it is
possible that the count says 1 but there are two waiters), saturates at
2 instead of 3 (because there is no benefit from differentiating those
two), and doesn't decrement the count on a successful wakeup.

Also, I noticed while debugging this that boost::noncopyable was causing
SharedMutex to be 8 bytes when it should only be 4.

One way the wakeup could be lost in the old code:

1. A calls lock()
2. A updates state <- kHasE
3. A returns
4. B calls lock()
5. B spins
6. B updates state <- kHasE + 1 * kIncrWaitingE
7. A calls unlock()
8. A updates state <- 0
9. A calls futexWake(), which returns 0
10. A calls lock()
11. A updates state <- kHasE
12. A returns
13. C calls lock()
14. C spins
15. C updates state <- kHasE + 1 * kIncrWaitingE
16. C calls futexWait, expecting kHasE + 1 * kIncrWaitingE
17. B calls futexWait, expecting kHasE + 1 * kIncrWaitingE
18. A calls unlock()
19. A updates state <- 0
20. A calls futexWake(), which returns 1
21. C receives the wakeup
22. C updates state <- kHasE
23. C returns
24. C calls unlock()
25. C updates state <- 0

B missed the wakeup that was intended for it (sent at step 9, wait
started at step 17), but went to sleep anyway because it saw the write
state at step 17. Now there are two waiters but only 1 recorded in the
SharedMutex, at which point failure is inevitable.

Test Plan:
1. DeterministicSchedule test using uniformSubset that can repro the problem
2. Test in production scenario that produced occasional deadlocks under high stress

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1980210

Tasks: 6720328

Signature: t1:1980210:1428623932:ef1c00c3f88154578b2b253ac0cfdbadf9f31d8c

9 years agoMove StringHandler to its own file
Dave Watson [Thu, 9 Apr 2015 16:59:36 +0000 (09:59 -0700)]
Move StringHandler to its own file

Summary: Move this from a test to its own file in codec folder

Test Plan:
already has a unittest
fbconfig -r folly/wangle; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959152

Signature: t1:1959152:1427935371:9d1338986df97fd94f98ad2bf08c731d229bc2b1

9 years agoLineBasedFrameDecoder
Dave Watson [Thu, 9 Apr 2015 17:00:48 +0000 (10:00 -0700)]
LineBasedFrameDecoder

Summary: Copy of netty's line based decoder.

Test Plan:
unittests
fbconfig folly/wangle/codec; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959155

Signature: t1:1959155:1427935150:e11280c5567df9ad9964dbb656aa090267856f57

9 years agoPipeline OutputBufferingHandler
Dave Watson [Wed, 8 Apr 2015 20:54:59 +0000 (13:54 -0700)]
Pipeline OutputBufferingHandler

Summary: Part 2 of pipeline.  Use OutputBufferingHandler, which is exactly the same output buffering thrift does currently.

Test Plan:
A couple canaries, unittests.
Specific issues of previous pipeline diffs I think are fixed already and should be fine with this one:
* Previously there were missing DestructorGuards.  I don't think anything in OutputBUfferHandler needs a DG
* previously broke http GET handling.  There is a unittest for it now.

Canary

Reviewed By: alandau@fb.com

Subscribers: doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, vloh, andreib

FB internal diff: D1970949

Signature: t1:1970949:1428360621:a0db142032894525f69c2b144cf946d63f790fe1

9 years agofolly: MemoryMapping: add offset/size params to advise
Lucian Grijincu [Wed, 8 Apr 2015 18:14:28 +0000 (11:14 -0700)]
folly: MemoryMapping: add offset/size params to advise

Test Plan: ran it

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1974936

Signature: t1:1974936:1428453023:1a83ed8336f75b745bffc633b8471231423b0fb0

9 years agofolly: MemoryMapping: respect writable when opening the file
Lucian Grijincu [Wed, 8 Apr 2015 18:14:17 +0000 (11:14 -0700)]
folly: MemoryMapping: respect writable when opening the file

Summary: the file needs to be opened with write-permissions before it can be mmaped with PROT_WRITE.

Test Plan: n/a

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1974895

Signature: t1:1974895:1428451664:1106019abf02b9de659ce0d305e3d788ae7d60bc

9 years agoAsyncSocketHandler cacheChainLength
Dave Watson [Wed, 8 Apr 2015 17:45:08 +0000 (10:45 -0700)]
AsyncSocketHandler cacheChainLength

Summary:
Is there any reason we shouldn't just cache the chain length everywhere?

Currently the pipeline does a lot of passing of buffers between IOBufQueues, I wonder if we can get even smarter about passing the length around

Test Plan: used in telnet server diff

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1959167

Signature: t1:1959167:1427934568:eb9a49514b2bb7d157879bb8efa99f12092d5e95

9 years agoRemove FiberManager move-constructor
Andrii Grynenko [Tue, 7 Apr 2015 18:19:46 +0000 (11:19 -0700)]
Remove FiberManager move-constructor

Summary:
LoopController and Fibers keep references to FiberManager, so implementing move-constructor for it is non-trivial.
The only purpose of move constructor was to have a create() static method, replaced it with a constructor accepting a type tag.

Test Plan: unit test + tao build

Reviewed By: stepan@fb.com, pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1971881

Signature: t1:1971881:1428374073:945d913e69eaa6f957dace981c23835105d91935

9 years agoAsyncUDPServerSocket passes socket in callback
Dave Watson [Tue, 7 Apr 2015 16:53:44 +0000 (09:53 -0700)]
AsyncUDPServerSocket passes socket in callback

Summary: AsyncUDPServerSocket doesn't make it easy to write to the same socket you read from.  Add the socket as a callback param, similar to AsyncServerSocket

Test Plan:
fbconfig -r folly; fbmake dbg

Will fixup any other spots contbuild finds

Reviewed By: hans@fb.com

Subscribers: bmatheny, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948936

Signature: t1:1948936:1427841651:20d13d73c06d31c75056624f051a6fd35b9701fb

9 years agoAdd ability to yield execution of the currently running fiber
Brian Watling [Tue, 7 Apr 2015 12:34:06 +0000 (05:34 -0700)]
Add ability to yield execution of the currently running fiber

Summary: This diff allows fibers to explicitly yield execution to other fibers and the event loop

Test Plan: unit tests

Reviewed By: andrii@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@

FB internal diff: D1965297

Signature: t1:1965297:1428082686:e524e9dd21b3fb951e1d3556e4cb3eedc3e6511a

9 years agoFix open-source Makefiles after fibers were moved to folly
Andrii Grynenko [Tue, 7 Apr 2015 02:32:01 +0000 (19:32 -0700)]
Fix open-source Makefiles after fibers were moved to folly

Test Plan: contbuild

Reviewed By: pavlo@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1966698

Signature: t1:1966698:1428367351:62f2218c23d4911c15055917c0fbae31a2420903

9 years agoGlobalExecutor.h include Executor.h and IOExecutor.h
Hans Fugal [Tue, 7 Apr 2015 01:20:00 +0000 (18:20 -0700)]
GlobalExecutor.h include Executor.h and IOExecutor.h

Summary: We were forward-declaring `Executor` and `IOExecutor` for great compilation speed. However, this leaves users of this header to include at least one of those headers in their cpp file to actually use it, and that's kind of annoying. `Executor` and `IOExecutor` are not templated and are quite small, so it seems like the compilation speed benefit is minimal compared to the convenience factor, so this diff just includes the two headers. (Probably including `Executor.h` is redundant since `IOExecutor.h` probably includes it, but for completeness and explicitness, and in case `IOExecutor` also just forward-declares `Executor` now or in the future.)

Test Plan: stuff still builds

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1970179

Signature: t1:1970179:1428351076:e9c3700a868688035e7ff50486b8cead751d3c02

9 years agoAdd FiberManager local type
Andrii Grynenko [Mon, 6 Apr 2015 19:06:58 +0000 (12:06 -0700)]
Add FiberManager local type

Summary: This adds local type for each FiberManager. Only local of given type can be created on fibers of this FiberManager. Locals of other types will be just treated as regualar thread-locals.

Test Plan: unit test

Reviewed By: bwatling@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D1969739

Signature: t1:1969739:1428345931:aff5deb526c179158e5881b29330ff3f6698149a

9 years agoOption to record precise stack on every N fiber
Pavlo Kushnir [Mon, 6 Apr 2015 02:51:13 +0000 (19:51 -0700)]
Option to record precise stack on every N fiber

Summary: title + some logging around

Test Plan: mcrouter unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1965097

Signature: t1:1965097:1428103637:341ff705c9d11c730197828256ec0b790ee7afaa

9 years agoshared ptr vector sockets
Dave Watson [Fri, 3 Apr 2015 20:07:13 +0000 (13:07 -0700)]
shared ptr vector sockets

Summary: promote the sockets vector to a shared_ptr, since both ServerWorkerPool and ServerBootstrap use it.  Otherwise there are destruction order issues between ServerBootstrap and any IOThreadPoolExecutor you use

Test Plan: Saw use after free in D1942242, gone after this.

Reviewed By: yfeldblum@fb.com

Subscribers: chalfant, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1947553

Signature: t1:1947553:1427484417:5b78f5c9c70d244d3f52a6f71b6d1fab7b29d106

9 years agoUse Fiber locals for TAO Fiber-locals
Andrii Grynenko [Fri, 3 Apr 2015 00:29:35 +0000 (17:29 -0700)]
Use Fiber locals for TAO Fiber-locals

Summary:
This simplifies TAO fibers locals, using FiberManager based fiber-local storage.
This depends on D1958135.

Test Plan: integration tests

Reviewed By: bwatling@fb.com

Subscribers: alikhtarov

FB internal diff: D1962214

Signature: t1:1962214:1427997755:b546b9039382a7600f234b0a2a60cc96da34e662

9 years agoFiber-local context
Andrii Grynenko [Fri, 3 Apr 2015 00:17:46 +0000 (17:17 -0700)]
Fiber-local context

Summary: This adds fiber-local context, which behaves more like static objects+fork rather than thread-locals.

Test Plan: unit test

Reviewed By: pavlo@fb.com, bwatling@fb.com

Subscribers: rushix, alikhtarov, bwatling

FB internal diff: D1958135

Signature: t1:1958135:1427999426:8e4b89f4af53a1a119b2e5a765fb549dd8442c50

9 years agoAdd static method to skip SSL init
Joel Goguen [Thu, 2 Apr 2015 23:37:50 +0000 (16:37 -0700)]
Add static method to skip SSL init

Summary:
Some things like gSOAP, initialize SSL only to have SSLContext stomp over their
work. Add a method to allow flagging SSLContext as "initialized" to use the
existing global state.

Test Plan:
Tested running mcrouter and paymentnegined to verify existing
functionality appears normal.

Reviewed By: subodh@fb.com

Subscribers: ssl-diffs@, folly-diffs@, yfeldblum, chalfant, dihde

FB internal diff: D1959936

Tasks: 6655251

Signature: t1:1959936:1427994292:8b2be1742ef17a5ec2c7b3ad37787d2cf4d142a6

9 years agoBump version to 34:0 v0.34.0
afrind [Thu, 2 Apr 2015 20:49:51 +0000 (13:49 -0700)]
Bump version to 34:0

9 years agoUpdate Makefile.am for AsyncSignalHandler
Alan Frindell [Thu, 2 Apr 2015 20:36:51 +0000 (13:36 -0700)]
Update Makefile.am for AsyncSignalHandler

Summary: facepalm

Test Plan: Built proxygen OSS that depends on this symbol

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1963679

Signature: t1:1963679:1428006035:2c4ceffc046b995431cde2fbcef757fa7ea1ea0b

9 years agoMove mcrouter/lib/fibers to folly/experimental/fibers
Rushi Desai [Thu, 2 Apr 2015 20:09:03 +0000 (13:09 -0700)]
Move mcrouter/lib/fibers to folly/experimental/fibers

Summary:
The mcrouter fibers library is quite general purpose and reusable by other projects. Hence I'm moving it into folly.

Test Plan:
fbconfig -r folly/experimental/fibers && fbmake runtests
fbconfig -r mcrouter && fbmake runtests
fbconfig -r tao && fbmake
fbconfig -r ti && fbmake

Reviewed By: andrii@fb.com

Subscribers: vikas, zhuohuang, jmkaldor, jhunt, pavlo, int, aap, trunkagent, fredemmott, alikhtarov, folly-diffs@, jsedgwick, yfeldblum, chalfant, chip

FB internal diff: D1958061

Signature: t1:1958061:1428005194:b57bfecfe9678e81c48526f57e6197270e2b5a27

9 years agoBump version to 33:0 v0.33.0
afrind [Thu, 2 Apr 2015 19:03:15 +0000 (12:03 -0700)]
Bump version to 33:0

9 years agoUDP bind via port or address
Dave Watson [Thu, 2 Apr 2015 18:48:01 +0000 (11:48 -0700)]
UDP bind via port or address

Summary: Bind via port or address, similar to tcp

Test Plan: I wrote a unittest that tested v4 vs v6 binding - can't check it in because we have v6 only clusters though.  Not sure how eles to write a unittest without picking specific ports

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948938

Signature: t1:1948938:1427841721:621031320981bf366a8252a0ea1d4491b694ea24

9 years agoMove TAsyncSignalHandler into folly
Alan Frindell [Thu, 2 Apr 2015 17:43:38 +0000 (10:43 -0700)]
Move TAsyncSignalHandler into folly

Summary:
TODO item, trying to remove unecessary dependencies on thrift

Test Plan: Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, fbcode-common-diffs@, davejwatson, andrewcox, alandau, bmatheny, anca, darshan, mshneer, folly-diffs@, bil, yfeldblum, haijunz, chalfant

FB internal diff: D1960215

Signature: t1:1960215:1427920934:8abd7e94c50676b05bf7ff79800df0db1bd04266

9 years agoAdd socket mocks
Alan Frindell [Thu, 2 Apr 2015 17:26:53 +0000 (10:26 -0700)]
Add socket mocks

Summary: These parallel the MockTAsync* from thrift/lib/cpp/test.  Once all the users of the thrift versions are gone they can be deleted

Test Plan: Unit tests

Reviewed By: yfeldblum@fb.com, alandau@fb.com

Subscribers: doug, net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1959979

Signature: t1:1959979:1427917909:19af219f88dd6847a064da986dd30765e29bdc99

9 years agoMove AsyncSocket tests from thrift to folly
Alan Frindell [Thu, 2 Apr 2015 17:20:49 +0000 (10:20 -0700)]
Move AsyncSocket tests from thrift to folly

Summary: These tests belong with the code that they test.  The old tests had a couple dependencies on TSocket/TSSLSocket, so I wrote a BlockingSocket wrapper for AsyncSocket/AsyncSSLSocket

Test Plan: Ran the tests

Reviewed By: alandau@fb.com

Subscribers: doug, net-systems@, alandau, bmatheny, mshneer, folly-diffs@, yfeldblum, chalfant

FB internal diff: D1959955

Signature: t1:1959955:1427917833:73d334846cf248f8bb215f3eb5b596df7f7cee4f

9 years agoMove AtomicLinkedList to folly
Rushi Desai [Thu, 2 Apr 2015 04:51:08 +0000 (21:51 -0700)]
Move AtomicLinkedList to folly

Summary:
Adding new atomic data structure from the mcrouter project.

Test Plan: fbconfig -r mcrouter && fbmake runtests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant, achao, jpearce

FB internal diff: D1960219

Signature: t1:1960219:1427921204:78e5313ea916b8a249d32e31cc7ba1aa9e890d95