folly.git
9 years agoConditionals for iOS / Android compilation
Sean Cannella [Thu, 16 Oct 2014 17:09:06 +0000 (10:09 -0700)]
Conditionals for iOS / Android compilation

Summary:
Conditional changes required to compile on iOS and Android.

Test Plan: compiled on all platforms

Reviewed By: meyering@fb.com

Subscribers: kmccray, trunkagent, shilin, njormrod, ranjeeth, subodh, bmatheny

FB internal diff: D1618028

Tasks: 5183325

9 years agoDon't throw in Singleton::get() if singleton is alive
Andrii Grynenko [Thu, 16 Oct 2014 00:40:15 +0000 (17:40 -0700)]
Don't throw in Singleton::get() if singleton is alive

Summary:
This is a fix for situations where you know that something is keeping singleton alive (by getting a weak_ptr and locking it), and request a singleton instance via get() method (if e.g. it's hard to pass a pointer to singleton instance from a method which locked it). If shutdown was scheduled - an exception was previously thrown even though the object was not destroyed yet.

This simplifies conversion of legacy code to folly::Singleton.

Test Plan:
fbconfig -r gatehouse/usersets/tests
fbmake runtests

Reviewed By: chip@fb.com

Subscribers: trunkagent, njormrod

FB internal diff: D1619311

9 years agoRemove global init of ThreadLocal in Random
Subodh Iyengar [Wed, 15 Oct 2014 23:51:46 +0000 (16:51 -0700)]
Remove global init of ThreadLocal in Random

Summary:
Remove global initialization of ThreadLocal in Random
See https://gcc.gnu.org/onlinedocs/gcc-4.8.3/gcc/Thread-Local.html

Static initializers and thread locals don't mix well, and we're seeing
some crashes which we think might be related to this.

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: trunkagent, seanc, njormrod

FB internal diff: D1617455

9 years agosupport for move-only types
James Sedgwick [Wed, 15 Oct 2014 22:58:44 +0000 (15:58 -0700)]
support for move-only types

Summary: universal references and perfect forwarding to the rescue

Test Plan: added demonstrative unit tests

Reviewed By: hans@fb.com

Subscribers: trunkagent, fugalh, njormrod, bmatheny

FB internal diff: D1592032

Tasks: 5283342

9 years agomerge wangle/Executor.h and experimental/wangle/concurrent/Executor.h
James Sedgwick [Wed, 15 Oct 2014 22:57:47 +0000 (15:57 -0700)]
merge wangle/Executor.h and experimental/wangle/concurrent/Executor.h

Summary:
the one in concurrent/ is a bit more generic, so I kept that as Executor and renamed the existing one ScheduledExecutor

because Hans is surfing I took the liberty of renaming Action->Func as an alias for std::function<void()>, because I think it's more reflective
also kept the version of add() that doesn't force rvalue-reference as it's more user friendly and probably not less performant in common cases (insert reference to "want speed? pass by value" here)

Test Plan: compiled some major relevant bits, will let contbuild show me anything I missed

Reviewed By: hans@fb.com

Subscribers: trunkagent, rushix, fbcode-common-diffs@, fugalh, msk, njormrod

FB internal diff: D1591237

Tasks: 5279196

9 years agoSchedule destroyInstances only when first singleton is created
Andrii Grynenko [Tue, 14 Oct 2014 03:08:15 +0000 (20:08 -0700)]
Schedule destroyInstances only when first singleton is created

Summary:
Right now destroyInstances is scheduled when SingletonVault is requested. This may change singleton destruction order (folly::Singleton-managed vs unmanaged singletons) when new singleton is registered with folly::Singleton (no matter if it is even used).
This diff changes it to be more stable.

Test Plan: servicerouter unittests

Reviewed By: chip@fb.com

Subscribers: njormrod, mshneer

FB internal diff: D1615213

Tasks: 5353022

9 years agoFix clang test compilation failures
Sean Cannella [Wed, 15 Oct 2014 21:30:46 +0000 (14:30 -0700)]
Fix clang test compilation failures

Summary:
While working on other code I noticed folly stopped compiling
with clang. Fix that.

Test Plan: fbmake runtests (clang and gcc)

Reviewed By: lucian@fb.com

Subscribers: mathieubaudet, njormrod, bmatheny

FB internal diff: D1618189

9 years ago(wangle) Use MicroSpinLock
Hans Fugal [Wed, 15 Oct 2014 21:05:59 +0000 (14:05 -0700)]
(wangle) Use MicroSpinLock

Summary: For great speed

Test Plan:
Before
============================================================================
folly/wangle/test/Benchmark.cpp                 relative  time/iter  iters/s
============================================================================
constantFuture                                             241.19ns    4.15M
promiseAndFuture                                 100.32%   240.42ns    4.16M
withThen                                          44.63%   540.47ns    1.85M
----------------------------------------------------------------------------
oneThen                                                    519.20ns    1.93M
twoThens                                          62.83%   826.41ns    1.21M
fourThens                                         36.80%     1.41us  708.75K
hundredThens                                       1.79%    29.05us   34.42K
----------------------------------------------------------------------------
no_contention                                                4.82ms   207.27
contention                                        62.91%     7.67ms   130.39
============================================================================

After
============================================================================
folly/wangle/test/Benchmark.cpp                 relative  time/iter  iters/s
============================================================================
constantFuture                                             159.79ns    6.26M
promiseAndFuture                                 101.23%   157.84ns    6.34M
withThen                                          41.78%   382.47ns    2.61M
----------------------------------------------------------------------------
oneThen                                                    358.23ns    2.79M
twoThens                                          63.07%   568.00ns    1.76M
fourThens                                         36.89%   971.07ns    1.03M
hundredThens                                       1.76%    20.34us   49.17K
----------------------------------------------------------------------------
no_contention                                                3.75ms   266.75
contention                                        59.83%     6.27ms   159.59
============================================================================

That's a 150% speedup.

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, fugalh, exa, njormrod

FB internal diff: D1617363

Tasks: 5278220

9 years agoDefer query string parsing from ctor to getQueryParams()
Jun LI [Wed, 15 Oct 2014 20:53:51 +0000 (13:53 -0700)]
Defer query string parsing from ctor to getQueryParams()

Summary:
Query string parsing uses a lot of CPU, it happens in ctor killing CPU
even in users that are not interested in query string. Move it to
getQueryParams().

Test Plan:
fbconfig folly/test
fbmake runtests_dbg

Reviewed By: ldemailly@fb.com

Subscribers: trunkagent, njormrod, zellux

FB internal diff: D1604973

Tasks: 5304484

Blame Revision: https://phabricator.fb.com/D1455158

9 years agoFix opensource build
Pavlo Kushnir [Wed, 15 Oct 2014 18:46:22 +0000 (11:46 -0700)]
Fix opensource build

Summary: D1615707 moved State.h to Core.h, but didn't update Makefile.am

Test Plan: visual

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1617562

Blame Revision: D1615707

9 years agoFix thread local random number generator static issue
Daniel Sommermann [Tue, 14 Oct 2014 23:39:00 +0000 (16:39 -0700)]
Fix thread local random number generator static issue

Summary:
See https://gcc.gnu.org/onlinedocs/gcc-4.8.3/gcc/Thread-Local.html
Thanks to @subodh for finding the first instance of this problem in the
proxygen codebase.

Test Plan:
unit tests, tested proxygen unit tests on my laptop and I don't
get segfaults on program exit now.

Reviewed By: subodh@fb.com

Subscribers: trunkagent, doug, njormrod, subodh

FB internal diff: D1616149

Tasks: 5217022

9 years agowangle::detail::FSM
Hans Fugal [Wed, 15 Oct 2014 16:32:03 +0000 (09:32 -0700)]
wangle::detail::FSM

Summary:
spaghetti-monster

Finagle uses a nifty state machine pattern in `Promise.scala`. Each state carries its data with it, then you switch statements with a simple state `cas`, within the functions (which are the transition inputs). See https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Promise.scala#L133-L139 and https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Promise.scala#L604-L635 for example.

I was thinking we could do something similar in C++, though we don't quite have the same luxury of type cases like scala. (Although as an aside I found this really cool paper on implementing type cases in C++, it's kind of evil but very cool. http://www.stroustrup.com/OOPSLA-typeswitch-draft.pdf)

I was looking at having a union of the different state classes, e.g.

union U {
State base;
Done done;
Waiting waiting;
...
};

and then you could use the memoized `dynamic_cast` trick in that paper to make a fairly efficient type case (and fairly readable, depending on how evil you want to get with the macros). However, `dynamic_cast<Done*>(&u.base)` blissfully segfaults. I'm not sure if this is something that should work and it's a compiler bug, or whether trying to (ab)use a union this way is against some arcane C++ rules. @hannesr suggested maybe a variant type might work. We can also do memory mangling on our own if it comes to it - however those are all more advanced techniques to play with at a later optimization time.

So instead, I went for a this-is-context helper base class. The mutable context is just `this`, and you inherit from `FSM`.

The magic macros make it more succint to lay out state transitions. See the tests for examples. Maybe in the future we can get really clever and find a way to generate state machine diagrams from code using this, especially when macros are being used.

Test Plan: unit tests were written and pass

Reviewed By: davejwatson@fb.com

Subscribers: meisner, trunkagent, net-systems@, fugalh, exa, njormrod, hannesr

FB internal diff: D1613497

9 years ago(wangle) s/State/Core/
Hans Fugal [Wed, 15 Oct 2014 16:31:52 +0000 (09:31 -0700)]
(wangle) s/State/Core/

Summary:
codemod
`State` is such an overloaded term, and not really the best to describe this backing future/promise object. Yes, it holds the state but it's more than that and it gets in the way of calling the states of the state machines `State`s.

Test Plan: builds and tests pass

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod

FB internal diff: D1615707

9 years agoBump version to 12:0
Andrii Grynenko [Wed, 15 Oct 2014 01:00:43 +0000 (18:00 -0700)]
Bump version to 12:0

9 years agoDon't re-create singletons
Andrii Grynenko [Wed, 8 Oct 2014 01:12:33 +0000 (18:12 -0700)]
Don't re-create singletons

Summary:
This forbids singleton creation/re-creation after destroyInstances() was called.
Only after reset() is called singletons can be created again.

registrationComplete() behavior is also slightly change. We disallow singleton registration after registrationComplete() is called even in Relaxed mode. Strict mode now only controlls whether singletons can be constructed before registation is complete.

Test Plan: unit test

Reviewed By: chip@fb.com

Subscribers: hphp-diffs@, ps, trunkagent, njormrod, mshneer, lins

FB internal diff: D1605136

9 years agoBasic tests for EventHandler.
Yedidya Feldblum [Fri, 10 Oct 2014 19:44:26 +0000 (12:44 -0700)]
Basic tests for EventHandler.

Summary:
[Folly] Basic tests for EventHandler.

Worthwhile adding some tests just for this class since it's rather fundamental. It is already tested indirectly via any number of things that make use of EventBase under the hood, but it's worth having a place for more direct tests.

Test Plan: Unit tests in `folly/io/async/test/EventHandlerTest.cpp`.

Reviewed By: njormrod@fb.com

Subscribers: njormrod, chalfant, dougw

FB internal diff: D1606284

9 years agoBetter estimateSpaceNeeded(double)
Soren Lassen [Fri, 10 Oct 2014 14:12:31 +0000 (07:12 -0700)]
Better estimateSpaceNeeded(double)

Test Plan: fbconfig -r folly/test && fbmake runtests

Reviewed By: mpawlowski@fb.com

Subscribers: njormrod

FB internal diff: D1604761

Blame Revision: D1420588

9 years agoGo back to a regular mutex
Hans Fugal [Thu, 9 Oct 2014 23:27:12 +0000 (16:27 -0700)]
Go back to a regular mutex

Summary:
D1428504 changed the `detail::State` mutex to be a recursive mutex to get around deadlock in our project, where a promise was being freed within a callback, racing with the mechanics happening while fulfilling the promise in the first place. At least, that's what seemed to be happening. I couldn't easily pull it into gdb because it's a python test.

I made my own test to repro, and it did in fact deadlock:

TEST(Future, DISABLED_promiseDestructedDuringCallback) {
auto p = folly::make_unique<Promise<void>>();
p->getFuture().then([&](Try<void>&&) {
// shouldn't deadlock.
p.reset();
});
p->setValue();
}

However, although this code fixes our project it does not fix this code, which still fails (not with a deadlock, but with a promiseAlreadyFulfilled exception). So whatever our project is doing is a bit more intricate.
I'm not convinced that even allowing this is ok - so I suspect out project is doing something bad. However, I also think it's probably bad to hold the lock while doing the callback so I am presenting this as a fix/compromise.

Test Plan:
unit tests

Reviewed By: hannesr@fb.com

Subscribers: net-systems@, fugalh, exa, njormrod, fsilva, davejwatson

FB internal diff: D1604829

Blame Revision: D1428504

9 years agoPort compilation fixes (1/3)
Sean Cannella [Thu, 9 Oct 2014 19:40:37 +0000 (12:40 -0700)]
Port compilation fixes (1/3)

Test Plan: existing tests

Reviewed By: meyering@fb.com

Subscribers: trunkagent, bmatheny, ranjeeth, njormrod, subodh

FB internal diff: D1605942

Tasks: 5183325

9 years agoMove static global variable randomDevice inside the scope of the function
Francis Ma [Thu, 9 Oct 2014 15:35:06 +0000 (08:35 -0700)]
Move static global variable randomDevice inside the scope of the function

Test Plan: Tests under folly passed

Reviewed By: seanc@fb.com

Subscribers: trunkagent, seanc, njormrod

FB internal diff: D1600266

Tasks: 5317470

9 years agoadd per node mutex for emulated futex
Nathan Bronson [Thu, 9 Oct 2014 02:44:16 +0000 (19:44 -0700)]
add per node mutex for emulated futex

Summary:
The emulated futex (not used on Linux) has an optimization to defer
notification until after the bucket mutex has been unlocked, to avoid
lock collisions between the waiter and waker.  That code will have a
use-after-free problem if the waiter's condition_variable has a spurious
wakeup, which is allowed by the spec.  That code also doesn't do
anything about contention between multiple waiters.

This diff adds a mutex to each wait node, relieving the waiters from
having to acquire the bucket lock on wakeup.  Rather than trying to
perform a racy late notification, we just make sure to release the node
lock immediately after calling notify_one, which seems to have the
desired effect.

Test Plan:
1) existing unit tests
2) new unit tests

Reviewed By: delong.j@fb.com

Subscribers: boseant, njormrod

FB internal diff: D1602360

9 years agomove TEventBaseManager to folly/io/async/EventBaseManager
James Sedgwick [Thu, 9 Oct 2014 00:15:17 +0000 (17:15 -0700)]
move TEventBaseManager to folly/io/async/EventBaseManager

Summary:
This class isn't thrift specific anymore, especially now that TEventBase->EventBase.
Specific use case in folly: folly/experimental/wangle/concurrent/IOThreadPoolExecutor
EventBaseManager itself needs some work/cleanup, but that is for a later diff
For instance, we might try to push towards only allowing access to the singleton, and towards
removing overrides. i.e. only getEventBase. But that's pending an audit of how people are using it now.

Note that the ProfiledMutex protecting the event base set has been made a regular std::mutex

Test Plan:
compiled thrift/lib/cpp|cpp2, made a pass at fixing forward declarations elsewhere
will let contbuild help me iterate

Reviewed By: hans@fb.com

Subscribers: trunkagent, nli, fbcode-common-diffs@, davejwatson, hero-diffs@, zeus-diffs@, andrewcox, netego-diffs@, alandau, apollo-diffs@, antonl, laser-diffs@, ads-dsp-eng@, darshan, micha, njormrod, panin, hdoshi, scuba-diffs@, bmatheny

FB internal diff: D1590827

Tasks: 5247981

9 years agofix flaky test from uninitialized std::atomic
Nathan Bronson [Wed, 8 Oct 2014 23:59:27 +0000 (16:59 -0700)]
fix flaky test from uninitialized std::atomic

Summary:
std::atomic<bool> default constructor doesn't initialize it to
false, so the flag to end the LifoSem.multi_try_wait test was sometimes
set too early.  This diff fixes that, makes the test more deterministic,
and also fixes a couple of other benign uninitialized values (so that
they won't be used as prototypes for places where it does matter).

Test Plan: unit tests

Reviewed By: mpawlowski@fb.com

Subscribers: njormrod, lovro

FB internal diff: D1604508

Tasks: 5336837

9 years agoFix via
Hans Fugal [Wed, 8 Oct 2014 19:06:38 +0000 (12:06 -0700)]
Fix via

Summary: Sometimes you just have to take a step back. :-P

Test Plan: All the unit tests including the one that had been disabled, now pass.

Reviewed By: hannesr@fb.com

Subscribers: meisner, trunkagent, net-systems@, fugalh, exa, njormrod, davejwatson

FB internal diff: D1596368

Tasks: 492068944805675306911

9 years agostop prereserving space in toAppend
Marcin Pawlowski [Wed, 8 Oct 2014 19:00:18 +0000 (12:00 -0700)]
stop prereserving space in toAppend

Summary:
previous change would cause reserving
as much as needed in toAppend for all arguements.
This had bad consequences for appending in a loop,
described here: https://phabricator.fb.com/D1420588#22
Not split the interfaces so that user has to decide

Test Plan: unit tests

Reviewed By: soren@fb.com

Subscribers: trunkagent, njormrod

FB internal diff: D1601614

Tasks: 5303991

9 years agowangle-bench gflags
Hans Fugal [Tue, 7 Oct 2014 22:50:32 +0000 (15:50 -0700)]
wangle-bench gflags

Summary: Because windtunnel quirkloads needs `--json` which is enabled with gflags

Test Plan:
$ wangle-bench --json
{
"%hundredThens" : 28133155.113220215,
"no_contention" : 4842268655.11322,
"%fourThens" : 1495655.1132202148,
"%twoThens" : 882311.3632202148,
"oneThen" : 581053.5507202148,
"-" : 0,
"%withThen" : 559830.8944702148,
"%promiseAndFuture" : 250840.66009521484,
"%contention" : 8074419655.11322,
"constantFuture" : 239916.83197021484
}

Reviewed By: meisner@fb.com

Subscribers: robbert, net-systems@, fugalh, exa, njormrod, davejwatson, jsedgwick

FB internal diff: D1601364

Tasks: 5277907

9 years agoserial/parallel benchmark
Hans Fugal [Tue, 7 Oct 2014 23:36:06 +0000 (16:36 -0700)]
serial/parallel benchmark

Summary: This is an attempt to see the effect of lock contention in the less-common case where a promise is fulfilled at the same time the future is still being set up. Although this is expected to be off the beaten path, it will still be a good idea to keep an eye on how we improve or worsen perf, when we play with different locking strategies (which I intend to do a lot of in the near term).

Test Plan:
============================================================================
folly/wangle/test/Benchmark.cpp                 relative  time/iter  iters/s
============================================================================
constantFuture                                             249.31ns    4.01M
promiseAndFuture                                 100.88%   247.13ns    4.05M
withThen                                          43.87%   568.30ns    1.76M
----------------------------------------------------------------------------
oneThen                                                    569.62ns    1.76M
twoThens                                          63.46%   897.60ns    1.11M
fourThens                                         39.64%     1.44us  695.90K
hundredThens                                       1.91%    29.75us   33.61K
----------------------------------------------------------------------------
serial                                                       4.97ms   201.21
parallel                                          60.22%     8.25ms   121.18
============================================================================

Reviewed By: jsedgwick@fb.com

Subscribers: meisner, net-systems@, fugalh, exa, njormrod

FB internal diff: D1593010

9 years agoadd emulation of futex() for non-Linux systems
Nathan Bronson [Wed, 1 Oct 2014 22:46:20 +0000 (15:46 -0700)]
add emulation of futex() for non-Linux systems

Summary:
Inside the Linux kernel, futex() works by hashing the source
address to a fixed set of wakeup lists.  When using folly on non-Linux
systems we can emulate something similar by using std::mutex and
std::condition_variable.

Emulating futex() using higher-level APIs is less crazy than it sounds,
because the emulated futex still provides the advantages of the real
one: it allows all of the fast-paths of a new synchronization construct
to be inlined; it is space efficient, taking only 1 word and allowing
the caller to encode state into all of the word's bits; and it avoids
system calls unless a thread actually needs to be put to sleep or
woken up.  Think of this as a way of boostrapping something with the
same properties as futex() on platforms that don't expose it publically.
(Presumably these platforms have private APIs that do something similar.)

This diff moves all of the Linux-specific futex stuff into Futex.cpp,
where it is gated by #ifdef __linux__.  It also adds an emulated
implementation.  The emulated futex will be selected by default on
non-Linux platforms, or it can be used on Linux by using an Atom template
type of folly::detail::EmulatedFutexAtomic.  This means, for example,
that you can test MPMCQueue on top of the emulated API by instantiating
a MPMCQueue<ElemType, EmulatedFutexAtomic>.

As a bonus, this refactoring provides a small speed boost by removing
an unnecessary evaluation of the errno macro in the success path of
futexWait.

Test Plan:
1. existing unit tests
2. new unit tests (including tests of Futex users)
3. compile Futex.cpp on OS X (some other build failures still occur)

Reviewed By: delong.j@fb.com

Subscribers: trunkagent, njormrod, yiding, boseant, mssarang

FB internal diff: D1596118

Tasks: 4952724

9 years agoMove static member inside the scope of the function
Francis Ma [Mon, 6 Oct 2014 23:34:06 +0000 (16:34 -0700)]
Move static member inside the scope of the function

Summary: We are seeing crashes which comes from the initialization of the static global variable. This particular variable is used only in a single function that was never invoked. So moving it into the scope of the function will at least solve the problem. The real issue still requires some deep investigation though.

Test Plan: unitest under folly passed

Reviewed By: subodh@fb.com

Subscribers: seanc, njormrod

FB internal diff: D1598048

Tasks: 5316441

9 years agodetail::MemoryIdler should use Malloc.h's mallctl decl
Nathan Bronson [Mon, 6 Oct 2014 04:04:53 +0000 (21:04 -0700)]
detail::MemoryIdler should use Malloc.h's mallctl decl

Summary:
Malloc.h now takes care to declare mallctl properly on platforms
that aren't using weak symbols, so we should just rely on that for
MemoryIdler.cpp.  This is one step toward fixing folly build on OS X.

Test Plan: fbmake runtests

Reviewed By: je@fb.com

Subscribers: yiding, njormrod

FB internal diff: D1596930

Tasks: 4952724

Blame Revision:

9 years agoLeak folly::SingletonVault
Andrii Grynenko [Thu, 2 Oct 2014 03:34:27 +0000 (20:34 -0700)]
Leak folly::SingletonVault

Summary:
This will ensure SingletonVault is always available. It matters for singletons, not managed by folly::Singleton. Singletons in it will actually be destroyed via static SingletonVaultDestructor.

This does the same as D1591270 (which got reverted), but doesn't change destruction order.

Test Plan:
Run tests which were broken by D1591270

Reviewed By: chip@fb.com

Subscribers: njormrod

FB internal diff: D1594898

9 years agoFix folly::Singleton to work with objects w/o default constructor
Andrii Grynenko [Wed, 17 Sep 2014 04:12:10 +0000 (21:12 -0700)]
Fix folly::Singleton to work with objects w/o default constructor

Test Plan:
unit tests

Reviewed By: ostap@fb.com

Subscribers: netego-diffs@, trunkagent, alikhtarov, marccelani, mshneer, zhuohuang, lins, njormrod

FB internal diff: D1560406

9 years agoAdd extra assertions in ExternalUnixAddr::free()
Anton Likhtarov [Thu, 2 Oct 2014 20:23:20 +0000 (13:23 -0700)]
Add extra assertions in ExternalUnixAddr::free()

Summary: Memory corruption investigation

Test Plan: fbconfig folly/test:network_address_test; fbmake runtests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, andrii, njormrod

FB internal diff: D1592539

Tasks: 5230657

Blame Revision: D1575098

9 years agoRevert "Leak folly::SingletonVault"
Daniel Sommermann [Thu, 2 Oct 2014 22:19:27 +0000 (15:19 -0700)]
Revert "Leak folly::SingletonVault"

Summary:
This reverts commit 3cf88fb680b4b9c47189d1e12699e2c24c7ac38b. It
was timing out our tests. probably static (de)initialization order fiasco related.

Test Plan: watch contbuild, git bisect

Reviewed By: afrind@fb.com

Subscribers: doug, njormrod

FB internal diff: D1593016

9 years agoLeak folly::SingletonVault
Andrii Grynenko [Thu, 2 Oct 2014 03:34:27 +0000 (20:34 -0700)]
Leak folly::SingletonVault

Summary: This will ensure SingletonVault is always available. It matters for singletons, not managed by folly::Singleton. Singletons in it will actually be destroyed via static SingletonVaultDestructor.

Test Plan: unit test

Reviewed By: chip@fb.com

Subscribers: njormrod

FB internal diff: D1591270

9 years agoadd some benchmarks
Hans Fugal [Thu, 2 Oct 2014 17:39:53 +0000 (10:39 -0700)]
add some benchmarks

Summary: For great speed

Test Plan:
$ fbmake opt && _bin/folly/wangle/wangle-bench
============================================================================
folly/wangle/test/Benchmark.cpp                 relative  time/iter  iters/s
============================================================================
constantFuture                                             235.92ns    4.24M
promiseAndFuture                                 100.37%   235.04ns    4.25M
withThen                                          41.97%   562.17ns    1.78M
----------------------------------------------------------------------------
oneThen                                                    539.03ns    1.86M
twoThens                                          64.01%   842.14ns    1.19M
fourThens                                         39.27%     1.37us  728.62K
hundredThens                                       1.82%    29.63us   33.75K
============================================================================

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod

FB internal diff: D1587871

9 years agosubscriptions
James Sedgwick [Thu, 2 Oct 2014 03:49:37 +0000 (20:49 -0700)]
subscriptions

Summary:
I'm not thrilled with this implementation, but it mostly works. The big bummer is enforcing that Observables are held in shared_ptrs, or rather that enforcing that condition is impossible.
The protected constructor / friended dumb make_shared() pattern is clunky, and it'd be really easy for a subclasser to shoot themselves in the foot or even in the face.

It does seem like maybe Observable should be made an interface again, and all these details should live in a subclass (FanoutObservable?) where the restriction are super obvious. For instance, the langtech AudioStream object doesn't need all this crap because it overrides subscribe() without using the observer list, but it subclasses anyways.

I'm noodling another approach that (if it works) will not require the shared_ptr dancing, but will probably have some additional overhead... the observable would have to keep track of the subscriptions itself.

I like the RAII subscriptions, though perhaps subscriptions should be optional as long as it's clear that your subscription will last forever it you opt out of them.

Thoughts?

Test Plan: added unit

Reviewed By: hans@fb.com

Subscribers: rushix, hannesr, trunkagent, fugalh, mwa, jgehring, fuegen, njormrod, bmatheny

FB internal diff: D1580443

9 years agoadd stringVPrintf() and stringVAppendf()
Adam Simpkins [Sun, 28 Sep 2014 01:08:34 +0000 (18:08 -0700)]
add stringVPrintf() and stringVAppendf()

Summary:
This adds versions of stringPrintf() and stringAppendf() that accept
va_list arguments.

This also fixes the existing stringPrintf() functions to properly call
va_end() even when an exception is thrown in stringPrintfImpl().

Test Plan: Included new unit tests for stringVPrintf() and stringVAppendf().

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, njormrod

FB internal diff: D1583675

9 years agoin thread pools, take factory as shared ptr
James Sedgwick [Wed, 1 Oct 2014 17:55:34 +0000 (10:55 -0700)]
in thread pools, take factory as shared ptr

Summary: needed for thrift server. if you have some sort of stateful/functional factory and you want to hold on to it, you should be allowed to

Test Plan: compiles with forthcoming thrift diff

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod

FB internal diff: D1584587

9 years agoexpose event base from IOThreadPoolExecutor
James Sedgwick [Wed, 1 Oct 2014 17:32:54 +0000 (10:32 -0700)]
expose event base from IOThreadPoolExecutor

Summary:
I'm not 100% sure this is the best way to go about this but I don't hate it either.
I'm going to start seeing how it might fit into tserver - my guess is that some sort Cpp2WorkerFactory which also manages those objects would get plugged in as the thread factory
Haven't fleshed out how this would relate to TEventBaseManager

Test Plan: added unit, starting to play with this in Thrift2 server

Reviewed By: davejwatson@fb.com

Subscribers: alandau, bmatheny, trunkagent, fugalh, njormrod

FB internal diff: D1574660

9 years agoBump version to 11:0
Dave Watson [Tue, 30 Sep 2014 23:17:25 +0000 (16:17 -0700)]
Bump version to 11:0

9 years ago(Wangle) Make via behave more like gate
Hans Fugal [Tue, 30 Sep 2014 22:52:52 +0000 (15:52 -0700)]
(Wangle) Make via behave more like gate

Summary:
Could the problem be that via continues the existing chain of futures,
whereas we actually want to start a new chain?

Is there any particular reason this wasn't implemented like this originally?

Test Plan:
Ran all the unit tests. I hope to try to reproduce the thread issue and
see if this improves things.

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, net-systems@, exa, njormrod, fugalh

FB internal diff: D1500225

Tasks: 4920689

9 years agoremove get_weak from singleton
Andrii Grynenko [Wed, 24 Sep 2014 00:50:05 +0000 (17:50 -0700)]
remove get_weak from singleton

Summary:

Test Plan: fbmake runtests, OBC no longer segfaults

Reviewed By: mshneer@fb.com

Subscribers: fbcode-common-diffs@, trunkagent, mcduff, marccelani, hitesh, mshneer, njormrod, lins

FB internal diff: D1573880

9 years agoFix compiler warning in Subprocess
Alan Frindell [Mon, 29 Sep 2014 20:01:49 +0000 (13:01 -0700)]
Fix compiler warning in Subprocess

Summary: I'm going to have hhvm depend on Subprocess, found this build error in open source.  I figured the intent was to set r to the return code.  I could also delete the whole thing since it's unused.

Test Plan: Unit tests

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, mwilliams, doug, njormrod

FB internal diff: D1583626

9 years agoFix bug and unit test failure in HHWheelTimer
Haijun Zhu [Mon, 29 Sep 2014 17:09:38 +0000 (10:09 -0700)]
Fix bug and unit test failure in HHWheelTimer

Summary: Some minor bug and a failed test caused by D1578466

Test Plan:
fbconfig thrift/lib/cpp/test:HHWheelTimerTest && fbmake
runtests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, alandau, bmatheny, njormrod

FB internal diff: D1581949

9 years agoUse RWSpinLock for Singleton mutex
Andrii Grynenko [Fri, 26 Sep 2014 02:02:37 +0000 (19:02 -0700)]
Use RWSpinLock for Singleton mutex

Summary: We only need exclusive lock when we add items to singletons_. Each SingletonEntry has its own mutex, so it's safe to rely on it for any modifications within individual entries.

Test Plan: Applied D1573880 and ran fbconfig -r servicerouter/client/cpp2 && fbmake runtests

Reviewed By: chip@fb.com

Subscribers: trunkagent, njormrod, hitesh, mshneer

FB internal diff: D1579877

9 years agoMalloc cannot include ScopeGuard
Nicholas Ormrod [Fri, 26 Sep 2014 22:19:51 +0000 (15:19 -0700)]
Malloc cannot include ScopeGuard

Summary:
Because we put FBString and Malloc into libgcc, we have to be
careful which dependencies they each have. We cannot include ScopeGuard.
Reorganize the code a bit to avoid the ScopeGuard.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: je@fb.com

Subscribers: trunkagent, sdwilsh, njormrod

FB internal diff: D1580957

9 years agoRare bug fix
Nicholas Ormrod [Fri, 26 Sep 2014 22:17:04 +0000 (15:17 -0700)]
Rare bug fix

Summary:
By the birthday paradox, this will fail about once every 8,100
runs. Dropping to 100 with cut that to 1 in 860,000.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: sdoroshenko@fb.com

Subscribers: sdwilsh, njormrod

FB internal diff: D1581238

9 years agoBump version to 10:0
Anton Likhtarov [Fri, 26 Sep 2014 22:29:05 +0000 (15:29 -0700)]
Bump version to 10:0

9 years agofix future executor test
James Sedgwick [Fri, 26 Sep 2014 19:17:01 +0000 (12:17 -0700)]
fix future executor test

Summary: This should be the last test abusing sleeps.

Test Plan: ran

Reviewed By: hannesr@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1580830

Tasks: 5225808

9 years agofolly/GroupVarint: fix a clang-caught heap-buffer-overrun
Jim Meyering [Thu, 25 Sep 2014 22:25:36 +0000 (15:25 -0700)]
folly/GroupVarint: fix a clang-caught heap-buffer-overrun

Summary:
Avoid a clang-caught heap-buffer-overrun
and document that the ssse3 decoder requires at least 17 bytes
of readable memory starting at the source pointer.
* folly/GroupVarint.h (decode_simple): Comment out dead code.
(decode)[__SSSE3__]: Add a comment describing this limitation.
* folly/test/GroupVarintTest.cpp (testGroupVarint32):
Include <algorithm> for use of std::max.
Ensure that the buffer we use has length at least 17.

Test Plan:
Now, all tests succeed with clang/asan

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, mathieubaudet, njormrod

FB internal diff: D1579109

Tasks: 5241437

9 years agoRFC: future executor
James Sedgwick [Fri, 26 Sep 2014 15:56:56 +0000 (08:56 -0700)]
RFC: future executor

Summary:
not necessarily for commit but i was playing around a bit with this today to see what it might look like, so i figured i'd put it up
i assume this shenanigan isn't remotely safe (threadwise) without some extra magic... @fugalh we chatted about this last time i was in mpk a bit.
maybe addFuture could take an executor to make sure the promise is fulfilled on the correct thread, or something.

you'd have to reimplement timeouts via this executor or via futures if you wanted to get them via the futures channel, which makes sense anyway as this could be used with any Executor and not just ThreadPoolExecutor which owns that implementation

also specialized makeFutureTry to take functions that return futures but treat them like they return the contained types, so fulfil() can be used liked then()
this specialization could just as easily be done on fulfil() itself if we wanted.

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: hannesr, trunkagent, craffert, njormrod, fugalh, bmatheny

FB internal diff: D1566369

9 years agofix more flakey tests
James Sedgwick [Fri, 26 Sep 2014 15:48:09 +0000 (08:48 -0700)]
fix more flakey tests

Summary: fixed the one brought up in the task, and tweaked another that could theoretically cause problems

Test Plan: ran, though I have not been able to repro the

Reviewed By: njormrod@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1575632

Tasks: 5225808

9 years agoSet DEFAULT_CATCHUP_EVERY_N to 10 to get more accurate task expiration when busy...
Simon Jenkins [Thu, 25 Sep 2014 21:25:47 +0000 (14:25 -0700)]
Set DEFAULT_CATCHUP_EVERY_N to 10 to get more accurate task expiration when busy event base

Summary:
when event base is busy we're timing out tasks too early.
BIG SCARY NOTE: I don't have the thrift knowledge to know if there's sideeffects of this, please let
me know if this change is harmful.

Test Plan:
tested internally

Reviewed By: davejwatson@fb.com

Subscribers: njormrod

FB internal diff: D1578466

Tasks: 4752162

9 years agoFix SocketAddress AF_UNIX support
Dave Watson [Wed, 24 Sep 2014 18:28:48 +0000 (11:28 -0700)]
Fix SocketAddress AF_UNIX support

Summary: check external_ vs. AF_UNIX.  Also fix reset() to reset storage_.addr

Test Plan: fbconfig folly/test:network_address_test; fbmake runtests

Reviewed By: soren@fb.com

Subscribers: tudorb, trunkagent, doug, njormrod

FB internal diff: D1575098

9 years agoAdd and fix BUCK files for folly
Andrew Gallagher [Thu, 25 Sep 2014 01:36:16 +0000 (18:36 -0700)]
Add and fix BUCK files for folly

Summary:
This gets almost all of folly building with buck.  There are a few
cases we don't support yet (custom test wrappers for spooky test
binaries, dlopen-enabled exception tracer binary, etc), but the
vast majority is covered.

Test Plan: built all build targets and ran all tests buck

Reviewed By: sdwilsh@fb.com

Subscribers: tjackson, sdwilsh, fugalh, njormrod

FB internal diff: D1577256

Tasks: 5055879

9 years agoexpose the class_name in an efficient way
Marc Horowitz [Fri, 5 Sep 2014 23:45:24 +0000 (16:45 -0700)]
expose the class_name in an efficient way

Summary:
Add a class_name method, which doesn't need to throw/catch
regardless how the exception is captured.

Test Plan: exception_wrapper_test

Reviewed By: mshneer@fb.com

Subscribers: njormrod

FB internal diff: D1543596

Tasks: 5025089

Blame Revision:

9 years agoDisable failing wangle test
Nicholas Ormrod [Thu, 25 Sep 2014 00:02:47 +0000 (17:02 -0700)]
Disable failing wangle test

Summary:
Under moderate load, this part of the test would fail. When
there is resource contention, it is quite possible that the timeout will
be exceeded. A further reduction of the time spread will render this
test next to useless.

Moderate load can be as little as running all of folly's tests at once,
which is something that our testing infrastructure does quite
frequently.

fbconfig -r folly
fbmake runtests

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: hannesr@fb.com

Subscribers: sdwilsh, fugalh, njormrod

FB internal diff: D1576570

Tasks: 5225808

9 years agoFix some folly memory leaks found with clang:dev + asan
Soren Lassen [Wed, 24 Sep 2014 20:34:17 +0000 (13:34 -0700)]
Fix some folly memory leaks found with clang:dev + asan

Summary:
Reverts the revert of the small_vector and test parts of D1569246.
I'm saving the SocketAddress fix for later, because it caused problems
and needs more research.

Test Plan:
unit tests with clang:dev/asan

Reviewed By: njormrod@fb.com

Subscribers: mathieubaudet, philipp, meyering, njormrod

FB internal diff: D1575574

Blame Revision: D1575190

9 years agoupdate thrift to use managed connection
Dave Watson [Wed, 24 Sep 2014 17:52:50 +0000 (10:52 -0700)]
update thrift to use managed connection

Summary:
Use the new managed connection code in folly.  Cleans things up a fair bit

The only catch was the current ConnectionManager code uses raw pointers, thrift expects shared_ptr so the request object can keep the channel around until the requests are finished

Test Plan: fbconfig -r thrift/lib/cpp2; fbmake runtests

Reviewed By: afrind@fb.com

Subscribers: trunkagent, njormrod, doug, fugalh, alandau, bmatheny, dcsommer, afrind

FB internal diff: D1519923

Tasks: 5002343

9 years agoRevert "Fix folly memory leaks found with clang:dev + asan."
Danny Guo [Wed, 24 Sep 2014 18:48:36 +0000 (11:48 -0700)]
Revert "Fix folly memory leaks found with clang:dev + asan."

Summary: This reverts commit 2c726dcf86bb176fb3695739a3d8bca2f95c41e6.

Test Plan: build it

Reviewed By: soren@fb.com

Subscribers: woo, mathieubaudet, njormrod

FB internal diff: D1575190

9 years agode-flake tests
James Sedgwick [Wed, 24 Sep 2014 17:30:21 +0000 (10:30 -0700)]
de-flake tests

Summary: make these more serialized / event based so they don't get flakey under high load

Test Plan:
ran under load - caveat: i was not able to repro the flakiness @njormrod reported
but by inspection these should be fine

Reviewed By: njormrod@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1574640

Tasks: 5225808

9 years agoFix folly memory leaks found with clang:dev + asan.
Soren Lassen [Tue, 23 Sep 2014 19:42:33 +0000 (12:42 -0700)]
Fix folly memory leaks found with clang:dev + asan.

Summary: I tried running folly/test with clang:dev and asan and found these leaks.

Test Plan:
unit tests with clang:dev/asan

Reviewed By: davejwatson@fb.com

Subscribers: davejwatson, trunkagent, mathieubaudet, njormrod, meyering, philipp

FB internal diff: D1569246

9 years agouser-defined expirations
James Sedgwick [Tue, 23 Sep 2014 18:17:03 +0000 (11:17 -0700)]
user-defined expirations

Summary:
Couple of notes:
1. is it a bummer not to have per-task callbacks of some kind? the interfaces set up here only tell you that some task expired, not which one expired. TM calls back with the Runnable object. is that useful?
2. std::chrono::* business is frustratingly verbose, but the safety/explicitness is nice. Not sure how I feel overall.
3. perhaps expirations should be given in microseconds even if we don't think we can accurately accomplish that

Test Plan: added unit

Reviewed By: hans@fb.com

Subscribers: fugalh, njormrod, bmatheny

FB internal diff: D1563520

9 years agoFix fbthrift build
Pavlo Kushnir [Tue, 23 Sep 2014 18:10:55 +0000 (11:10 -0700)]
Fix fbthrift build

Summary: Mcrouter open source build is failing because Codel is missing in fbthrift.

Test Plan: visual

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1572122

Blame Revision: D1566128

9 years agofix unittest
Dave Watson [Mon, 22 Sep 2014 22:18:51 +0000 (15:18 -0700)]
fix unittest

Summary: Looks like the test explicitly looks for InvalidAddressFamilyException.  I don't think any users are depending on this particular error type either way, I just switched it to InvalidAddressFamilyException

Test Plan:
unit tests

Reviewed By: ldbrandy@fb.com

Subscribers: doug, njormrod

FB internal diff: D1570363

Tasks: 5153819

9 years agothread safety for Observable::observers_
James Sedgwick [Tue, 23 Sep 2014 13:12:18 +0000 (06:12 -0700)]
thread safety for Observable::observers_

Summary: this way we can subscribe to an observable and blast data through it simultaneously from different threads

Test Plan: not much... the one client of rx compiles

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, bmatheny

FB internal diff: D1560647

9 years agostats for ThreadPoolExecutor
James Sedgwick [Tue, 23 Sep 2014 13:08:00 +0000 (06:08 -0700)]
stats for ThreadPoolExecutor

Summary:
pool-wide stats via a call on the pool, and per-task stats (e.g. to be funneled into a histogram) via an rx subscription
rx needs a little work before this diff is safe - e.g. synchronization around the subscriber list, and perhaps exposing whether there are any subscribers so we can skip stat tracking if no one is listening
won't commit this without moving rx into folly/experimental of course

the idea is that timeout/expiration notifications can also go through the same subscription channel

haven't run the benchmarks yet and have to leave for the evening but tmrw i'll commit changes to the benchmark and get this stuff into windtunnel so i don't have to do so much manual output inspection on future diffs

Test Plan: added unit

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, bmatheny

FB internal diff: D1558424

Tasks: 50023925002425

9 years agomove rx to folly/experimental
James Sedgwick [Tue, 23 Sep 2014 12:48:08 +0000 (05:48 -0700)]
move rx to folly/experimental

Summary: As above. I want to use this for the thread pools and it probably belongs in folly long-term anywya (if we stick with it)

Test Plan: compiled the one user

Reviewed By: hans@fb.com

Subscribers: fugalh, mwa, jgehring, fuegen, njormrod

FB internal diff: D1560578

9 years agomove Codel to wangle
James Sedgwick [Tue, 23 Sep 2014 12:46:37 +0000 (05:46 -0700)]
move Codel to wangle

Summary: under folly::wangle::

Test Plan: ran unit for experimental/wangle, thrift/lib/cpp/concurrency, and a user of FbThreadManager

Reviewed By: hans@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, fugalh, alandau, bmatheny, everstore-dev@, njormrod

FB internal diff: D1566128

9 years agoFix lint errors
Nicholas Ormrod [Mon, 22 Sep 2014 21:13:14 +0000 (14:13 -0700)]
Fix lint errors

Summary: Pass const StringPieces by value instead of reference.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod

FB internal diff: D1569488

9 years agorfc : -Wshorten-64-to-32 warnings in folly liger dependencies
Ranjeeth Dasineni [Wed, 20 Aug 2014 19:38:57 +0000 (12:38 -0700)]
rfc : -Wshorten-64-to-32 warnings in folly liger dependencies

Summary:
This is not a fix but me seeking advice here. The context here is that in liger (our common network
stack for mobile platforms), we saw bugs due to truncation and want to enable these warnings. To find those in our code,
I had to first resolve the folly ones. I just got around by making the truncation explicit so that I can get to real errors.
Ideally I would like owners to examine each case and fix the original/accept the explicit truncate if its intended. Let me
know what you think. The last (least preferred) is for us to keep this as a liger only change. We have a couple of ways to
do that but it would be nice to fix.

N.B : this covers only liger dependencies

Test Plan: errors resolved after these changes.

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, doug, shilin, njormrod, seanc, pgriess

FB internal diff: D1509355

9 years agofix leak in MPMCQueueTest's use of boost::intrusive_ptr
Nathan Bronson [Mon, 22 Sep 2014 18:33:28 +0000 (11:33 -0700)]
fix leak in MPMCQueueTest's use of boost::intrusive_ptr

Summary:
MPMCQueueTest's intrusive reference count test implementation was
incorrect, so it simultaneously didn't test what it should and had a leak.

Test Plan: tests, plus new __thread to track lifetimes

Reviewed By: meyering@fb.com

Subscribers: njormrod, soren, meyering

FB internal diff: D1569448

9 years agoFix lint errors
Nicholas Ormrod [Mon, 22 Sep 2014 18:27:18 +0000 (11:27 -0700)]
Fix lint errors

Summary: Lines exceeding 80 characters have been restructured.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, fugalh, njormrod

FB internal diff: D1568086

9 years agoIncrease wangle test timers
Nicholas Ormrod [Mon, 22 Sep 2014 18:20:47 +0000 (11:20 -0700)]
Increase wangle test timers

Summary:
When running all folly tests at once, as with
fbconfig -r folly && fbmake runtests
the wangle tests sometimes fail. Enough tests are running that the
1-millisecond timer is too small, and is sometimes missed. Bumping
it to 10 milliseconds.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: sdwilsh, fugalh, njormrod

FB internal diff: D1568124

Tasks: 5180879

9 years agoFaster accessors for dynamic values
Pavlo Kushnir [Sat, 20 Sep 2014 00:02:35 +0000 (17:02 -0700)]
Faster accessors for dynamic values

Summary:
Sometimes it is useful to have a fast accessor to a value stored in dynamic, without any type conversion. These still do the type check, but compiler has more room for optimization.

Test Plan: build folly, will add unit tests/benchmarks if this makes sense at all.

Reviewed By: njormrod@fb.com

Subscribers: jdelong, njormrod

FB internal diff: D1564867

9 years agoadd function to push data from another cursor
Guohui Wang [Fri, 29 Aug 2014 01:32:59 +0000 (18:32 -0700)]
add function to push data from another cursor

Summary:
Add a function to Writable to push data from another cursor. Data in the input
cursor could be in a buffer chain.

Added test in IOBufCurosTest.cpp to test the new function.

Test Plan: fbmake runtests;

Reviewed By: simpkins@fb.com

Subscribers: njormrod, net-systems@

FB internal diff: D1563209

Tasks: 4886058

Blame Revision:

9 years agoBump version to 9:0
dcsommer [Fri, 19 Sep 2014 22:31:17 +0000 (15:31 -0700)]
Bump version to 9:0

9 years agoDo not barf on invalid ELF files
Tudor Bosman [Fri, 19 Sep 2014 19:58:10 +0000 (12:58 -0700)]
Do not barf on invalid ELF files

Summary: The Haskell runtime does magic, and loads an ET_REL file directly.

Test Plan: ran it

Reviewed By: jon.coens@fb.com

Subscribers: njormrod, rkroll

FB internal diff: D1566853

Tasks: 4401996

@override-unit-failures

9 years agomove socketaddress to folly
Dave Watson [Fri, 12 Sep 2014 18:30:59 +0000 (11:30 -0700)]
move socketaddress to folly

Summary:
Goes with IPAddress and MACAddress already in folly.

* change namespace, add typedef in thrift.  Had to update forward decls to folly namespace.
* change exception types to std:: types instead of TTransportException.  I fbgs'd for everywhere I could find trying to catch these and updated the type, there looked like there were only ~9 files or so.

Test Plan: contbuild.

@override-unit-failures

Tests already passed in last run

Reviewed By: simpkins@fb.com

Subscribers: doug, ps, alandau, everstore-dev@, njormrod, fugalh, alikhtarov, bmatheny, jsedgwick

FB internal diff: D1556916

9 years agofix occasional crash in cpu pool
James Sedgwick [Fri, 19 Sep 2014 10:19:17 +0000 (03:19 -0700)]
fix occasional crash in cpu pool

Summary: need to initialize threadsToStop_ and make it signed so it doesn't underflow in rare races

Test Plan: cpu pool unit tests no longer crash if run a ton of times

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1563524

9 years agoinstall connectionmanager headers
Dave Watson [Thu, 18 Sep 2014 19:12:33 +0000 (12:12 -0700)]
install connectionmanager headers

Summary: Install headers / compile files

Test Plan: watch jenkins build

Reviewed By: dcsommer@fb.com

Subscribers: doug, fugalh, njormrod

FB internal diff: D1564098

9 years agoBump version to 8:0
Dave Watson [Thu, 18 Sep 2014 16:45:05 +0000 (09:45 -0700)]
Bump version to 8:0

9 years agomerge https://github.com/facebook/folly/pull/88
dungcoi [Thu, 18 Sep 2014 16:19:57 +0000 (09:19 -0700)]
merge https://github.com/facebook/folly/pull/88

Summary:
merge https://github.com/facebook/folly/pull/88, missing install header

Looks like fbthrift build needs this

Test Plan: watch jenkins build still works

Reviewed By: ckwalsh@fb.com

Subscribers: doug, njormrod

FB internal diff: D1563651

9 years agoRemove dependency on std::to_string
Sean Cannella [Thu, 18 Sep 2014 14:20:04 +0000 (07:20 -0700)]
Remove dependency on std::to_string

Summary:
Mobile platform clang doesn't currently support std::to_string
so remove folly's dependency on it.

Test Plan: existing tests

Reviewed By: mpawlowski@fb.com

Subscribers: kmdent, fma, benyluo, shikong, ranjeeth, subodh, pgriess, njormrod

FB internal diff: D1562179

Tasks: 5183325

9 years agofix race in Future::waitWithSemaphore
Matt Dordal [Wed, 17 Sep 2014 23:52:03 +0000 (16:52 -0700)]
fix race in Future::waitWithSemaphore

Summary:
There's a race condition in waitWithSemaphore, specifically because we're
returning a new future that's completed by the input future. As a result,
that future may not be completed by the time waitWithSemaphore returns,
although completion should be imminent.

Test Plan:
`var=0; while true ; do echo $((var++)); _bin/folly/wangle/wangle-test --gtest_filter='Future*' || break ; done`

Before change two runs yielded 303 and 371.

After change I killed the test at 11000.

Reviewed By: njormrod@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1561281

Tasks: 5180879

9 years agoremove eof whitespace lines
Nicholas Ormrod [Wed, 17 Sep 2014 20:21:40 +0000 (13:21 -0700)]
remove eof whitespace lines

Summary:
Remove lint errors from folly: trailing newlines

find folly | xargs arc lint | grep -B3 EOF | grep ">>>" | egrep -o "folly[^:]*" | xargs -n1 sed -i '$d'

folly/experimental/io/test/AsyncIOTest.cpp had two trailing newlines,
the second was removed by hand

@override-unit-failures

Test Plan:
fbconfig -r folly && fbmake runtests

Pretty overkill for a whitespace diff, but why not.

Reviewed By: robbert@fb.com

Subscribers: sdwilsh, fugalh, njormrod

FB internal diff: D1561213

9 years agoBump version to 7:0
Dave Watson [Wed, 17 Sep 2014 18:23:54 +0000 (11:23 -0700)]
Bump version to 7:0

9 years agomove connectionmanager/managed connection to wangle
Dave Watson [Wed, 17 Sep 2014 17:20:31 +0000 (10:20 -0700)]
move connectionmanager/managed connection to wangle

Summary: Moar code sharing between proxygen/thrift.  Code must be moved to a common library: The only one we currently have is folly, but if someone wants to argue for a separate lib, that's cool too

Test Plan: fbconfig -r proxygen; fbmake dev

Reviewed By: dcsommer@fb.com

Subscribers: doug, fugalh, bmatheny, njormrod

FB internal diff: D1519919

Tasks: 5002343

9 years agoThreadPoolExecutor and its children CPUThreadPoolExecutor and IOThreadPoolExecutor
James Sedgwick [Wed, 17 Sep 2014 09:58:06 +0000 (02:58 -0700)]
ThreadPoolExecutor and its children CPUThreadPoolExecutor and IOThreadPoolExecutor

Summary:
Spun off from https://phabricator.fb.com/D1534506 as this seemed different enough for a new diff

Similar to previous diff but attempts to reuse a common thread management process between cpu and io bound thread pools. Also sets the stage for other common functionality, e.g. stats, monitoring, timeouts, and so on

Here is some output from the queue benchmark in common/concurrent with both of these pools added (changes to BM not in this diff): https://phabricator.fb.com/P16308560

Test Plan: added a unit test, ran benchmark

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, bmatheny

FB internal diff: D1555443

Tasks: 50023925002425

9 years agoFix folly signed/unsigned comparisons
Sean Cannella [Wed, 17 Sep 2014 00:08:52 +0000 (17:08 -0700)]
Fix folly signed/unsigned comparisons

Summary:
Fix a few sign comparison issues in folly::IPAddress

@override-unit-failures

Test Plan: existing tests, compiled with clang

Reviewed By: meyering@fb.com, davejwatson@fb.com

Subscribers: njormrod, bmatheny, subodh, ranjeeth, pgriess

FB internal diff: D1559152

9 years agomake with_exception() const work in all cases
Marc Horowitz [Thu, 11 Sep 2014 00:39:19 +0000 (17:39 -0700)]
make with_exception() const work in all cases

Summary:
Abstract the with_exception() logic into a static template
function which will work with a const or non-const object.

Test Plan: exception_wrapper_test

Reviewed By: vloh@fb.com

Subscribers: njormrod

FB internal diff: D1549175

Tasks: 5127267

9 years agoAdding IO stats in AsyncIO.
Shao-Chuan Wang [Tue, 16 Sep 2014 05:56:18 +0000 (22:56 -0700)]
Adding IO stats in AsyncIO.

Summary:
It would be great if we have IO stats tracking built within AsyncIO.
It would enable upper layer application to better track the number of I/O
that was submitted and completed

Test Plan:
$ fbmake runtests
Test Results Summary:
Passed: 1734
100% successful

Reviewed By: philipp@fb.com

Subscribers: njormrod, schen, stanislav

FB internal diff: D1557472

Tasks: 5008299

9 years agoBump version to 6:0
Dave Watson [Mon, 15 Sep 2014 21:30:23 +0000 (14:30 -0700)]
Bump version to 6:0

9 years agoAdd non-strict mode to folly::Singleton
Chip Turner [Fri, 12 Sep 2014 00:36:50 +0000 (17:36 -0700)]
Add non-strict mode to folly::Singleton

Summary:
It is difficult to guarantee every binary can call
registrationComplete; for now, support a mode where we don't enforce the
strict registration lifecycle.  In the fullness of time, we can make
strictness the default.

Test Plan: runtests

Reviewed By: lins@fb.com

Subscribers: lins, anca, njormrod

FB internal diff: D1551688

9 years agoequality for exception_wrapper
Victor Loh [Fri, 12 Sep 2014 22:03:39 +0000 (15:03 -0700)]
equality for exception_wrapper

Summary:
std::exception_ptr expose methods to compare equality and it
will be great to do so for folly::exception_wrapper (it helps in gmock
testing).

Test Plan: added some unit tests to test this new functionality

Reviewed By: mhorowitz@fb.com

Subscribers: njormrod

FB internal diff: D1546597

9 years agoMake TSocketAddress use folly::IPAddress
Dave Watson [Wed, 3 Sep 2014 21:41:47 +0000 (14:41 -0700)]
Make TSocketAddress use folly::IPAddress

Summary:
Step 1 of moving TAsyncSocket and friends to folly.

folly::IPAddress changes (can split these off if requested):
* isLoopback looks for a single address, as opposed to subnet
* For V6, isPrivate & isLoopback don't seem to check mapped addresses

TSocketAddress changes:
* Make union use folly::IPAddress instead of sockaddr struct.  external_ bool to tell if using AF_UNIX instead.
* manage the port and scope/zone info ourselves

@override-unit-failures

Overriding failures, can't get contbuild to finish all builds after several tries.  None of the failures look related

Test Plan:
fbconfig -r thrift
fbmake runtests
fbconfig folly/test:network_address_test
fbmake runtests

Reviewed By: simpkins@fb.com

Subscribers: snaptu-diffs@, trunkagent, hphp-diffs@, njormrod, fbcode-common-diffs@, doug, ruibalp, ps, alandau, bmatheny

FB internal diff: D1479365

Tasks: 3658665

9 years agofolly: avoid used-uninitialized bug in a test
Jim Meyering [Thu, 11 Sep 2014 22:23:17 +0000 (15:23 -0700)]
folly: avoid used-uninitialized bug in a test

Summary:
Clang complained about this.
* folly/gen/test/FileBenchmark.cpp (BENCHMARK): Initialize "rfd" to -1
and verify that it is set to something nonnegative in the first
BENCHMARK_SUSPEND block.

Test Plan:
Run this:
fbconfig -r --clang --with-project-version clang:dev folly/gen && fbmake runtests
and observe that compilation now succeeds. (and test results remain unchanged)

Reviewed By: tudorb@fb.com

Subscribers: njormrod

FB internal diff: D1551180

Tasks: 4090011

9 years agoBump version to 5:0
Sara Golemon [Wed, 10 Sep 2014 23:59:58 +0000 (16:59 -0700)]
Bump version to 5:0

9 years agoInstall EvictingCacheMap as part of OSS build
Daniel Sommermann [Wed, 10 Sep 2014 23:38:04 +0000 (16:38 -0700)]
Install EvictingCacheMap as part of OSS build

Summary: This may have just been forgotten when imported to folly

Test Plan: live. verified file name

Reviewed By: ranjeeth@fb.com

Subscribers: doug, njormrod

FB internal diff: D1548804