5 years agoupdate stats APIs to use TimePoint vs Duration correctly
Adam Simpkins [Fri, 16 Sep 2016 19:23:41 +0000 (12:23 -0700)]
update stats APIs to use TimePoint vs Duration correctly

Update the stats APIs to correcly distinguish between TimePoint and Duration

This does leave addValue() and update() APIs in place that accept Duration
values, for backwards compatibility.  These should eventually be removed once
all code has been converted to call the new APIs.

Reviewed By: yfeldblum

Differential Revision: D3808805

fbshipit-source-id: 36d6574ba4a09db7eb9f1a35e47addd3e07f8461

5 years agoReturn rvalue references from &&-qualified members of dynamic
Tom Jackson [Fri, 16 Sep 2016 17:09:32 +0000 (10:09 -0700)]
Return rvalue references from &&-qualified members of dynamic

Summary: Let the caller do a move, don't force one.

Reviewed By: yfeldblum

Differential Revision: D3873129

fbshipit-source-id: 40c6bf564bcbf794830c99ea1248a9c1bb30e9b0

5 years agoReduce footprint of ScribeClient
Vladimir Slaykovskiy [Fri, 16 Sep 2016 09:53:58 +0000 (02:53 -0700)]
Reduce footprint of ScribeClient

I've found all code paths that contributed to the number of threads in new ScribeClient. Here's a list of flags that reduce the number of thread:
  --default_tls_thread_count=1 # was 32
  --has_thrift_dispatcher_reporter=0 # creates a thrift server with a few extra threads
  --observer_manager_pool_size=1 # default is 4

Optimizations in this diff:
- Don't initialize OBCClient object if it is not used
- Added FLAG_observer_manager_pool_size to control size of the pool in ObserverManager

Currently OBC counters are switched off globally in ScribeCliean, it means that new code path that creates SR instances is disabled, but eventually we plan to switch it on. Clients that don't use SR and want to keep their RSS/threads profile low, should pass flags listed above. I'll announce this in ScribeUsers group as well.

Differential Revision: D3870704

fbshipit-source-id: 0efad6b3dc43c072ab11cac7e9461c09532ea11c

5 years agowork around LLVM#30305 in folly::Expected, use unified initialization syntax for...
Eric Niebler [Fri, 16 Sep 2016 00:00:30 +0000 (17:00 -0700)]
work around LLVM#30305 in folly::Expected, use unified initialization syntax for extra goodness

Summary: The behavior of inheriting constructors changed in clang 3.9 (see, and folly::Expected needs to change accordingly. As a drive-by improvement, change all invocations of default constructors to use unified initialization syntax.

Reviewed By: igorsugak

Differential Revision: D3872994

fbshipit-source-id: fdaea8b35980df21b8522e2c3d5a8c3be1d84efa

5 years agofolly/.../ExceptionTracerLib.cpp: provide less DOF fodder (avoid heap use-after-free)
Jim Meyering [Thu, 15 Sep 2016 23:26:34 +0000 (16:26 -0700)]
folly/.../ExceptionTracerLib.cpp: provide less DOF fodder (avoid heap use-after-free)

Before this change, an application would fail consistently with a shutdown-time heap use-after-free (i.e., destructor-order-fiasco when one of the following symbols was used after its static-destruction-triggered free):


Each of those would define a classic meyers singleton.
Since each destructor is trivial, we can fix this by making each a pseudo-leaky (indestructible) meyers singleton instead. Since each static value is never destroyed,
it can never cause a DOF error again.

Differential Revision: D3870740

fbshipit-source-id: 625f5d5268768ca0e4125bed72bc66c53618be29

5 years agoKill unneeded dependency on portability/Memory.h
Christopher Dykes [Thu, 15 Sep 2016 20:00:46 +0000 (13:00 -0700)]
Kill unneeded dependency on portability/Memory.h

Summary: This dependency was added but isn't actually needed, so get rid of it.

Reviewed By: mzlee

Differential Revision: D3863923

fbshipit-source-id: 407d2b3db759e30c37d876dde133fc962daeaeae

5 years agoAdd flag to indicate a session resumption attempt
Neel Goyal [Thu, 15 Sep 2016 19:00:17 +0000 (12:00 -0700)]
Add flag to indicate a session resumption attempt

Summary: Track when session resumption was attempted during connect

Reviewed By: shamdor-fb

Differential Revision: D3861028

fbshipit-source-id: 26ca41084faeeb64666b6833896b9510ef2b4c25

5 years agoa simple first step towards using clocks properly in the stats code
Adam Simpkins [Thu, 15 Sep 2016 18:02:47 +0000 (11:02 -0700)]
a simple first step towards using clocks properly in the stats code

Update the timeseries and histogram classes to accept a clock parameter as a
template parameter, instead of a time duration type.

This is a first step towards transitioning the code to correctly distinguishing
between time_point and duration types.  This defines TimePoint and Duration
type aliases, but does not start using them yet.

In upcoming diffs I will start converting more APIs to correctly use TimePoint
instead of just Duration.

For now the default clock type is folly::LegacyStatsClock, which still uses
std::chrono::seconds as the default duration.  At the moment the stats code is
optimized for second granularity--the addValue() code has a fast path when
called in the same second as the last update.  When using finer granularity
durations this fast path can't be used as often.  I will also send out
subsequent diffs to make the code optimized for updates within the same bucket,
rather than just updates with the exact same time value.

Reviewed By: yfeldblum

Differential Revision: D3807715

fbshipit-source-id: 77696c4f44a8d85e4d6ff84d7656fe7a9709797c

5 years agoignore `$SHELL` in `shellify`
Dominik Gabi [Thu, 15 Sep 2016 14:57:15 +0000 (07:57 -0700)]
ignore `$SHELL` in `shellify`

Why `$SHELL` is a bad idea:

- getenv() is not thread safe. (In practice it should work if other threads aren't calling setenv(), but still seems undesirable if we can avoid it.)
- It seems confusing for the program to have different behavior for different developers.
- Other shells besides /bin/sh may have different quoting behaviors. For instance, I don't think csh allows unescaped newlines inside single quotes.
- SHELL might be set to other non-shell-like programs in some cases. (Say, if this gets run from inside a git or mercurial hook it might end up being set to the restricted git or hg shell that only lets you run specific commands.)
Anyway, this isn't related to your diff, so nothing needs to be done for now,
but I would vote for changing this to always use /bin/sh in a future diff.

Both the C system() call and python's subprocess module appear to
always use /bin/sh and ignore $SHELL.

Reviewed By: simpkins

Differential Revision: D3867047

fbshipit-source-id: dab0e6afbe1c20ff13d9a52f212df95af425dd77

5 years agoformatting support for `Subprocess::shellify`
Dominik Gabi [Thu, 15 Sep 2016 14:57:14 +0000 (07:57 -0700)]
formatting support for `Subprocess::shellify`

Adding formatting support with automatic escaping of parameters. Will
subsequentyl try and make it harder for people to use non-statically determined
format strings for this.

Reviewed By: simpkins

Differential Revision: D3866174

fbshipit-source-id: 80bb5886386692643876116fbf338ca0d6986e6a

5 years agodeprecate `folly::Subprocess(std::string, ...)`
Dominik Gabi [Thu, 15 Sep 2016 14:57:13 +0000 (07:57 -0700)]
deprecate `folly::Subprocess(std::string, ...)`

introducing `Subprocess::shellify` to get around this. Still thinking
about how to make sure people escape arguments passing into that function. I'd
like to have something along the lines of Phabricator's `csprintf` here.

Reviewed By: yfeldblum

Differential Revision: D3857827

fbshipit-source-id: 8afbc9f1c62c62e0fc91782e11b808145b370933

5 years agoMake ElfCache follow .gnu_debuglink
Mark Williams [Wed, 14 Sep 2016 22:48:52 +0000 (15:48 -0700)]
Make ElfCache follow .gnu_debuglink

If you split out debug info into a separate file, folly::Symbolizer
didn't find it, and failed to include file and line number info in

This adds a new open mode which follows the .gnu_debuginfo link, and
uses it from ElfCache and SignalSafeElfCache.

Reviewed By: meyering, luciang

Differential Revision: D3852311

fbshipit-source-id: fec9e57378ae59fa1b82d41a163bb9cfcf9ca23c

5 years agoSwap a newly added include of gtest.h with portability/GTest.h
Christopher Dykes [Wed, 14 Sep 2016 21:12:21 +0000 (14:12 -0700)]
Swap a newly added include of gtest.h with portability/GTest.h

Summary: Because it's needed with the way the portability layer works.

Reviewed By: yfeldblum

Differential Revision: D3863954

fbshipit-source-id: 3b682dcc2d2799f17c61fe16a4391fc9f0c2b127

5 years agoTest for folly::SingletonThreadLocal
Phil Willoughby [Wed, 14 Sep 2016 08:53:09 +0000 (01:53 -0700)]
Test for folly::SingletonThreadLocal

Summary: Validates that we create a distinct singleton for each thread and that each such singleton is destroyed at its owning-thread's exit.

Reviewed By: yfeldblum

Differential Revision: D3849146

fbshipit-source-id: af878b32ecfc6c82b866d7a805e1385d74a8a5f5

5 years agoRemove Benchmark.h dependency in SpookyHash tests
Michael Lee [Tue, 13 Sep 2016 23:02:19 +0000 (16:02 -0700)]
Remove Benchmark.h dependency in SpookyHash tests

Summary: It would seem the folly/Benchmark.h dependency was to pull in glog/logging.h. Include what you use.

Reviewed By: yfeldblum

Differential Revision: D3857112

fbshipit-source-id: 8d5bd105e10eee8f5b6e5d377d58ae84d908ed57

5 years agoOptimize toJson() Performance
Milo Yip [Tue, 13 Sep 2016 18:24:35 +0000 (11:24 -0700)]
Optimize toJson() Performance

Removing string::reserve() which causes O(n^2) penalty.

Fixes #477

Reviewed By: yfeldblum

Differential Revision: D3850509

Pulled By: Orvid

fbshipit-source-id: ecf44c35b2aedadc5385d23c325cacab1abfd02d

5 years agoFix recordStack feature in FiberManager
Stepan Palamarchuk [Tue, 13 Sep 2016 02:49:22 +0000 (19:49 -0700)]
Fix recordStack feature in FiberManager

It appears that a feature of recording exact stack usage on fibers is broken for a long time.
This diff brings it back.

Reviewed By: andriigrynenko

Differential Revision: D3847035

fbshipit-source-id: d9cf532774204b6ac2f40e1b39fd98ea06c08f55

5 years agoFix a bug in ObserverManager destructor
Andrii Grynenko [Tue, 13 Sep 2016 00:53:52 +0000 (17:53 -0700)]
Fix a bug in ObserverManager destructor

Summary: facepaw

Reviewed By: ckwalsh

Differential Revision: D3853363

fbshipit-source-id: f863a3e3ae830256f5cff6c56f23254ff6fdee2a

5 years agoFix TLObserver copy constructor
Andrii Grynenko [Tue, 13 Sep 2016 00:53:09 +0000 (17:53 -0700)]
Fix TLObserver copy constructor

Reviewed By: yfeldblum

Differential Revision: D3853494

fbshipit-source-id: 23f5cadcb55a9307fb2a825fe4c53a03c5978d12

5 years agoDo not mess with NDEBUG in fbstring
Giuseppe Ottaviano [Mon, 12 Sep 2016 21:00:40 +0000 (14:00 -0700)]
Do not mess with NDEBUG in fbstring

Summary: Temporarily overriding `NDEBUG` can have unexpected side-effects depending on the implementation of `<assert.h>`. Better use an explicit macro for `assert()` that we can explicitly disable.

Reviewed By: Gownta

Differential Revision: D3850717

fbshipit-source-id: b1e7fce89ab4120b0224e9c6bf668d983b3d2d31

5 years agoUse the GTest portability headers v2016.09.12.00 v2016.09.12.01
Christopher Dykes [Sat, 10 Sep 2016 00:44:01 +0000 (17:44 -0700)]
Use the GTest portability headers

Switch all of the uses of `<gtest/gtest.h>` to `<folly/portability/GTest.h>`.

This is painful but necessary to get the tests to compile nicely under MSVC.

Reviewed By: yfeldblum

Differential Revision: D3837300

fbshipit-source-id: 7ae43a5d5ba94c8c24fa23a485f18546416e7191

5 years agoUse the GMock portability header
Christopher Dykes [Sat, 10 Sep 2016 00:44:00 +0000 (17:44 -0700)]
Use the GMock portability header

Switch all of the uses of <gmock/gmock.h> to <folly/portability/GMock.h>.

This is painful but necessary to get the tests to compile nicely under MSVC.

Reviewed By: yfeldblum

Differential Revision: D3837529

fbshipit-source-id: 7221dfea8f2a880919690b5b0601ca91642991ae

5 years agoAdd portability headers for GTest and GMock
Christopher Dykes [Sat, 10 Sep 2016 00:43:59 +0000 (17:43 -0700)]
Add portability headers for GTest and GMock

Both the gtest and gmock headers include `<io.h>` on Windows, which conflicts with how the portability headers include it, so a specific include order is required before either of those headers can be included. As that's brittle and messy, create a pair of portability headers for them instead, so that those can be included instead.

This only adds the headers. The switch to using them will be done in a later pair of diffs.

Reviewed By: mzlee

Differential Revision: D3837221

fbshipit-source-id: 82a273485cdd4536f1153d958c171bfe3ec32e0b

5 years agoRefactor basic_fbstring
Eric Niebler [Fri, 9 Sep 2016 23:19:15 +0000 (16:19 -0700)]
Refactor basic_fbstring

Summary: Move fbstring_core and basic_fbstring member functions from in-situ to out-of-class inlines and refactor for readability. Remove superfluous dependence on scope_guard.

Reviewed By: ot, luciang

Differential Revision: D3795250

fbshipit-source-id: f6edca25d4771181faff9e0a4339bbaffd71a370

5 years agoNomenclature in Synchronized - prefer read to shared
Yedidya Feldblum [Fri, 9 Sep 2016 21:44:11 +0000 (14:44 -0700)]
Nomenclature in Synchronized - prefer read to shared

[Folly] Nomenclature in `Synchronized` - prefer `read` to `shared`.

`folly::Synchronized` is a higher-level abstraction, so it will use higher-level nomenclature than will lower-level tools like `std::mutex` and `folly::LockTraits`. `shared` describes the lock state, and is used in the C++ libraries. `read` describes the class of operations that calling code is permitted to perform.

Reviewed By: simpkins

Differential Revision: D3840060

fbshipit-source-id: 4b23eaa391cb59d1eca2bfacf72db89d3c7c591e

5 years agoAdd forwarding gather() function to IOBufQueue
Daniel Sommermann [Fri, 9 Sep 2016 20:50:08 +0000 (13:50 -0700)]
Add forwarding gather() function to IOBufQueue

I'm working with a parser that requires a certain number of
contiguous bytes to be able to make forward progress. I'm also using
IOBufQueue to receive data from an AsyncReader::ReadCallback with
pre/postallocate. So, I need to call gather() to ensure that the queue's
front IOBuf has the right number of contiguous bytes available.

Reviewed By: djwatson

Differential Revision: D3838079

fbshipit-source-id: 9f1ec5c86895eb1b2b109f9f145ca42d2dbba4c6

5 years agoMake folly::PasswordCollector::getPassword const
Anirudh Ramachandran [Fri, 9 Sep 2016 17:23:15 +0000 (10:23 -0700)]
Make folly::PasswordCollector::getPassword const

Summary: As in title

Reviewed By: siyengar

Differential Revision: D3794648

fbshipit-source-id: f0b7052f34ecce65cf4e21d546d08c7a6b0a8ee3

5 years ago#if FOLLY_HAVE_INT128_T rather than #ifdef
Michael Lee [Fri, 9 Sep 2016 00:02:37 +0000 (17:02 -0700)]
#if FOLLY_HAVE_INT128_T rather than #ifdef

Summary: Switch from #ifdef to #if so the configuration can define the macro to 0

Reviewed By: yfeldblum, Orvid

Differential Revision: D3838748

fbshipit-source-id: e287b07f0fdfdc86c882538e96f2078795b85bfd

5 years agomove AsyncSSLSocket logging level for errors to vlog
Denis Samoylov [Thu, 8 Sep 2016 20:50:13 +0000 (13:50 -0700)]
move AsyncSSLSocket logging level for errors to vlog

Summary: current log entries ("E0906 AsyncSSLSocket.cpp:117] TCP connect failed: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused)") are not very helpful to debug due lack of details and can spam logs of application that uses async library without ability to disable logging

Reviewed By: djwatson

Differential Revision: D3825048

fbshipit-source-id: 1c97f14e1ea3f1b276d04bb12483d42372a0d186

5 years agoDynamic expansion of folly MPMC queue
Maged Michael [Thu, 8 Sep 2016 17:43:23 +0000 (10:43 -0700)]
Dynamic expansion of folly MPMC queue

This diff allows queues to start with small capacity and expand as needed up to the specified capacity.
The main additions and changes:
- Extra template parameter `Dynamic` that enables dynamic expansion (`default 'false').
- `ClosedArray` type.
- Extra members:
  -- `dstate_`: a packed 64 bit unsigned int that contains a seqlock (which implicitly indicates the number of expansions and the lowest ticket for the current `dslots_/dcapacity_/dstride_` configuration.
 -- `dcapacity_`: current dynamic capacity.
 -- `dslots_`: current dynamic slots array. (in anonymous union with `slots_`)
 -- `dstride_`: current dynamic stride. (in anonymous union with `stride_`)
 -- `closed_` a logarithmic-sized array of ClosedArray to hold information about earlier smaller queue arrays for use by lagging consumers.

Design sketch:
- Reallocate a new larger array on expansion
- Expansion uses a seqlock. The common case critical path includes a seqlock read-only section.
- Lagging consumers and lagging blocking producers use a logarithmic-sized array for info about closed arrays
- Tickets are adjusted by an offset (to accounts for the tickets associated with earlier closed arrays) in order to calculate appropriate index and turn.
- The synching of `pushTicket_` with the ticket offset packed in `dstate_` is tricky. `pushTicket_` is accessed outside `dstate_`'s seqlock.

Reviewed By: djwatson

Differential Revision: D3462592

fbshipit-source-id: d442a7694190cca3c33753409ffac941d7463f83

5 years agoRemove runAfterDrain()
Andrii Grynenko [Thu, 8 Sep 2016 17:29:01 +0000 (10:29 -0700)]
Remove runAfterDrain()

Reviewed By: djwatson

Differential Revision: D3832835

fbshipit-source-id: b5cabc00758fd80b314424e4224458e2d50ddb5c

5 years agoExplicitly include <random> to use std::mt19937 in MathBenchmark.
Zonr Chang [Thu, 8 Sep 2016 13:18:44 +0000 (06:18 -0700)]
Explicitly include <random> to use std::mt19937 in MathBenchmark.

This fixes build with Clang.

Differential Revision: D3834865

Pulled By: nbronson

fbshipit-source-id: 02c1ca192c6b6af2cc0a8fdaa0a854510cb0bca4

5 years agoprvalues from get_ref_default()'s default functor
Tom Jackson [Thu, 8 Sep 2016 09:36:58 +0000 (02:36 -0700)]
prvalues from get_ref_default()'s default functor

Summary: This previously allowed `get_ref_default(map, 4, []{ return 6; })`, even though this would form a reference to a temporary, then **use that invalid reference**.

Reviewed By: yfeldblum

Differential Revision: D3802707

fbshipit-source-id: 384d965f69c9d7b6bd3f011c8eff7fe55be7023a

5 years agoMove the alignment attribute in TokenBucket to a place where MSVC supports it
Christopher Dykes [Thu, 8 Sep 2016 01:44:32 +0000 (18:44 -0700)]
Move the alignment attribute in TokenBucket to a place where MSVC supports it

Summary: MSVC doesn't support the aligment attribute being after the field name.

Reviewed By: yfeldblum

Differential Revision: D3832314

fbshipit-source-id: 6a56db245e01922ede6b9b93eb1c8aecc835e2bf

5 years agointeger division with controlled rounding in Math.h
Nathan Bronson [Wed, 7 Sep 2016 22:50:38 +0000 (15:50 -0700)]
integer division with controlled rounding in Math.h

C++'s integer division performs truncation, but it is fairly
common that people actually want to round up.  There are actually
four rounding modes that make some sense: toward negative infinity
(floor), toward positive infinity (ceil), toward zero (truncation),
and away from zero (?).  It is pretty common that code that wants ceil
actually writes (a + b - 1) / b, which doesn't work at all for negative
values (and has a potential overflow issue).  This diff adds 4 templated
functions for performing integer division: divFloor, divCeil, divTrunc,
and divRoundAway.  They are not subject to unnecessary internal underflow
or overflow, and they work correctly across their entire input domain.

I did a bit of benchmarking across x86_64, arm64, and 32-bit ARM.
Only 32-bit ARM was different.  That's not surprising since it doesn't
have an integer division instruction, and the function that implements
integer division doesn't produce the remainder for free.  On 32-bit ARM
a branchful version that doesn't need the modulus is used.

Reviewed By: yfeldblum

Differential Revision: D3806743

fbshipit-source-id: c14c56717e96f135321920e64acbfe9dcb1fe039

5 years agoExpand DSched interface for managing auxiliary functions
Maged Michael [Wed, 7 Sep 2016 15:31:10 +0000 (08:31 -0700)]
Expand DSched interface for managing auxiliary functions

Changed DSched interface for managing auxiliary functions to allow separate auxiliary functions for single actions (applicable to the next shared access by a specific thread) and repeating actions (applicable to all subsequent shared accesses).

[Note: I have a dependent diff that depends on both this diff and the diff for dynamic MPMCQueue (/D3462592). I don't think I can submit a diff that depends on multiple diffs that haven't landed yet. So, I'll wait until this one lands.]

Reviewed By: djwatson

Differential Revision: D3792669

fbshipit-source-id: 52654fffda2dc905b19ff91f4459f15da11f7735

5 years agoRework the de-allocation guard on the munmap implementation
Christopher Dykes [Tue, 6 Sep 2016 23:22:33 +0000 (16:22 -0700)]
Rework the de-allocation guard on the munmap implementation

The previous version assumed that `RegionSize` would always be the full size of the allocation done by `VirtualAlloc`. However, `RegionSize` actually only includes the following pages that have the same attributes, so, if you change the access permissions via `mprotect`, the `RegionSize` would exclude that region, which is not what was intended.
This instead stores the length and a dummy magic value after the end of the requested allocation.

Reviewed By: yfeldblum

Differential Revision: D3812949

fbshipit-source-id: 53bbbcc371accbed08adaffa82fc082ec44f316f

5 years agoEliminate a string allocation in folly::SocketAddr and bug in char[] conversion
Chip Turner [Tue, 6 Sep 2016 18:27:41 +0000 (11:27 -0700)]
Eliminate a string allocation in folly::SocketAddr and bug in char[] conversion

We were doing an unnecessary and wasteful string conversion in the
SocketAddr codepath.  This eliminates it.  I also noticed we had an off-by-one
in the "convert string to char buffer" code path, so I added a test to confirm
the bug and fixed it.

Reviewed By: yfeldblum, meyering

Differential Revision: D3817959

fbshipit-source-id: 51fed8331ab23c0888a3d1f9e0cc9cea5ea8329b

5 years agoImplement more of the sockets API
Christopher Dykes [Tue, 6 Sep 2016 17:07:44 +0000 (10:07 -0700)]
Implement more of the sockets API

This gets the socket portability layer functional enough that most of the socket tests are passing, the few that are left are depending on specific error messages.
This also switches all of the additional overloads for some of the socket functions to forward to a single implementation, to make adjustments to these functions easier in the future. (most of the functions already needed adjustments and would have had to change regardless)

Reviewed By: yfeldblum

Differential Revision: D3814011

fbshipit-source-id: c6793ee74a91d9e164775a2d52c96f54b28b9f24

5 years agodelete const rvalue reference ctor of folly::Function v2016.09.05.00
Philip Pronin [Fri, 2 Sep 2016 16:51:19 +0000 (09:51 -0700)]
delete const rvalue reference ctor of folly::Function

This code compiles and is causing stack overflow at runtime:

using F = folly::Function<void()>;
void foo(F);

F bar;
auto baz = [bar = std::move(bar)] {

The bug there is that `baz` is missing `mutable` keyword, so when
constructing argument for `foo`, `F(const F&&)` is called, which is selecting
`template <typename Fun> F(Fun&&)` (where `Fun = F const`) and we end up in
an infinite `F<Fun>(Fun&&) <-> F<Fun>(Fun&&, SmallTag|HeapTag)` cycle.

This diff transforms this easy-to-make-bug into compile-time error.

Reviewed By: yfeldblum

Differential Revision: D3810269

fbshipit-source-id: f80a18ab02bd0715d692cf67c3c8943f557c2982

5 years agozstd is no longer in beta -- s/ZSTD_BETA/ZSTD/g
Chip Turner [Thu, 1 Sep 2016 23:40:08 +0000 (16:40 -0700)]
zstd is no longer in beta -- s/ZSTD_BETA/ZSTD/g


Need to override unit test failures after repeated landing failures unrelated
to the change.

Reviewed By: yfeldblum

Differential Revision: D3803301

fbshipit-source-id: b6581e3d2ff6baba57eb0f61b890a61107118bf7

5 years agoFix folly conversions for Clang with GCC5's libstdc++
Phil Willoughby [Thu, 1 Sep 2016 20:09:24 +0000 (13:09 -0700)]
Fix folly conversions for Clang with GCC5's libstdc++

Summary: Now builds correctly

Reviewed By: meyering

Differential Revision: D3763879

fbshipit-source-id: e272fbcc28a74fcf36b63a0135534c5063a7c4ee

5 years agoAdd ScopeGuard.h include in places where it is needed.
Eric Niebler [Thu, 1 Sep 2016 18:06:06 +0000 (11:06 -0700)]
Add ScopeGuard.h include in places where it is needed.

Summary: Fix code that is transitively including ScopeGuard.h via FBString.h.

Reviewed By: yfeldblum

Differential Revision: D3801328

fbshipit-source-id: 7e1dc37cd6dd3aa691dd1baf31385d3a8c51eed0

5 years agoProperly report observable returning nullptr
Andrii Grynenko [Wed, 31 Aug 2016 21:12:57 +0000 (14:12 -0700)]
Properly report observable returning nullptr

Summary: Make sure exception is propagated to caller (especially when Observer is just created).

Reviewed By: yfeldblum

Differential Revision: D3798991

fbshipit-source-id: cee8452cfd40dcfbaf0e2ae2c2ee628af0362c6e

5 years agoUpdate
Colin Ni [Wed, 31 Aug 2016 19:59:54 +0000 (12:59 -0700)]

Minor changes. Simplified the memory-handling explanation; also fixed some grammar issues and re-worded some confusing paragraphs. (I suggest reading the edited and unedited versions separately.)

Reviewed By: simpkins

Differential Revision: D3779296

Pulled By: Orvid

fbshipit-source-id: 24b086cbd0b67e4c592731aeec6a7ffc14ff0319

5 years agoupdate TimeseriesHistogram::rate to return a template type
Adam Simpkins [Wed, 31 Aug 2016 19:21:13 +0000 (12:21 -0700)]
update TimeseriesHistogram::rate to return a template type

Update TimeseriesHistogram::rate(int level) to have a configurable return type,
similar to the rate(TimeType start, TimeType end) function, as well as the
avg() functions.  I believe it was simply an oversight initially that this
version of rate did not have a configurable return type.

Since rate() and avg() are template methods, their full definitions should
really be available in TimeseriesHistogram.h rather than
TimeseriesHistogram-defs.h.  This also fixes that problem.  Most of the logic
in these functions isn't actually dependent on the return type, so that was
split out into separate non-template helper functions that are still in

Reviewed By: yfeldblum

Differential Revision: D3776017

fbshipit-source-id: 7deebf5b9ea1be143b5d772a15246706cb0cae80

5 years agoConsolidate namespace for using GFlags.
Zonr Chang [Wed, 31 Aug 2016 18:37:58 +0000 (11:37 -0700)]
Consolidate namespace for using GFlags.

Default namespace for GFlags has been changed from "google" to "gflags".


Reviewed By: yfeldblum

Differential Revision: D3787144

Pulled By: Orvid

fbshipit-source-id: db7bcaf4e2bcd46b022a48b17b50ef155570f296

5 years agoReverted commit D3755446
Philip Pronin [Wed, 31 Aug 2016 12:10:43 +0000 (05:10 -0700)]
Reverted commit D3755446

Move ThreadLocal object destruction to occur under the lock to avoid races.
This causes a few cascading changes - the Tag lock needs to be a recursive_mutex so
constructing a new object while destroying another st. Also, forking requires
a new mutex to avoid deadlocking on accessing a recursive_mutex across a fork()

Reviewed By: andriigrynenko

Differential Revision: D3755446

fbshipit-source-id: f1f1f92175eb39e77aaa2add6915e5c9bb68d0fb

5 years agoFix folly::SocketAddress::reset
Naizhi Li [Wed, 31 Aug 2016 05:56:57 +0000 (22:56 -0700)]
Fix folly::SocketAddress::reset

Currently reset does not actually reset it. This change
make sure it does the job.

Reviewed By: yfeldblum

Differential Revision: D3788869

fbshipit-source-id: 771a641e31bb3cf307e5bebb979b81a3d3feea55

5 years agoExpose SSL key materials to debug SSL
Gabriel Grise [Wed, 31 Aug 2016 02:55:12 +0000 (19:55 -0700)]
Expose SSL key materials to debug SSL

Summary: Adding two methods to export the parameters used to generate the key material   (key_block). These parameter can be used to decrypt a TLS session from a packet capture.

Reviewed By: anirudhvr

Differential Revision: D3687099

fbshipit-source-id: 04137f34dd32c387a1b7aec04b3ed6066f123a8e

5 years agoUse a normal variable if thread local is not available.
Michael Lee [Wed, 31 Aug 2016 02:29:59 +0000 (19:29 -0700)]
Use a normal variable if thread local is not available.

Summary: This should not affect correctness, but it means SharedMutex would need to share the variable across threads.

Reviewed By: yfeldblum

Differential Revision: D3621227

fbshipit-source-id: dc1baa7c47cd2d459cd0ef89451bcd7fd4d553aa

5 years agoFix joining threads twice when exiting EventBaseTest.RunInThread normally
Zonr Chang [Tue, 30 Aug 2016 22:46:50 +0000 (15:46 -0700)]
Fix joining threads twice when exiting EventBaseTest.RunInThread normally

10e9cd3 set up SCOPE_EXIT to join threads (wait for them to exit) on
exceptions. However, it is also called on normal exit. Remove the
original code at the end for joining threads.

This fixes EventBaseTest regression.

Reviewed By: yfeldblum

Differential Revision: D3792977

Pulled By: Orvid

fbshipit-source-id: e4b12202ae3e35fd942ad2ca1d8d405cf1a28f68

5 years agoDefine "__STDC_FORMAT_MACROS" for using PRI{u,x}64 in C++.
Zonr Chang [Tue, 30 Aug 2016 20:41:51 +0000 (13:41 -0700)]
Define "__STDC_FORMAT_MACROS" for using PRI{u,x}64 in C++.

Summary: Closes

Reviewed By: yfeldblum

Differential Revision: D3787151

Pulled By: Orvid

fbshipit-source-id: e255c9daea9917082f684e52e27984770c9f08b1

5 years agoMake type conversion from Expected<> to bool explicitly.
Zonr Chang [Tue, 30 Aug 2016 20:37:29 +0000 (13:37 -0700)]
Make type conversion from Expected<> to bool explicitly.

Summary: Closes

Reviewed By: yfeldblum

Differential Revision: D3787146

Pulled By: Orvid

fbshipit-source-id: 17433a256338bd107eec41e69af3eef58de3339b

5 years agoAdded a better check for openssl
Bruno Goncalves [Tue, 30 Aug 2016 20:37:13 +0000 (13:37 -0700)]
Added a better check for openssl

For some systems like my Fedora 23 is missing -lcrypto library to be together with -lssl

The following error ocurr running 'make check':
avoid undefined reference to symbol 'EVP_DigestInit_ex

Reviewed By: yfeldblum

Differential Revision: D3787160

Pulled By: Orvid

fbshipit-source-id: 8ff69d00c9c752d0a005d2a5e8d68e65347cc43b

5 years agoModified ref-qualifiers return type for Optional::value() and Optional::operator*
Anand Mazumdar [Tue, 30 Aug 2016 06:21:40 +0000 (23:21 -0700)]
Modified ref-qualifiers return type for Optional::value() and Optional::operator*

Optional::value() returns a temporary object when the object is an rvalue. This is different in semantics then what boost::optional/std::optional do.

The decision to make the copy or not should be up to the user and not the library. Consider an example:

void F(Optional<T> &&opt) {
  T&& t = std::move(opt).get();
  // I know `opt` is alive in this scope, I should be able to keep a rvalue ref to the internals
// if we were to return a `T`, that would actually return a new temporary.

void G(T&& t);
G(std::move(opt).get()); // This could have surprising behavior too !

This change modified the return type to be `T&&` and also introduces an extra overload for `const T&&`. Also, deleted two test-cases that assume the lifetime to be extended. This is a breaking change but this brings folly::Optional on parity with other siblings.

Reviewed By: ddrcoder

Differential Revision: D3714962

Pulled By: yfeldblum

fbshipit-source-id: 1794d51590062db4ad02fc8688cb28a06712c076

5 years agoFix ThreadCachedInt race condition
Dan Schatzberg [Mon, 29 Aug 2016 23:42:44 +0000 (16:42 -0700)]
Fix ThreadCachedInt race condition

Move ThreadLocal object destruction to occur under the lock to avoid races.
This causes a few cascading changes - the Tag lock needs to be a recursive_mutex so
constructing a new object while destroying another st. Also, forking requires
a new mutex to avoid deadlocking on accessing a recursive_mutex across a fork()

Reviewed By: andriigrynenko

Differential Revision: D3755446

fbshipit-source-id: bb4c4f29bab98d763490df29b460066f124303e0

5 years agoResolve fibers-futures dependency
Andrii Grynenko [Mon, 29 Aug 2016 23:02:38 +0000 (16:02 -0700)]
Resolve fibers-futures dependency

Reviewed By: mzlee

Differential Revision: D3780312

fbshipit-source-id: c42c8f0a06b82520ee1b46f105a2a85ad524c442

5 years agogen::dereference should perfectly-forward unwrapped values
Yedidya Feldblum [Mon, 29 Aug 2016 21:57:05 +0000 (14:57 -0700)]
gen::dereference should perfectly-forward unwrapped values

[Folly] `gen::dereference` should perfectly-forward unwrapped values.

The problem comes in when the wrapped value is not actually a pointer, but is actually an rvalue-ref to some other kind of wrapper type with `Inner&& operator*() &&`. In such cases, the compiler emits a type mismatch error that it cannot cast `Inner` to `Inner&&`, with the errors originating in `Dereference::foreach` and `Dereference::apply`.

Fixes a couple other missing-forwarding and extra-forwarding bugs.

Reviewed By: ddrcoder

Differential Revision: D3776617

fbshipit-source-id: 6926fc18244a572846b22d428bd407d37fb20aa1

5 years agoUse Synchronized in RequestContext
Yedidya Feldblum [Mon, 29 Aug 2016 21:01:15 +0000 (14:01 -0700)]
Use Synchronized in RequestContext

[Folly] Use `Synchronized` in `RequestContext`.

Because we can. And it makes the code a tad simpler and also enforces access correctness a tad.

Also use `folly::make_unique` in `RequestContextTest` to keep balance between the explicit `new` and `delete` ops.

Reviewed By: markisaa

Differential Revision: D3781115

fbshipit-source-id: 63b41ddd8009e9546e3be5f89bdd23a4d791105c

5 years agoAdd default constructor to folly::IOBuf::Iterator.
Nick Terrell [Mon, 29 Aug 2016 16:50:36 +0000 (09:50 -0700)]
Add default constructor to folly::IOBuf::Iterator.

Iterators must be default constructible.
folly::IOBuf can now be used as a range in ranges-v3.

Reviewed By: yfeldblum

Differential Revision: D3782536

fbshipit-source-id: 854813b4e3336aba50048649e6ae7b375d49e382

5 years agoFix up TokenBucket.h in Makefile headers
Jon Maltiel Swenson [Mon, 29 Aug 2016 16:30:52 +0000 (09:30 -0700)]
Fix up TokenBucket.h in Makefile headers

Summary: Title.

Reviewed By: andreazevedo

Differential Revision: D3785212

fbshipit-source-id: c551e0367196cbc68cf39a5d40f2b324883fcdf4

5 years agoGeneralized and polished folly::TokenBucket v2016.08.29.00
Philipp Unterbrunner [Mon, 29 Aug 2016 13:27:23 +0000 (06:27 -0700)]
Generalized and polished folly::TokenBucket

Summary: Added support for user-defined clock classes, improved comments, and removed part of the std::atomics use that had no effect on thread-safety.

Reviewed By: yfeldblum

Differential Revision: D3708378

fbshipit-source-id: 1a933c3707c12311584a3b33afd773ee91577167

5 years agoAdded fiber-compatible semaphore.
Lee Howes [Fri, 26 Aug 2016 22:58:45 +0000 (15:58 -0700)]
Added fiber-compatible semaphore.

Summary: Adds a standard semaphore type with signal and wait methods that is safe to use in both multi-threaded contexts and from fibers.

Reviewed By: andriigrynenko

Differential Revision: D3778943

fbshipit-source-id: 6997f1fb870739e07f982399dbebfd8b3e45daa2

5 years agoUse folly::Random and ensure M_PI is defined
Christopher Dykes [Fri, 26 Aug 2016 20:11:03 +0000 (13:11 -0700)]
Use folly::Random and ensure M_PI is defined

Summary: Because MSVC doesn't have `random()` and `M_PI` is guarded by a pre-processor macro.

Reviewed By: yfeldblum

Differential Revision: D3767159

fbshipit-source-id: fcb1c41cd75925455c76efad38423cd52bd98aeb

5 years agocollectOne
Yunqiao Zhang [Fri, 26 Aug 2016 19:53:24 +0000 (12:53 -0700)]

The resultant future of collectOne will be fulfilled when the first one of
the future in the list completes without exception. If all input futures throws
exception, the resultant future will get the last exception that was thrown.

Reviewed By: andriigrynenko

Differential Revision: D3764760

fbshipit-source-id: 76484254e35182eddc8266865853d65c28170f82

5 years agoAdd CachelinePadded<T> to folly.
David Goldblatt [Fri, 26 Aug 2016 19:40:58 +0000 (12:40 -0700)]
Add CachelinePadded<T> to folly.

This class allows easy insertion of padding bytes after an object to ensure
that two cacheline-padded elements of an array don't engage in false sharing.

Reviewed By: yfeldblum

Differential Revision: D3485144

fbshipit-source-id: a3ece1e34566b20f94ff9f66532b2386ab19a9b1

5 years agoRemove now-useless FOLLY_MALLOC_H_
Nicholas Ormrod [Fri, 26 Aug 2016 18:41:38 +0000 (11:41 -0700)]
Remove now-useless FOLLY_MALLOC_H_

Malloc.h is intentionally included multiple times, once from folly and once from libstdc++. The current ###pragma once## implicitly allows this to happen. FBString.h has an undef for Malloc.h's include guard, which originally accomplished this goal.

The undefing code is presently moot, since its functionality has been replaced by ###pragma once##. Remove it.

I noticed this when ott was copying FBString over to libstdc++ in D3757853.
The diff that switched the include guards to pragmas was a codemod, and was not specific to fbstring. D3054492

Reviewed By: ot

Differential Revision: D3758119

fbshipit-source-id: e796d039a031d5f842ed39bf55a6b1aeb2686bc4

5 years agoUse intrusive base hook rather than a member hook
Christopher Dykes [Fri, 26 Aug 2016 17:51:13 +0000 (10:51 -0700)]
Use intrusive base hook rather than a member hook

Summary: Because MSVC is not happy about the member hook when used in complex inheritence scenarios.

Reviewed By: yfeldblum

Differential Revision: D3774513

fbshipit-source-id: 9e1ef8dd76d966de339f8486ff1a1d0ab1571849

5 years agoSuppress an abort in FileUtilTest under MSVC
Christopher Dykes [Thu, 25 Aug 2016 21:53:29 +0000 (14:53 -0700)]
Suppress an abort in FileUtilTest under MSVC

Summary: Because MSVC is right, this shouldn't be happening, but we test weird things.

Reviewed By: yfeldblum

Differential Revision: D3773486

fbshipit-source-id: 50f28bf0ac8ff53f63231b4f6f9ce050b509b464

5 years agolseek returns the new offset, not 0
Christopher Dykes [Thu, 25 Aug 2016 21:52:59 +0000 (14:52 -0700)]
lseek returns the new offset, not 0

With how this was written it would fail if the call to `lseek` failed, or if it succeeded, unless it was setting the length to 0, in which case it would succeed.
This makes it work right.

Reviewed By: yfeldblum

Differential Revision: D3773311

fbshipit-source-id: 3dc94502d0c4259f6f2766b4c0903c081d7c36ab

5 years agoAllow locking nullptr if the length is 0
Christopher Dykes [Thu, 25 Aug 2016 21:52:05 +0000 (14:52 -0700)]
Allow locking nullptr if the length is 0

Because apparently this is valid (and we test it).
This also adds a check in mmap to make sure we aren't passing an invalid handle to `MapViewOfFileEx`.

Reviewed By: yfeldblum

Differential Revision: D3772853

fbshipit-source-id: 11593997a3fb12b7b391c5e52661060b71341aef

5 years agoFix SimplerObservable build with -Werror=unused-local-typedefs
Andrii Grynenko [Thu, 25 Aug 2016 18:23:20 +0000 (11:23 -0700)]
Fix SimplerObservable build with -Werror=unused-local-typedefs

Reviewed By: yfeldblum

Differential Revision: D3765642

fbshipit-source-id: 9b1cc4007c553da5082799fa4ed0af8f7850ec8d

5 years agoDon't pass the single quote format parameter when formatting things
Christopher Dykes [Thu, 25 Aug 2016 17:18:25 +0000 (10:18 -0700)]
Don't pass the single quote format parameter when formatting things

We are formatting an integer value, so it is doing absolutely nothing at all, as confirmed by the version I implemented for MSVC, which does output the value with digit separators, which fails the unit tests because the unit test expects the digit separators to not be present. What's more, whe specifically assert that the user has not requested digit separators in the format string.
This also kills the MSVC special case entirely because it's not needed.

Reviewed By: yfeldblum

Differential Revision: D3768405

fbshipit-source-id: 388cd0ca9699e257c71798b9bf355aa651748e60

5 years agoEasy: Mark folly::static_function_deleter::operator() const
Felix Handte [Thu, 25 Aug 2016 00:42:37 +0000 (17:42 -0700)]
Easy: Mark folly::static_function_deleter::operator() const

First, folly::static_function_deleter::operator() is in fact a const
operation, so this should be a reasonable change. The motivation for this is
to make folly::ThreadLocalPtr and folly::static_function_deleter play nice with
each other.

Minimal example:

void deleter(int* ptr) {

int main(int argc, char* argv[]) {
  folly::ThreadLocalPtr<int> tl;
  tl.reset(std::unique_ptr<int, folly::static_function_deleter<int, deleter>>(
      new int(0)));
  return 0;

Currently produces:
folly/ThreadLocal.h:207:7: error: no matching function for call to object of type 'const folly::static_function_deleter<int, &deleter>'
Test.cpp:10:6: note: in instantiation of function template specialization 'folly::ThreadLocalPtr<int, void>::reset<int, folly::static_function_deleter<int, &deleter>, void>' requested here
  tl.reset(std::unique_ptr<int, folly::static_function_deleter<int, deleter>>(new int(0)));
folly/Memory.h:91:8: note: candidate function not viable: 'this' argument has type 'const folly::static_function_deleter<int, &deleter>', but method is not marked const
  void operator()(T* t) { f(t); }
1 error generated.

With the fix, the build succeeds.

Reviewed By: yfeldblum

Differential Revision: D3764624

fbshipit-source-id: c28c791b79f1415704c205c36bfda2d888d6c010

5 years agoFix a typo in ScopeGuard.h
Stephane Sezer [Wed, 24 Aug 2016 23:26:18 +0000 (16:26 -0700)]
Fix a typo in ScopeGuard.h

Summary: Closes

Reviewed By: yfeldblum

Differential Revision: D3765911

Pulled By: Orvid

fbshipit-source-id: 30de0b99ec622471e52a775835b5a0564437febc

5 years agoDon't assume unsigned long is 64-bit
Christopher Dykes [Wed, 24 Aug 2016 23:25:11 +0000 (16:25 -0700)]
Don't assume unsigned long is 64-bit

Summary: Because it isn't. Not on MSVC anyways.

Reviewed By: yfeldblum

Differential Revision: D3767326

fbshipit-source-id: 0f9421773819999f99be38ae0646ae3e81bde4e4

5 years agoUse cbegin and cend in static reflection container traits
Yedidya Feldblum [Wed, 24 Aug 2016 21:25:31 +0000 (14:25 -0700)]
Use cbegin and cend in static reflection container traits

[Thrift] Use `cbegin` and `cend` in static reflection container traits.

These are functions that all the C++ standard library containers have.

And add the two methods to `folly::sorted_vector_set` and `folly::sorted_vector_map`.

Reviewed By: juchem

Differential Revision: D3763187

fbshipit-source-id: 9d467a1cf391206eba671f454da428323c4a566a

5 years agoAllow building the NotificationQueue tests without event FDs
Christopher Dykes [Wed, 24 Aug 2016 20:59:55 +0000 (13:59 -0700)]
Allow building the NotificationQueue tests without event FDs

Because event file descriptors don't exist on Windows.
This also disables the fork() test on Windows, as we don't have fork().

Reviewed By: yfeldblum

Differential Revision: D3766266

fbshipit-source-id: af8d81615957a3d7bdfcb204afbf648723921f76

5 years agoExplicit typecast of unused constant to make 32 bit compiler happy.
Michael Callahan [Wed, 24 Aug 2016 20:48:02 +0000 (13:48 -0700)]
Explicit typecast of unused constant to make 32 bit compiler happy.

GCC 5.3 is confused that we might be trying to assign a 64
bit value to a 32 bit size_t although we are clearly not.  Just
typecast the confusion away.

Reviewed By: yfeldblum

Differential Revision: D3762839

fbshipit-source-id: 1b705c4e70b782fdb106844fee52ab25aa038acd

5 years agoCycle detection
Andrii Grynenko [Wed, 24 Aug 2016 18:24:33 +0000 (11:24 -0700)]
Cycle detection

1. This implements a GraphCycleDetector which can check if newly added edge belongs to a cycle in a directed graph. GraphCycleDetector is used to detect cycles between Observers when creator function is run.
2. This also fixes a bug where new dependencies could be saved even if Observer creator failed.

Reviewed By: yfeldblum

Differential Revision: D3746743

fbshipit-source-id: 99d10446c56fa4d8f7485f38309e8a282cd21bdf

5 years agofolly: fix enumeral vs non-enumeral type in conditional expression
Huapeng Zhou [Wed, 24 Aug 2016 17:07:45 +0000 (10:07 -0700)]
folly: fix enumeral vs non-enumeral type in conditional expression

mcrouter/lib/carbon/CarbonProtocolReader.h:143:49:   required from here
folly/small_vector.h:441:38: error: enumeral and non-enumeral type in conditional expression [-Werror]
cc1plus: all warnings being treated as errors

Reviewed By: yfeldblum

Differential Revision: D3763073

fbshipit-source-id: da490e91cdb12dacf37d71c1796239bdc361a31b

5 years agoPrevent compiler warning about mixing enumeral and non-enumeral types in ternary...
Jon Maltiel Swenson [Wed, 24 Aug 2016 16:35:48 +0000 (09:35 -0700)]
Prevent compiler warning about mixing enumeral and non-enumeral types in ternary expression

The GCC warning "warning: enumeral and non-enumeral type in conditional expression"
is logged a lot in many builds since lots of code depends on small_vector.  This diff
should prevent those warnings.

Reviewed By: yfeldblum

Differential Revision: D3762835

fbshipit-source-id: 49831e4364e716592287c05d1dbf1912326324f6

5 years agoFix a folly build failure with GCC5
Phil Willoughby [Wed, 24 Aug 2016 08:05:41 +0000 (01:05 -0700)]
Fix a folly build failure with GCC5

I tested it doesn't break the gcc 4.9 and clang builds - I didn't try any other versions of anything.


Reviewed By: meyering

Differential Revision: D3757152

fbshipit-source-id: a126cd31cb89057dececfed1b265f452ee29fbff

5 years agoAsyncSSLSocket getRawBytes fix
Heng Hong Lee [Tue, 23 Aug 2016 23:31:17 +0000 (16:31 -0700)]
AsyncSSLSocket getRawBytes fix

While adding logging around our socket and looking into `AsyncSSLSocket` it seems like the data that is actually written into the socket is not correctly attributed.  I
added logs and printed out what happens on the socket layer, P56563098 in this paste you can see that the `[fishhook]` logs are actually those added in D3698728.

Those are the actual bytes written onto the socket, in the paste, the bytes that are written by the AsyncSocket::bioWrite method are
the ones that actually get written onto the socket. some of the bytes written into the bio comes from bf_buff.c and bss_mem.c which are not eventually attributed to a
socket message and will be incorrectly counted when getRawBytesWritten/Read invoked on AsyncSSLSocket.cpp

Unfortunately/Fortunately this issue is not symmetrical and does not manifest in the getRawBytesReceived in AsyncSSLSocket, reading the bio for read bytes
correctly attributes the actual number of bytes written on the socket. moreover, pulling the asyncsocket data for getRawBytesRead doesnt give the full read bytes on
wire because SSL_connect and SSL_read dont return the number of bytes they read but return the number of bytes without the TLS bytes used.

siyengar seems like a right person to look at this. so adding him here.

Would love to discuss more about this and am open to iterating more on this solution

Reviewed By: knekritz

Differential Revision: D3698744

fbshipit-source-id: 541aa478778b9607f51db194fcbfe28bd23c737f

5 years agoMake it build on Ubuntu 16.04
Meng Zhang [Tue, 23 Aug 2016 22:17:52 +0000 (15:17 -0700)]
Make it build on Ubuntu 16.04

include linux/membarrier.h if it is available.

Reviewed By: lukenels

Differential Revision: D3714952

Pulled By: Orvid

fbshipit-source-id: 8c85756af2cb132152b2182816becfea138f0149

5 years agoMake fbstring::assign() tight, and simplify operator=() and append()
Giuseppe Ottaviano [Tue, 23 Aug 2016 20:00:48 +0000 (13:00 -0700)]
Make fbstring::assign() tight, and simplify operator=() and append()

`fbstring::assign()` uses `append()`, which triggers exponential growth, but it's preferable that `assign()` behaves like the `fbstring(const char*, size_t)` constructor, which is tight.

Also rewrite `operator=()` in terms of `assign()`, to avoid duplicating the logic, and refactor the logic of `append()` for the aliasing case so that it uses the same expansion operation as the non-aliasing case.

Reviewed By: luciang

Differential Revision: D3754932

fbshipit-source-id: 5423f2a360b4268b6a05dd0ae9d2fe5bd1eb855d

5 years agoMake Malloc.h self-contained again
Giuseppe Ottaviano [Tue, 23 Aug 2016 17:14:40 +0000 (10:14 -0700)]
Make Malloc.h self-contained again

Summary: D3743475 adds to `Malloc.h` a dependency on another folly header, which breaks it when used in stand-alone mode. This diff moves the include to the right `#ifdef` section.

Reviewed By: Gownta

Differential Revision: D3757819

fbshipit-source-id: 71664ca6a3a47b6e4449a4ef603fedf052c5df3b

5 years agoFixup a reference to detail/FunctionalExcept.cpp
Christopher Dykes [Mon, 22 Aug 2016 22:55:49 +0000 (15:55 -0700)]
Fixup a reference to detail/FunctionalExcept.cpp

Summary: I missed this reference when I moved it to the portability folder. It is already being compiled at its new location.

Reviewed By: yfeldblum

Differential Revision: D3754227

fbshipit-source-id: 357b0c26ddbcefdc7640f6a334150abba90ed711

5 years agoUse a constexpr char* under MSVC for the constexpr StringPiece test
Christopher Dykes [Mon, 22 Aug 2016 21:15:39 +0000 (14:15 -0700)]
Use a constexpr char* under MSVC for the constexpr StringPiece test

Summary: MSVC can't handle the array as a constexpr value, but works fine with the char*.

Reviewed By: yfeldblum

Differential Revision: D3705189

fbshipit-source-id: e8208b3f2831a720095641f0e1e72ac63ed845a0

5 years agoMore fixup of the build on Mac for HHVM
Christopher Dykes [Mon, 22 Aug 2016 16:52:34 +0000 (09:52 -0700)]
More fixup of the build on Mac for HHVM

Summary: `std::__throw_bad_alloc()` is defined in `<new>` on OSX, so bring back the `#ifdef` guards that were there previously.

Reviewed By: markw65

Differential Revision: D3749714

fbshipit-source-id: 0338a4cece928fce0b9d33d41c17cfa99a319abe

5 years agofix(FBString): fix bugs v2016.08.22.00
pp__qq [Mon, 22 Aug 2016 04:50:45 +0000 (21:50 -0700)]
fix(FBString): fix bugs

fix(FBString): compile error on instantiate `basic_fbstring` with a `Storage` that is not `fbstring_core<E>`

Reviewed By: ot

Differential Revision: D3714957

Pulled By: yfeldblum

fbshipit-source-id: 1c5d2538b674049f7e1872a0b623ec330dc8d7b2

5 years agoGet ObserverManager compiling under MSVC
Christopher Dykes [Sun, 21 Aug 2016 17:02:55 +0000 (10:02 -0700)]
Get ObserverManager compiling under MSVC

Summary: It only needed a minor tweak, but the change still had to be made :(

Reviewed By: andriigrynenko

Differential Revision: D3745021

fbshipit-source-id: 286c6db706c3571842006537c6b17f506609e51d

5 years agoFix folly::to<> under MSVC
Christopher Dykes [Sun, 21 Aug 2016 17:02:29 +0000 (10:02 -0700)]
Fix folly::to<> under MSVC

Summary: MSVC didn't like the conversion to using `Expected`, and, after `Expected` was fixed to work under MSVC, conv still needs more changes. Specifically MSVC doesn't understand the single-instantiation form of getting the last element, so we have to switch to a tuple-backed version instead, which requires more template instantions, but produces the same performance at runtime.

Reviewed By: ericniebler

Differential Revision: D3744063

fbshipit-source-id: affcab1574c721d8b9529784d7ca2a46233d9935

5 years agoSuggestion of fixing folly::to<std::string>(double/float) performance issue
Bi Xue [Sun, 21 Aug 2016 04:10:36 +0000 (21:10 -0700)]
Suggestion of fixing folly::to<std::string>(double/float) performance issue

When calling folly::to<SomeString>(double), generic implementation will
firstly reserve 24 (or 25 when negative value) bytes. This will introduce
a malloc call for most of mainstream string implementation.

But for most cases, a floating point doesn't need 24 (or 25) bytes to
be converted as a string.

This diff try to introduce a special version which does not do string reserve.

Reviewed By: ericniebler

Differential Revision: D3728171

fbshipit-source-id: d70ead396ad6c8d0df1f542c5516f7534e82cb97

5 years agoAn intro to the upgrade mutex in the Synchronized docs
Yedidya Feldblum [Sat, 20 Aug 2016 06:17:28 +0000 (23:17 -0700)]
An intro to the upgrade mutex in the Synchronized docs

[Folly] An intro to the upgrade mutex in the `Synchronized` docs.

Describes what the upgrade mutex is. Extends the existing docs which describe `Synchronized` and `LockPtr` interface and behavior in the presence of an upgrade mutex.

Reviewed By: snarkmaster, simpkins

Differential Revision: D3746177

fbshipit-source-id: 68b0570a36cc1f4393d5ccca535efa02752ca11d

5 years agofix flaky EventHandlerTest
Aravind Anbudurai [Fri, 19 Aug 2016 23:33:50 +0000 (16:33 -0700)]
fix flaky EventHandlerTest

Summary: Lets the kernel choose the port instead of hardcoding it.

Reviewed By: djwatson

Differential Revision: D3745911

fbshipit-source-id: d9680ec286e8015abb9274c30b572ff1d91548ce

5 years agoRemove a dead comment in folly/test/SynchronizedTest.cpp
Yedidya Feldblum [Fri, 19 Aug 2016 23:27:36 +0000 (16:27 -0700)]
Remove a dead comment in folly/test/SynchronizedTest.cpp

Summary: [Folly] Remove a dead comment in `folly/test/SynchronizedTest.cpp`.

Reviewed By: simpkins

Differential Revision: D3745609

fbshipit-source-id: acdbd3eaa6d947213b72fe13cec0291545a60c87

5 years agoInclude <unordered_set> in experimental/observer/detail/Core.h
Christopher Dykes [Fri, 19 Aug 2016 21:53:37 +0000 (14:53 -0700)]
Include <unordered_set> in experimental/observer/detail/Core.h

Summary: Because it, and a whole bunch of other things, are used in the header but not properly included.

Reviewed By: yfeldblum

Differential Revision: D3744653

fbshipit-source-id: c10dbb83109200b6186b8ed5ef0d2447d4200f69

5 years agoMark the long overload of digits_to as noexcept
Christopher Dykes [Fri, 19 Aug 2016 20:51:26 +0000 (13:51 -0700)]
Mark the long overload of digits_to as noexcept

Summary: Well, the explicit instantion for it anyways. The extern template declaration for it declares it as `noexcept`, so MSVC generates an error because the explicit instantiation is not also marked as `noexcept`

Reviewed By: yfeldblum, ericniebler

Differential Revision: D3744090

fbshipit-source-id: 4e756b2761c23a436097a6131b9b2ecd59faf4f9