folly.git
7 years agoFix the pkg-config filter
Michael Steinert [Wed, 15 Mar 2017 16:23:02 +0000 (09:23 -0700)]
Fix the pkg-config filter

Summary:
I encountered a situation where "pkg-config --libs openssl" included
some linker flags, i.e.

    $ pkg-config --libs openssl
    -Wl,-z,relro -lssl -lcrypto -ldl -lz

This resulted in these linker flags being passed to sed. This change
filters out the linker flags from the pkg-config output.
Closes https://github.com/facebook/folly/pull/559

Reviewed By: pixelb

Differential Revision: D4700120

Pulled By: yfeldblum

fbshipit-source-id: 530d524caf965919418d19a1df53d2b418f67717

7 years agoAdd String Support to Compression Codec
Felix Handte [Wed, 15 Mar 2017 15:48:25 +0000 (08:48 -0700)]
Add String Support to Compression Codec

Summary:
This diff adds string-based compress and uncompress support to
folly::io::Codec. By default, the string API is implemented by translating to
and from IOBufs. Room is left for individual implementations to override that
behavior and go straight to the underlying library.

Reviewed By: yfeldblum

Differential Revision: D4690770

fbshipit-source-id: 7e2615da9add8328654505cdda6a2c79583b4558

7 years agoIgnore the generated file folly/libfolly.pc
Michael Steinert [Tue, 14 Mar 2017 17:54:57 +0000 (10:54 -0700)]
Ignore the generated file folly/libfolly.pc

Summary: Closes https://github.com/facebook/folly/pull/560

Reviewed By: pixelb

Differential Revision: D4700198

Pulled By: yfeldblum

fbshipit-source-id: 5e251e5719c6783c9f8a4fd9bab357ed362340c0

7 years agoFix the --disable-follytestmain option
Michael Steinert [Tue, 14 Mar 2017 14:35:08 +0000 (07:35 -0700)]
Fix the --disable-follytestmain option

Summary:
This option is useful when compiling on systems that do not want to
include libunwind.
Closes https://github.com/facebook/folly/pull/558

Reviewed By: andrewjcg

Differential Revision: D4701878

Pulled By: yfeldblum

fbshipit-source-id: 61765e2e04ea1da510f06d7805c2c54381aa771e

7 years agosmall_vector should not require a copy-ctr from its element type unless it needs it.
Dmitry Pleshkov [Tue, 14 Mar 2017 06:33:57 +0000 (23:33 -0700)]
small_vector should not require a copy-ctr from its element type unless it needs it.

Summary:
Use default constructor directly, instead of copying the element introduced by default parameter value.
This behavior is compatible with `std::vector`.

Reviewed By: ot, bixue2010, luciang

Differential Revision: D4700611

fbshipit-source-id: d3bd82c46c2857f5abcbec5dd2a64aaa85eaefe4

7 years agoAdd a way to determine if a compression codec is supported at runtime
Christopher Dykes [Mon, 13 Mar 2017 21:47:40 +0000 (14:47 -0700)]
Add a way to determine if a compression codec is supported at runtime

Summary: Folly supports being compiled without all of the compression libraries, so we need a way to determine at runtime what libraries Folly has been compiled to support.

Reviewed By: yfeldblum

Differential Revision: D4692556

fbshipit-source-id: 0ec459db70a4b1d64f1e07c87a1f51ae584bccd0

7 years agoFill buffer before allocating more
Nick Terrell [Mon, 13 Mar 2017 19:50:55 +0000 (12:50 -0700)]
Fill buffer before allocating more

Summary:
Modify `QueueAppender::pushAtMost(const uint8_t*, size_t)` to fill the end of
the current buffer before it allocates another.

Reviewed By: yfeldblum

Differential Revision: D4687832

fbshipit-source-id: 7c21f4359da9f0cf26d9577abddfd640b0c877e9

7 years agoget_ptr(mapOfMaps, key, key...)
Tom Jackson [Mon, 13 Mar 2017 18:09:00 +0000 (11:09 -0700)]
get_ptr(mapOfMaps, key, key...)

Summary:
Since this is a not-uncommon pattern now:

```lang=cpp
if (auto found1 = get_ptr(m, k1)) {
  if (auto found2 = get_ptr(*found, k2)) {
    return use(*found2);
  }
}
```

This diff enables:

```lang=cpp
if (auto found = get_ptr(m, k1, k2)) {
  return use(*found);
}

```

Note that it works for const and non-const maps, returning a correspondingly mutable pointer.

Reviewed By: luciang, yfeldblum

Differential Revision: D3812701

fbshipit-source-id: 77ace9f5dca6cdc4cae7e6dfb9e5fc1075b5b571

7 years agoUpdate folly::Singleton documentation block
Phil Willoughby [Mon, 13 Mar 2017 10:03:16 +0000 (03:03 -0700)]
Update folly::Singleton documentation block

Summary:
With more experience of using folly::Singleton in our code we have found new
recommendations for how it should be used. This change updates the
documentation block to include those recommendations.

Reviewed By: yfeldblum

Differential Revision: D4688700

fbshipit-source-id: 40350c2330084a28f1dc0744176d0ac8a0e64d39

7 years agoMake EventBaseLoopController only support VirtualEventBase v2017.03.13.00
Andrii Grynenko [Sat, 11 Mar 2017 01:06:58 +0000 (17:06 -0800)]
Make EventBaseLoopController only support VirtualEventBase

Summary: EventBase support is achieved through default VirtualEventBase mechanism.

Reviewed By: yfeldblum

Differential Revision: D4685948

fbshipit-source-id: 15c8c789a55776984aa9087455e3f7b79d7604ad

7 years agoFix data race in IndexedMemPool
Maged Michael [Fri, 10 Mar 2017 23:34:07 +0000 (15:34 -0800)]
Fix data race in IndexedMemPool

Summary:
IndexedMemPool uses regular memory for the global and local next links. There can be concurrent reads and writes of such links. In order to avoid C++ undefined behavior due to the data races, these links should be atomic. Relaxed loads and stores are sufficient for correctness.

Depends on D4680286

Reviewed By: nbronson

Differential Revision: D4680317

fbshipit-source-id: 27c0d888e08bd5d862aee4c6dc6ee393033b32e3

7 years agoFix problems with clock_gettime and OSX < 10.12 + XCode 8
Christopher Dykes [Fri, 10 Mar 2017 21:52:11 +0000 (13:52 -0800)]
Fix problems with clock_gettime and OSX < 10.12 + XCode 8

Summary: It's a pain, so rely on macros to figure a few things about the build configuration.

Reviewed By: yfeldblum

Differential Revision: D4229128

fbshipit-source-id: 78b414c21cae6ba51ade2ab75b117cccad5c608c

7 years agoUse VirtualEventBase in mcrouter
Andrii Grynenko [Fri, 10 Mar 2017 19:40:59 +0000 (11:40 -0800)]
Use VirtualEventBase in mcrouter

Summary:
This switches mcrouter to use VirtualEventBase as the unified way of managing its threads. This allows multiple mcrouters to be run on the same thread-pool, without having to keep separate shutdown logic.

As an immediate win - this eliminates any special shutdown logic from tcc and standalone mcrouter.

Reviewed By: jmswen

Differential Revision: D4570982

fbshipit-source-id: 3974279005150e3e1db16230fde6b5951ae016b9

7 years agoFix use after move in AsyncSSLSocket constructor.
Kyle Nekritz [Fri, 10 Mar 2017 17:36:52 +0000 (09:36 -0800)]
Fix use after move in AsyncSSLSocket constructor.

Summary: The new AsyncSocket is already constructed at this point so just use the event base from it.

Reviewed By: djwatson

Differential Revision: D4689395

fbshipit-source-id: aac898c1c6e6e0e0ffcb20b16bbf2b842cc31d54

7 years agoImplementing a callback interface for folly::AsyncSocket allowing to supply an ancill...
Maxim Georgiev [Fri, 10 Mar 2017 06:19:15 +0000 (22:19 -0800)]
Implementing a callback interface for folly::AsyncSocket allowing to supply an ancillary data buffer with msghdr structure to sendmsg() system call

Summary: Implementing a callback interface for folly::AsyncSocket allowing to supply an ancillary data buffer with msghdr structure to sendmsg() system call.

Reviewed By: afrind

Differential Revision: D4422168

fbshipit-source-id: 29a23b05f704aff796d368f4ac9514c49b7ce578

7 years agoRemove runInLoop
Andrii Grynenko [Fri, 10 Mar 2017 04:34:14 +0000 (20:34 -0800)]
Remove runInLoop

Summary: Existing runInLoop implementation was generally not safe (it did not capture the KeepAlive token). runInLoop() is mostly used in library code though, and often times it's ok not capture the KeepAlive token. It's better to just have the caller make the decision, rather then keep runInLoop() and make it always capture the KeepAlive token.

Reviewed By: yfeldblum

Differential Revision: D4684025

fbshipit-source-id: 65907bbe9c774e2a7b92580d0c0387d346b495d5

7 years agoteach gtest how to pretty-print StringPiece values
Adam Simpkins [Fri, 10 Mar 2017 01:54:21 +0000 (17:54 -0800)]
teach gtest how to pretty-print StringPiece values

Summary:
Even though an ostream operator<<() is defined for StringPiece, gtest doesn't
use it when printing StringPiece values in test failure messages.  Because
StringPiece defines a nested iterator type gtest instead treats StringPiece as
a container, and prints each of its elements (characters) one-by-one.  This is
fairly awkward to read.

This diff defines an explicit PrintTo() function for StringPiece in
folly/test/TestUtils.h.  This makes gtest print StringPiece values nicely if
you include TestUtils.h in your test sources.

Reviewed By: yfeldblum

Differential Revision: D4672257

fbshipit-source-id: 4b39ccc116e5382c29c37c2abe879293d310faf5

7 years agoDetect data race in IndexedMemPool
Maged Michael [Thu, 9 Mar 2017 19:21:50 +0000 (11:21 -0800)]
Detect data race in IndexedMemPool

Summary: IndexedMemPool uses regular memory for the global and local next links. There is a data race as there can be concurrent reads and writes of such links. Added a test for data races.

Reviewed By: nbronson

Differential Revision: D4680286

fbshipit-source-id: 9ef3ed439b9df07d69afe570e516c146caa53a6c

7 years agoReplace MSG_PEEK with a pre-received data interface.
Kyle Nekritz [Thu, 9 Mar 2017 16:25:27 +0000 (08:25 -0800)]
Replace MSG_PEEK with a pre-received data interface.

Summary: MSG_PEEK was difficult if not impossible to use well since we do not provide a way wait for more data to arrive. If you are using setPeek on AsyncSocket, and you do not receive the amount of data you want, you must either abandon your peek attempt, or spin around the event base waiting for more data. This diff replaces the peek interface on AsyncSocket with a pre-received data interface, allowing users to insert data back onto the front of connections after reading some data in another layer.

Reviewed By: djwatson

Differential Revision: D4626315

fbshipit-source-id: c552e64f5b3ac9e40ea3358d65b4b9db848f5d74

7 years agoFixes to always call Singleton's registrationComplete()/initFacebook()
Andrii Grynenko [Wed, 8 Mar 2017 22:13:08 +0000 (14:13 -0800)]
Fixes to always call Singleton's registrationComplete()/initFacebook()

Reviewed By: yfeldblum

Differential Revision: D4671818

fbshipit-source-id: 80154adfef238f0be62da12fb843e22de1ae65ca

7 years agoMark non-captured constexpr variable as static
Christopher Dykes [Wed, 8 Mar 2017 20:38:27 +0000 (12:38 -0800)]
Mark non-captured constexpr variable as static

Summary:
GCC & Clang allow the use of a non-captured constexpr local within a lambda, MSVC does not.

All 3 compilers allow this when the local has static storage.

Reviewed By: yfeldblum

Differential Revision: D4675147

fbshipit-source-id: 3e07eb77731cbd3ebda6aee4b0b57ac19ac34de5

7 years agoDon't define MALLOCX_* if building with JEMalloc
Christopher Dykes [Wed, 8 Mar 2017 18:48:01 +0000 (10:48 -0800)]
Don't define MALLOCX_* if building with JEMalloc

Summary:
With the way it is currently, if `jemalloc.h` is included after `folly/Malloc.h`, `jemalloc.h` ends up redefining these.
This changes it to just include `jemalloc.h` if we know we're compiling with JEMalloc.

Reviewed By: yfeldblum

Differential Revision: D4660285

fbshipit-source-id: 814e36432676e89cf7ba2bc5247b7dde2bfdc14a

7 years agoFix mcrouter-oss build
Anirudh Ramachandran [Wed, 8 Mar 2017 18:10:42 +0000 (10:10 -0800)]
Fix mcrouter-oss build

Summary: D4624754 broke mcrouter-oss because that uses older OpenSSL (1.0.1).

Reviewed By: yfeldblum, jmswen

Differential Revision: D4670767

fbshipit-source-id: f297c8d3e013525bb36bd059eb40c70f0c4c2dcc

7 years agostd::make_unique -> folly::make_unique for older platforms
Jon Maltiel Swenson [Wed, 8 Mar 2017 17:41:26 +0000 (09:41 -0800)]
std::make_unique -> folly::make_unique for older platforms

Summary: Title.

Reviewed By: yfeldblum

Differential Revision: D4674125

fbshipit-source-id: 71ad38d7a52ab1c27fcc7ff14d130978b0ece382

7 years agoDefault VirtualEventBase
Andrii Grynenko [Wed, 8 Mar 2017 00:54:44 +0000 (16:54 -0800)]
Default VirtualEventBase

Summary:
Make each EventBase have a defuault VirtualEventBase which is attached to it and is joined on destruction. Default VirtualEventBase is lazily created on first request.
This makes it trivial to use code switched to VirtualEventBase both with VirtualEventBase and EventBase.

Reviewed By: yfeldblum

Differential Revision: D4644639

fbshipit-source-id: cf28a3632463a1c61404c225ce1186f5a4a062a3

7 years agoDial EventBase loop time logging up to 11
Alexander Blom [Tue, 7 Mar 2017 11:27:21 +0000 (03:27 -0800)]
Dial EventBase loop time logging up to 11

Summary:
There is no need for EventBase to log every loop iteration on a relatively low VLOG level.
Just move it up to 11 like the other verbose logging in EventBase.cpp

Reviewed By: yfeldblum

Differential Revision: D4642379

fbshipit-source-id: 351636afb186ba94b2447b206d7d1389cbdba110

7 years agoAdd a few more methods to OpenSSLCertUtils
Anirudh Ramachandran [Tue, 7 Mar 2017 08:26:12 +0000 (00:26 -0800)]
Add a few more methods to OpenSSLCertUtils

Summary:
Add a few more getters (subject, issuer, notbefore, notafter) to
OpenSSLCertUtils.

Also add an additional API to AsyncSSLSocket to add application-generated
alert strings during the handshake, for e.g., during certificate verification

Reviewed By: knekritz

Differential Revision: D4624754

fbshipit-source-id: f01998b9e0e58b88ece8c6dc51ab590988bf0a8f

7 years agofixing a compile error on windows v2017.03.06.00
Johannes Kopf [Sun, 5 Mar 2017 23:58:16 +0000 (15:58 -0800)]
fixing a compile error on windows

Summary:
This file only compiles with MSVC after I made the change in this diff.

The code was difficult to follow, so I am not sure if the template is needed here?

Reviewed By: yfeldblum

Differential Revision: D4656575

fbshipit-source-id: 19415623bdb1757b573f12cdc9d8544d716ce6f2

7 years agofolly/Foreach.h: avoid shadowing warnings
Jim Meyering [Sun, 5 Mar 2017 18:32:27 +0000 (10:32 -0800)]
folly/Foreach.h: avoid shadowing warnings

Summary:
Fix warnings exposed by gcc's -Wshadow-compatible-local.
Don't use a hard-coded names that would cause shadowing with each nested use of this macro.

This avoids errors like the following:

  folly/Foreach.h:94:18: error: declaration of 'FOR_EACH_privateCount' shadows a previous local [-Werror=shadow-compatible-local]
  folly/Foreach.h:91:12: error: shadowed declaration is here [-Werror=shadow-compatible-local]

Reviewed By: yfeldblum

Differential Revision: D4656783

fbshipit-source-id: d7cd820fb04ccd78242f7487e2b219dc300709b6

7 years agoAdd getTotalConnectTimeout method
Subodh Iyengar [Sun, 5 Mar 2017 01:46:28 +0000 (17:46 -0800)]
Add getTotalConnectTimeout method

Summary:
AsyncSSLSocket can be connected using connect with
total timeouts.

This adds a method to get the total timeout that
was set.

Reviewed By: ngoyal, yfeldblum

Differential Revision: D4656331

fbshipit-source-id: a55ad9f081449d358b8133e9598a2063f625a2e6

7 years agoRemove a RequestContext deadlock
Marc Celani [Sat, 4 Mar 2017 06:27:50 +0000 (22:27 -0800)]
Remove a RequestContext deadlock

Summary:
It is not unusual to write a RequestData that will send a request to a backend service upon destruction (for example, to log the request data). This typically involves calling EventBase::runInEventBaseThread() to do the logging. Normally, if you wait until the RequestContext is destructing for this to happen, everything works fine. However, if you explicitly call clearContextData(), then this can cause a deadlock. clearContextData() holds the write lock on the data_. Imagine the NotificationQueue thread tries to set the context (for some other reason), and it is waiting on the read lock on the data_. It does so while holding a lock on its own queue. Now, the clearContextData() calls the destructor while holding the write lock on data_, and the destructor calls runInEventBaseThread, which grabs the queue's lock. Deadlock.

This can either be fixed in NotificationQueue or RequestContext. I decided to fix it in RequestContext because there may be other deadlocks that don't involve NotificationQueue (see the test plan).

There are other deadlocks in RequestContext if you call write methods in RequestData::onSet or onUnset. But fixing this would require a massive perf regression (I imagine using shared_ptr instead of unique_ptr, and copying the ptrs under readlock and calling the callbacks after releasing the lock - yuck!) So instead, warn people against doing that.

Reviewed By: yfeldblum

Differential Revision: D4655400

fbshipit-source-id: cfebe696953a19dc4dba49976bbcd0ec1363bc42

7 years agoTweak size handling in Endian
Yedidya Feldblum [Fri, 3 Mar 2017 05:32:09 +0000 (21:32 -0800)]
Tweak size handling in Endian

Summary:
[Folly] Tweak size handling in `Endian`.

Either multiply by 8 in one place, or divide by 8 in another place. Go with the latter, because `sizeof` in C++ emits byte sizes, not bit sizes.

Reviewed By: simpkins

Differential Revision: D4645020

fbshipit-source-id: cb78600ba4c20bebc66aed506d4b5d6c378fc998

7 years agoRevert D4618623: Give observer manager threads a name
Christopher Dykes [Thu, 2 Mar 2017 21:44:59 +0000 (13:44 -0800)]
Revert D4618623: Give observer manager threads a name

Summary: This reverts commit 21b86944f513da0700039a3e3736937fd72e8887

Differential Revision: D4618623

fbshipit-source-id: 6ab6498a2ce22895d7052e258d1af216d46ca178

7 years agoAdd a method to AsyncSSLSocket to supply connect timeouts
Subodh Iyengar [Thu, 2 Mar 2017 17:00:14 +0000 (09:00 -0800)]
Add a method to AsyncSSLSocket to supply connect timeouts

Summary:
The current method to supply connect timeouts to AsyncSSLSocket
is to supply only the max connect + ssl connect time.

However in some cases when connect time is known, it is desirable
to supply it so that if connect does not succeed in connect time
we can error out quicker and retry.

This adds a new method for connect time in addition to the total
Connect time.

An alternative to this would be to create a AsyncSocket, connect it
and then pass it's fd to AsyncSSLSocket, however that approach does
not work well when TFO is being used, because TFO State is a part of
the AsyncSocket's state and that is not encapsulated with the fd transfer.

We could move the state around, but that is error prone, and this is much
simpler and isolated to AsyncSSLSocket.

Reviewed By: knekritz

Differential Revision: D4626924

fbshipit-source-id: d802d035efbced68873ab59314d9f0e661e5509b

7 years agoRefactor Endian
Yedidya Feldblum [Thu, 2 Mar 2017 05:20:44 +0000 (21:20 -0800)]
Refactor Endian

Summary:
[Folly] Refactor `Endian`.

Define `swap` for all types, instead of only declaring it for all types, but have its implementation call a function that only has a given set of non-template overloads for a given set of supported types.

The effective change is that now `Endian::swap<std::string>` will fail to compile, rather than compile but fail to link.

And support floating-point types.

Reviewed By: nbronson

Differential Revision: D4615706

fbshipit-source-id: 5034e1e4466b8118a258d917ce8f4094460ca01a

7 years agoFix ASAN integration with detect_stack_use_after_return turned on
Andrii Grynenko [Wed, 1 Mar 2017 18:37:55 +0000 (10:37 -0800)]
Fix ASAN integration with detect_stack_use_after_return turned on

Summary: detect_stack_use_after_return enables fake stack logic. Previous implementation was relying on fiber releasing its fake stack before switch, which is not safe, because that fake stack may actually be in use. This diff implements a hacky workaround by building a freeFakeStack function, which can release fake stack outside of the fiber. Ideally this function should be provided by ASAN itself.

Reviewed By: yfeldblum

Differential Revision: D4631962

fbshipit-source-id: 6213e2a6b3cbc35fae3793fcf86b0614dd8ff825

7 years agoDisable MemoryIdler::unmapUnusedStack with ASAN
Andrii Grynenko [Wed, 1 Mar 2017 18:35:36 +0000 (10:35 -0800)]
Disable MemoryIdler::unmapUnusedStack with ASAN

Summary: It doesn't work nicely with fake stacks (which are used with detect_stack_use_after_return).

Reviewed By: yfeldblum

Differential Revision: D4631995

fbshipit-source-id: f139041314f8f64c0cf0a15874d6a4710fb2a5b6

7 years agorename Range template arg for increased readability
Nathan Bronson [Wed, 1 Mar 2017 17:53:28 +0000 (09:53 -0800)]
rename Range template arg for increased readability

Summary:
Parameterizing Range by the template arg T is potentially
confusing because it violates the typical container pattern of T being
the value type.  This diff uses Range<Iter> consistently in Range.h when
the template arg refers to a fully general iterator.

Reviewed By: yfeldblum

Differential Revision: D4635270

fbshipit-source-id: f70bb2e3e23ab9875fba3dbf7b11492613a037b8

7 years agoNew C++17 backport: folly::enable_shared_from_this
Philipp Unterbrunner [Wed, 1 Mar 2017 06:14:32 +0000 (22:14 -0800)]
New C++17 backport: folly::enable_shared_from_this

Summary:
C++14 has no direct means of creating a std::weak_ptr, one must always create a (temporary) std::shared_ptr first. C++17 adds weak_from_this() to std::enable_shared_from_this to avoid that overhead.
Alas code that is meant to compile under different language versions cannot rely on std::enable_shared_from_this::weak_from_this().
This new utility class uses SFINAE to call std::enable_shared_from_this<T>::weak_from_this() if available. Falls back to std::enable_shared_from_this<T>::shared_from_this() otherwise.
Use as a drop-in replacement for std::enable_shared_from_this where weak_from_this() is desired.

Reviewed By: yfeldblum

Differential Revision: D4616394

fbshipit-source-id: 73bf6cd8852d4a33478a9280cc69c81e7ea73423

7 years agoGive observer manager threads a name
Haijun Zhu [Tue, 28 Feb 2017 06:45:34 +0000 (22:45 -0800)]
Give observer manager threads a name

Summary: As title, convenient for debugging. Asked for from D4542856

Reviewed By: nkg-

Differential Revision: D4618623

fbshipit-source-id: 21b86944f513da0700039a3e3736937fd72e8887

7 years agoUpdate googletest to 1.8.0
Ted Percival [Mon, 27 Feb 2017 23:01:14 +0000 (15:01 -0800)]
Update googletest to 1.8.0

Summary:
`make check` was failing because <gmock.h> is included but not supplied. Updating to googletest 1.8.0 includes both googletest & googlemock, so `make check` now runs (and passes).
Closes https://github.com/facebook/folly/pull/547

Reviewed By: igorsugak

Differential Revision: D4593403

Pulled By: Orvid

fbshipit-source-id: 02f80e7b67e17562ff981e9c80a37f985eb685be

7 years agoMove some tests out of FOLLY_ALLOW_TFO ifdef.
Kyle Nekritz [Mon, 27 Feb 2017 23:00:11 +0000 (15:00 -0800)]
Move some tests out of FOLLY_ALLOW_TFO ifdef.

Summary: I think this ended up encompassing more than intended.

Reviewed By: yfeldblum, maxgeorg

Differential Revision: D4622645

fbshipit-source-id: 26597098a479b85d6ecc64abef2ecc53637ee759

7 years agoDon't call memcpy with empty buffer in IOBuf::copyBuffer().
Kyle Nekritz [Mon, 27 Feb 2017 18:48:39 +0000 (10:48 -0800)]
Don't call memcpy with empty buffer in IOBuf::copyBuffer().

Summary: Fixes "undefined behavior" in ubsan builds when using an empty ByteRange.

Reviewed By: siyengar

Differential Revision: D4613663

fbshipit-source-id: 4b53ba764609acc986340f10613f84585fa697cf

7 years agoKill checks in configure.ac that aren't used
Christopher Dykes [Mon, 27 Feb 2017 03:20:16 +0000 (19:20 -0800)]
Kill checks in configure.ac that aren't used

Summary: They are checks that aren't actually used anywhere to guard anything, so kill them. Many are for things that have long since been standard, or aren't even used.

Reviewed By: yfeldblum

Differential Revision: D4615947

fbshipit-source-id: 2179caff4ef8583c89efedbff3136ae5fc552424

7 years agoUse cstdint instead of inttypes.h
Christopher Dykes [Mon, 27 Feb 2017 03:19:28 +0000 (19:19 -0800)]
Use cstdint instead of inttypes.h

Summary: This is the only place `inttypes.h` is used in Folly and it's used in a context where `cstdint` would have sufficed.

Reviewed By: yfeldblum

Differential Revision: D4619248

fbshipit-source-id: e67dccb7d8bf1ad5f7539d4b039590253e8d8c79

7 years agoKill unused include of sched.h
Christopher Dykes [Mon, 27 Feb 2017 00:18:08 +0000 (16:18 -0800)]
Kill unused include of sched.h

Summary: These are the only includes of `sched.h` in Folly, and they really are unused:

Reviewed By: yfeldblum

Differential Revision: D4615501

fbshipit-source-id: 6a3a1aad7a91a92a322fd93d18e1284aa3f86b89

7 years agoreduce consumed stack size in symbolizer
Philip Pronin [Sun, 26 Feb 2017 23:30:35 +0000 (15:30 -0800)]
reduce consumed stack size in symbolizer

Summary:
It currently takes ~32kB of stack trace to run symbolizer, which is
very close to ASan alt stack size (32 kB).

If we exclude `demangle` (which can use unbound stack size in extreme cases),
the heaviest path in symbolizer includes `FrameArray<100>` and three arrays of
`PATH_MAX` (4 kB) size.  This diff removes the former and one of the latters,
reducing this code path from 32 kB to ~10 kB.

Reviewed By: ot, yfeldblum

Differential Revision: D4618467

fbshipit-source-id: e6a53b61b3d5f6e8b892216d2e9b839ed8430d0e

7 years agoDrop FOLLY_HAVE_STD_THIS_THREAD_SLEEP_FOR
Christopher Dykes [Sun, 26 Feb 2017 19:13:26 +0000 (11:13 -0800)]
Drop FOLLY_HAVE_STD_THIS_THREAD_SLEEP_FOR

Summary: It was only needed for GCC 4.7 support. GCC 4.8 and above support it.

Reviewed By: yfeldblum

Differential Revision: D4616044

fbshipit-source-id: 177d7b3b7090134f9d8084a3ba7a984446c87d5e

7 years agoFix -Wshadow warning in sorted_vector_types
Michael Lee [Sat, 25 Feb 2017 02:08:13 +0000 (18:08 -0800)]
Fix -Wshadow warning in sorted_vector_types

Summary: `last` was already passed in as a variable, so switch this so there isn't a local variable at all.

Reviewed By: mlogan, yfeldblum

Differential Revision: D4615206

fbshipit-source-id: dd8e3157e54dc2abe3dfd97837ce327371f4f7b2

7 years agoDrop support for gcc 4.7 with byteswap shim
Yedidya Feldblum [Fri, 24 Feb 2017 22:33:37 +0000 (14:33 -0800)]
Drop support for gcc 4.7 with byteswap shim

Summary: [Folly] Drop support for gcc 4.7 with byteswap shim

Reviewed By: igorsugak, Orvid

Differential Revision: D4610746

fbshipit-source-id: 092cd432c7bda9ca5507d0230b6fcaa43b96c500

7 years agoFix a corner case in EliasFanoReader::previousValue
Giuseppe Ottaviano [Fri, 24 Feb 2017 01:51:16 +0000 (17:51 -0800)]
Fix a corner case in EliasFanoReader::previousValue

Summary:
`previousValue` makes the incorrect assumption that `outer_`
is a multiple of the word size. This is incorrect because `skipToNext`
can reposition at an arbitrary point.

The bug is very rare because it can only happen if there is a gap
larger than the skip quantum after the first element in the upper
bits.

Reviewed By: philippv

Differential Revision: D4607270

fbshipit-source-id: ff016f09f3f1f87314b68370e3dc137b82694f45

7 years agoRemove duplicates during bulk insertion.
Mark Logan [Thu, 23 Feb 2017 20:21:31 +0000 (12:21 -0800)]
Remove duplicates during bulk insertion.

Summary:
antonl noticed that bulk_insert doesn't remove duplicates.
We now run a std::unique() pass after merging.

Reviewed By: yfeldblum

Differential Revision: D4595304

fbshipit-source-id: 538364150aeea64b95488da158c09e07a6597e7c

7 years agoAdd getCommonName and getSubjectAltName Cert utility helpers
Michael Lee [Tue, 21 Feb 2017 18:24:08 +0000 (10:24 -0800)]
Add getCommonName and getSubjectAltName Cert utility helpers

Summary: `getCommonName` and `getSubjectAltName` are useful shared utility functions.

Reviewed By: anirudhvr

Differential Revision: D4546866

fbshipit-source-id: bcefc1ff3a02df93fbf0cd2d4f24569928bd1753

7 years agoPrevent a crash in folly::Symbolizer
Mark Williams [Mon, 20 Feb 2017 22:53:22 +0000 (14:53 -0800)]
Prevent a crash in folly::Symbolizer

Summary:
dbg and dbgo builds of hhvm with gcc-5 crash when generating
backtraces using folly::Symbolizer, because the .debug_aranges for
libc-2.23.so are SHF_COMPRESSED, and folly::Symbolizer doesn't
recognize that.

Just pretend that the section doesn't exist if it has SHF_COMPRESSED
set.

We might eventually want to support decompressing such sections under
an option - but folly::Symbolizer's goal is to just mmap and walk the
debug info without allocating memory (so that it can run while
handling signals etc).

Reviewed By: pixelb

Differential Revision: D4586762

fbshipit-source-id: bef61ed670d1a80caa4f7aac1f80fd2a92cc4ba9

7 years agoAdd Baton variants with multiple posters and with a non-blocking waiter
Maged Michael [Fri, 17 Feb 2017 13:13:28 +0000 (05:13 -0800)]
Add Baton variants with multiple posters and with a non-blocking waiter

Summary:
Added two variants of Baton by adding two Boolean template parameters with defaults that correspond to the original single poster blocking waiter version:
- SinglePoster (single poster vs multiple posters). Default true.
- Blocking (vs spinning). Default true.

The multi-poster baton (SinglePoster = false) can be handed over by multiple concurrent posters, where redundant post()-s are idempotent but if a post() arrives after the baton is reset() then it unblocks the waiter.

The nonblocking Baton (Blocking = false) has faster common case post(), by avoiding CAS in the critical path. It does so by disallowing waiter blocking and timed_wait().

Reviewed By: nbronson

Differential Revision: D4506728

fbshipit-source-id: e5b65ad93e443fbba22164319c985ebc8424554a

7 years agoFix exception_wrapper::throwException when called before main
Yedidya Feldblum [Fri, 17 Feb 2017 05:07:49 +0000 (21:07 -0800)]
Fix exception_wrapper::throwException when called before main

Summary:
[Folly] Fix `exception_wrapper::throwException` when called before `main`.

When the `exception_wrapper` is empty, then `throwException` emits a message to `std::cerr` and terminates. If called before `main` begins or after `main` ends, then `std::cerr` might not have been constructed yet or might have already been destructed. Bad. Segfault, etc. So we need to ensure that `std::cerr` is alive by holding an instance of `std::ios_base::Init`.

Reviewed By: Orvid

Differential Revision: D4576190

fbshipit-source-id: dd84b5e82eb6bd817d405baf530714e0f4c314c5

7 years agoDrop support for Clang < 3.9
Yedidya Feldblum [Thu, 16 Feb 2017 22:45:30 +0000 (14:45 -0800)]
Drop support for Clang < 3.9

Summary: [Folly] Drop support for Clang < 3.9.

Reviewed By: igorsugak

Differential Revision: D4567233

fbshipit-source-id: cd94c711f0390d249b6736ab292721a477022b02

7 years agoWorkaround MSVC 2015 limitations in Traits.h
Christopher Dykes [Thu, 16 Feb 2017 22:02:54 +0000 (14:02 -0800)]
Workaround MSVC 2015 limitations in Traits.h

Summary: MSVC 2015 has issues with templated `using` declarations using `decltype`, so adjust the definition to use constexpr evaluation to fill a template parameter of a base type instead.

Reviewed By: yfeldblum

Differential Revision: D4510155

fbshipit-source-id: cc2e33625b515085dff4e682971299810905fef0

7 years agoAdjust a few things to avoid triggering warnings under MSVC
Christopher Dykes [Thu, 16 Feb 2017 21:10:26 +0000 (13:10 -0800)]
Adjust a few things to avoid triggering warnings under MSVC

Summary: This includes both unused parameters (W4100) and conditional expression is constant (W4127).

Reviewed By: yfeldblum

Differential Revision: D4525057

fbshipit-source-id: 7c057dbe239d02fa2e3ce96373d53814591994ec

7 years agoUse std::this_thread::yield rather than sched_yield
Christopher Dykes [Thu, 16 Feb 2017 21:08:47 +0000 (13:08 -0800)]
Use std::this_thread::yield rather than sched_yield

Summary: They do the same thing, and the first is portable.

Reviewed By: yfeldblum

Differential Revision: D4569649

fbshipit-source-id: db0434766f674a7789d6e59335e122b4d2131e06

7 years agoFix data race in Futex<EmulatedFutexAtomic>
Maged Michael [Thu, 16 Feb 2017 20:19:29 +0000 (12:19 -0800)]
Fix data race in Futex<EmulatedFutexAtomic>

Summary: Fixed a data race between an atomic store by the waker and a nonatomic memcpy by the waiter.

Reviewed By: nbronson

Differential Revision: D4572410

fbshipit-source-id: 3982ca433e0f628636916516e35aeb7738ae030f

7 years agoAdd AtomicIntrusiveLinkedList::reverseSweep()
Lovro Puzar [Thu, 16 Feb 2017 11:33:51 +0000 (03:33 -0800)]
Add AtomicIntrusiveLinkedList::reverseSweep()

Summary: In D4558451 I want to pull elements off an atomic list onto a thread-local (single-consumer) vector and pop to get elements in insertion order.  A sweep that provides elements in LIFO order will avoid a redundant pair of list reversals.

Reviewed By: nbronson

Differential Revision: D4564816

fbshipit-source-id: 38cf50418e6afe0be3eec96ce85d571c65f06d7e

7 years agooptimize EliasFanoEncoder::defaultNumLowerBits
Philip Pronin [Wed, 15 Feb 2017 20:10:01 +0000 (12:10 -0800)]
optimize EliasFanoEncoder::defaultNumLowerBits

Summary:
`defaultNumLowerBits` is heavily used during PL encoding with
optimally-partitioned EF. This diff improves its performance by 3.5x by
avoiding expensive 64-bit division.

Reviewed By: ot, luciang

Differential Revision: D4564615

fbshipit-source-id: 4521d3c05f19573cd2c84f702fcfeaace5f98e77

7 years agoReplace the future-splitting system
Phil Willoughby [Wed, 15 Feb 2017 11:37:16 +0000 (03:37 -0800)]
Replace the future-splitting system

Summary:
The previous mechanism in SharedPromise doesn't work if the lifetime of the
SharedPromise object ends before the Future which it was splitting is
completed.

This variation on the same theme doesn't have that problem.

Reviewed By: spacedentist

Differential Revision: D4339670

fbshipit-source-id: f619762bff1390481715575b3e638ec26b5c4edd

7 years agoFix undefined behaviour
Anton Likhtarov [Wed, 15 Feb 2017 01:35:35 +0000 (17:35 -0800)]
Fix undefined behaviour

Summary: memcpy's src cannot be nullptr even if the size is 0.

Reviewed By: djwatson

Differential Revision: D4560100

fbshipit-source-id: 37cd3ed73f902a136f711a5c1a918258f94d1063

7 years agoAdd overloads of Range::{start,end}sWith with custom comparator
Alex Malyshev [Tue, 14 Feb 2017 19:09:42 +0000 (11:09 -0800)]
Add overloads of Range::{start,end}sWith with custom comparator

Summary: Intended to be used for case-insensitive prefix/suffix matching.

Reviewed By: yfeldblum

Differential Revision: D4552336

fbshipit-source-id: 0030b883426dd67bea27c3f878279359d51e1d33

7 years agoTimestamping callback interface in folly::AsyncSocket
Maxim Georgiev [Tue, 14 Feb 2017 01:32:45 +0000 (17:32 -0800)]
Timestamping callback interface in folly::AsyncSocket

Summary: Adding an interface to folly::AsyncSocket allowing to receive timestamp notifications collected using recvmsg(fd, MSG_ERRQUEUE, ...).

Reviewed By: djwatson

Differential Revision: D4329066

fbshipit-source-id: 154f5c0d04e5c0e410081d48c937af4069479fc2

7 years agoadd Range constructors from std::array
Adam Simpkins [Sat, 11 Feb 2017 01:11:44 +0000 (17:11 -0800)]
add Range constructors from std::array

Summary:
Add explicit constructors to create Range objects referring to the contents of
a std::array.

D4165608 previously added a folly::range() factory function to create Ranges
around const arrays.  However, it did not support non-const arrays.  However,
providing this as a constructor seems better.  This new constructor does not
appear ambiguous with any existing constructors, and a constructor allows for
more natural support of non-const arrays: the caller can explicitly indicate
if they want to construct a Range to const or non-const data in this case.

Reviewed By: yfeldblum

Differential Revision: D4523515

fbshipit-source-id: c1a262f9e8f76907d87d80b03f252576506e70ab

7 years agoMinor GDB script fix
Andrii Grynenko [Fri, 10 Feb 2017 20:56:56 +0000 (12:56 -0800)]
Minor GDB script fix

Reviewed By: lbrandy

Differential Revision: D4544705

fbshipit-source-id: d60f2026d31c94e020df51f8c887b995986ddcd7

7 years agochanged Synchronized.md to reflect SharedMutex as default
Jack Montgomery [Fri, 10 Feb 2017 20:39:08 +0000 (12:39 -0800)]
changed Synchronized.md to reflect SharedMutex as default

Summary: folly/docs/Synchronized.md now reflects the fact that SharedMutex is default mutex for Synchronized

Reviewed By: yfeldblum

Differential Revision: D4529851

fbshipit-source-id: b3b8eafe908ba92e3629392fbc4b856b5a99b658

7 years agoDetect identity conversion in toDynamic
Andrew Krieger [Fri, 10 Feb 2017 00:54:55 +0000 (16:54 -0800)]
Detect identity conversion in toDynamic

Summary:
D4499520 added typedefs to `dynamic` which made it start matching
tests for ranges/containers. However, the typedefs are unconditional on the
actual contents of the `dynamic`. This made toDynamic(dynamic) select the
range-based conversion operator, always, which immediately asserts when
trying to do range-based iteration over an Object or a primitive. Add tests
to the converters that enable/disable depending on whether the object is
already a `dynamic` and early-out in that case.

Reviewed By: mzlee

Differential Revision: D4538617

fbshipit-source-id: f3a5aafab07946a221dcead782fc27de51afa0a6

7 years agoadd folly:hasher<bool>
Den Raskovalov [Thu, 9 Feb 2017 20:00:33 +0000 (12:00 -0800)]
add folly:hasher<bool>

Summary:
hasher<bool> is a trivial thing, but it is useful, if we implement a hash function for a class by writing something like:
  return folly::Hash()(boolField1_, boolField2_, intField3_, boolField4_, someField5_)

Reviewed By: ot

Differential Revision: D4535563

fbshipit-source-id: 5e6672c2eeaebbbac80a8a066a6a0ec4838810f4

7 years agoAdd keepAlive() mechanism
Andrii Grynenko [Thu, 9 Feb 2017 18:58:07 +0000 (10:58 -0800)]
Add keepAlive() mechanism

Summary:
EventBase and VirtualEventBase already had a loopKeepAlive() mechanism, which enabled libraries to prevent EventBase/VirtualEventBase from being destroyed until all keep-alive tokens were released.

This change adds generic keepAlive() support into folly::Executor. folly::Executors which don't support keep-alive mechanism yet, will just return a no-op KeepAlive token.

Reviewed By: yfeldblum

Differential Revision: D4516649

fbshipit-source-id: 869779621c746cb14d985aa73bc4536859914c03

7 years agoBenchmark string copying via the formatter
Phil Willoughby [Thu, 9 Feb 2017 12:35:54 +0000 (04:35 -0800)]
Benchmark string copying via the formatter

Summary:
Using the formatter is always much slower than using the copy constructor (News at 11).

Comparing the two idioms for formatting out a single string input:
* the unsafe method (`format(theString)`) is (annoyingly) fastest for small strings; probably because cache-hits dominate the need to inspect every character to see if it is `{` or `}`
* the safe method (`format("{}",theString)`) is fastest on large strings (crossover at roughly 256 bytes)
* there isn't a lot in it between 64 and 512 bytes.

Reviewed By: yfeldblum

Differential Revision: D4423088

fbshipit-source-id: 7f8590653ab33fe62887a506ea79a43989fbfb29

7 years agoNon-const `dynamic` iterators
Andrew Krieger [Thu, 9 Feb 2017 00:03:39 +0000 (16:03 -0800)]
Non-const `dynamic` iterators

Summary:
Add non-const iterators corresponding to existing iterator types
where it makes sense.

Reviewed By: yfeldblum

Differential Revision: D4499520

fbshipit-source-id: dc4ec583d3da1c6701805b30b25389fbf9311a7c

7 years agoSpecialise std::hash for folly::Optional
Sebastian Messmer [Wed, 8 Feb 2017 19:41:13 +0000 (11:41 -0800)]
Specialise std::hash for folly::Optional

Summary: This allows using folly::Optional<T> in std::unordered_map and std::unordered_set.

Reviewed By: luciang

Differential Revision: D4469938

fbshipit-source-id: b86b1a4510379b337e9de83471a4aafce6d5a6cb

7 years agoThrow custom exception type when ABD fails a result because some other token did...
Ameya Limaye [Wed, 8 Feb 2017 19:23:24 +0000 (11:23 -0800)]
Throw custom exception type when ABD fails a result because some other token did not call dispatch

Summary:
Throw custom exception type when ABD fails a result because some other token did not call dispatch:
- This is useful because the caller can catch this exception and discard it if they find some other exception
- The custom exception does not give an indication of the problem. The other exception/s thrown will do that.

Reviewed By: yfeldblum

Differential Revision: D4395742

fbshipit-source-id: be80f66b1297e9faf625a2fb087590a7d0a9335d

7 years agoMake a few coersions to bool explicit
Christopher Dykes [Tue, 7 Feb 2017 19:27:44 +0000 (11:27 -0800)]
Make a few coersions to bool explicit

Summary: MSVC has warning 4800 which is triggered on implicit coercions to `bool` when not in use as the condition to a control-flow statement.

Reviewed By: yfeldblum

Differential Revision: D4518465

fbshipit-source-id: 858ab9e68215a2a667cb3ea55daf51b74368174d

7 years agoModify BaseFormatter to help static analysers
Phil Willoughby [Tue, 7 Feb 2017 08:45:01 +0000 (00:45 -0800)]
Modify BaseFormatter to help static analysers

Summary:
This decouples the format-string-parser from the storage of the arguments to be
formatted: a static analyser can now define a class which derives from
`BaseFormatter` with no storage and use `BaseFormatter::operator()` to verify
the syntax of the format string and determine which arguments will be
referenced.

This method of allowing overrides is resolved at compile time; the benchmarks
confirm that there is no run-time difference.

Reviewed By: yfeldblum

Differential Revision: D4507689

fbshipit-source-id: f109d81ae54dc074ac363720ef6e565520435d26

7 years agoRun clang-format over Format*.h
Phil Willoughby [Tue, 7 Feb 2017 08:44:56 +0000 (00:44 -0800)]
Run clang-format over Format*.h

Summary:
Changes are planned, this will make it easier for the new code to look good
without making the older code look weird.

Reviewed By: yfeldblum

Differential Revision: D4514839

fbshipit-source-id: 43f8cd7d85901e1915cc434a0f4b5fd6bce1ec7a

7 years agoSupport compiling in C++14 mode
Christopher Dykes [Mon, 6 Feb 2017 21:44:18 +0000 (13:44 -0800)]
Support compiling in C++14 mode

Summary:
The rest of Folly isn't dependent on C++17 mode in MSVC, so modify this to support not having it enabled.
This will make adoption easier as C++17 mode acts on the things the standard says are removed.

Reviewed By: yfeldblum

Differential Revision: D4509919

fbshipit-source-id: f4644f5f5ba78de8ab5192b89ac6320767b51084

7 years agoexecute callbacks as rvalue references
Sven Over [Mon, 6 Feb 2017 16:13:01 +0000 (08:13 -0800)]
execute callbacks as rvalue references

Summary:
Callable objects may implement a separate `operator()` for when
they are called as rvalue references. For example, `folly::partial`
will move captured objects to the function call when invoked as
rvalue reference. That allows e.g. to capture pass a `unique_ptr`
to `folly::partial` for a function that takes a `unique_ptr` by
value or rvalue-reference.

Callbacks for `folly::Future`s are only ever executed once. They
may consume captured data. If the callback is an object that
implements a `operator()() &&`, then that one should be invoked.

This diff makes sure, callbacks passed to `folly::Future` are
invoked as rvalue references.

Reviewed By: ericniebler, fugalh

Differential Revision: D4404186

fbshipit-source-id: 9f33e442a634acb8797183d3d869840d85bd5d15

7 years agoExpose testing callbacks from AsyncSocketTest2.
Andrey Ignatov [Sat, 4 Feb 2017 09:28:42 +0000 (01:28 -0800)]
Expose testing callbacks from AsyncSocketTest2.

Summary:
`TestConnectionEventCallback` and `TestAcceptCallback` are very handy and can
be used in other tests outside folly. Specifically I'm going to use them in
`ConnectHookTest`. It'll also allow to avoid copy/paste in `TransparentTlsTest`.

Reviewed By: plapukhov

Differential Revision: D4507461

fbshipit-source-id: dfaa97e26036ebb11da17a53d4a73431a295f4d4

7 years agoFix environ enumeration on Apple platforms
Matt Glazar [Sat, 4 Feb 2017 00:59:31 +0000 (16:59 -0800)]
Fix environ enumeration on Apple platforms

Summary:
environ is documented to not work very well from .dylib-s (dynamic libraries) on macOS. Use the _NSGetEnviron function instead to get a pointer to environ.

<crt_externs.h> is missing from some iOS SDKs, so forward-declare _NSGetEnviron in such cases.

Reviewed By: Orvid, mzlee

Differential Revision: D4491044

fbshipit-source-id: 886d19bdd63dea14225992446b7ba132faf35054

7 years agoExpose more functions/constructors on BucketedTimeSeries
Xuli Liu [Fri, 3 Feb 2017 22:53:09 +0000 (14:53 -0800)]
Expose more functions/constructors on BucketedTimeSeries

Summary: Currently BucketedTimeSeries does not expose enough functions to allow assessing its data, therefore there is no way to do serialization/deserialization. Adding functions/constructors to support this.

Reviewed By: simpkins

Differential Revision: D4500075

fbshipit-source-id: 656ac8a208547d8d3fadf9ea150606b6e74775c9

7 years agoAdd ContainerT&& ctors to sorted_vector_map and sorted_vector_set.
Mark Logan [Fri, 3 Feb 2017 18:09:57 +0000 (10:09 -0800)]
Add ContainerT&& ctors to sorted_vector_map and sorted_vector_set.

Summary:
Add ContainerT&& ctors to sorted_vector_map and sorted_vector_set,
to support more efficient bulk-construction of these containers. With the
prior constructors, the only way to do bulk-construction efficiently was
by using the iterator constructors with a pre-sorted range. If you didn't
have a presorted range, you would need to make a temporary container in
order to get a sorted range. Repeatedly calling insert() without a sorted
range is quadratic, hence not an option.

Reviewed By: yfeldblum

Differential Revision: D4491299

fbshipit-source-id: 041c546578f44ee6928c5506e8d8191092143b12

7 years agoFix worst-case quadratic behavior of iterator constructors and range insert()
Mark Logan [Fri, 3 Feb 2017 18:09:52 +0000 (10:09 -0800)]
Fix worst-case quadratic behavior of iterator constructors and range insert()

Summary:
The iterator constructors and the range insert() method previously
had quadratic runtime if given an unsorted range. This is now fixed. We
append the entire range to the container, sort that subrange, and merge it
into the already-sorted container. Sorting and merging is skipped when possible.

Reviewed By: yfeldblum

Differential Revision: D4493534

fbshipit-source-id: e6d73b5c19e374001f9e340ff527c5257bef2ca3

7 years agoFix build failure with GCC 5
Pádraig Brady [Thu, 2 Feb 2017 19:35:54 +0000 (11:35 -0800)]
Fix build failure with GCC 5

Summary:
GCC 5 doesn't recognize LOG(FATAL) as [[noreturn]],
so remove [[noreturn]] to avoid that warning and build failure.

Reviewed By: yfeldblum

Differential Revision: D4484625

fbshipit-source-id: ca129ca3a52239becd7d80f99107e64745bf8838

7 years agoCorrect the API of some functions in the portability headers
Christopher Dykes [Thu, 2 Feb 2017 01:22:36 +0000 (17:22 -0800)]
Correct the API of some functions in the portability headers

Summary: They were based on the signatures of the original functions as defined by MSVC. They should be based on the signatures as defined by Posix instead.

Reviewed By: yfeldblum

Differential Revision: D4471711

fbshipit-source-id: 8a6fd60ba2b326ca57058c119fc77c46f1d21d10

7 years agoAssorted cleanups for folly/portability/BitsFunctexcept.h
Yedidya Feldblum [Wed, 1 Feb 2017 21:57:34 +0000 (13:57 -0800)]
Assorted cleanups for folly/portability/BitsFunctexcept.h

Summary: [Folly] Assorted cleanups for `folly/portability/BitsFunctexcept.h`.

Reviewed By: Orvid, smeenai

Differential Revision: D4492264

fbshipit-source-id: 5e85ac2ad96ef84627cc5473ec0b16c4c9b5da26

7 years agoMPMCQueue Wrapper with priorities
Scott Michelson [Wed, 1 Feb 2017 20:03:48 +0000 (12:03 -0800)]
MPMCQueue Wrapper with priorities

Summary: In many cases where MPMCQueue would be used, it's useful to be able to prioritize requests. This implements a thin wrapper on MPMCQueue to accomplish this (albeit in a bit of a memory inefficient way)

Reviewed By: haijunz

Differential Revision: D4465498

fbshipit-source-id: 6630b80ccf3138b5c135e7f7f281133b37d82b4d

7 years agoSwap a few APIs to reduce sign and implicit truncations required to work with it
Christopher Dykes [Wed, 1 Feb 2017 01:35:51 +0000 (17:35 -0800)]
Swap a few APIs to reduce sign and implicit truncations required to work with it

Summary: This results in a more uniform API within Folly, allowing for a more uniform API to use outside of Folly.

Reviewed By: yfeldblum

Differential Revision: D4471471

fbshipit-source-id: f798a6498bd1a05ed12adea362ff4aedd25789ee

7 years agoUse std::chrono::high_resolution_clock for folly::Benchmark
Christopher Dykes [Wed, 1 Feb 2017 00:34:49 +0000 (16:34 -0800)]
Use std::chrono::high_resolution_clock for folly::Benchmark

Summary: `clock_gettime(CLOCK_REALTIME)` is based on `std::system_clock` on Windows, but that only has a resolution that's the same as `FILETIME` (100s of nanoseconds), so modernize things and use `std::chrono::high_resolution_clock` which is intended for this purpose.

Reviewed By: yfeldblum

Differential Revision: D4476671

fbshipit-source-id: 3db1debc8f408f689b5c9fe1966a72b8dad4da93

7 years agoFunctionScheduler - set running_ when it has actually started running
Igor Adamski [Tue, 31 Jan 2017 10:15:36 +0000 (02:15 -0800)]
FunctionScheduler - set running_ when it has actually started running

Summary:
When start() throws because of reasons (in my case it was std::system_error("Resource temporarily unavailalble") coming from std::thread constructo)
and during the exception propagation we will have to destroy FunctionScheduler then
thread_.join() in shutdown will throw again.
This diff sets running_ after the thread is created.

Reviewed By: yfeldblum

Differential Revision: D4469816

fbshipit-source-id: cde54dfbf39f04d3ea9dfa02a65295f5440e5ea4

7 years agoRequire compile-time constant format strings to `shellify`
Phil Willoughby [Tue, 31 Jan 2017 09:37:27 +0000 (01:37 -0800)]
Require compile-time constant format strings to `shellify`

Summary:
Because it's impossible to require that a parameter to a function is a compile-time constant string this replaces `shellify()` with the user-defined-literal suffix `_shellify()`.

It's trivial to convert previously-correct code: `shellify("whatever {}", A)` => `"whatever {}"_shellify(A)`

The previous `folly::shellify()` API is still present as a transition measure. Compilers will issue a deprecation warning if it is used.

Reviewed By: yfeldblum

Differential Revision: D4435512

fbshipit-source-id: 6639cd91280dc72108e47a8a7775c5160a4e644f

7 years agocrt_externs.h is not available on all platforms
Michael Lee [Tue, 31 Jan 2017 00:00:33 +0000 (16:00 -0800)]
crt_externs.h is not available on all platforms

Summary: Different deployments of don't always have access to crt_externs.h.

Reviewed By: yfeldblum, Orvid

Differential Revision: D4480621

fbshipit-source-id: f0b18d15fa253d6ba28dee68859cf9a34bf41255

7 years agoFix folly/Makefile.am
Michael Lee [Mon, 30 Jan 2017 21:15:26 +0000 (13:15 -0800)]
Fix folly/Makefile.am

Summary: Missed a `\`

Reviewed By: Orvid

Differential Revision: D4483602

fbshipit-source-id: 91451c3476756125abdbd0dd98ab9fb38bcf2c32

7 years agoSplit up experimental/TestUtil
Michael Lee [Mon, 30 Jan 2017 20:03:04 +0000 (12:03 -0800)]
Split up experimental/TestUtil

Summary: TestUtil is primarily a temporary file and temporary directory.  Split out the env variable manipulation into a different file.

Reviewed By: Orvid

Differential Revision: D4482071

fbshipit-source-id: 9d1a4a08010a8fad270aa56a7e1432829eb6484c

7 years agochar ** enivron is inconsistent on some platforms
Michael Lee [Sun, 29 Jan 2017 03:43:50 +0000 (19:43 -0800)]
char ** enivron is inconsistent on some platforms

Summary: The forward declaration of `extern char ** environ` is inconsistent on some platforms.

Reviewed By: strager, Orvid

Differential Revision: D4474856

fbshipit-source-id: c73a7c7d5a247e962fab06aeb65df7815f0e3bcb

7 years agoRFC: Include timeout duration in exception message
Jason Rahman [Sat, 28 Jan 2017 06:24:55 +0000 (22:24 -0800)]
RFC: Include timeout duration in exception message

Summary: For debugging purposes, it's often extremely useful to have the duration of the expired timeout in the exception message (in case the timeout was excessively low, or to understand exactly how long the operation ran before failing due to timeout). Here we make a best effort to include the timeout when possible.

Reviewed By: yfeldblum

Differential Revision: D4476038

fbshipit-source-id: f2d0aa5e1f6367a3cfe20579c27577ab91fd8d9c