9 years agomove assignment operators for folly::Synchronized
Philip Pronin [Fri, 5 Jul 2013 03:01:37 +0000 (20:01 -0700)]
move assignment operators for folly::Synchronized

* added move assignment operators,
* fixed `operator=(const Synchronized& rhs)` (it had a typo),
* fixed deadlock on self-assignment,
* changed `swap` to call `swap(lhs.datum_, rhs.datum_)` instead of `lhs.datum_.swap(rhs.datum_)`.

Test Plan: fbconfig -r folly/test && fbmake opt -j32 && fbmake runtests_opt

Reviewed By:

FB internal diff: D875977

9 years agomemrchr and *timed_mutex are platform-specific
Sean Cannella [Wed, 3 Jul 2013 20:36:28 +0000 (13:36 -0700)]
memrchr and *timed_mutex are platform-specific

- conditionally compile rfind overrides
- conditionally add support for timed_mutex/recursive_timed_mutex

Test Plan:
- compiled on OSX
- unit tests

Reviewed By:

FB internal diff: D872272

9 years agostrerror_r is XSI compliant on Apple/FreeBSD
Sean Cannella [Mon, 1 Jul 2013 15:06:12 +0000 (08:06 -0700)]
strerror_r is XSI compliant on Apple/FreeBSD

- Noticed this due to an -fpermissive compiler warning while compiling
HHVM for OSX (complaint of trying to cast int to char*)

Test Plan:
- Compiled
- Confirmed the build warning is fixed by this on the Mac OS X build

Reviewed By:

FB internal diff: D865169

9 years agotoDynamic(T)
Tom Jackson [Tue, 23 Apr 2013 17:02:07 +0000 (10:02 -0700)]

Summary: It only makes sense to be able to go the other direction, too.

Test Plan: Unit tests

Reviewed By:

FB internal diff: D785282

9 years agoRange<T>::rfind()
Tom Jackson [Wed, 26 Jun 2013 23:53:58 +0000 (16:53 -0700)]

Test Plan: Unit tests

Reviewed By:

FB internal diff: D865951

9 years agoMulti-producer multi-consumer queue with optional blocking
Nathan Bronson [Fri, 28 Jun 2013 20:42:14 +0000 (13:42 -0700)]
Multi-producer multi-consumer queue with optional blocking

MPMCQueue<T> is a high-performance bounded concurrent queue that
supports multiple producers, multiple consumers, and optional blocking.
The queue has a fixed capacity, for which all memory will be allocated
up front.  The bulk of the work of enqueuing and dequeuing can be
performed in parallel.

To make an MPMCQueue<T>, T must satisfy either of two conditions:
- both the constructor used during enqueue and the move operator are
marked noexcept.

This diff extracts the generic component from tao/queues/ConcurrentQueue
and renames identifiers to match those of existing folly queues.
It also includes an extraction of Futex, which wraps the futex syscall,
and DeterministicScheduler, which allows for deterministic exploration
of thread interleavings for components built from std::atomic and Futex.

Test Plan: new unit tests

Reviewed By:

FB internal diff: D866566

9 years agoconst'ing sorted_vector_map::count()
Tom Jackson [Wed, 26 Jun 2013 19:32:28 +0000 (12:32 -0700)]
const'ing sorted_vector_map::count()

Test Plan: Use it

Reviewed By:

FB internal diff: D865211

9 years agofix tautological comparisons in Conv.h
Louis Brandy [Thu, 20 Jun 2013 00:01:38 +0000 (17:01 -0700)]
fix tautological comparisons in Conv.h

We have an off-by-one in our enable_if/sfinae logic here. We do not want to actually do the comparison in `less_than` when the rhs is exactly the minimum possible lhs. This results in a tautological comparison.

I added a unit test for these traits that test all the various sfinae cases.

Test Plan: See the clang tautological warnings go away. Under gcc, rerun all tests. All pass.

Reviewed By:

FB internal diff: D856869

9 years agoSpeed up EventCount, now 64-bit only
Tudor Bosman [Fri, 21 Jun 2013 03:14:35 +0000 (20:14 -0700)]
Speed up EventCount, now 64-bit only

Don't use two atomic variables when only one will suffice (otherwise, the
increments in doNotify() and prepareWait() would require memory_model_seq_cst,
because they need to globally order a store before a load, and no weaker
model allows you to do that)

Of course, this makes this 64-bit only, but I don't care.

Test Plan: eventcount_test, tests outside of folly

Reviewed By:

FB internal diff: D858963

9 years agoStringPiece comparisons are broken
Tudor Bosman [Mon, 24 Jun 2013 19:48:43 +0000 (12:48 -0700)]
StringPiece comparisons are broken

8-bit StringPiece comparisons are broken.
The reason is char_traits is magic for 'char', but not 'const char'.

Test Plan: test added

Reviewed By:

FB internal diff: D861521

9 years agoadd Chrono.h to patch around a gcc 4.6 issue
Tudor Bosman [Fri, 21 Jun 2013 16:18:37 +0000 (09:18 -0700)]
add Chrono.h to patch around a gcc 4.6 issue

Test Plan: used it

Reviewed By:

FB internal diff: D859244

9 years agoUse libunwind instead of the heavyweight thing from libgcc
Tudor Bosman [Thu, 20 Jun 2013 21:15:17 +0000 (14:15 -0700)]
Use libunwind instead of the heavyweight thing from libgcc

Test Plan: exception_tracer_test

Reviewed By:

FB internal diff: D858195

9 years agoAdd MultiLevelTimeSeries to folly.
Stephen Chen [Fri, 14 Jun 2013 22:46:44 +0000 (15:46 -0700)]
Add MultiLevelTimeSeries to folly.

Add MultiLevelTimeSeries class which represents a timeseries which keeps several
levels of data granularity (similar in principle to the loads reported by the
UNIX 'uptime' command).  It uses several instances (one per level) of
BucketedTimeSeries as the underlying storage.

This can easily be used to track sums (and thus rates or averages) over several
predetermined time periods, as well as all-time sums.  For example, you would
use to it to track query rate or response speed over the last 5, 15, 30, and 60

Test Plan: unittest included.

Reviewed By:

FB internal diff: D851444

9 years agorename template parameters in folly::greater_than
Louis Brandy [Wed, 19 Jun 2013 23:39:45 +0000 (16:39 -0700)]
rename template parameters in folly::greater_than

Summary: This template is only used in folly::Conv (to avoid tautological comparisons) and as best as I can tell, the rhs/lhs names are not correct as it's currently used (and not consistent with `less_than`). Just swap rhs/lhs variable names.

Test Plan:
Test Results Summary:
Passed: 471
100% successful

Reviewed By:

FB internal diff: D856785

9 years agofolly: accommodate use of -Wshadow in other projects
Jim Meyering [Thu, 13 Jun 2013 16:54:17 +0000 (09:54 -0700)]
folly: accommodate use of -Wshadow in other projects

Without this change, any folly-using project that attempts to enable
gcc's -Wshadow option will be overwhelmed by warnings about violations
in folly's class definitions.  Violations are often due to local
variable or parameter names shadowed by a member function.

Bracket offending content with directives like this:

#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wshadow"
#pragma GCC diagnostic pop

This set of modified files is the minimal set to allow tao
to use -Wshadow with no folly-related warning.

Test Plan:
fbmake folly/test && fbmake opt

Reviewed By:

FB internal diff: D848455

9 years agorework the Subprocess::communicate() API
Adam Simpkins [Thu, 16 May 2013 02:03:40 +0000 (19:03 -0700)]
rework the Subprocess::communicate() API

This eliminates the CommunicateFlags argument to communicate().  It now
always reads from both stdout and stderr if they were set up a pipes.
If a non-empty input buffer was supplied, it always writes that to

This mimics the communicate() behavior of python's module.
This also makes it impossible to have buffering deadlocks by forgetting
to call communicate() with readStderr().

Test Plan:
Ran the existing subprocess tests, and also added a more complicated
duplex test that requires communication on stdin, stdout, and stderr all
at the same time.

Also grepped for all existing users of Subprocess::communicate(), and
made sure they will work correctly with the new behavior.

Reviewed By:

FB internal diff: D814405

9 years agoFolly Changes for Clang, revision 1
Todd Nowacki [Tue, 4 Jun 2013 21:43:22 +0000 (14:43 -0700)]
Folly Changes for Clang, revision 1

Depends on D829162 and D830052

A few changes are needed to get folly compiling with clang.
There are likely more changes to compile all of folly

Moved ifunc check, and added an additional check for clang, instead of hard
coding it to be 1

Moved the declaration of a function to get rid of clang's unused function

Test Plan:
This bit of code was a dependency in another project.
That code compiled as it needed to

See other diff for more test details

Reviewed By:

FB internal diff: D830075

9 years agoFix some issues in allocator_delete
Jordan DeLong [Mon, 3 Jun 2013 23:14:50 +0000 (16:14 -0700)]
Fix some issues in allocator_delete

The unique_ptr implementation was getting confused about how
to determine the pointer type, so just tell it.

Facebook: Broke the fxl build; test plan included recompiling it.

Test Plan: Built.

Reviewed By:

FB internal diff: D834371

9 years agoMake folly::allocator_delete take advantage of EBO
Jordan DeLong [Sun, 2 Jun 2013 03:55:24 +0000 (20:55 -0700)]
Make folly::allocator_delete take advantage of EBO

Summary: It wasn't.

Test Plan:
I have a case using this that now passes a static_assert
that it's the same size as the default_deleter version.  I didn't add
a static_assert in folly's tests because right now
is_simple_allocator<int,std::allocator<int>>::value is true (I think
because the tested members come from a base class), which breaks easy
test cases.

Reviewed By:

FB internal diff: D833142

9 years agoosx on folly take 2
Paul Tarjan [Fri, 31 May 2013 22:01:14 +0000 (15:01 -0700)]
osx on folly take 2

Summary: using ULL broke everything. Lets try the case @danslo originally suggested.

Test Plan: will check errors for this file

Reviewed By:

FB internal diff: D832165

9 years agorevert folly/Benchmark.cpp
Paul Tarjan [Fri, 31 May 2013 21:55:52 +0000 (14:55 -0700)]
revert folly/Benchmark.cpp

Summary: revert the breaking change

Test Plan: none

Reviewed By:

FB internal diff: D832143

Blame Revision: D830883

9 years agomake folly build on OSX
Daniel Sloof [Tue, 28 May 2013 08:17:55 +0000 (10:17 +0200)]
make folly build on OSX

A squash of

Github Author: danslo

Test Plan: he said it compiles on OSX

Reviewed By:

FB internal diff: D830883

9 years agoadd missing include
Owen Yamauchi [Thu, 30 May 2013 20:24:04 +0000 (13:24 -0700)]
add missing include

Newer versions of boost fail without this. I guess mutex.hpp doesn't
include lock_guard.hpp indirectly anymore, or something.

Test Plan:
built on OS X. This was part of what I had to do for my OS
X-building attempts from a few weeks ago.

Reviewed By:

FB internal diff: D830157

9 years agoeachToTuple<Types...>()
Tom Jackson [Wed, 29 May 2013 01:17:37 +0000 (18:17 -0700)]

Summary: For parsing records. `eachToPair` might be a worthwhile specialization.

Test Plan: Unit tests, benchmarks maybe?

Reviewed By:

FB internal diff: D827441

9 years agoHACK: New Gen operators: zip, interleave
Mike Curtiss [Sat, 2 Mar 2013 06:27:56 +0000 (22:27 -0800)]
HACK: New Gen operators: zip, interleave

Zip: inspired by python's zip()
o Combine a generator with the contents of a container to form
a tuple.  Note that we combine with a container (and not
another generator) because of a fundamental constraint
in how control-flow in Generators works.  Containers give us 90%
of the utility without all the hassle.  We could theoretically
also add a version of zip where the extra source is generated
concurrently in another thread.

Interleave: similar to zip, but inspired by Clojure's interleave()
o Instead of creating a tuple like zip, just flatten the values.

Added some tuple creation/concatenation functions.  These are mostly
meant as a way to enable zip'ing multiple containers together into an
N-tuple. (My variadic-fu was not strong enough to get this working
within a single Zip function).

Test Plan: Added unit-tests

Reviewed By:

FB internal diff: D740518

9 years agocorrect broken testcase
John Fremlin VII [Tue, 28 May 2013 18:28:14 +0000 (11:28 -0700)]
correct broken testcase

Turns out that phabricator and arc unit do not actually run
tests in our internal repo and this one had a bunch of syntax errors.


No test cases were changed, just fixing syntax.

Test Plan:
- fbconfig folly/test

- fbmake runtests

- saw all ok

- _bin/folly/test/json_test

[==========] Running 10 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 10 tests from Json
[ RUN      ] Json.Unicode
[       OK ] Json.Unicode (0 ms)
[ RUN      ] Json.Parse
[       OK ] Json.Parse (1 ms)
[ RUN      ] Json.JavascriptSafe
[       OK ] Json.JavascriptSafe (0 ms)
[ RUN      ] Json.Produce
[       OK ] Json.Produce (0 ms)
[ RUN      ] Json.JsonEscape
[       OK ] Json.JsonEscape (0 ms)
[ RUN      ] Json.JsonNonAsciiEncoding
[       OK ] Json.JsonNonAsciiEncoding (0 ms)
[ RUN      ] Json.UTF8Retention
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0528 11:29:43.879976 17977 JsonTest.cpp:257] input: ♥ => json: "♥"
I0528 11:29:43.880077 17977 JsonTest.cpp:259] output: ♥ => json: "♥"
[       OK ] Json.UTF8Retention (1 ms)
[ RUN      ] Json.UTF8EncodeNonAsciiRetention
I0528 11:29:43.880122 17977 JsonTest.cpp:284] input: ♥ => json: "\u2665"
I0528 11:29:43.880131 17977 JsonTest.cpp:286] output: ♥ => json: "\u2665"
[       OK ] Json.UTF8EncodeNonAsciiRetention (0 ms)
[ RUN      ] Json.UTF8Validation
[       OK ] Json.UTF8Validation (0 ms)
[ RUN      ] Json.ParseNonStringKeys
[       OK ] Json.ParseNonStringKeys (0 ms)
[----------] 10 tests from Json (2 ms total)

[----------] Global test environment tear-down
[==========] 10 tests from 1 test case ran. (2 ms total)
[  PASSED  ] 10 tests.

Reviewed By:

FB internal diff: D826413

Blame Revision: D805218

9 years agoallow reading maps from number -> value
John Fremlin VII [Tue, 21 May 2013 23:04:40 +0000 (16:04 -0700)]
allow reading maps from number -> value

The serialization of PHP values often uses integer -> value
maps in JSON arrays. These are emitted by the standard stream <<
operator on dynamics but cannot be read. This diff fixes that.

Test Plan: - read in serialized value with array "bucketing":{"days_stale":{0:{2:null,1:14.01,0:"more_than_two_weeks_stale"}}}

Reviewed By:

FB internal diff: D805218

9 years agoadd openNoInt, truncateNoInt, ftruncateNoInt
Tudor Bosman [Wed, 15 May 2013 04:28:50 +0000 (21:28 -0700)]
add openNoInt, truncateNoInt, ftruncateNoInt

Test Plan: it compiles

Reviewed By:

FB internal diff: D825286

9 years agoAdd QueueAppender
Tudor Bosman [Tue, 21 May 2013 17:59:39 +0000 (10:59 -0700)]
Add QueueAppender

Add an Appender that appends to a IOBufQueue.  You can pass in the maximum
append size if you know it, and it will throw on overflow, and will never
allocate IOBufs larger than the remaining size.

Test Plan: test added

Reviewed By:

FB internal diff: D820160

9 years agomake folly:make_unique support customized deleter
Xu Ning [Fri, 24 May 2013 23:02:12 +0000 (16:02 -0700)]
make folly:make_unique support customized deleter

Summary: just follow the same template arguments as unique_ptr

Test Plan: compile

Reviewed By:

FB internal diff: D825025

9 years agomember(&Foo::getter), field(&Foo::field)
Tom Jackson [Thu, 23 May 2013 22:34:03 +0000 (15:34 -0700)]
member(&Foo::getter), field(&Foo::field)

Summary: For easily calling a getter on every item in a sequence.

Test Plan: Unit tests, benchmarks

Reviewed By:

FB internal diff: D651206

9 years agoSome documentation updates in lazy
Jordan DeLong [Wed, 22 May 2013 21:23:51 +0000 (14:23 -0700)]
Some documentation updates in lazy

Summary: Per discussion after commit on the previous diff.

Test Plan: Compiled.

Reviewed By:

FB internal diff: D821985

9 years agoAdd folly::lazy
Jordan DeLong [Sun, 19 May 2013 21:47:39 +0000 (14:47 -0700)]
Add folly::lazy

A thin wrapper around Optional for terse creation of
lazily-initialized values.

Test Plan: New tests, and a use case in hphp.

Reviewed By:

FB internal diff: D817906

9 years agoAllow escapeString in folly/json.cpp to be called by other methods.
Maxime Boucher [Mon, 20 May 2013 08:26:55 +0000 (01:26 -0700)]
Allow escapeString in folly/json.cpp to be called by other methods.

Summary: See title.

Test Plan:
From fbocde, type:
fbconfig folly/test; fbmake opt -j 32; fbmake runtests_opt -j 32

Reviewed By:

FB internal diff: D818077

9 years agoGen apply comment
Mike Curtiss [Thu, 16 May 2013 18:16:37 +0000 (11:16 -0700)]
Gen apply comment

Summary: Comment

Test Plan: Builds

Reviewed By:

FB internal diff: D814986

9 years agodelete unused variable
Paul Tarjan [Mon, 20 May 2013 18:33:17 +0000 (11:33 -0700)]
delete unused variable

Test Plan: none

Reviewed By:

FB internal diff: D818326

9 years agogen::sample
Mike Curtiss [Tue, 14 May 2013 05:05:06 +0000 (22:05 -0700)]

Take a random sample of size N from a range.  Clients
can also pass in a custom random number generator.

Test Plan: Added test and benchmark.

Reviewed By:

FB internal diff: D811260

9 years agoUpdate Subprocess to throw if exec() fails
Adam Simpkins [Tue, 16 Apr 2013 00:58:47 +0000 (17:58 -0700)]
Update Subprocess to throw if exec() fails

Add a new SubprocessSpawnError, and change the Subprocess constructor to
throw this if the child process encounters an error before calling
execve().  Error information is passed back to the parent process over a

Previosly in this case the Subprocess constructor would fail, and
clients would simply get a return code of 126 or 127 when waiting on the
process.  There was no way to distinguish this from a successful
execve() followed by the child process exiting with status 127.

Test Plan:
Added tests to check the exception behavior, and also to check for file
descriptor leaks in the parent process.

Reviewed By:

FB internal diff: D776273

9 years agoExpose a human-readable type name for a dynamic
Chip Turner [Tue, 14 May 2013 23:44:27 +0000 (16:44 -0700)]
Expose a human-readable type name for a dynamic

Summary: Aids in debugging dynamic types, etc.

Test Plan: unit tests

Reviewed By:

FB internal diff: D812385

9 years agoNew feature support in folly::AtomicHash*
Mark Williams [Thu, 9 May 2013 18:41:55 +0000 (11:41 -0700)]
New feature support in folly::AtomicHash*

AtomicHashMap/AtomicHashArray didn't support a custom equality
function, and didnt support pointer keys. We only ever do
compare-and-swap against the 3 "magic" keys, so as long as we're
careful we can continue to do that, while using the equality
function elsewhere.

Also add a user-configurable growth rate, independent of

Test Plan: automated tests

Reviewed By:

FB internal diff: D806936

9 years agoAdd fsyncNoInt, fdatasyncNoInt
Tudor Bosman [Sat, 11 May 2013 21:35:43 +0000 (14:35 -0700)]
Add fsyncNoInt, fdatasyncNoInt

Test Plan: compiled file_util_test

Reviewed By:

FB internal diff: D809250

9 years agoReplace CHECK in Range.h by throw std::out_of_range
Maxime Boucher [Sun, 12 May 2013 04:33:07 +0000 (21:33 -0700)]
Replace CHECK in Range.h by throw std::out_of_range

Calling CHECK() in folly will force the program to abort in case of a failure.
On the other hand, for range checking, the standard library throws std::out_of_range for many functions.

Thus it could be a good idea to throw the same exception in folly so that errors can be handled using try {} catch (...) {} blocks.

Test Plan:
from fbcode, type:
fbconfig -r folly; fbmake opt -j32; fbmake runtests_opt -j 32

What other tests should I run?

Reviewed By:

FB internal diff: D808204

9 years agoReplace CHECK in Range.h by throw std::out_of_range
Maxime Boucher [Fri, 10 May 2013 18:42:20 +0000 (11:42 -0700)]
Replace CHECK in Range.h by throw std::out_of_range

Calling CHECK() in folly will force the program to abort in case of a failure.
On the other hand, for range checking, the standard library throws std::out_of_range for many functions.

Thus it could be a good idea to throw the same exception in folly so that errors can be handled using try {} catch (...) {} blocks.

Test Plan:
from fbcode, type:
fbconfig -r folly; fbmake opt -j32; fbmake runtests_opt -j 32

What other tests should I run?

Reviewed By:

FB internal diff: D808204

9 years agoUnbreak build on glibc 2.5.1
Tudor Bosman [Thu, 9 May 2013 22:50:02 +0000 (15:50 -0700)]
Unbreak build on glibc 2.5.1

Also move manually-generated stuff away from folly-config.h and into
Portability.h.  This should be only for things that differs between the
various (compiler, library) pairs used internally at FB; everything else
should be autoconf-ed and therefore go into folly-config.h.

Test Plan: built and ran file_util_test on various platforms

Reviewed By:

FB internal diff: D807067

Blame Revision: D806781

9 years agofix segfault on fb platform
Tudor Bosman [Thu, 9 May 2013 22:17:13 +0000 (15:17 -0700)]
fix segfault on fb platform

Summary: Strings can be moved around while the vector grows.

Test Plan: file_util_test

Reviewed By:

FB internal diff: D806966

Blame Revision: D806781

9 years agoconvert RecordIO to pwritev
Tudor Bosman [Thu, 9 May 2013 21:27:56 +0000 (14:27 -0700)]
convert RecordIO to pwritev

Summary: zero-copy

Test Plan: record_io_test

Reviewed By:

FB internal diff: D806813

9 years agoAdd missing FileUtil functions (p?readvFull, p?writevFull) , add test
Tudor Bosman [Thu, 9 May 2013 21:16:35 +0000 (14:16 -0700)]
Add missing FileUtil functions (p?readvFull, p?writevFull) , add test

Testing incomplete reads / writes is hard, so I'm only testing the wrappers
(template functions that take the underlying operation and retry it in
case of incomplete operations).  Note the cute hack of using variadic
templates to use the same wrapper for both file pointer- and explicit-position
flavors of the functions (the offset argument becomes optional).

Test Plan: test added

Reviewed By:

FB internal diff: D806781

9 years agoMake folly::AsyncIO thread safe
Tudor Bosman [Wed, 8 May 2013 21:40:10 +0000 (14:40 -0700)]
Make folly::AsyncIO thread safe

You can now submit to the same AsyncIO object from different threads, but you
must still reap from only one thread at a time.

Test Plan: async_io_test, added MT test

Reviewed By:

FB internal diff: D804914

9 years agoBreak dependency on endian.h
Owen Yamauchi [Fri, 3 May 2013 19:16:14 +0000 (12:16 -0700)]
Break dependency on endian.h

gcc and clang both give you these three macros pre-defined. I've found a
reference on the interweb to this being broken
( but that was with gcc
4.4. I've tested with 4.6, 4.7 and clang 3.2 and they all do the right
thing on little-endian (internal build and OS X). Alas, I don't have a
big-endian system handy to test with.

Test Plan: fbconfig/fbmake runtests.

Reviewed By:

FB internal diff: D799416

9 years agoBreak dependency on byteswap.h
Owen Yamauchi [Fri, 3 May 2013 17:43:36 +0000 (10:43 -0700)]
Break dependency on byteswap.h

Unfortunately, this doesn't exist on all platforms. Fortunately, both
gcc and clang support builtins that do the same thing as the bswap_*
functions we were including byteswap.h for.

I changed the test file to hardcode expected results for the swapping
functions. Is that OK? It seemed a little silly to be testing the
Endian::whatever functions by comparing them against the functions that
they're implemented in terms of.

Test Plan:
fbconfig/fbmake runtests with gcc 4.6 and 4.7. Verified
separately that clang supports these builtins.

Reviewed By:

FB internal diff: D799244

9 years agoBreak dependency on features.h
Owen Yamauchi [Fri, 3 May 2013 16:25:39 +0000 (09:25 -0700)]
Break dependency on features.h

It doesn't exist on some systems (at least Mac OS X). We're only using
it for __GNUC_PREREQ, which is easy to provide our own definition for.

I moved the definitions of FOLLY_FINAL and FOLLY_OVERRIDE into
folly-config.h so we can autoconf them in the open-source build. The
hardcoded stuff for the internal build is a little ugly, unfortunately.

folly can't be built with gcc versions earlier than 4.6, so that check
in ThreadLocal.h is pointless by now. (Plus we use noexcept without a
macro wrapper all over the place.) That stuff was also not
clang-friendly. clang has supported static_assert since 2.9 and noexcept
since... I'm not sure but at least 3.0.

Test Plan:
fbconfig/fbmake runtests, with gcc 4.6 and 4.7. clang can't
build folly right now, but I verified separately that it supports
noexcept and static_assert.

Reviewed By:

FB internal diff: D799143

9 years ago(Folly) Remove unused variable according to -Wunused-variable
Hannes Roth [Fri, 3 May 2013 16:37:17 +0000 (09:37 -0700)]
(Folly) Remove unused variable according to -Wunused-variable

I am trying to move more code into a HPHP extension, and it's complaining about

Test Plan: `fbconfig -r folly && fbmake runtests`

Reviewed By:

FB internal diff: D799096

9 years agoFail the configure script if you don't have boost
Owen Yamauchi [Tue, 30 Apr 2013 21:33:21 +0000 (14:33 -0700)]
Fail the configure script if you don't have boost

Discovered this while trying to build folly on OS X. It would print a
warning about Boost being missing that got drowned out in the rest of
the configure output, and then the build would fail. This makes the
failure more friendly.

Test Plan:
`autoreconf; ./configure` on a system without boost
installed; make sure configure fails and the last thing it prints is the
boost error message.

Reviewed By:

FB internal diff: D797389

9 years agoAdd read*String() methods to Cursor
Peter Griess [Wed, 1 May 2013 03:06:55 +0000 (20:06 -0700)]
Add read*String() methods to Cursor

- Add some convenience methods for reading std::string objects via

Test Plan: - Unit tests

Reviewed By:

FB internal diff: D795428

9 years agoFixed-size split()
Tom Jackson [Tue, 30 Apr 2013 20:13:34 +0000 (13:13 -0700)]
Fixed-size split()

There are quite a few places where we split strings into a fixed number
of fields. This enables this to be done a bit faster by not using any
variable-length structures at runtime.

Test Plan: Unit tests, Benchmarks

Reviewed By:

FB internal diff: D794523

9 years agoRecordIO: robust record-based streaming I/O
Tudor Bosman [Thu, 25 Apr 2013 03:29:08 +0000 (20:29 -0700)]
RecordIO: robust record-based streaming I/O

RecordIO provides an interface to write and read streams of variable-length
records that is resilient in the face of failure and data corruption.  If
the stream is corrupted in any way, you will lose records, but the stream
will resynchronize.

We have one implementation of RecordIO reader/writer that reads from / writes
to regular files (using mmap(), even for writes, so we can preserve state
even in case of process death -- the buffer cache will flush things to disk
eventually) and we expose enough of the guts (in a reasonably clean way)
so you can build your own on top of other backends.

Test Plan: test added

Reviewed By:

FB internal diff: D790275

9 years agoflock locks in folly::File, FileUtil, Exception.h fixes and tests
Tudor Bosman [Fri, 26 Apr 2013 22:32:36 +0000 (15:32 -0700)]
flock locks in folly::File, FileUtil, Exception.h fixes and tests

Test required a separate process, as fcntl locks are always re-granted to
a process that holds the lock.

Test Plan: new tests, all tests under folly

Reviewed By:

FB internal diff: D791370

9 years agoFix build for folly_fb_platform
Jordan DeLong [Sun, 28 Apr 2013 16:27:03 +0000 (09:27 -0700)]
Fix build for folly_fb_platform

This wasn't building with an error from a warning about
narrowing conversions---probably we didn't notice because it was
previously broken due to folly docs being broken forever.  Folly tests
should always pass.  (I want to add it to the fbcode-tests-must-pass
commit hook.)

Test Plan:
fbmake runtests_opt in this platform, except with stl_tests
turned off because it takes years to compile.

Reviewed By:

FB internal diff: D792059

9 years agoMaking from(dynamic.items()) work
Tom Jackson [Tue, 30 Apr 2013 01:46:29 +0000 (18:46 -0700)]
Making from(dynamic.items()) work

Summary: It was broken because the `dynamic`'s iterator wasn't complete.

Test Plan: Unit test

Reviewed By:

FB internal diff: D793618

9 years agodistinctBy()
Tom Jackson [Fri, 26 Apr 2013 20:58:08 +0000 (13:58 -0700)]

Test Plan: Unit tests

Reviewed By:

FB internal diff: D791149

9 years agoDisabling conversion with contained value for Optional
Tom Jackson [Mon, 22 Apr 2013 18:07:07 +0000 (11:07 -0700)]
Disabling conversion with contained value for Optional

Summary: Comparisons with values can lead to confusion, especially when the value itself is truthy. To avoid this confusion, I'm disabling comparison with contained value. Note that Optionals can still be constructed and assigned these values, but comparsion must be done separately for the container and the contained.

Test Plan: Unit tests, contbuild

Reviewed By:

FB internal diff: D783621

9 years agoFix async_io_test to work with larger block sizes
Tudor Bosman [Wed, 24 Apr 2013 21:59:06 +0000 (14:59 -0700)]
Fix async_io_test to work with larger block sizes

Test Plan: ran it

Reviewed By:

FB internal diff: D787733

9 years agomove rateHelper() to detail/Stats.h
Adam Simpkins [Wed, 17 Apr 2013 03:41:29 +0000 (20:41 -0700)]
move rateHelper() to detail/Stats.h

The BucketedTimeSeries::rateHelper() function is generic, and not
specific to the BucketedTimeSeries class.  This diff moves it to
folly/detail/Stats.h, so other parts of the stats code can access it as

Test Plan: Ran the folly unit tests.

Reviewed By:

FB internal diff: D778115

9 years agoMake BucketedTimeSeries::rate() more accurate
Adam Simpkins [Wed, 17 Apr 2013 01:32:23 +0000 (18:32 -0700)]
Make BucketedTimeSeries::rate() more accurate

Make rate() and countRate() more accurate when queried for a specific
time range.

Previously these functions divided the estimated sum/count by the entire
time range specified.  This underestimated the rate if we don't actually
have data for the entire time period.  (Since the sum computed only
takes into account the time range for which we have data.)

For example, if the timeseries duration was 60 seconds, but only 30
seconds of data had been entered so far, rate(now - 60, now) would
underestimate the rate by half, since there was only 30 seconds worth of
data available.  The no-argument version of rate() did work correctly in
that case.

Test Plan:
Added a new unit test for this behavior.

Also fixed the existing rate test code, which had the same bug and
expected the underestimated rate.

Reviewed By:

FB internal diff: D778114

9 years agofix fbstring move assignment operator
Philip Pronin [Tue, 23 Apr 2013 06:36:47 +0000 (23:36 -0700)]
fix fbstring move assignment operator

21.4.2 [string.cons] / 23 says

> If *this and str are the same object, the member has no effect.

That means we have to support self-move-assignment.

Test Plan: added test which triggered assertion, ran it

Reviewed By:

FB internal diff: D785057

9 years agotoAppendDelimited, toDelimited
Tudor Bosman [Sat, 20 Apr 2013 20:07:39 +0000 (13:07 -0700)]
toAppendDelimited, toDelimited

Test Plan: test added

Reviewed By:

FB internal diff: D783100

Tudor Bosman [Sat, 20 Apr 2013 20:17:18 +0000 (13:17 -0700)]

Summary: Seems like a good idea.

Test Plan: compiled all of folly

Reviewed By:

FB internal diff: D783104

9 years agoModernize TemporaryFile, add TemporaryDirectory
Tudor Bosman [Tue, 16 Apr 2013 20:22:41 +0000 (13:22 -0700)]
Modernize TemporaryFile, add TemporaryDirectory

Test Plan: fbconfig $(find folly -name test) && fbmake runtests_opt

Reviewed By:

FB internal diff: D777186

9 years agoRevert "Revert "URI parsing in folly""
Tudor Bosman [Wed, 17 Apr 2013 18:57:05 +0000 (11:57 -0700)]
Revert "Revert "URI parsing in folly""

Now that the third_party link was updated in, we're good.

Test Plan: fbconfig -r thrift && fbmake runtests_opt

Reviewed By:

FB internal diff: D778707

9 years agoRevert "URI parsing in folly"
Tudor Bosman [Wed, 17 Apr 2013 18:47:25 +0000 (11:47 -0700)]
Revert "URI parsing in folly"

Summary: Unbreak build.  third-party link not yet updated after

Test Plan: no

Reviewed By:

FB internal diff: D778669

9 years agoURI parsing in folly
Tudor Bosman [Wed, 10 Apr 2013 00:34:52 +0000 (17:34 -0700)]
URI parsing in folly

Summary: Cleaned up from common/strings/URL.h, and it should be URI, not URL.

Test Plan: tests added

Reviewed By:

FB internal diff: D768880

9 years agoFix bug in reserve() and shrink_to_fit().
Christian Kamm [Mon, 25 Mar 2013 08:54:26 +0000 (09:54 +0100)]
Fix bug in reserve() and shrink_to_fit().

impl_.e_ += newB - impl_.b_; fails when the difference
between newB and impl_.b_ isn't a multiple of sizeof(T).

Test Plan: .

Reviewed By:

FB internal diff: D774754

9 years agoMake Subprocess::spawn more robust
Tudor Bosman [Sat, 13 Apr 2013 06:49:32 +0000 (23:49 -0700)]
Make Subprocess::spawn more robust

We can't throw after the process is created, because we don't know what to do
with it (and the Subprocess object goes up in smoke, so we can't rely on the
caller to clean up, either).  So don't throw.

If we throw before the process is created, make sure we clean up.

Test Plan: subprocess_test

Reviewed By:

FB internal diff: D774722

9 years agodo not include iostream from Range.h
Philip Pronin [Sun, 14 Apr 2013 02:06:37 +0000 (19:06 -0700)]
do not include iostream from Range.h

folly/Range.h is extensively used in fbcode, try to avoid including
<iostream> (which is pretty heavy), include forward declarations
(<iosfwd>) instead.

Also transitioned it from 'std type_traits' + 'boost type_traits' to
'std type_traits'.

Test Plan: compiled, ran tests

Reviewed By:

FB internal diff: D774834

9 years agomove Histogram.h to stats/
Adam Simpkins [Sat, 30 Mar 2013 01:55:16 +0000 (18:55 -0700)]
move Histogram.h to stats/

Move Histogram.h into the stats/ subdirectory, along side the
BucketedTimeSeries code.  Eventually I plan to land more of our stats
code in this subdirectory too.

This also renames Histogram-inl.h to Histogram-defs.h, and no longer
includes it by default from Histogram.h.  Instead, this adds a new
stats/Instantiations.cpp file which explicitly instantiates
Histogram<int64_t> and BucketedTimeSeries<int64_t>.  Most callers use
these instantiations, and they now no longer need to include
Histogram-defs.h.  Only callers that need other instantiations need to
include Histogram-defs.h.  This will speed up build times.

Test Plan:
Ran "arc unit" to build all projects depending on folly, and verified
they all still built and passed tests.

Reviewed By:

FB internal diff: D761377

9 years agoeasier rebinding of allocators
Marcelo Juchem [Mon, 8 Apr 2013 20:38:22 +0000 (13:38 -0700)]
easier rebinding of allocators

Summary: rebinding allocators is too cumbersome, this diff implements a helper to make this job easier.

Test Plan: unit test added

Reviewed By:

FB internal diff: D766451

9 years agoMaking StlAllocator<Alloc, void> usable when rebinding.
Marcelo Juchem [Mon, 8 Apr 2013 21:08:18 +0000 (14:08 -0700)]
Making StlAllocator<Alloc, void> usable when rebinding.

currently, StlAllocator<Alloc, void> can't be used when you
want an untyped allocator that can be rebound later since it doesn't
carry the SimpleAllocator pointer with it. This diff fixes that.

Test Plan: unit test added

Reviewed By:

FB internal diff: D766559

9 years agoInitialize variable before using in own initialization
Michael Connor [Wed, 27 Mar 2013 17:16:59 +0000 (10:16 -0700)]
Initialize variable before using in own initialization

Clang throws error because the callback refers to itself inside its lambda
function definition.  I prevent this by declaring the variable first then the
compiler does not complain when it is used within its lambda definition.

folly/test/TimeoutQueueTest.cpp:99:37: error: variable 'cb' is uninitialized
when used within its own initialization [-Werror,-Wuninitialized]

Test Plan:
fbconfig --clang --platform=gcc-4.7.1-glibc-2.14.1 --with-project-version
boost:1.51.0 folly/test/
fbmake opt
fbmake runtests_opt

Reviewed By:

FB internal diff: D753061

9 years agoSoft-limit for arenas
Jonathan Coens [Thu, 4 Apr 2013 22:25:52 +0000 (15:25 -0700)]
Soft-limit for arenas

Summary: Create an artificial limit on an arena to start throwing bad_alloc before running out of system memory

Test Plan: adjust unit test

Reviewed By:

FB internal diff: D762695

9 years agoFixing clang compatibility issues
Marcelo Juchem [Sat, 30 Mar 2013 07:13:31 +0000 (00:13 -0700)]
Fixing clang compatibility issues

Test Plan: all folly unit tests

Reviewed By:

FB internal diff: D757374

9 years agoImplementing a traits class to check for incomplete types
Marcelo Juchem [Wed, 3 Apr 2013 03:22:58 +0000 (20:22 -0700)]
Implementing a traits class to check for incomplete types

Summary: A traits class to check for incomplete types

Test Plan: unit tests added

Reviewed By:

FB internal diff: D760676

9 years agofolly: speed up fastpath of ThreadLocal::get()
Lucian Grijincu [Wed, 3 Apr 2013 04:29:02 +0000 (21:29 -0700)]
folly: speed up fastpath of ThreadLocal::get()

A smaller function makes it more likely it will be inlined
(it wasn't before, is now).

Test Plan: n/a

Reviewed By:

FB internal diff: D759996

9 years agofolly: speed up fastpath of StaticMeta::get()
Lucian Grijincu [Wed, 3 Apr 2013 00:56:32 +0000 (17:56 -0700)]
folly: speed up fastpath of StaticMeta::get()

A smaller function makes it more likely it will be inlined
(it wasn't before, is now).

Test Plan: n/a

Reviewed By:

FB internal diff: D760000

9 years agoas_stl_allocator "template alias"
Marcelo Juchem [Mon, 1 Apr 2013 23:00:01 +0000 (16:00 -0700)]
as_stl_allocator "template alias"

Summary: Implementing as_stl_allocator as a companion to the existing make_stl_allocator

Test Plan: unit tests added

Reviewed By:

FB internal diff: D755207

9 years agoSuppress unused variable warning
Misha Shneerson [Tue, 2 Apr 2013 19:14:41 +0000 (12:14 -0700)]
Suppress unused variable warning

HPHP compiler treats warnings as errors and the unused 'constuctor'
variable breaks the build.

Test Plan: Build

Reviewed By:

FB internal diff: D759559

9 years agoOptionally, kill subprocess when parent dies
Tudor Bosman [Thu, 28 Mar 2013 23:48:51 +0000 (16:48 -0700)]
Optionally, kill subprocess when parent dies

Summary: Non-portable.

Test Plan: test added

Reviewed By:

FB internal diff: D755528

9 years agoremove folly/eventfd.h
Tudor Bosman [Wed, 27 Mar 2013 21:04:01 +0000 (14:04 -0700)]
remove folly/eventfd.h

Test Plan: compile: fbconfig folly/experimental/io/test thrift/test unicorn/io/test common/system_features common/concurrency && fbmake opt

Reviewed By:

FB internal diff: D753552

9 years agoRemove an unused variable in FormatValue<double>
Brett Simmers [Wed, 27 Mar 2013 20:14:37 +0000 (13:14 -0700)]
Remove an unused variable in FormatValue<double>

Test Plan: Built a program that uses it

Reviewed By:

FB internal diff: D753460

9 years agomake sse4.2 functions in folly/Range.h build on -fb platform
Tudor Bosman [Mon, 25 Mar 2013 18:37:39 +0000 (11:37 -0700)]
make sse4.2 functions in folly/Range.h build on -fb platform

Test Plan: fbconfig --platform=gcc-4.7.1-glibc-2.14.1-fb folly/test && fbmake runtests_opt

Reviewed By:

FB internal diff: D749992

9 years agofix clang warnings in folly::Gen
Louis Brandy [Mon, 25 Mar 2013 17:16:15 +0000 (10:16 -0700)]
fix clang warnings in folly::Gen

Summary: Two clang warnings. It wants user defined consturctor for static initialization, and it wants fwd-declare to agree with full decleration in regards to struct v class.

Test Plan:
`fbconfig --clang --platform=gcc-4.7.1-glibc-2.14.1 folly/test/`

And make sure it doesn't break normal build.

Reviewed By:

FB internal diff: D749857

9 years agoAdd read/write mode to ElfFile
Peter Griess [Mon, 25 Mar 2013 16:31:49 +0000 (09:31 -0700)]
Add read/write mode to ElfFile

- Add a mode to ElfFile that allows opening the file for read/write
access via PROT_WRITE.

Test Plan:
- Used it in some other code

Reviewed By:

FB internal diff: D740184

9 years agoHandle non-Intel platforms in Range and CpuId
Owen Yamauchi [Thu, 21 Mar 2013 14:32:47 +0000 (07:32 -0700)]
Handle non-Intel platforms in Range and CpuId

Compile out the SSE versions of these functions in Range, based on a new
entry in folly-config.h.

The change to CpuId feels slightly iffy to me. It seems like it would be
more rigorous to make compiling CpuId.h on non-Intel an error, and force
clients to handle non-Intel platforms at the callsite. However, I think
that would be too susceptible to unintentional breakage on non-Intel
platforms, since most people (including automated systems) aren't
building and testing regularly on any. Falling back to saying "none of
these features exist on this processor" seems like a reasonable

Test Plan:
fbmake runtests, with FOLLY_HAVE_EMMINTRIN_H set to 0 and 1.
Make sure the SSE functions are getting compiled in or out as
appropriate. ##autoreconf## and ##./configure## to regenerate

Reviewed By:

FB internal diff: D746872

9 years agoAdd symbol name resolution and value retrieval
Peter Griess [Fri, 15 Mar 2013 16:12:46 +0000 (09:12 -0700)]
Add symbol name resolution and value retrieval

- Add ElfFile::getSymbolByName(), which finds a Symbol object
corresponding to the symbol w/ the given name
- Add ElfFile::getSymbolValue(), which resolves the Symbol object to a
value in the mapped file, following pointers if necessary

Test Plan: - Unit tests

Reviewed By:

FB internal diff: D740183

9 years agoEnable Elf::at<T> only for POD data types
Peter Griess [Fri, 22 Mar 2013 15:12:30 +0000 (08:12 -0700)]
Enable Elf::at<T> only for POD data types

- Make sure that the utility function Elf::at<T> only works for POD
datatypes, as it uses reinterpret_cast<T>.

Test Plan: - Unit tests

Reviewed By:

FB internal diff: D748314

9 years agoMove folly::symbolizer::systemError() into Exception.h
Peter Griess [Fri, 22 Mar 2013 14:55:54 +0000 (07:55 -0700)]
Move folly::symbolizer::systemError() into Exception.h

- This is pretty similar to some stuff that we already have in
Exception.h. Move (and rename) it.

Test Plan: - Unit tests

Reviewed By:

FB internal diff: D748313

9 years agoCompile out GroupVarint on non-Intel
Owen Yamauchi [Thu, 21 Mar 2013 19:13:39 +0000 (12:13 -0700)]
Compile out GroupVarint on non-Intel

Compile out instead of erroring.

In an ideal world, we'd have a fallback that would work across platforms
(i.e. no SSE, no unaligned 32-bit writes etc.) and compile some version
of GroupVarint in all environments. I actually tried this; the SSE stuff
is all behind #if __SSSE3__ already, so I thought it could work (modulo
the unaligned-writes problem). I ran into problems with the
SSSE3-vs.-not distinction that @simpkins alluded to in D652764, and
decided I'd rather not open that can of worms at the moment.

Test Plan:
fbmake runtests. Manually force the #ifs to false and make
sure fbmake runtests still passes (although GroupVarintTest is empty).

Reviewed By:

FB internal diff: D747150

9 years agoStlAllocator.h + MakeUnique.h -> Memory.h
Louis Brandy [Wed, 20 Mar 2013 21:54:45 +0000 (14:54 -0700)]
StlAllocator.h + MakeUnique.h -> Memory.h

Go with the fat header approach. Merge these two into Memory.h. We could, potentially, include Malloc.h as well, but it fbstring header uses the once define for some special magic. Leave it alone for now.

An alternate approach might be moving all three leaner headers into a `memory/` subdir with `folly/Memory.h` just #including the three.

Test Plan:
fbconfig folly/tests && fbmake runtests_opt

Reviewed By:

FB internal diff: D745873

9 years agoHACK: Static detection of infinite sequences
Mike Curtiss [Wed, 6 Mar 2013 07:22:54 +0000 (23:22 -0800)]
HACK: Static detection of infinite sequences

Certain operations should not be performed on infinite sequences
(e.g. sorting, left-folds, summation).  In some cases, we can
detect that a sequence is infinite at compile-time and provide
a static_assert to prevent such dangerous operations.

Test Plan:
Manually created cases where the operation should
be disallowed.  Compiler correctly raised an error.

Reviewed By:

FB internal diff: D740011

9 years agoCopyright 2012 -> 2013
Louis Brandy [Wed, 20 Mar 2013 22:09:03 +0000 (15:09 -0700)]
Copyright 2012 -> 2013

Summary: See title.

Test Plan: Inspection.

Reviewed By:

FB internal diff: D745883

9 years agoIOBuf::getIov
Hans Fugal [Thu, 14 Mar 2013 00:32:00 +0000 (17:32 -0700)]

Generate an `fbvector` of `struct iovec` suitable for using with `writev` or

This code is pretty straightforward, but Adam pointed out that something along
these lines has already been done in thrift, so I followed that code closely.

Test Plan:
fbmake runtests

I am using also this in a prototype and it's working there.

Reviewed By:

FB internal diff: D744055