8 years agofix off-by-one error in IPAddress::longestCommonPrefix()
Adam Simpkins [Fri, 18 Apr 2014 18:53:32 +0000 (11:53 -0700)]
fix off-by-one error in IPAddress::longestCommonPrefix()

Fix an off-by-one error causing an ASAN abort.  When calling
longestCommonPrefix() with a full mask, it would compare one byte past
the end of the address data, and would end up writing past the end of
the ba array on the stack.

Test Plan:
Built with ASAN, ran the unit tests, and verified the ASAN failure was

Reviewed By:

FB internal diff: D1284750

8 years agoMove common/network/IPAddress.h and related to folly/
Anton Likhtarov [Sat, 5 Apr 2014 02:00:31 +0000 (19:00 -0700)]
Move common/network/IPAddress.h and related to folly/

Moving our internal IP/Mac address libraries to folly/

We want to get rid of common/ dependencies in Mcrouter since we're going to open source it. Also looking at the original commit, seems like it's been the intention all along, so I just did it.

I tried to keep dependencies intact as much as possible. Changing projects to use this directly should be in separate diffs.

Test Plan:
Run folly/network and common/network unit tests.

Generate the list of targets with:

fbgs /$FILE.h | cut -f1 -d: | xargs -L1 dirname | cut -f2- -d/ | sort | uniq

Then fbconfig + fbmake. Will fix contbuild failures.

Revert Plan:

Reviewed By:

FB internal diff: D1261089

8 years agoFix type mismatch in IOBuf::getIov()
Peter Griess [Fri, 18 Apr 2014 20:06:07 +0000 (13:06 -0700)]
Fix type mismatch in IOBuf::getIov()

Summary: - On iOS, iovec::iov_len is a size_t, which is a uint32_t; convert

Test Plan:
- fbconfig -r folly && fbmake runtests
- Builds in fbobjc

Reviewed By:

FB internal diff: D1284931

8 years agoFix header ordering lint failures
Nicholas Ormrod [Fri, 18 Apr 2014 17:16:33 +0000 (10:16 -0700)]
Fix header ordering lint failures

Headers whose includes are not sufficient cause problems when included
from other files. An easy way to prevent this is to include the header
first in its associated cpp file. In fact, we have a lint rule for this.
However, there are places where this rule is ignored. This diff fixes
many of them.

Note that a few top-level directories were excluded.

Reviewed By:

FB internal diff: D1281032

8 years agox64 detection missing an include
Elizabeth Smith [Fri, 18 Apr 2014 16:28:00 +0000 (09:28 -0700)]
x64 detection missing an include

Summary: rwspinlock.h was missing the portability.h header which made the x64 detection go wonky, but only shows up when used with isolated rwspinlock use

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1284387

Blame Revision: D1282140

8 years agomacro for cross platform x64 detection
Elizabeth Smith [Fri, 18 Apr 2014 15:18:54 +0000 (08:18 -0700)]
macro for cross platform x64 detection

Summary: Another cross platform fun macro - this one detects x64 status and requires some portability.h includes (which should only define portability stuff and not actually include anything)

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1282140

8 years agoFix flag naming
Bryan Alger [Thu, 17 Apr 2014 22:06:36 +0000 (15:06 -0700)]
Fix flag naming

Summary: doh

Test Plan: compiled under android

Reviewed By:

FB internal diff: D1283177

8 years agofbstring likely/unlikley ifdef for stuff that doesn't support it
Elizabeth Smith [Thu, 17 Apr 2014 20:36:10 +0000 (13:36 -0700)]
fbstring likely/unlikley ifdef for stuff that doesn't support it

Summary: the fbstring specific unlikely/likely defs were missing the ifdef blocks to avoid breaking things without builtin_expect

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1282166

8 years ago(Folly/Gen) Make ranges and sequences take a stepping functor
Hannes Roth [Thu, 17 Apr 2014 16:26:56 +0000 (09:26 -0700)]
(Folly/Gen) Make ranges and sequences take a stepping functor

The functor allows to create ranges and sequences that advance the
iterator by more than 1 per iteration.

Test Plan: Unit test.

Reviewed By:

FB internal diff: D1228065

8 years ago(Folly/Gen) Fix compilation with clang
Hannes Roth [Thu, 17 Apr 2014 15:49:51 +0000 (08:49 -0700)]
(Folly/Gen) Fix compilation with clang

Clang chokes because it tries to instantiate both versions of `from<const int*>`, one of which calls `std::begin(const int*)`, which doesn't work. By casting the intializer list to the right type, it can pick the overload.

Clang, because it makes debugging these templates so much better.

Test Plan: `fbconfig --clang folly/gen/test && fbmake runtests_dbg`

Reviewed By:

FB internal diff: D1280888

8 years agoabstract thread_local support
Elizabeth Smith [Thu, 17 Apr 2014 14:49:10 +0000 (07:49 -0700)]
abstract thread_local support

change from using __thread to using FOLLY_THREAD_LOCAL macro, this will allow abstraction over gcc and msvc implementations of thread local (__thread and __declspec(thread)) which have the same semantices and will also allow drop in replacement of thread_local when compiler support for the feature is complete  This doesn't do anything about apple, however, which still has broken __thread support

This doesn't actually change any implementation for now, simply allows for correct compilation

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1278726

8 years agoAdd missing includes for msvc
Elizabeth Smith [Thu, 17 Apr 2014 14:45:39 +0000 (07:45 -0700)]
Add missing includes for msvc

Summary: MSVC is a stricter compiler and needs additional includes explicitly declared

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1277004

8 years agoparallel(pipeline)
Tom Jackson [Sat, 5 Apr 2014 00:58:07 +0000 (17:58 -0700)]

Adding `... | parallel(my | pipe | line) | ...` for parallelizing a portion of a generator pipeline.

auto factored = from(values)
| parallel(filter(isEven) | map(square) | sub(count))
| sum;

Work is divided evenly among a fixed number of threads using a `MPMCQueue`.

Test Plan: Unit tests and benchmarks testing for a variety of workloads and performance characteristics, including sub-linear (blocking) workloads, linear (mostly math) workloads, and superlinear (sleeping) workloads to simulate real-world use.

Reviewed By:

FB internal diff: D638551

8 years agoIntroduce LifoSem
Marc Celani [Thu, 17 Apr 2014 00:19:31 +0000 (17:19 -0700)]
Introduce LifoSem

Summary: A LIFO semaphore is useful for building a thread pool because we can keep cpu caches warmer by continually reusing the last used thread. This diff introduces a LIFO semaphore to folly.

Test Plan: unit tests

Reviewed By:

FB internal diff: D1280405

8 years agoFix folly Uri::host() return value for IPv6 address
Jun LI [Wed, 16 Apr 2014 20:58:54 +0000 (13:58 -0700)]
Fix folly Uri::host() return value for IPv6 address

Summary: folly Uri::host() returns leading and trailing brackets for IPv6 host, for example, for URI "http:://[::1]:8080/index.html", host() method returns "[::1]". But square brackets are not part of host in fact, this diff is going to remove brackets for IPv6 host.

Test Plan: fbconfig -r folly/test && fbmake runtests_dbg

Reviewed By:

FB internal diff: D1276513

8 years agoMSVC translation of noreturn attribute
Elizabeth Smith [Wed, 16 Apr 2014 20:49:35 +0000 (13:49 -0700)]
MSVC translation of noreturn attribute

Provide translations for gcc noreturn attribute

__attribute__((noreturn)) is gcc specific, clang understands it on nix systems, but for msvc __declspec(noreturn) is the compiler specific version, which clang will imitate/use on windows.  There was already a FOLLY_NORETURN in portability.h, however because of __declspec limitations it must prefix the declaration, not postfix.  The gcc noreturn attribute does not have this limitation and will work prefixed OR postfixed, so to keep from turning code into spaghetti nonsense, the macro was moved to the front of declations where it is currently used.  This will allow it to work with the proper definitions on clang, clang on windows, gcc, and msvc

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1279466

8 years agoMSVC always_inline and noinline __attributes__ translation
Elizabeth Smith [Wed, 16 Apr 2014 20:45:37 +0000 (13:45 -0700)]
MSVC always_inline and noinline __attributes__ translation

Provide translations for gcc always_inline and noinline attribute

Change to using a macro from portability.h for attributes for using always_inline and noinline for cross platform support

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1279652

8 years agoUse StringPiece instead of String
Victor Loh [Wed, 16 Apr 2014 04:57:23 +0000 (21:57 -0700)]
Use StringPiece instead of String

Using StringPiece makes it easier than std::string

Facebook: Motivation for this diff is found in D1253595

Test Plan:
Added new unittest

fbconfig folly/io/test && fbmake runtests_opt

Reviewed By:

FB internal diff: D1276185

8 years agouint isn't portable and makes some compilers angry, change to unsigned int
Elizabeth Smith [Wed, 16 Apr 2014 01:52:19 +0000 (18:52 -0700)]
uint isn't portable and makes some compilers angry, change to unsigned int

Summary: As part of the windows port, uint is not defined as a type and is not standard, it makes msvc choke, a simple change to unsigned int fixes the issue

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1278487

8 years agoAdd non-const operator[] to TypedIOBuf
Tudor Bosman [Tue, 15 Apr 2014 16:24:11 +0000 (09:24 -0700)]
Add non-const operator[] to TypedIOBuf

Test Plan: used it

Reviewed By:

FB internal diff: D1277063


8 years agofolly: improve ProducerConsumerQueueBenchmark (misplaced doNotOptimizeAway)
Lucian Grijincu [Tue, 15 Apr 2014 03:15:10 +0000 (20:15 -0700)]
folly: improve ProducerConsumerQueueBenchmark (misplaced doNotOptimizeAway)

as per discussion from

After changing the queue length to be a small value (100) I didn't see
improvement from cache alignment of atomics (too much noise). Dropping
that and just fixing the benchmark.

Test Plan: n/a

Reviewed By:

FB internal diff: D1276591

8 years agoFix apparent race in SubprocessTest
Nicholas Ormrod [Mon, 14 Apr 2014 21:21:11 +0000 (14:21 -0700)]
Fix apparent race in SubprocessTest

The code originally contained the line

EXPECT_EQ((rcount == lineCount), r); // EOF iff at lineCount

Which asserted that, on the last iteration, EOF was being read (and thus
r was true).

This line has been causing errors in contbuild. There seem to be two
issues at play.

First, the left hand side of the EXPECT_EQ is always false. rcount is
incremented three lines down, and the (implicit) loop ends when rcount
equals lineCount. This means that, on the last iteration, rcount is
actually one less than lineCount. Hence the check is effectively
asserting that r is false.

The second issue is that r is rarely true. Empirically, a necessary but
not sufficient condition is that there be lots of other processes
running at the same time (such as when running fbconfig -r folly &&
fbmake runtests_opt). This would explain why the test passes during
development but fails in contbuild.

I am guessing that, when there are few processes running, Subprocess is
predictably ceasing to read before whichever other process yields EOF,
and that this does not always happen when there is resource contention.

As an immediate fix, I am asserting that r may only be true on the last
iteration. Could someone (cc @tudorb) who is more familiar with the
intricacies of Subprocess please check to see if this is an idiosyncrasy
of the test, or if there is actually an underlying bug.

Test Plan:
Before the change, run fbconfig -r folly && fbmake runtests_opt
repeatedly. Notice that most of the iterations yield an error from line
376 (which is consistent with contbuild error logs).

After the change, notice that the error disappears.

Reviewed By:

FB internal diff: D1269124

8 years agofolly: gen: pmap: parallel version of map
Lucian Grijincu [Sun, 13 Apr 2014 06:10:11 +0000 (23:10 -0700)]
folly: gen: pmap: parallel version of map

same as map, but runs it's argument in parallel over a number of threads.


Test Plan: added test

Reviewed By:

FB internal diff: D1258364

8 years agoChange child's working directory
Alexey Spiridonov [Thu, 10 Apr 2014 07:26:23 +0000 (00:26 -0700)]
Change child's working directory

Summary: Changing the parent's WD is prone to race conditions of various sorts, and needlessly alters parent state. Python's subprocess also has this feature.

Test Plan: fbmake dbg _bin/folly/test/subprocess_test ; ../_bin/folly/test/subprocess_test

Reviewed By:

FB internal diff: D1269526

8 years agofolly: allow to build with (flex arrays vs. lambda)
Jim Meyering [Fri, 4 Apr 2014 20:14:20 +0000 (13:14 -0700)]
folly: allow to build with (flex arrays vs. lambda)

* folly/test/CacheLocalityTest.cpp (contentionAtWidth): Work around
clang's prohibition against using flexible arrays in a lambda.

Test Plan:
fbconfig --clang --with-project-version clang:dev folly/test \
&& fbmake runtests

Reviewed By:

FB internal diff: D1263620

8 years agoconfigurable alignment for Arena
Philip Pronin [Tue, 8 Apr 2014 18:38:55 +0000 (11:38 -0700)]
configurable alignment for Arena

Summary: Allow specifying custom alignment.

Test Plan: fbconfig -r folly/test && fbmake opt -j32


Reviewed By:

FB internal diff: D1264851

8 years agoMake header compatible with warp
ptarjan [Wed, 9 Apr 2014 04:13:38 +0000 (21:13 -0700)]
Make header compatible with warp

8 years agofix ConcurrentSkipList::Recycler layout
Philip Pronin [Tue, 8 Apr 2014 18:08:02 +0000 (11:08 -0700)]
fix ConcurrentSkipList::Recycler layout

D1261546 introduced regression in `sizeof(ConcurrentSkipList)`.
In case of `NodeAlloc = folly::SysAlloc`, size of an empty `NodeAlloc` struct
is 1 byte, due to the alignment requirements of the next field, it got
translated into wasting 8 bytes.

Test Plan: fbconfig -r folly/test && fbmake opt -j32


Reviewed By:

FB internal diff: D1264730

8 years agofolly: accommodate fact that clang's strlen is not constexpr
Jim Meyering [Tue, 8 Apr 2014 04:11:54 +0000 (21:11 -0700)]
folly: accommodate fact that clang's strlen is not constexpr

* folly/folly-config.h (FOLLY_HAVE_CONSTEXPR_STRLEN): Define to
zero when compiling with clang, since clang's strlen is not constexpr.

Test Plan:
Run this:
fbconfig --clang --with-project-version clang:dev folly/test && fbconfig dbg
and observe that the following no longer strikes:
folly/test/RangeTest.cpp:304:25: error: constexpr variable 'hello1' must be initialized by a constant expression

I also tried using clang-3.4 (the current default),
fbconfig --clang folly/test && fbmake dbg
with these results:

Reviewed By:

FB internal diff: D1262840

8 years agoFix make check
Dave Watson [Fri, 4 Apr 2014 19:17:54 +0000 (12:17 -0700)]
Fix make check

Summary: Fix make check.  Benchmark.cpp was included twice, eventfd doesn't exist anymore.

Test Plan: autoreconf -i; ./configure; make check on ubuntu

Reviewed By:

FB internal diff: D1259706

8 years agofix automake
Dave Watson [Thu, 3 Apr 2014 20:40:58 +0000 (13:40 -0700)]
fix automake

Automake complains there are two Malloc.cpp rules - rename one of them.

Facebook: Actually this file doesn't even build in fbconfig - is it still necessary?

Test Plan: autoreconf -i on ubuntu doens't fail

Reviewed By:

FB internal diff: D1257372

8 years ago(wangle) codegen then() variant tests
Hans Fugal [Mon, 7 Apr 2014 16:57:42 +0000 (09:57 -0700)]
(wangle) codegen then() variant tests

When all the variants I am dreaming of work, there's a little more than
100 tests generated.

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1258754

8 years ago(Folly/FBString) Fix compilation time of test with opt
Hannes Roth [Mon, 7 Apr 2014 16:44:38 +0000 (09:44 -0700)]
(Folly/FBString) Fix compilation time of test with opt

Summary: Now compiles in 18 seconds. Took two minutes before.

Test Plan: `fbmake runtests_opt`

Reviewed By:

FB internal diff: D1243689

8 years agoconfigurable node allocator for ConcurrentSkipList
Philip Pronin [Sat, 5 Apr 2014 21:54:37 +0000 (14:54 -0700)]
configurable node allocator for ConcurrentSkipList

Allow specifying `SimpleAllocator` to use for node allocation.
Added optimizations for arena allocators.

Test Plan: fbconfig -r folly/test && fbmake opt -j32

Reviewed By:

FB internal diff: D1261546

8 years agomake ConcurrentSkipList ctor public
Philip Pronin [Sat, 5 Apr 2014 04:50:27 +0000 (21:50 -0700)]
make ConcurrentSkipList ctor public

There is no reason to force heap allocation for

Test Plan: fbconfig -r unicorn/test && fbmake opt -j32


Reviewed By:

FB internal diff: D1261183

8 years agoAdding support for signed integers
Tom Jackson [Fri, 4 Apr 2014 23:55:09 +0000 (16:55 -0700)]
Adding support for signed integers

Summary: For use with Thrift::Frozen, especially.

Test Plan: Unit tests

Reviewed By:

FB internal diff: D1255257

8 years agoBugfix in iterator emplace
Nicholas Ormrod [Fri, 4 Apr 2014 03:02:55 +0000 (20:02 -0700)]
Bugfix in iterator emplace

Subtle bugfix to fbvector

Reproduction requirements:
- emplace into a vector at a non-end position
- the vector has enough capacity for the extra elements
- the value type's internal state is poisoned after a move

What should happen, explained by picture:


want to insert two elements here

FIRST STEP: uninitialized move-construction
x and y are initialized, but could be in an invalid state


SECOND STEP: move-assignment, in reverse order


What actually happens:
An indexing error causes the entire tail (##1234xy##) to be
move-assigned, instead of the intended head of the tail (##1234##).

If the data type's move operator does not invalidate its own data (i.e.
move is essentially a copy, as is the case with atomic types) then the
move assignment of ##y## onto ##Y## and ##x## onto ##X## is basically a
no-op. However, for types that do invalidate their own data when being
the source of a move (e.g. non-empty vectors and fbvectors), then the
end result will have bad data in place of ##X## and ##Y##.

This bug has lain dormant for a while.

Very few people emplace into a vector, and those few who do could be
saved by using a copy-movable type like int.

The original test case did not use a data type which poisoned itself
after being the source of a move, so the tests did not notice any

A new testsuite, which does poison itself, did notice.

Test Plan:
re-enable the test/stl_test mega-test. Run it. All tests

fbconfig -r folly && fbmake runtests_opt

Run fbvector through the new test suite (which initially caught the
error). It now passes.

Reviewed By:

FB internal diff: D1257258

8 years agoA few missing includes
Hans Fugal [Thu, 3 Apr 2014 18:05:32 +0000 (11:05 -0700)]
A few missing includes

Summary: These were required to build folly on OSX.

Test Plan: build

Reviewed By:

FB internal diff: D1256856

8 years ago(wangle) Use an atomic_flag to make FutureObject threadsafe
Hans Fugal [Thu, 3 Apr 2014 15:48:29 +0000 (08:48 -0700)]
(wangle) Use an atomic_flag to make FutureObject threadsafe

There are two operations on `FutureObject` (the backing object that both `Future` and `Promise` front), which are not threadsafe currently. Those are `setContinuation` and `fulfil`. They're not threadsafe because they have a race condition between checking to see whether they should run the continuation and the other side making its modification. This diff introduces a `std::atomic_flag` variable to avoid the race and make these threadsafe.

NB Making `Future`/`Promise` threadsafe does not make this pattern safe:


(In a hypothetical world where `Future::via` is another name for `executeWithSameThread`)
There is a race between the future from `via` and the call of `then` (and between the first and second `then`s), so the `then`s could execute in the far thread as intended, or they could execute in the current thread (probably not what was intended). This diff does not solve that semantic problem of which thread this runs in. But it does make it safer, in that all those continuations will always execute, just maybe not in the thread you had intended.

You may ask what is the benefit if that semantic problem still exists? That's up for debate. I think the safety is worth it: at least everything a n00b throws at it will *work*. The question of thread semantics can iterate. If we end up with thread semantics that really don't need locks (e.g. maybe because we move to an Rx/Later style "launch" at the end of a chain) we can always toss this atomic.

How does this affect performance? I'm doing some experiments and going to write up my findings. Early results indicate that outlier performance is impacted but not by a lot, but don't count those chickens until I hatch them. Part of the reason for sending this diff out is to run windtunnel experiments.

Test Plan: tests, benchmark, close inspection

Reviewed By:

FB internal diff: D1241822

Louis Kruger [Tue, 1 Apr 2014 20:22:42 +0000 (13:22 -0700)]

TIMED_SYNCHRONIZED_CONST is acquiring a read-write lock but releasing a shared lock.  Chaos ensues.

Test Plan: Add unit test, test times out before fix, passes after fix.

Reviewed By:

FB internal diff: D1251518

Blame Revision: D327492

8 years agoAdd ability to silence callbacks for Subprocess::communicate
Tudor Bosman [Tue, 1 Apr 2014 17:36:27 +0000 (10:36 -0700)]
Add ability to silence callbacks for Subprocess::communicate

Subprocess::communicate callbacks are level-triggered, which makes writing
"chatty" communication protocols difficult -- you often want to silence the
write callback until you read the expected message. Add
Subprocess::enableNotifications() for this purpose.

Test Plan: test added

Reviewed By:

FB internal diff: D1251564


8 years agoAdd loopOnce() method to folly::EventBase
Stepan Palamarchuk [Mon, 31 Mar 2014 21:19:54 +0000 (14:19 -0700)]
Add loopOnce() method to folly::EventBase

Summary: Add loopOnce() method which provides functionality similar to event_base_loop(base, EVLOOP_ONCE);

Test Plan: fbmake

Reviewed By:

FB internal diff: D1249753

8 years agoUtility that converts from prettyPrint format back to double (e.g. 10M = 10 000 000)
Mikhail Okunev [Mon, 31 Mar 2014 18:51:55 +0000 (11:51 -0700)]
Utility that converts from prettyPrint format back to double (e.g. 10M = 10 000 000)

Summary: Utility is doing reverse of prettyPrint. This is a first sketch.

Test Plan:
1) We can reverse all tests from prettyPrint test and use for our function. For now I just reversed several of them
2) We can also test prettyPrint and prettyToDouble together by checking whether prettyToDouble(prettyPrint(X)) == X. This is implemented.

Reviewed By:

FB internal diff: D1159879

8 years ago(Folly/FBString) Use a lambda to throw an exception for NULL input
Hannes Roth [Thu, 27 Mar 2014 21:47:00 +0000 (14:47 -0700)]
(Folly/FBString) Use a lambda to throw an exception for NULL input

This should be standard compliant. `__PRETTY_FUNCTION__` seems
unnecessary, it just prints `basic_fbstring`, might as well use that and
simplify the code.

Test Plan:
`fbconfig --platform-all=gcc-4.8.1-glibc-2.17-fb folly/test`
`fbmake runtests_dbg`
`fbmake runtests_opt`

`fbconfig --platform-all=gcc-4.7.1-glibc-2.14.1 folly/test`
`fbmake runtests_dbg`
`fbmake runtests_opt`

`fbconfig --clang folly/test`

Reviewed By:

FB internal diff: D1243670

8 years agoNULL -> nullptr
Nicholas Ormrod [Wed, 26 Mar 2014 09:33:48 +0000 (02:33 -0700)]
NULL -> nullptr

Change NULLs to nullptrs.

I was tired of seeing lint errors for NULL -> nullptr, so I
fixed it.

How: modified flint's tokenizer to output lines of the form

sed -i '__LINE__s|\<NULL\>|nullptr|g' __FILE__

for each NULL token encountered.

Ran the sed lines, restricted to filepaths with extensions
h,hpp,cc,cpp,tcc. Did not apply to FacebookUpdate, due to weird
formatting; hphp, to be done in a separate diff; payment, due to a large
number of generated text which didn't play well with my flow; and
memcache, which has lots of .h's that cannot contain nullptr's because
they are included from .c's.


Test Plan:
fbconfig -r common folly && fbmake opt && fbmake runtests_opt
^ of the 4k+ test cases, about two dozen are failing. Slightly more
failed in master than with my diff applied.

arc unit

Reviewed By:

FB internal diff: D1234261

8 years agoRange::contains(x)
Tom Jackson [Mon, 24 Mar 2014 18:54:02 +0000 (11:54 -0700)]

Test Plan: Unit tests

Reviewed By:

FB internal diff: D1236630

8 years agoOptionally specify page size in MemoryMapping ctor
Tudor Bosman [Thu, 20 Mar 2014 20:53:44 +0000 (13:53 -0700)]
Optionally specify page size in MemoryMapping ctor

For proper rounding, and also so memOpInChunks doesn't try to mlock / munlock /
madvise / munmap in chunks smaller than the page size.

Test Plan: memory_mapping_test

Reviewed By:

FB internal diff: D1232153

8 years ago(Wangle) Sanity check for the small_vector test
Hannes Roth [Thu, 20 Mar 2014 17:11:14 +0000 (10:11 -0700)]
(Wangle) Sanity check for the small_vector test

Let's double check that `small_vector` is doing the right thing. So
it doesn't just accidentally work.

Test Plan:
`fbconfig folly/wangle && fbmake runtests`
`fbconfig --clang folly/wangle && fbmake runtests`

Reviewed By:

FB internal diff: D1231360

8 years agoEventBase cleanup
Dave Watson [Thu, 13 Mar 2014 17:18:56 +0000 (10:18 -0700)]
EventBase cleanup

Summary: Moar c++11 features.  marccelani's comments in D1195393, but in a separate diff.

Test Plan:
fbconfig thrift/lib/cpp/test/
fbmake runtests

Reviewed By:

FB internal diff: D1219158

8 years agoPerformance Improvement for AsciiCaseInsensitive
Nima Aghdaii [Tue, 11 Mar 2014 00:58:29 +0000 (17:58 -0700)]
Performance Improvement for AsciiCaseInsensitive

This implementation improves the performance of AsciiCaseInsensitive.
AsciiCaseInsensitive uses toupper, and it's slow.
Since toupper only works for ascii characters and this function does exactly the same thing over the range of [0..256), we can gain 3x performance here.

Test Plan: unit test and benchmark

Reviewed By:

FB internal diff: D1217596


8 years ago(Wangle) Fix tests for clang
Hannes Roth [Wed, 19 Mar 2014 23:45:58 +0000 (16:45 -0700)]
(Wangle) Fix tests for clang

Summary: `std::async` doesn't play nicely with our clang. Replace it with `std::thread`.

Test Plan: `fbconfig --clang folly/wangle && fbmake runtests`

Reviewed By:

FB internal diff: D1230197

8 years agoHarden usingJEMalloc()
Tudor Bosman [Wed, 19 Mar 2014 04:57:17 +0000 (21:57 -0700)]
Harden usingJEMalloc()

In a shared library, we're using whichever memory allocator the main program
uses, but we may still believe we're using jemalloc if we load
(because rallocm is defined). This means we'll pass to rallocm pointers that
were not allocated by jemalloc, which is bad.

Test Plan: tests added

Reviewed By:

FB internal diff: D1228483

8 years agorange(), for making Range<T*> from arrays and std::vector
Tom Jackson [Wed, 12 Mar 2014 23:31:41 +0000 (16:31 -0700)]
range(), for making Range<T*> from arrays and std::vector

Summary: So people can more easily write functions which take contiguous values from memory.

Test Plan: Unit tests

Reviewed By:

FB internal diff: D1217809

8 years agoAdd constexpr specifier for small_vector::max_size
Tianjiao Yin [Tue, 18 Mar 2014 00:33:48 +0000 (17:33 -0700)]
Add constexpr specifier for small_vector::max_size

Summary: Now we could use small_vector::max_size in constant expressions

Test Plan: unittest

Reviewed By:

FB internal diff: D1224274

8 years agoUnbreak Symbolizer with small ElfCache
Tudor Bosman [Mon, 17 Mar 2014 20:24:07 +0000 (13:24 -0700)]
Unbreak Symbolizer with small ElfCache

Test Plan: fbconfig -r folly/experimental/symbolizer && fbmake runtests_opt && fbmake runtests

Reviewed By:

FB internal diff: D1224552


8 years agoMove commited bits out of folly
Alexey Spiridonov [Mon, 17 Mar 2014 22:59:25 +0000 (15:59 -0700)]
Move commited bits out of folly

Summary: Preparing to land the other diffs. No code changes here, just moving files around.

Test Plan: fbconfig bistro/common/cron/test bistro/common/cron/utils/test ; fbmake runtests

Reviewed By:

FB internal diff: D1225159

8 years agosorted_vector_types have move inserts
Marc Celani [Mon, 17 Mar 2014 16:17:50 +0000 (09:17 -0700)]
sorted_vector_types have move inserts

Summary: sorted_vector_types were missing move inserts that come with std::map in c++11. This prevents me from using move iterators in folly::merge use case.

Test Plan: unit tests

Reviewed By:

FB internal diff: D1223426

8 years agofolly::merge() - std::merge() with stronger guarantees (probably same implementation...
Marc Celani [Sun, 16 Mar 2014 03:04:47 +0000 (20:04 -0700)]
folly::merge() - std::merge() with stronger guarantees (probably same implementation in practice)

std::merge() does not guarantee the ordering when equal elements belong in two ranges(comparator(it_a, it_b) == comparator(it_b, it_a) == 0). For maps, it is important that we can specify the ordering (see array_merge in php, where we guarantee which array's value will be present in the output if a key is present in both inputs).

Also removes folly::merge that is specfic for sorted_vector_map since this will not be needed. NOTE: I expect this to break feed, will fix in a separate non-folly diff.

Test Plan: This implementation is directly ripped from, but unit tests added none-the-less. Specifically, one is added where the output is a std::map to demonstrate its usefulness.

Reviewed By:

FB internal diff: D1223401


8 years agosorted_vector_types does not work with std::inserter
Marc Celani [Sun, 16 Mar 2014 03:00:45 +0000 (20:00 -0700)]
sorted_vector_types does not work with std::inserter

The signature of insert(hint, const val& v) is wrong. It should return an iterator, not pair<iterator, bool>. This breaks std::inserter, making it impossible to use this for std::merge. I plan to create folly::merge (std::merge with stronger semantics when equal values are present in the two input ranges), so I need this fixed.

Given that the documentation claims this is a "drop in replacement" for std::map, I figure we should fix this.

Test Plan: reran unit tests

Reviewed By:

FB internal diff: D1223396

8 years agoIntroduce a merge() function for folly::sorted_vector_map
Marc Celani [Fri, 14 Mar 2014 02:52:09 +0000 (19:52 -0700)]
Introduce a merge() function for folly::sorted_vector_map

Feed spends a non trivial amount of time merging two sorted_vector_maps. Writing code for this efficiently is actually a little tricky, and its much easier to maintain the simpler code. This adds merge() to folly so that its easy for feed and fast. Benchmarks with large input sizes show this is a gigantic win (moving from O(n^2) to O(n).

Test Plan: unit tests, benchmark

Reviewed By:

FB internal diff: D1221725


8 years agoMake sure folly::dynamic arrays can be used with folly::gen
Marcus Holland-Moritz [Thu, 13 Mar 2014 00:27:09 +0000 (17:27 -0700)]
Make sure folly::dynamic arrays can be used with folly::gen

Define a value_type for folly::dynamic to make arrays work with

Test Plan:
- added new unit test
- fbconfig folly/test && fbmake runtests

Reviewed By:

FB internal diff: D1216358

8 years agoMove thrift/lib/cpp/async to folly.
Dave Watson [Wed, 12 Mar 2014 16:25:43 +0000 (09:25 -0700)]
Move thrift/lib/cpp/async to folly.

Move the minimum amount of stuff and still have everything compile.  Would like to move TAsyncSocket/ServerSocket/SSL/UDP eventually, but not this round.

thrift async is used very widely now: thrift, proxygen, newer mysql async interface, even trying it out in memcache.  A common complaint is that it doesn't get wide enough notice under thrift/, so let's move it to folly/.  After moving TAsyncSocket, both HHVM and proxygen could avoid a dep on thrift as well.

* mv files to folly/io/async
* remove 'T' prefix on classes/filenames
* change namespace to 'folly'
* remove any thrift references.

Tried this before in D470080, this time doesn't attempt to abstract libevent


Test Plan:
fbconfig -r thrift; fbmake dev.
Will iterate on any other contbuild failures

Reviewed By:

FB internal diff: D1195393

Philip Pronin [Wed, 12 Mar 2014 08:00:52 +0000 (01:00 -0700)]

Let's make sure that boost version is intented to mean exactly
the same as `std::is_trivially_copyable` (see 9 [class] / 6) to avoid any
confusion (current boost path is more close to
`std::is_trivially_copy_constructible` than to
`std::is_trivially_copyable`;  UPD: unfortunately, old versions of boost
don't support `boost::has_trivial_move_constructor` and friends, so I
can't completely mimic `std::is_trivially_copyable` here).

As mentioned in the original comment, `__has_trivial_copy` returns 1 in
case of deleted copy ctor in clang (contradicts 12.8 [class.copy] / 13 +
8.4.2 [dcl.fct.def.default] / 4), which makes `FOLLY_IS_TRIVIALLY_COPYABLE`
accept `std::unique_ptr<>`; using `boost::has_trivial_destructor` would
at least save us from the problems in this case.

Alternative "solution" may be to rename `FOLLY_IS_TRIVIALLY_COPYABLE` to
`folly::small_vector` won't call dtor when memcopies the data around.

Test Plan: fbconfig --clang folly/test:small_vector_test && fbmake runtests_opt (fails in trunk, passes with this diff)

Reviewed By:

FB internal diff: D1216158

8 years agoAdd control of floating point serialization to JSON serialization
David Vickrey [Mon, 10 Mar 2014 22:55:41 +0000 (15:55 -0700)]
Add control of floating point serialization to JSON serialization

Summary: Title.  Note that this is a no-op with default parameters because folly::toAppend(double, &dest) calls folly::toAppend(double, &dest, DtoaMode, numDigits) with DtoaMode = SHORTEST and numDigits = 0.

Test Plan: Tested new functionality in D1212547.

Reviewed By:

FB internal diff: D1212617

8 years agofix Cursor's copy constructor
Adam Simpkins [Thu, 6 Mar 2014 00:28:09 +0000 (16:28 -0800)]
fix Cursor's copy constructor

The Cursor class didn't provide a copy constructor accepting a
"const Cursor&".  The existing constructor only accepted references to
non-const Cursor.

This updates the constructor to also accept "const Cursor&".  I also
made the template parameter checking more strict, so that it only
accepts other CursorBase classes.  This prevents the compiler from
thinking that a Cursor can be constructed from any other possible type
when performing overload resolution.

Test Plan: Ran all of the folly cursor tests.

Reviewed By:

FB internal diff: D1208428

8 years agoSupport numeric types as targets for folly::split
Marcus Holland-Moritz [Mon, 10 Mar 2014 18:35:00 +0000 (11:35 -0700)]
Support numeric types as targets for folly::split

This extends the fixed version of folly::split to support numeric types
as targets in addition to string pieces. I was almost certain this was
already possible when I recently reviewed some code that did

folly::StringPiece source, target, strFrequency;
int frequency;
if (folly::split('\t', line, source, target, strFrequency) &&
(frequency = folly::to<unsigned int>(strFrequency)) > 0)

and was about to suggest changing the above into:

folly::StringPiece source, target;
int frequency;
if (folly::split('\t', line, source, target, frequency) && frequency > 0)

I double checked and saw that only splitting to string pieces was supported.

In the meantime I came across this pattern again and decided to just make
it work because it's really convenient.

The implementation should be fully backwards compatible.

Test Plan:
- New unit tests
- fbconfig -r folly && fbmake runtests
- Applied to github release, ./configure && make check

Reviewed By:

FB internal diff: D1187004

8 years agoAdd setPort() to folly::URI class.
Will Hodges [Mon, 10 Mar 2014 17:15:35 +0000 (10:15 -0700)]
Add setPort() to folly::URI class.

Add setPort() method to URI class for easy port replacement in

Perform port replacement in WriteProcessor.cpp in a way that
supports URI schemes. Add setPort() method to URI class to replace
the port number in the authority portion of the URI.

Test Plan: Test in debugger for dfsrouter changes.

Reviewed By:

FB internal diff: D1209120

8 years agofolly/experimental/Bits.h + ASan
Philip Pronin [Fri, 7 Mar 2014 08:49:24 +0000 (00:49 -0800)]
folly/experimental/Bits.h + ASan

Summary: Allow disabling ASan for `folly::Bits`.

Test Plan: [fb-only]

Reviewed By:

FB internal diff: D1208584

8 years agoFix comment
Andrey Goder [Fri, 7 Mar 2014 22:50:07 +0000 (14:50 -0800)]
Fix comment

Summary: This should be 'docs' not 'doc'. Too bad now it exceeds 80 chars :(.

Test Plan: n/a

Reviewed By:

FB internal diff: D1209738


8 years ago(Wangle) Support .then(function pointer) syntax
Hannes Roth [Fri, 7 Mar 2014 16:02:11 +0000 (08:02 -0800)]
(Wangle) Support .then(function pointer) syntax

Support `.then(&static_function)`, `.then(&static_member_function)`, `.then(&Class::static_member_function)`, `.then(instance, &Class::member_function)` in Wangle.

C++ does not allow `.then(instance, &member_function)`, sadly.

This implementation just creates the closure for you and defers to the existing `then` implementations.

Test Plan: Added a test.

Reviewed By:

FB internal diff: D1204954

8 years agouse 0666 as the default permissions for new files
Philip Pronin [Thu, 6 Mar 2014 01:45:28 +0000 (17:45 -0800)]
use 0666 as the default permissions for new files

Summary: Seems like there is no real need to restrict permissions by default?

Test Plan: fbconfig -r folly/test && fbmake runtests_opt


Reviewed By:

FB internal diff: D1205527

8 years ago(Wangle) Remove some early returns in tests
Hannes Roth [Thu, 6 Mar 2014 01:39:47 +0000 (17:39 -0800)]
(Wangle) Remove some early returns in tests

I guess they sneaked in when testing.

Also remove an `EXPECT_NE` that depended on `std::move` actually moving.

Test Plan: All the tests.

Reviewed By:

FB internal diff: D1204971

8 years ago(Wangle) Actually support the universal reference used in whenAll
Hannes Roth [Wed, 5 Mar 2014 17:50:28 +0000 (09:50 -0800)]
(Wangle) Actually support the universal reference used in whenAll

Need to use `std::decay` to actually support the universal reference.

The futures are moved in, and invalidated.

Test Plan: Test; does not compile before.

Reviewed By:

FB internal diff: D1199841

8 years agoReplace random_device/mt19937 with folly/Random
Shuo Li [Tue, 4 Mar 2014 19:55:59 +0000 (11:55 -0800)]
Replace random_device/mt19937 with folly/Random

Summary: Replace random_device/mt19937 with folly/Random to increase hygiene of using Random

Test Plan:
For each file is changed, fbmake -opt is run; except one file:, since the TARGETS file is not under the path.
All fbmake are successfully passing

Reviewed By:

FB internal diff: D1199483

Blame Revision: v1

8 years ago(Wangle) whenAll should take rvalue references instead of references
Hannes Roth [Mon, 3 Mar 2014 22:45:10 +0000 (14:45 -0800)]
(Wangle) whenAll should take rvalue references instead of references

Summary: Don't need to keep the old futures around.

Test Plan: `FutureTest.cpp`

Reviewed By:

FB internal diff: D1197360

8 years agoFix off-by-one affecting benchmark numbers in folly/io.
Yedidya Feldblum [Mon, 3 Mar 2014 22:33:02 +0000 (14:33 -0800)]
Fix off-by-one affecting benchmark numbers in folly/io.

Summary: Fix off-by-one affecting benchmark numbers in folly/io.

Test Plan:
$ fbconfig -r folly/io/test
$ fbmake --fast opt
$ _build/opt/folly/io/test/iobuf_cursor_test --benchmark
$ _build/opt/folly/io/test/iobuf_network_bench --benchmark

Reviewed By:

FB internal diff: D1199679

Blame Revision: D344018

8 years agoAdd Wangle
Hans Fugal [Mon, 3 Mar 2014 20:57:29 +0000 (12:57 -0800)]
Add Wangle

Moving from Dex.
Transliteration from whatever wiki format Dex uses to Markdown
Some minor corrections/additions.

Test Plan: Pasted into a private gist and it looks reasonable on github.

Reviewed By:

FB internal diff: D1199510

8 years agoMake it easy to wrap pre-existing cob-style async apis
Marc Celani [Mon, 3 Mar 2014 19:36:05 +0000 (11:36 -0800)]
Make it easy to wrap pre-existing cob-style async apis

Tao neesd a way to wrap its c-style async apis with Later. Although my comments suggest that you can do this, it turns out I never got around to implementing it. Well, here is the implementation.

Basically, we supply the callback to the pre-existing api, and that callback will fulfill a promise that is used internally within Later. This is thread safe because the async call is not made until the starter is fired, and we can use the future immediately for chaining then() calls.

Test Plan: unit test

Reviewed By:

FB internal diff: D1197721

8 years agoMaking folly/Benchmark.cpp clangable
Marcelo Juchem [Wed, 26 Feb 2014 23:46:51 +0000 (15:46 -0800)]
Making folly/Benchmark.cpp clangable

Summary: trunk fails to compile under clang:

Test Plan: ran some benchmarks and checked output

Reviewed By:

FB internal diff: D1192933

8 years ago(Folly) Fix error when compiling with clang
Hannes Roth [Mon, 3 Mar 2014 17:18:59 +0000 (09:18 -0800)]
(Folly) Fix error when compiling with clang

folly/Benchmark.cpp:427:9: error: logical not is only applied to the left hand side of this comparison
if (!strcmp(get<1>(benchmarks[i]), "-") == 0) {

Since the `!` binds to `strcmp`, removing both the `!` and the `== 0`
should be equivalent.

Reviewers of diff that introduced this line: D490324

Test Plan: I compiled `tao/client` with/without this diff.

Reviewed By:

FB internal diff: D1196452

Blame Revision: rFBCODE6f5eea5fcd23044a5c579b040bf45d17006b4adf

8 years agoMake Folly cpp_library targets more granular
Peter Griess [Tue, 25 Feb 2014 23:54:23 +0000 (15:54 -0800)]
Make Folly cpp_library targets more granular

- Some users of Folly don't want to pull in the entire library just to
use one feature (e.g. folly::to<T>). Throw those users a bone by
breaking Folly up into a few discrete chunks: :uri, :range, :string,
:format, etc. Leave the top-level :folly target alone so that existing
users don't have to be changed.
- Update some Thrift and Proxygen users to use the more granular

Test Plan: - Phabricator contbuild

Reviewed By:

FB internal diff: D1192765

8 years agokill unused vars in folly
Paul Tarjan [Fri, 28 Feb 2014 09:25:57 +0000 (01:25 -0800)]
kill unused vars in folly

Summary: these print as errors when compiling hhvm opensource

Test Plan: none

Reviewed By:

FB internal diff: D1195649

8 years agoEasy: fix signed/unsigned comparisons
Anton Likhtarov [Thu, 27 Feb 2014 01:51:49 +0000 (17:51 -0800)]
Easy: fix signed/unsigned comparisons

Test Plan: build with -Wsign-compare, no warnings

Reviewed By:

FB internal diff: D1193502

8 years agomake FB_LOG_EVERY_MS thread safe
Nathan Bronson [Thu, 27 Feb 2014 17:32:50 +0000 (09:32 -0800)]
make FB_LOG_EVERY_MS thread safe

Use compare_exchange_strong to update the previous time so that
there is no race.  This diff also adds microbenchmarks.

Test Plan: unit tests

Reviewed By:

FB internal diff: D1191217

8 years ago(wangle) more comment tweaks
Hans Fugal [Sat, 22 Feb 2014 00:22:15 +0000 (16:22 -0800)]
(wangle) more comment tweaks

Test Plan: eyeballs

Reviewed By:

FB internal diff: D1185813

8 years ago(wangle) comment cleanup
Hans Fugal [Fri, 21 Feb 2014 18:52:22 +0000 (10:52 -0800)]
(wangle) comment cleanup

Summary: Cleaning up some inaccuracies, adding just a little detail in a few places, and trying very hard not to get OCD about the commenting style.

Test Plan: still builds

Reviewed By:

FB internal diff: D1184910

8 years agos/valueTry/getTry/
Hans Fugal [Fri, 21 Feb 2014 18:33:24 +0000 (10:33 -0800)]

Summary: This is a better name

Test Plan: unit tests. fbgs shows no external usage of this.

Reviewed By:

FB internal diff: D1184905

8 years ago(wangle) remove Future::wait
Hans Fugal [Fri, 21 Feb 2014 18:15:31 +0000 (10:15 -0800)]
(wangle) remove Future::wait

Summary: pulling the trigger

Test Plan: Unit tests. Wait for contbuild and dependent unit tests

Reviewed By:

FB internal diff: D1184842

8 years agoTry Contains enum as an enum class
Hans Fugal [Fri, 21 Feb 2014 18:10:03 +0000 (10:10 -0800)]
Try Contains enum as an enum class

Summary: Somebody set us up the enum

Test Plan: Unit tests still build and pass. Will see if dependencies build and their unit tests pass. But I don't think anyone uses this outside of `Try` (they shouldn't anyway).

Reviewed By:

FB internal diff: D1184839

8 years agoAdd method, that constructs Try<T> based on the result of functor.
Stepan Palamarchuk [Tue, 25 Feb 2014 22:42:55 +0000 (14:42 -0800)]
Add method, that constructs Try<T> based on the result of functor.

Usually we construct Try<T> as a result of execution of some functor.
And we need to treat void and non-void functors in similar way, but that requires different, yet similar pieces of code.

This diff simplifies future usage of Try.

Test Plan: compile only

Reviewed By:

FB internal diff: D1190296

8 years agoadd operator +, ==, and != to Cursor classes
Adam Simpkins [Thu, 20 Feb 2014 23:28:50 +0000 (15:28 -0800)]
add operator +, ==, and != to Cursor classes

Add operator+, which returns a new Cursor pointing the specified number
of bytes ahead.  Cursor already implemented a += operator.

Also add == and != operators for checking to see if two cursors are
pointing to the same location.  Note that some derived cursor classes do
contain additional state, and we don't consider that state when
comparing for equality.  For instance, two Appenders pointing to the
same location will compare as equal, even if they are configured with
different growth parameters.  It seems like this is the behavior most
users will expect, but let me know if you have concerns about this.

Test Plan: Updated the unit tests to exercise the new operators.

Reviewed By:

FB internal diff: D1183537

8 years agofolly: install missing headers from
Andrew Gallagher [Tue, 25 Feb 2014 22:18:00 +0000 (14:18 -0800)]
folly: install missing headers from

Test Plan: Built folly on ubuntu 13.04 and 13.10

Reviewed By:

FB internal diff: D1190155

8 years agoFix inclusions of double-conversion to use proper path
Andrew Gallagher [Tue, 25 Feb 2014 19:53:29 +0000 (11:53 -0800)]
Fix inclusions of double-conversion to use proper path

Summary: Will commit in tandem with D1171409.

Test Plan: built folly

Reviewed By:

FB internal diff: D1189713

8 years agoFix random tests
Ben Maurer [Tue, 25 Feb 2014 18:40:58 +0000 (10:40 -0800)]
Fix random tests

Summary: Was broken but didn't notice it in all the random test failures

Test Plan: fbmake runtests

Reviewed By:

FB internal diff: D1189113

8 years agoMove wangle to folly
Dave Watson [Fri, 21 Feb 2014 20:59:25 +0000 (12:59 -0800)]
Move wangle to folly

* git mv
* codemod facebook::wangle folly::wangle
* Change 'runnable' to be a base class in wangle instead of thrift

* std::future doesn't have then, whenall, etc.
* boost::future doesn't support executors


Test Plan: contbuild and pray

Reviewed By:

FB internal diff: D1185194

8 years agoLeak ELF file cache that the signal handler uses
Tudor Bosman [Mon, 24 Feb 2014 23:39:46 +0000 (15:39 -0800)]
Leak ELF file cache that the signal handler uses

Summary: so we can still handle signals during global destruction. (d'oh)

Test Plan: unittests

Reviewed By:

FB internal diff: D1188159

8 years agoHarden failure signal handler in the face of memory corruptions
Tudor Bosman [Mon, 24 Feb 2014 22:56:23 +0000 (14:56 -0800)]
Harden failure signal handler in the face of memory corruptions

@mhorowitz got

*** Aborted at 1393267847 (Unix time, try 'date -d @1393267847') ***
*** Signal 11 (SIGSEGV) (0x0) received by PID 12652 (TID 0x7f59cbfff700), stack trace: ***
pure virtual method called
terminate called without an active exception
Entered fatal signal handler recursively. We're in trouble.

in a test, and no stack trace.

The first time we enter recursively (ie. the second time overall), try to dump
without symbolization. The next time, give up.

Test Plan: folly/experimental/symbolizer/test, ran crash with a modified dumpStackTrace to force it to enter recursively

Reviewed By:

FB internal diff: D1187942

8 years agoRandom number generator utilities for folly
Ben Maurer [Tue, 25 Feb 2014 02:37:24 +0000 (18:37 -0800)]
Random number generator utilities for folly

In looking at how people were using common/base/Random, I noticed
a number of issues with our current usage of random number generators

1) People would simply declare a RandomInt32 without seeding it. This
results in a predictable seed
2) We initialize a Mersenne Twister RNG from a single int32. This
causes us to have a more predictable starting sequence of numbers
3) People aren't consistently using thread-local RNGs
4) Many of the APIs lack consistency. For example random32 takes a
max parameter that is exclusive while uniformRandom32 uses inclusive

I'm hoping a better API can fix this. thread_prng implements the Generator
concept with result_type = int32. It isn't actually a random number generator,
but it uses a thread local to point to a real generator. An advantage
of this is that it can be used in existing APIs but that it doesn't expose
the implementation of the RNG via the header file. One thing that's a bit
weird about it is that if you copy the object across threads it could
cause an error.

The Random class provides utilities that take any type of random number
generator. This has the advantage of allowing the user to pass a RNG
meant for testing or a secure RNG based on /dev/random. Another advnatage
is if you're woried about the performance of TLS lookups, you can
cache a local thread_prng which memoizes the TLS lookup.

If available, we use a SIMD optimized MT API

Some open questions:

1) What functions should be in random
2) Should the default RNG be a 64 or 32 bit based RNG

Test Plan: Benchmark runs

Reviewed By:

FB internal diff: D1181864

8 years agoFix clang folly::to failure
Tom Conerly [Tue, 18 Feb 2014 07:10:21 +0000 (23:10 -0800)]
Fix clang folly::to failure

Converting an enum class to a string fails with clang.
Adfinder clang build failure is
Our version of clang supports std::underlying_type so use that implementation instead.

Test Plan:
fbconfig -r folly && fbmake runtests (only failure is folly/experimental/symbolizer/test:symbolizer_test which failed before this diff)
fbconfig --clang folly/test:conv_test && fbmake runtests (fbconfig -r --clang folly fails and fbconfig --clang folly/test && fbmake has some unrelated compile errors)
Build adfinder with clang

Reviewed By:

FB internal diff: D1178770

8 years agoFix FBString's new includes
Nicholas Ormrod [Mon, 24 Feb 2014 20:53:47 +0000 (12:53 -0800)]
Fix FBString's new includes

The headers <ext/hash*> were being included but not used when
_LIBSTDCXX_FBSTRING is defined. They have been relocated to within the
appropiate header block.

Test Plan:
fbconfig -r folly && fbmake runtests_opt

copy FBString and Malloc into libgcc, then
tp2_build libgcc/4.8.1/gcc-4.8.1-glibc-2.17-fb

Reviewed By:

FB internal diff: D1187345