folly.git
11 years agoconvert RecordIO to pwritev
Tudor Bosman [Thu, 9 May 2013 21:27:56 +0000 (14:27 -0700)]
convert RecordIO to pwritev

Summary: zero-copy

Test Plan: record_io_test

Reviewed By: lucian@fb.com

FB internal diff: D806813

11 years agoAdd missing FileUtil functions (p?readvFull, p?writevFull) , add test
Tudor Bosman [Thu, 9 May 2013 21:16:35 +0000 (14:16 -0700)]
Add missing FileUtil functions (p?readvFull, p?writevFull) , add test

Summary:
Testing incomplete reads / writes is hard, so I'm only testing the wrappers
(template functions that take the underlying operation and retry it in
case of incomplete operations).  Note the cute hack of using variadic
templates to use the same wrapper for both file pointer- and explicit-position
flavors of the functions (the offset argument becomes optional).

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D806781

11 years agoMake folly::AsyncIO thread safe
Tudor Bosman [Wed, 8 May 2013 21:40:10 +0000 (14:40 -0700)]
Make folly::AsyncIO thread safe

Summary:
You can now submit to the same AsyncIO object from different threads, but you
must still reap from only one thread at a time.

Test Plan: async_io_test, added MT test

Reviewed By: philipp@fb.com

FB internal diff: D804914

11 years agoBreak dependency on endian.h
Owen Yamauchi [Fri, 3 May 2013 19:16:14 +0000 (12:16 -0700)]
Break dependency on endian.h

Summary:
gcc and clang both give you these three macros pre-defined. I've found a
reference on the interweb to this being broken
(http://gcc.gnu.org/ml/gcc/2011-08/msg00152.html) but that was with gcc
4.4. I've tested with 4.6, 4.7 and clang 3.2 and they all do the right
thing on little-endian (internal build and OS X). Alas, I don't have a
big-endian system handy to test with.

Test Plan: fbconfig/fbmake runtests.

Reviewed By: simpkins@fb.com

FB internal diff: D799416

11 years agoBreak dependency on byteswap.h
Owen Yamauchi [Fri, 3 May 2013 17:43:36 +0000 (10:43 -0700)]
Break dependency on byteswap.h

Summary:
Unfortunately, this doesn't exist on all platforms. Fortunately, both
gcc and clang support builtins that do the same thing as the bswap_*
functions we were including byteswap.h for.

I changed the test file to hardcode expected results for the swapping
functions. Is that OK? It seemed a little silly to be testing the
Endian::whatever functions by comparing them against the functions that
they're implemented in terms of.

Test Plan:
fbconfig/fbmake runtests with gcc 4.6 and 4.7. Verified
separately that clang supports these builtins.

Reviewed By: simpkins@fb.com

FB internal diff: D799244

11 years agoBreak dependency on features.h
Owen Yamauchi [Fri, 3 May 2013 16:25:39 +0000 (09:25 -0700)]
Break dependency on features.h

Summary:
It doesn't exist on some systems (at least Mac OS X). We're only using
it for __GNUC_PREREQ, which is easy to provide our own definition for.

I moved the definitions of FOLLY_FINAL and FOLLY_OVERRIDE into
folly-config.h so we can autoconf them in the open-source build. The
hardcoded stuff for the internal build is a little ugly, unfortunately.

folly can't be built with gcc versions earlier than 4.6, so that check
in ThreadLocal.h is pointless by now. (Plus we use noexcept without a
macro wrapper all over the place.) That stuff was also not
clang-friendly. clang has supported static_assert since 2.9 and noexcept
since... I'm not sure but at least 3.0.

Test Plan:
fbconfig/fbmake runtests, with gcc 4.6 and 4.7. clang can't
build folly right now, but I verified separately that it supports
noexcept and static_assert.

Reviewed By: simpkins@fb.com

FB internal diff: D799143

11 years ago(Folly) Remove unused variable according to -Wunused-variable
Hannes Roth [Fri, 3 May 2013 16:37:17 +0000 (09:37 -0700)]
(Folly) Remove unused variable according to -Wunused-variable

Summary:
I am trying to move more code into a HPHP extension, and it's complaining about
this.

Test Plan: `fbconfig -r folly && fbmake runtests`

Reviewed By: delong.j@fb.com

FB internal diff: D799096

11 years agoFail the configure script if you don't have boost
Owen Yamauchi [Tue, 30 Apr 2013 21:33:21 +0000 (14:33 -0700)]
Fail the configure script if you don't have boost

Summary:
Discovered this while trying to build folly on OS X. It would print a
warning about Boost being missing that got drowned out in the rest of
the configure output, and then the build would fail. This makes the
failure more friendly.

Test Plan:
`autoreconf; ./configure` on a system without boost
installed; make sure configure fails and the last thing it prints is the
boost error message.

Reviewed By: delong.j@fb.com

FB internal diff: D797389

11 years agoAdd read*String() methods to Cursor
Peter Griess [Wed, 1 May 2013 03:06:55 +0000 (20:06 -0700)]
Add read*String() methods to Cursor

Summary:
- Add some convenience methods for reading std::string objects via
folly::Cursor

Test Plan: - Unit tests

Reviewed By: simpkins@fb.com

FB internal diff: D795428

11 years agoFixed-size split()
Tom Jackson [Tue, 30 Apr 2013 20:13:34 +0000 (13:13 -0700)]
Fixed-size split()

Summary:
There are quite a few places where we split strings into a fixed number
of fields. This enables this to be done a bit faster by not using any
variable-length structures at runtime.

Test Plan: Unit tests, Benchmarks

Reviewed By: philipp@fb.com

FB internal diff: D794523

11 years agoRecordIO: robust record-based streaming I/O
Tudor Bosman [Thu, 25 Apr 2013 03:29:08 +0000 (20:29 -0700)]
RecordIO: robust record-based streaming I/O

Summary:
RecordIO provides an interface to write and read streams of variable-length
records that is resilient in the face of failure and data corruption.  If
the stream is corrupted in any way, you will lose records, but the stream
will resynchronize.

We have one implementation of RecordIO reader/writer that reads from / writes
to regular files (using mmap(), even for writes, so we can preserve state
even in case of process death -- the buffer cache will flush things to disk
eventually) and we expose enough of the guts (in a reasonably clean way)
so you can build your own on top of other backends.

Test Plan: test added

Reviewed By: mmcurtiss@fb.com

FB internal diff: D790275

11 years agoflock locks in folly::File, FileUtil, Exception.h fixes and tests
Tudor Bosman [Fri, 26 Apr 2013 22:32:36 +0000 (15:32 -0700)]
flock locks in folly::File, FileUtil, Exception.h fixes and tests

Summary:
Test required a separate process, as fcntl locks are always re-granted to
a process that holds the lock.

Test Plan: new tests, all tests under folly

Reviewed By: lucian@fb.com

FB internal diff: D791370

11 years agoFix build for folly_fb_platform
Jordan DeLong [Sun, 28 Apr 2013 16:27:03 +0000 (09:27 -0700)]
Fix build for folly_fb_platform

Summary:
This wasn't building with an error from a warning about
narrowing conversions---probably we didn't notice because it was
previously broken due to folly docs being broken forever.  Folly tests
should always pass.  (I want to add it to the fbcode-tests-must-pass
commit hook.)

Test Plan:
fbmake runtests_opt in this platform, except with stl_tests
turned off because it takes years to compile.

Reviewed By: andrewjcg@fb.com

FB internal diff: D792059

11 years agoMaking from(dynamic.items()) work
Tom Jackson [Tue, 30 Apr 2013 01:46:29 +0000 (18:46 -0700)]
Making from(dynamic.items()) work

Summary: It was broken because the `dynamic`'s iterator wasn't complete.

Test Plan: Unit test

Reviewed By: jbrewer@fb.com

FB internal diff: D793618

11 years agodistinctBy()
Tom Jackson [Fri, 26 Apr 2013 20:58:08 +0000 (13:58 -0700)]
distinctBy()

Test Plan: Unit tests

Reviewed By: jbrewer@fb.com

FB internal diff: D791149

11 years agoDisabling conversion with contained value for Optional
Tom Jackson [Mon, 22 Apr 2013 18:07:07 +0000 (11:07 -0700)]
Disabling conversion with contained value for Optional

Summary: Comparisons with values can lead to confusion, especially when the value itself is truthy. To avoid this confusion, I'm disabling comparison with contained value. Note that Optionals can still be constructed and assigned these values, but comparsion must be done separately for the container and the contained.

Test Plan: Unit tests, contbuild

Reviewed By: tudorb@fb.com

FB internal diff: D783621

11 years agoFix async_io_test to work with larger block sizes
Tudor Bosman [Wed, 24 Apr 2013 21:59:06 +0000 (14:59 -0700)]
Fix async_io_test to work with larger block sizes

Test Plan: ran it

Reviewed By: lucian@fb.com

FB internal diff: D787733

11 years agomove rateHelper() to detail/Stats.h
Adam Simpkins [Wed, 17 Apr 2013 03:41:29 +0000 (20:41 -0700)]
move rateHelper() to detail/Stats.h

Summary:
The BucketedTimeSeries::rateHelper() function is generic, and not
specific to the BucketedTimeSeries class.  This diff moves it to
folly/detail/Stats.h, so other parts of the stats code can access it as
well.

Test Plan: Ran the folly unit tests.

Reviewed By: delong.j@fb.com

FB internal diff: D778115

11 years agoMake BucketedTimeSeries::rate() more accurate
Adam Simpkins [Wed, 17 Apr 2013 01:32:23 +0000 (18:32 -0700)]
Make BucketedTimeSeries::rate() more accurate

Summary:
Make rate() and countRate() more accurate when queried for a specific
time range.

Previously these functions divided the estimated sum/count by the entire
time range specified.  This underestimated the rate if we don't actually
have data for the entire time period.  (Since the sum computed only
takes into account the time range for which we have data.)

For example, if the timeseries duration was 60 seconds, but only 30
seconds of data had been entered so far, rate(now - 60, now) would
underestimate the rate by half, since there was only 30 seconds worth of
data available.  The no-argument version of rate() did work correctly in
that case.

Test Plan:
Added a new unit test for this behavior.

Also fixed the existing rate test code, which had the same bug and
expected the underestimated rate.

Reviewed By: delong.j@fb.com

FB internal diff: D778114

11 years agofix fbstring move assignment operator
Philip Pronin [Tue, 23 Apr 2013 06:36:47 +0000 (23:36 -0700)]
fix fbstring move assignment operator

Summary:
21.4.2 [string.cons] / 23 says

> If *this and str are the same object, the member has no effect.

That means we have to support self-move-assignment.

Test Plan: added test which triggered assertion, ran it

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D785057

11 years agotoAppendDelimited, toDelimited
Tudor Bosman [Sat, 20 Apr 2013 20:07:39 +0000 (13:07 -0700)]
toAppendDelimited, toDelimited

Test Plan: test added

Reviewed By: delong.j@fb.com

FB internal diff: D783100

11 years agoFOLLY_NORETURN
Tudor Bosman [Sat, 20 Apr 2013 20:17:18 +0000 (13:17 -0700)]
FOLLY_NORETURN

Summary: Seems like a good idea.

Test Plan: compiled all of folly

Reviewed By: delong.j@fb.com

FB internal diff: D783104

11 years agoModernize TemporaryFile, add TemporaryDirectory
Tudor Bosman [Tue, 16 Apr 2013 20:22:41 +0000 (13:22 -0700)]
Modernize TemporaryFile, add TemporaryDirectory

Test Plan: fbconfig $(find folly -name test) && fbmake runtests_opt

Reviewed By: tjackson@fb.com

FB internal diff: D777186

11 years agoRevert "Revert "URI parsing in folly""
Tudor Bosman [Wed, 17 Apr 2013 18:57:05 +0000 (11:57 -0700)]
Revert "Revert "URI parsing in folly""

Summary:
Now that the third_party link was updated in
https://phabricator.fb.com/D778617, we're good.

Test Plan: fbconfig -r thrift && fbmake runtests_opt

Reviewed By: chip@fb.com

FB internal diff: D778707

11 years agoRevert "URI parsing in folly"
Tudor Bosman [Wed, 17 Apr 2013 18:47:25 +0000 (11:47 -0700)]
Revert "URI parsing in folly"

Summary: Unbreak build.  third-party link not yet updated after https://phabricator.fb.com/D776457

Test Plan: no

Reviewed By: chip@fb.com

FB internal diff: D778669

11 years agoURI parsing in folly
Tudor Bosman [Wed, 10 Apr 2013 00:34:52 +0000 (17:34 -0700)]
URI parsing in folly

Summary: Cleaned up from common/strings/URL.h, and it should be URI, not URL.

Test Plan: tests added

Reviewed By: chip@fb.com

FB internal diff: D768880

11 years agoFix bug in reserve() and shrink_to_fit().
Christian Kamm [Mon, 25 Mar 2013 08:54:26 +0000 (09:54 +0100)]
Fix bug in reserve() and shrink_to_fit().

Summary:
impl_.e_ += newB - impl_.b_; fails when the difference
between newB and impl_.b_ isn't a multiple of sizeof(T).

Test Plan: .

Reviewed By: oyamauchi@fb.com

FB internal diff: D774754

11 years agoMake Subprocess::spawn more robust
Tudor Bosman [Sat, 13 Apr 2013 06:49:32 +0000 (23:49 -0700)]
Make Subprocess::spawn more robust

Summary:
We can't throw after the process is created, because we don't know what to do
with it (and the Subprocess object goes up in smoke, so we can't rely on the
caller to clean up, either).  So don't throw.

If we throw before the process is created, make sure we clean up.

Test Plan: subprocess_test

Reviewed By: delong.j@fb.com

FB internal diff: D774722

11 years agodo not include iostream from Range.h
Philip Pronin [Sun, 14 Apr 2013 02:06:37 +0000 (19:06 -0700)]
do not include iostream from Range.h

Summary:
folly/Range.h is extensively used in fbcode, try to avoid including
<iostream> (which is pretty heavy), include forward declarations
(<iosfwd>) instead.

Also transitioned it from 'std type_traits' + 'boost type_traits' to
'std type_traits'.

Test Plan: compiled, ran tests

Reviewed By: soren@fb.com

FB internal diff: D774834

11 years agomove Histogram.h to stats/
Adam Simpkins [Sat, 30 Mar 2013 01:55:16 +0000 (18:55 -0700)]
move Histogram.h to stats/

Summary:
Move Histogram.h into the stats/ subdirectory, along side the
BucketedTimeSeries code.  Eventually I plan to land more of our stats
code in this subdirectory too.

This also renames Histogram-inl.h to Histogram-defs.h, and no longer
includes it by default from Histogram.h.  Instead, this adds a new
stats/Instantiations.cpp file which explicitly instantiates
Histogram<int64_t> and BucketedTimeSeries<int64_t>.  Most callers use
these instantiations, and they now no longer need to include
Histogram-defs.h.  Only callers that need other instantiations need to
include Histogram-defs.h.  This will speed up build times.

Test Plan:
Ran "arc unit" to build all projects depending on folly, and verified
they all still built and passed tests.

Reviewed By: ldbrandy@fb.com

FB internal diff: D761377

11 years agoeasier rebinding of allocators
Marcelo Juchem [Mon, 8 Apr 2013 20:38:22 +0000 (13:38 -0700)]
easier rebinding of allocators

Summary: rebinding allocators is too cumbersome, this diff implements a helper to make this job easier.

Test Plan: unit test added

Reviewed By: tudorb@fb.com

FB internal diff: D766451

11 years agoMaking StlAllocator<Alloc, void> usable when rebinding.
Marcelo Juchem [Mon, 8 Apr 2013 21:08:18 +0000 (14:08 -0700)]
Making StlAllocator<Alloc, void> usable when rebinding.

Summary:
currently, StlAllocator<Alloc, void> can't be used when you
want an untyped allocator that can be rebound later since it doesn't
carry the SimpleAllocator pointer with it. This diff fixes that.

Test Plan: unit test added

Reviewed By: jon.coens@fb.com

FB internal diff: D766559

11 years agoInitialize variable before using in own initialization
Michael Connor [Wed, 27 Mar 2013 17:16:59 +0000 (10:16 -0700)]
Initialize variable before using in own initialization

Summary:
Clang throws error because the callback refers to itself inside its lambda
function definition.  I prevent this by declaring the variable first then the
compiler does not complain when it is used within its lambda definition.

folly/test/TimeoutQueueTest.cpp:99:37: error: variable 'cb' is uninitialized
when used within its own initialization [-Werror,-Wuninitialized]

Test Plan:
fbconfig --clang --platform=gcc-4.7.1-glibc-2.14.1 --with-project-version
boost:1.51.0 folly/test/
fbmake opt
fbmake runtests_opt

Reviewed By: ldbrandy@fb.com

FB internal diff: D753061

11 years agoSoft-limit for arenas
Jonathan Coens [Thu, 4 Apr 2013 22:25:52 +0000 (15:25 -0700)]
Soft-limit for arenas

Summary: Create an artificial limit on an arena to start throwing bad_alloc before running out of system memory

Test Plan: adjust unit test

Reviewed By: marcelo.juchem@fb.com

FB internal diff: D762695

11 years agoFixing clang compatibility issues
Marcelo Juchem [Sat, 30 Mar 2013 07:13:31 +0000 (00:13 -0700)]
Fixing clang compatibility issues

Test Plan: all folly unit tests

Reviewed By: andrewjcg@fb.com

FB internal diff: D757374

11 years agoImplementing a traits class to check for incomplete types
Marcelo Juchem [Wed, 3 Apr 2013 03:22:58 +0000 (20:22 -0700)]
Implementing a traits class to check for incomplete types

Summary: A traits class to check for incomplete types

Test Plan: unit tests added

Reviewed By: delong.j@fb.com

FB internal diff: D760676

11 years agofolly: speed up fastpath of ThreadLocal::get()
Lucian Grijincu [Wed, 3 Apr 2013 04:29:02 +0000 (21:29 -0700)]
folly: speed up fastpath of ThreadLocal::get()

Summary:
A smaller function makes it more likely it will be inlined
(it wasn't before, is now).

Test Plan: n/a

Reviewed By: tudorb@fb.com

FB internal diff: D759996

11 years agofolly: speed up fastpath of StaticMeta::get()
Lucian Grijincu [Wed, 3 Apr 2013 00:56:32 +0000 (17:56 -0700)]
folly: speed up fastpath of StaticMeta::get()

Summary:
A smaller function makes it more likely it will be inlined
(it wasn't before, is now).

Test Plan: n/a

Reviewed By: tudorb@fb.com

FB internal diff: D760000

11 years agoas_stl_allocator "template alias"
Marcelo Juchem [Mon, 1 Apr 2013 23:00:01 +0000 (16:00 -0700)]
as_stl_allocator "template alias"

Summary: Implementing as_stl_allocator as a companion to the existing make_stl_allocator

Test Plan: unit tests added

Reviewed By: jon.coens@fb.com

FB internal diff: D755207

11 years agoSuppress unused variable warning
Misha Shneerson [Tue, 2 Apr 2013 19:14:41 +0000 (12:14 -0700)]
Suppress unused variable warning

Summary:
HPHP compiler treats warnings as errors and the unused 'constuctor'
variable breaks the build.

Test Plan: Build

Reviewed By: simpkins@fb.com

FB internal diff: D759559

11 years agoOptionally, kill subprocess when parent dies
Tudor Bosman [Thu, 28 Mar 2013 23:48:51 +0000 (16:48 -0700)]
Optionally, kill subprocess when parent dies

Summary: Non-portable.

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D755528

11 years agoremove folly/eventfd.h
Tudor Bosman [Wed, 27 Mar 2013 21:04:01 +0000 (14:04 -0700)]
remove folly/eventfd.h

Test Plan: compile: fbconfig folly/experimental/io/test thrift/test unicorn/io/test common/system_features common/concurrency && fbmake opt

Reviewed By: soren@fb.com

FB internal diff: D753552

11 years agoRemove an unused variable in FormatValue<double>
Brett Simmers [Wed, 27 Mar 2013 20:14:37 +0000 (13:14 -0700)]
Remove an unused variable in FormatValue<double>

Test Plan: Built a program that uses it

Reviewed By: tudorb@fb.com

FB internal diff: D753460

11 years agomake sse4.2 functions in folly/Range.h build on -fb platform
Tudor Bosman [Mon, 25 Mar 2013 18:37:39 +0000 (11:37 -0700)]
make sse4.2 functions in folly/Range.h build on -fb platform

Test Plan: fbconfig --platform=gcc-4.7.1-glibc-2.14.1-fb folly/test && fbmake runtests_opt

Reviewed By: oyamauchi@fb.com

FB internal diff: D749992

11 years agofix clang warnings in folly::Gen
Louis Brandy [Mon, 25 Mar 2013 17:16:15 +0000 (10:16 -0700)]
fix clang warnings in folly::Gen

Summary: Two clang warnings. It wants user defined consturctor for static initialization, and it wants fwd-declare to agree with full decleration in regards to struct v class.

Test Plan:
`fbconfig --clang --platform=gcc-4.7.1-glibc-2.14.1 folly/test/`

And make sure it doesn't break normal build.

Reviewed By: tulloch@fb.com

FB internal diff: D749857

11 years agoAdd read/write mode to ElfFile
Peter Griess [Mon, 25 Mar 2013 16:31:49 +0000 (09:31 -0700)]
Add read/write mode to ElfFile

Summary:
- Add a mode to ElfFile that allows opening the file for read/write
access via PROT_WRITE.

Test Plan:
- Used it in some other code

Reviewed By: simpkins@fb.com

FB internal diff: D740184

11 years agoHandle non-Intel platforms in Range and CpuId
Owen Yamauchi [Thu, 21 Mar 2013 14:32:47 +0000 (07:32 -0700)]
Handle non-Intel platforms in Range and CpuId

Summary:
Compile out the SSE versions of these functions in Range, based on a new
entry in folly-config.h.

The change to CpuId feels slightly iffy to me. It seems like it would be
more rigorous to make compiling CpuId.h on non-Intel an error, and force
clients to handle non-Intel platforms at the callsite. However, I think
that would be too susceptible to unintentional breakage on non-Intel
platforms, since most people (including automated systems) aren't
building and testing regularly on any. Falling back to saying "none of
these features exist on this processor" seems like a reasonable
alternative.

Test Plan:
fbmake runtests, with FOLLY_HAVE_EMMINTRIN_H set to 0 and 1.
Make sure the SSE functions are getting compiled in or out as
appropriate. ##autoreconf## and ##./configure## to regenerate
folly-config.h.

Reviewed By: delong.j@fb.com

FB internal diff: D746872

11 years agoAdd symbol name resolution and value retrieval
Peter Griess [Fri, 15 Mar 2013 16:12:46 +0000 (09:12 -0700)]
Add symbol name resolution and value retrieval

Summary:
- Add ElfFile::getSymbolByName(), which finds a Symbol object
corresponding to the symbol w/ the given name
- Add ElfFile::getSymbolValue(), which resolves the Symbol object to a
value in the mapped file, following pointers if necessary

Test Plan: - Unit tests

Reviewed By: simpkins@fb.com

FB internal diff: D740183

11 years agoEnable Elf::at<T> only for POD data types
Peter Griess [Fri, 22 Mar 2013 15:12:30 +0000 (08:12 -0700)]
Enable Elf::at<T> only for POD data types

Summary:
- Make sure that the utility function Elf::at<T> only works for POD
datatypes, as it uses reinterpret_cast<T>.

Test Plan: - Unit tests

Reviewed By: simpkins@fb.com

FB internal diff: D748314

11 years agoMove folly::symbolizer::systemError() into Exception.h
Peter Griess [Fri, 22 Mar 2013 14:55:54 +0000 (07:55 -0700)]
Move folly::symbolizer::systemError() into Exception.h

Summary:
- This is pretty similar to some stuff that we already have in
Exception.h. Move (and rename) it.

Test Plan: - Unit tests

Reviewed By: simpkins@fb.com

FB internal diff: D748313

11 years agoCompile out GroupVarint on non-Intel
Owen Yamauchi [Thu, 21 Mar 2013 19:13:39 +0000 (12:13 -0700)]
Compile out GroupVarint on non-Intel

Summary:
Compile out instead of erroring.

In an ideal world, we'd have a fallback that would work across platforms
(i.e. no SSE, no unaligned 32-bit writes etc.) and compile some version
of GroupVarint in all environments. I actually tried this; the SSE stuff
is all behind #if __SSSE3__ already, so I thought it could work (modulo
the unaligned-writes problem). I ran into problems with the
SSSE3-vs.-not distinction that @simpkins alluded to in D652764, and
decided I'd rather not open that can of worms at the moment.

Test Plan:
fbmake runtests. Manually force the #ifs to false and make
sure fbmake runtests still passes (although GroupVarintTest is empty).

Reviewed By: delong.j@fb.com

FB internal diff: D747150

11 years agoStlAllocator.h + MakeUnique.h -> Memory.h
Louis Brandy [Wed, 20 Mar 2013 21:54:45 +0000 (14:54 -0700)]
StlAllocator.h + MakeUnique.h -> Memory.h

Summary:
Go with the fat header approach. Merge these two into Memory.h. We could, potentially, include Malloc.h as well, but it fbstring header uses the once define for some special magic. Leave it alone for now.

An alternate approach might be moving all three leaner headers into a `memory/` subdir with `folly/Memory.h` just #including the three.

Test Plan:
fbconfig folly/tests && fbmake runtests_opt

Reviewed By: delong.j@fb.com

FB internal diff: D745873

11 years agoHACK: Static detection of infinite sequences
Mike Curtiss [Wed, 6 Mar 2013 07:22:54 +0000 (23:22 -0800)]
HACK: Static detection of infinite sequences

Summary:
Certain operations should not be performed on infinite sequences
(e.g. sorting, left-folds, summation).  In some cases, we can
detect that a sequence is infinite at compile-time and provide
a static_assert to prevent such dangerous operations.

Test Plan:
Manually created cases where the operation should
be disallowed.  Compiler correctly raised an error.

Reviewed By: tjackson@fb.com

FB internal diff: D740011

11 years agoCopyright 2012 -> 2013
Louis Brandy [Wed, 20 Mar 2013 22:09:03 +0000 (15:09 -0700)]
Copyright 2012 -> 2013

Summary: See title.

Test Plan: Inspection.

Reviewed By: delong.j@fb.com

FB internal diff: D745883

11 years agoIOBuf::getIov
Hans Fugal [Thu, 14 Mar 2013 00:32:00 +0000 (17:32 -0700)]
IOBuf::getIov

Summary:
Generate an `fbvector` of `struct iovec` suitable for using with `writev` or
`sendmsg`.

This code is pretty straightforward, but Adam pointed out that something along
these lines has already been done in thrift, so I followed that code closely.
http://fburl.com/11586814

Test Plan:
fbmake runtests

I am using also this in a prototype and it's working there.

Reviewed By: agartrell@fb.com

FB internal diff: D744055

11 years agoallow to dequeue the first IOBuf in an IOBufQueue
Alessandro Salvatori [Sat, 9 Mar 2013 02:16:59 +0000 (18:16 -0800)]
allow to dequeue the first IOBuf in an IOBufQueue

Summary: allow to dequeue the first IOBuf in an IOBufQueue

Test Plan: throughly tested with some dependent code in proxygen

Reviewed By: tudorb@fb.com

FB internal diff: D732484

11 years agoUse aligned loads for Range::find_first_of
Mike Curtiss [Fri, 22 Feb 2013 22:28:05 +0000 (14:28 -0800)]
Use aligned loads for Range::find_first_of

Summary:
Aligned loads are faster on some architectures. Let's refactor
qfind_first_of so that it uses aligned loads when possible.

Also modify the benchmarks to account for begin/end-of-string
logic.

Test Plan:
Tests pass. Updated benchmarks.  Generally seeing a 5-20%
speed-up, depending on the situation.

Reviewed By: philipp@fb.com

FB internal diff: D720369

11 years agoShort-circuit operator== based on size()
Tom Jackson [Wed, 13 Mar 2013 22:44:59 +0000 (15:44 -0700)]
Short-circuit operator== based on size()

Summary:
We don't do this today, but it looks like std::string does. For longer, similar strings, this is a big win.

Before:

```lang=text
============================================================================
./folly/test/FBStringTestBenchmarks.cpp.h       relative  time/iter  iters/s
============================================================================
BM_equality_string(65536)                                    5.13ms   194.87
BM_equality_fbstring(65536)                                 11.34ms    88.18
============================================================================
```

After:

```lang=text
============================================================================
./folly/test/FBStringTestBenchmarks.cpp.h       relative  time/iter  iters/s
============================================================================
BM_equality_string(65536)                                    5.01ms   199.74
BM_equality_fbstring(65536)                                  6.63ms   150.78
============================================================================
```

Test Plan: Benchmark, unit  tests

Reviewed By: tudorb@fb.com

FB internal diff: D737482

11 years agoFix unsplit(", ")
Tom Jackson [Wed, 13 Mar 2013 20:56:29 +0000 (13:56 -0700)]
Fix unsplit(", ")

Summary: Otherwise you get errors like `error: array used as initializer ./folly/experimental/StringGen-inl.h: In constructor ‘folly::gen::detail::UnsplitBuffer<Delimiter, OutputBuffer>::UnsplitBuffer(const Delimiter&, OutputBuffer*) [with Delimiter = char [3] ...]`, since literal strings bind as reference to fixed-length character arrays. Providing an explicit overload for `const char*` fixes this.

Test Plan: Unit tests

Reviewed By: tulloch@fb.com

FB internal diff: D737117

11 years agoadding is_non_positive traits
Marcelo Juchem [Tue, 12 Mar 2013 22:55:06 +0000 (15:55 -0700)]
adding is_non_positive traits

Summary:
template <typename SomeInt>
void foo(SomeInt x) {
// yields an error in clang when SomeInt is unsigned and -Werror is used
if(x <= 0) {
//...
}
}

Test Plan: added unit tests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D735735

11 years agoMake hash_combine accept a configurable hash function
David Vickrey [Mon, 11 Mar 2013 22:34:18 +0000 (15:34 -0700)]
Make hash_combine accept a configurable hash function

Summary:
std::hash is not awesome and not configurable.  Typical cases you might want to customize are:
string: I happen to know that fnv isn't super awesome, for example, and that's what folly uses for std::hash fbstring.
pointers: you may want to hash the contents of the pointer instead of the address for certain types.

This is a very simple diff that lets you do that.  It provides StdHasher that passes through to std::hash and uses that for hash_combine, so this should be 100% backward compatible.

Test Plan: test_hash.  I will add another test for using a hasher besides StdHasher shortly.

Reviewed By: delong.j@fb.com

FB internal diff: D733899

11 years agoFixing namespace for GeneratorBuilder, more moves for Params
Tom Jackson [Fri, 8 Mar 2013 03:26:51 +0000 (19:26 -0800)]
Fixing namespace for GeneratorBuilder, more moves for Params

Summary:
GENERATOR was broken if you didn't `using namespace folly::gen`, and
we're still copying quite a few functors where we could move them.

Test Plan: Unit tests

Reviewed By: chaoyc@fb.com

FB internal diff: D731253

11 years agoAdd explicit assignment operator definitions to Optional
Lovro Puzar [Mon, 11 Mar 2013 20:52:42 +0000 (13:52 -0700)]
Add explicit assignment operator definitions to Optional

Summary:
See new test.  Under GCC 4.6 (which is what the folly tests build with) the old code works fine but under 4.7 building fails with:

folly/test/OptionalTest.cpp: In member function ‘virtual void Optional_AssignmentContained_Test::TestBody()’:
folly/test/OptionalTest.cpp:122:14: error: use of deleted function ‘ContainsOptional& ContainsOptional::operator=(const ContainsOptional&)’
folly/test/OptionalTest.cpp:106:21: note: ‘ContainsOptional& ContainsOptional::operator=(const ContainsOptional&)’ is implicitly deleted because the default definition would be ill-formed:
folly/test/OptionalTest.cpp:106:21: error: use of deleted function ‘folly::Optional<int>& folly::Optional<int>::operator=(const folly::Optional<int>&)’
In file included from folly/test/OptionalTest.cpp:17:0:
./folly/Optional.h:84:7: note: ‘folly::Optional<int>& folly::Optional<int>::operator=(const folly::Optional<int>&)’ is implicitly declared as deleted because ‘folly::Optional<int>’ declares a move constructor or move assignment operator
folly/test/OptionalTest.cpp:129:30: error: use of deleted function ‘ContainsOptional& ContainsOptional::operator=(ContainsOptional&&)’
folly/test/OptionalTest.cpp:108:21: note: ‘ContainsOptional& ContainsOptional::operator=(ContainsOptional&&)’ is implicitly deleted because the default definition would be ill-formed:
folly/test/OptionalTest.cpp:108:21: error: non-static data member ‘ContainsOptional::opt_’ does not have a move assignment operator or trivial copy assignment operator
folly/test/OptionalTest.cpp:137:14: error: use of deleted function ‘ContainsOptional& ContainsOptional::operator=(const ContainsOptional&)’

Test Plan:
(1) fbconfig folly/test && fbmake dbg && _build/dbg/folly/test/optional_test
(2) Remove folly/PLATFORM to build with gcc 4.7, then repeat (1).  Without the code fix, the new test fails to build.  With the fix, the test builds and runs fine.

Reviewed By: tjackson@fb.com

FB internal diff: D732402

11 years agofolly (easy): Disable GCC-specific warning disabling hacks in clang
Ben Gertzfield [Mon, 11 Mar 2013 18:27:34 +0000 (11:27 -0700)]
folly (easy): Disable GCC-specific warning disabling hacks in clang

Summary:
When compiling folly with clang, the compiler warns about our
use of GCC-specific pragmas to silence incorrect compiler warnings:

folly/Optional.h:79:33: warning: unknown warning group '-Wpragmas', ignored [-Wunknown-pragmas]
folly/Optional.h:80:33: warning: unknown warning group '-Wmaybe-uninitialized', ignored [-Wunknown-pragmas]

Thankfully, those incorrect compiler warnings are not emitted by
clang, so we can just disable the pragmas in clang.

Test Plan:
Built folly in gcc and ran it through clang. Warning above
is gone.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D733323

11 years agoAdd CHECK for out-of-range minRequests
Tudor Bosman [Thu, 7 Mar 2013 15:45:16 +0000 (07:45 -0800)]
Add CHECK for out-of-range minRequests

Test Plan: async_io_test

Reviewed By: philipp@fb.com

FB internal diff: D730100

11 years agoRemove File::tryOpen
Tom Jackson [Wed, 6 Mar 2013 23:27:38 +0000 (15:27 -0800)]
Remove File::tryOpen

Summary:
In hopes of keeping 'busywork' helpers out of folly, I've moved this
closer to the code that needed to do this.

Test Plan: Unit tests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D729194

Blame Revision: D726916

11 years agoFile::tryOpen
Tom Jackson [Tue, 5 Mar 2013 19:05:28 +0000 (11:05 -0800)]
File::tryOpen

Summary:
Now that we have truthy files, it would be nice to be able to simply
//try// opening files and return a possibly-initialized File.

Test Plan: Unit tests

Reviewed By: chaoyc@fb.com

FB internal diff: D726916

11 years agoTruthy File
Tom Jackson [Tue, 5 Mar 2013 19:05:42 +0000 (11:05 -0800)]
Truthy File

Summary:
File has a default constructor so it can be initialized late, but it
doesn't have a good canonical way to see if it's been initialized. This adds an
//explicit// operator bool so you can test files like `if (file) ...`.

Test Plan: Unit tests

Reviewed By: chaoyc@fb.com

FB internal diff: D726914

11 years agoAdd resizing constructor to folly::padded::Adaptor
Tudor Bosman [Tue, 5 Mar 2013 00:31:58 +0000 (16:31 -0800)]
Add resizing constructor to folly::padded::Adaptor

Summary: Added Adaptor(size_t, const value_type&)

Test Plan: test added

Reviewed By: soren@fb.com

FB internal diff: D726358

11 years agoAdding useful error message for File
Tom Jackson [Mon, 4 Mar 2013 19:52:58 +0000 (11:52 -0800)]
Adding useful error message for File

Summary:
'open() failed' isn't too helpful. Seeing a filename there is. So I
added it.

Test Plan: Unit test

Reviewed By: mohittalwar@fb.com

FB internal diff: D725204

11 years agoprovide a flag to control the minimum number of iterations
Adam Simpkins [Fri, 1 Mar 2013 05:33:29 +0000 (21:33 -0800)]
provide a flag to control the minimum number of iterations

Summary:
Add a --bm_min_iters flag to control the minimum number of iterations
that the benchmark code starts with on each epoch.

This can be used on benchmarks that test very cheap operations, but take
a long time to set up.  Otherwise the benchmark code may have to retry
many times before it hits a large enough number of iterations to get a
meaningful result, and each time it still pays the fixed setup cost.

This also helps with benchmarks when some of the setup cost cannot be
hidden with BenchmarkSuspender for some reason.  --bm_min_iters can be
set to a large enough value so that the extra startup cost will not
affect the measurements too much.

Test Plan:
Used this with the thread local stats benchmark.  During setup/cleanup,
this benchmark starts and synchronizes with many threads.  The entire
setup time cannot be reliably hidden with BenchmarkSuspender; the
synchronization between the threads takes a relatively long time
compared to the cost of the operation being benchmarked.  --bm_min_iters
allows a relatively high number of iterations to be used, masking this
cost.

Reviewed By: rajat@fb.com

FB internal diff: D723304

11 years agofolly::make_optional
Mike Curtiss [Thu, 28 Feb 2013 03:53:14 +0000 (19:53 -0800)]
folly::make_optional

Summary:
Helper method for creating a folly::Optional<T> by just passing
in a T reference.  Analogous to boost::make_optional.

Test Plan: Added test case

Reviewed By: tjackson@fb.com

FB internal diff: D721762

11 years agoMemoryMapping::data() returns StringPiece ::range() returns ByteRange
Lucian Grijincu [Wed, 27 Feb 2013 23:44:37 +0000 (15:44 -0800)]
MemoryMapping::data() returns StringPiece ::range() returns ByteRange

Summary: MemoryMapping::data() returns StringPiece ::range() returns ByteRange

Test Plan: tests

Reviewed By: philipp@fb.com

FB internal diff: D720985

11 years agoAbstract ifunc support into a define
Owen Yamauchi [Tue, 19 Feb 2013 20:20:43 +0000 (12:20 -0800)]
Abstract ifunc support into a define

Summary:
There are platforms other than clang that don't support ifuncs. (The one
I'm concerned about is ARM.) I changed the ifdef __clang__ around the
ifunc attributes to be more abstract, so we can can pass in this flag on
the command line, or use autoconf to detect it.

Test Plan:
fbmake runtests. Manually define HAVE_IFUNC 0 and make sure the
popcount() and popcountll() functions get compiled as calls to
popcount_builtin.

Run autoreconf, ./configure, make sure the feature gets detected
properly by looking at config.h.

Reviewed By: andrewjcg@fb.com

FB internal diff: D712192

11 years agoRenaming flag in MemoryMapping
Tom Jackson [Sat, 23 Feb 2013 03:07:11 +0000 (19:07 -0800)]
Renaming flag in MemoryMapping

Summary:
This flag conflicts with the flag with the same purpose in
`common/files/MemoryMappedFile.cpp`. Just renaming it for now.

Test Plan: Build

Reviewed By: lucian@fb.com

FB internal diff: D717067

11 years agoMemoryMapping
Tom Jackson [Thu, 21 Feb 2013 00:13:00 +0000 (16:13 -0800)]
MemoryMapping

Summary: MemoryMapping is a C++ wrapper object around mmap. It works with `folly::File`s, and provides bitwise-range access for reading and writing files.

Test Plan: Unit test

Reviewed By: lucian@fb.com

FB internal diff: D452384

11 years agofolly: AsyncIO: add debuging helper
Lucian Grijincu [Fri, 22 Feb 2013 18:36:01 +0000 (10:36 -0800)]
folly: AsyncIO: add debuging helper

Summary: Change State to enum class, and add debugging helper to AsyncIOOp and AsyncIO.

Test Plan: n/a

Reviewed By: philipp@fb.com

FB internal diff: D715676

11 years agofolly/{experimental => .}/File
Tom Jackson [Thu, 21 Feb 2013 00:50:50 +0000 (16:50 -0800)]
folly/{experimental => .}/File

Summary: Moving File into `/folly`.

Test Plan: Same unit tests, rebuild affected code outside folly.

Reviewed By: philipp@fb.com

FB internal diff: D714462

11 years agoFix SIGSEGV in StringPiece::find_first_of
Mike Curtiss [Tue, 12 Feb 2013 22:39:13 +0000 (14:39 -0800)]
Fix SIGSEGV in StringPiece::find_first_of

Summary:
Our SSE version of find_first_of was reading past the end of
the StringPiece in some cases, which (very rarely) caused a seg-fault
when we were reading outside of our allotted virtual address space.

Modify the code to never read past the end of the underlying buffers
except when we think it's "safe" because we're still within the same
page. (ASSUMPTION: if a process is allowed to read a byte within a
page, then it is allowed to read _all_ bytes within that page.)

Test Plan:
Added tests that verify we won't go across page boundaries.

Sadly, this code hurts our benchmarks -- sometimes by up to 50% for
smaller strings.

Reviewed By: philipp@fb.com

FB internal diff: D707923

Blame Revision: D638500

11 years agoRework folly::AsyncIO interface to make it easier for other classes to use Op
Tudor Bosman [Thu, 14 Feb 2013 23:26:26 +0000 (15:26 -0800)]
Rework folly::AsyncIO interface to make it easier for other classes to use Op

Summary:
AsyncIOOp no longer requires derivation to be able to use callbacks; the
callback is passed in.  This makes composition easier (see AsyncIOQueue, added
in this diff).

Test Plan: async_io_test, test added

Reviewed By: lucian@fb.com

FB internal diff: D709648

11 years agofix comment
Tudor Bosman [Fri, 15 Feb 2013 00:48:32 +0000 (16:48 -0800)]
fix comment

Test Plan: No

Reviewed By: philipp@fb.com

FB internal diff: D709795

11 years agovalue_ might be uninitialized
Rajat Goel [Wed, 13 Feb 2013 20:38:02 +0000 (12:38 -0800)]
value_ might be uninitialized

Summary: Compilation is failing in 'opt' build

Test Plan: compile

Reviewed By: andrewjcg@fb.com

FB internal diff: D707869

11 years agoDefine ALLOCM_LG_ALIGN
Tudor Bosman [Wed, 13 Feb 2013 20:04:41 +0000 (12:04 -0800)]
Define ALLOCM_LG_ALIGN

Test Plan: No

Reviewed By: philipp@fb.com

FB internal diff: D707792

11 years agoMinor clang compiler fixes
Gaurav Jain [Wed, 13 Feb 2013 00:59:25 +0000 (16:59 -0800)]
Minor clang compiler fixes

Summary: Minor clang compiler fixes

Test Plan: - fbmake runtest

Reviewed By: andrewjcg@fb.com

FB internal diff: D707663

11 years agoMake AsyncIO::capacity_ const
Sergey Doroshenko [Tue, 12 Feb 2013 17:17:53 +0000 (09:17 -0800)]
Make AsyncIO::capacity_ const

Summary: It is set once, and is never changed afterwards.

Test Plan: compiled, unit tests

Reviewed By: tudorb@fb.com

FB internal diff: D706125

11 years agoCodemod time\(NULL\) to time(nullptr)
Jordan DeLong [Thu, 31 Jan 2013 00:09:43 +0000 (16:09 -0800)]
Codemod time\(NULL\) to time(nullptr)

Summary: codemod with 'Yes to all'.

Test Plan: None really.

Reviewed By: abirchall@fb.com

FB internal diff: D693769

11 years agoFix subtle double-free in trimEnd; optionally pack IOBufQueue after append
Tudor Bosman [Fri, 8 Feb 2013 01:56:55 +0000 (17:56 -0800)]
Fix subtle double-free in trimEnd; optionally pack IOBufQueue after append

Test Plan: folly/io/test, both dbg and opt, standalone and in valgrind

Reviewed By: philipp@fb.com

FB internal diff: D702755

11 years agoall(), better any()
Tom Jackson [Thu, 7 Feb 2013 18:18:15 +0000 (10:18 -0800)]
all(), better any()

Summary: TSIA

Test Plan: Unit tests

Reviewed By: tulloch@fb.com

FB internal diff: D701890

11 years agofix typos
Tudor Bosman [Tue, 5 Feb 2013 21:12:26 +0000 (13:12 -0800)]
fix typos

Test Plan: async_io_test

Reviewed By: lucian@fb.com

FB internal diff: D699175

11 years agoAdd some CursorBase::operator-() implementations
Peter Griess [Mon, 28 Jan 2013 18:05:16 +0000 (10:05 -0800)]
Add some CursorBase::operator-() implementations

Summary:
- Add CursorBase::operator-() implementations for Cursor and BufType;
useful for figuring out the distance between two objects

Test Plan: - Used in some other code

Reviewed By: simpkins@fb.com

FB internal diff: D690046

11 years agoassert on self move assignment
Philip Pronin [Sat, 2 Feb 2013 06:41:53 +0000 (22:41 -0800)]
assert on self move assignment

Test Plan: ran tests in dbg

Reviewed By: tudorb@fb.com

FB internal diff: D696964

11 years agoAsyncIO: CHECK preconditions
Tudor Bosman [Tue, 5 Feb 2013 18:51:48 +0000 (10:51 -0800)]
AsyncIO: CHECK preconditions

Test Plan: async_io_test

Reviewed By: delong.j@fb.com

FB internal diff: D698919

11 years agoAsyncIO in folly
Tudor Bosman [Tue, 5 Feb 2013 02:38:14 +0000 (18:38 -0800)]
AsyncIO in folly

Summary:
Interface extended and cleaned up.  Also, now
actually allows you to retrieve IO errors.  Also moved some useful functions
out of Subprocess.cpp into a separate header file.

Test Plan: async_io_test, subprocess_test

Reviewed By: philipp@fb.com

FB internal diff: D698412

11 years agounsplit
Andrew Tulloch [Tue, 5 Feb 2013 00:54:37 +0000 (16:54 -0800)]
unsplit

Summary:
1. Incorporates @tjackson's offline comments.
2. See docstrings and examples for basic usage.
3. The usecase this covers (for me and others) was where I have some map or
vector of elements, and I want to form a string representation of this (for
logging, fb303 exported values, etc.). Various uses have existed in fbcode (e.g.
UP2X shard representations), and this seemed like a useful utility.

Test Plan: unit tests.

Reviewed By: tjackson@fb.com

FB internal diff: D696794

11 years agoUpdate example in folly::gen
Jordan DeLong [Mon, 4 Feb 2013 03:58:18 +0000 (19:58 -0800)]
Update example in folly::gen

Summary: asVector() doesn't exist.

Test Plan: Nope.

Reviewed By: tjackson@fb.com

FB internal diff: D697240

11 years agocontains()
Tom Jackson [Wed, 16 Jan 2013 04:02:16 +0000 (20:02 -0800)]
contains()

Summary: TSIA

Test Plan: Unit tests

Reviewed By: tulloch@fb.com

FB internal diff: D680177

11 years agofolly/dynamic: fix explicit specializations after instantiation
Andrew Gallagher [Tue, 29 Jan 2013 06:54:19 +0000 (22:54 -0800)]
folly/dynamic: fix explicit specializations after instantiation

Summary:
clang complains that the explicit specializations in dynamic.cpp
come after references in dynamic-inl.h.

Test Plan:
- built dyamic.o with clang
- built folly normally and ran unittests

Reviewed By: delong.j@fb.com

FB internal diff: D691081

11 years agoadding some missing remove_reference in make_stl_allocator and is_simple_allocator
Marcelo Juchem [Sat, 26 Jan 2013 01:22:57 +0000 (17:22 -0800)]
adding some missing remove_reference in make_stl_allocator and is_simple_allocator

Summary: adding some missing remove_reference in make_stl_allocator and is_simple_allocator

Test Plan: unit tests

Reviewed By: jon.coens@fb.com

FB internal diff: D689524

11 years agoRemove a bit of debug code in folly::Range
Jordan DeLong [Sun, 20 Jan 2013 20:18:07 +0000 (12:18 -0800)]
Remove a bit of debug code in folly::Range

Summary:
Seems like we can drop this, as the case that motivated
adding it wouldn't have been caught by it.

Test Plan: Compiled folly/test.

Reviewed By: tudorb@fb.com

FB internal diff: D684200

11 years agofixing rwspinlock test Summary:
Xin Liu [Fri, 25 Jan 2013 18:21:44 +0000 (10:21 -0800)]
fixing rwspinlock test Summary:

Summary:
gcc seems to treat this as declaration:

RWSpinLock::UpgradedHolder ug(
RWSpinLock::WriteHolder(
RWSpinLock::ReadHolder(&lock)));

Test Plan: add LOG(INFO) to make sure the holder converstions did get executed.

Reviewed By: marcelo.juchem@fb.com

FB internal diff: D688748