folly.git
11 years agoImprove digits_to() performance
Eitan Frachtenberg [Wed, 13 Jun 2012 00:07:02 +0000 (17:07 -0700)]
Improve digits_to() performance

Summary:
This optimization eliminates about 75% of the multiplications and bounds
checking in the original code by using constant lookup tables to convert
from decical digit to (shifted) binary number, instead of arithmetic.
The total cost of the lookup tables is 2KB of static memory, but the
average throughput gain across the range from 1 to 19 characters is 27%.
The throughput distributes as follows: (in million iters/sec)
Bytes   Orig    New     Speedup
1       101.31  110.12  109%
2       97.42   110.12  113%
3       97.42   105.53  108%
4       79.13   101.31  128%
5       74.48   97.42   131%
6       74.48   93.81   126%
7       70.35   93.81   133%
8       61.77   79.14   128%
9       58.9    72.35   123%
10      58.9    74.48   126%
11      56.28   74.48   132%
12      50.22   64.93   129%
13      48.52   60.3    124%
14      47.74   61.77   129%
15      46.88   61.77   132%
16      42.54   55.06   129%
17      41.51   51.69   125%
18      40.97   52.76   129%
19      39.57   52.76   133%

Test Plan: Passes all unit tests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D493245

11 years agoAdd multi-bit operations to folly/experimental/Bits.h
Tudor Bosman [Wed, 4 Jul 2012 00:02:05 +0000 (17:02 -0700)]
Add multi-bit operations to folly/experimental/Bits.h

Summary: Add ability to set and get N consecutive bits from the bit sequence.

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D511438

11 years agoadding StringPiece constructor that takes a piece of another StringPiece
Rafael Sagula [Thu, 28 Jun 2012 23:05:21 +0000 (16:05 -0700)]
adding StringPiece constructor that takes a piece of another StringPiece

Summary:
adding what seems to be a missing constructor to StringPiece --
I need to be able to take a piece of another StringPiece. (It's possible
to do that with all sorts of strings already, except StringPiece...)

Test Plan: na -- tested as part of other (dependent) diffs

Reviewed By: delong.j@fb.com

FB internal diff: D508545

11 years agoRemove unnecessary member variable.
Tudor Bosman [Mon, 2 Jul 2012 00:40:06 +0000 (17:40 -0700)]
Remove unnecessary member variable.

Test Plan: No

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D509324

11 years agoadd bm_max/min_iters and bm_regex
Spencer Ahrens [Sat, 30 Jun 2012 18:01:06 +0000 (11:01 -0700)]
add bm_max/min_iters and bm_regex

Summary: These were handy in the old framework and easy enough to add.

Test Plan:
ran ThreadCachedIntTest - got better (but not perfect due to
subtraction of single-threaded baseline from multi-threaded results).

Reviewed By: sherman.ye@fb.com

FB internal diff: D490324

11 years agofolly: get building with gcc-4.7.1
Andrew Gallagher [Fri, 29 Jun 2012 23:47:49 +0000 (16:47 -0700)]
folly: get building with gcc-4.7.1

Summary:
A lot of libstdc++-4.7 headers dropped includes of unistdc.h, so
we'll probably see more of these.

Test Plan: built folly

Reviewed By: simpkins@fb.com

FB internal diff: D508871

11 years agoMinor changes to folly/experimental/io
Tudor Bosman [Mon, 11 Jun 2012 23:24:30 +0000 (16:24 -0700)]
Minor changes to folly/experimental/io

Reviewed By: philipp@fb.com

FB internal diff: D491952

11 years agoAdding support to output benchmarks result in JSON
Rajat Goel [Mon, 25 Jun 2012 20:41:49 +0000 (13:41 -0700)]
Adding support to output benchmarks result in JSON

Summary: The main reason being so that they can be fed to some automated systems, if required.

Test Plan: Just ran few benchmarks locally

Reviewed By: andrewjcg@fb.com

FB internal diff: D501025

11 years agoAdd TypedIOBuf.
Tudor Bosman [Fri, 15 Jun 2012 00:07:00 +0000 (17:07 -0700)]
Add TypedIOBuf.

Summary: Simple class to handle IOBuf as an array of objects of fixed size.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D497287

11 years agoAdd sizeGuess() member to ProducerConsumerQueue
Mike Curtiss [Thu, 7 Jun 2012 07:32:14 +0000 (00:32 -0700)]
Add sizeGuess() member to ProducerConsumerQueue

Summary:
This is part 1 of a change to add hysteretic behavior to a blocking queue based on ProducerConsumerQueue.

Knowing the size is useful for monitoring and possibly for objects that contain a ProducerConsumerQueue.

Test Plan:
Added tiny test-case.

Tests pass

Reviewed By: delong.j@fb.com

FB internal diff: D496787

11 years agoAdd an allocate() member function on IOBufQueue
Tudor Bosman [Wed, 13 Jun 2012 23:52:38 +0000 (16:52 -0700)]
Add an allocate() member function on IOBufQueue

Summary: Function is documented in the header.

Reviewed By: soren@fb.com

FB internal diff: D494315

11 years agoFix a typo in benchmark comments
Sherman Ye [Wed, 9 May 2012 16:26:13 +0000 (09:26 -0700)]
Fix a typo in benchmark comments

Summary: As title

Test Plan: Deployed to wormhole_apps.dragon.dev

Reviewed By: asharma@fb.com

FB internal diff: D493253

11 years agoUnbreak Ubuntu build.
Tudor Bosman [Thu, 14 Jun 2012 19:14:04 +0000 (12:14 -0700)]
Unbreak Ubuntu build.

Summary: follybenchmark and follyfingerprint need folly to work.

Test Plan: built on ubuntu

Reviewed By: andrewjcg@fb.com

FB internal diff: D494955

11 years agoFix clang narrowing error in GroupVarintTables
Andrew Gallagher [Thu, 14 Jun 2012 19:24:31 +0000 (12:24 -0700)]
Fix clang narrowing error in GroupVarintTables

Summary:
The __m128i type is a pair of 64-bit signed ints and the values
generated in generate_varint_tables.py overflow this range, which
causes clang to complain.

Test Plan:
Built and ran unittests.  Also verified that the GroupVarintTable.o
data section was identical with and w/o this change.

Reviewed By: tudorb@fb.com

FB internal diff: D494645

11 years agoClarify split / splitTo comments.
Tudor Bosman [Thu, 14 Jun 2012 01:57:34 +0000 (18:57 -0700)]
Clarify split / splitTo comments.

Test Plan: No

Reviewed By: delong.j@fb.com

FB internal diff: D494525

11 years agoMove and refactor code
Chip Turner [Fri, 8 Jun 2012 21:53:51 +0000 (14:53 -0700)]
Move and refactor code

Summary:

Moves some string manipulation code like humanify into folly

Moves hash-related functions into folly and removes some old includes
to clean up some code

Adds generic hashing for tuples, similar to pairs

Updates all of the build breakages from the above

Test Plan: run unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D490478

11 years agoFix narrowing warning (if enabled).
Tudor Bosman [Wed, 13 Jun 2012 00:37:21 +0000 (17:37 -0700)]
Fix narrowing warning (if enabled).

Summary: They're (signed) long long.

Test Plan: built, folly/test

Reviewed By: andrewjcg@fb.com

FB internal diff: D493270

11 years agoFix intermittent failure in concurrent_skip_list_test
Tudor Bosman [Wed, 13 Jun 2012 00:08:04 +0000 (17:08 -0700)]
Fix intermittent failure in concurrent_skip_list_test

Summary: It creates too many threads, stop when we run out of resources.

Test Plan: by hand, in a loop, in a memory-constrained environment

Reviewed By: delong.j@fb.com

FB internal diff: D493239

11 years agoFix build and test.
Tudor Bosman [Tue, 12 Jun 2012 22:44:21 +0000 (15:44 -0700)]
Fix build and test.

Summary:
Fix bug in FBStringTest (don't reuse the same file name)

Don't use VLOG, as that requires that glog is compiled with gflags
support, which it isn't always.

Add other targets and tests, make all tests pass.

Test Plan: tests

Reviewed By: delong.j@fb.com

FB internal diff: D493018

11 years agoFix build and test.
Tudor Bosman [Tue, 12 Jun 2012 22:44:21 +0000 (15:44 -0700)]
Fix build and test.

Summary:
Fix bug in FBStringTest (don't reuse the same file name)

Don't use VLOG, as that requires that glog is compiled with gflags
support, which it isn't always.

Add other targets and tests, make all tests pass.

Test Plan: tests

Reviewed By: delong.j@fb.com

FB internal diff: D493018

11 years agoMove Rabin fingerprinting code to folly.
Tudor Bosman [Tue, 12 Jun 2012 16:20:03 +0000 (09:20 -0700)]
Move Rabin fingerprinting code to folly.

Summary:
Also generate fingerprint tables every time, so the code doesn't rot.

TODO(tudorb): move benchmark to folly
TODO(tudorb): Include the program used to generate the polynomials
(can't build as it requires NTL from http://www.shoup.net/ntl/)

Test Plan: folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D492455

11 years agoInternalBuf doesn't need capacity
Tudor Bosman [Tue, 12 Jun 2012 00:09:43 +0000 (17:09 -0700)]
InternalBuf doesn't need capacity

Summary: as it's always kMaxInternalDataSize.

Test Plan: all tests in folly/experimental/io/test, both opt and dbg

Reviewed By: brianp@fb.com

FB internal diff: D492008

11 years agoProperly align internal buffer in IOBuf.
Tudor Bosman [Mon, 11 Jun 2012 23:18:22 +0000 (16:18 -0700)]
Properly align internal buffer in IOBuf.

Summary:
Yes, it's gcc specific, because alignas and max_align_t aren't in gcc
4.6.2.

Test Plan: iobuf_test, test added

Reviewed By: brianp@fb.com

FB internal diff: D491939

11 years agoAdd IOBuf::takeOwnership(unique_ptr<T>&&).
Tudor Bosman [Mon, 11 Jun 2012 20:58:15 +0000 (13:58 -0700)]
Add IOBuf::takeOwnership(unique_ptr<T>&&).

Summary:
Useful for in-place serialization.  Dangerous, obviously, but anything
involving IOBuf usually is.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D491663

11 years agoTypos.
Tudor Bosman [Sat, 9 Jun 2012 22:32:10 +0000 (15:32 -0700)]
Typos.

Test Plan: No

Reviewed By: delong.j@fb.com

FB internal diff: D491038

11 years agoD'oh: fix size computation in ThreadLocalDetail.h
Tudor Bosman [Sat, 9 Jun 2012 04:30:50 +0000 (21:30 -0700)]
D'oh: fix size computation in ThreadLocalDetail.h

Test Plan: thread_local_test

Reviewed By: ldemailly@fb.com

FB internal diff: D490912

11 years agofolly::detail::futex should be inline
Philip Pronin [Fri, 8 Jun 2012 08:37:24 +0000 (01:37 -0700)]
folly::detail::futex should be inline

Summary: It breaks build.

Test Plan: will check *_try builds

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D489935

11 years agoAdd IOBufQueue::wrapBuffer, which handles buffers > 4GB.
Tudor Bosman [Thu, 7 Jun 2012 20:59:38 +0000 (13:59 -0700)]
Add IOBufQueue::wrapBuffer, which handles buffers > 4GB.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D489158

11 years agoAdd links to README.
Tudor Bosman [Thu, 7 Jun 2012 21:26:44 +0000 (14:26 -0700)]
Add links to README.

Test Plan: No

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D489199

11 years agoDeuglify unaligned accesses in GroupVarint.
Tudor Bosman [Thu, 7 Jun 2012 02:56:43 +0000 (19:56 -0700)]
Deuglify unaligned accesses in GroupVarint.

Test Plan: folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D488631

11 years agoEventCount: "condition variable" for lock-free algorithms
Tudor Bosman [Thu, 17 May 2012 21:31:04 +0000 (14:31 -0700)]
EventCount: "condition variable" for lock-free algorithms

Summary:
This allows you to convert non-blocking lock-free algorithms into
blocking versions.

See
http://www.1024cores.net/home/lock-free-algorithms/eventcounts,
http://software.intel.com/en-us/forums/showthread.php?t=62364,
http://www.akkadia.org/drepper/futex.pdf

Test Plan: new test

Reviewed By: delong.j@fb.com

FB internal diff: D474570

11 years agofbstring: remove reference to /home/engshare/ paths
Andrew Gallagher [Tue, 5 Jun 2012 23:13:31 +0000 (16:13 -0700)]
fbstring: remove reference to /home/engshare/ paths

Summary: Reported by https://github.com/facebook/folly/issues/8

Test Plan: no

Reviewed By: tudorb@fb.com

FB internal diff: D487572

11 years agoFix some memory_order parameters in AtomicHashMap
Jordan DeLong [Sun, 3 Jun 2012 21:58:50 +0000 (14:58 -0700)]
Fix some memory_order parameters in AtomicHashMap

Summary: Herb Sutter noticed an error in the AHM find pseudo-code at
the C++ conference, and the error is in the real implementation too.
We should be using memory_order_acquire, because we need the load of
the key to happen before any subsequent loads of the value itself.
Our implementation is conservative for a memory_order_relaxed load
(we generate the same code as if it were memory_order_acquire---i.e. a
compiler barrier but no memory barriers), so this should not change the
generated code.  Correctness is a good idea, though (when we upgrade
to gcc 4.7's atomics it might not be as conservative).  Also, erase was
doing three loads when one is enough.  ... also fixes an exception safety
bug in insert.  (Phew.)

Test Plan: Ran the AHM unit tests, debug and opt.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D485744

11 years agoSome typos corrects, and collapsed the two descriptions of Bits.h into one.
Christopher C. Aycock [Sun, 3 Jun 2012 03:59:02 +0000 (00:59 -0300)]
Some typos corrects, and collapsed the two descriptions of Bits.h into one.

Summary: https://github.com/facebook/folly/pull/1

Test Plan: N/A.

Reviewed By: andrewjcg@fb.com

FB internal diff: D485738

11 years agoAdd #include <new> to folly/Malloc.h.
Tudor Bosman [Tue, 5 Jun 2012 07:04:29 +0000 (00:04 -0700)]
Add #include <new> to folly/Malloc.h.

Test Plan: folly tests

Reviewed By: philipp@fb.com, andrei.alexandrescu@fb.com

FB internal diff: D486884D487376

11 years agoCheck the return value from malloc / realloc.
Tudor Bosman [Tue, 5 Jun 2012 05:24:21 +0000 (22:24 -0700)]
Check the return value from malloc / realloc.

Summary:
https://github.com/facebook/folly/issues/7

Wrappers: checkedMalloc / checkedRealloc / checkedCalloc

Test Plan: all folly tests

FB internal diff: D486841

11 years agoFix error in ProducerQueue::isEmpty
Michael Curtiss [Tue, 5 Jun 2012 05:16:44 +0000 (22:16 -0700)]
Fix error in ProducerQueue::isEmpty

Summary:
Oops.

Also: documented the slightly confusing behavior w.r.t. 'size'.

Test Plan: Added a unit test.

Reviewers: tjackson, jdelong

Reviewed By: jdelong

CC: folly@lists, lr, bagashe
Differential Revision: https://phabricator.fb.com/D486832

11 years agoTypos.
Tudor Bosman [Tue, 5 Jun 2012 02:50:18 +0000 (19:50 -0700)]
Typos.

Summary:
As pointed out by orbitcowboy at
https://github.com/facebook/folly/issues/6

Test Plan: No

Reviewers: jdelong, aalexandre

Reviewed By: jdelong

CC: folly@lists, bagashe
Differential Revision: https://phabricator.fb.com/D486754

11 years agoMove LICENCE and README to top level directory
Jordan DeLong [Sun, 3 Jun 2012 22:34:03 +0000 (15:34 -0700)]
Move LICENCE and README to top level directory

Summary: For the public version of the repo.

Test Plan: N/A.

11 years agoadd a .gitignore and remove folly-config.h (generated file) from the repo
Chip Turner [Sun, 3 Jun 2012 04:23:59 +0000 (21:23 -0700)]
add a .gitignore and remove folly-config.h (generated file) from the repo

11 years agoAdding support for in-place use of ProducerConsumerQueue.
Tom Jackson [Mon, 4 Jun 2012 23:57:49 +0000 (16:57 -0700)]
Adding support for in-place use of ProducerConsumerQueue.

Summary: As it is, ProducerConsumerQueue requires that values are moved or copied on the way out of the queue. It would be nice if it was possible to get a reference to the front of the queue, use it in place, then destruct it.

Test Plan: Unit tests

FB internal diff: D484538

11 years agoAdd a LICENSE file for folly
Jordan DeLong [Sun, 3 Jun 2012 02:43:28 +0000 (19:43 -0700)]
Add a LICENSE file for folly

Summary: Adds a copy of the apache license 2.0.

Test Plan: None.

11 years agoUpdate the folly/README
Jordan DeLong [Sat, 2 Jun 2012 23:13:53 +0000 (16:13 -0700)]
Update the folly/README

Summary: Now that we have an Overview.md.

Test Plan: None.

11 years agoPull from FB rev 63ce89e2f2301e6bba44a111cc7d4218022156f6
Jordan DeLong [Sat, 2 Jun 2012 18:19:29 +0000 (11:19 -0700)]
Pull from FB rev 63ce89e2f2301e6bba44a111cc7d4218022156f6

11 years agoinitial
Jordan DeLong [Fri, 1 Jun 2012 21:00:42 +0000 (14:00 -0700)]
initial