parallel(pipeline)
authorTom Jackson <tjackson@fb.com>
Sat, 5 Apr 2014 00:58:07 +0000 (17:58 -0700)
committerSara Golemon <sgolemon@fb.com>
Fri, 18 Apr 2014 19:04:15 +0000 (12:04 -0700)
commitd3cc50e00280b1ca5ffd4f713ac312181b1d6810
tree5d6eed263daa267c45d4711c118d82058852fe70
parent7fbec7196afc262f23dd26431441f5c231d4d801
parallel(pipeline)

Summary:
Adding `... | parallel(my | pipe | line) | ...` for parallelizing a portion of a generator pipeline.

```lang=cpp
auto factored = from(values)
| parallel(filter(isEven) | map(square) | sub(count))
| sum;
```

Work is divided evenly among a fixed number of threads using a `MPMCQueue`.

Test Plan: Unit tests and benchmarks testing for a variety of workloads and performance characteristics, including sub-linear (blocking) workloads, linear (mostly math) workloads, and superlinear (sleeping) workloads to simulate real-world use.

Reviewed By: lucian@fb.com

FB internal diff: D638551
folly/gen/Base-inl.h
folly/gen/Base.h
folly/gen/Core-inl.h
folly/gen/Parallel-inl.h [new file with mode: 0644]
folly/gen/Parallel.h [new file with mode: 0644]
folly/gen/test/Bench.h [new file with mode: 0644]
folly/gen/test/ParallelBenchmark.cpp [new file with mode: 0644]
folly/gen/test/ParallelTest.cpp [new file with mode: 0644]