Squashed 'third_party/boostorg/range/' content from commit 4cfd4d8
Change-Id: I641c49f21039952b16f888223a952503e43a28a9
git-subtree-dir: third_party/boostorg/range
git-subtree-split: 4cfd4d8287ca949d7f29256adf3e796a0d1775ec
diff --git a/doc/reference/adaptors/tokenized.qbk b/doc/reference/adaptors/tokenized.qbk
new file mode 100644
index 0000000..2105323
--- /dev/null
+++ b/doc/reference/adaptors/tokenized.qbk
@@ -0,0 +1,67 @@
+[/
+ Copyright 2010 Neil Groves
+ Distributed under the Boost Software License, Version 1.0.
+ (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+/]
+[section:tokenized tokenized]
+
+[table
+ [[Syntax] [Code]]
+ [
+ [Pipe]
+ [
+ ``
+ rng | boost::adaptors::tokenized(regex)
+ rng | boost::adaptors::tokenized(regex, i)
+ rng | boost::adaptors::tokenized(regex, rndRng)
+ rng | boost::adaptors::tokenized(regex, i, flags)
+ rng | boost::adaptors::tokenized(regex, rndRng, flags)
+ ``
+ ]
+ ]
+ [
+ [Function]
+ [
+ ``
+ boost::adaptors::tokenize(rng, regex)
+ boost::adaptors::tokenize(rng, regex, i)
+ boost::adaptors::tokenize(rng, regex, rndRng)
+ boost::adaptors::tokenize(rng, regex, i, flags)
+ boost::adaptors::tokenize(rng, regex, rndRng, flags)
+ ``
+ ]
+ ]
+]
+
+* [*Precondition:]
+ * Let `T` denote `typename range_value<decltype(rng)>::type`, then `regex` has the type `basic_regex<T>` or is implicitly convertible to one of these types.
+ * `i` has the type `int`.
+ * the `value_type` of `rndRng` is `int`.
+ * `flags` has the type `regex_constants::syntax_option_type`.
+* [*Returns:] A range whose iterators behave as if they were the original iterators wrapped in `regex_token_iterator`. The first iterator in the range would be constructed by forwarding all the arguments of `tokenized()` to the `regex_token_iterator` constructor.
+* [*Throws:] Whatever constructing and copying equivalent `regex_token_iterator`s might throw.
+* [*Range Category:] __random_access_range__
+* [*Range Return Type:] `boost::tokenized_range<decltype(rng)>`
+* [*Returned Range Category:] __random_access_range__
+
+[section:tokenized_example tokenized_example]
+[import ../../../test/adaptor_test/tokenized_example.cpp]
+[tokenized_example]
+[endsect]
+
+This would produce the output:
+``
+a
+b
+c
+d
+e
+f
+g
+hijklmnopqrstuvwxyz
+
+``
+
+[endsect]
+
+