-.. _yamlio:
-
=====================
YAML I/O
=====================
In relational database theory there is a design step called normalization in
which you reorganize fields and tables. The same considerations need to
go into the design of your YAML encoding. But, you may not want to change
-your exisiting native data structures. Therefore, when writing out YAML
+your existing native data structures. Therefore, when writing out YAML
there may be a normalization step, and when reading YAML there would be a
corresponding denormalization step.
YAML I/O uses a non-invasive, traits based design. YAML I/O defines some
abstract base templates. You specialize those templates on your data types.
-For instance, if you have an eumerated type FooBar you could specialize
+For instance, if you have an enumerated type FooBar you could specialize
ScalarEnumerationTraits on that type and define the enumeration() method:
.. code-block:: c++
As with all YAML I/O template specializations, the ScalarEnumerationTraits is used for
both reading and writing YAML. That is, the mapping between in-memory enum
-values and the YAML string representation is only in place.
+values and the YAML string representation is only in one place.
This assures that the code for writing and parsing of YAML stays in sync.
To specify a YAML mappings, you define a specialization on
-llvm::yaml::MapppingTraits.
+llvm::yaml::MappingTraits.
If your native data structure happens to be a struct that is already normalized,
then the specialization is simple. For example:
.. code-block:: c++
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
template <>
- struct Mapp
pingTraits<Person> {
+ struct MappingTraits<Person> {
static void mapping(IO &io, Person &info) {
io.mapRequired("name", info.name);
io.mapOptional("hat-size", info.hatSize);
};
-A YAML sequence is automatically infered if you data type has begin()/end()
+A YAML sequence is automatically inferred if you data type has begin()/end()
iterators and a push_back() method. Therefore any of the STL containers
(such as std::vector<>) will automatically translate to YAML sequences.
* float
* double
* StringRef
+* std::string
* int64_t
* int32_t
* int16_t
* uint16_t
* uint8_t
-That is, you can use those types in fields of MapppingTraits or as element type
+That is, you can use those types in fields of MappingTraits or as element type
in sequence. When reading, YAML I/O will validate that the string found
is convertible to that type and error out if not.
.. code-block:: c++
using llvm::yaml::ScalarEnumerationTraits;
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
template <>
};
template <>
- struct MapppingTraits<Info> {
+ struct MappingTraits<Info> {
static void mapping(IO &io, Info &info) {
io.mapRequired("cpu", info.cpu);
io.mapOptional("flags", info.flags, 0);
flagsRound = 8
};
- LLVM_YAML_UNIQUE_TYPE(MyFlags, uint32_t)
+ LLVM_YAML_STRONG_TYPEDEF(uint32_t, MyFlags)
To support reading and writing of MyFlags, you specialize ScalarBitSetTraits<>
on MyFlags and provide the bit values and their names.
.. code-block:: c++
using llvm::yaml::ScalarBitSetTraits;
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
template <>
};
template <>
- struct MapppingTraits<Info> {
+ struct MappingTraits<Info> {
static void mapping(IO &io, Info& info) {
io.mapRequired("name", info.name);
io.mapRequired("flags", info.flags);
name: Tom
flags: [ pointy, flat ]
+Sometimes a "flags" field might contains an enumeration part
+defined by a bit-mask.
+
+.. code-block:: c++
+
+ enum {
+ flagsFeatureA = 1,
+ flagsFeatureB = 2,
+ flagsFeatureC = 4,
+
+ flagsCPUMask = 24,
+
+ flagsCPU1 = 8,
+ flagsCPU2 = 16
+ };
+
+To support reading and writing such fields, you need to use the maskedBitSet()
+method and provide the bit values, their names and the enumeration mask.
+
+.. code-block:: c++
+
+ template <>
+ struct ScalarBitSetTraits<MyFlags> {
+ static void bitset(IO &io, MyFlags &value) {
+ io.bitSetCase(value, "featureA", flagsFeatureA);
+ io.bitSetCase(value, "featureB", flagsFeatureB);
+ io.bitSetCase(value, "featureC", flagsFeatureC);
+ io.maskedBitSetCase(value, "CPU1", flagsCPU1, flagsCPUMask);
+ io.maskedBitSetCase(value, "CPU2", flagsCPU2, flagsCPUMask);
+ }
+ };
+
+YAML I/O (when writing) will apply the enumeration mask to the flags field,
+and compare the result and values from the bitset. As in case of a regular
+bitset, each that matches will cause the corresponding string to be added
+to the flow sequence.
Custom Scalar
-------------
custom formatting and parsing of scalar types by specializing ScalarTraits<> on
your data type. When writing, YAML I/O will provide the native type and
your specialization must create a temporary llvm::StringRef. When reading,
-YAML I/O will provide a llvm::StringRef of scalar and your specialization
+YAML I/O will provide an llvm::StringRef of scalar and your specialization
must convert that to your native data type. An outline of a custom scalar type
looks like:
static StringRef input(StringRef scalar, T &value) {
// do custom parsing here. Return the empty string on success,
// or an error message on failure.
- return StringRef();
+ return StringRef();
}
+ // Determine if this scalar needs quotes.
+ static bool mustQuote(StringRef) { return true; }
};
========
To be translated to or from a YAML mapping for your type T you must specialize
-llvm::yaml::MapppingTraits on T and implement the "void mapping(IO &io, T&)"
+llvm::yaml::MappingTraits on T and implement the "void mapping(IO &io, T&)"
method. If your native data structures use pointers to a class everywhere,
you can specialize on the class pointer. Examples:
.. code-block:: c++
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
// Example of struct Foo which is used by value
template <>
- struct MapppingTraits<Foo> {
+ struct MappingTraits<Foo> {
static void mapping(IO &io, Foo &foo) {
io.mapOptional("size", foo.size);
...
// Example of struct Bar which is natively always a pointer
template <>
- struct MapppingTraits<Bar*> {
+ struct MappingTraits<Bar*> {
static void mapping(IO &io, Bar *&bar) {
io.mapOptional("size", bar->size);
...
.. code-block:: c++
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
template <>
- struct Mapp
pingTraits<Person> {
+ struct MappingTraits<Person> {
static void mapping(IO &io, Person &info) {
io.mapRequired("name", info.name);
io.mapOptional("hat-size", info.hatSize);
x: 10.3
y: -4.7
-You can support this by defining a MapppingTraits that normalizes the polar
+You can support this by defining a MappingTraits that normalizes the polar
coordinates to x,y coordinates when writing YAML and denormalizes x,y
-coordindates into polar when reading YAML.
+coordinates into polar when reading YAML.
.. code-block:: c++
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
template <>
- struct Mapp
pingTraits<Polar> {
+ struct MappingTraits<Polar> {
class NormalizedPolar {
public:
y(polar.distance * sin(polar.angle)) {
}
Polar denormalize(IO &) {
- return Polar(sqrt(x*x+y*y, arctan(x,y));
+ return Polar(sqrt(x*x+y*y), arctan(x,y));
}
float x;
};
When writing YAML, the local variable "keys" will be a stack allocated
-instance of NormalizedPolar, constructed from the suppled polar object which
+instance of NormalizedPolar, constructed from the supplied polar object which
initializes it x and y fields. The mapRequired() methods then write out the x
and y values as key/value pairs.
normalized instance is stack allocated. In these cases, the utility template
MappingNormalizationHeap<> can be used instead. It just like
MappingNormalization<> except that it heap allocates the normalized object
-when reading YAML. It never destroyes the normalized object. The denormalize()
+when reading YAML. It never destroys the normalized object. The denormalize()
method can this return "this".
.. code-block:: c++
- using llvm::yaml::MapppingTraits;
+ using llvm::yaml::MappingTraits;
using llvm::yaml::IO;
struct Info {
};
template <>
- struct MapppingTraits<Info> {
+ struct MappingTraits<Info> {
static void mapping(IO &io, Info &info) {
io.mapRequired("cpu", info.cpu);
// flags must come after cpu for this to work when reading yaml
};
+Tags
+----
+
+The YAML syntax supports tags as a way to specify the type of a node before
+it is parsed. This allows dynamic types of nodes. But the YAML I/O model uses
+static typing, so there are limits to how you can use tags with the YAML I/O
+model. Recently, we added support to YAML I/O for checking/setting the optional
+tag on a map. Using this functionality it is even possbile to support different
+mappings, as long as they are convertable.
+
+To check a tag, inside your mapping() method you can use io.mapTag() to specify
+what the tag should be. This will also add that tag when writing yaml.
+
+Validation
+----------
+
+Sometimes in a yaml map, each key/value pair is valid, but the combination is
+not. This is similar to something having no syntax errors, but still having
+semantic errors. To support semantic level checking, YAML I/O allows
+an optional ``validate()`` method in a MappingTraits template specialization.
+
+When parsing yaml, the ``validate()`` method is call *after* all key/values in
+the map have been processed. Any error message returned by the ``validate()``
+method during input will be printed just a like a syntax error would be printed.
+When writing yaml, the ``validate()`` method is called *before* the yaml
+key/values are written. Any error during output will trigger an ``assert()``
+because it is a programming error to have invalid struct values.
+
+
+.. code-block:: c++
+
+ using llvm::yaml::MappingTraits;
+ using llvm::yaml::IO;
+
+ struct Stuff {
+ ...
+ };
+
+ template <>
+ struct MappingTraits<Stuff> {
+ static void mapping(IO &io, Stuff &stuff) {
+ ...
+ }
+ static StringRef validate(IO &io, Stuff &stuff) {
+ // Look at all fields in 'stuff' and if there
+ // are any bad values return a string describing
+ // the error. Otherwise return an empty string.
+ return StringRef();
+ }
+ };
+
+
Sequence
========
To be translated to or from a YAML sequence for your type T you must specialize
llvm::yaml::SequenceTraits on T and implement two methods:
-“size_t size(IO &io, T&)” and “T::value_type& element(IO &io, T&, size_t indx)”.
-For example:
+``size_t size(IO &io, T&)`` and
+``T::value_type& element(IO &io, T&, size_t indx)``. For example:
.. code-block:: c++
template <>
struct SequenceTraits<MySeq> {
static size_t size(IO &io, MySeq &list) { ... }
- static MySeqEl element(IO &io, MySeq &list, size_t index) { ... }
+ static MySeqEl &element(IO &io, MySeq &list, size_t index) { ... }
};
The size() method returns how many elements are currently in your sequence.
template <>
struct SequenceTraits<MyList> {
static size_t size(IO &io, MyList &list) { ... }
- static MyListEl element(IO &io, MyList &list, size_t index) { ... }
+ static MyListEl &element(IO &io, MyList &list, size_t index) { ... }
// The existence of this member causes YAML I/O to use a flow sequence
static const bool flow = true;
};
With the above, if you used MyList as the data type in your native data
-strucutures, then then when converted to YAML, a flow sequence of integers
+structures, then then when converted to YAML, a flow sequence of integers
will be used (e.g. [ 10, -3, 4 ]).
Utility Macros
--------------
-Since a common source of sequences is std::vector<>, YAML I/O provids macros:
+Since a common source of sequences is std::vector<>, YAML I/O provides macros:
LLVM_YAML_IS_SEQUENCE_VECTOR() and LLVM_YAML_IS_FLOW_SEQUENCE_VECTOR() which
can be used to easily specify SequenceTraits<> on a std::vector type. YAML
I/O does not partial specialize SequenceTraits on std::vector<> because that
projects / oota-llvm.git / blobdiff