337 lines
11 KiB
Markdown
337 lines
11 KiB
Markdown
|
# Custom YAML data types
|
||
|
|
||
|
Sometimes you need to serialize complex data types such as classes. To
|
||
|
do this you could use plain nodes such as mappings with classes' fields.
|
||
|
YAML also supports custom types with identifiers called *tags*. That is
|
||
|
the topic of this tutorial.
|
||
|
|
||
|
Each YAML node has a tag specifying its type. For instance: strings use
|
||
|
the tag `tag:yaml.org,2002:str`. Tags of most default types are
|
||
|
*implicitly resolved* during parsing - you don't need to specify tag for
|
||
|
each float, integer, etc. D:YAML can also implicitly resolve custom
|
||
|
tags, as we will show later.
|
||
|
|
||
|
## Constructor
|
||
|
|
||
|
D:YAML uses the [Constructor](https://dyaml.dpldocs.info/dyaml.constructor.Constructor.html)
|
||
|
class to process each node to hold data type corresponding to its tag.
|
||
|
*Constructor* stores functions to process each supported tag. These are
|
||
|
supplied by the user using the *addConstructorXXX()* methods, where
|
||
|
*XXX* is *Scalar*, *Sequence* or *Mapping*. *Constructor* is then passed
|
||
|
to *Loader*, which parses YAML input.
|
||
|
|
||
|
Structs and classes must implement the *opCmp()* operator for YAML
|
||
|
support. This is used for duplicate detection in mappings, sorting and
|
||
|
equality comparisons of nodes. The signature of the operator that must
|
||
|
be implemented is `const int opCmp(ref const MyStruct s)` for structs
|
||
|
where *MyStruct* is the struct type, and `int opCmp(Object o)` for
|
||
|
classes. Note that the class *opCmp()* should not alter the compared
|
||
|
values - it is not const for compatibility reasons.
|
||
|
|
||
|
We will implement support for an RGB color type. It is implemented as
|
||
|
the following struct:
|
||
|
|
||
|
```D
|
||
|
struct Color
|
||
|
{
|
||
|
ubyte red;
|
||
|
ubyte green;
|
||
|
ubyte blue;
|
||
|
|
||
|
const int opCmp(ref const Color c)
|
||
|
{
|
||
|
if(red != c.red) {return red - c.red;}
|
||
|
if(green != c.green){return green - c.green;}
|
||
|
if(blue != c.blue) {return blue - c.blue;}
|
||
|
return 0;
|
||
|
}
|
||
|
}
|
||
|
```
|
||
|
|
||
|
First, we need a function to construct our data type. The function will
|
||
|
take a reference to *Node* to construct from. The node is guaranteed to
|
||
|
contain either a *string*, an array of *Node* or of *Node.Pair*,
|
||
|
depending on whether we're constructing our value from a scalar,
|
||
|
sequence, or mapping, respectively. If this function throws any
|
||
|
exception, D:YAML handles it and adds its message to a *YAMLException*
|
||
|
that will be thrown when loading the file.
|
||
|
|
||
|
In this tutorial, we have functions to construct a color from a scalar,
|
||
|
using CSS-like format, RRGGBB, or from a mapping, where we use the
|
||
|
following format: {r:RRR, g:GGG, b:BBB} . Code of these functions:
|
||
|
|
||
|
```D
|
||
|
Color constructColorScalar(ref Node node)
|
||
|
{
|
||
|
string value = node.as!string;
|
||
|
|
||
|
if(value.length != 6)
|
||
|
{
|
||
|
throw new Exception("Invalid color: " ~ value);
|
||
|
}
|
||
|
//We don't need to check for uppercase chars this way.
|
||
|
value = value.toLower();
|
||
|
|
||
|
//Get value of a hex digit.
|
||
|
uint hex(char c)
|
||
|
{
|
||
|
import std.ascii;
|
||
|
if(!std.ascii.isHexDigit(c))
|
||
|
{
|
||
|
throw new Exception("Invalid color: " ~ value);
|
||
|
}
|
||
|
|
||
|
if(std.ascii.isDigit(c))
|
||
|
{
|
||
|
return c - '0';
|
||
|
}
|
||
|
return c - 'a' + 10;
|
||
|
}
|
||
|
|
||
|
Color result;
|
||
|
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
|
||
|
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
|
||
|
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
|
||
|
|
||
|
return result;
|
||
|
}
|
||
|
|
||
|
Color constructColorMapping(ref Node node)
|
||
|
{
|
||
|
ubyte r,g,b;
|
||
|
|
||
|
//Might throw if a value is missing is not an integer, or is out of range.
|
||
|
//If this happens, D:YAML will handle the exception and use its message
|
||
|
//in a YAMLException thrown when loading.
|
||
|
r = node["r"].as!ubyte;
|
||
|
g = node["g"].as!ubyte;
|
||
|
b = node["b"].as!ubyte;
|
||
|
|
||
|
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
|
||
|
}
|
||
|
```
|
||
|
|
||
|
Next, we need some YAML data using our new tag. Create a file called
|
||
|
`input.yaml` with the following contents:
|
||
|
|
||
|
```YAML
|
||
|
scalar-red: !color FF0000
|
||
|
scalar-orange: !color FFFF00
|
||
|
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
|
||
|
mapping-orange:
|
||
|
!color-mapping
|
||
|
r: 255
|
||
|
g: 255
|
||
|
b: 0
|
||
|
```
|
||
|
|
||
|
You can see that we're using tag `!color` for scalar colors, and
|
||
|
`!color-mapping` for colors expressed as mappings.
|
||
|
|
||
|
Finally, the code to put it all together:
|
||
|
|
||
|
```D
|
||
|
void main()
|
||
|
{
|
||
|
auto red = Color(255, 0, 0);
|
||
|
auto orange = Color(255, 255, 0);
|
||
|
|
||
|
try
|
||
|
{
|
||
|
auto constructor = new Constructor;
|
||
|
//both functions handle the same tag, but one handles scalar, one mapping.
|
||
|
constructor.addConstructorScalar("!color", &constructColorScalar);
|
||
|
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
|
||
|
|
||
|
auto loader = Loader("input.yaml");
|
||
|
loader.constructor = constructor;
|
||
|
|
||
|
auto root = loader.load();
|
||
|
|
||
|
if(root["scalar-red"].as!Color == red &&
|
||
|
root["mapping-red"].as!Color == red &&
|
||
|
root["scalar-orange"].as!Color == orange &&
|
||
|
root["mapping-orange"].as!Color == orange)
|
||
|
{
|
||
|
writeln("SUCCESS");
|
||
|
return;
|
||
|
}
|
||
|
}
|
||
|
catch(YAMLException e)
|
||
|
{
|
||
|
writeln(e.msg);
|
||
|
}
|
||
|
|
||
|
writeln("FAILURE");
|
||
|
}
|
||
|
```
|
||
|
|
||
|
First, we create a *Constructor* and pass functions to handle the
|
||
|
`!color` and `!color-mapping` tag. We construct a *Loader* and pass the
|
||
|
*Constructor* to it. We then load the YAML document, and finally, read
|
||
|
the colors to test if they were loaded as expected.
|
||
|
|
||
|
You can find the source code for what we've done so far in the
|
||
|
`examples/constructor` directory in the D:YAML package.
|
||
|
|
||
|
## Resolver
|
||
|
|
||
|
Specifying tag for every color can be tedious. D:YAML can implicitly
|
||
|
resolve scalar tags using regular expressions. This is how default types
|
||
|
are resolved. We will use the [Resolver](../api/dyaml.resolver.html)
|
||
|
class to add implicit tag resolution for the Color data type (in its
|
||
|
scalar form).
|
||
|
|
||
|
We use the *addImplicitResolver()* method of *Resolver*, passing the
|
||
|
tag, regular expression the scalar must match to resolve to this tag,
|
||
|
and a string of possible starting characters of the scalar. Then we pass
|
||
|
the *Resolver* to *Loader*.
|
||
|
|
||
|
Note that resolvers added first override ones added later. If no
|
||
|
resolver matches a scalar, YAML string tag is used. Therefore our custom
|
||
|
values must not be resolvable as any non-string YAML data type.
|
||
|
|
||
|
Add this to your code to add implicit resolution of `!color`.
|
||
|
|
||
|
```D
|
||
|
//code from the previous example...
|
||
|
|
||
|
auto resolver = new Resolver;
|
||
|
import std.regex;
|
||
|
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
|
||
|
"0123456789abcdefABCDEF");
|
||
|
|
||
|
auto loader = Loader("input.yaml");
|
||
|
|
||
|
loader.constructor = constructor;
|
||
|
loader.resolver = resolver;
|
||
|
|
||
|
//code from the previous example...
|
||
|
```
|
||
|
|
||
|
Now, change contents of `input.yaml` to this:
|
||
|
|
||
|
```YAML
|
||
|
scalar-red: FF0000
|
||
|
scalar-orange: FFFF00
|
||
|
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
|
||
|
mapping-orange:
|
||
|
!color-mapping
|
||
|
r: 255
|
||
|
g: 255
|
||
|
b: 0
|
||
|
```
|
||
|
|
||
|
We no longer need to specify the tag for scalar color values. Compile
|
||
|
and test the example. If everything went as expected, it should report
|
||
|
success.
|
||
|
|
||
|
You can find the complete code in the `examples/resolver` directory in
|
||
|
the D:YAML package.
|
||
|
|
||
|
## Representer
|
||
|
|
||
|
Now that you can load custom data types, it might be good to know how to
|
||
|
dump them. D:YAML uses the [Representer](https://dyaml.dpldocs.info/dyaml.representer.Representer.html)
|
||
|
class for this purpose.
|
||
|
|
||
|
*Representer* processes YAML nodes into plain mapping, sequence or
|
||
|
scalar nodes ready for output. Just like with *Constructor*, this is
|
||
|
done by user specified functions. These functions take references to a
|
||
|
node to process and to the *Representer*, and return the processed node.
|
||
|
|
||
|
Representer functions can be added with the *addRepresenter()* method.
|
||
|
The *Representer* is then passed to *Dumper*, which dumps YAML
|
||
|
documents. Only one function per type can be specified. This is asserted
|
||
|
in *addRepresenter()* preconditions. Default YAML types already have
|
||
|
representer functions specified, but you can disable them by
|
||
|
constructing *Representer* with the *useDefaultRepresenters* parameter
|
||
|
set to false.
|
||
|
|
||
|
By default, tags are explicitly output for all non-default types. To
|
||
|
make dumped tags implicit, you can pass a *Resolver* that will resolve
|
||
|
them implicitly. Of course, you will need to use an identical *Resolver*
|
||
|
when loading the output.
|
||
|
|
||
|
With the following code, we will add support for dumping the our Color
|
||
|
type.
|
||
|
|
||
|
```D
|
||
|
Node representColor(ref Node node, Representer representer)
|
||
|
{
|
||
|
//The node is guaranteed to be Color as we add representer for Color.
|
||
|
Color color = node.as!Color;
|
||
|
|
||
|
static immutable hex = "0123456789ABCDEF";
|
||
|
|
||
|
//Using the color format from the Constructor example.
|
||
|
string scalar;
|
||
|
foreach(channel; [color.red, color.green, color.blue])
|
||
|
{
|
||
|
scalar ~= hex[channel / 16];
|
||
|
scalar ~= hex[channel % 16];
|
||
|
}
|
||
|
|
||
|
//Representing as a scalar, with custom tag to specify this data type.
|
||
|
return representer.representScalar("!color", scalar);
|
||
|
}
|
||
|
```
|
||
|
|
||
|
First we get the *Color* from the node. Then we convert it to a string
|
||
|
with the CSS-like format we've used before. Finally, we use the
|
||
|
*representScalar()* method of *Representer* to get a scalar node ready
|
||
|
for output. There are corresponding *representMapping()* and
|
||
|
*representSequence()* methods as well, with examples in the [Resolver
|
||
|
API documentation](https://dyaml.dpldocs.info/dyaml.resolver.html).
|
||
|
|
||
|
Since a type can only have one representer function, we don't dump
|
||
|
*Color* both in the scalar and mapping formats we've used before.
|
||
|
However, you can decide to dump the node with different formats/tags in
|
||
|
the representer function itself. E.g. you could dump the Color as a
|
||
|
mapping based on some arbitrary condition, such as the color being
|
||
|
white.
|
||
|
|
||
|
```D
|
||
|
void main()
|
||
|
{
|
||
|
try
|
||
|
{
|
||
|
auto representer = new Representer;
|
||
|
representer.addRepresenter!Color(&representColor);
|
||
|
|
||
|
auto resolver = new Resolver;
|
||
|
import std.regex;
|
||
|
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
|
||
|
"0123456789abcdefABCDEF");
|
||
|
|
||
|
auto dumper = Dumper("output.yaml");
|
||
|
dumper.representer = representer;
|
||
|
dumper.resolver = resolver;
|
||
|
|
||
|
auto document = Node([Color(255, 0, 0),
|
||
|
Color(0, 255, 0),
|
||
|
Color(0, 0, 255)]);
|
||
|
|
||
|
dumper.dump(document);
|
||
|
}
|
||
|
catch(YAMLException e)
|
||
|
{
|
||
|
writeln(e.msg);
|
||
|
}
|
||
|
}
|
||
|
```
|
||
|
|
||
|
We construct a new *Representer*, and specify a representer function for
|
||
|
the *Color* (the template argument) type. We also construct a
|
||
|
*Resolver*, same as in the previous section, so the `!color` tag will be
|
||
|
implicit. Of course, identical *Resolver* would then have to be used
|
||
|
when loading the file. You don't need to do this if you want the tag to
|
||
|
be explicit.
|
||
|
|
||
|
We construct a *Dumper* to file `output.yaml` and pass the *Representer*
|
||
|
and *Resolver* to it. Then, we create a simple node containing a
|
||
|
sequence of colors and finally, we dump it.
|
||
|
|
||
|
Source code for this section can be found in the `examples/representer`
|
||
|
directory of the D:YAML package.
|