Custom YAML data types¶
Often you might want to serialize complex data types such as classes. You can use functions to process nodes such as a mapping containing class data members indexed by name. Alternatively, YAML supports custom data types using identifiers called tags. That is the topic of this tutorial.
Each YAML node has a tag specifying its type. For instance: strings use the tag tag:yaml.org,2002:str. Tags of most default types are implicitly resolved during parsing, so you don’t need to specify tag for each float, integer, etc. It is also possible to implicitly resolve custom tags, as we will show later.
Constructor¶
D:YAML uses the Constructor class to process each node to hold data type corresponding to its tag. Constructor stores a function for each supported tag to process it. These functions are supplied by the user using the addConstructorXXX() methods, where XXX is Scalar, Sequence or Mapping. Constructor is then passed to Loader, which parses YAML input.
Struct types have no specific requirements for YAML support. Class types should define the opEquals() operator, as this is used in equality comparisons of nodes. Default class opEquals() compares references, which means two identical objects might be considered unequal.
We will implement support for an RGB color type. It is implemented as the following struct:
struct Color
{
ubyte red;
ubyte green;
ubyte blue;
}
First, we need a function to construct our data type. It must take two Mark structs, which store position of the node in the file, and a reference to Node to construct from. The node is guaranteed to contain either a string, an array of Node or of Node.Pair, depending on whether we’re constructing our value from a scalar, sequence, or mapping, respectively. In this tutorial, we have functions to construct a color from a scalar, using CSS-like format, RRGGBB, or from a mapping, where we use the following format: {r:RRR, g:GGG, b:BBB} . Code of these functions:
Color constructColorScalar(Mark start, Mark end, ref Node node)
{
string value = node.get!string;
if(value.length != 6)
{
throw new ConstructorException("Invalid color: " ~ value, start, end);
}
//We don't need to check for uppercase chars this way.
value = value.toLower();
//Get value of a hex digit.
uint hex(char c)
{
if(!std.ascii.isHexDigit(c))
{
throw new ConstructorException("Invalid color: " ~ value, start, end);
}
if(std.ascii.isDigit(c))
{
return c - '0';
}
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
}
Color constructColorMapping(Mark start, Mark end, ref Node node)
{
int r,g,b;
bool error = false;
//Might throw if a value is missing or is not an integer.
try
{
r = node["r"].get!int;
g = node["g"].get!int;
b = node["b"].get!int;
}
catch(NodeException e)
{
error = true;
}
if(error || r < 0 || r > 255 || g < 0 || g > 255 || b < 0 || b > 255)
{
throw new ConstructorException("Invalid color", start, end);
}
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
Next, we need some YAML data using our new tag. Create a file called input.yaml with the following contents:
scalar-red: !color FF0000
scalar-orange: !color FFFF00
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
mapping-orange:
!color-mapping
r: 255
g: 255
b: 0
You can see that we’re using tag !color for scalar colors, and !color-mapping for colors expressed as mappings.
Finally, the code to put it all together:
void main()
{
auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0);
try
{
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto loader = Loader("input.yaml");
loader.constructor = constructor;
auto root = loader.load();
if(root["scalar-red"].get!Color == red &&
root["mapping-red"].get!Color == red &&
root["scalar-orange"].get!Color == orange &&
root["mapping-orange"].get!Color == orange)
{
writeln("SUCCESS");
return;
}
}
catch(YAMLException e)
{
writeln(e.msg);
}
writeln("FAILURE");
}
First, we create a Constructor and pass functions to handle the !color and !color-mapping tag. We construct a Loader and pass the Constructor to it. We then load the YAML document, and finally, read the colors using get() method to test if they were loaded as expected.
You can find the source code for what we’ve done so far in the examples/constructor directory in the D:YAML package.
Resolver¶
Specifying tag for every color value can be tedious. D:YAML can implicitly resolve scalar tags using regular expressions. This is how default types such as int are resolved. We will use the Resolver class to add implicit tag resolution for the Color data type (in its scalar form).
We use the addImplicitResolver() method of Resolver, passing the tag, regular expression the scalar must match to resolve to this tag, and a string of possible starting characters of the scalar. Then we pass the Resolver to Loader.
Note that resolvers added first override ones added later. If no resolver matches a scalar, YAML string tag is used. Therefore our custom values must not be resolvable as any non-string YAML data type.
Add this to your code to add implicit resolution of !color.
//code from the previous example...
auto resolver = new Resolver;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto loader = Loader("input.yaml");
loader.constructor = constructor;
loader.resolver = resolver;
//code from the previous example...
Now, change contents of input.yaml to this:
scalar-red: FF0000
scalar-orange: FFFF00
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
mapping-orange:
!color-mapping
r: 255
g: 255
b: 0
We no longer need to specify the tag for scalar color values. Compile and test the example. If everything went as expected, it should report success.
You can find the complete code in the examples/resolver directory in the D:YAML package.
Representer¶
Now that you know how to load custom data types, it might also be useful to know how to dump them. D:YAML uses the Representer class for this purpose.
Representer processes YAML nodes into plain mapping, sequence or scalar nodes ready for output. Just like with Constructor, this is done by user specified functions. These functions take references to a node to process and to the Representer, and return the processed node.
Representer functions can be added with the addRepresenter() method. The Representer is then passed to Dumper, which dumps YAML documents. Only one representer can be added for a type. This is asserted in addRepresenter() preconditions. By default, the default YAML types already have representer functions, but you can disable them by constructing Representer with the useDefaultRepresenters parameter set to false.
By default, tags are explicitly specified for all non-default types. If you want the tags to be implicit, you can pass a Resolver that will resolve them implicitly. Of course, you will then need to use an identical Resolver when loading the output.
With the following code, we will add support for dumping the our Color type.
Node representColor(ref Node node, Representer representer)
{
//The node is guaranteed to be Color as we add representer for Color.
Color color = node.get!Color;
static immutable hex = "0123456789ABCDEF";
//Using the color format from the Constructor example.
string scalar;
foreach(channel; [color.red, color.green, color.blue])
{
scalar ~= hex[channel / 16];
scalar ~= hex[channel % 16];
}
//Representing as a scalar, with custom tag to specify this data type.
return representer.representScalar("!color", scalar);
}
First we get the Color from the node. Then we convert it to a string with the CSS-like format we’ve used before. Finally, we use the representScalar() method of Representer to get a scalar node ready for output. There are corresponding representMapping() and representSequence() methods as well, with examples in the Resolver API documentation.
Since a type can only have one representer function, we don’t dump Color both in the scalar and mapping formats we’ve used before. However, you can decide to dump the node with different formats/tags in the representer function itself. E.g. you could dump the Color as a mapping based on some arbitrary condition, such as the color being white.
void main()
{
try
{
auto representer = new Representer;
representer.addRepresenter!Color(&representColor);
auto resolver = new Resolver;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto dumper = Dumper("output.yaml");
dumper.representer = representer;
dumper.resolver = resolver;
auto document = Node([Color(255, 0, 0),
Color(0, 255, 0),
Color(0, 0, 255)]);
dumper.dump(document);
}
catch(YAMLException e)
{
writeln(e.msg);
}
}
We construct a new Representer, and specify a representer function for the Color (the template argument) type. We also construct a Resolver, same as in the previous section, so the !color tag will be implicit. Of course, identical Resolver would then have to be used when loading the file. You don’t need to do this if you want the tag to be explicit.
We construct a Dumper to file output.yaml and pass the Representer and Resolver to it. Then, we create a simple node containing a sequence of colors and finally, we dump it.
Source code for this section can be found in the examples/representer directory of the D:YAML package.
