debian-packages/dyaml
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Move custom types to Node (#213)

Browse source 
Move custom types to Node
merged-on-behalf-of: BBasile <BBasile@users.noreply.github.com>
This commit is contained in:
Cameron Ross 2019-01-15 04:07:50 -03:30 committed by The Dlang Bot
parent beb160f1eb
commit 7f913246ea
20 changed files with 1152 additions and 1679 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

166
docs/tutorials/custom_types.md
View file

@ -13,20 +13,8 @@ tags, as we will show later.
## Constructor
D:YAML uses the [Constructor](https://dyaml.dpldocs.info/dyaml.constructor.Constructor.html)
class to process each node to hold data type corresponding to its tag.
*Constructor* stores functions to process each supported tag. These are
supplied by the user using the *addConstructorXXX()* methods, where
*XXX* is *Scalar*, *Sequence* or *Mapping*. *Constructor* is then passed
to *Loader*, which parses YAML input.
Structs and classes must implement the *opCmp()* operator for YAML
support. This is used for duplicate detection in mappings, sorting and
equality comparisons of nodes. The signature of the operator that must
be implemented is `const int opCmp(ref const MyStruct s)` for structs
where *MyStruct* is the struct type, and `int opCmp(Object o)` for
classes. Note that the class *opCmp()* should not alter the compared
values - it is not const for compatibility reasons.
D:YAML supports conversion to user-defined types. Adding a constructor to read
the data from the node is all that is needed.
We will implement support for an RGB color type. It is implemented as
the following struct:
@ -37,31 +25,31 @@ struct Color
ubyte red;
ubyte green;
ubyte blue;
const int opCmp(ref const Color c)
{
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
}
}
```
First, we need a function to construct our data type. The function will
take a reference to *Node* to construct from. The node is guaranteed to
First, we need our type to have an appropriate constructor. The constructor
will take a const *Node* to construct from. The node is guaranteed to
contain either a *string*, an array of *Node* or of *Node.Pair*,
depending on whether we're constructing our value from a scalar,
sequence, or mapping, respectively. If this function throws any
exception, D:YAML handles it and adds its message to a *YAMLException*
that will be thrown when loading the file.
sequence, or mapping, respectively.
In this tutorial, we have functions to construct a color from a scalar,
In this tutorial, we have a constructor to construct a color from a scalar,
using CSS-like format, RRGGBB, or from a mapping, where we use the
following format: {r:RRR, g:GGG, b:BBB} . Code of these functions:
```D
Color constructColorScalar(ref Node node)
this(const Node node, string tag) @safe
{
if (tag == "!color-mapping")
{
//Will throw if a value is missing, is not an integer, or is out of range.
red = node["r"].as!ubyte;
green = node["g"].as!ubyte;
blue = node["b"].as!ubyte;
}
else
{
string value = node.as!string;
@ -88,26 +76,10 @@ Color constructColorScalar(ref Node node)
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
}
Color constructColorMapping(ref Node node)
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
```
@ -138,15 +110,7 @@ void main()
try
{
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto loader = Loader("input.yaml");
loader.constructor = constructor;
auto root = loader.load();
auto root = Loader.fromFile("input.yaml").load();
if(root["scalar-red"].as!Color == red &&
root["mapping-red"].as!Color == red &&
@ -166,10 +130,8 @@ void main()
}
```
First, we create a *Constructor* and pass functions to handle the
`!color` and `!color-mapping` tag. We construct a *Loader* and pass the
*Constructor* to it. We then load the YAML document, and finally, read
the colors to test if they were loaded as expected.
First we load the YAML document, and then have the resulting *Node*s converted
to Colors via their constructor.
You can find the source code for what we've done so far in the
`examples/constructor` directory in the D:YAML package.
@ -194,19 +156,14 @@ values must not be resolvable as any non-string YAML data type.
Add this to your code to add implicit resolution of `!color`.
```D
//code from the previous example...
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
auto resolver = new Resolver;
resolver.addImplicitResolver("!color", regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto loader = Loader("input.yaml");
auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
loader.resolver = resolver;
//code from the previous example...
```
Now, change contents of `input.yaml` to this:
@ -232,62 +189,45 @@ the D:YAML package.
## Representer
Now that you can load custom data types, it might be good to know how to
dump them. D:YAML uses the [Representer](https://dyaml.dpldocs.info/dyaml.representer.Representer.html)
class for this purpose.
dump them.
*Representer* processes YAML nodes into plain mapping, sequence or
scalar nodes ready for output. Just like with *Constructor*, this is
done by user specified functions. These functions take references to a
node to process and to the *Representer*, and return the processed node.
The *Node* struct simply attempts to cast all unrecognized types to *Node*.
This gives each type a consistent and simple way of being represented in a
document. All we need to do is specify a `Node opCast(T: Node)()` method for
any types we wish to support. It is also possible to specify specific styles
for each representation.
Representer functions can be added with the *addRepresenter()* method.
The *Representer* is then passed to *Dumper*, which dumps YAML
documents. Only one function per type can be specified. This is asserted
in *addRepresenter()* preconditions. Default YAML types already have
representer functions specified, but you can disable them by
constructing *Representer* with the *useDefaultRepresenters* parameter
set to false.
By default, tags are explicitly output for all non-default types. To
make dumped tags implicit, you can pass a *Resolver* that will resolve
them implicitly. Of course, you will need to use an identical *Resolver*
when loading the output.
Each type may only have one opCast!Node. Default YAML types are already
supported.
With the following code, we will add support for dumping the our Color
type.
```D
Node representColor(ref Node node, Representer representer)
Node opCast(T: Node)() const
{
//The node is guaranteed to be Color as we add representer for Color.
Color color = node.as!Color;
static immutable hex = "0123456789ABCDEF";
//Using the color format from the Constructor example.
string scalar;
foreach(channel; [color.red, color.green, color.blue])
foreach(channel; [red, green, blue])
{
scalar ~= hex[channel / 16];
scalar ~= hex[channel % 16];
}
//Representing as a scalar, with custom tag to specify this data type.
return representer.representScalar("!color", scalar);
return Node(scalar, "!color");
}
```
First we get the *Color* from the node. Then we convert it to a string
with the CSS-like format we've used before. Finally, we use the
*representScalar()* method of *Representer* to get a scalar node ready
for output. There are corresponding *representMapping()* and
*representSequence()* methods as well, with examples in the [Resolver
API documentation](https://dyaml.dpldocs.info/dyaml.resolver.html).
First we convert the colour data to a string with the CSS-like format we've
used before. Then, we create a scalar *Node* with our desired tag.
Since a type can only have one representer function, we don't dump
Since a type can only have one opCast!Node method, we don't dump
*Color* both in the scalar and mapping formats we've used before.
However, you can decide to dump the node with different formats/tags in
the representer function itself. E.g. you could dump the Color as a
the method itself. E.g. you could dump the Color as a
mapping based on some arbitrary condition, such as the color being
white.
@ -296,17 +236,7 @@ void main()
{
try
{
auto representer = new Representer;
representer.addRepresenter!Color(&representColor);
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto dumper = Dumper("output.yaml");
dumper.representer = representer;
dumper.resolver = resolver;
auto dumper = dumper(File("output.yaml", "w").lockingTextWriter);
auto document = Node([Color(255, 0, 0),
Color(0, 255, 0),
@ -321,16 +251,8 @@ void main()
}
```
We construct a new *Representer*, and specify a representer function for
the *Color* (the template argument) type. We also construct a
*Resolver*, same as in the previous section, so the `!color` tag will be
implicit. Of course, identical *Resolver* would then have to be used
when loading the file. You don't need to do this if you want the tag to
be explicit.
We construct a *Dumper* to file `output.yaml` and pass the *Representer*
and *Resolver* to it. Then, we create a simple node containing a
sequence of colors and finally, we dump it.
We construct a *Dumper* to file `output.yaml`. Then, we create a simple node
containing a sequence of colors and finally, we dump it.
Source code for this section can be found in the `examples/representer`
directory of the D:YAML package.

64
examples/constructor/main.d
View file

@ -8,16 +8,23 @@ struct Color
ubyte green;
ubyte blue;
const int opCmp(ref const Color c)
this(ubyte r, ubyte g, ubyte b) @safe
{
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
}
red = r;
green = g;
blue = b;
}
Color constructColorScalar(ref Node node) @safe
this(const Node node, string tag) @safe
{
if (tag == "!color-mapping")
{
//Will throw if a value is missing, is not an integer, or is out of range.
red = node["r"].as!ubyte;
green = node["g"].as!ubyte;
blue = node["b"].as!ubyte;
}
else
{
string value = node.as!string;
@ -44,44 +51,27 @@ Color constructColorScalar(ref Node node) @safe
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
}
}
}
Color constructColorMapping(ref Node node) @safe
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
void main()
void main(string[] args)
{
auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0);
string path = "input.yaml";
if (args.length > 1)
{
path = args[1];
}
try
{
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
auto root = loader.load();
auto root = Loader.fromFile(path).load();
if(root["scalar-red"].as!Color == red &&
root["mapping-red"].as!Color == red &&

29
examples/representer/main.d
View file

@ -7,49 +7,28 @@ struct Color
ubyte green;
ubyte blue;
const int opCmp(ref const Color c)
Node opCast(T: Node)() const
{
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
}
}
Node representColor(ref Node node, Representer representer) @safe
{
//The node is guaranteed to be Color as we add representer for Color.
Color color = node.as!Color;
static immutable hex = "0123456789ABCDEF";
//Using the color format from the Constructor example.
string scalar;
foreach(channel; [color.red, color.green, color.blue])
foreach(channel; [red, green, blue])
{
scalar ~= hex[channel / 16];
scalar ~= hex[channel % 16];
}
//Representing as a scalar, with custom tag to specify this data type.
return representer.representScalar("!color", scalar);
return Node(scalar, "!color");
}
}
void main()
{
try
{
auto representer = new Representer;
representer.addRepresenter!Color(&representColor);
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto dumper = dumper(File("output.yaml", "w").lockingTextWriter);
dumper.representer = representer;
dumper.resolver = resolver;
auto document = Node([Color(255, 0, 0),
Color(0, 255, 0),

89
examples/resolver/main.d
View file

@ -1,101 +1,31 @@
import std.regex;
import std.stdio;
import std.string;
import dyaml;
struct Color
int main(string[] args)
{
ubyte red;
ubyte green;
ubyte blue;
const int opCmp(ref const Color c)
string path = "input.yaml";
if (args.length > 1)
{
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
path = args[1];
}
}
Color constructColorScalar(ref Node node) @safe
{
string value = node.as!string;
if(value.length != 6)
{
throw new Exception("Invalid color: " ~ value);
}
//We don't need to check for uppercase chars this way.
value = value.toLower();
//Get value of a hex digit.
uint hex(char c)
{
import std.ascii;
if(!std.ascii.isHexDigit(c))
{
throw new Exception("Invalid color: " ~ value);
}
if(std.ascii.isDigit(c))
{
return c - '0';
}
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
}
Color constructColorMapping(ref Node node) @safe
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
void main()
{
auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0);
try
{
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
resolver.addImplicitResolver("!color", regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
loader.resolver = resolver;
auto root = loader.load();
if(root["scalar-red"].as!Color == red &&
root["mapping-red"].as!Color == red &&
root["scalar-orange"].as!Color == orange &&
root["mapping-orange"].as!Color == orange)
if(root["scalar-red"].tag == "!color" &&
root["scalar-orange"].tag == "!color")
{
writeln("SUCCESS");
return;
return 0;
}
}
catch(YAMLException e)
@ -104,4 +34,5 @@ void main()
}
writeln("FAILURE");
return 1;
}

4
examples/yaml_bench/yaml_bench.d
View file

@ -91,7 +91,6 @@ void main(string[] args) //@safe
// Instead of constructing a resolver/constructor with each Loader,
// construct them once to remove noise when profiling.
auto resolver = new Resolver();
auto constructor = new Constructor();
auto constructTime = stopWatch.peek();
@ -113,7 +112,6 @@ void main(string[] args) //@safe
}
loader.resolver = resolver;
loader.constructor = constructor;
nodes = loader.array;
}
void runDumpBenchmark() @safe
@ -150,7 +148,7 @@ void main(string[] args) //@safe
{
writeln("Time to load file: ", loadTime);
}
writeln("Time to set up resolver & constructor: ", constructTime);
writeln("Time to set up resolver: ", constructTime);
}
writeln("Runs: ", runs);
foreach(time, func, enabled; lockstep(totalTime[], only("Loader", "Dumper", "Get"), only(true, dump, get)))

22
source/dyaml/composer.d
View file

@ -46,8 +46,6 @@ final class Composer
Parser parser_;
///Resolver resolving tags (data types).
Resolver resolver_;
///Constructor constructing YAML values.
Constructor constructor_;
///Nodes associated with anchors. Used by YAML aliases.
Node[string] anchors_;
@ -70,13 +68,11 @@ final class Composer
*
* Params: parser = Parser to provide YAML events.
* resolver = Resolver to resolve tags (data types).
* constructor = Constructor to construct nodes.
*/
this(Parser parser, Resolver resolver, Constructor constructor) @safe
this(Parser parser, Resolver resolver) @safe
{
parser_ = parser;
resolver_ = resolver;
constructor_ = constructor;
}
///Destroy the composer.
@ -84,7 +80,6 @@ final class Composer
{
parser_ = null;
resolver_ = null;
constructor_ = null;
anchors_.destroy();
anchors_ = null;
}
@ -222,8 +217,9 @@ final class Composer
const tag = resolver_.resolve(NodeID.scalar, event.tag, event.value,
event.implicit);
Node node = constructor_.node(event.startMark, event.endMark, tag,
event.value, event.scalarStyle);
Node node = constructNode(event.startMark, event.endMark, tag,
event.value);
node.scalarStyle = event.scalarStyle;
return node;
}
@ -248,8 +244,9 @@ final class Composer
nodeAppender.put(composeNode(pairAppenderLevel, nodeAppenderLevel + 1));
}
Node node = constructor_.node(startEvent.startMark, parser_.front.endMark,
tag, nodeAppender.data.dup, startEvent.collectionStyle);
Node node = constructNode(startEvent.startMark, parser_.front.endMark,
tag, nodeAppender.data.dup);
node.collectionStyle = startEvent.collectionStyle;
parser_.popFront();
nodeAppender.clear();
@ -370,8 +367,9 @@ final class Composer
enforce(numUnique == pairAppender.data.length,
new ComposerException("Duplicate key found in mapping", parser_.front.startMark));
Node node = constructor_.node(startEvent.startMark, parser_.front.endMark,
tag, pairAppender.data.dup, startEvent.collectionStyle);
Node node = constructNode(startEvent.startMark, parser_.front.endMark,
tag, pairAppender.data.dup);
node.collectionStyle = startEvent.collectionStyle;
parser_.popFront();
pairAppender.clear();

615
source/dyaml/constructor.d
View file

@ -28,9 +28,10 @@ import dyaml.node;
import dyaml.exception;
import dyaml.style;
package:
// Exception thrown at constructor errors.
package class ConstructorException : YAMLException
class ConstructorException : YAMLException
{
/// Construct a ConstructorException.
///
@ -59,203 +60,6 @@ package class ConstructorException : YAMLException
*
* If a tag is detected with no known constructor function, it is considered an error.
*/
final class Constructor
{
private:
// Constructor functions from scalars.
Node delegate(ref Node) @safe[string] fromScalar_;
// Constructor functions from sequences.
Node delegate(ref Node) @safe[string] fromSequence_;
// Constructor functions from mappings.
Node delegate(ref Node) @safe[string] fromMapping_;
public:
/// Construct a Constructor.
///
/// If you don't want to support default YAML tags/data types, you can use
/// defaultConstructors to disable constructor functions for these.
///
/// Params: defaultConstructors = Use constructors for default YAML tags?
this(const Flag!"useDefaultConstructors" defaultConstructors = Yes.useDefaultConstructors)
@safe nothrow
{
if(!defaultConstructors){return;}
addConstructorScalar("tag:yaml.org,2002:null", &constructNull);
addConstructorScalar("tag:yaml.org,2002:bool", &constructBool);
addConstructorScalar("tag:yaml.org,2002:int", &constructLong);
addConstructorScalar("tag:yaml.org,2002:float", &constructReal);
addConstructorScalar("tag:yaml.org,2002:binary", &constructBinary);
addConstructorScalar("tag:yaml.org,2002:timestamp", &constructTimestamp);
addConstructorScalar("tag:yaml.org,2002:str", &constructString);
///In a mapping, the default value is kept as an entry with the '=' key.
addConstructorScalar("tag:yaml.org,2002:value", &constructString);
addConstructorSequence("tag:yaml.org,2002:omap", &constructOrderedMap);
addConstructorSequence("tag:yaml.org,2002:pairs", &constructPairs);
addConstructorMapping("tag:yaml.org,2002:set", &constructSet);
addConstructorSequence("tag:yaml.org,2002:seq", &constructSequence);
addConstructorMapping("tag:yaml.org,2002:map", &constructMap);
addConstructorScalar("tag:yaml.org,2002:merge", &constructMerge);
}
/** Add a constructor function from scalar.
*
* The function must take a reference to $(D Node) to construct from.
* The node contains a string for scalars, $(D Node[]) for sequences and
* $(D Node.Pair[]) for mappings.
*
* Any exception thrown by this function will be caught by D:YAML and
* its message will be added to a $(D YAMLException) that will also tell
* the user which type failed to construct, and position in the file.
*
*
* The value returned by this function will be stored in the resulting node.
*
* Only one constructor function can be set for one tag.
*
*
* Structs and classes must implement the $(D opCmp()) operator for D:YAML
* support. The signature of the operator that must be implemented
* is $(D const int opCmp(ref const MyStruct s)) for structs where
* $(I MyStruct) is the struct type, and $(D int opCmp(Object o)) for
* classes. Note that the class $(D opCmp()) should not alter the compared
* values - it is not const for compatibility reasons.
*
* Params: tag = Tag for the function to handle.
* ctor = Constructor function.
*/
void addConstructorScalar(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!string)[t] = deleg;
}
///
@safe unittest
{
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructScalar(ref Node node) @safe
{
//Guaranteed to be string as we construct from scalar.
//!mystruct x:y:z
auto parts = node.as!string().split(":");
// If this throws, the D:YAML will handle it and throw a YAMLException.
return MyStruct(to!int(parts[0]), to!int(parts[1]), to!int(parts[2]));
}
import dyaml.loader : Loader;
auto loader = Loader.fromString("!mystruct 12:34:56");
auto constructor = new Constructor;
constructor.addConstructorScalar("!mystruct", &constructMyStructScalar);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(12, 34, 56));
}
/** Add a constructor function from sequence.
*
* See_Also: addConstructorScalar
*/
void addConstructorSequence(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!(Node[]))[t] = deleg;
}
///
@safe unittest
{
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructSequence(ref Node node) @safe
{
//node is guaranteed to be sequence.
//!mystruct [x, y, z]
return MyStruct(node[0].as!int, node[1].as!int, node[2].as!int);
}
import dyaml.loader : Loader;
auto loader = Loader.fromString("!mystruct [1,2,3]");
auto constructor = new Constructor;
constructor.addConstructorSequence("!mystruct", &constructMyStructSequence);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(1, 2, 3));
}
/** Add a constructor function from a mapping.
*
* See_Also: addConstructorScalar
*/
void addConstructorMapping(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!(Node.Pair[]))[t] = deleg;
}
///
@safe unittest {
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructMapping(ref Node node) @safe
{
//node is guaranteed to be mapping.
//!mystruct {"x": x, "y": y, "z": z}
return MyStruct(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
import dyaml.loader : Loader;
auto loader = Loader.fromString(`!mystruct {"x": 11, "y": 22, "z": 33}`);
auto constructor = new Constructor;
constructor.addConstructorMapping("!mystruct", &constructMyStructMapping);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(11, 22, 33));
}
package:
/*
* Construct a node.
*
@ -267,200 +71,137 @@ final class Constructor
*
* Returns: Constructed node.
*/
Node node(T, U)(const Mark start, const Mark end, const string tag,
T value, U style) @safe
if((is(T : string) || is(T == Node[]) || is(T == Node.Pair[])) &&
(is(U : CollectionStyle) || is(U : ScalarStyle)))
Node constructNode(T)(const Mark start, const Mark end, const string tag,
T value) @safe
if((is(T : string) || is(T == Node[]) || is(T == Node.Pair[])))
{
enum type = is(T : string) ? "scalar" :
is(T == Node[]) ? "sequence" :
is(T == Node.Pair[]) ? "mapping" :
"ERROR";
enforce((tag in *delegates!T) !is null,
new ConstructorException("No constructor function from " ~ type ~
" for tag " ~ tag, start, end));
Node node = Node(value);
Node newNode;
try
{
static if(is(U : ScalarStyle))
switch(tag)
{
auto newNode = (*delegates!T)[tag](node);
newNode.startMark_ = start;
newNode.scalarStyle = style;
return newNode;
}
else static if(is(U : CollectionStyle))
case "tag:yaml.org,2002:null":
newNode = Node(YAMLNull(), tag);
break;
case "tag:yaml.org,2002:bool":
static if(is(T == string))
{
auto newNode = (*delegates!T)[tag](node);
newNode.startMark_ = start;
newNode.collectionStyle = style;
return newNode;
newNode = Node(constructBool(value), tag);
break;
}
else throw new Exception("Only scalars can be bools");
case "tag:yaml.org,2002:int":
static if(is(T == string))
{
newNode = Node(constructLong(value), tag);
break;
}
else throw new Exception("Only scalars can be ints");
case "tag:yaml.org,2002:float":
static if(is(T == string))
{
newNode = Node(constructReal(value), tag);
break;
}
else throw new Exception("Only scalars can be floats");
case "tag:yaml.org,2002:binary":
static if(is(T == string))
{
newNode = Node(constructBinary(value), tag);
break;
}
else throw new Exception("Only scalars can be binary data");
case "tag:yaml.org,2002:timestamp":
static if(is(T == string))
{
newNode = Node(constructTimestamp(value), tag);
break;
}
else throw new Exception("Only scalars can be timestamps");
case "tag:yaml.org,2002:str":
static if(is(T == string))
{
newNode = Node(constructString(value), tag);
break;
}
else throw new Exception("Only scalars can be strings");
case "tag:yaml.org,2002:value":
static if(is(T == string))
{
newNode = Node(constructString(value), tag);
break;
}
else throw new Exception("Only scalars can be values");
case "tag:yaml.org,2002:omap":
static if(is(T == Node[]))
{
newNode = Node(constructOrderedMap(value), tag);
break;
}
else throw new Exception("Only sequences can be ordered maps");
case "tag:yaml.org,2002:pairs":
static if(is(T == Node[]))
{
newNode = Node(constructPairs(value), tag);
break;
}
else throw new Exception("Only sequences can be pairs");
case "tag:yaml.org,2002:set":
static if(is(T == Node.Pair[]))
{
newNode = Node(constructSet(value), tag);
break;
}
else throw new Exception("Only mappings can be sets");
case "tag:yaml.org,2002:seq":
static if(is(T == Node[]))
{
newNode = Node(constructSequence(value), tag);
break;
}
else throw new Exception("Only sequences can be sequences");
case "tag:yaml.org,2002:map":
static if(is(T == Node.Pair[]))
{
newNode = Node(constructMap(value), tag);
break;
}
else throw new Exception("Only mappings can be maps");
case "tag:yaml.org,2002:merge":
newNode = Node(YAMLMerge(), tag);
break;
default:
newNode = Node(value, tag);
break;
}
else static assert(false);
}
catch(Exception e)
{
throw new ConstructorException("Error constructing " ~ typeid(T).toString()
~ ":\n" ~ e.msg, start, end);
}
newNode.startMark_ = start;
return newNode;
}
private:
/*
* Add a constructor function.
*
* Params: tag = Tag for the function to handle.
* ctor = Constructor function.
*/
auto addConstructor(T)(const string tag, T function(ref Node) @safe ctor)
{
assert((tag in fromScalar_) is null &&
(tag in fromSequence_) is null &&
(tag in fromMapping_) is null,
"Constructor function for tag " ~ tag ~ " is already " ~
"specified. Can't specify another one.");
return (ref Node n) @safe
{
return Node(ctor(n), tag);
};
}
//Get the array of constructor functions for scalar, sequence or mapping.
@property auto delegates(T)()
{
static if(is(T : string)) {return &fromScalar_;}
else static if(is(T : Node[])) {return &fromSequence_;}
else static if(is(T : Node.Pair[])){return &fromMapping_;}
else static assert(false);
}
}
///Construct a struct from a scalar
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructScalar(ref Node node) @safe
{
// Guaranteed to be string as we construct from scalar.
auto parts = node.as!string().split(":");
return MyStruct(to!int(parts[0]), to!int(parts[1]), to!int(parts[2]));
}
import dyaml.loader : Loader;
string data = "!mystruct 1:2:3";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorScalar("!mystruct", &constructMyStructScalar);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
///Construct a struct from a sequence
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructSequence(ref Node node) @safe
{
// node is guaranteed to be sequence.
return MyStruct(node[0].as!int, node[1].as!int, node[2].as!int);
}
import dyaml.loader : Loader;
string data = "!mystruct [1, 2, 3]";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorSequence("!mystruct", &constructMyStructSequence);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
///Construct a struct from a mapping
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructMapping(ref Node node) @safe
{
// node is guaranteed to be mapping.
return MyStruct(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
import dyaml.loader : Loader;
string data = "!mystruct {x: 1, y: 2, z: 3}";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorMapping("!mystruct", &constructMyStructMapping);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
/// Construct a _null _node.
YAMLNull constructNull(ref Node node) @safe pure nothrow @nogc
{
return YAMLNull();
}
/// Construct a merge _node - a _node that merges another _node into a mapping.
YAMLMerge constructMerge(ref Node node) @safe pure nothrow @nogc
{
return YAMLMerge();
}
/// Construct a boolean _node.
bool constructBool(ref Node node) @safe
// Construct a boolean _node.
bool constructBool(const string str) @safe
{
static yes = ["yes", "true", "on"];
static no = ["no", "false", "off"];
string value = node.as!string().toLower();
string value = str.toLower();
if(yes.canFind(value)){return true;}
if(no.canFind(value)) {return false;}
throw new Exception("Unable to parse boolean value: " ~ value);
}
/// Construct an integer (long) _node.
long constructLong(ref Node node) @safe
// Construct an integer (long) _node.
long constructLong(const string str) @safe
{
string value = node.as!string().replace("_", "");
string value = str.replace("_", "");
const char c = value[0];
const long sign = c != '-' ? 1 : -1;
if(c == '-' || c == '+')
@ -505,12 +246,6 @@ long constructLong(ref Node node) @safe
}
@safe unittest
{
long getLong(string str) @safe
{
auto node = Node(str);
return constructLong(node);
}
string canonical = "685230";
string decimal = "+685_230";
string octal = "02472256";
@ -518,18 +253,18 @@ long constructLong(ref Node node) @safe
string binary = "0b1010_0111_0100_1010_1110";
string sexagesimal = "190:20:30";
assert(685230 == getLong(canonical));
assert(685230 == getLong(decimal));
assert(685230 == getLong(octal));
assert(685230 == getLong(hexadecimal));
assert(685230 == getLong(binary));
assert(685230 == getLong(sexagesimal));
assert(685230 == constructLong(canonical));
assert(685230 == constructLong(decimal));
assert(685230 == constructLong(octal));
assert(685230 == constructLong(hexadecimal));
assert(685230 == constructLong(binary));
assert(685230 == constructLong(sexagesimal));
}
/// Construct a floating point (real) _node.
real constructReal(ref Node node) @safe
// Construct a floating point (real) _node.
real constructReal(const string str) @safe
{
string value = node.as!string().replace("_", "").toLower();
string value = str.replace("_", "").toLower();
const char c = value[0];
const real sign = c != '-' ? 1.0 : -1.0;
if(c == '-' || c == '+')
@ -576,12 +311,6 @@ real constructReal(ref Node node) @safe
return a >= (b - epsilon) && a <= (b + epsilon);
}
real getReal(string str) @safe
{
auto node = Node(str);
return constructReal(node);
}
string canonical = "6.8523015e+5";
string exponential = "685.230_15e+03";
string fixed = "685_230.15";
@ -589,21 +318,20 @@ real constructReal(ref Node node) @safe
string negativeInf = "-.inf";
string NaN = ".NaN";
assert(eq(685230.15, getReal(canonical)));
assert(eq(685230.15, getReal(exponential)));
assert(eq(685230.15, getReal(fixed)));
assert(eq(685230.15, getReal(sexagesimal)));
assert(eq(-real.infinity, getReal(negativeInf)));
assert(to!string(getReal(NaN)) == "nan");
assert(eq(685230.15, constructReal(canonical)));
assert(eq(685230.15, constructReal(exponential)));
assert(eq(685230.15, constructReal(fixed)));
assert(eq(685230.15, constructReal(sexagesimal)));
assert(eq(-real.infinity, constructReal(negativeInf)));
assert(to!string(constructReal(NaN)) == "nan");
}
/// Construct a binary (base64) _node.
ubyte[] constructBinary(ref Node node) @safe
// Construct a binary (base64) _node.
ubyte[] constructBinary(const string value) @safe
{
import std.ascii : newline;
import std.array : array;
string value = node.as!string;
// For an unknown reason, this must be nested to work (compiler bug?).
try
{
@ -621,16 +349,15 @@ ubyte[] constructBinary(ref Node node) @safe
char[] buffer;
buffer.length = 256;
string input = Base64.encode(test, buffer).idup;
auto node = Node(input);
const value = constructBinary(node);
const value = constructBinary(input);
assert(value == test);
assert(value == [84, 104, 101, 32, 65, 110, 115, 119, 101, 114, 58, 32, 52, 50]);
}
/// Construct a timestamp (SysTime) _node.
SysTime constructTimestamp(ref Node node) @safe
// Construct a timestamp (SysTime) _node.
SysTime constructTimestamp(const string str) @safe
{
string value = node.as!string;
string value = str;
auto YMDRegexp = regex("^([0-9][0-9][0-9][0-9])-([0-9][0-9]?)-([0-9][0-9]?)");
auto HMSRegexp = regex("^[Tt \t]+([0-9][0-9]?):([0-9][0-9]):([0-9][0-9])(\\.[0-9]*)?");
@ -704,8 +431,7 @@ SysTime constructTimestamp(ref Node node) @safe
{
string timestamp(string value)
{
auto node = Node(value);
return constructTimestamp(node).toISOString();
return constructTimestamp(value).toISOString();
}
string canonical = "2001-12-15T02:59:43.1Z";
@ -727,18 +453,18 @@ SysTime constructTimestamp(ref Node node) @safe
assert(timestamp(ymd) == "20021214T000000Z");
}
/// Construct a string _node.
string constructString(ref Node node) @safe
// Construct a string _node.
string constructString(const string str) @safe
{
return node.as!string;
return str;
}
/// Convert a sequence of single-element mappings into a sequence of pairs.
Node.Pair[] getPairs(string type, Node[] nodes) @safe
// Convert a sequence of single-element mappings into a sequence of pairs.
Node.Pair[] getPairs(string type, const Node[] nodes) @safe
{
Node.Pair[] pairs;
pairs.reserve(nodes.length);
foreach(ref node; nodes)
foreach(node; nodes)
{
enforce(node.isMapping && node.length == 1,
new Exception("While constructing " ~ type ~
@ -750,10 +476,10 @@ Node.Pair[] getPairs(string type, Node[] nodes) @safe
return pairs;
}
/// Construct an ordered map (ordered sequence of key:value pairs without duplicates) _node.
Node.Pair[] constructOrderedMap(ref Node node) @safe
// Construct an ordered map (ordered sequence of key:value pairs without duplicates) _node.
Node.Pair[] constructOrderedMap(const Node[] nodes) @safe
{
auto pairs = getPairs("ordered map", node.as!(Node[]));
auto pairs = getPairs("ordered map", nodes);
//Detect duplicates.
//TODO this should be replaced by something with deterministic memory allocation.
@ -791,38 +517,30 @@ Node.Pair[] constructOrderedMap(ref Node node) @safe
return pairs;
}
bool hasDuplicates(Node[] nodes) @safe
{
auto node = Node(nodes);
return null !is collectException(constructOrderedMap(node));
assertThrown(constructOrderedMap(alternateTypes(8) ~ alternateTypes(2)));
assertNotThrown(constructOrderedMap(alternateTypes(8)));
assertThrown(constructOrderedMap(sameType(64) ~ sameType(16)));
assertThrown(constructOrderedMap(alternateTypes(64) ~ alternateTypes(16)));
assertNotThrown(constructOrderedMap(sameType(64)));
assertNotThrown(constructOrderedMap(alternateTypes(64)));
}
assert(hasDuplicates(alternateTypes(8) ~ alternateTypes(2)));
assert(!hasDuplicates(alternateTypes(8)));
assert(hasDuplicates(sameType(64) ~ sameType(16)));
assert(hasDuplicates(alternateTypes(64) ~ alternateTypes(16)));
assert(!hasDuplicates(sameType(64)));
assert(!hasDuplicates(alternateTypes(64)));
// Construct a pairs (ordered sequence of key: value pairs allowing duplicates) _node.
Node.Pair[] constructPairs(const Node[] nodes) @safe
{
return getPairs("pairs", nodes);
}
/// Construct a pairs (ordered sequence of key: value pairs allowing duplicates) _node.
Node.Pair[] constructPairs(ref Node node) @safe
// Construct a set _node.
Node[] constructSet(const Node.Pair[] pairs) @safe
{
return getPairs("pairs", node.as!(Node[]));
}
/// Construct a set _node.
Node[] constructSet(ref Node node) @safe
{
auto pairs = node.as!(Node.Pair[]);
// In future, the map here should be replaced with something with deterministic
// memory allocation if possible.
// Detect duplicates.
ubyte[Node] map;
Node[] nodes;
nodes.reserve(pairs.length);
foreach(ref pair; pairs)
foreach(pair; pairs)
{
enforce((pair.key in map) is null, new Exception("Duplicate entry in a set"));
map[pair.key] = 0;
@ -862,27 +580,26 @@ Node[] constructSet(ref Node node) @safe
return true;
}
auto nodeDuplicatesShort = Node(DuplicatesShort.dup);
auto nodeNoDuplicatesShort = Node(noDuplicatesShort.dup);
auto nodeDuplicatesLong = Node(DuplicatesLong.dup);
auto nodeNoDuplicatesLong = Node(noDuplicatesLong.dup);
auto nodeDuplicatesShort = DuplicatesShort.dup;
auto nodeNoDuplicatesShort = noDuplicatesShort.dup;
auto nodeDuplicatesLong = DuplicatesLong.dup;
auto nodeNoDuplicatesLong = noDuplicatesLong.dup;
assert(null !is collectException(constructSet(nodeDuplicatesShort)));
assert(null is collectException(constructSet(nodeNoDuplicatesShort)));
assert(null !is collectException(constructSet(nodeDuplicatesLong)));
assert(null is collectException(constructSet(nodeNoDuplicatesLong)));
assertThrown(constructSet(nodeDuplicatesShort));
assertNotThrown(constructSet(nodeNoDuplicatesShort));
assertThrown(constructSet(nodeDuplicatesLong));
assertNotThrown(constructSet(nodeNoDuplicatesLong));
}
/// Construct a sequence (array) _node.
Node[] constructSequence(ref Node node) @safe
// Construct a sequence (array) _node.
Node[] constructSequence(Node[] nodes) @safe
{
return node.as!(Node[]);
return nodes;
}
/// Construct an unordered map (unordered set of key:value _pairs without duplicates) _node.
Node.Pair[] constructMap(ref Node node) @safe
// Construct an unordered map (unordered set of key:value _pairs without duplicates) _node.
Node.Pair[] constructMap(Node.Pair[] pairs) @safe
{
auto pairs = node.as!(Node.Pair[]);
//Detect duplicates.
//TODO this should be replaced by something with deterministic memory allocation.
auto keys = redBlackTree!Node();

67
source/dyaml/dumper.d
View file

@ -23,6 +23,7 @@ import dyaml.node;
import dyaml.representer;
import dyaml.resolver;
import dyaml.serializer;
import dyaml.style;
import dyaml.tagdirective;
@ -45,8 +46,6 @@ struct Dumper(Range)
private:
//Resolver to resolve tags.
Resolver resolver_;
//Representer to represent data types.
Representer representer_;
//Stream to write to.
Range stream_;
@ -71,7 +70,24 @@ struct Dumper(Range)
//Name of the output file or stream, used in error messages.
string name_ = "<unknown>";
// Default style for scalar nodes.
ScalarStyle defaultScalarStyle_ = ScalarStyle.invalid;
// Default style for collection nodes.
CollectionStyle defaultCollectionStyle_ = CollectionStyle.invalid;
public:
///Set default _style for scalars. If style is $(D ScalarStyle.invalid), the _style is chosen automatically.
@property void defaultScalarStyle(ScalarStyle style) pure @safe nothrow
{
defaultScalarStyle_ = style;
}
///Set default _style for collections. If style is $(D CollectionStyle.invalid), the _style is chosen automatically.
@property void defaultCollectionStyle(CollectionStyle style) pure @safe nothrow
{
defaultCollectionStyle_ = style;
}
@disable this();
@disable bool opEquals(ref Dumper!Range);
@disable int opCmp(ref Dumper!Range);
@ -84,7 +100,6 @@ struct Dumper(Range)
this(Range stream) @safe
{
resolver_ = new Resolver();
representer_ = new Representer();
stream_ = stream;
}
@ -100,12 +115,6 @@ struct Dumper(Range)
resolver_ = resolver;
}
///Specify custom Representer to use.
@property void representer(Representer representer) @safe
{
representer_ = representer;
}
///Write scalars in _canonical form?
@property void canonical(bool canonical) pure @safe nothrow
{
@ -219,7 +228,8 @@ struct Dumper(Range)
explicitEnd_, YAMLVersion_, tags_);
foreach(ref document; documents)
{
representer_.represent(serializer, document);
auto data = representData(document, defaultScalarStyle_, defaultCollectionStyle_);
serializer.serialize(data);
}
}
catch(YAMLException e)
@ -277,18 +287,47 @@ struct Dumper(Range)
auto node = Node([1, 2, 3, 4, 5]);
dumper(stream).dump(node);
}
///Use a custom representer/resolver to support custom data types and/or implicit tags
///Use a custom resolver to support custom data types and/or implicit tags
@safe unittest
{
auto node = Node([1, 2, 3, 4, 5]);
auto representer = new Representer();
auto resolver = new Resolver();
//Add representer functions / resolver expressions here...
auto dumper = dumper(new Appender!string());
dumper.representer = representer;
dumper.resolver = resolver;
dumper.dump(node);
}
/// Set default scalar style
@safe unittest
{
auto stream = new Appender!string();
auto node = Node("Hello world!");
auto dumper = dumper(stream);
dumper.defaultScalarStyle = ScalarStyle.singleQuoted;
dumper.dump(node);
}
/// Set default collection style
@safe unittest
{
auto stream = new Appender!string();
auto node = Node(["Hello", "world!"]);
auto dumper = dumper(stream);
dumper.defaultCollectionStyle = CollectionStyle.flow;
dumper.dump(node);
}
// Make sure the styles are actually used
@safe unittest
{
auto stream = new Appender!string();
auto node = Node([Node("Hello world!"), Node(["Hello", "world!"])]);
auto dumper = dumper(stream);
dumper.defaultScalarStyle = ScalarStyle.singleQuoted;
dumper.defaultCollectionStyle = CollectionStyle.flow;
dumper.explicitEnd = false;
dumper.explicitStart = false;
dumper.YAMLVersion = null;
dumper.dump(node);
assert(stream.data == "[!!str 'Hello world!', [!!str 'Hello', !!str 'world!']]\n");
}
// Explicit document start/end markers
@safe unittest
{

13
source/dyaml/loader.d
View file

@ -41,8 +41,6 @@ struct Loader
Parser parser_;
// Resolves tags (data types).
Resolver resolver_;
// Constructs YAML data types.
Constructor constructor_;
// Name of the input file or stream, used in error messages.
string name_ = "<unknown>";
// Are we done loading?
@ -177,12 +175,6 @@ struct Loader
resolver_ = resolver;
}
/// Specify custom Constructor to use.
void constructor(Constructor constructor) pure @safe nothrow @nogc
{
constructor_ = constructor;
}
/** Load single YAML document.
*
* If none or more than one YAML document is found, this throws a YAMLException.
@ -230,7 +222,7 @@ struct Loader
if (!rangeInitialized)
{
lazyInitConstructorResolver();
composer = new Composer(parser_, resolver_, constructor_);
composer = new Composer(parser_, resolver_);
rangeInitialized = true;
}
assert(!done_, "Loader.popFront called on empty range");
@ -301,7 +293,6 @@ struct Loader
void lazyInitConstructorResolver() @safe
{
if(resolver_ is null) { resolver_ = new Resolver(); }
if(constructor_ is null) { constructor_ = new Constructor(); }
}
}
/// Load single YAML document from a file:
@ -410,13 +401,11 @@ struct Loader
"Hello world!\n"~
"...\n"
);
auto constructor = new Constructor();
auto resolver = new Resolver();
// Add constructor functions / resolver expressions here...
auto loader = Loader.fromFile("example.yaml");
loader.constructor = constructor;
loader.resolver = resolver;
auto rootNode = loader.load();
}

389
source/dyaml/node.d
View file

@ -58,59 +58,6 @@ struct YAMLNull
// Merge YAML type, used to support "tag:yaml.org,2002:merge".
package struct YAMLMerge{}
// Interface for YAMLContainer - used for user defined YAML types.
package interface YAMLObject
{
public:
// Get type of the stored value.
@property TypeInfo type() const pure @safe nothrow;
protected:
// Compare with another YAMLObject.
int cmp(const YAMLObject) const @system;
}
// Stores a user defined YAML data type.
package class YAMLContainer(T) if (!Node.allowed!T): YAMLObject
{
private:
// Stored value.
T value_;
// Construct a YAMLContainer holding specified value.
this(T value) @safe
{
value_ = value;
}
public:
// Get type of the stored value.
@property override TypeInfo type() const pure @safe nothrow
{
return typeid(T);
}
// Get string representation of the container.
override string toString() @system
{
return format("YAMLContainer(%s)", value_);
}
protected:
// Compare with another YAMLObject.
override int cmp(const YAMLObject rhs) const @system
{
const typeCmp = type.opCmp(rhs.type);
if(typeCmp != 0){return typeCmp;}
// Const-casting here as Object opCmp is not const.
T* v1 = cast(T*)&value_;
T* v2 = cast(T*)&((cast(YAMLContainer)rhs).value_);
return (*v1).opCmp(*v2);
}
}
// Key-value pair of YAML nodes, used in mappings.
private struct Pair
{
@ -150,6 +97,20 @@ private struct Pair
@disable int opCmp(ref Pair);
}
enum NodeType
{
null_,
merge,
boolean,
integer,
decimal,
binary,
timestamp,
string,
mapping,
sequence
}
/** YAML node.
*
* This is a pseudo-dynamic type that can store any YAML value, including a
@ -164,9 +125,9 @@ struct Node
package:
// YAML value type.
alias Value = Algebraic!(YAMLNull, YAMLMerge, bool, long, real, ubyte[], SysTime, string,
Node.Pair[], Node[], YAMLObject);
Node.Pair[], Node[]);
// Can Value hold this type without wrapping it in a YAMLObject?
// Can Value hold this type naturally?
enum allowed(T) = isIntegral!T ||
isFloatingPoint!T ||
isSomeString!T ||
@ -212,7 +173,8 @@ struct Node
* be in full form, e.g. "tag:yaml.org,2002:int", not
* a shortcut, like "!!int".
*/
this(T)(T value, const string tag = null)
this(T)(T value, const string tag = null) @safe
if (allowed!T || isArray!T || isAssociativeArray!T || is(Unqual!T == Node) || castableToNode!T)
{
tag_ = tag;
@ -333,15 +295,10 @@ struct Node
{
{
auto node = Node(42);
assert(node.isScalar && !node.isSequence && !node.isMapping && !node.isUserType);
assert(node.isScalar && !node.isSequence && !node.isMapping);
assert(node.as!int == 42 && node.as!float == 42.0f && node.as!string == "42");
assert(!node.isUserType);
}
{
auto node = Node(new class{int a = 5;});
assert(node.isUserType);
}
{
auto node = Node("string");
assert(node.as!string == "string");
@ -351,7 +308,7 @@ struct Node
{
with(Node([1, 2, 3]))
{
assert(!isScalar() && isSequence && !isMapping && !isUserType);
assert(!isScalar() && isSequence && !isMapping);
assert(length == 3);
assert(opIndex(2).as!int == 3);
}
@ -364,7 +321,7 @@ struct Node
aa["2"] = 2;
with(Node(aa))
{
assert(!isScalar() && !isSequence && isMapping && !isUserType);
assert(!isScalar() && !isSequence && isMapping);
assert(length == 2);
assert(opIndex("2").as!int == 2);
}
@ -433,7 +390,7 @@ struct Node
{
with(Node(["1", "2"], [1, 2]))
{
assert(!isScalar() && !isSequence && isMapping && !isUserType);
assert(!isScalar() && !isSequence && isMapping);
assert(length == 2);
assert(opIndex("2").as!int == 2);
}
@ -464,12 +421,6 @@ struct Node
return isType!(Pair[]);
}
/// Is this node a user defined type?
@property bool isUserType() const @safe nothrow
{
return isType!YAMLObject;
}
/// Is this node null?
@property bool isNull() const @safe nothrow
{
@ -552,18 +503,38 @@ struct Node
* the value is out of range of requested type.
*/
inout(T) get(T, Flag!"stringConversion" stringConversion = Yes.stringConversion)() inout
if (allowed!(Unqual!T) || hasNodeConstructor!(Unqual!T))
{
if(isType!(Unqual!T)){return getValue!T;}
/// Must go before others, as even string/int/etc could be stored in a YAMLObject.
static if(!allowed!(Unqual!T)) if(isUserType)
static if(!allowed!(Unqual!T))
{
auto object = getValue!YAMLObject();
enforce(object.type is typeid(T),
new NodeException("Node has unexpected type: " ~ object.type.toString() ~
". Expected: " ~ typeid(T).toString, startMark_));
return (cast(inout YAMLContainer!(Unqual!T))(object)).value_;
static if (hasSimpleNodeConstructor!T)
{
alias params = AliasSeq!(this);
}
else static if (hasExpandedNodeConstructor!T)
{
alias params = AliasSeq!(this, tag_);
}
else
{
static assert(0, "Unknown Node constructor?");
}
static if (is(T == class))
{
return new inout T(params);
}
else static if (is(T == struct))
{
return T(params);
}
else
{
static assert(0, "Unhandled user type");
}
} else {
// If we're getting from a mapping and we're not getting Node.Pair[],
// we're getting the default value.
@ -611,6 +582,7 @@ struct Node
else throw new NodeException("Node stores unexpected type: " ~ type.toString() ~
". Expected: " ~ typeid(T).toString, startMark_);
}
}
/// Automatic type conversion
@safe unittest
{
@ -620,6 +592,186 @@ struct Node
assert(node.get!string == "42");
assert(node.get!double == 42.0);
}
/// Scalar node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
this(int x, int y, int z) @safe
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
auto parts = node.as!string().split(":");
x = parts[0].to!int;
y = parts[1].to!int;
z = parts[2].to!int;
}
Node opCast(T: Node)() @safe
{
//Using custom scalar format, x:y:z.
auto scalar = format("%s:%s:%s", x, y, z);
//Representing as a scalar, with custom tag to specify this data type.
return Node(scalar, "!mystruct.tag");
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Sequence node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
this(int x, int y, int z) @safe
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
x = node[0].as!int;
y = node[1].as!int;
z = node[2].as!int;
}
Node opCast(T: Node)()
{
return Node([x, y, z], "!mystruct.tag");
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Mapping node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
Node opCast(T: Node)()
{
auto pairs = [Node.Pair("x", x),
Node.Pair("y", y),
Node.Pair("z", z)];
return Node(pairs, "!mystruct.tag");
}
this(int x, int y, int z)
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
x = node["x"].as!int;
y = node["y"].as!int;
z = node["z"].as!int;
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Classes can be used too
@system unittest {
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static class MyClass
{
int x, y, z;
this(int x, int y, int z)
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe inout
{
auto parts = node.as!string().split(":");
x = parts[0].to!int;
y = parts[1].to!int;
z = parts[2].to!int;
}
///Useful for Node.as!string.
override string toString()
{
return format("MyClass(%s, %s, %s)", x, y, z);
}
Node opCast(T: Node)() @safe
{
//Using custom scalar format, x:y:z.
auto scalar = format("%s:%s:%s", x, y, z);
//Representing as a scalar, with custom tag to specify this data type.
return Node(scalar, "!myclass.tag");
}
override bool opEquals(Object o)
{
if (auto other = cast(MyClass)o)
{
return (other.x == x) && (other.y == y) && (other.z == z);
}
return false;
}
}
auto appender = new Appender!string;
// Dump class to yaml document
dumper(appender).dump(Node(new MyClass(1,2,3)));
// Read yaml document back as a MyClass
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyClass == new MyClass(1,2,3));
}
@safe unittest
{
assertThrown!NodeException(Node("42").get!int);
@ -1876,10 +2028,6 @@ struct Node
const t2 = rhs.getValue!SysTime;
return cmp(t1, t2);
}
else if(isUserType)
{
return getValue!YAMLObject.cmp(rhs.getValue!YAMLObject);
}
assert(false, "Unknown type of node for comparison : " ~ type.toString());
}
@ -1923,12 +2071,58 @@ struct Node
assert(false);
}
// Get type of the node value (YAMLObject for user types).
// Get type of the node value.
@property TypeInfo type() const @safe nothrow
{
return value_.type;
}
// Get type of the node value.
@property NodeType newType() const @safe nothrow
{
if (value_.type is typeid(bool))
{
return NodeType.boolean;
}
else if (value_.type is typeid(long))
{
return NodeType.integer;
}
else if (value_.type is typeid(Node[]))
{
return NodeType.sequence;
}
else if (value_.type is typeid(ubyte[]))
{
return NodeType.binary;
}
else if (value_.type is typeid(string))
{
return NodeType.string;
}
else if (value_.type is typeid(Node.Pair[]))
{
return NodeType.mapping;
}
else if (value_.type is typeid(SysTime))
{
return NodeType.timestamp;
}
else if (value_.type is typeid(YAMLNull))
{
return NodeType.null_;
}
else if (value_.type is typeid(YAMLMerge))
{
return NodeType.merge;
}
else if (value_.type is typeid(real))
{
return NodeType.decimal;
}
else assert(0, text(value_.type));
}
public:
// Determine if the value stored by the node is of specified type.
//
@ -2210,7 +2404,7 @@ struct Node
}
else
{
value_ = cast(YAMLObject)new YAMLContainer!T(value);
value_ = (cast(Node)value).value_;
}
}
}
@ -2232,3 +2426,30 @@ void merge(ref Appender!(Node.Pair[]) pairs, Node.Pair[] toMerge) @safe
pairs.put(pair);
}
}
enum hasNodeConstructor(T) = hasSimpleNodeConstructor!T || hasExpandedNodeConstructor!T;
template hasSimpleNodeConstructor(T)
{
static if (is(T == struct))
{
enum hasSimpleNodeConstructor = is(typeof(T(Node.init)));
}
else static if (is(T == class))
{
enum hasSimpleNodeConstructor = is(typeof(new inout T(Node.init)));
}
else enum hasSimpleNodeConstructor = false;
}
template hasExpandedNodeConstructor(T)
{
static if (is(T == struct))
{
enum hasExpandedNodeConstructor = is(typeof(T(Node.init, "")));
}
else static if (is(T == class))
{
enum hasExpandedNodeConstructor = is(typeof(new inout T(Node.init, "")));
}
else enum hasExpandedNodeConstructor = false;
}
enum castableToNode(T) = (is(T == struct) || is(T == class)) && is(typeof(T.opCast!Node()) : Node);

2
source/dyaml/package.d
View file

@ -5,13 +5,11 @@
module dyaml;
public import dyaml.constructor;
public import dyaml.dumper;
public import dyaml.encoding;
public import dyaml.exception;
public import dyaml.linebreak;
public import dyaml.loader;
public import dyaml.representer;
public import dyaml.resolver;
public import dyaml.style;
public import dyaml.node;

898
source/dyaml/representer.d
View file

File diff suppressed because it is too large Load diff

83
source/dyaml/scanner.d
View file

@ -1951,91 +1951,36 @@ final class Scanner
/// In case of an error, error_ is set. Use throwIfError() to handle this.
void scanURIEscapesToSlice(string name)(const Mark startMark)
{
import core.exception : UnicodeException;
// URI escapes encode a UTF-8 string. We store UTF-8 code units here for
// decoding into UTF-32.
char[4] bytes;
size_t bytesUsed;
Appender!string buffer;
// Get one dchar by decoding data from bytes.
//
// This is probably slow, but simple and URI escapes are extremely uncommon
// in YAML.
//
// Returns the number of bytes used by the dchar in bytes on success,
// size_t.max on failure.
static size_t getDchar(char[] bytes, Reader reader_) @safe
{
size_t nextChar;
dchar c;
if(bytes[0] < 0x80)
{
c = bytes[0];
++nextChar;
}
else
{
try
{
c = decode(bytes[], nextChar);
}
catch (UTFException)
{
return size_t.max;
}
}
reader_.sliceBuilder.write(c);
if(bytes.length - nextChar > 0)
{
copy(bytes[nextChar..bytes.length], bytes[0..bytes.length-nextChar]);
}
return bytes.length - nextChar;
}
enum contextMsg = "While scanning a " ~ name;
while(reader_.peekByte() == '%')
{
reader_.forward();
if(bytesUsed == bytes.length)
{
bytesUsed = getDchar(bytes[], reader_);
if(bytesUsed == size_t.max)
{
error(contextMsg, startMark,
"Invalid UTF-8 data encoded in URI escape sequence",
reader_.mark);
return;
}
}
char b = 0;
uint mult = 16;
// Converting 2 hexadecimal digits to a byte.
foreach(k; 0 .. 2)
{
const dchar c = reader_.peek(k);
if(!c.isHexDigit)
char[2] nextByte = [reader_.peekByte(), reader_.peekByte(1)];
if(!nextByte[0].isHexDigit || !nextByte[1].isHexDigit)
{
auto msg = expected("URI escape sequence of 2 hexadecimal " ~
"numbers", c);
"numbers", nextByte);
error(contextMsg, startMark, msg, reader_.mark);
return;
}
uint digit;
if(c - '0' < 10) { digit = c - '0'; }
else if(c - 'A' < 6) { digit = 10 + c - 'A'; }
else if(c - 'a' < 6) { digit = 10 + c - 'a'; }
else { assert(false); }
b += mult * digit;
mult /= 16;
}
bytes[bytesUsed++] = b;
buffer ~= nextByte[].to!ubyte(16);
reader_.forward(2);
}
bytesUsed = getDchar(bytes[0 .. bytesUsed], reader_);
if(bytesUsed == size_t.max)
try
{
foreach (dchar chr; buffer.data)
{
reader_.sliceBuilder.write(chr);
}
}
catch (UnicodeException)
{
error(contextMsg, startMark,
"Invalid UTF-8 data encoded in URI escape sequence",

59
source/dyaml/test/constructor.d
View file

@ -328,21 +328,12 @@ class TestClass
this.z = z;
}
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
override int opCmp(Object o) @safe
Node opCast(T: Node)() @safe
{
TestClass s = cast(TestClass)o;
if(s is null){return -1;}
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
override string toString() @safe
{
return format("TestClass(", x, ", ", y, ", ", z, ")");
auto pairs = [Node.Pair("x", x),
Node.Pair("y", y),
Node.Pair("z", z)];
return Node(pairs, "!tag1");
}
}
@ -351,43 +342,22 @@ struct TestStruct
{
int value;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const TestStruct s) const @safe
this (int x) @safe
{
return value - s.value;
}
}
///Constructor function for TestClass.
TestClass constructClass(ref Node node) @safe
{
return new TestClass(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
Node representClass(ref Node node, Representer representer) @safe
{
auto value = node.as!TestClass;
auto pairs = [Node.Pair("x", value.x),
Node.Pair("y", value.y),
Node.Pair("z", value.z)];
auto result = representer.representMapping("!tag1", pairs);
return result;
value = x;
}
///Constructor function for TestStruct.
TestStruct constructStruct(ref Node node) @safe
this(ref Node node) @safe
{
return TestStruct(to!int(node.as!string));
value = node.as!string.to!int;
}
///Representer function for TestStruct.
Node representStruct(ref Node node, Representer representer) @safe
Node opCast(T: Node)() @safe
{
string[] keys, values;
auto value = node.as!TestStruct;
return representer.representScalar("!tag2", to!string(value.value));
return Node(value.to!string, "!tag2");
}
}
/**
@ -403,12 +373,7 @@ void testConstructor(string dataFilename, string codeDummy) @safe
enforce((base in expected) !is null,
new Exception("Unimplemented constructor test: " ~ base));
auto constructor = new Constructor;
constructor.addConstructorMapping("!tag1", &constructClass);
constructor.addConstructorScalar("!tag2", &constructStruct);
auto loader = Loader.fromFile(dataFilename);
loader.constructor = constructor;
loader.resolver = new Resolver;
Node[] exp = expected[base];

3
source/dyaml/test/emitter.d
View file

@ -90,7 +90,6 @@ void testEmitterOnData(string dataFilename, string canonicalFilename) @safe
auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver;
auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents));
@ -119,7 +118,6 @@ void testEmitterOnCanonical(string canonicalFilename) @safe
}
auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver;
auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents));
@ -177,7 +175,6 @@ void testEmitterStyles(string dataFilename, string canonicalFilename) @safe
}
auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver;
auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents));

8
source/dyaml/test/representer.d
View file

@ -53,21 +53,13 @@ void testRepresenterTypes(string codeFilename) @safe
}
auto emitStream = new Appender!(immutable(encoding)[]);
auto representer = new Representer;
representer.addRepresenter!TestClass(&representClass);
representer.addRepresenter!TestStruct(&representStruct);
auto dumper = dumper(emitStream);
dumper.representer = representer;
dumper.dump!encoding(expectedNodes);
output = emitStream.data;
auto constructor = new Constructor;
constructor.addConstructorMapping("!tag1", &constructClass);
constructor.addConstructorScalar("!tag2", &constructStruct);
auto loader = Loader.fromString(emitStream.data.toUTF8);
loader.name = "TEST";
loader.constructor = constructor;
readNodes = loader.array;
assert(expectedNodes.length == readNodes.length);

13
source/dyaml/test/resolver.d
View file

@ -16,12 +16,6 @@ import std.string;
import dyaml.test.common;
string construct(ref Node node) @safe
{
return node.as!string;
}
/**
* Implicit tag resolution unittest.
*
@ -44,12 +38,7 @@ void testImplicitResolver(string dataFilename, string detectFilename) @safe
correctTag = readText(detectFilename).strip();
auto constructor = new Constructor;
constructor.addConstructorScalar("tag:example.com,2000:app/tag🤔", &construct);
auto loader = Loader.fromFile(dataFilename);
loader.constructor = constructor;
node = loader.load();
node = Loader.fromFile(dataFilename).load();
assert(node.isSequence);
foreach(ref Node scalar; node)
{

1
test/data/empty-python-module.loader-error
View file

@ -1 +0,0 @@
--- !!python:module:

1
test/data/empty-python-name.loader-error
View file

@ -1 +0,0 @@
--- !!python/name: empty

1
test/data/undefined-constructor.loader-error
View file

@ -1 +0,0 @@
--- !foo bar