debian-packages/dyaml
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Move custom types to Node (#213)

Browse source 
Move custom types to Node
merged-on-behalf-of: BBasile <BBasile@users.noreply.github.com>
This commit is contained in:
Cameron Ross 2019-01-15 04:07:50 -03:30 committed by The Dlang Bot
parent beb160f1eb
commit 7f913246ea
20 changed files with 1152 additions and 1679 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

166
docs/tutorials/custom_types.md
View file

@ -13,20 +13,8 @@ tags, as we will show later.
## Constructor ## Constructor
D:YAML uses the [Constructor](https://dyaml.dpldocs.info/dyaml.constructor.Constructor.html) D:YAML supports conversion to user-defined types. Adding a constructor to read
class to process each node to hold data type corresponding to its tag. the data from the node is all that is needed.
*Constructor* stores functions to process each supported tag. These are
supplied by the user using the *addConstructorXXX()* methods, where
*XXX* is *Scalar*, *Sequence* or *Mapping*. *Constructor* is then passed
to *Loader*, which parses YAML input.
Structs and classes must implement the *opCmp()* operator for YAML
support. This is used for duplicate detection in mappings, sorting and
equality comparisons of nodes. The signature of the operator that must
be implemented is `const int opCmp(ref const MyStruct s)` for structs
where *MyStruct* is the struct type, and `int opCmp(Object o)` for
classes. Note that the class *opCmp()* should not alter the compared
values - it is not const for compatibility reasons.
We will implement support for an RGB color type. It is implemented as We will implement support for an RGB color type. It is implemented as
the following struct: the following struct:
@ -37,31 +25,31 @@ struct Color
ubyte red; ubyte red;
ubyte green; ubyte green;
ubyte blue; ubyte blue;
const int opCmp(ref const Color c)
{
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
}
} }
``` ```
First, we need a function to construct our data type. The function will First, we need our type to have an appropriate constructor. The constructor
take a reference to *Node* to construct from. The node is guaranteed to will take a const *Node* to construct from. The node is guaranteed to
contain either a *string*, an array of *Node* or of *Node.Pair*, contain either a *string*, an array of *Node* or of *Node.Pair*,
depending on whether we're constructing our value from a scalar, depending on whether we're constructing our value from a scalar,
sequence, or mapping, respectively. If this function throws any sequence, or mapping, respectively.
exception, D:YAML handles it and adds its message to a *YAMLException*
that will be thrown when loading the file.
In this tutorial, we have functions to construct a color from a scalar, In this tutorial, we have a constructor to construct a color from a scalar,
using CSS-like format, RRGGBB, or from a mapping, where we use the using CSS-like format, RRGGBB, or from a mapping, where we use the
following format: {r:RRR, g:GGG, b:BBB} . Code of these functions: following format: {r:RRR, g:GGG, b:BBB} . Code of these functions:
```D ```D
Color constructColorScalar(ref Node node)
this(const Node node, string tag) @safe
{
if (tag == "!color-mapping")
{
//Will throw if a value is missing, is not an integer, or is out of range.
red = node["r"].as!ubyte;
green = node["g"].as!ubyte;
blue = node["b"].as!ubyte;
}
else
{ {
string value = node.as!string; string value = node.as!string;
@ -88,26 +76,10 @@ Color constructColorScalar(ref Node node)
return c - 'a' + 10; return c - 'a' + 10;
} }
Color result; red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1])); green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3])); blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
} }
Color constructColorMapping(ref Node node)
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
} }
``` ```
@ -138,15 +110,7 @@ void main()
try try
{ {
auto constructor = new Constructor; auto root = Loader.fromFile("input.yaml").load();
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto loader = Loader("input.yaml");
loader.constructor = constructor;
auto root = loader.load();
if(root["scalar-red"].as!Color == red && if(root["scalar-red"].as!Color == red &&
root["mapping-red"].as!Color == red && root["mapping-red"].as!Color == red &&
@ -166,10 +130,8 @@ void main()
} }
``` ```
First, we create a *Constructor* and pass functions to handle the First we load the YAML document, and then have the resulting *Node*s converted
`!color` and `!color-mapping` tag. We construct a *Loader* and pass the to Colors via their constructor.
*Constructor* to it. We then load the YAML document, and finally, read
the colors to test if they were loaded as expected.
You can find the source code for what we've done so far in the You can find the source code for what we've done so far in the
`examples/constructor` directory in the D:YAML package. `examples/constructor` directory in the D:YAML package.
@ -194,19 +156,14 @@ values must not be resolvable as any non-string YAML data type.
Add this to your code to add implicit resolution of `!color`. Add this to your code to add implicit resolution of `!color`.
```D ```D
//code from the previous example...
auto resolver = new Resolver;
import std.regex; import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"), auto resolver = new Resolver;
resolver.addImplicitResolver("!color", regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF"); "0123456789abcdefABCDEF");
auto loader = Loader("input.yaml"); auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
loader.resolver = resolver; loader.resolver = resolver;
//code from the previous example...
``` ```
Now, change contents of `input.yaml` to this: Now, change contents of `input.yaml` to this:
@ -232,62 +189,45 @@ the D:YAML package.
## Representer ## Representer
Now that you can load custom data types, it might be good to know how to Now that you can load custom data types, it might be good to know how to
dump them. D:YAML uses the [Representer](https://dyaml.dpldocs.info/dyaml.representer.Representer.html) dump them.
class for this purpose.
*Representer* processes YAML nodes into plain mapping, sequence or The *Node* struct simply attempts to cast all unrecognized types to *Node*.
scalar nodes ready for output. Just like with *Constructor*, this is This gives each type a consistent and simple way of being represented in a
done by user specified functions. These functions take references to a document. All we need to do is specify a `Node opCast(T: Node)()` method for
node to process and to the *Representer*, and return the processed node. any types we wish to support. It is also possible to specify specific styles
for each representation.
Representer functions can be added with the *addRepresenter()* method. Each type may only have one opCast!Node. Default YAML types are already
The *Representer* is then passed to *Dumper*, which dumps YAML supported.
documents. Only one function per type can be specified. This is asserted
in *addRepresenter()* preconditions. Default YAML types already have
representer functions specified, but you can disable them by
constructing *Representer* with the *useDefaultRepresenters* parameter
set to false.
By default, tags are explicitly output for all non-default types. To
make dumped tags implicit, you can pass a *Resolver* that will resolve
them implicitly. Of course, you will need to use an identical *Resolver*
when loading the output.
With the following code, we will add support for dumping the our Color With the following code, we will add support for dumping the our Color
type. type.
```D ```D
Node representColor(ref Node node, Representer representer) Node opCast(T: Node)() const
{ {
//The node is guaranteed to be Color as we add representer for Color.
Color color = node.as!Color;
static immutable hex = "0123456789ABCDEF"; static immutable hex = "0123456789ABCDEF";
//Using the color format from the Constructor example. //Using the color format from the Constructor example.
string scalar; string scalar;
foreach(channel; [color.red, color.green, color.blue]) foreach(channel; [red, green, blue])
{ {
scalar ~= hex[channel / 16]; scalar ~= hex[channel / 16];
scalar ~= hex[channel % 16]; scalar ~= hex[channel % 16];
} }
//Representing as a scalar, with custom tag to specify this data type. //Representing as a scalar, with custom tag to specify this data type.
return representer.representScalar("!color", scalar); return Node(scalar, "!color");
} }
``` ```
First we get the *Color* from the node. Then we convert it to a string First we convert the colour data to a string with the CSS-like format we've
with the CSS-like format we've used before. Finally, we use the used before. Then, we create a scalar *Node* with our desired tag.
*representScalar()* method of *Representer* to get a scalar node ready
for output. There are corresponding *representMapping()* and
*representSequence()* methods as well, with examples in the [Resolver
API documentation](https://dyaml.dpldocs.info/dyaml.resolver.html).
Since a type can only have one representer function, we don't dump Since a type can only have one opCast!Node method, we don't dump
*Color* both in the scalar and mapping formats we've used before. *Color* both in the scalar and mapping formats we've used before.
However, you can decide to dump the node with different formats/tags in However, you can decide to dump the node with different formats/tags in
the representer function itself. E.g. you could dump the Color as a the method itself. E.g. you could dump the Color as a
mapping based on some arbitrary condition, such as the color being mapping based on some arbitrary condition, such as the color being
white. white.
@ -296,17 +236,7 @@ void main()
{ {
try try
{ {
auto representer = new Representer; auto dumper = dumper(File("output.yaml", "w").lockingTextWriter);
representer.addRepresenter!Color(&representColor);
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto dumper = Dumper("output.yaml");
dumper.representer = representer;
dumper.resolver = resolver;
auto document = Node([Color(255, 0, 0), auto document = Node([Color(255, 0, 0),
Color(0, 255, 0), Color(0, 255, 0),
@ -321,16 +251,8 @@ void main()
} }
``` ```
We construct a new *Representer*, and specify a representer function for We construct a *Dumper* to file `output.yaml`. Then, we create a simple node
the *Color* (the template argument) type. We also construct a containing a sequence of colors and finally, we dump it.
*Resolver*, same as in the previous section, so the `!color` tag will be
implicit. Of course, identical *Resolver* would then have to be used
when loading the file. You don't need to do this if you want the tag to
be explicit.
We construct a *Dumper* to file `output.yaml` and pass the *Representer*
and *Resolver* to it. Then, we create a simple node containing a
sequence of colors and finally, we dump it.
Source code for this section can be found in the `examples/representer` Source code for this section can be found in the `examples/representer`
directory of the D:YAML package. directory of the D:YAML package.

64
examples/constructor/main.d
View file

@ -8,16 +8,23 @@ struct Color
ubyte green; ubyte green;
ubyte blue; ubyte blue;
const int opCmp(ref const Color c) this(ubyte r, ubyte g, ubyte b) @safe
{ {
if(red != c.red) {return red - c.red;} red = r;
if(green != c.green){return green - c.green;} green = g;
if(blue != c.blue) {return blue - c.blue;} blue = b;
return 0;
}
} }
Color constructColorScalar(ref Node node) @safe this(const Node node, string tag) @safe
{
if (tag == "!color-mapping")
{
//Will throw if a value is missing, is not an integer, or is out of range.
red = node["r"].as!ubyte;
green = node["g"].as!ubyte;
blue = node["b"].as!ubyte;
}
else
{ {
string value = node.as!string; string value = node.as!string;
@ -44,44 +51,27 @@ Color constructColorScalar(ref Node node) @safe
return c - 'a' + 10; return c - 'a' + 10;
} }
Color result; red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1])); green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3])); blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5])); }
}
return result;
} }
Color constructColorMapping(ref Node node) @safe void main(string[] args)
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
void main()
{ {
auto red = Color(255, 0, 0); auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0); auto orange = Color(255, 255, 0);
string path = "input.yaml";
if (args.length > 1)
{
path = args[1];
}
try try
{ {
auto constructor = new Constructor; auto root = Loader.fromFile(path).load();
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
auto root = loader.load();
if(root["scalar-red"].as!Color == red && if(root["scalar-red"].as!Color == red &&
root["mapping-red"].as!Color == red && root["mapping-red"].as!Color == red &&

29
examples/representer/main.d
View file

@ -7,49 +7,28 @@ struct Color
ubyte green; ubyte green;
ubyte blue; ubyte blue;
const int opCmp(ref const Color c) Node opCast(T: Node)() const
{ {
if(red != c.red) {return red - c.red;}
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
}
}
Node representColor(ref Node node, Representer representer) @safe
{
//The node is guaranteed to be Color as we add representer for Color.
Color color = node.as!Color;
static immutable hex = "0123456789ABCDEF"; static immutable hex = "0123456789ABCDEF";
//Using the color format from the Constructor example. //Using the color format from the Constructor example.
string scalar; string scalar;
foreach(channel; [color.red, color.green, color.blue]) foreach(channel; [red, green, blue])
{ {
scalar ~= hex[channel / 16]; scalar ~= hex[channel / 16];
scalar ~= hex[channel % 16]; scalar ~= hex[channel % 16];
} }
//Representing as a scalar, with custom tag to specify this data type. //Representing as a scalar, with custom tag to specify this data type.
return representer.representScalar("!color", scalar); return Node(scalar, "!color");
}
} }
void main() void main()
{ {
try try
{ {
auto representer = new Representer;
representer.addRepresenter!Color(&representColor);
auto resolver = new Resolver;
import std.regex;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF");
auto dumper = dumper(File("output.yaml", "w").lockingTextWriter); auto dumper = dumper(File("output.yaml", "w").lockingTextWriter);
dumper.representer = representer;
dumper.resolver = resolver;
auto document = Node([Color(255, 0, 0), auto document = Node([Color(255, 0, 0),
Color(0, 255, 0), Color(0, 255, 0),

89
examples/resolver/main.d
View file

@ -1,101 +1,31 @@
import std.regex;
import std.stdio; import std.stdio;
import std.string;
import dyaml; import dyaml;
struct Color int main(string[] args)
{ {
ubyte red; string path = "input.yaml";
ubyte green; if (args.length > 1)
ubyte blue;
const int opCmp(ref const Color c)
{ {
if(red != c.red) {return red - c.red;} path = args[1];
if(green != c.green){return green - c.green;}
if(blue != c.blue) {return blue - c.blue;}
return 0;
} }
}
Color constructColorScalar(ref Node node) @safe
{
string value = node.as!string;
if(value.length != 6)
{
throw new Exception("Invalid color: " ~ value);
}
//We don't need to check for uppercase chars this way.
value = value.toLower();
//Get value of a hex digit.
uint hex(char c)
{
import std.ascii;
if(!std.ascii.isHexDigit(c))
{
throw new Exception("Invalid color: " ~ value);
}
if(std.ascii.isDigit(c))
{
return c - '0';
}
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
}
Color constructColorMapping(ref Node node) @safe
{
ubyte r,g,b;
//Might throw if a value is missing is not an integer, or is out of range.
//If this happens, D:YAML will handle the exception and use its message
//in a YAMLException thrown when loading.
r = node["r"].as!ubyte;
g = node["g"].as!ubyte;
b = node["b"].as!ubyte;
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
void main()
{
auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0);
try try
{ {
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructorScalar("!color", &constructColorScalar);
constructor.addConstructorMapping("!color-mapping", &constructColorMapping);
auto resolver = new Resolver; auto resolver = new Resolver;
import std.regex; resolver.addImplicitResolver("!color", regex("[0-9a-fA-F]{6}"),
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}"),
"0123456789abcdefABCDEF"); "0123456789abcdefABCDEF");
auto loader = Loader.fromFile("input.yaml"); auto loader = Loader.fromFile("input.yaml");
loader.constructor = constructor;
loader.resolver = resolver; loader.resolver = resolver;
auto root = loader.load(); auto root = loader.load();
if(root["scalar-red"].as!Color == red && if(root["scalar-red"].tag == "!color" &&
root["mapping-red"].as!Color == red && root["scalar-orange"].tag == "!color")
root["scalar-orange"].as!Color == orange &&
root["mapping-orange"].as!Color == orange)
{ {
writeln("SUCCESS"); writeln("SUCCESS");
return; return 0;
} }
} }
catch(YAMLException e) catch(YAMLException e)
@ -104,4 +34,5 @@ void main()
} }
writeln("FAILURE"); writeln("FAILURE");
return 1;
} }

4
examples/yaml_bench/yaml_bench.d
View file

@ -91,7 +91,6 @@ void main(string[] args) //@safe
// Instead of constructing a resolver/constructor with each Loader, // Instead of constructing a resolver/constructor with each Loader,
// construct them once to remove noise when profiling. // construct them once to remove noise when profiling.
auto resolver = new Resolver(); auto resolver = new Resolver();
auto constructor = new Constructor();
auto constructTime = stopWatch.peek(); auto constructTime = stopWatch.peek();
@ -113,7 +112,6 @@ void main(string[] args) //@safe
} }
loader.resolver = resolver; loader.resolver = resolver;
loader.constructor = constructor;
nodes = loader.array; nodes = loader.array;
} }
void runDumpBenchmark() @safe void runDumpBenchmark() @safe
@ -150,7 +148,7 @@ void main(string[] args) //@safe
{ {
writeln("Time to load file: ", loadTime); writeln("Time to load file: ", loadTime);
} }
writeln("Time to set up resolver & constructor: ", constructTime); writeln("Time to set up resolver: ", constructTime);
} }
writeln("Runs: ", runs); writeln("Runs: ", runs);
foreach(time, func, enabled; lockstep(totalTime[], only("Loader", "Dumper", "Get"), only(true, dump, get))) foreach(time, func, enabled; lockstep(totalTime[], only("Loader", "Dumper", "Get"), only(true, dump, get)))

22
source/dyaml/composer.d
View file

@ -46,8 +46,6 @@ final class Composer
Parser parser_; Parser parser_;
///Resolver resolving tags (data types). ///Resolver resolving tags (data types).
Resolver resolver_; Resolver resolver_;
///Constructor constructing YAML values.
Constructor constructor_;
///Nodes associated with anchors. Used by YAML aliases. ///Nodes associated with anchors. Used by YAML aliases.
Node[string] anchors_; Node[string] anchors_;
@ -70,13 +68,11 @@ final class Composer
* *
* Params: parser = Parser to provide YAML events. * Params: parser = Parser to provide YAML events.
* resolver = Resolver to resolve tags (data types). * resolver = Resolver to resolve tags (data types).
* constructor = Constructor to construct nodes.
*/ */
this(Parser parser, Resolver resolver, Constructor constructor) @safe this(Parser parser, Resolver resolver) @safe
{ {
parser_ = parser; parser_ = parser;
resolver_ = resolver; resolver_ = resolver;
constructor_ = constructor;
} }
///Destroy the composer. ///Destroy the composer.
@ -84,7 +80,6 @@ final class Composer
{ {
parser_ = null; parser_ = null;
resolver_ = null; resolver_ = null;
constructor_ = null;
anchors_.destroy(); anchors_.destroy();
anchors_ = null; anchors_ = null;
} }
@ -222,8 +217,9 @@ final class Composer
const tag = resolver_.resolve(NodeID.scalar, event.tag, event.value, const tag = resolver_.resolve(NodeID.scalar, event.tag, event.value,
event.implicit); event.implicit);
Node node = constructor_.node(event.startMark, event.endMark, tag, Node node = constructNode(event.startMark, event.endMark, tag,
event.value, event.scalarStyle); event.value);
node.scalarStyle = event.scalarStyle;
return node; return node;
} }
@ -248,8 +244,9 @@ final class Composer
nodeAppender.put(composeNode(pairAppenderLevel, nodeAppenderLevel + 1)); nodeAppender.put(composeNode(pairAppenderLevel, nodeAppenderLevel + 1));
} }
Node node = constructor_.node(startEvent.startMark, parser_.front.endMark, Node node = constructNode(startEvent.startMark, parser_.front.endMark,
tag, nodeAppender.data.dup, startEvent.collectionStyle); tag, nodeAppender.data.dup);
node.collectionStyle = startEvent.collectionStyle;
parser_.popFront(); parser_.popFront();
nodeAppender.clear(); nodeAppender.clear();
@ -370,8 +367,9 @@ final class Composer
enforce(numUnique == pairAppender.data.length, enforce(numUnique == pairAppender.data.length,
new ComposerException("Duplicate key found in mapping", parser_.front.startMark)); new ComposerException("Duplicate key found in mapping", parser_.front.startMark));
Node node = constructor_.node(startEvent.startMark, parser_.front.endMark, Node node = constructNode(startEvent.startMark, parser_.front.endMark,
tag, pairAppender.data.dup, startEvent.collectionStyle); tag, pairAppender.data.dup);
node.collectionStyle = startEvent.collectionStyle;
parser_.popFront(); parser_.popFront();
pairAppender.clear(); pairAppender.clear();

615
source/dyaml/constructor.d
View file

@ -28,9 +28,10 @@ import dyaml.node;
import dyaml.exception; import dyaml.exception;
import dyaml.style; import dyaml.style;
package:
// Exception thrown at constructor errors. // Exception thrown at constructor errors.
package class ConstructorException : YAMLException class ConstructorException : YAMLException
{ {
/// Construct a ConstructorException. /// Construct a ConstructorException.
/// ///
@ -59,203 +60,6 @@ package class ConstructorException : YAMLException
* *
* If a tag is detected with no known constructor function, it is considered an error. * If a tag is detected with no known constructor function, it is considered an error.
*/ */
final class Constructor
{
private:
// Constructor functions from scalars.
Node delegate(ref Node) @safe[string] fromScalar_;
// Constructor functions from sequences.
Node delegate(ref Node) @safe[string] fromSequence_;
// Constructor functions from mappings.
Node delegate(ref Node) @safe[string] fromMapping_;
public:
/// Construct a Constructor.
///
/// If you don't want to support default YAML tags/data types, you can use
/// defaultConstructors to disable constructor functions for these.
///
/// Params: defaultConstructors = Use constructors for default YAML tags?
this(const Flag!"useDefaultConstructors" defaultConstructors = Yes.useDefaultConstructors)
@safe nothrow
{
if(!defaultConstructors){return;}
addConstructorScalar("tag:yaml.org,2002:null", &constructNull);
addConstructorScalar("tag:yaml.org,2002:bool", &constructBool);
addConstructorScalar("tag:yaml.org,2002:int", &constructLong);
addConstructorScalar("tag:yaml.org,2002:float", &constructReal);
addConstructorScalar("tag:yaml.org,2002:binary", &constructBinary);
addConstructorScalar("tag:yaml.org,2002:timestamp", &constructTimestamp);
addConstructorScalar("tag:yaml.org,2002:str", &constructString);
///In a mapping, the default value is kept as an entry with the '=' key.
addConstructorScalar("tag:yaml.org,2002:value", &constructString);
addConstructorSequence("tag:yaml.org,2002:omap", &constructOrderedMap);
addConstructorSequence("tag:yaml.org,2002:pairs", &constructPairs);
addConstructorMapping("tag:yaml.org,2002:set", &constructSet);
addConstructorSequence("tag:yaml.org,2002:seq", &constructSequence);
addConstructorMapping("tag:yaml.org,2002:map", &constructMap);
addConstructorScalar("tag:yaml.org,2002:merge", &constructMerge);
}
/** Add a constructor function from scalar.
*
* The function must take a reference to $(D Node) to construct from.
* The node contains a string for scalars, $(D Node[]) for sequences and
* $(D Node.Pair[]) for mappings.
*
* Any exception thrown by this function will be caught by D:YAML and
* its message will be added to a $(D YAMLException) that will also tell
* the user which type failed to construct, and position in the file.
*
*
* The value returned by this function will be stored in the resulting node.
*
* Only one constructor function can be set for one tag.
*
*
* Structs and classes must implement the $(D opCmp()) operator for D:YAML
* support. The signature of the operator that must be implemented
* is $(D const int opCmp(ref const MyStruct s)) for structs where
* $(I MyStruct) is the struct type, and $(D int opCmp(Object o)) for
* classes. Note that the class $(D opCmp()) should not alter the compared
* values - it is not const for compatibility reasons.
*
* Params: tag = Tag for the function to handle.
* ctor = Constructor function.
*/
void addConstructorScalar(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!string)[t] = deleg;
}
///
@safe unittest
{
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructScalar(ref Node node) @safe
{
//Guaranteed to be string as we construct from scalar.
//!mystruct x:y:z
auto parts = node.as!string().split(":");
// If this throws, the D:YAML will handle it and throw a YAMLException.
return MyStruct(to!int(parts[0]), to!int(parts[1]), to!int(parts[2]));
}
import dyaml.loader : Loader;
auto loader = Loader.fromString("!mystruct 12:34:56");
auto constructor = new Constructor;
constructor.addConstructorScalar("!mystruct", &constructMyStructScalar);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(12, 34, 56));
}
/** Add a constructor function from sequence.
*
* See_Also: addConstructorScalar
*/
void addConstructorSequence(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!(Node[]))[t] = deleg;
}
///
@safe unittest
{
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructSequence(ref Node node) @safe
{
//node is guaranteed to be sequence.
//!mystruct [x, y, z]
return MyStruct(node[0].as!int, node[1].as!int, node[2].as!int);
}
import dyaml.loader : Loader;
auto loader = Loader.fromString("!mystruct [1,2,3]");
auto constructor = new Constructor;
constructor.addConstructorSequence("!mystruct", &constructMyStructSequence);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(1, 2, 3));
}
/** Add a constructor function from a mapping.
*
* See_Also: addConstructorScalar
*/
void addConstructorMapping(T)(const string tag, T function(ref Node) @safe ctor)
{
const t = tag;
const deleg = addConstructor!T(t, ctor);
(*delegates!(Node.Pair[]))[t] = deleg;
}
///
@safe unittest {
static struct MyStruct
{
int x, y, z;
//Any D:YAML type must have a custom opCmp operator.
//This is used for ordering in mappings.
int opCmp(ref const MyStruct s) const
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructMapping(ref Node node) @safe
{
//node is guaranteed to be mapping.
//!mystruct {"x": x, "y": y, "z": z}
return MyStruct(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
import dyaml.loader : Loader;
auto loader = Loader.fromString(`!mystruct {"x": 11, "y": 22, "z": 33}`);
auto constructor = new Constructor;
constructor.addConstructorMapping("!mystruct", &constructMyStructMapping);
loader.constructor = constructor;
Node node = loader.load();
assert(node.get!MyStruct == MyStruct(11, 22, 33));
}
package:
/* /*
* Construct a node. * Construct a node.
* *
@ -267,200 +71,137 @@ final class Constructor
* *
* Returns: Constructed node. * Returns: Constructed node.
*/ */
Node node(T, U)(const Mark start, const Mark end, const string tag, Node constructNode(T)(const Mark start, const Mark end, const string tag,
T value, U style) @safe T value) @safe
if((is(T : string) || is(T == Node[]) || is(T == Node.Pair[])) && if((is(T : string) || is(T == Node[]) || is(T == Node.Pair[])))
(is(U : CollectionStyle) || is(U : ScalarStyle)))
{ {
enum type = is(T : string) ? "scalar" : Node newNode;
is(T == Node[]) ? "sequence" :
is(T == Node.Pair[]) ? "mapping" :
"ERROR";
enforce((tag in *delegates!T) !is null,
new ConstructorException("No constructor function from " ~ type ~
" for tag " ~ tag, start, end));
Node node = Node(value);
try try
{ {
static if(is(U : ScalarStyle)) switch(tag)
{ {
auto newNode = (*delegates!T)[tag](node); case "tag:yaml.org,2002:null":
newNode.startMark_ = start; newNode = Node(YAMLNull(), tag);
newNode.scalarStyle = style; break;
return newNode; case "tag:yaml.org,2002:bool":
} static if(is(T == string))
else static if(is(U : CollectionStyle))
{ {
auto newNode = (*delegates!T)[tag](node); newNode = Node(constructBool(value), tag);
newNode.startMark_ = start; break;
newNode.collectionStyle = style; }
return newNode; else throw new Exception("Only scalars can be bools");
case "tag:yaml.org,2002:int":
static if(is(T == string))
{
newNode = Node(constructLong(value), tag);
break;
}
else throw new Exception("Only scalars can be ints");
case "tag:yaml.org,2002:float":
static if(is(T == string))
{
newNode = Node(constructReal(value), tag);
break;
}
else throw new Exception("Only scalars can be floats");
case "tag:yaml.org,2002:binary":
static if(is(T == string))
{
newNode = Node(constructBinary(value), tag);
break;
}
else throw new Exception("Only scalars can be binary data");
case "tag:yaml.org,2002:timestamp":
static if(is(T == string))
{
newNode = Node(constructTimestamp(value), tag);
break;
}
else throw new Exception("Only scalars can be timestamps");
case "tag:yaml.org,2002:str":
static if(is(T == string))
{
newNode = Node(constructString(value), tag);
break;
}
else throw new Exception("Only scalars can be strings");
case "tag:yaml.org,2002:value":
static if(is(T == string))
{
newNode = Node(constructString(value), tag);
break;
}
else throw new Exception("Only scalars can be values");
case "tag:yaml.org,2002:omap":
static if(is(T == Node[]))
{
newNode = Node(constructOrderedMap(value), tag);
break;
}
else throw new Exception("Only sequences can be ordered maps");
case "tag:yaml.org,2002:pairs":
static if(is(T == Node[]))
{
newNode = Node(constructPairs(value), tag);
break;
}
else throw new Exception("Only sequences can be pairs");
case "tag:yaml.org,2002:set":
static if(is(T == Node.Pair[]))
{
newNode = Node(constructSet(value), tag);
break;
}
else throw new Exception("Only mappings can be sets");
case "tag:yaml.org,2002:seq":
static if(is(T == Node[]))
{
newNode = Node(constructSequence(value), tag);
break;
}
else throw new Exception("Only sequences can be sequences");
case "tag:yaml.org,2002:map":
static if(is(T == Node.Pair[]))
{
newNode = Node(constructMap(value), tag);
break;
}
else throw new Exception("Only mappings can be maps");
case "tag:yaml.org,2002:merge":
newNode = Node(YAMLMerge(), tag);
break;
default:
newNode = Node(value, tag);
break;
} }
else static assert(false);
} }
catch(Exception e) catch(Exception e)
{ {
throw new ConstructorException("Error constructing " ~ typeid(T).toString() throw new ConstructorException("Error constructing " ~ typeid(T).toString()
~ ":\n" ~ e.msg, start, end); ~ ":\n" ~ e.msg, start, end);
} }
newNode.startMark_ = start;
return newNode;
} }
private: private:
/* // Construct a boolean _node.
* Add a constructor function. bool constructBool(const string str) @safe
*
* Params: tag = Tag for the function to handle.
* ctor = Constructor function.
*/
auto addConstructor(T)(const string tag, T function(ref Node) @safe ctor)
{
assert((tag in fromScalar_) is null &&
(tag in fromSequence_) is null &&
(tag in fromMapping_) is null,
"Constructor function for tag " ~ tag ~ " is already " ~
"specified. Can't specify another one.");
return (ref Node n) @safe
{
return Node(ctor(n), tag);
};
}
//Get the array of constructor functions for scalar, sequence or mapping.
@property auto delegates(T)()
{
static if(is(T : string)) {return &fromScalar_;}
else static if(is(T : Node[])) {return &fromSequence_;}
else static if(is(T : Node.Pair[])){return &fromMapping_;}
else static assert(false);
}
}
///Construct a struct from a scalar
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructScalar(ref Node node) @safe
{
// Guaranteed to be string as we construct from scalar.
auto parts = node.as!string().split(":");
return MyStruct(to!int(parts[0]), to!int(parts[1]), to!int(parts[2]));
}
import dyaml.loader : Loader;
string data = "!mystruct 1:2:3";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorScalar("!mystruct", &constructMyStructScalar);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
///Construct a struct from a sequence
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructSequence(ref Node node) @safe
{
// node is guaranteed to be sequence.
return MyStruct(node[0].as!int, node[1].as!int, node[2].as!int);
}
import dyaml.loader : Loader;
string data = "!mystruct [1, 2, 3]";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorSequence("!mystruct", &constructMyStructSequence);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
///Construct a struct from a mapping
@safe unittest
{
static struct MyStruct
{
int x, y, z;
int opCmp(ref const MyStruct s) const pure @safe nothrow
{
if(x != s.x){return x - s.x;}
if(y != s.y){return y - s.y;}
if(z != s.z){return z - s.z;}
return 0;
}
}
static MyStruct constructMyStructMapping(ref Node node) @safe
{
// node is guaranteed to be mapping.
return MyStruct(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
import dyaml.loader : Loader;
string data = "!mystruct {x: 1, y: 2, z: 3}";
auto loader = Loader.fromString(data);
auto constructor = new Constructor;
constructor.addConstructorMapping("!mystruct", &constructMyStructMapping);
loader.constructor = constructor;
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1, 2, 3));
}
/// Construct a _null _node.
YAMLNull constructNull(ref Node node) @safe pure nothrow @nogc
{
return YAMLNull();
}
/// Construct a merge _node - a _node that merges another _node into a mapping.
YAMLMerge constructMerge(ref Node node) @safe pure nothrow @nogc
{
return YAMLMerge();
}
/// Construct a boolean _node.
bool constructBool(ref Node node) @safe
{ {
static yes = ["yes", "true", "on"]; static yes = ["yes", "true", "on"];
static no = ["no", "false", "off"]; static no = ["no", "false", "off"];
string value = node.as!string().toLower(); string value = str.toLower();
if(yes.canFind(value)){return true;} if(yes.canFind(value)){return true;}
if(no.canFind(value)) {return false;} if(no.canFind(value)) {return false;}
throw new Exception("Unable to parse boolean value: " ~ value); throw new Exception("Unable to parse boolean value: " ~ value);
} }
/// Construct an integer (long) _node. // Construct an integer (long) _node.
long constructLong(ref Node node) @safe long constructLong(const string str) @safe
{ {
string value = node.as!string().replace("_", ""); string value = str.replace("_", "");
const char c = value[0]; const char c = value[0];
const long sign = c != '-' ? 1 : -1; const long sign = c != '-' ? 1 : -1;
if(c == '-' || c == '+') if(c == '-' || c == '+')
@ -505,12 +246,6 @@ long constructLong(ref Node node) @safe
} }
@safe unittest @safe unittest
{ {
long getLong(string str) @safe
{
auto node = Node(str);
return constructLong(node);
}
string canonical = "685230"; string canonical = "685230";
string decimal = "+685_230"; string decimal = "+685_230";
string octal = "02472256"; string octal = "02472256";
@ -518,18 +253,18 @@ long constructLong(ref Node node) @safe
string binary = "0b1010_0111_0100_1010_1110"; string binary = "0b1010_0111_0100_1010_1110";
string sexagesimal = "190:20:30"; string sexagesimal = "190:20:30";
assert(685230 == getLong(canonical)); assert(685230 == constructLong(canonical));
assert(685230 == getLong(decimal)); assert(685230 == constructLong(decimal));
assert(685230 == getLong(octal)); assert(685230 == constructLong(octal));
assert(685230 == getLong(hexadecimal)); assert(685230 == constructLong(hexadecimal));
assert(685230 == getLong(binary)); assert(685230 == constructLong(binary));
assert(685230 == getLong(sexagesimal)); assert(685230 == constructLong(sexagesimal));
} }
/// Construct a floating point (real) _node. // Construct a floating point (real) _node.
real constructReal(ref Node node) @safe real constructReal(const string str) @safe
{ {
string value = node.as!string().replace("_", "").toLower(); string value = str.replace("_", "").toLower();
const char c = value[0]; const char c = value[0];
const real sign = c != '-' ? 1.0 : -1.0; const real sign = c != '-' ? 1.0 : -1.0;
if(c == '-' || c == '+') if(c == '-' || c == '+')
@ -576,12 +311,6 @@ real constructReal(ref Node node) @safe
return a >= (b - epsilon) && a <= (b + epsilon); return a >= (b - epsilon) && a <= (b + epsilon);
} }
real getReal(string str) @safe
{
auto node = Node(str);
return constructReal(node);
}
string canonical = "6.8523015e+5"; string canonical = "6.8523015e+5";
string exponential = "685.230_15e+03"; string exponential = "685.230_15e+03";
string fixed = "685_230.15"; string fixed = "685_230.15";
@ -589,21 +318,20 @@ real constructReal(ref Node node) @safe
string negativeInf = "-.inf"; string negativeInf = "-.inf";
string NaN = ".NaN"; string NaN = ".NaN";
assert(eq(685230.15, getReal(canonical))); assert(eq(685230.15, constructReal(canonical)));
assert(eq(685230.15, getReal(exponential))); assert(eq(685230.15, constructReal(exponential)));
assert(eq(685230.15, getReal(fixed))); assert(eq(685230.15, constructReal(fixed)));
assert(eq(685230.15, getReal(sexagesimal))); assert(eq(685230.15, constructReal(sexagesimal)));
assert(eq(-real.infinity, getReal(negativeInf))); assert(eq(-real.infinity, constructReal(negativeInf)));
assert(to!string(getReal(NaN)) == "nan"); assert(to!string(constructReal(NaN)) == "nan");
} }
/// Construct a binary (base64) _node. // Construct a binary (base64) _node.
ubyte[] constructBinary(ref Node node) @safe ubyte[] constructBinary(const string value) @safe
{ {
import std.ascii : newline; import std.ascii : newline;
import std.array : array; import std.array : array;
string value = node.as!string;
// For an unknown reason, this must be nested to work (compiler bug?). // For an unknown reason, this must be nested to work (compiler bug?).
try try
{ {
@ -621,16 +349,15 @@ ubyte[] constructBinary(ref Node node) @safe
char[] buffer; char[] buffer;
buffer.length = 256; buffer.length = 256;
string input = Base64.encode(test, buffer).idup; string input = Base64.encode(test, buffer).idup;
auto node = Node(input); const value = constructBinary(input);
const value = constructBinary(node);
assert(value == test); assert(value == test);
assert(value == [84, 104, 101, 32, 65, 110, 115, 119, 101, 114, 58, 32, 52, 50]); assert(value == [84, 104, 101, 32, 65, 110, 115, 119, 101, 114, 58, 32, 52, 50]);
} }
/// Construct a timestamp (SysTime) _node. // Construct a timestamp (SysTime) _node.
SysTime constructTimestamp(ref Node node) @safe SysTime constructTimestamp(const string str) @safe
{ {
string value = node.as!string; string value = str;
auto YMDRegexp = regex("^([0-9][0-9][0-9][0-9])-([0-9][0-9]?)-([0-9][0-9]?)"); auto YMDRegexp = regex("^([0-9][0-9][0-9][0-9])-([0-9][0-9]?)-([0-9][0-9]?)");
auto HMSRegexp = regex("^[Tt \t]+([0-9][0-9]?):([0-9][0-9]):([0-9][0-9])(\\.[0-9]*)?"); auto HMSRegexp = regex("^[Tt \t]+([0-9][0-9]?):([0-9][0-9]):([0-9][0-9])(\\.[0-9]*)?");
@ -704,8 +431,7 @@ SysTime constructTimestamp(ref Node node) @safe
{ {
string timestamp(string value) string timestamp(string value)
{ {
auto node = Node(value); return constructTimestamp(value).toISOString();
return constructTimestamp(node).toISOString();
} }
string canonical = "2001-12-15T02:59:43.1Z"; string canonical = "2001-12-15T02:59:43.1Z";
@ -727,18 +453,18 @@ SysTime constructTimestamp(ref Node node) @safe
assert(timestamp(ymd) == "20021214T000000Z"); assert(timestamp(ymd) == "20021214T000000Z");
} }
/// Construct a string _node. // Construct a string _node.
string constructString(ref Node node) @safe string constructString(const string str) @safe
{ {
return node.as!string; return str;
} }
/// Convert a sequence of single-element mappings into a sequence of pairs. // Convert a sequence of single-element mappings into a sequence of pairs.
Node.Pair[] getPairs(string type, Node[] nodes) @safe Node.Pair[] getPairs(string type, const Node[] nodes) @safe
{ {
Node.Pair[] pairs; Node.Pair[] pairs;
pairs.reserve(nodes.length); pairs.reserve(nodes.length);
foreach(ref node; nodes) foreach(node; nodes)
{ {
enforce(node.isMapping && node.length == 1, enforce(node.isMapping && node.length == 1,
new Exception("While constructing " ~ type ~ new Exception("While constructing " ~ type ~
@ -750,10 +476,10 @@ Node.Pair[] getPairs(string type, Node[] nodes) @safe
return pairs; return pairs;
} }
/// Construct an ordered map (ordered sequence of key:value pairs without duplicates) _node. // Construct an ordered map (ordered sequence of key:value pairs without duplicates) _node.
Node.Pair[] constructOrderedMap(ref Node node) @safe Node.Pair[] constructOrderedMap(const Node[] nodes) @safe
{ {
auto pairs = getPairs("ordered map", node.as!(Node[])); auto pairs = getPairs("ordered map", nodes);
//Detect duplicates. //Detect duplicates.
//TODO this should be replaced by something with deterministic memory allocation. //TODO this should be replaced by something with deterministic memory allocation.
@ -791,38 +517,30 @@ Node.Pair[] constructOrderedMap(ref Node node) @safe
return pairs; return pairs;
} }
bool hasDuplicates(Node[] nodes) @safe assertThrown(constructOrderedMap(alternateTypes(8) ~ alternateTypes(2)));
{ assertNotThrown(constructOrderedMap(alternateTypes(8)));
auto node = Node(nodes); assertThrown(constructOrderedMap(sameType(64) ~ sameType(16)));
return null !is collectException(constructOrderedMap(node)); assertThrown(constructOrderedMap(alternateTypes(64) ~ alternateTypes(16)));
assertNotThrown(constructOrderedMap(sameType(64)));
assertNotThrown(constructOrderedMap(alternateTypes(64)));
} }
assert(hasDuplicates(alternateTypes(8) ~ alternateTypes(2))); // Construct a pairs (ordered sequence of key: value pairs allowing duplicates) _node.
assert(!hasDuplicates(alternateTypes(8))); Node.Pair[] constructPairs(const Node[] nodes) @safe
assert(hasDuplicates(sameType(64) ~ sameType(16))); {
assert(hasDuplicates(alternateTypes(64) ~ alternateTypes(16))); return getPairs("pairs", nodes);
assert(!hasDuplicates(sameType(64)));
assert(!hasDuplicates(alternateTypes(64)));
} }
/// Construct a pairs (ordered sequence of key: value pairs allowing duplicates) _node. // Construct a set _node.
Node.Pair[] constructPairs(ref Node node) @safe Node[] constructSet(const Node.Pair[] pairs) @safe
{ {
return getPairs("pairs", node.as!(Node[]));
}
/// Construct a set _node.
Node[] constructSet(ref Node node) @safe
{
auto pairs = node.as!(Node.Pair[]);
// In future, the map here should be replaced with something with deterministic // In future, the map here should be replaced with something with deterministic
// memory allocation if possible. // memory allocation if possible.
// Detect duplicates. // Detect duplicates.
ubyte[Node] map; ubyte[Node] map;
Node[] nodes; Node[] nodes;
nodes.reserve(pairs.length); nodes.reserve(pairs.length);
foreach(ref pair; pairs) foreach(pair; pairs)
{ {
enforce((pair.key in map) is null, new Exception("Duplicate entry in a set")); enforce((pair.key in map) is null, new Exception("Duplicate entry in a set"));
map[pair.key] = 0; map[pair.key] = 0;
@ -862,27 +580,26 @@ Node[] constructSet(ref Node node) @safe
return true; return true;
} }
auto nodeDuplicatesShort = Node(DuplicatesShort.dup); auto nodeDuplicatesShort = DuplicatesShort.dup;
auto nodeNoDuplicatesShort = Node(noDuplicatesShort.dup); auto nodeNoDuplicatesShort = noDuplicatesShort.dup;
auto nodeDuplicatesLong = Node(DuplicatesLong.dup); auto nodeDuplicatesLong = DuplicatesLong.dup;
auto nodeNoDuplicatesLong = Node(noDuplicatesLong.dup); auto nodeNoDuplicatesLong = noDuplicatesLong.dup;
assert(null !is collectException(constructSet(nodeDuplicatesShort))); assertThrown(constructSet(nodeDuplicatesShort));
assert(null is collectException(constructSet(nodeNoDuplicatesShort))); assertNotThrown(constructSet(nodeNoDuplicatesShort));
assert(null !is collectException(constructSet(nodeDuplicatesLong))); assertThrown(constructSet(nodeDuplicatesLong));
assert(null is collectException(constructSet(nodeNoDuplicatesLong))); assertNotThrown(constructSet(nodeNoDuplicatesLong));
} }
/// Construct a sequence (array) _node. // Construct a sequence (array) _node.
Node[] constructSequence(ref Node node) @safe Node[] constructSequence(Node[] nodes) @safe
{ {
return node.as!(Node[]); return nodes;
} }
/// Construct an unordered map (unordered set of key:value _pairs without duplicates) _node. // Construct an unordered map (unordered set of key:value _pairs without duplicates) _node.
Node.Pair[] constructMap(ref Node node) @safe Node.Pair[] constructMap(Node.Pair[] pairs) @safe
{ {
auto pairs = node.as!(Node.Pair[]);
//Detect duplicates. //Detect duplicates.
//TODO this should be replaced by something with deterministic memory allocation. //TODO this should be replaced by something with deterministic memory allocation.
auto keys = redBlackTree!Node(); auto keys = redBlackTree!Node();

67
source/dyaml/dumper.d
View file

@ -23,6 +23,7 @@ import dyaml.node;
import dyaml.representer; import dyaml.representer;
import dyaml.resolver; import dyaml.resolver;
import dyaml.serializer; import dyaml.serializer;
import dyaml.style;
import dyaml.tagdirective; import dyaml.tagdirective;
@ -45,8 +46,6 @@ struct Dumper(Range)
private: private:
//Resolver to resolve tags. //Resolver to resolve tags.
Resolver resolver_; Resolver resolver_;
//Representer to represent data types.
Representer representer_;
//Stream to write to. //Stream to write to.
Range stream_; Range stream_;
@ -71,7 +70,24 @@ struct Dumper(Range)
//Name of the output file or stream, used in error messages. //Name of the output file or stream, used in error messages.
string name_ = "<unknown>"; string name_ = "<unknown>";
// Default style for scalar nodes.
ScalarStyle defaultScalarStyle_ = ScalarStyle.invalid;
// Default style for collection nodes.
CollectionStyle defaultCollectionStyle_ = CollectionStyle.invalid;
public: public:
///Set default _style for scalars. If style is $(D ScalarStyle.invalid), the _style is chosen automatically.
@property void defaultScalarStyle(ScalarStyle style) pure @safe nothrow
{
defaultScalarStyle_ = style;
}
///Set default _style for collections. If style is $(D CollectionStyle.invalid), the _style is chosen automatically.
@property void defaultCollectionStyle(CollectionStyle style) pure @safe nothrow
{
defaultCollectionStyle_ = style;
}
@disable this(); @disable this();
@disable bool opEquals(ref Dumper!Range); @disable bool opEquals(ref Dumper!Range);
@disable int opCmp(ref Dumper!Range); @disable int opCmp(ref Dumper!Range);
@ -84,7 +100,6 @@ struct Dumper(Range)
this(Range stream) @safe this(Range stream) @safe
{ {
resolver_ = new Resolver(); resolver_ = new Resolver();
representer_ = new Representer();
stream_ = stream; stream_ = stream;
} }
@ -100,12 +115,6 @@ struct Dumper(Range)
resolver_ = resolver; resolver_ = resolver;
} }
///Specify custom Representer to use.
@property void representer(Representer representer) @safe
{
representer_ = representer;
}
///Write scalars in _canonical form? ///Write scalars in _canonical form?
@property void canonical(bool canonical) pure @safe nothrow @property void canonical(bool canonical) pure @safe nothrow
{ {
@ -219,7 +228,8 @@ struct Dumper(Range)
explicitEnd_, YAMLVersion_, tags_); explicitEnd_, YAMLVersion_, tags_);
foreach(ref document; documents) foreach(ref document; documents)
{ {
representer_.represent(serializer, document); auto data = representData(document, defaultScalarStyle_, defaultCollectionStyle_);
serializer.serialize(data);
} }
} }
catch(YAMLException e) catch(YAMLException e)
@ -277,18 +287,47 @@ struct Dumper(Range)
auto node = Node([1, 2, 3, 4, 5]); auto node = Node([1, 2, 3, 4, 5]);
dumper(stream).dump(node); dumper(stream).dump(node);
} }
///Use a custom representer/resolver to support custom data types and/or implicit tags ///Use a custom resolver to support custom data types and/or implicit tags
@safe unittest @safe unittest
{ {
auto node = Node([1, 2, 3, 4, 5]); auto node = Node([1, 2, 3, 4, 5]);
auto representer = new Representer();
auto resolver = new Resolver(); auto resolver = new Resolver();
//Add representer functions / resolver expressions here...
auto dumper = dumper(new Appender!string()); auto dumper = dumper(new Appender!string());
dumper.representer = representer;
dumper.resolver = resolver; dumper.resolver = resolver;
dumper.dump(node); dumper.dump(node);
} }
/// Set default scalar style
@safe unittest
{
auto stream = new Appender!string();
auto node = Node("Hello world!");
auto dumper = dumper(stream);
dumper.defaultScalarStyle = ScalarStyle.singleQuoted;
dumper.dump(node);
}
/// Set default collection style
@safe unittest
{
auto stream = new Appender!string();
auto node = Node(["Hello", "world!"]);
auto dumper = dumper(stream);
dumper.defaultCollectionStyle = CollectionStyle.flow;
dumper.dump(node);
}
// Make sure the styles are actually used
@safe unittest
{
auto stream = new Appender!string();
auto node = Node([Node("Hello world!"), Node(["Hello", "world!"])]);
auto dumper = dumper(stream);
dumper.defaultScalarStyle = ScalarStyle.singleQuoted;
dumper.defaultCollectionStyle = CollectionStyle.flow;
dumper.explicitEnd = false;
dumper.explicitStart = false;
dumper.YAMLVersion = null;
dumper.dump(node);
assert(stream.data == "[!!str 'Hello world!', [!!str 'Hello', !!str 'world!']]\n");
}
// Explicit document start/end markers // Explicit document start/end markers
@safe unittest @safe unittest
{ {

13
source/dyaml/loader.d
View file

@ -41,8 +41,6 @@ struct Loader
Parser parser_; Parser parser_;
// Resolves tags (data types). // Resolves tags (data types).
Resolver resolver_; Resolver resolver_;
// Constructs YAML data types.
Constructor constructor_;
// Name of the input file or stream, used in error messages. // Name of the input file or stream, used in error messages.
string name_ = "<unknown>"; string name_ = "<unknown>";
// Are we done loading? // Are we done loading?
@ -177,12 +175,6 @@ struct Loader
resolver_ = resolver; resolver_ = resolver;
} }
/// Specify custom Constructor to use.
void constructor(Constructor constructor) pure @safe nothrow @nogc
{
constructor_ = constructor;
}
/** Load single YAML document. /** Load single YAML document.
* *
* If none or more than one YAML document is found, this throws a YAMLException. * If none or more than one YAML document is found, this throws a YAMLException.
@ -230,7 +222,7 @@ struct Loader
if (!rangeInitialized) if (!rangeInitialized)
{ {
lazyInitConstructorResolver(); lazyInitConstructorResolver();
composer = new Composer(parser_, resolver_, constructor_); composer = new Composer(parser_, resolver_);
rangeInitialized = true; rangeInitialized = true;
} }
assert(!done_, "Loader.popFront called on empty range"); assert(!done_, "Loader.popFront called on empty range");
@ -301,7 +293,6 @@ struct Loader
void lazyInitConstructorResolver() @safe void lazyInitConstructorResolver() @safe
{ {
if(resolver_ is null) { resolver_ = new Resolver(); } if(resolver_ is null) { resolver_ = new Resolver(); }
if(constructor_ is null) { constructor_ = new Constructor(); }
} }
} }
/// Load single YAML document from a file: /// Load single YAML document from a file:
@ -410,13 +401,11 @@ struct Loader
"Hello world!\n"~ "Hello world!\n"~
"...\n" "...\n"
); );
auto constructor = new Constructor();
auto resolver = new Resolver(); auto resolver = new Resolver();
// Add constructor functions / resolver expressions here... // Add constructor functions / resolver expressions here...
auto loader = Loader.fromFile("example.yaml"); auto loader = Loader.fromFile("example.yaml");
loader.constructor = constructor;
loader.resolver = resolver; loader.resolver = resolver;
auto rootNode = loader.load(); auto rootNode = loader.load();
} }

389
source/dyaml/node.d
View file

@ -58,59 +58,6 @@ struct YAMLNull
// Merge YAML type, used to support "tag:yaml.org,2002:merge". // Merge YAML type, used to support "tag:yaml.org,2002:merge".
package struct YAMLMerge{} package struct YAMLMerge{}
// Interface for YAMLContainer - used for user defined YAML types.
package interface YAMLObject
{
public:
// Get type of the stored value.
@property TypeInfo type() const pure @safe nothrow;
protected:
// Compare with another YAMLObject.
int cmp(const YAMLObject) const @system;
}
// Stores a user defined YAML data type.
package class YAMLContainer(T) if (!Node.allowed!T): YAMLObject
{
private:
// Stored value.
T value_;
// Construct a YAMLContainer holding specified value.
this(T value) @safe
{
value_ = value;
}
public:
// Get type of the stored value.
@property override TypeInfo type() const pure @safe nothrow
{
return typeid(T);
}
// Get string representation of the container.
override string toString() @system
{
return format("YAMLContainer(%s)", value_);
}
protected:
// Compare with another YAMLObject.
override int cmp(const YAMLObject rhs) const @system
{
const typeCmp = type.opCmp(rhs.type);
if(typeCmp != 0){return typeCmp;}
// Const-casting here as Object opCmp is not const.
T* v1 = cast(T*)&value_;
T* v2 = cast(T*)&((cast(YAMLContainer)rhs).value_);
return (*v1).opCmp(*v2);
}
}
// Key-value pair of YAML nodes, used in mappings. // Key-value pair of YAML nodes, used in mappings.
private struct Pair private struct Pair
{ {
@ -150,6 +97,20 @@ private struct Pair
@disable int opCmp(ref Pair); @disable int opCmp(ref Pair);
} }
enum NodeType
{
null_,
merge,
boolean,
integer,
decimal,
binary,
timestamp,
string,
mapping,
sequence
}
/** YAML node. /** YAML node.
* *
* This is a pseudo-dynamic type that can store any YAML value, including a * This is a pseudo-dynamic type that can store any YAML value, including a
@ -164,9 +125,9 @@ struct Node
package: package:
// YAML value type. // YAML value type.
alias Value = Algebraic!(YAMLNull, YAMLMerge, bool, long, real, ubyte[], SysTime, string, alias Value = Algebraic!(YAMLNull, YAMLMerge, bool, long, real, ubyte[], SysTime, string,
Node.Pair[], Node[], YAMLObject); Node.Pair[], Node[]);
// Can Value hold this type without wrapping it in a YAMLObject? // Can Value hold this type naturally?
enum allowed(T) = isIntegral!T || enum allowed(T) = isIntegral!T ||
isFloatingPoint!T || isFloatingPoint!T ||
isSomeString!T || isSomeString!T ||
@ -212,7 +173,8 @@ struct Node
* be in full form, e.g. "tag:yaml.org,2002:int", not * be in full form, e.g. "tag:yaml.org,2002:int", not
* a shortcut, like "!!int". * a shortcut, like "!!int".
*/ */
this(T)(T value, const string tag = null) this(T)(T value, const string tag = null) @safe
if (allowed!T || isArray!T || isAssociativeArray!T || is(Unqual!T == Node) || castableToNode!T)
{ {
tag_ = tag; tag_ = tag;
@ -333,15 +295,10 @@ struct Node
{ {
{ {
auto node = Node(42); auto node = Node(42);
assert(node.isScalar && !node.isSequence && !node.isMapping && !node.isUserType); assert(node.isScalar && !node.isSequence && !node.isMapping);
assert(node.as!int == 42 && node.as!float == 42.0f && node.as!string == "42"); assert(node.as!int == 42 && node.as!float == 42.0f && node.as!string == "42");
assert(!node.isUserType);
} }
{
auto node = Node(new class{int a = 5;});
assert(node.isUserType);
}
{ {
auto node = Node("string"); auto node = Node("string");
assert(node.as!string == "string"); assert(node.as!string == "string");
@ -351,7 +308,7 @@ struct Node
{ {
with(Node([1, 2, 3])) with(Node([1, 2, 3]))
{ {
assert(!isScalar() && isSequence && !isMapping && !isUserType); assert(!isScalar() && isSequence && !isMapping);
assert(length == 3); assert(length == 3);
assert(opIndex(2).as!int == 3); assert(opIndex(2).as!int == 3);
} }
@ -364,7 +321,7 @@ struct Node
aa["2"] = 2; aa["2"] = 2;
with(Node(aa)) with(Node(aa))
{ {
assert(!isScalar() && !isSequence && isMapping && !isUserType); assert(!isScalar() && !isSequence && isMapping);
assert(length == 2); assert(length == 2);
assert(opIndex("2").as!int == 2); assert(opIndex("2").as!int == 2);
} }
@ -433,7 +390,7 @@ struct Node
{ {
with(Node(["1", "2"], [1, 2])) with(Node(["1", "2"], [1, 2]))
{ {
assert(!isScalar() && !isSequence && isMapping && !isUserType); assert(!isScalar() && !isSequence && isMapping);
assert(length == 2); assert(length == 2);
assert(opIndex("2").as!int == 2); assert(opIndex("2").as!int == 2);
} }
@ -464,12 +421,6 @@ struct Node
return isType!(Pair[]); return isType!(Pair[]);
} }
/// Is this node a user defined type?
@property bool isUserType() const @safe nothrow
{
return isType!YAMLObject;
}
/// Is this node null? /// Is this node null?
@property bool isNull() const @safe nothrow @property bool isNull() const @safe nothrow
{ {
@ -552,18 +503,38 @@ struct Node
* the value is out of range of requested type. * the value is out of range of requested type.
*/ */
inout(T) get(T, Flag!"stringConversion" stringConversion = Yes.stringConversion)() inout inout(T) get(T, Flag!"stringConversion" stringConversion = Yes.stringConversion)() inout
if (allowed!(Unqual!T) || hasNodeConstructor!(Unqual!T))
{ {
if(isType!(Unqual!T)){return getValue!T;} if(isType!(Unqual!T)){return getValue!T;}
/// Must go before others, as even string/int/etc could be stored in a YAMLObject. static if(!allowed!(Unqual!T))
static if(!allowed!(Unqual!T)) if(isUserType)
{ {
auto object = getValue!YAMLObject(); static if (hasSimpleNodeConstructor!T)
enforce(object.type is typeid(T), {
new NodeException("Node has unexpected type: " ~ object.type.toString() ~ alias params = AliasSeq!(this);
". Expected: " ~ typeid(T).toString, startMark_));
return (cast(inout YAMLContainer!(Unqual!T))(object)).value_;
} }
else static if (hasExpandedNodeConstructor!T)
{
alias params = AliasSeq!(this, tag_);
}
else
{
static assert(0, "Unknown Node constructor?");
}
static if (is(T == class))
{
return new inout T(params);
}
else static if (is(T == struct))
{
return T(params);
}
else
{
static assert(0, "Unhandled user type");
}
} else {
// If we're getting from a mapping and we're not getting Node.Pair[], // If we're getting from a mapping and we're not getting Node.Pair[],
// we're getting the default value. // we're getting the default value.
@ -611,6 +582,7 @@ struct Node
else throw new NodeException("Node stores unexpected type: " ~ type.toString() ~ else throw new NodeException("Node stores unexpected type: " ~ type.toString() ~
". Expected: " ~ typeid(T).toString, startMark_); ". Expected: " ~ typeid(T).toString, startMark_);
} }
}
/// Automatic type conversion /// Automatic type conversion
@safe unittest @safe unittest
{ {
@ -620,6 +592,186 @@ struct Node
assert(node.get!string == "42"); assert(node.get!string == "42");
assert(node.get!double == 42.0); assert(node.get!double == 42.0);
} }
/// Scalar node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
this(int x, int y, int z) @safe
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
auto parts = node.as!string().split(":");
x = parts[0].to!int;
y = parts[1].to!int;
z = parts[2].to!int;
}
Node opCast(T: Node)() @safe
{
//Using custom scalar format, x:y:z.
auto scalar = format("%s:%s:%s", x, y, z);
//Representing as a scalar, with custom tag to specify this data type.
return Node(scalar, "!mystruct.tag");
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Sequence node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
this(int x, int y, int z) @safe
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
x = node[0].as!int;
y = node[1].as!int;
z = node[2].as!int;
}
Node opCast(T: Node)()
{
return Node([x, y, z], "!mystruct.tag");
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Mapping node to struct and vice versa
@safe unittest
{
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static struct MyStruct
{
int x, y, z;
Node opCast(T: Node)()
{
auto pairs = [Node.Pair("x", x),
Node.Pair("y", y),
Node.Pair("z", z)];
return Node(pairs, "!mystruct.tag");
}
this(int x, int y, int z)
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe
{
x = node["x"].as!int;
y = node["y"].as!int;
z = node["z"].as!int;
}
}
auto appender = new Appender!string;
// Dump struct to yaml document
dumper(appender).dump(Node(MyStruct(1,2,3)));
// Read yaml document back as a MyStruct
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyStruct == MyStruct(1,2,3));
}
/// Classes can be used too
@system unittest {
import dyaml.dumper : dumper;
import dyaml.loader : Loader;
static class MyClass
{
int x, y, z;
this(int x, int y, int z)
{
this.x = x;
this.y = y;
this.z = z;
}
this(Node node) @safe inout
{
auto parts = node.as!string().split(":");
x = parts[0].to!int;
y = parts[1].to!int;
z = parts[2].to!int;
}
///Useful for Node.as!string.
override string toString()
{
return format("MyClass(%s, %s, %s)", x, y, z);
}
Node opCast(T: Node)() @safe
{
//Using custom scalar format, x:y:z.
auto scalar = format("%s:%s:%s", x, y, z);
//Representing as a scalar, with custom tag to specify this data type.
return Node(scalar, "!myclass.tag");
}
override bool opEquals(Object o)
{
if (auto other = cast(MyClass)o)
{
return (other.x == x) && (other.y == y) && (other.z == z);
}
return false;
}
}
auto appender = new Appender!string;
// Dump class to yaml document
dumper(appender).dump(Node(new MyClass(1,2,3)));
// Read yaml document back as a MyClass
auto loader = Loader.fromString(appender.data);
Node node = loader.load();
assert(node.as!MyClass == new MyClass(1,2,3));
}
@safe unittest @safe unittest
{ {
assertThrown!NodeException(Node("42").get!int); assertThrown!NodeException(Node("42").get!int);
@ -1876,10 +2028,6 @@ struct Node
const t2 = rhs.getValue!SysTime; const t2 = rhs.getValue!SysTime;
return cmp(t1, t2); return cmp(t1, t2);
} }
else if(isUserType)
{
return getValue!YAMLObject.cmp(rhs.getValue!YAMLObject);
}
assert(false, "Unknown type of node for comparison : " ~ type.toString()); assert(false, "Unknown type of node for comparison : " ~ type.toString());
} }
@ -1923,12 +2071,58 @@ struct Node
assert(false); assert(false);
} }
// Get type of the node value (YAMLObject for user types). // Get type of the node value.
@property TypeInfo type() const @safe nothrow @property TypeInfo type() const @safe nothrow
{ {
return value_.type; return value_.type;
} }
// Get type of the node value.
@property NodeType newType() const @safe nothrow
{
if (value_.type is typeid(bool))
{
return NodeType.boolean;
}
else if (value_.type is typeid(long))
{
return NodeType.integer;
}
else if (value_.type is typeid(Node[]))
{
return NodeType.sequence;
}
else if (value_.type is typeid(ubyte[]))
{
return NodeType.binary;
}
else if (value_.type is typeid(string))
{
return NodeType.string;
}
else if (value_.type is typeid(Node.Pair[]))
{
return NodeType.mapping;
}
else if (value_.type is typeid(SysTime))
{
return NodeType.timestamp;
}
else if (value_.type is typeid(YAMLNull))
{
return NodeType.null_;
}
else if (value_.type is typeid(YAMLMerge))
{
return NodeType.merge;
}
else if (value_.type is typeid(real))
{
return NodeType.decimal;
}
else assert(0, text(value_.type));
}
public: public:
// Determine if the value stored by the node is of specified type. // Determine if the value stored by the node is of specified type.
// //
@ -2210,7 +2404,7 @@ struct Node
} }
else else
{ {
value_ = cast(YAMLObject)new YAMLContainer!T(value); value_ = (cast(Node)value).value_;
} }
} }
} }
@ -2232,3 +2426,30 @@ void merge(ref Appender!(Node.Pair[]) pairs, Node.Pair[] toMerge) @safe
pairs.put(pair); pairs.put(pair);
} }
} }
enum hasNodeConstructor(T) = hasSimpleNodeConstructor!T || hasExpandedNodeConstructor!T;
template hasSimpleNodeConstructor(T)
{
static if (is(T == struct))
{
enum hasSimpleNodeConstructor = is(typeof(T(Node.init)));
}
else static if (is(T == class))
{
enum hasSimpleNodeConstructor = is(typeof(new inout T(Node.init)));
}
else enum hasSimpleNodeConstructor = false;
}
template hasExpandedNodeConstructor(T)
{
static if (is(T == struct))
{
enum hasExpandedNodeConstructor = is(typeof(T(Node.init, "")));
}
else static if (is(T == class))
{
enum hasExpandedNodeConstructor = is(typeof(new inout T(Node.init, "")));
}
else enum hasExpandedNodeConstructor = false;
}
enum castableToNode(T) = (is(T == struct) || is(T == class)) && is(typeof(T.opCast!Node()) : Node);

2
source/dyaml/package.d
View file

@ -5,13 +5,11 @@
module dyaml; module dyaml;
public import dyaml.constructor;
public import dyaml.dumper; public import dyaml.dumper;
public import dyaml.encoding; public import dyaml.encoding;
public import dyaml.exception; public import dyaml.exception;
public import dyaml.linebreak; public import dyaml.linebreak;
public import dyaml.loader; public import dyaml.loader;
public import dyaml.representer;
public import dyaml.resolver; public import dyaml.resolver;
public import dyaml.style; public import dyaml.style;
public import dyaml.node; public import dyaml.node;

898
source/dyaml/representer.d
View file

File diff suppressed because it is too large Load diff

83
source/dyaml/scanner.d
View file

@ -1951,91 +1951,36 @@ final class Scanner
/// In case of an error, error_ is set. Use throwIfError() to handle this. /// In case of an error, error_ is set. Use throwIfError() to handle this.
void scanURIEscapesToSlice(string name)(const Mark startMark) void scanURIEscapesToSlice(string name)(const Mark startMark)
{ {
import core.exception : UnicodeException;
// URI escapes encode a UTF-8 string. We store UTF-8 code units here for // URI escapes encode a UTF-8 string. We store UTF-8 code units here for
// decoding into UTF-32. // decoding into UTF-32.
char[4] bytes; Appender!string buffer;
size_t bytesUsed;
// Get one dchar by decoding data from bytes.
//
// This is probably slow, but simple and URI escapes are extremely uncommon
// in YAML.
//
// Returns the number of bytes used by the dchar in bytes on success,
// size_t.max on failure.
static size_t getDchar(char[] bytes, Reader reader_) @safe
{
size_t nextChar;
dchar c;
if(bytes[0] < 0x80)
{
c = bytes[0];
++nextChar;
}
else
{
try
{
c = decode(bytes[], nextChar);
}
catch (UTFException)
{
return size_t.max;
}
}
reader_.sliceBuilder.write(c);
if(bytes.length - nextChar > 0)
{
copy(bytes[nextChar..bytes.length], bytes[0..bytes.length-nextChar]);
}
return bytes.length - nextChar;
}
enum contextMsg = "While scanning a " ~ name; enum contextMsg = "While scanning a " ~ name;
while(reader_.peekByte() == '%') while(reader_.peekByte() == '%')
{ {
reader_.forward(); reader_.forward();
if(bytesUsed == bytes.length) char[2] nextByte = [reader_.peekByte(), reader_.peekByte(1)];
{ if(!nextByte[0].isHexDigit || !nextByte[1].isHexDigit)
bytesUsed = getDchar(bytes[], reader_);
if(bytesUsed == size_t.max)
{
error(contextMsg, startMark,
"Invalid UTF-8 data encoded in URI escape sequence",
reader_.mark);
return;
}
}
char b = 0;
uint mult = 16;
// Converting 2 hexadecimal digits to a byte.
foreach(k; 0 .. 2)
{
const dchar c = reader_.peek(k);
if(!c.isHexDigit)
{ {
auto msg = expected("URI escape sequence of 2 hexadecimal " ~ auto msg = expected("URI escape sequence of 2 hexadecimal " ~
"numbers", c); "numbers", nextByte);
error(contextMsg, startMark, msg, reader_.mark); error(contextMsg, startMark, msg, reader_.mark);
return; return;
} }
buffer ~= nextByte[].to!ubyte(16);
uint digit;
if(c - '0' < 10) { digit = c - '0'; }
else if(c - 'A' < 6) { digit = 10 + c - 'A'; }
else if(c - 'a' < 6) { digit = 10 + c - 'a'; }
else { assert(false); }
b += mult * digit;
mult /= 16;
}
bytes[bytesUsed++] = b;
reader_.forward(2); reader_.forward(2);
} }
try
bytesUsed = getDchar(bytes[0 .. bytesUsed], reader_); {
if(bytesUsed == size_t.max) foreach (dchar chr; buffer.data)
{
reader_.sliceBuilder.write(chr);
}
}
catch (UnicodeException)
{ {
error(contextMsg, startMark, error(contextMsg, startMark,
"Invalid UTF-8 data encoded in URI escape sequence", "Invalid UTF-8 data encoded in URI escape sequence",

59
source/dyaml/test/constructor.d
View file

@ -328,21 +328,12 @@ class TestClass
this.z = z; this.z = z;
} }
//Any D:YAML type must have a custom opCmp operator. Node opCast(T: Node)() @safe
//This is used for ordering in mappings.
override int opCmp(Object o) @safe
{ {
TestClass s = cast(TestClass)o; auto pairs = [Node.Pair("x", x),
if(s is null){return -1;} Node.Pair("y", y),
if(x != s.x){return x - s.x;} Node.Pair("z", z)];
if(y != s.y){return y - s.y;} return Node(pairs, "!tag1");
if(z != s.z){return z - s.z;}
return 0;
}
override string toString() @safe
{
return format("TestClass(", x, ", ", y, ", ", z, ")");
} }
} }
@ -351,43 +342,22 @@ struct TestStruct
{ {
int value; int value;
//Any D:YAML type must have a custom opCmp operator. this (int x) @safe
//This is used for ordering in mappings.
int opCmp(ref const TestStruct s) const @safe
{ {
return value - s.value; value = x;
}
}
///Constructor function for TestClass.
TestClass constructClass(ref Node node) @safe
{
return new TestClass(node["x"].as!int, node["y"].as!int, node["z"].as!int);
}
Node representClass(ref Node node, Representer representer) @safe
{
auto value = node.as!TestClass;
auto pairs = [Node.Pair("x", value.x),
Node.Pair("y", value.y),
Node.Pair("z", value.z)];
auto result = representer.representMapping("!tag1", pairs);
return result;
} }
///Constructor function for TestStruct. ///Constructor function for TestStruct.
TestStruct constructStruct(ref Node node) @safe this(ref Node node) @safe
{ {
return TestStruct(to!int(node.as!string)); value = node.as!string.to!int;
} }
///Representer function for TestStruct. ///Representer function for TestStruct.
Node representStruct(ref Node node, Representer representer) @safe Node opCast(T: Node)() @safe
{ {
string[] keys, values; return Node(value.to!string, "!tag2");
auto value = node.as!TestStruct; }
return representer.representScalar("!tag2", to!string(value.value));
} }
/** /**
@ -403,12 +373,7 @@ void testConstructor(string dataFilename, string codeDummy) @safe
enforce((base in expected) !is null, enforce((base in expected) !is null,
new Exception("Unimplemented constructor test: " ~ base)); new Exception("Unimplemented constructor test: " ~ base));
auto constructor = new Constructor;
constructor.addConstructorMapping("!tag1", &constructClass);
constructor.addConstructorScalar("!tag2", &constructStruct);
auto loader = Loader.fromFile(dataFilename); auto loader = Loader.fromFile(dataFilename);
loader.constructor = constructor;
loader.resolver = new Resolver; loader.resolver = new Resolver;
Node[] exp = expected[base]; Node[] exp = expected[base];

3
source/dyaml/test/emitter.d
View file

@ -90,7 +90,6 @@ void testEmitterOnData(string dataFilename, string canonicalFilename) @safe
auto loader2 = Loader.fromString(emitStream.data); auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST"; loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver; loader2.resolver = new Resolver;
auto newEvents = loader2.parse(); auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents)); assert(compareEvents(events, newEvents));
@ -119,7 +118,6 @@ void testEmitterOnCanonical(string canonicalFilename) @safe
} }
auto loader2 = Loader.fromString(emitStream.data); auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST"; loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver; loader2.resolver = new Resolver;
auto newEvents = loader2.parse(); auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents)); assert(compareEvents(events, newEvents));
@ -177,7 +175,6 @@ void testEmitterStyles(string dataFilename, string canonicalFilename) @safe
} }
auto loader2 = Loader.fromString(emitStream.data); auto loader2 = Loader.fromString(emitStream.data);
loader2.name = "TEST"; loader2.name = "TEST";
loader2.constructor = new Constructor;
loader2.resolver = new Resolver; loader2.resolver = new Resolver;
auto newEvents = loader2.parse(); auto newEvents = loader2.parse();
assert(compareEvents(events, newEvents)); assert(compareEvents(events, newEvents));

8
source/dyaml/test/representer.d
View file

@ -53,21 +53,13 @@ void testRepresenterTypes(string codeFilename) @safe
} }
auto emitStream = new Appender!(immutable(encoding)[]); auto emitStream = new Appender!(immutable(encoding)[]);
auto representer = new Representer;
representer.addRepresenter!TestClass(&representClass);
representer.addRepresenter!TestStruct(&representStruct);
auto dumper = dumper(emitStream); auto dumper = dumper(emitStream);
dumper.representer = representer;
dumper.dump!encoding(expectedNodes); dumper.dump!encoding(expectedNodes);
output = emitStream.data; output = emitStream.data;
auto constructor = new Constructor;
constructor.addConstructorMapping("!tag1", &constructClass);
constructor.addConstructorScalar("!tag2", &constructStruct);
auto loader = Loader.fromString(emitStream.data.toUTF8); auto loader = Loader.fromString(emitStream.data.toUTF8);
loader.name = "TEST"; loader.name = "TEST";
loader.constructor = constructor;
readNodes = loader.array; readNodes = loader.array;
assert(expectedNodes.length == readNodes.length); assert(expectedNodes.length == readNodes.length);

13
source/dyaml/test/resolver.d
View file

@ -16,12 +16,6 @@ import std.string;
import dyaml.test.common; import dyaml.test.common;
string construct(ref Node node) @safe
{
return node.as!string;
}
/** /**
* Implicit tag resolution unittest. * Implicit tag resolution unittest.
* *
@ -44,12 +38,7 @@ void testImplicitResolver(string dataFilename, string detectFilename) @safe
correctTag = readText(detectFilename).strip(); correctTag = readText(detectFilename).strip();
auto constructor = new Constructor; node = Loader.fromFile(dataFilename).load();
constructor.addConstructorScalar("tag:example.com,2000:app/tag🤔", &construct);
auto loader = Loader.fromFile(dataFilename);
loader.constructor = constructor;
node = loader.load();
assert(node.isSequence); assert(node.isSequence);
foreach(ref Node scalar; node) foreach(ref Node scalar; node)
{ {

1
test/data/empty-python-module.loader-error
View file

@ -1 +0,0 @@
--- !!python:module:

1
test/data/empty-python-name.loader-error
View file

@ -1 +0,0 @@
--- !!python/name: empty

1
test/data/undefined-constructor.loader-error
View file

@ -1 +0,0 @@
--- !foo bar