debian-packages/dyaml
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Initial commit.

Browse source
This commit is contained in:
Ferdinand Majerech 2011-08-16 14:53:13 +02:00
commit 283c42bf8f
592 changed files with 26392 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

227
docsrc/tutorials/custom_types.rst Normal file
View file

@ -0,0 +1,227 @@
======================
Custom YAML data types
======================
Often you will want to serialize complex data types such as classes. You can use
functions to process nodes; e.g. a mapping containing class data members indexed
by name. Alternatively, YAML supports custom data types using identifiers called
*tags*. That is the topic of this tutorial.
Each YAML node has a tag specifying its type. For instance: strings use the tag
``tag:yaml.org,2002:str``. Tags of most default types are *implicitly resolved*
during parsing, so you don't need to specify tag for each float, integer, etc.
It is also possible to implicitly resolve custom tags, as we will show later.
-----------
Constructor
-----------
D:YAML uses the *Constructor* class to process each node to hold data type
corresponding to its tag. *Constructor* stores a function for each supported
tag to process it. These functions can be supplied by the user using the
*addConstructor()* method. *Constructor* is then passed to *Loader*, which will
parse YAML input.
We will implement support for an RGB color type. It is implemented as the
following struct:
.. code-block:: d
struct Color
{
ubyte red;
ubyte green;
ubyte blue;
}
First, we need a function to construct our data type. It must take two *Mark*
structs, which store position of the node in the file, and either a *string*, an
array of *Node* or of *Node.Pair*, depending on whether we're constructing our
value from a scalar, sequence, or mapping, respectively. In this tutorial, we
have functions to construct a color from a scalar, using HTML-like format,
RRGGBB, or from a mapping, where we use the following format:
{r:RRR, g:GGG, b:BBB} . Code of these functions:
.. code-block:: d
Color constructColorScalar(Mark start, Mark end, string value)
{
if(value.length != 6)
{
throw new ConstructorException("Invalid color: " ~ value, start, end);
}
//We don't need to check for uppercase chars this way.
value = value.toLower();
//Get value of a hex digit.
uint hex(char c)
{
if(!std.ascii.isHexDigit(c))
{
throw new ConstructorException("Invalid color: " ~ value, start, end);
}
if(std.ascii.isDigit(c))
{
return c - '0';
}
return c - 'a' + 10;
}
Color result;
result.red = cast(ubyte)(16 * hex(value[0]) + hex(value[1]));
result.green = cast(ubyte)(16 * hex(value[2]) + hex(value[3]));
result.blue = cast(ubyte)(16 * hex(value[4]) + hex(value[5]));
return result;
}
Color constructColorMapping(Mark start, Mark end, Node.Pair[] pairs)
{
int r, g, b;
r = g = b = -1;
bool error = pairs.length != 3;
foreach(ref pair; pairs)
{
//Key might not be a string, and value might not be an int,
//so we need to check for that
try
{
switch(pair.key.get!string)
{
case "r": r = pair.value.get!int; break;
case "g": g = pair.value.get!int; break;
case "b": b = pair.value.get!int; break;
default: error = true;
}
}
catch(NodeException e)
{
error = true;
}
}
if(error || r < 0 || r > 255 || g < 0 || g > 255 || b < 0 || b > 255)
{
throw new ConstructorException("Invalid color", start, end);
}
return Color(cast(ubyte)r, cast(ubyte)g, cast(ubyte)b);
}
Next, we need some YAML code using our new tag. Create a file called input.yaml
with the following contents:
.. code-block:: yaml
scalar-red: !color FF0000
scalar-orange: !color FFFF00
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
mapping-orange:
!color-mapping
r: 255
g: 255
b: 0
You can see that we're using tag ``!color`` for scalar colors, and
``!color-mapping`` for colors expressed as mappings.
Finally, the code to put it all together:
.. code-block:: d
void main()
{
auto red = Color(255, 0, 0);
auto orange = Color(255, 255, 0);
try
{
auto constructor = new Constructor;
//both functions handle the same tag, but one handles scalar, one mapping.
constructor.addConstructor("!color", &constructColorScalar);
constructor.addConstructor("!color-mapping", &constructColorMapping);
auto loader = new Loader("input.yaml", constructor, new Resolver);
auto root = loader.loadSingleDocument();
if(root["scalar-red"].get!Color == red &&
root["mapping-red"].get!Color == red &&
root["scalar-orange"].get!Color == orange &&
root["mapping-orange"].get!Color == orange)
{
writeln("SUCCESS");
return;
}
}
catch(YAMLException e)
{
writeln(e.msg);
}
writeln("FAILURE");
}
First, we create a *Constructor* and pass functions to handle the ``!color``
and ``!color-mapping`` tag. We construct a *Loader* using the *Constructor*.
We also need a *Resolver*, but for now we just default-construct it. We then
load the YAML document, and finally, read the colors using *get()* method to
test if they were loaded as expected.
You can find the source code for what we've done so far in the
``examples/constructor`` directory in the D:YAML package.
--------
Resolver
--------
Specifying tag for every color value can be tedious. D:YAML can implicitly
resolve tag of a scalar using a regular expression. This is how default types,
e.g. int, are resolved. We will use the *Resolver* class to add implicit tag
resolution for the Color data type (in its scalar form).
We use the *addImplicitResolver* method of *Resolver*, passing the tag, regular
expression the value must match to resolve to this tag, and a string of possible
starting characters of the value. Then we pass the *Resolver* to the constructor
of *Loader*.
Note that resolvers added first override ones added later. If no resolver
matches a scalar, YAML string tag is used. Therefore our custom values must not
be resolvable as any non-string YAML data type.
Add this to your code to add implicit resolution of ``!color``.
.. code-block:: d
//code from the previous example...
auto resolver = new Resolver;
resolver.addImplicitResolver("!color", std.regex.regex("[0-9a-fA-F]{6}",
"0123456789abcdefABCDEF"));
auto loader = new Loader("input.yaml", constructor, resolver);
//code from the previous example...
Now, change contents of input.dyaml to this:
.. code-block:: yaml
scalar-red: FF0000
scalar-orange: FFFF00
mapping-red: !color-mapping {r: 255, g: 0, b: 0}
mapping-orange:
!color-mapping
r: 255
g: 255
b: 0
We no longer need to specify the tag for scalar color values. Compile and test
the example. If everything went as expected, it should report success.
You can find the complete code in the ``examples/resolver`` directory in the
D:YAML package.

168
docsrc/tutorials/getting_started.rst Normal file
View file

@ -0,0 +1,168 @@
===============
Getting started
===============
Welcome to D:YAML! D:YAML is a `YAML <http://en.wikipedia.org/wiki/YAML>`_ parser
library for the `D programming language <http://d-p-l.org>`_. This tutorial will
explain how to set D:YAML up and use it in your projects.
This is meant to be the **simplest possible** introduction to D:YAML. Some of the
information present might already be known to you. Only basic usage is covered.
More advanced usage is described in other tutorials.
----------
Setting up
----------
^^^^^^^^^^^^^^^^^^^^^^^^
Install the DMD compiler
^^^^^^^^^^^^^^^^^^^^^^^^
Digital Mars D compiler, or DMD, is the most commonly used D compiler. You can
find its newest version `here <http://www.digitalmars.com/d/download.html>`_.
Download the version of DMD for your operating system and install it.
.. note::
Other D compilers exist, such as
`GDC <http://bitbucket.org/goshawk/gdc/wiki/Home>`_ and
`LDC <http://www.dsource.org/projects/ldc/>`_.
Setting up with either one of them should be similar to DMD,
however, at the moment they are not as up to date as DMD.
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Download and compile D:YAML
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The newest version of D:YAML can be found `here <TODO>`_. Download a source
archive, extract it, and move to the extracted directory.
D:YAML uses a modified version of the `CDC <http://dsource.org/projects/cdc/>`_
script for compilation. To compile D:YAML, you first need to build CDC.
Do this by typing the following command into the console::
dmd cdc.d
Now you can use CDC to compile D:YAML.
To do this on Unix/Linux, use the following command::
./cdc
On Windows::
cdc.exe
This will compile the library to a file called ``libdyaml.a`` on Unix/Linux or
``libdyaml.lib`` on Windows.
-------------------------
Your first D:YAML project
-------------------------
Create a directory for your project and in that directory, create a file called
``input.yaml`` with the following contents:
.. code-block:: yaml
Hello World :
- Hello
- World
Answer: 42
This will serve as input for our example.
Now we need to parse it. Create a file called "main.d". Paste following code
into the file:
.. code-block:: d
import std.stdio;
import yaml;
void main()
{
yaml.Node root = yaml.load("input.yaml");
foreach(string word; root["Hello World"])
{
writeln(word);
}
writeln("The answer is ", root["Answer"].get!int);
}
^^^^^^^^^^^^^^^^^^^^^^^
Explanation of the code
^^^^^^^^^^^^^^^^^^^^^^^
First, we import the *yaml* module. This is the only module you need to import
to use D:YAML - it automatically imports all needed modules.
Next we load the file using the *yaml.load()* function - this loads the file as
**one** YAML document and throws *YAMLException*, D:YAML exception type, if the
file could not be parsed or does not contain exactly one document. Note that we
don't do any error checking here in order to keep the example as simple as
possible.
*yaml.Node* represents a node in a YAML document. It can be a sequence (array),
mapping (associative array) or a scalar (value). Here the root node is a
mapping, and we use the index operator to get subnodes with keys "Hello World"
and "Answer". We iterate over the first, as it is a sequence, and use the
*yaml.Node.get()* method on the second to get its value as an integer.
You can iterate over a mapping or sequence as if it was an associative or normal
array. If you try to iterate over a scalar, it will throw a *YAMLException*.
You can iterate over subnodes using yaml.Node as the iterated type, or specify
the type subnodes are expected to have. D:YAML will automatically convert
iterated subnodes to that type if possible. Here we specify the *string* type,
so we iterate over the "Hello World" sequence as an array of strings. If it is
not possible to convert to iterated type, a *YAMLException* is thrown. For
instance, if we specified *int* here, we would get an error, as "Hello"
cannot be converted to an integer.
The *yaml.Node.get()* method is used to get value of a scalar node as specified
type. D:YAML will try to return the scalar as specified type, converting if
needed, throwing *YAMLException* if not possible.
^^^^^^^^^
Compiling
^^^^^^^^^
To compile your project, you must give DMD the directories containing import
modules and the library. You also need to tell it to link with D:YAML. The import
directory should be the D:YAML package directory. You can specify it using the
``-I`` option of DMD. The library directory should point to where you put the
compiled D:YAML library. On Unix/Linux you can specify it using the ``-L-L``
option, and link with D:YAML using the ``-L-l`` option. On Windows, the import
directory is used as the library directory. To link with the library on Windows,
just add the path to it relative to the current directory.
For example, if you extracted D:YAML to ``/home/xxx/dyaml`` and compiled it in
that directory, your project is in ``/home/xxx/dyaml-project``, and you are
currently in that directory, you can compile the project with the following
command on Unix/Linux::
dmd -I../dyaml -L-L../dyaml -L-ldyaml main.d
And the following on Windows::
dmd -I../dyaml ../dyaml/libdyaml.lib main.d
This will produce an executable called ``main`` or ``main.exe`` in your
directory. When you run it, it should produce the following output::
Hello
World
The answer is 42
^^^^^^^^^^
Conclusion
^^^^^^^^^^
You should now have a basic idea about how to use D:YAML. To learn more, look at
the `API documentation <../api/index.html>`_ and other tutorials. You can find code for this
example in the ``example/getting_started`` directory in the package.

273
docsrc/tutorials/yaml_syntax.rst Normal file
View file

@ -0,0 +1,273 @@
===========
YAML syntax
===========
This is an introduction to the most common YAML constructs. For more detailed
information, see `PyYAML documentation <http://pyyaml.org/wiki/PyYAMLDocumentation>`_,
which this article is based on,
`Chapter 2 of the YAML specification <http://yaml.org/spec/1.1/#id857168>`_
or the `Wikipedia page <http://en.wikipedia.org/wiki/YAML>`_.
YAML is a data serialization format designed to be as human readable as
possible. YAML is a recursive acronym for "YAML Ain't Markup Language".
YAML is similar to JSON, and in fact, JSON is a subset of YAML 1.2; but YAML has
some more advanced features and is easier to read. However, YAML is also more
difficult to parse (and probably somewhat slower). Data is stored in mappings
(associative arrays), sequences (lists) and scalars (single values). Data
structure hierarchy either depends on indentation (block context, similar to
Python code), or nesting of brackets and braces (flow context, similar to JSON).
YAML comments begin with ``#`` and continue until the end of line.
---------
Documents
---------
A YAML stream consists of one or more documents starting with ``---`` and
optionally ending with ``...`` . If there is only one document, ``---`` can be
left out.
Single document with no explicit start or end:
.. code-block:: yaml
- Red
- Green
- Blue
Same document with explicit start and end:
.. code-block:: yaml
---
- Red
- Green
- Blue
...
A stream containing multiple documents:
.. code-block:: yaml
---
- Red
- Green
- Blue
---
- Linux
- BSD
---
answer : 42
---------
Sequences
---------
Sequences are arrays of nodes of any type, similar e.g. to Python lists.
In block context, each item begins with hyphen+space "- ". In flow context,
sequences have syntax similar to D arrays.
.. code-block:: yaml
#Block context
- Red
- Green
- Blue
.. code-block:: yaml
#Flow context
[Red, Green, Blue]
.. code-block:: yaml
#Nested
-
- Red
- Green
- Blue
-
- Linux
- BSD
.. code-block:: yaml
#Nested flow
[[Red, Green, Blue], [Linux, BSD]]
.. code-block:: yaml
#Nested in a mapping
Colors:
- Red
- Green
- Blue
Operating systems:
- Linux
- BSD
--------
Mappings
--------
Mappings are associative arrays where each key and value can be of any type,
similar e.g. to Python dictionaries. In block context, keys and values are
separated by colon+space ": ". In flow context, mappings have syntax similar
to D associative arrays, but with braces instead of brackets:
.. code-block:: yaml
#Block context
CPU: Athlon
GPU: Radeon
OS: Linux
.. code-block:: yaml
#Flow context
{CPU: Athlon, GPU: Radeon, OS: Linux}
.. code-block:: yaml
#Nested
PC:
CPU: Athlon
GPU: Radeon
OS: Debian
Phone:
CPU: Cortex
GPU: PowerVR
OS: Android
.. code-block:: yaml
#Nested flow
{PC: {CPU: Athlon, GPU: Radeon, OS: Debian},
Phone: {CPU: Cortex, GPU: PowerVR, OS: Android}}
.. code-block:: yaml
#Nested in a sequence
- CPU: Athlon
GPU: Radeon
OS: Debian
- CPU: Cortex
GPU: PowerVR
OS: Android
Complex keys start with question mark+space "? ".
.. code-block:: yaml
#Nested in a sequence
? [CPU, GPU]: [Athlon, Radeon]
OS: Debian
-------
Scalars
-------
Scalars are simple values such as integers, strings, timestamps and so on.
There are multiple scalar styles.
Plain scalars use no quotes, start with the first non-space and end with the
last non-space character:
.. code-block:: yaml
scalar: Plain scalar
Single quoted scalars start and end with single quotes. A single quote is
represented by a pair of single quotes ''.
.. code-block:: yaml
scalar: 'Single quoted scalar ending with some spaces '
Double quoted scalars support C-style escape sequences.
.. code-block:: yaml
scalar: "Double quoted scalar \n with some \\ escape sequences"
Block scalars are convenient for multi-line values. They start either with
``|`` or with ``>``. With ``|``, the newlines in the scalar are preserved.
With ``>``, the newlines between two non-empty lines are removed.
.. code-block:: yaml
scalar: |
Newlines are preserved
First line
Second line
.. code-block:: yaml
scalar: >
Newlines are folded
This is still the first paragraph
This is the second
paragraph
-------------------
Anchors and aliases
-------------------
Anchors and aliases can reduce size of YAML code by allowing you to define a
value once, assign an anchor to it and use alias referring to that anchor
anywhere else you need that value. It is possible to use this to create
recursive data structures and some parsers support this; however, D:YAML does
not (this might change in the future, but it is unlikely).
.. code-block:: yaml
Person: &AD
gender: male
name: Arthur Dent
Clone: *AD
----
Tags
----
Tags are identifiers that specify data types of YAML nodes. Most default YAML
tags are resolved implicitly, so there is no need to specify them. D:YAML also
supports implicit resolution for custom, user specified tags.
Explicitly specified tags:
.. code-block:: yaml
answer: !!int "42"
name: !!str "Arthur Dent"
Implicit tags:
.. code-block:: yaml
answer: 42 #int
name: Arthur Dent #string
This table shows D types stored in *yaml.Node* default YAML tags are converted to.
Some of these might change in the future (especially !!map and !!set).
====================== ================
YAML tag D type
====================== ================
!!null yaml.YAMLNull
!!bool bool
!!int long
!!float real
!!binary ubyte[]
!!timestamp datetime.SysTime
!!map, !!omap, !!pairs Node.Pair[]
!!seq, !!set Node[]
!!str string
====================== ================