|
|
|
|
|
==========================
|
|
|
|
|
|
Docutils_ Hacker's Guide
|
|
|
|
|
|
==========================
|
|
|
|
|
|
|
|
|
|
|
|
:Author: Lea Wiemann
|
|
|
|
|
|
:Contact: docutils-develop@lists.sourceforge.net
|
|
|
|
|
|
:Revision: $Revision: 7302 $
|
|
|
|
|
|
:Date: $Date: 2012-01-03 20:23:53 +0100 (Di, 03 Jan 2012) $
|
|
|
|
|
|
:Copyright: This document has been placed in the public domain.
|
|
|
|
|
|
|
|
|
|
|
|
:Abstract: This is the introduction to Docutils for all persons who
|
|
|
|
|
|
want to extend Docutils in some way.
|
|
|
|
|
|
:Prerequisites: You have used reStructuredText_ and played around with
|
|
|
|
|
|
the `Docutils front-end tools`_ before. Some (basic) Python
|
|
|
|
|
|
knowledge is certainly helpful (though not necessary, strictly
|
|
|
|
|
|
speaking).
|
|
|
|
|
|
|
|
|
|
|
|
.. _Docutils: http://docutils.sourceforge.net/
|
|
|
|
|
|
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
|
|
|
|
|
|
.. _Docutils front-end tools: ../user/tools.html
|
|
|
|
|
|
|
|
|
|
|
|
.. contents::
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Overview of the Docutils Architecture
|
|
|
|
|
|
=====================================
|
|
|
|
|
|
|
|
|
|
|
|
To give you an understanding of the Docutils architecture, we'll dive
|
|
|
|
|
|
right into the internals using a practical example.
|
|
|
|
|
|
|
|
|
|
|
|
Consider the following reStructuredText file::
|
|
|
|
|
|
|
|
|
|
|
|
My *favorite* language is Python_.
|
|
|
|
|
|
|
|
|
|
|
|
.. _Python: http://www.python.org/
|
|
|
|
|
|
|
|
|
|
|
|
Using the ``rst2html.py`` front-end tool, you would get an HTML output
|
|
|
|
|
|
which looks like this::
|
|
|
|
|
|
|
|
|
|
|
|
[uninteresting HTML code removed]
|
|
|
|
|
|
<body>
|
|
|
|
|
|
<div class="document">
|
|
|
|
|
|
<p>My <em>favorite</em> language is <a class="reference" href="http://www.python.org/">Python</a>.</p>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</body>
|
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
|
|
While this looks very simple, it's enough to illustrate all internal
|
|
|
|
|
|
processing stages of Docutils. Let's see how this document is
|
|
|
|
|
|
processed from the reStructuredText source to the final HTML output:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Reading the Document
|
|
|
|
|
|
--------------------
|
|
|
|
|
|
|
|
|
|
|
|
The **Reader** reads the document from the source file and passes it
|
|
|
|
|
|
to the parser (see below). The default reader is the standalone
|
|
|
|
|
|
reader (``docutils/readers/standalone.py``) which just reads the input
|
|
|
|
|
|
data from a single text file. Unless you want to do really fancy
|
|
|
|
|
|
things, there is no need to change that.
|
|
|
|
|
|
|
|
|
|
|
|
Since you probably won't need to touch readers, we will just move on
|
|
|
|
|
|
to the next stage:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Parsing the Document
|
|
|
|
|
|
--------------------
|
|
|
|
|
|
|
|
|
|
|
|
The **Parser** analyzes the the input document and creates a **node
|
|
|
|
|
|
tree** representation. In this case we are using the
|
|
|
|
|
|
**reStructuredText parser** (``docutils/parsers/rst/__init__.py``).
|
|
|
|
|
|
To see what that node tree looks like, we call ``quicktest.py`` (which
|
|
|
|
|
|
can be found in the ``tools/`` directory of the Docutils distribution)
|
|
|
|
|
|
with our example file (``test.txt``) as first parameter (Windows users
|
|
|
|
|
|
might need to type ``python quicktest.py test.txt``)::
|
|
|
|
|
|
|
|
|
|
|
|
$ quicktest.py test.txt
|
|
|
|
|
|
<document source="test.txt">
|
|
|
|
|
|
<paragraph>
|
|
|
|
|
|
My
|
|
|
|
|
|
<emphasis>
|
|
|
|
|
|
favorite
|
|
|
|
|
|
language is
|
|
|
|
|
|
<reference name="Python" refname="python">
|
|
|
|
|
|
Python
|
|
|
|
|
|
.
|
|
|
|
|
|
<target ids="python" names="python" refuri="http://www.python.org/">
|
|
|
|
|
|
|
|
|
|
|
|
Let us now examine the node tree:
|
|
|
|
|
|
|
|
|
|
|
|
The top-level node is ``document``. It has a ``source`` attribute
|
|
|
|
|
|
whose value is ``text.txt``. There are two children: A ``paragraph``
|
|
|
|
|
|
node and a ``target`` node. The ``paragraph`` in turn has children: A
|
|
|
|
|
|
text node ("My "), an ``emphasis`` node, a text node (" language is "),
|
|
|
|
|
|
a ``reference`` node, and again a ``Text`` node (".").
|
|
|
|
|
|
|
|
|
|
|
|
These node types (``document``, ``paragraph``, ``emphasis``, etc.) are
|
|
|
|
|
|
all defined in ``docutils/nodes.py``. The node types are internally
|
|
|
|
|
|
arranged as a class hierarchy (for example, both ``emphasis`` and
|
|
|
|
|
|
``reference`` have the common superclass ``Inline``). To get an
|
|
|
|
|
|
overview of the node class hierarchy, use epydoc (type ``epydoc
|
|
|
|
|
|
nodes.py``) and look at the class hierarchy tree.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Transforming the Document
|
|
|
|
|
|
-------------------------
|
|
|
|
|
|
|
|
|
|
|
|
In the node tree above, the ``reference`` node does not contain the
|
|
|
|
|
|
target URI (``http://www.python.org/``) yet.
|
|
|
|
|
|
|
|
|
|
|
|
Assigning the target URI (from the ``target`` node) to the
|
|
|
|
|
|
``reference`` node is *not* done by the parser (the parser only
|
|
|
|
|
|
translates the input document into a node tree).
|
|
|
|
|
|
|
|
|
|
|
|
Instead, it's done by a **Transform**. In this case (resolving a
|
|
|
|
|
|
reference), it's done by the ``ExternalTargets`` transform in
|
|
|
|
|
|
``docutils/transforms/references.py``.
|
|
|
|
|
|
|
|
|
|
|
|
In fact, there are quite a lot of Transforms, which do various useful
|
|
|
|
|
|
things like creating the table of contents, applying substitution
|
|
|
|
|
|
references or resolving auto-numbered footnotes.
|
|
|
|
|
|
|
|
|
|
|
|
The Transforms are applied after parsing. To see how the node tree
|
|
|
|
|
|
has changed after applying the Transforms, we use the
|
|
|
|
|
|
``rst2pseudoxml.py`` tool:
|
|
|
|
|
|
|
|
|
|
|
|
.. parsed-literal::
|
|
|
|
|
|
|
|
|
|
|
|
$ rst2pseudoxml.py test.txt
|
|
|
|
|
|
<document source="test.txt">
|
|
|
|
|
|
<paragraph>
|
|
|
|
|
|
My
|
|
|
|
|
|
<emphasis>
|
|
|
|
|
|
favorite
|
|
|
|
|
|
language is
|
|
|
|
|
|
<reference name="Python" **refuri="http://www.python.org/"**>
|
|
|
|
|
|
Python
|
|
|
|
|
|
.
|
|
|
|
|
|
<target ids="python" names="python" ``refuri="http://www.python.org/"``>
|
|
|
|
|
|
|
|
|
|
|
|
For our small test document, the only change is that the ``refname``
|
|
|
|
|
|
attribute of the reference has been replaced by a ``refuri``
|
|
|
|
|
|
attribute |---| the reference has been resolved.
|
|
|
|
|
|
|
|
|
|
|
|
While this does not look very exciting, transforms are a powerful tool
|
|
|
|
|
|
to apply any kind of transformation on the node tree.
|
|
|
|
|
|
|
|
|
|
|
|
By the way, you can also get a "real" XML representation of the node
|
|
|
|
|
|
tree by using ``rst2xml.py`` instead of ``rst2pseudoxml.py``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Writing the Document
|
|
|
|
|
|
--------------------
|
|
|
|
|
|
|
|
|
|
|
|
To get an HTML document out of the node tree, we use a **Writer**, the
|
|
|
|
|
|
HTML writer in this case (``docutils/writers/html4css1.py``).
|
|
|
|
|
|
|
|
|
|
|
|
The writer receives the node tree and returns the output document.
|
|
|
|
|
|
For HTML output, we can test this using the ``rst2html.py`` tool::
|
|
|
|
|
|
|
|
|
|
|
|
$ rst2html.py --link-stylesheet test.txt
|
|
|
|
|
|
<?xml version="1.0" encoding="utf-8" ?>
|
|
|
|
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
|
|
|
|
<head>
|
|
|
|
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
|
|
|
|
<meta name="generator" content="Docutils 0.3.10: http://docutils.sourceforge.net/" />
|
|
|
|
|
|
<title></title>
|
|
|
|
|
|
<link rel="stylesheet" href="../docutils/writers/html4css1/html4css1.css" type="text/css" />
|
|
|
|
|
|
</head>
|
|
|
|
|
|
<body>
|
|
|
|
|
|
<div class="document">
|
|
|
|
|
|
<p>My <em>favorite</em> language is <a class="reference" href="http://www.python.org/">Python</a>.</p>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</body>
|
|
|
|
|
|
</html>
|
|
|
|
|
|
|
|
|
|
|
|
So here we finally have our HTML output. The actual document contents
|
|
|
|
|
|
are in the fourth-last line. Note, by the way, that the HTML writer
|
|
|
|
|
|
did not render the (invisible) ``target`` node |---| only the
|
|
|
|
|
|
``paragraph`` node and its children appear in the HTML output.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Extending Docutils
|
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
|
|
Now you'll ask, "how do I actually extend Docutils?"
|
|
|
|
|
|
|
|
|
|
|
|
First of all, once you are clear about *what* you want to achieve, you
|
|
|
|
|
|
have to decide *where* to implement it |---| in the Parser (e.g. by
|
|
|
|
|
|
adding a directive or role to the reStructuredText parser), as a
|
|
|
|
|
|
Transform, or in the Writer. There is often one obvious choice among
|
|
|
|
|
|
those three (Parser, Transform, Writer). If you are unsure, ask on
|
|
|
|
|
|
the Docutils-develop_ mailing list.
|
|
|
|
|
|
|
|
|
|
|
|
In order to find out how to start, it is often helpful to look at
|
|
|
|
|
|
similar features which are already implemented. For example, if you
|
|
|
|
|
|
want to add a new directive to the reStructuredText parser, look at
|
|
|
|
|
|
the implementation of a similar directive in
|
|
|
|
|
|
``docutils/parsers/rst/directives/``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Modifying the Document Tree Before It Is Written
|
|
|
|
|
|
------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
You can modify the document tree right before the writer is called.
|
|
|
|
|
|
One possibility is to use the publish_doctree_ and
|
|
|
|
|
|
publish_from_doctree_ functions.
|
|
|
|
|
|
|
|
|
|
|
|
To retrieve the document tree, call::
|
|
|
|
|
|
|
|
|
|
|
|
document = docutils.core.publish_doctree(...)
|
|
|
|
|
|
|
|
|
|
|
|
Please see the docstring of publish_doctree for a list of parameters.
|
|
|
|
|
|
|
|
|
|
|
|
.. XXX Need to write a well-readable list of (commonly used) options
|
|
|
|
|
|
of the publish_* functions. Probably in api/publisher.txt.
|
|
|
|
|
|
|
|
|
|
|
|
``document`` is the root node of the document tree. You can now
|
|
|
|
|
|
change the document by accessing the ``document`` node and its
|
|
|
|
|
|
children |---| see `The Node Interface`_ below.
|
|
|
|
|
|
|
|
|
|
|
|
When you're done with modifying the document tree, you can write it
|
|
|
|
|
|
out by calling::
|
|
|
|
|
|
|
|
|
|
|
|
output = docutils.core.publish_from_doctree(document, ...)
|
|
|
|
|
|
|
|
|
|
|
|
.. _publish_doctree: ../api/publisher.html#publish_doctree
|
|
|
|
|
|
.. _publish_from_doctree: ../api/publisher.html#publish_from_doctree
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The Node Interface
|
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
|
|
As described in the overview above, Docutils' internal representation
|
|
|
|
|
|
of a document is a tree of nodes. We'll now have a look at the
|
|
|
|
|
|
interface of these nodes.
|
|
|
|
|
|
|
|
|
|
|
|
(To be completed.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
What Now?
|
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
|
|
This document is not complete. Many topics could (and should) be
|
|
|
|
|
|
covered here. To find out with which topics we should write about
|
|
|
|
|
|
first, we are awaiting *your* feedback. So please ask your questions
|
|
|
|
|
|
on the Docutils-develop_ mailing list.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _Docutils-develop: ../user/mailing-lists.html#docutils-develop
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. |---| unicode:: 8212 .. em-dash
|
|
|
|
|
|
:trim:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
..
|
|
|
|
|
|
Local Variables:
|
|
|
|
|
|
mode: indented-text
|
|
|
|
|
|
indent-tabs-mode: nil
|
|
|
|
|
|
sentence-end-double-space: t
|
|
|
|
|
|
fill-column: 70
|
|
|
|
|
|
End:
|
