<?xml version='1.0' encoding='iso-8859-1'?>
<par>&xist; is an extensible &html;/&xml; generator written in Python.
It was developed as a replacement for an &html; preprocessor named
<app moreinfo="http://www.linguistik.uni-erlangen.de/~msbethke/hsc/">HSC</app>
and borrows some features and ideas from it. It also borrows the basic ideas
(&xml;/&html; elements as Python objects) from
<app moreinfo="http://starship.python.net/crew/friedrich/HTMLgen/html/main.html">HTMLgen</app>
or <app moreinfo="http://dustman.net/andy/python/HyperText/">HyperText</app>.</par>

<par>(If you're impatient, there's also a
<link href="Examples.html">list of examples</link> that shows what can
be done with &xist;.)</par>

<section><title>Overview</title>
<par>&xist; can be used as a compiler that reads an input &xml; file and
generates a transformed output file, or it could be used for generating &xml;
dynamically inside a web server (but note that handling object trees <em>is</em>
slower than simply sending string fragments). In either case generating the
final &html; or &xml; output requires the following three steps:</par>
<ulist>
<item>Generating a source &xml; tree: This can be done either by
parsing an &xml; file, or by directly constructing the
tree &mdash; as <app>HTMLgen</app> and <app>HyperText</app>
do &mdash; as a tree of Python objects. &xist; provides a very natural
and pythonic &api; for that.</item>
<item>Converting the source tree into a target tree: This target
tree can be a &html; tree or a &svg; tree or XSL-FO tree or any other
&xml; tree you like. Every node class provides a
<method>convert</method> method for performing this conversion. For
your own &xml; element types you have to define your
own element classes and implement an appropriate
<method>convert</method> method. This is possible for
processing instructions and entity references too.</item>
<item>Publishing the target tree: For generating the final
output a <class>Publisher</class> object is used that generates
the encoded byte string fragments that can be written to an output stream
(or yielded from a WSGI application, etc.).</item>
</ulist>
</section>


<section><title>Constructing &xml; trees</title>
<par>Like any other &xml; tree &api;, &xist; provides the usual classes:</par>
<ulist>
<item><pyref module="ll.xist.xsc" class="Element"><class>Element</class></pyref> for &xml; elements;</item>
<item><pyref module="ll.xist.xsc" class="Attr"><class>Attr</class></pyref> for attributes;</item>
<item><pyref module="ll.xist.xsc" class="Attrs"><class>Attrs</class></pyref> for attribute mappings;</item>
<item><pyref module="ll.xist.xsc" class="Text"><class>Text</class></pyref> for text data;</item>
<item><pyref module="ll.xist.xsc" class="Frag"><class>Frag</class></pyref> for document fragments,
(a <pyref module="ll.xist.xsc" class="Frag"><class>Frag</class></pyref> object is simply a list
of nodes);</item>
<item><pyref module="ll.xist.xsc" class="Comment"><class>Comment</class></pyref> for &xml; comments
(e.g. <markup>&lt;!-- the comment --&gt;</markup>);</item>
<item><pyref module="ll.xist.xsc" class="ProcInst"><class>ProcInst</class></pyref> for processing instructions
(e.g. <markup>&lt;?php echo $spam;?&gt;</markup>);</item>
<item><pyref module="ll.xist.xsc" class="Entity"><class>Entity</class></pyref> for entity references
(e.g. <markup>&amp;parrot;</markup>) and</item>
<item><pyref module="ll.xist.xsc" class="DocType"><class>DocType</class></pyref> for document type
declarations (e.g. <markup>&lt;!DOCTYPE html PUBLIC <rep>...</rep>&gt;</markup>).</item>
</ulist>

<section><title>&xml; trees as Python objects</title>
<par>&xist; works somewhat different from a normal &dom; &api;.
Instead of only one element class, &xist; has one class for every element
type. All the elements from different &xml; vocabularies known to &xist; are
defined in modules in the <pyref module="ll.xist.ns"><module>ll.xist.ns</module></pyref>
subpackage. (Of course it's possible to define additional namespaces for your
own &xml; vocabulary). The definition of &html; can be found in
<pyref module="ll.xist.ns.html"><module>ll.xist.ns.html</module></pyref>
for example.</par>

<par>Every element class has a constructor of the form:</par>
<prog>
__init__(self, *content, **attrs)
</prog>
<par>Positional arguments (i.e. items in <arg>content</arg>)
will be the child nodes of the element node. Keyword arguments will be attributes.
You can pass most of Python's builtin types to such a constructor.
Strings (<class>str</class> and <class>unicode</class>) and integers
will be automatically converted to
<pyref module="ll.xist.xsc" class="Text"><class>Text</class></pyref> objects.
Constructing an &html; element works like this:</par>
<example><title>The first example</title>
<prog>
from ll.xist.ns import html

node = html.div(
	"Hello ",
	html.a("Python", href="http://www.python.org/"),
	" world!"
)
</prog>
</example>

<par>For attribute names that collide with Python keywords
or are not legal identifiers (most notably <lit>class</lit> in &html;)
the attribute name must be slightly modified, so that it's a legal
Python identifier (for <lit>class</lit> an underscore is appended):</par>
<example><title>Illegal attribute names</title>
<prog>
node = html.div(
	"Hello world!",
	class_="greeting"
)
</prog>
</example>
<par>(Don't worry: This modified attribute name will be mapped to the real official attribute
name once the output is generated.)</par>

<par>You can pass attributes as a dictionary too:</par>
<example><title>Passing attributes as dictionaries</title>
<prog>
node = html.div(
	"Hello world!",
	{
		"class_": "greeting",
		"id": 42,
		"title": "Greet the world"
	}
)
</prog>
</example>
</section>

<section><title>Generating &xml; trees from &xml; files</title>
<par>&xml; trees can also be generated by parsing &xml; files.
For this the module <pyref module="ll.xist.parsers"><module>ll.xist.parsers</module></pyref>
provides several functions:</par>
<prog>
def parseString(text, base=None, sysid=None, **parserargs)
def parseURL(url, base=None, sysid=None, **parserargs)
def parseFile(filename, base=None, sysid=None, **parserargs)
def parse(stream, base=None, sysid=None, **parserargs)
</prog>
<par><pyref module="ll.xist.parsers" function="parseString"><function>parseString</function></pyref>
is for parsing strings (<class>str</class> and <class>unicode</class>) and
<pyref module="ll.xist.parsers" function="parseURL"><function>parseURL</function></pyref> is for
parsing resources from &url;s. With
<pyref module="ll.xist.parsers" function="parseFile"><function>parseFile</function></pyref>
you can parse local files and with <pyref module="ll.xist.parsers" function="parse"><function>parse</function></pyref>
you can directly parse from a file-like object.</par>
<par>All four functions create a parser internally, parse the supplied source document and
return the resulting object tree.</par>
<par>For example, parsing a string can be done like this:</par>
<example><title>Parsing a string</title>
<prog>
from ll.xist import parsers
from ll.xist.ns import html

node = parsers.parseString(
	"&lt;p&gt;Hello &lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt; world!&lt;/p&gt;"
)
</prog>
</example>

<par>For further info about the rest of the arguments to the parsing
functions, see the documentation for
<pyref module="ll.xist.parsers" class="Parser"><class>ll.xist.parsers.Parser</class></pyref>.</par>
</section>
</section>


<section><title>Defining new elements and converting &xml; trees</title>

<par>To be able to parse an &xml; file, you have to provide an element class
for every element type that appears in the file. These classes either come from
modules provided by &xist; or you can define your own. Defining your own
element class for an element named <lit>cool</lit> works like this:</par>

<example><title>Defining a new element</title>
<prog>
class cool(xsc.Element):
	def convert(self, converter):
		node = html.b(self.content, u" is cool!")
		return node.convert(converter)
</prog>
</example>

<par>You have to derive your new class from
<pyref module="ll.xist.xsc" class="Element"><class>xsc.Element</class></pyref>.
The name of the class will be the element name. For element type names that
are invalid Python identifiers, you can use the class attribute <lit>xmlname</lit>
in the element class to overwrite the element name.</par>
<par>To be able to convert an element of this type to a new &xml; tree
(probably &html; in most cases), you have to implement the
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
method. In this method you can build a new &xml; tree from the content and attributes
of the object.</par>

<par>Using this new element is simple</par>

<example><title>Using the new element</title>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool("Python")</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.conv().asBytes()</input>
&lt;b&gt;Python is cool!&lt;/b&gt;
</tty>
</example>

<par><pyref module="ll.xist.xsc" class="Node" method="conv"><method>conv</method></pyref> simply
calls
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
with a default <pyref module="ll.xist.converters" class="Converter"><arg>converter</arg></pyref>
argument. We'll come to converters in a minute.
<pyref module="ll.xist.xsc" class="Node" method="asBytes"><method>asBytes</method></pyref>
is a method that converts the node to a byte string. This method will be explained
when we discuss the publishing interface.</par>

<par>Note that it is vital for your own <method>convert</method>
methods that you recursively call <method>convert</method>
on your own content, because otherwise some unconverted nodes
might remain in the tree. Let's define a new element:</par>
<prog>
class python(xsc.Element):
	def convert(self, converter):
		return html.a(u"Python", href=u"http://www.python.org/")
</prog>
<par>Now we can do the following:</par>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool(python())</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.conv().asBytes()</input>
&lt;b&gt;&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt; is cool!&lt;/b&gt;
</tty>
<par>But if we forget to call
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
for our own content, i.e. if the element <pyref class="cool"><class>cool</class></pyref>
was written like this:</par>
<prog>
class cool(xsc.Element):
	def convert(self, converter):
		return html.b(self.content, " is cool!")
</prog>
<par>we would get:</par>
<prog>
&gt;&gt;&gt; node = cool(python())
&gt;&gt;&gt; print node.conv().asBytes()
&lt;b&gt;&lt;python /&gt; is cool!&lt;/b&gt;
</prog>

<par>Furthermore <method>convert</method> should never modify <self/>, because
<method>convert</method> might be called multiple times for the same node.</par>

<section><title>Converters</title>
<par><pyref module="ll.xist.xsc" class="Node" method="conv"><method>conv</method></pyref>
is a convenience method that creates a default converter for you and calls
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>.
This converter is created once and is passed to all <method>convert</method>
calls. It is used to store parameters for the conversion process and it allows
elements to pass information to other nodes. You can also call
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
yourself, which would look like this:</par>

<prog>
from ll.xist import converters
from ll.xist.ns import html

node = cool(python())
node = node.convert(converters.Converter())
</prog>

<par>You can pass the following arguments to the
<pyref module="ll.xist.converters" class="Converter"><class>Converter</class></pyref> constructor:</par>
<dlist>
<term><arg>root</arg></term><item><arg>root</arg> (which defaults to <lit>None</lit>) is the root &url;
for the conversion process. When you want to resolve a link in some of your own <method>convert</method> methods,
the &url; must be interpreted relative to this root &url; (You can use
<pyref module="ll.xist.xsc" class="URLAttr" method="forInput"><method>URLAttr.forInput</method></pyref>
for that).</item>
<term><arg>mode</arg></term><item><arg>mode</arg> (which defaults to <lit>None</lit>) works the same way
as modes in &xslt;. You can use this for implementing different conversion modes.</item>
<term><arg>stage</arg></term><item><arg>stage</arg> (which defaults to <lit>"deliver"</lit>)
allows you to implement multi stage conversion: Suppose that you want to deliver a dynamically
constructed web page with &xist; that contains results from a database query and the current time.
The data in the database changes infrequently, so it doesn't make sense to do the query on
every request. The query is done every few minutes and the resulting &html; tree is
stored in the servlet (using any of the available Python servlet technologies).
For this conversion the <arg>stage</arg> would be <lit>"cache"</lit> and your database &xml;
element would do the query when <lit><arg>stage</arg>=="cache"</lit>. Your time display element
would do the conversion when <lit><arg>stage</arg>=="deliver"</lit> and simply returns itself
when <lit><arg>stage</arg>=="cache"</lit>, so it would still be part of the cached &xml; tree
and would be converted to &html; on every request.</item>
<term><arg>target</arg></term><item><arg>target</arg> (which defaults to
<pyref module="ll.xist.ns.html"><module>ll.xist.ns.html</module></pyref>) specifies what the output should be.
Values must be <pyref module="ll.xist.xsc" class="Namespace">namespace subclasses</pyref>
(see below for an explanation of namespaces).</item>
<term><arg>lang</arg></term><item><par><arg>lang</arg> (which defaults to <lit>None</lit>) is the language
in which the result tree should be. This can be used in the <method>convert</method> method
to implement different conversions for different languages, e.g.:</par>
<prog>
class note(xsc.Element):
	def convert(self, converter):
		if converter.lang==u"de":
			title = u"Anmerkung"
		elif converter.lang==u"en":
			title = u"Note"
		else:
			title = u"???"
		node = xsc.Frag(
			html.h1(title),
			html.div(self.content)
		)
		return node.convert(converter)
</prog>
</item>
</dlist>

<par>Additional arguments are passed when a converter is created in the context of
a <pyref module="ll.make"><module>make</module></pyref> script.</par>

</section>

<section><title>Attributes</title>
<par>Setting and accessing the attributes of an element works via
the dictionary interface:</par>
<example>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.a(u"Python", href=u"http://www.python.org/")</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node[u"href"].asBytes()</input>
http://www.python.org/
<prompt>&gt;&gt;&gt; </prompt><input>del node[u"href"]</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node[u"href"].asBytes()</input>

<prompt>&gt;&gt;&gt; </prompt><input>node[u"href"] = u"http://www.python.org"</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node[u"href"].asBytes()</input>
http://www.python.org/
</tty>
</example>

<par>All attribute values are instances of subclasses of the class
<pyref module="ll.xist.xsc" class="Attr"><class>Attr</class></pyref>.
Available subclasses are:</par>

<ulist>
<item><pyref module="ll.xist.xsc" class="TextAttr"><class>TextAttr</class></pyref>, for normal text attributes;</item>
<item><pyref module="ll.xist.xsc" class="URLAttr"><class>URLAttr</class></pyref>, for attributes that are &url;s;</item>
<item><pyref module="ll.xist.xsc" class="BoolAttr"><class>BoolAttr</class></pyref>, for boolean attributes (for such an attribute
only its presence is important, it's value will always be the same as the attribute name when publishing);</item>
<item><pyref module="ll.xist.xsc" class="IntAttr"><class>IntAttr</class></pyref>, for integer attributes;</item>
<item><pyref module="ll.xist.xsc" class="ColorAttr"><class>ColorAttr</class></pyref>, for color attributes (e.g. <lit>#ffffff</lit>).</item>
</ulist>

<par><class>IntAttr</class> and <class>ColorAttr</class> mostly serve as documentation
of the attributes purpose. Both classes have no added functionality.</par>

<par><pyref module="ll.xist.xsc" class="Attr"><class>Attr</class></pyref> itself is derived from
<pyref module="ll.xist.xsc" class="Frag"><class>Frag</class></pyref> so it is possible
to use all the sequence methods on an attribute.</par>

<par>Unset attributes will be treated like empty ones so the following is possible:</par>

<example>
<prog>
del node["spam"]
node["spam"].append("ham")
</prog>
</example>

<par>This also means that after <lit>del node["spam"][:]</lit> the attribute
will be empty again and will be considered to be unset.
Such attributes will be skipped when publishing.</par>

<par>The main purpose of this is to allow you to construct values conditionally
and then use those values as attribute values:</par>

<prog>
import random

if random.random() &lt; 0.5:
	class_ = None
else:
	class_ = u"foo"

node = html.div(u"foo", class_=class_)
</prog>

<par>In 50% of the cases the generated <class>div</class> element will not
have a <lit>class</lit> attribute.</par>


<section><title>Defining attributes</title>

<par>When you define a new element you have to specify the attributes allowed
for this element. For this use the class attribute <lit>Attrs</lit>
(which must be a class derived from
<pyref module="ll.xist.xsc" class="Element.Attrs"><class>xsc.Element.Attrs</class></pyref>)
and define the attributes by deriving them from one of the existing attribute classes.
We could extend our example element in the following way:</par>

<example><title>Using attributes</title>
<prog>
class cool(xsc.Element):
	class Attrs(xsc.Element.Attrs):
		class adj(xsc.TextAttr): pass

	def convert(self, converter):
		node = xsc.Frag(self.content, u" is")
		if u"adj" in self.attrs:
			node.append(u" ", html.em(self[u"adj"]))
		node.append(u" cool!")
		return node.convert(converter)
</prog>
</example>

<par>and use it like this:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool(python(), adj=u"totally")</input>
<prompt>&gt;&gt;&gt; </prompt><input>node.conv().asBytes()</input>
&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt; is &lt;em&gt;totally&lt;/em&gt; cool!
</tty>

</section>


<section><title>Default attributes</title>
<par>It is possible to define default values for attributes via
the class attribute <lit>default</lit>:</par>

<example><title>Defining default attributes</title>
<prog>
class cool(xsc.Element):
	class Attrs(xsc.Element.Attrs):
		class adj(xsc.TextAttr):
			default = u"absolutely"

	def convert(self, converter):
		node = xsc.Frag(self.content, u" is")
		if u"adj" in self.attrs:
			node.append(u" ", html.em(self[u"adj"]))
		node.append(u" cool!")
		return node.convert(converter)
</prog>
</example>

<par>Now if we instantiate the class without specifying
<lit>adj</lit> we'll get the default:</par>

<example><title>Using default attributes</title>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool(python())</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.conv().asBytes()</input>
&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt; is &lt;em&gt;absolutely&lt;/em&gt; cool!
</tty>
</example>

<par>If we want a <class>cool</class> instance without
an <lit>adj</lit> attribute, we can pass <lit>None</lit>
as the attribute value:</par>

<example><title>Removing default attributes</title>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool(python(), adj=None)</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.conv().asBytes()</input>
&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt; is cool!
</tty>
</example>

</section>

<section><title>Attribute value sets</title>
<par>It's possible to specify that an attribute has a fixed set of allowed
values. This can be done with the class attribute <lit>values</lit>. We could
extend our example to look like this:</par>

<example><title>Defining attributes value sets</title>
<prog>
class cool(xsc.Element):
	class Attrs(xsc.Element.Attrs):
		class adj(xsc.TextAttr):
			default = "absolutely"
			values = (u"absolutely", u"totally", u"very")

	def convert(self, converter):
		node = xsc.Frag(self.content, u" is")
		if u"adj" in self.attrs:
			node.append(" ", html.em(self[u"adj"]))
		node.append(u" cool!")
		return node.convert(converter)
</prog>
</example>

<par>These values won't be checked when we create our <class>cool</class>
instance. Only when this node is parsed from a file will the warning
be issued:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>s = '&lt;cool adj="pretty"&gt;&lt;python/&gt;&lt;/cool&gt;'</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = parsers.parseString(s)</input>
/home/walter/pythonroot/ll/xist/xsc.py:1665: IllegalAttrValueWarning: Attribute value 'pretty' not allowed for __main__:cool.Attrs.adj.
  warnings.warn(errors.IllegalAttrValueWarning(self))
</tty>

<par>The warning will also be issued if we publish such a node,
but note that for warnings Python's warning framework is used,
so the warning will be printed only once (but of course
you can change that with <function>warnings.filterwarnings</function>):</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>node = cool(python(), adj=u"pretty")</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes()</input>
/home/walter/pythonroot/ll/xist/xsc.py:1665: IllegalAttrValueWarning: Attribute value 'pretty' not allowed for __main__:cool.Attrs.adj.
  warnings.warn(errors.IllegalAttrValueWarning(self))
&lt;cool adj="very"&gt;&lt;python /&gt;&lt;/cool&gt;
</tty>

</section>

<section><title>Required attributes</title>
<par>Finally it's possible to specify that an attribute is required.
This again will only be checked when parsing or publishing. To specify that an
attribute is required simply add the class attribute <lit>required</lit>
with the value <lit>True</lit>. The attribute <lit>alt</lit> of the
class <pyref module="ll.xist.ns.html" class="img"><class>ll.xist.ns.html.img</class></pyref>
is such an attribute, so we'll get:</par>

<example><title>Missing required attributes</title>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.img(src="eggs.png")</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes()</input>
/home/walter/pythonroot/ll/xist/xsc.py:2046: RequiredAttrMissingWarning: Required attribute 'alt' missing in ll.xist.ns.html:img.Attrs.
  warnings.warn(errors.RequiredAttrMissingWarning(self, attrs.keys()))
&lt;img src="eggs.png" /&gt;
</tty>
</example>

</section>

</section>

<section><title>Namespaces</title>

<par>Now that you've defined your own elements, you have to tell the parser
about them, so they can be instantiated when a file is parsed. This is done
with namespace classes.</par>

<par>Namespace classes can be thought of as object oriented versions of
&xml; namespaces. Two class attributes can be used to configure the namespace:
<lit>xmlname</lit> specifies the default namespace prefix to use for the
namespace and <lit>xmlurl</lit> is the namespace name. All element classes
nested inside the namespace class belong to the namespace.</par>

<par>It's also possible to define your namespace class without any nested
element classes and later add those classes to the namespace by attribute
assignment or with the class method
<pyref module="ll.xist.xsc" class="Namespace" method="update"><method>update</method></pyref>,
which expects a dictionary as an argument. All objects found in the values of
the dictionary will be added to the namespace as attributes. So you can put the
namespace class at the end of your Python module after all the element classes
are defined and add all the objects from the local scope to the namespace. Your
complete namespace might looks like this:</par>
<example><title>Defining and populating a namespace</title>
<prog>
class python(xsc.Element):
	def convert(self, converter):
		return html.a(
			u"Python",
			href=u"http://www.python.org/"
		)

class cool(xsc.Element):
	def convert(self, converter):
		node = html.b(self.content, u" is cool!")
		return node.convert(converter)

class __ns__(xsc.Namespace):
	xmlname = "foo"
	xmlurl = "http://www.example.com/foo"
__ns__.update(vars())
</prog>
</example>

<par>All defined namespace classes will be registered with the parser automatically,
so all elements belonging to the namespace will be used when parsing files.</par>

<section><title>Namespaces as modules</title>

<par>It is convenient to define all classes that belong to one namespace in one
Python module. However this means that the resulting module will only contain
one <z>interesting</z> object: the namespace class. To make using this class
more convenient, it's possible to turn the namespace class into a module by
using the class method <pyref module="ll.xist.xsc" class="Namespace" method="makemod"><method>makemod</method></pyref>
instead of <method>update</method>:</par>

<example><title>Turning a namespace into a module</title>
<prog>
class python(xsc.Element):
	def convert(self, converter):
		return html.a(
			u"Python",
			href=u"http://www.python.org/"
		)

class cool(xsc.Element):
	def convert(self, converter):
		node = html.b(self.content, u" is cool!")
		return node.convert(converter)

class __ns__(xsc.Namespace):
	xmlname = "foo"
	xmlurl = "http://www.example.com/foo"
__ns__.<em>makemod</em>(vars())
</prog>
</example>

<par>Suppose that the above code is in the file <filename>foo.py</filename>.
Doing an <lit>import foo</lit> will then give you the namespace class instead
of the module:</par>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>import foo</input>
<prompt>&gt;&gt;&gt; </prompt><input>foo</input>
&lt;foo:__ns__ namespace name=u"foo" url=u"http://www.example.com/foo" with 2 elements from "foo.py" at 0x87654321&gt;
<prompt>&gt;&gt;&gt; </prompt><input>foo.python</input>
&lt;foo:python element at 0x87654321&gt;
</tty>
</section>

<section><title>Global attributes</title>
<par>You can define global attributes belonging to a certain namespace in the
same way as defining local attributes belonging to a certain element type:
Define a nested <class>Attrs</class> class inside the namespace class
(derived from <class>ll.xist.xsc.Namespace.Attrs</class>):</par>

<prog>
from ll.xist import xsc

class __ns__(xsc.Namespace):
	xmlname = "foo"
	xmlurl = "http://www.example.com/foo"

	class Attrs(xsc.Namespace.Attrs):
		class foo(xsc.TextAttr): pass
__ns__.makemod(vars())
</prog>

<par>Setting and accessing such an attribute can be done like this:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>import foo</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(u"foo", {(foo, u"foo"): u"bar")</input>
<prompt>&gt;&gt;&gt; </prompt><input>str(node[foo, u"foo"])</input>
'bar'
</tty>

<par>An alternate way of specifying a global attribute in a constructor looks
like this:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>import foo</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(u"foo", foo.Attrs(foo=u"baz"))</input>
<prompt>&gt;&gt;&gt; </prompt><input>str(node[foo, u"foo"])</input>
'baz'
</tty>

</section>

<section><title>Subclassing namespaces</title>
<par>Each element class that belongs to a namespace can access its
namespace via the class attribute <lit>__ns__</lit>. When you're subclassing
namespace classes, the elements in the base namespace will be automatically
subclassed too. Of course you can explicitly subclass an element class too.
The following example shows the usefulness of this feature. Define your base
namespace like this and put it into <filename>navns.py</filename>:</par>

<prog>
from ll.xist import xsc
from ll.xist.ns import html

languages = [
	(u"Python", u"http://www.python.org/"),
	(u"Perl", u"http://www.perl.org/"),
	(u"PHP", u"http://www.php.net/"),
	(u"Java", u"http://java.sun.com/")
]

class navigation(xsc.Element):
	def convert(self, converter):
		node = self.__ns__.links()
		for (name, url) in languages:
			node.append(self.__ns__.link(name, href=url))
		return node.convert(converter)

class links(xsc.Element):
	def convert(self, converter):
		node = self.content
		return node.convert(converter)

class link(xsc.Element):
	class Attrs(xsc.Element.Attrs):
		class href(xsc.URLAttr): pass

	def convert(self, converter):
		node = html.div(html.a(self.content, href=self[u"href"]))
		return node.convert(converter)

class __ns__(xsc.Namespace):
	xmlname = "nav"
	xmlurl = "http://www.example.com/nav"
__ns__.makemod(vars())
</prog>

<par>This namespace defines a navigation element that generates <class>div</class>s
with links to various homepages for programming languages. We can use it like this:</par>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>import navns</input>
<prompt>&gt;&gt;&gt; </prompt><input>print navns.navigation().conv().asBytes()</input>
&lt;div&gt;&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt;&lt;/div&gt;
&lt;div&gt;&lt;a href="http://www.perl.org/"&gt;Perl&lt;/a&gt;&lt;/div&gt;
&lt;div&gt;&lt;a href="http://www.php.net/"&gt;PHP&lt;/a&gt;&lt;/div&gt;
&lt;div&gt;&lt;a href="http://java.sun.com/"&gt;Java&lt;/a&gt;&lt;/div&gt;
</tty>
<par>(Of course the output will all be on one line.)</par>

<par>Now we can define a derived namespace (in the file <filename>nav2ns.py</filename>)
that overwrites the element classes <class>links</class> and <class>link</class>
to change how the navigation looks:</par>

<prog>
from ll.xist import xsc
from ll.xist.ns import html

import navns

class __ns__(navns):
	class links(navns.links):
		def convert(self, converter):
			node = html.table(
				self.content,
				border=0,
				cellpadding=0,
				cellspacing=0,
				class_=u"navigation",
			)
			return node.convert(converter)

	class link(navns.link):
		def convert(self, converter):
			node = html.tr(
				html.td(
					html.a(
						self.content,
						href=self[u"href"],
					)
				)
			)
			return node.convert(converter)
__ns__.makemod(vars())
</prog>

<par>When we use the navigation element from the derived namespace we'll get
the following output:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>import nav2ns</input>
<prompt>&gt;&gt;&gt; </prompt><input>print nav2ns.navigation().conv().asBytes()</input>
&lt;table border="0" cellpadding="0" cellspacing="0" class="navigation"&gt;
&lt;tr&gt;&lt;td&gt;&lt;a href="http://www.python.org/"&gt;Python&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;&lt;a href="http://www.perl.org/"&gt;Perl&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;&lt;a href="http://www.php.net/"&gt;PHP&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td&gt;&lt;a href="http://java.sun.com/"&gt;Java&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;/table&gt;
</tty>
<par>(again all on one line.)</par>

<par>Notice that we automatically got an element class <class>nav2ns.navigation</class>,
that this element class inherited the <method>convert</method> method from its
base class and that the call to <method>convert</method> on the derived class
did instantiate the link classes from the derived namespace.</par>

</section>

<section><title>Namespaces as conversion targets</title>

<par>The <arg>converter</arg> argument passed to the <method>convert</method> method
has an attribute <lit>target</lit> which is a namespace class and specifies the target
namespace to which <self/> should be converted.</par>

<par>You can check which conversion is wanted with <function>issubclass</function>.
Once this is determined you can use element classes from the target to create the
required &xml; object tree. This makes it possible to customize the conversion by
passing a derived namespace to the <method>convert</method> method. To demonstrate
this, we change our example namespace to use the conversion target like this:</par>

<prog>
import navns

class __ns__(navns):
	class links(nav.links):
		def convert(self, converter):
			node = converter.target.table(
				self.content,
				border=0,
				cellpadding=0,
				cellspacing=0,
				class_=u"navigation",
			)
			return node.convert(converter)

	class link(nav.link):
		def convert(self, converter):
			target = converter.target
			node = target.tr(
				target.td(
					target.a(
						self.content,
						href=self[u"href"],
					)
				)
			)
			return node.convert(converter)
</prog>

<par>What we might want to do is have all links (i.e. all <class>ll.xist.ns.html.a</class>
elements) generated with an attribute <lit>target="_top"</lit>. For this we derive
a new namespace from <class>ll.xist.ns.html</class> and overwrite the <class>a</class>
element:</par>

<prog>
from ll.xist.ns import html

class __ns__(html):
	class a(html.a):
		def convert(self, converter):
			node = html.a(self.content, self.attrs, target=u"_top")
			return node.convert(converter)
</prog>

<par>Now we can pass this namespace as the conversion target and all links
will have a <lit>target="_top"</lit>.</par>
</section>

</section>


<section><title>Validation and content models</title>

<par>When generating &html; you might want to make sure that your generated
code doesn't contain any illegal tag nesting (i.e. something bad like
<markup>&lt;p&gt;&lt;p&gt;Foo&lt;/p&gt;&lt;/p&gt;</markup>). The module
<module>ll.xist.ns.html</module> does this automatically:</par>

<example>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.p(html.p(u"foo"))</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes()</input>
/home/walter/pythonroot/ll/xist/sims.py:238: WrongElementWarning: element &lt;ll.xist.ns.html:p&gt; may not contain element &lt;ll.xist.ns.html:p&gt;
  warnings.warn(WrongElementWarning(node, child, self.elements))
&lt;p&gt;&lt;p&gt;foo&lt;/p&gt;&lt;/p&gt;
</tty>
</example>

<par>For your own elements you can specify the content model too. This is done
by setting the class attribute <lit>model</lit> inside the element class.
<lit>model</lit> must be an object that provides a <method>checkvalid</method>
method. This method will be called during parsing or publishing with the element
as an argument. When a validation violation is detected, the Python warning
framework should be used to issue a warning.</par>

<par>The module <module>ll.xist.sims</module> contains several classes that
provide simple validation methods:
<pyref module="ll.xist.sims" class="Empty"><class>Empty</class></pyref>
can be used to ensure that the element doesn't have any content (like
<markup>br</markup> and <markup>img</markup> in &html;).
<pyref module="ll.xist.sims" class="Any"><class>Any</class></pyref>
does allow any content.
<pyref module="ll.xist.sims" class="NoElements"><class>NoElements</class></pyref>
will warn about elements from the same namespace (elements from other namespaces
will be OK).
<pyref module="ll.xist.sims" class="NoElementsOrText"><class>NoElementsOrText</class></pyref>
will warn about elements from the same namespace and non-whitespace text content.
<pyref module="ll.xist.sims" class="Elements"><class>Elements</class></pyref>
will only allow the elements specified in the constructor.
<pyref module="ll.xist.sims" class="ElementsOrText"><class>ElementsOrText</class></pyref>
will only allow the elements specified in the constructor and text.</par>

<par>None of these classes will check the number of child elements or their
order.</par>

<par>For more info see the <pyref module="ll.xist.sims"><module>sims</module></pyref>
module.</par>
</section>


<section><title>Entities</title>

<par>In the same way as defining new element types, you can define new entities.
But to be able to use the new entities in an &xml; file you have to use a parser
that supports reporting undefined entities to the application via
<method>skippedEntity</method>
(<pyref module="ll.xist.parsers" class="SGMLOPParser"><class>SGMLOPParser</class></pyref>
and <pyref module="ll.xist.parsers" class="ExpatParser"><class>ExpatParser</class></pyref>
in the module <pyref module="ll.xist.parsers"><module>ll.xist.parsers</module></pyref>
do that). The following example is from the module
<pyref module="ll.xist.ns.abbr"><module>ll.xist.ns.abbr</module></pyref>:</par>

<example><title>Defining new entities</title>
<prog>
from ll.xist import xsc
from ll.xist.ns import html

class html(xsc.Entity):
	def convert(self, converter):
		return html.abbr(
			u"HTML",
			title=u"Hypertext Markup Language",
			lang=u"en"
		)
</prog>
</example>

<par>You can use this entity in your &xml; files like this:</par>

<example><title>Using the newly defined entity</title>
<prog>
&lt;cool adj="very"&gt;&amp;html;&lt;/cool&gt;
</prog>
</example>

</section>

<section><title>Processing instructions</title>
<par>Defining processing instructions works just like elements and entities.
Derive a new class from
<pyref module="ll.xist.xsc" class="ProcInst"><class>ll.xist.xsc.ProcInst</class></pyref>
and implement <pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>.
The following example implements a processing instruction that returns an
uppercase version of its content as a text node.</par>

<example><title>Defining new processing instructions</title>
<prog>
class upper(xsc.ProcInst):
	def convert(self, converter):
		return xsc.Text(self.content.upper())
</prog>
</example>

<par>It can be used in an &xml; file like this:</par>

<example><title>Using the newly defined processing instruction</title>
<prog>
&lt;cool&gt;&lt;?upper foo?&gt;&lt;/cool&gt;
</prog>
</example>

<par>There are namespaces containing processing instruction classes that don't
provide a <method>convert</method> method. These processing instruction objects
will then be published as &xml; processing instructions. One example is the
namespace <pyref module="ll.xist.ns.php"><module>ll.xist.ns.php</module></pyref>.</par>

<par>Other namespaces (like <pyref module="ll.xist.ns.jsp"><module>ll.xist.ns.jsp</module></pyref>)
contain processing instruction classes, but they will be published in a different
(not &xml; compatible) format. For example <lit>ll.xist.ns.jsp.expression("foo")</lit>
will be published as <lit>&lt;%= foo&gt;</lit>.</par>

</section>

</section>

<section><title>Publishing &xml; trees</title>

<par>After creating the &xml; tree and converting the tree
into its final output form, you have to write the resulting tree
to a file. This can be done with the publishing &api;. Three methods
that use the publishing &api; are
<pyref module="ll.xist.xsc" class="Node" method="bytes"><method>bytes</method></pyref>,
<pyref module="ll.xist.xsc" class="Node" method="asBytes"><method>asBytes</method></pyref>
and
<pyref module="ll.xist.xsc" class="Node" method="write"><method>write</method></pyref>.
<pyref module="ll.xist.xsc" class="Node" method="bytes"><method>bytes</method></pyref>
is a generator that will yield the complete 8-bit &xml; string in fragments.
<pyref module="ll.xist.xsc" class="Node" method="asBytes"><method>asBytes</method></pyref>
returns the complete 8-bit &xml; string.</par>

<par>You can specify the encoding with the parameter <arg>encoding</arg>
(with <lit>"utf-8"</lit> being the default). Unencodable characters will be
escaped with character references when possible (i.e. inside text nodes, for
comments or processing instructions you'll get an exception):</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xsc</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>s = u"A\e4\u03a9\u8a9e"</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(s)</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes(encoding="ascii")</input>
&lt;div&gt;A&amp;#228;&amp;#937;&amp;#35486;&gt;
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes(encoding="iso-8859-1")</input>
&lt;div&gt;Aä&amp;#937;&amp;#35486;&gt;
<prompt>&gt;&gt;&gt; </prompt><input>print xsc.Comment(s).asBytes()</input>
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in ?
  File "/home/walter/pythonroot/ll/xist/xsc.py", line 600, in asBytes
    publisher.publish(stream, self, base)
  File "/home/walter/pythonroot/ll/xist/publishers.py", line 205, in publish
    self.node.publish(self)
  File "/home/walter/pythonroot/ll/xist/xsc.py", line 1305, in publish
    publisher.write(self.content)
  File "/usr/local/lib/python2.3/codecs.py", line 178, in write
    data, consumed = self.encode(object, self.errors)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-4: ordinal not in range(128)
</tty>

<par>When you include an <pyref module="ll.xist.ns.xml" class="XML10">&xml; header</pyref>
or an <pyref module="ll.xist.ns.meta" class="contenttype">&html; meta header</pyref>,
&xist; will automatically insert the correct encoding when publishing:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xsc</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import xml, meta</input>
<prompt>&gt;&gt;&gt; </prompt><input>e = xsc.Frag(xml.XML10(), u"\n", meta.contenttype())</input>
<prompt>&gt;&gt;&gt; </prompt><input>print e.conv().asBytes(encoding="iso-8859-15")</input>
&lt;?xml version='1.0' encoding='iso-8859-15'?&gt;
&lt;meta content="text/html; charset=iso-8859-15" http-equiv="Content-Type" /&gt;
</tty>

<par>Another useful parameter is <arg>xhtml</arg>,
it specifies whether you want pure &html; or &xhtml; as output:</par>

<dlist>
<term><lit>xhtml==0</lit></term><item>This will give you pure &html;, i.e. no
final <lit>/</lit> for elements with an empty content model, so you'll get e.g.
<markup>&lt;br&gt;</markup> in the output. Elements that don't have an empty
content model, but are empty will be published with a start and end tag
(i.e. <markup>&lt;div&gt;&lt;/div&gt;</markup>).</item>
<term><lit>xhtml==1</lit></term><item>This gives &html; compatible &xhtml;.
Elements with an empty content model will be published like this:
<markup>&lt;br /&gt;</markup> (This is the default).</item>
<term><lit>xhtml==2</lit></term><item>This gives full &xml; output. Every empty
element will be published with an empty tag (without an additional space):
<markup>&lt;br/&gt;</markup> or <markup>&lt;div/&gt;</markup>.</item>
</dlist>

<par>Writing a node to a file can be done with the method
<pyref module="ll.xist.xsc" class="Node" method="write"><method>write</method></pyref>:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(u"äöü", html.br(), u"ÄÖÜ")</input>
<prompt>&gt;&gt;&gt; </prompt><input>node.write(open("foo.html", "wb"), encoding="ascii")</input>
</tty>

<par>All these methods use the method
<pyref module="ll.xist.xsc" class="Node" method="publish"><method>publish</method></pyref> internally.
<pyref module="ll.xist.xsc" class="Node" method="publish"><method>publish</method></pyref> gets passed
an instance of <pyref module="ll.xist.publishers" class="Publisher"><class>ll.xist.publisher.Publisher</class></pyref>.</par>
</section>


<section><title>Searching trees</title>

<par>There are two methods available for iterating through an &xml; tree and
finding nodes in the tree: The <method>walk</method> method and XFind
expressions.</par>

<section><title>The <method>walk</method> method</title>
<par>The method <pyref module="ll.xist.xsc" class="Node" method="walk"><method>walk</method></pyref>
is a generator. You pass a callable object to <method>walk</method>
which is used for determining which part of the tree should be searched and
which nodes should be returned.</par>

<par><module>ll.xist.xsc</module> provides several useful predefined classes for
specifying what should be returned from <method>walk</method>:
<pyref module="ll.xist.xsc" class="FindType"><class>FindType</class></pyref>
will search only the first level of the tree and will return any node that is an
instance of one of the classes passed to the constructor. So if you have an
instance of <class>ll.xist.ns.html.ul</class> named <lit>node</lit> you could
search for all <class>ll.xist.ns.html.li</class> elements inside with the
following code:</par>

<example><title>Searching for <class>li</class> inside <class>ul</class> with <method>walk</method></title>
<prog>
for cursor in node.content.walk(xsc.FindType(html.li)):
	print unicode(cursor.node)
</prog>
</example>

<par><pyref module="ll.xist.xsc" class="FindTypeAll"><class>FindTypeAll</class></pyref>
can be used when you want to search the complete tree. The following example
extracts all the links on the
<link href="http://www.python.org/">Python home page</link>:</par>

<example><title>Finding all links on the Python home page</title>
<prog>
from ll.xist import xsc, parsers
from ll.xist.ns import html

node = parsers.parseURL("http://www.python.org/", tidy=True)

for cursor in node.walk(xsc.FindTypeAll(html.a)):
	print cursor.node[u"href"]
</prog>
</example>

<par>This gives the output:</par>

<tty>
http://www.python.org/
http://www.python.org/search/
http://www.python.org/download/
http://www.python.org/doc/
http://www.python.org/Help.html
http://www.python.org/dev/
<rep>...</rep>
</tty>

<par>The following example will find all external links on the Python home
page:</par>

<example><title>Finding external links on the Python home page</title>
<prog>
from ll.xist import xsc, parsers
from ll.xist.ns import html

node = parsers.parseURL("http://www.python.org/", tidy=True)

def isextlink(cursor):
	if isinstance(cursor.node, html.a) and not unicode(cursor.node[u"href"]).startswith(u"http://www.python.org"):
		return (True, xsc.entercontent)
	return (xsc.entercontent,)

for cursor in node.walk(isextlink):
	print cursor.node[u"href"]
</prog>
</example>

<par>This gives the output:</par>

<tty>
http://www.jython.org/
http://sourceforge.net/tracker/?atid=105470&amp;group%5fid=5470
http://sourceforge.net/tracker/?atid=305470&amp;group%5fid=5470
http://sourceforge.net/cvs/?group%5fid=5470
http://www.python-in-business.org/
http://www.europython.org/
mailto:webmaster@python.org
<rep>...</rep>
</tty>

<par>The callable (<function>isextlink</function> in the example) will be called
for each node visited. The <arg>cursor</arg> argument has an attribute <lit>node</lit>
that is the node in question. For the other attributes see the
<pyref module="ll.xist.xsc" class="Cursor"><class>Cursor</class> class</pyref>.</par>

<par>The callable must return a sequence with the following entries:</par>

<dlist>
<term><lit>ll.xist.xsc.entercontent</lit></term><item>enter the content of this
element and continue searching;</item>
<term><lit>ll.xist.xsc.enterattrs</lit></term><item>enter the attributes of this
element and continue searching;</item>
<term>boolean value</term><item>If true, the node will be part of the result.</item>
</dlist>

<par>The sequence will be <z>executed</z> in the order you specify. To change
the top down traversal from our example to a bottom up traversal we could change
<function>isextlink</function> to the following (note the swapped tuple entries):</par>

<example><title>Bottom up link traversal function</title>
<prog>
def isextlink(node):
	if isinstance(node, html.a) and not unicode(node[u"href"]).startswith(u"http://www.python.org"):
		return <em>(xsc.entercontent, True)</em>
	return (xsc.entercontent,)
</prog>
</example>

<par>Note that the cursor yielded from <method>walk</method> will be reused by
subsequent <method>next</method> calls, so you should not modify the cursor and
you can't rely on attributes of the cursor after reentry to
<method>walk</method>.</par>

</section>

<section><title>XFind expressions</title>

<par>A second method exists for iterating through a tree: XFind expressions.
An XFind expression looks somewhat like an XPath expression, but is implemented
as a pure Python expression (overloading the division operators).</par>

<par>Our example from above that searched for <class>li</class>s inside
<class>ul</class>s can be rewritten as follows:</par>

<example><title>Searching for <class>li</class> inside <class>ul</class> with an XFind expression</title>
<prog>
for child in node/html.li:
	print unicode(child)
</prog>
</example>

<par>A XFind expression returns an iterator for certain parts of the &xml; tree.
In an XFind expression <lit><rep>a</rep>/<rep>b</rep></lit>,
<lit><rep>a</rep></lit> must be either a node or an iterator producing nodes
(note that an XFind expression itself is such an iterator, so
<lit><rep>a</rep></lit> itself might be a XFind expression).
<lit><rep>b</rep></lit> must be an XFind operator.</par>

<par>Every subclass of
<pyref module="ll.xist.xsc" class="Node"><class>ll.xist.xsc.Node</class></pyref>
is a XFind operator. If <lit><rep>b</rep></lit> is such a subclass,
<lit><rep>a</rep>/<rep>b</rep></lit> will produce any child nodes of the nodes
from <lit><rep>a</rep></lit> that is an instance of <lit><rep>b</rep></lit>.
If <lit><rep>b</rep></lit> is an attribute class, you will get attribute nodes
instead of child nodes. Other XFind operators can be found in the module
<pyref module="ll.xist.xfind"><module>ll.xist.xfind</module></pyref>. The
<lit>all</lit> operator will produce every node in the tree (except for
attributes):</par>

<prog>
from ll.xist import xfind
from ll.xist.ns import html

node = html.div(
	html.div(
		html.div(id=3),
		html.div(id=4),
		id=2,
	),
	html.div(
		html.div(id=6),
		html.div(id=7),
		id=5,
	),
	id=1
)

for child in node/xfind.all:
	print child["id"]
</prog>

<par>The output of this is:</par>

<tty>
1
2
3
4
5
6
7
</tty>

<par>The following example demonstrates how to find all links on the Python
homepage via an XFind expression:</par>

<prog>
from ll.xist import xfind, parsers
from ll.xist.ns import html

node = parsers.parseURL("http://www.python.org/", tidy=True)
for link in node/xfind.all/html.a:
	print link["href"]
</prog>

<par>An <lit>all</lit> operator in the middle of an XFind expression can be
abbreviated. The XFind expression from the last example
(<lit>node/xfind.all/html.a</lit>) can be rewritten like this:
<lit>node//html.a</lit>.</par>

<par>Another XFind operator is
<pyref module="ll.xist.xfind" class="contains"><class>contains</class></pyref>.
It acts as a filter, i.e. the nodes produced by
<lit><rep>a</rep>/xfind.contains(<rep>b</rep>)</lit> are a subset of the nodes
produced by <lit><rep>a</rep></lit>, those that contain child nodes of type
<lit>b</lit>. Searching for all links on the Python home page that contain
images can be done like this:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xfind, parsers</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = parsers.parseURL("http://www.python.org/", tidy=True)</input>
<prompt>&gt;&gt;&gt; </prompt><input>for link in node//html.a/xfind.contains(html.img):</input>
<prompt>... </prompt><input>	print link["href"]</input>
<prompt>... </prompt><input></input>
http://www.python.org/
http://www.python.org/psf/donations.html
http://www.opensource.org/
</tty>

<par>Note that using the <lit>all</lit> operator twice in an XFind expression
currently won't give you the expected result, as nodes might be produced twice.</par>

<par>Calling <method>__getitem__</method> on an XFind operator gives you an
item operator. Such an item operator only returns a specific item (or slice) of
those nodes returned by the base iterator. An example:</par>

<prog>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>e = html.table(html.tr(html.td(j) for j in xrange(i, i+3)) for i in xrange(1, 10, 3))</input>
<prompt>&gt;&gt;&gt; </prompt><input>print e.pretty().asBytes()</input>
&lt;table&gt;
	&lt;tr&gt;
		&lt;td&gt;1&lt;/td&gt;
		&lt;td&gt;2&lt;/td&gt;
		&lt;td&gt;3&lt;/td&gt;
	&lt;/tr&gt;
	&lt;tr&gt;
		&lt;td&gt;4&lt;/td&gt;
		&lt;td&gt;5&lt;/td&gt;
		&lt;td&gt;6&lt;/td&gt;
	&lt;/tr&gt;
	&lt;tr&gt;
		&lt;td&gt;7&lt;/td&gt;
		&lt;td&gt;8&lt;/td&gt;
		&lt;td&gt;9&lt;/td&gt;
	&lt;/tr&gt;
&lt;/table&gt;
<prompt>&gt;&gt;&gt; </prompt><input># Every cell</input>
<prompt>&gt;&gt;&gt; </prompt><input>for td in e/html.tr/html.td:</input>
<prompt>... </prompt><input>	print td</input>
<prompt>... </prompt><input></input>
1
2
3
4
5
6
7
8
9
<prompt>&gt;&gt;&gt; </prompt><input># Every first cell in each row</input>
<prompt>&gt;&gt;&gt; </prompt><input>for td in e/html.tr/html.td[0]:</input>
<prompt>... </prompt><input>	print td</input>
1
4
7
<prompt>&gt;&gt;&gt; </prompt><input># Every cell in the first row</input>
<prompt>&gt;&gt;&gt; </prompt><input>for td in e/html.tr[0]/html.td:</input>
<prompt>... </prompt><input>	print td</input>
1
2
3
<prompt>&gt;&gt;&gt; </prompt><input># The first of all cells</input>
<prompt>&gt;&gt;&gt; </prompt><input>for td in e/(html.tr/html.td)[0]:</input>
<prompt>... </prompt><input>	print td</input>
1
</prog>

</section>

</section>


<section><title>Manipulating trees</title>
<par>&xist; provides many methods for manipulating an &xml; tree.</par>

<par>The method <pyref module="ll.xist.xsc" class="Frag" method="withsep"><method>withsep</method></pyref>
can be used to put a seperator node between the child nodes of an
<pyref module="ll.xist.xsc" class="Element"><class>Element</class></pyref>
or a <pyref module="ll.xist.xsc" class="Frag"><class>Frag</class></pyref>:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xsc</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(*xrange(10))</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.withsep(", ").asBytes()</input>
&lt;div&gt;0, 1, 2, 3, 4, 5, 6, 7, 8, 9&lt;/div&gt;
</tty>

<par>The method <pyref module="ll.xist.xsc" class="Frag" method="shuffled"><method>shuffled</method></pyref>
returns a shuffled version of the <pyref module="ll.xist.xsc" class="Element"><class>Element</class></pyref>
or <pyref module="ll.xist.xsc" class="Frag"><class>Frag</class></pyref>:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xsc</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(*xrange(10))</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.shuffled().withsep(", ").asBytes()</input>
&lt;div&gt;8, 1, 3, 6, 7, 5, 2, 9, 4, 0&lt;/div&gt;
</tty>

<par>There are methods named <pyref module="ll.xist.xsc" class="Frag" method="reversed"><method>reversed</method></pyref>
and <pyref module="ll.xist.xsc" class="Frag" method="sorted"><method>sorted</method></pyref> that
return a reversed or sorted version of an element or fragment:</par>

<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import xsc</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>def key(n):</input>
<prompt>... </prompt><input>   return unicode(n)</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = html.div(8,4,2,1,9,6,3,0,7,5)</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.sorted(key=key).reversed().withsep(",").asBytes()</input>
&lt;div&gt;9,8,7,6,5,4,3,2,1,0&lt;/div&gt;
</tty>

<par>The method <pyref module="ll.xist.xsc" class="Node" method="mapped"><method>mapped</method></pyref>
recursively walks the tree and generates a new tree, where all the nodes are mapped
through a function. An example: To replace <lit>Python</lit> with <lit>Parrot</lit>
in every text node on the <link href="http://www.python.org/">Python page</link>, do the following:</par>

<prog>
from ll.xist import xsc, parsers

def p2p(node, converter):
	if isinstance(node, xsc.Text):
		node = node.replace(u"Python", u"Parrot")
		node = node.replace(u"python", u"parrot")
	return node

node = parsers.parseURL("http://www.python.org/", tidy=True)
node = node.mapped(p2p)
node.write(open("parrot_index.html", "wb"))
</prog>

<par>The function must either return a new node, in which case this
new node will be used instead of the old one, or return the
old node to tell <pyref module="ll.xist.xsc" class="Node" method="mapped"><method>mapped</method></pyref>
that it should recursively continue with the content of the node.</par>
</section>

<section><title>&url;s</title>

<par>For &url; handling &xist; uses the module
<pyref module="ll.url"><module>ll.url</module></pyref>. Refer to its documentation
for the basic functionality (especially regarding the methods
<pyref module="ll.url" class="URL" method="__div__"><method>__div__</method></pyref>
and <pyref module="ll.url" class="URL" method="relative"><method>relative</method></pyref>).</par>

<par>When &xist; parses an &xml; resource it uses a so called <z>base</z> &url;.
This base &url; can be passed to all parsing functions. If it isn't specified
it defaults to the &url; of the resource being parsed. This base &url; will
be prepended to all &url;s that are read during parsing:</par>
<tty>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist import parsers</input>
<prompt>&gt;&gt;&gt; </prompt><input>from ll.xist.ns import html</input>
<prompt>&gt;&gt;&gt; </prompt><input>node = parsers.parseString('&lt;img src="eggs.png"/&gt;', base="root:spam/index.html")</input>
<prompt>&gt;&gt;&gt; </prompt><input>print node.asBytes()</input>
&lt;img src="root:spam/eggs.png" /&gt;
</tty>

<par>For publishing a base &url; can be specified too. &url;s will be published
relative to this base &url; with the exception of relative &url;s in the tree.
This means:</par>
<ulist>
<item>When you have a relative &url; (e.g. <lit>#top</lit>) generated by a <method>convert</method>
call, this &url; will stay the same when publishing.</item>
<item>Base &url;s for parsing should never be relative: Relative base
&url;s will be prepended to all relative &url;s in the file, but this will not be
reverted for publishing. In most cases the base &url; should be a
<lit>root</lit> &url; when you parse local files.</item>
<item>When you parse remote web pages you can either
omit the <arg>base</arg> argument, so it will default to the
&url; being parsing, so that links, images, etc. on the page
will still point back to their original location, or you
might want to use the empty &url; <lit>URL()</lit> as the
base, so you'll get all &url;s in the page as they are.</item>
<item><par>When &xist; is used as a compiler for static pages, you're
going to read source &xml; files, do a conversion and write the
result to a new target file. In this case you should probably
use the &url; of the target file for both parsing and
publishing. Let's assume we have an &url; <lit>#top</lit>
in the source file. When we use the <z>real</z> file names
for parsing and publishing like this:</par>
<prog>
node = parsers.parseFile("spam.htmlxsc", base="root:spam.htmlxsc")
node = node.conv()
node.write(open("spam.html", "wb"), base="root:spam.html")
</prog>
<par>the following will happen: The &url; <lit>#top</lit>
will be parsed as <lit>root:spam.htmlxsc#top</lit>. After
conversion this will be written to <filename>spam.html</filename>
relative to the &url; <lit>root:spam.html</lit>, which results
in <lit>spam.html#top</lit>, which works, but is not what you
want.</par>
<par>When you use <lit>root:spam.html</lit> both for parsing
and publishing, <lit>#top</lit> will be written to the target file
as expected.</par></item>
</ulist>

</section>


<section><title>Pretty printing &xml;</title>
<par>The method <method>pretty</method> can be used for pretty printing &xml;.
It returns a new version of the node, with additional white space between the
elements:</par>
<example>
<prog>
from ll.xist.ns import html
node = html.html(
	html.head(
		html.title(u"foo"),
	),
	html.body(
		html.div(
			html.h1(u"The ", html.em(u"foo"), u" page!"),
			html.p(u"Welcome to the ", html.em(u"foo"), u" page."),
		),
	),
)

print node.pretty().asBytes()
</prog>
</example>
<par>This will print:</par>
<example>
<tty>
&lt;html&gt;
	&lt;head&gt;
		&lt;title&gt;foo&lt;/title&gt;
	&lt;/head&gt;
	&lt;body&gt;
		&lt;div&gt;
			&lt;h1&gt;The &lt;em&gt;foo&lt;/em&gt; page!&lt;/h1&gt;
			&lt;p&gt;Welcome to the &lt;em&gt;foo&lt;/em&gt; page.&lt;/p&gt;
		&lt;/div&gt;
	&lt;/body&gt;
&lt;/html&gt;
</tty>
</example>
<par>Element content will only be modified if it doesn't contain
<class>Text</class> nodes, so mixed content will not be touched.</par>
</section>


<section><title>Automatic generation of image size attributes</title>

<par>The module <pyref module="ll.xist.ns.htmlspecials"><module>ll.xist.ns.htmlspecials</module></pyref>
contains an element <pyref module="ll.xist.ns.htmlspecials" class="autoimg"><class>autoimg</class></pyref>
that extends <pyref module="ll.xist.ns.html" class="img"><class>ll.xist.ns.html.img</class></pyref>.
When converted to &html; via the <pyref module="ll.xist.ns" class="Node" method="convert"><method>convert</method></pyref>
method the size of the image will be determined and the <lit>height</lit>
and <lit>width</lit> attributes will be set accordingly (if those attributes
are not set already).</par>

</section>


<section><title>Embedding Python code</title>
<par>It's possible to embed Python code into &xist; &xml; files. For this
&xist; supports two new processing instructions:
<pyref module="ll.xist.ns.code" class="pyexec"><lit>pyexec</lit></pyref>
and <pyref module="ll.xist.ns.code" class="pyeval"><lit>pyeval</lit></pyref> (in the module
<pyref module="ll.xist.ns.code"><module>ll.xist.ns.code</module></pyref>). The content of
<pyref module="ll.xist.ns.code" class="pyexec"><lit>pyexec</lit></pyref> will be
executed when the processing instruction node is converted.</par>

<par>The result of a call to <pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
for a <pyref module="ll.xist.ns.code" class="pyeval"><lit>pyeval</lit></pyref> processing instruction is whatever the
Python code in the content returns. The processing instruction content is treated as the body
of a function, so you can put multiple return statements there.
The converter is available as the parameter <arg>converter</arg> inside
the processing instruction. For example, consider the following &xml; file:</par>

<prog>
&lt;?pyexec
	# sum
	def gauss(top=100):
		sum = 0
		for i in xrange(top+1):
			sum += i
		return sum
?&gt;
&lt;b&gt;&lt;?pyeval return gauss()?&gt;&lt;/b&gt;
</prog>

<par>Parsing this file and calling
<pyref module="ll.xist.xsc" class="Node" method="convert"><method>convert</method></pyref>
results in the following:</par>

<tty>
&lt;b&gt;5050&lt;/b&gt;
</tty>

</section>
