Title:  0.9.5 Almost There, But Stumped On Templates

Since the 0.9.4 release I've rewritten the main part of the decoding parser 
so that it's much cleaner and handles more edge conditions.  If there's one 
word that defines what makes MIME horrible it would be "edge".

It's amazing the kind of stupid crap clients are allowed to send out in headers.
I've actually got test cases from Mutt where it breaks headers up into completely
different encodings across multiple lines for no damn reason at all.  Here's a 
test I had to write just to cover the one worst case I found:

<pre class="code prettyprint">
def test_dumb_shit():
    # this is a sample of possibly the worst case Mutt can produce
    idiot = '=?iso-8859-1?B?SOlhdnkgTel05WwgVW7uY/hk?=\n\t=?iso-8859-1?Q?=E9?='
    should_be = u'H\xe9avy M\xe9t\xe5l Un\xeec\xf8d\xe9'
    assert_equal(encoding.header_from_mime_encoding(idiot), should_be)
</pre>

If you can decode what that actually is and why it's retarded then you'll win a
prize.  I'll give you a hint:  @B != Q@.

The example of the above working is "on this page at oneshotblog.com":http://oneshotblog.com/posts/zedshaw@zedshaw.com/test.html so that's solved (for now).


h2. Now About Those Templates

I originally wanted to make writing templates as easy as writing an email or a web page, 
so I let you do things like this:

<pre class="code prettyprint">
To: {{ message['from'] }}
From: {{ confirm_address }}@{{host}}
Subject:  Please confirm your new blog {{ post_name }}
Reply-To: {{ confirm_address }}@{{host}}

Hi there, you requested that I create a blog with a post named:

{{ post_name }} : {{ message['subject'] }}
...
</pre>

This is a "Jinja2":http://jinja.pocoo.org/2/ that "oneshotblog.com":oneshotblog.com sends
out when it makes you confirm creating a blog.

This seems all normal, since it's the headers you want and the body, 
like an email, but it is *a total fantasy*.  The reason is you can't actually
decode the above template if any part of it has non-ascii chars.  I'll explain, and
then hopefully I can get some suggestions on what people would like.

First, Lamson does the right thing and in your templates you are getting perfectly
usable and future proof @unicode@ objects to work with in your handlers.  Also,
Jinja2 has no problem with taking this unicode and processing the above template.
It works fine, and Jinja2 or Mako aren't to blame for the problem with
templates now.

The problem is actually in how email has to be parsed, since the email library
expects you to hand it a @ascii@ encoded string, no unicode allowed.  Nothing else.
Lamson's new code can handle the unicode fine, the problem is actually in 
this one function:


<pre class="code prettyprint">
def respond(template, variables, html=False):
    results = render(template, variables)

    data = email.message_from_string(results.encode('ascii', 'replace'))

    if html:
        msg = mail.MailResponse(Html=data.get_payload())
    else:
        msg = mail.MailResponse(Body=data.get_payload())

    msg.update(data)

    return msg
</pre>


The above function has to go, but where I'm stuck is that it'd be way more
elegant to ditch this whole mess completely and remove headers from
the template.

It's too hard to write these headers by hand unless they are simple ones,
and there needs to be a way to pass in more complex headers that you want
to add.  If I'm providing a way to pass in headers to @lamson.view.respond@
then why not take the headers out of the template and simplify the
whole rendering process?

I'd like to hear from people using Lamson or those who might have ideas on this.
If taking your headers out of the template and moving them to the render call
will totally screw up your day, then let me know.  If it comes down to it, and I 
have to do it, I'll probably write a conversion tool to help.

Also, if you have a suggestion for doing the above better but keeping the headers
then let me know.

Thanks for your help folks, and I'll be hanging out in the #lamson channel on irc.freenode.org
if people want to hash out a design.


