Decrypted Mind

libotp - email rendering in kube

Feb 5, 2016 kolab kde kontact kube

The important part of a mailreader is rendering an email. Nobody likes to read the raw mime message. For kube we looked around what we should use for mail rendering. and came to the conclusion, that we will use parts of kdepim for that task. But the current situation was not the usable for us, because is was tangled together with other things of the mailviewer (like Akonadi, Widgetcode,...) , so we would end up depending on nearly everything in kdepim. What was a nogo for us. But after a week of untangeling the rendering part out of messageviewer, we end up with a nice library that does only mail rendering called libotp branch dev/libotp. But for the moment it is just a working name. Maybe someone come up with a better one?

Why a lib - email rendering is easy?

Well if you look from the outside, it really looks looks like an easy task to solve. But in detail the task is quite complicated. We have crypted and signed mail parts, html/not html mailparts, alternate mime structure, broken mail clients and so on. And than we also want user interaction, do we want to decrypt the mail by default? Do we want to verify mails by default? Do we allow external html links? Does the user user prefer html over non html? ...

In total you have to keep many things in the mind while rendering a mail. And we are not talking, that we also want a pluginable system, where we be able to create own rendering for special types of mails. All these thing a already solved by the messageviewer of kdepim: We have high integrated crypto support, support for many different mime types and it was already used for years, additionally the test couverage is quite high. Like you see in the image above we are already been able to decrypt and verify mails.

libotp

libotp is a library that renders emails to html. Maybe you ask, why the hell html? I hate html mails, too :D But html is easy to display and back in time there was no alternative, to show something that is that dynamic. Nowadays we have other solutions like QML, but still we have html message, that we wanna be able to display. Currently, we have no way, to try out QML rendering for mails, because the output of libotp is limited to html. I hopefully can also solve this to give libotp the ability to redner to different output formats, by splitting the monolithic task of render an email to html into a parse step, in which the structure of the email is translated into the visible parts (a signed part is followed by a encrypted one, that has as child a html part,...) and the pure rendering step.

If you follow the link libotp branch dev/libotp, you may wonder, if a fork of messagelib is happening. No the repo is created, to use libotp now in kube and I made many shortcuts and use ugly hacks to get it working. The plan is that libotp is part of the messagelib repo and currently I have already made it to push the first part of (polished) patches upstream. If everything went fine, I will have everything upstreamed by next week.

How to use it ?

At the moment it is still work in process, so it may change. Also if other step up and give input about the way they wanna use it. Let's have a look how kube it is using kube/framework/mail/maillistmodel.cpp

// mail -> kmime tree
const auto mailData = KMime::CRLFtoLF(file.readAll());
KMime::Message::Ptr msg(new KMime::Message);
msg->setContent(mailData);
msg->parse();

first step - load the mail into KMime to have a tree with the mime parts. file is a mail in mbox format located in a local file.

// render the mail
StringHtmlWriter htmlWriter;
QImage paintDevice;
CSSHelper cssHelper(&paintDevice);
MessageViewer::NodeHelper nodeHelper;
ObjectTreeSource source(&htmlWriter, &cssHelper);
MessageViewer::ObjectTreeParser otp(&source, &nodeHelper);

now initalize the ObjectTreeParser. Therefore we need

  • HtmlWriter, that gets html output while rendering and do what ever the user wants to do with the html output (in our case just save it for later use - see htmlWriter.html()).

  • CSSHelper creates the header of the html with css, you have the possibility to set color schemas and fonts that are used for html rendering.

  • NodeHelper is the place, where information are stored, that need to be stored for longer (pointers to crypted part, that are currently in asynchronously been decrypted, or extra mail parts, that are visible but only on mime part in the mail). NodeHelper also informs you if any async job has endend. At the moment, we don't use the async mode nor we have mail internal links working, that's why the NodeHelper is a local variable here.

  • ObjectTreeSource is the setting object, here you can store, if decryption is allowed, if you prefer html output, if you like emotionicons, ...

And last not least the ObjectTreeParser(Otp) itself. It's doing the real work of parsing and rendering the mail :)

htmlWriter.begin(QString());
otp.parseObjectTree(msg.data());
htmlWriter.end();

return htmlWriter.html();

After initializing the Otp we can render the mail. This is done with otp.parseObjectTree(msg.data());. Around that we need to tell htmlWriter that a html creation has begun and ended afterwards.

As you may noticed, except the ObjectTreeParser and the NodeHelper kube has overloads of the objects. This makes libotp highly configurable for others needs already.

next steps

After the week of hacking now the current task is to push things upstream, to not create a fork and focus on one solution to render mails for kmail and kube together. After upstreaming I will start to extract the parts of libotp out of messageviewer (currently it is only another cmake target and not really divided) and make messageviewer to depend on libotp. With that libotp is a independent library that is used by both projects and I can focus again to polish libotp and messageviewer.

tl;dr;

Here you see kube can now render your spam now nicely:

Like the spam says (and spam can't lie) - get the party started!