Decrypted Mind: Articles About KDE

QML Dependency tracking in Debian

2024-12-01T00:00:00Z

Tracking library dependencies work in Debian to resolve from symbols usage to a library and add this to the list of dependencies. That is working for years now. The KDE community nowadays create more and more QML based applications. Unfortunately QML is a interpreted language, this means missing QML dependencies will only be an issue at runtime.

To fix this I created dh_qmldeps, that searches for QML dependencies at build time and will fail if it can't resolve the QML dependency.

Me didn't create an own QML interpreter, just using qmlimportscanner behind the scenes and process the output further to resolve the QML modules to Debian packages.

The workflow is like follows:

The package compiles normally and split to the binary packages. Than dh_qmldeps scans through the package content to find QML content ( .qml files, or qmldirfor QML modules). All founded files will be scanned by qmlimportscanner, the output is a list of depended QML modules. As QML modules have a standardized file path, we can ask the Debian system, which packages ship this file path. We end up with a list of Debian packages in the variable ${qml6:Depends}. This variable can be attached to the list of dependencies of the scanned package. A maintainer can also lower some dependencies to Recommends or Suggest, if needed.

You can find the source code on salsa and usage documentation you can find on https://qt-kde-team.pages.debian.net/dh_qmldeps.html.

The last weeks I now enabled dh_qmldeps for newly every package, that creates a QML6 module package. So the first bugs are solved and it should be usable for more packages.

By scanning with qmlimportscanner trough all code, I found several non-existing QML modules:

import QtQuick3DPrivate qt6-multimedia - no Private QML module QTBUG-131753.
import QtQuickPrivate qt6-graphs - no Private QML module QTBUG-131754.
import QtQuickTimeline qt6-quicktimeline - the correct QML name is QtQuick.Timeline QTBUG-131755.
import QtQuickControls2 qt6-webengine - looks like a porting bug as the QML6 modules name is QtQuick.Controls QTBUG-131756.
import QtGraphicalEffects kquickimageeditor - the correct name is for QML6 is qt5compat.graphicaleffects, properly as it is an example nobody checks it kquickimageeditor!7.

YEAH - the first milestone is reached. We are able to simply handle QML modules.

But QML applications there is still room for improvement. In apps the QML files are inside the executable. Additionally applications create internal QML modules, that are shipped directly in the same executable. I still search for a good way to analyse an executable to get a list of internal QML modules and a list of included QML files. Any ideas are welcomed :)

As workaround dh_qmldeps scans currently all QML files inside the application source code.

Akademy 2024 in Würzburg

2024-11-26T00:00:00Z

In order to prepare for the Akademy I started some days before to give my Librem 5 ( an Open Hardware Phone) another try and ended up with a non starting Plasma 6. Actually this issue was known already, but hasn't been addressed. In the end I reached the Akademy with my Librem 5 having phosh installed (which is Gnome based), in order to have something working.

I met Bushan and Bart who took care and the issue was fixed two days later I could finally install Plasma 6 on it. The last time I tested my Librem 5 with Plasma 5 it felt sluggish and not well working. But this time I was impressed how well the system reacts. Sure there are some things here and there, but in the bigger picture it is quite useable. One annoying issue is that the camera is only working with one app and the other issue is the battery capacity, you have to charge it once a day. Because of missing a QR reader that can use the camera, getting data to the phone was quite challenging. Unfortunately the conference Wifi separated the devices and I couldn't use KDE Connect to transfer data. In the end the only way to import data was taking five photos from the QR Code to import my D-Ticket to Itinerary.

With a device with Plasma Mobile, it directly was used for a experiment: How well does Dolphin works on a Plasma Mobile device. Together with Felix Ernst we tried it out and were quite impressed, that Dolphin does work very well on Plasma Mobile, after some simple modifications on the UI. That resulted in a patch to add a mobile UI for Dolphin !826.

With more time to play with my Librem 5 I also found an bug in KWeather, that is missing a Refresh option, when used in a Plasma Mobile environment #493656.

Akademy is a good place to identify and solve some issues. It is always like that, you chat with someone and they can tell you who to ask to answer the concrete question and in the end you can solve things, that seems unsolvable in the beginning.

There was also time to look into the travelling app Itinerary. A lot people are faced with a lot of real world issues, when not in their home town. Itinerary is the best traveling apps I know about. It can import nearly every ticket you have and can get location information from restaurant websites and allow routing to that place. It does add many useful information, while traveling like current delays, platform changes, live updates for elevator, weather information at the destination, a station map and all those features with strong focus on privacy.

In detail I found some small things to improve:

If you search for a bus ride and enter the correct name for the bus stop, it will still add some walk from and to the station. The issue here is that we use different backends and not all backends share the same geo coordinate. That's why Itinerary needs to add some heuristics to delete those paths.
Instead of displaying just a small station map of one bus stop in the inner city, it showed complete Würzburg inner city, as there is one big park around the inner city (named "Ringpark").
Würzburg has a quite big bus station but the platform information were missing in the map, so we tweaked the CSS to display the platform. To be sure, that we don't fix only Würzburg, we also looked at Greifswald and Aix-en-Provence if they are following the same name scheme.

I additionally learned that it has a lot of details that helps people who have special needs. That is the reason why Daniel Kraut wants to get Itinerary available for iOS. As spoken out, that Daniel wants to reach this goal, others already started to implement the first steps to build apps for iOS.

This year I was volunteering in helping out at Akademy. For me it was a lot of fun to meet everyone at the infodesk or help the speakers setup the beamer and microphone. It is also a good opportunity to meet many new faces and get in contact with them. I see also room for improvement. As we were quite busy at the Welcome Event to get out the badges to everyone, I couldn't answer the questions from newcomers, as the queue was too long. I propose that some people volunteer to be available for questions from newcomers. Often it is hard for newcomers to get their first contact(s) in a new community. There is a lot of space for improvement to make it easier for newcomers to join. Some ideas in my head are: Make an event for the newcomers to get them some links into the community and show that everyone is friendly. The tables at the BoFs should make a circle, so everyone can see each other. It was also hard for me to understand everyone as they mostly spoken towards the front. And then BoFs are sometimes full of very specific words and if you are not already deep in the topic you are lost. I can see the problem, on the one side BoFs are also the place where the person that knows the topic already wants to get things done. On the other side new comers join BoFs, are overwhelmed by to many new words get frustrated and think, that they are not welcome. Maybe at least everyone should present itself with name and ask new faces, why they joined the BoF to help them joining.

I'm happy, that the food provided for the attendees was very delicious and that I'm not the only one mostly vegetarian with a big amount to be vegan.

At the conference the KDE Eco initiation really caught me, as I see a lot of new possibilities in giving more reasons to switch to an Open Source system. The talk from Natalie was great to see how pupils get excited about Open Source and also help their grandparents to move to a Linux system. As I also will start to work as a teacher, I really got ideas what I can do at school. Together with Joseph and Nicole, we finally started to think about how to drive an exploration on what kind of old hardware is still KDE software running. The ones with the oldest hardware will get an old KDE shirt. For more information see #40.

The conference was very motivating for me, I also had still energy at the evening to do some Debian packaging and finally pushed kweathercore to Debian and started to work on KWeather. Now I'm even more interested in the KDE apps focusing the mobile world, as I now have some hardware that can actually use those apps.

I really enjoyed the workshop how to contribute to Qt by Volker Hilsheimer, especially the way how Volker explained things in a very friendly way, answered every question, sometime postponed some questions but came back to them later. All in all I now have a good overview how Qt is doing development and how I can fix bugs.

The daytrip to Rothenburg ob der Tauber was very interesting for me. It was the first time I visited the village. But in my memory it feels like I know the village already. I grew up with reading a lot of comic albums including the good SiFi comic album series "Yoku Tsuno" created by the Belgian writer Roger Leloup. Yoku Tsuno is an electronics engineer, raised in Japan but now living in Belgium. In "On the edge of life" she helps her friend Ingard, who actually lives in Rothenburg. Leloup invested a lot of time to travel to make the make his drawings as accurate as possible.

In order to not have a hard cut from Akademy to normal life, I had a lunch with Carlos, to discuss KDE Neon and how we can improve the interaction with Debian. In the future this should have less friction and make both communities work together more smoothly. Additionally as I used to develop on KDEPIM with the help of Docker images based on Neon I ask for a meta kf6 dev meta package. That should help to get rid of most hand written lists of dev packages in the Docker file in order to make it more simple for new contributors to start hacking on KDEPIM.

The rest of the day I finally found time to do the normal tourist stuff: Going to the Wine bridge and having a walk to the castle of Würzburg. Unfortunately you hear a lot of car noises up there, but I could finally relaxe in a Japanese designed garden.

Finally at Saturday I started my trip back. The trains towards Eberswalde are broken and I needed to find alternative routing. I got a little bit nervous, as it was the first time I travelled with my Librem 5 and Itinerary only and needed to reach the next train in less than two mins. With the indoor maps provided, I could prepare my run through the train station so I reached successfully my next train.

By the way, also if you only only use KDE software, I would recommend everyone to join Akademy ;)

Overhaul Enncryption Support in Kontact

2022-10-07T00:00:00Z

For a long time I have been fixing issues behind the scenes to support Autocrypt and fixing bugs around encryption.

But the best crypto support does not help if it is too complicated for users to use the system. PGP is complex and a lot of things can go wrong, so the UI should support the user to find solutions, if things are going the wrong way. For me it was obvious that I cannot do this on my own and found Eileen Wagner a UX designer who is experienced in crypto UX. It was a lot fun to work together with Eileen to improve the UX in Kontact ;)

It soon became obvious that the part that needs an overhaul is mostly sending. There is a lot that happens AFTER you press send. You may be faced with information that the keys are not good enough, or that a used key is near expiry. So we tried to improve the UX so that these issues will bubble up earlier so you can fix the issues before pressing send.

At least for me, it is often that I concentrate in order to finish a message before I need to go, and then press send in a hurry. So all dialogs and warnings are facing me while I'm in a hurry and I just want them to disappear. If instead, I know of those things in advance, I will have time to ask for a new key or search for the correct key for a particular recipient.

Here you see a sample of creating a message to several recipients after our improvments.

Eileen created a blog post about the thoughts behind the UX decisions made for Kontact. After several months of working together with Eileen, I realized that for outsiders, it is still hard to distinguish Kontact and KMail. Kontact is a bundle of several applications that are presented together. You are free to start and use every of those applications directly and will see no difference. You only will miss the small left-column to switch between the applications. KMail is the application that Kontact is using for the mail tab.

This work is possible because Eileen and I were funded by nlnet to improve Email Encryption in Kontact.

Near key expiry

I really like the feature that Kontact informs you about keys that will soon expire. Doing this makes it clear that I need to care about a key update and I can trigger it in advance. I know there is a lot of discussion about automatic key updates and several attempts to do this.

I am still using parcimonie for this task. But unfortunately not everyone is using a key server to communicate key updates, and nowadays there are several sources for keys update keyserver, WKD, PKA, DANE, ... Sure it would be easy to try an update in background, but starting a network connection without users consent is a no-go. So the first step is now to show the user that keys are near to expiry. In the second step, we will make it possible for the user to directly trigger an update. This will also be true if no suitable key was found for a recipient.

As I was inside the near-expiry feature code, I also added a fourth category for your own keys. I want to get informed about my own key expiry long before the key expires, to create and upload the key update, so when others are searching for the keys they already find the updated key.

Key cache usage

Until now Kontact always talked to gnupg directly using gpgme. In itself this is not an issue, but this connection is slow. This is why libkleo started to implement a key cache a while ago to cache all keys. Before, we had to wait for gnupg to answer all our requests. In my experience, that means sometimes I wait a minute or longer.

Now Kontact is also using this cache and we now can instantly show that we found PGP keys for all recipients, while typing the mail address. Do you see how fast the "green check mark" toggles while writing in the video? This "green check mark" indicates, that we found keys for all recipients. That's possible because of the key cache.

Trust levels

Gnupg now has the TOFU (trust on first use) feature that creates statistics about key usage and when we seen keys in our messages. When a key is used for a long time, the key is trusted more, and now you can detect a key with no history. This makes it harder for someone to present you with a new fake key. Of course, you get the best security by checking fingerprints and signing the recipient keys. But let’s be honest, who can do this for every key that one is using in our busy lives. For those who do not check every key, TOFU is actually a great improvement, as you build trust while using the keys.

In Kontact we are now display the trust levels instead of just validity, as the trust levels are taking the TOFU history into account. I personally cannot see any disadvantage to enabling trust-model 'tofu+pgp' in gnupg via (~/.gnupg/gpg.conf) and would highly suggest that everyone enable that in gnupg. It gives you the best of two worlds: You will be able to build trust on the keys by just using the keys (tofu part) and still can also check fingerprints (pgp part).

After enabling it I actually found out that until now the tofu data is not updated when I sent an encrypted message. However, it is done, if I use gnupg from the command line. I created a upstream bugreport for this. Until this is fixed tofu is a little bit useless, because no statistics are created. Key cache and key resolver also need to learn trust levels, as they are in charge to select the most trusted key to send messages to.

Settings

The settings in Kontact are in some corners a big list of checkboxes and it is not obvious where to find what. For Encryption we decided to merge several tabs together to present one page and name it Encryption.

There is also the signature feature, that is not connected with Cryptography but just about the mail signatures.

The critical point is the defaults for new users and we ended up having default encryption settings that can be overridden for each identity.

As all the work was done with Autocrypt in mind, I also mark Autocrypt support as stable. Now the user can enable Autocrypt within the settings page of the identity.

In the end I think these improvements take Kontact a big step forward and lets us use encryption more easily. I'm proud about the current state, but my To-Do list is still full until we have looked into all the corner cases.

January/February in KDE PIM

2022-03-02T00:00:00Z

Since the last summary two months ago we have seen the 21.12.3 patch releases of Kontact, and more than 1200 changes by 35 contributors have been integrated. Around 20 people helped to process nearly 120 merge requests the last two months, reviewing and finally merging over 80 of them.

Calendaring

The new notification-based reminder service can now launch KOrganizer or Kalendar respectively to view or edit the corresponding event. The notifications with geolocation data attached to your event or task will now open your default mapping application. The reminder service is planned to replace the previous dialog-based one for 22.04.

The identity management library has been restructured to reduce the runtime dependencies of the reminder daemon as well as other background services.

The public holidays list for several countries has been updated.

Kalendar

The Kalendar team released Kalendar 1.0, our last standalone release before moving to the KDE Gear release service.

Slawek Kaplonski worked on some small usability improvements by making for example drag and drop possible for all-day events too, and added some animation when using the 'Now' button.

Claudio Cambra continued working on making Kalendar more stable by fixing a lot of bugs. He also made many subtle user interface improvements: for example, he added the calendar's color in a few more places in the user interface.

Carl Schwan worked on making Kalendar also work with Right-To-Left languages like Arabic and Hebrew and added a progress indicator to the task list.

You can find all the changes here.

Kleopatra

Joey Berkovitz made sure that keys stored on smart cards are fully usable with Kleopatra as soon as Kleopatra has encountered the smart card. T5782, pim/kleopatra!11
Secret subkeys can now be exported without the secret primary key. This is useful if you want to share the encryption subkey of a shared email address. This is possible from the Details dialog of OpenPGP keys.
Settings that are marked as read-only (via KDE's Kiosk support or via global GnuPG options) cannot be changed anymore in the configuration dialog. T5791
One can now easily retrieve all OpenPGP keys that another OpenPGP key was certified with. Additionally, it's now possible to configure Kleopatra to retrieve all certification keys of newly imported keys automatically. T5805
One can now trigger a restart of the background processes (gpg-agent, dirmngr, scdaemon, etc.), e.g. after one has changed their configuration. T5775
Usability and accessibility of Kleopatra's main view, i.e. the certificate list, T5841 and of the OpenPGP key generation T5832 was improved. This is part of the major goal to make Kleopatra fully accessible. T5824

KAlarm

Allow calendars and date picker to be shown together in side panel #440250
Cancel sound file playback if audio alarm edit dialog is closed after clicking Try.
Fix failure to create a missing calendar file after enabling a resource.
Fix crash after Defer is selected in alarm notification message #448212
Fix deleted calendar resources reappearing when KAlarm restarts.
Fix auto-close not working for message windows after the timeout period.

KMail

Laurent Montel finished a new kmail plugin “Open Url With”, that allows one to launch specific apps for a specific URL. For example RocketChat can send notification email when a user wants to speak to you (when ruqola/rocketchat client is down). Now you can directly open ruqola and not Firefox, when clicking the URL.

Contacts

The address formatting code in the KContacts framework has been rewritten and now covers significantly more countries, supports multiple formatting styles (single- or multi-line, domestic or international, postal mail) and scripts other than Latin. This is expected in KF 5.92 and will subsequently be used in applications. frameworks/kcontacts/!34.

Itinerary

Itinerary receiving new features, such as manually added train and bus trips, or new additional information displayed in the timeline. It can now display the currency conversion rates for your trips. When you use Plasma 5.24 you can now select the default application to open geo links.

There is an own bi-monthly blog that you can check out here.

One patch highlights

Alexander Basov fixed bugs in AppArmor profile usage with Postgres database pim/akonadi!86
Oleg Solovyov fixed a Kontact crash when clicking New. (#424252)
Jonathan Marten fixed the wrong identification of MS Word attachments as encrypted data. (#90002)
Sandro Knauß fixed "X-Face can break cryptographic signatures" (#439958)

Infrastructure

A lot of work was put into building KDE PIM libraries and applications successfully on Qt6 and KDE Frameworks 6. Additionally libraries get new names in Qt6 (e.g. like Grantlee is renamed to ktexttemplate). The new name expresses better what the library is used for.

Inside kdepim repository names do not necessarily match to the Bugzilla projects/components. In the past the release team was not updating Bugzilla version information for those repositories. Together with the release team we helped to handle the connection repository <-> Bugzilla correctly to get a full list of versions for all Bugzilla products. sysadmin/release-tools!18

The benefits are that we now get more precise metadata on bug reports, and users can see and select from the full list of released versions. Additionally we have correct links in Invent to Bugzilla.

The technical details about Bugzilla information in the project API can be seen here.

Help us make Kontact even better!

Check out some of our open junior jobs! They are simple, mostly programming-focused tasks, but they don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list. And/or get in touch with us by email, Matrix #kontact:kde.org or IRC #kontact:libera.chat! We’ll be happy to guide you and answer all your questions. Read more here…

Autocrypt support in Kontact

2021-02-08T00:00:00Z

Autocrypt support is now in Kontact! This has been several weeks of work. Autocrypt makes it easier for you to use encrypted messages, as is handles key transfer for you automatically.

There are several parts involved in supporting Autocrypt. First, Autocrypt uses Protected Headers, implemented already. Within Autocrypt I found some issues and fixed them. Than I began implementing the receiving of Autocrypt messages. The key concept of Autocrypt is to always send the public key within each email, so the receivers are always able to answer encrypted. The first step was extraction of the key and saving it to disk. Because Autocrypt sends keys unverified at the moment, I decided to not import the Autocrypt keys into the users' keyrings, but keep them separately in json files under ~/.local/share/autocrypt.

After having all data of processed mails stored. I started to implement the sending part of Autocrypt. While implementing I learned that GnuPG does not like to handle two different keyrings anymore. I solved this by creating a temporary keyrings with all keys needed to encrypt a mail. This is certainly not perfect but works for now. There is room for improvement, of course. But as I needed to touch the code of KeyResolver I found out that contact preferences have been disabled for around 2 years, because back then Akonadi locked sometimes during search queries. I think this is not a problem anymore, so I will enable it again pim/messagelib!35. Hopefully with the feedback will ensure that it now just works, or we'll receive more informative failure reports that help to fix the issue.

For testing I used a mail address with an underscore and could reproduce #370385, and then made a journey into akonadi-search. It took a while before I understand the issue, because mail addresses in From or To header can be found correctly. In the end I found a fix: pim/akonadi-search!5. It would be helpful got get feedback from people who know Xapian to improve this code, because for me it feels like poking around in the dark.

Back from the journey, I needed to add Autocrypt support into identities, so you can now enable Autocrypt on an identity basis. I think the complete part is ready for enthusiasts to enable it. This as an experimental feature for now, and you need to enable it in the configuration file by hand. So if you ride the master branch you can enable it by setting Autocrypt = true for each identity in ~/.config/emailidentities. At the moment I cannot tell if this will be stable enough to show users the Autocrypt checkbox in the next release 21.04.

What is working so far?

Syncing Autocrypt storage while reading mails
sending mails to users that only have Autocrypt keys
Adding your key to the Autocrypt header and adding a Gossip header if needed

I tried to implement it in a way that will not break existing workflows. If there is at least one valid key of an user in your keyring, this will be preferred and Autocrypt does not kicks in. A valid key is a key that is neither expired, nor revoked nor disabled and suitable for encryption. Only if no valid key is found, a valid key is searched within Autocrypt storage. This ensures that existing encrypted communication is as safe as before.

What is missing?

Creating and processing Setup messages
A way to transfer your private key material between your devices
Detecting if an Autocrypt key is available while writing the mail. Currently the recipient is displayed as if no encryption would be possible
You need to request encryption explicitly
No support for prefer-encrypt preferences
signatures of mails cannot be verified against an Autocrypt key

I hope you enjoy Autocrypt support in Kontact. The next steps are to finish the missing parts and refactor the crypto support in Kontact. This work I do is supported by funding from nlnet to improve mail encryption.

Bugzilla integration for KDE Project API

2020-11-02T00:00:00Z

The KDE Bugzilla handles a lot of projects and they often match with the repo name, but not always. For instance we have ancient products and components at Bugzilla, as projects have a lifecycle from playground into Release Service, or Frameworks, sometimes with a change of name. So you may end up searching Bugzilla quite awhile for the correct product and component to be able to confirm or create bug reports against an application. Let's have a look at KPeople, and see why the situation is complicated. You find two products in KDE Bugzilla: kpeople (the repository's name) and on the other hand Frameworks have the scheme of a "frameworks-" prefix: frameworks-kpeople. From the data displayed even I as a developer am unable to tell which is the correct product to add new bug reports. Both have bug reports this year that got fixed and the number of bug reports is too low to get a clear picture of which to choose.

This is not only a problem of KDE; it is a general problem in different communities that it is hard for newcomers to find the correct place to search and add new bug reports.

That's why Debian added the bug report information for every package. This should help users to search the upstream bug reports or create new ones (Bug-Submit and Bug-Database): https://wiki.debian.org/UpstreamMetadata#Fields

While I was collecting this information for Frameworks and KDE PIM, I wondered why KDE does not have links between each project and Bugzilla. After some searching and discussions it became obvious, that KDE does not have this information that can be processed. Okay let's fix this. The obvious place to reach this information is the Project API available under https://projects.kde.org/api/. To reach this goal I began adding Bugzilla information to the data source of the Project API named Git Repo Metadata. Then a merge request later the Project API is able to generate the links to Bugzilla invent:sysadmin/projects-api!2.

Where you should search for bugs in kontact? Go to https://projects.kde.org/api/v1/identifier/kontact:

bug_database: https://bugs.kde.org/buglist.cgi?product=kontact&resolution=---
bug_submit: https://bugs.kde.org/enter_bug.cgi?product=kontact

After implementing the needed bits I found out that Nicolas Alvarez had the same idea, to store Bugzilla information in Git Repo Metadata invent:sysadmin/repo-metadata@085d878ea. Fortunately I can say that since June the information are used by Project API.

So now, back to my task to add upstream metadata to Debian packages. After I filled the needed information to Repo Metadata I created a script to update the links in Debian salsa:qt-kde-team/pkg-kde-dev-scripts/function_collection/functions_plasma.py:addMissingBugMetadatafields. This should hopefully help to always point to the correct Bugzilla links in future.

A random list came to my mind of places that may benefit from the Bugzilla information in Git Metadata Repository:

Links from API documentation directly to the Bugzilla.
We can use this information in scripts that adds the new version to Bugzilla (e.g. invent:sdk/releaseme/plasma/plasma-add-bugzilla-versions has currently a static list of Bugzilla products).
For DrKonqui this information may also useful. The mapping binary -> bugzilla is currently handcrafted in invent:plasma/drkonqi/src/data/mappings. This is NOT the low hanging fruit, but maybe someone feels inspired to play with the available data and who is checking each handcrafted entry in that file to still be correct? The mapping repo -> binary we get by our CI, because when we build the stuff, we know what we built.

The truth is, that Nicolas is right that adding the Bugzilla links are a manual task. But on the other hand, let's add this information once at one place and than we can use it at several places.

Please help adding this information; it is simple a yaml file. If you see the data missing example commit and create a merge request. Or you can also give me the data and I'll add it.

Akademy 2019

2019-12-09T00:00:00Z

At this year's Akademy I had great moments with new and already known people. Akedemy gives me much power for hopefully the rest of the year. I really enjoyed the daytrip to the lake. It was calm and beautiful environment. The daytrip helped me to calm down again. Together with Leiner, Florian and Valorie we sat down to discuss issues for newcomers attending Akademy the first time while having an amazing lunch. Is it often hard to remember how hard it can be to attend the Akademy the first time without knowing lots of people. The outcome of this discussion will feed back to community after some more cleanup of our notes. Hopefully we can make the next Akademy even better for newcomers next year!

My highlights from the first two days of great talks are Kirogi and "Developers Italia". I really enjoyed seeing that Open Source reaches more and more domains and now you can even control your drone with Open Source named Kirogi. The software itself looks already quite usable and I'm looking forward what features we will see there in future...

"Developers Italia" was an eye opener, in how governments can change the laws so administrations must invest in Open Source. In Italy, administrations are forced to search for an existing solution in Open Source and then use this solution. If the software does not work for them they can pay developers to implement their needed features, but still the code will be owned by the administration and they need to publish the code afterwards under an Open Source license. I'm very interested to see how this will develop in future, because at the moment I still have the bad feeling that some big companies may have the ability and also the desire to destroy this revolutionary idea, with the result that only some big companies will get all the big grants, and the result will be bloated unusable Open Source software. But none the less, let's give the Italy administrations a warm welcome and give them a hand to become good Open Source citizens.

I also enjoyed the talk by Albert about the status of fuzzing KDE software. Albert explained, that the first Frameworks are covered by fuzzing, and the results that were found by the fuzzer. The first days and weeks spit out a lot of interesting issues, but nowadays, the fuzzer takes a lot of time to find new issues. So it is time now to add the next set ready to be fuzzed. I talked with Albert about what would be the most valuable parts of KDEPIM that should be covered by fuzzing. The first set is KMime, KContacts and KCalenderCore as they handle input without any user interaction.

As Qt6 is planned for next year, KDE needs to plan KDE Frameworks 6. In a BoF we discussed the timetable and also some things to do to prepare. Within the same day a Phabicator board was set up and the first review request was accepted. The first big task is not to get rid of KDE4Support and where possible remove deprecated Qt5 API.

Within the KDEPim BoF we discussed, what parts of KDEPIM are ready to become a Framework. Volker has done a really good job in getting KContacts and KCalendarCore into Frameworks. Then we looked at what external applications depend on KDEPIM and why they depend on our stuff. We can see clearly, that most of external applications want to use an address book. The solution we favor is that the external application use kpeople and kpeople has a plugin system where Akonadi can have a plugin. Bhushan and Nico are willing to implement this. It is great that we now also get a stronger connection to Plasma Mobile, as they are interested in a PIM solution for the phone too. We all know that the current stack with Akonadi is not ready to be pushed to the phone, but we can make sure that as much code as possible is reused on the phone. We named the two parts PIM dinosaurs (the desktop apps) and PIM mobile. The most important part is that we are in one team.

As there was also some time in between the BoFs. I spent time to make MemoryHole (encrypted mail headers) support ready for KMail. Together with some time after Akademy, we now have MemoryHole support for the MailViewer implemented. The next step, to implement the sending part, is still being developed. I also took the opportunity to talk with Jonathan and Daniel to get AppArmor support for Akonadi Server. The state before Akademy was that Neon and Debian had both implemented a AppArmor profile, and they were not same, so we sat down together and started to took into it. This unified profile is now ready and will be shipped with the next release 19.12.0.

The last topic I want to talk about is the privacy goal. This goal is surely not finished and we want to continue. At this point we now slowly starting to really work as a strong team of people. And we have a hard task, because privacy is difficult. This makes it hard to pinpoint big issues we can improve. Privacy is more about the small details, which made it hard to write a guide for developers. So we planned how to make it visible that we are not dead, so we want to do a bimonthly blog post series to show the world what we accomplished within those two months. Also we tried to identify the KDE Applications that are in the scope of the Privacy goal.

I'm not able to write everything that happened during Akademy, as it is simply too much. Talking and discussing things or just getting status updates of people's lives is so rich and give me a lot of energy for the year ahead until the next Akademy!

Akademy 2018

2018-11-11T00:00:00Z

You have probably read a lot about Akademy 2018 recently, and how great it was. For me it was a great experience too and this year I met a lot of KDE people, both old and new. This is always nice. I arrived on Thursday so I had one day to set everything up and had a little bit of time to get to know the city. On Friday evening I enjoyed the "Welcoming evening", but I was very surprised when Volker told me that I would be on stage the next day, talking about privacy. He told me that someone should have informed me several days before. The scheduled speaker, Sebastian, couldn't make it to Akademy.

That was really a pity. If I had known earlier, I would have prepared a proper presentation, as I think I have a lot to say on the topic. I care a lot about good encryption support in KDE PIM and I have also been busy with Tails, a distribution strongly focused on privacy and anonymity.

So the next day I rambled a bit about encrypting mail headers (also known as Memory Hole), but it wasn't a great talk. It was also very stressful, as we started with a lot of delay: Setting up the audio took much longer than expected so all talks started late. As I was in such a hurry. I forgot a lot of what I wanted to say and failed to draft a greater picture for the privacy goal.

To compensate, let me say it here. Privacy starts with encrypting everything, starting with the content of the connection. This is mostly done via TLS. Here KDE is lacking some bits, as we cannot display the encryption methods, so we are sometimes forced to downgrade to old and insecure encryption methods without user visual feedback. But at least we support TLS everywhere! Unfortunately metadata is often enough to track where people are. To hide this information, we need TOR/VPN support within the applications. For most applications you can use the TOR network by running

torsocks <cmd>

but this is not really user-friendly. It also has the disadvantage that you can't see if DNS requests are transmitted via the TOR tunnel. And then there's deamons, which you don't normally run from the command line. Furthermore, e.g. GnuPG has an option to enable TOR support and does not like to run using torsocks.

We need more control over the used TLS certificates and a way to display the TLS parameters, like accepted ciphers, protocol versions etc. Because at the moment KDE just asks you about unknown certificates, but you can't view/ control the other important details of a TLS connection to harden your system. Sure a default user can't decide whether something is good or not and a user may need to access some rotten company side. But we can provide a rating like ssllabs to show that the TLS connection is not good.

Another topic is to have a good support for TOR/VPN for applications and KDE in total in a user friendly way. And be sure that everything within these applications really uses the tunnel. I think at usecases like: I want TOR for everything except this application. After we have good solutions for those, we can focus on all the different applications. I haven't a good overview what each application should do in regards to the Privacy goal additionally. As I'm busy within KDE PIM I see some things there:

Encrypting headers of mails to give less metadata to surveillance T742.
Make it possible to search in encrypted mails. Until now, the encrypted data are a binary blob to the search. The search is important to find mails again and is often a reason, why users do not use encrypted messages. T8447.
Protect Akregator webviewer against ad trackers T7528.
Add GnuPG TOFU (Trust On First Use) trust model support for KMail. With TOFU you will get more security without exchanging the gpg fingerprints. To make it clear, exchanging fingerprints gives more security than is achieved with TOFU, but it is exhausting. Only a small number of users exchange fingerprints. So many users will profit from having extra security by the TOFU security model being supported by KMail.

To move forward in reaching the privacy goal, I volunteered to organize a Sprint together with Louise Galathea. It will hopefully take place next Spring.

But back to the Akademy - two days of many talks which was pretty overwhelming, in a good way. I really enjoyed the mix of non technical talks and technical talks. My highlight was a talk about Kontact from Markus Feilner. It is interessting to see how users master issues around Kontact and what solutions they find. The talk would have benefited from Markus talking to KDE PIM before, as some of his suggestions weren't correct. But it was nice to see that there are happy users who love all bells and whistles from Kontact. The days after the talk I encountered another user who gave Kontact a try after years, because of that talk.

The BOF session started on Monday. I started the Mondey joining the Promo BoF to get some insights into how Promo works and how KDE PIM can have better promotion to attract new contributors. Together we created a rough plan to focus first on the new website and then create several blog posts to promote KDE PIM. Internaly we started to create several junior jobs and are discussing who can mentor new people when they come along. This is still ongoing and hopefully new people will join the new spirit of KDE PIM and aren't overwhelmed by the big amount of repositories.

After the AGM I joined the Distros BoF and was impressed by how many Distros have a KDE focus. We started to talk about issues regarding KDE and downstream. Especially if distros are time based, they need a lot of manpower to validate if bugs have already been fixed upstream, or if it is a downstream bug. The fast release cycle of different KDE bundels (Plasma, Frameworks & Applications) makes it even more difficult to follow. Maybe this BoF helps the distros to talk more about their issues in the distributions@kde.org mailinglist directly and solutions can be found for common issues.

The next day I attend KConfig BOF and talked about issues reagarding config migration and how to store config. Also a very common issue is that you need to get updates of config values in your application. KDE PIM has some examples of bad decisions reagarding config handeling, as you can configure a lot of stuff and everything is seperated into several files. But the seperation is not well formed from the beginning - an issue that's grown with the years. It is also written down as task to do cleanups and make the configs easier to consume. Because a clearer seperation also helps users who want to move their KDE PIM from one computer to another.

In the afternoon, the KDE PIM BoF took place, where we followed up on what has been going on since the last Akademy and what new things we want to do. We discussed a lot of how we can improve the onbording for KDE PIM. As we want to make it easier for other applications to use PIM related data, we have a long standing goal to move single parts of KDE PIM to Frameworks. For the next KDE Application 18.12 we want to push syndication into Frameworks release. If we would have more manpower, we would move those things faster to Frameworks. If you are interested, or currently using some parts of KDE PIM within your application, it is a good idea to help us getting things cleaned up and moved to Frameworks. A bigger audience can consume PIM data more easily.

In the evening I joined the LGBT & Queer Dinner and it was nice to cook together and chat.

The next days I mostly sat down and tried to get some smaller things fixed, like the issue with the suspend icon within the logout Applet. It is very good to sit next to Kai-Uwe so he could see and propose workarounds directly. Finally he found the hard coded color in the theme and the issue is solved in the next release.

Since there were many open discussions I did not managed to write a line of code within these days - I was busy talking. Additionally I wanted to see the newest KDE PIM within Debian, so this also took some time.

On Friday I took a train back to Germany. As Volker is always interested about live data from delays for improving KItinerary, he wished me some delays. And I had a lot of delay - I arrived in Berlin 12h later than expected. I had to sleep in Dresden, as the last train from Prag leaves at 18:07 towards Berlin. And since I missed that one, I had to take a local train to Dresden and there was no train anymore to Berlin. And it wasn't really late - just midnight. I never thought about that Dresden is that badly connected to rest of Germany... But it was nice as I stayed at a friends place, who I wouldn't have met otherwise. From the improving KItinerary it was not a success, as I only got one update from Deutsche Bahn at 23:25 - telling me that the train from Leipzig to Berlin would use another track. The more useful information, that I never would reach that train would be more useful.

KDE PIM sprint 2018

2018-08-05T00:00:00Z

Attending the yearly KDE Pim Sprint in April in Toulouse was nice. For me it is often leaving a cold rainy Germany and arriving warm, almost summer weather with a lot of sun. This time the weather in Germany was also sunny and warm when I left, but spring's always further in Toulouse. As only around ten people attended the sprint, it was also a time to get to know the people behind the nicknames. Unfortunately there were no new faces this time, but a new contributor joined the Pim team and attended remotely.

As the trains from Germany to Toulouse take some time, for me, the sprint normally starts with entering the train and having time to hack. The first things I looked at, were some cleanups in the dependency chain in KDE Pim, by moving stuff around.

Reaching Toulouse, David and I started to dig into the problem, that sometimes connections to remote servers stall and nothing goes back and forth without an error being triggered. This issue is only visible if the internet connection is not stable, like a connection while riding the train. Yes, it's a good thing that sometime developers have to face real world, to be able to reproduce bugs. To solve these issues we first had to reproduce them, which leads into the problem of how to reproduce an unstable internet connection. It took a while before we had a setup running to reproduce the issue and after a lot of trial and error, we finally managed to fix the issues we'd found.

Email security is a big issue in KDE Pim and KDE made privacy a goal for the next years. As a whole team we discussed what we can improve in the context of KDE Pim and added some topics in Phabricator, what we want to improve T7050 (have a look at the subtasks). As the sprint was before the public announcement of EFail, but we were already informed about this, we could discuss the outcome of EFail and add some improvements. You can read about the current situation of KMail and EFail on dot.kde.org.

We also looked at how we can support emails using Memory Hole. Memory Hole is a way to also encrypt headers in emails. The discussion has started but I havn't found time to implement it yet, see T742 for more information.

Together with Daniel I had a discussion on how to improve the way to debug issues with Akonadi. Currently it is very hard to debug issues with Akonadi, but we already have a tool for debugging: Akonadi Console. But still Akonadi Console has some issues, as you can't keep it running for days to wait for issues that happen very rarely. This is because it slows down your computer too much after some time. The Debug view shows the transmitted data between Akonadi and all clients. To make it possible to keep Debug running for days, we had to refactor the Debug view to use a TableView to display the logs instead of a simple TextView. A TableView scales much better with a big amount of data. We can now keep it running and save those logs and scan for interesting entries by hand later. Another advantage is that we can now change the filter settings while logging and have the ability to add more logic to find the interesting entries. Together with this switch we replaced the hand written Debug log protocol with JSON. This also removed the need to implement a separate parser for the protocol and you can use QJson directly for further filtering.

Another issue, that we've been having for a long time, was that you needed to restart Akonadi to view the logs, as those are only printed in the console you start Akonadi from and if you had disabled logging, then no logs were available. Also, you had to enable the correct categories to view a specific issue. Daniel used the time of the sprint to implement another tab in Akonadi Console to display those logs. So you don't need to restart Akonadi anymore to view the logs, which is a great step forward to debugging more easily.

Thanks to KDE e.V. for supporting me in joining the sprint.

Kontact on Debian

2017-12-21T00:00:00Z

When coding at Kontact you normally don't have to care a lot about dependencies in between the different KDE Pim packages, because there are great tools available already. kdesrc-build finds a solution to build all KDE Pim packages in the correct order. The KDE Pim docker image gives you an environment with all dependencies preinstalled, so you can start hacking on KDE Pim directly.

While hacking on master is nice, most users are not using master on their computers in daily life. To reach the users, distributions need to compile and ship KDE Pim. I am active within Debian and would like to make the newest version of KDE Pim available to Debian users. Because Qt deprecated Qt Webkit within Qt 5.5 KDE Pim had to switch from Qt Webkit to Qt WebEngine. Unfortunately Qt WebEngine wasn't available in Debian, so I had to package Qt WebEngine for Debian before packaging KDE Pim for Debian. Qt WebEngine itself is a beast to package. It was only possible to package Qt WebEngine for the last stable release named "Stretch" in time with the help of others of the Debian Qt/KDE mainatiners especially Scarlett Clark, Dmitry Shachnev and Simon Quigley, and we could only upload it some hours before the deep freeze. So if you have asked yourself why Debian doesn't ship 16.08 within their last stable release, this is the answer. The missing dependency for KDE Pim named Qt WebEngine. There is a second consequence of the switch: Kontact will only be available for those architectures that are supported by Qt WebEngine. Of 19 supported architectures for 16.04, we can only support five architectures in future.

Now after Debian has woken up again from its slumber, we first had to update Qt and KDE Frameworks. After the first attempt at packaging KDE Pim 17.08.0, that was released for experimental, we are now finally reaching the point where we can package and deliver KDE Pim 17.08.3 to Debian unstable. Because Pino Toscano and I had time we started packaging it and stumbled across the issue of having to package 58 source packages, all dependent on each other. Keep in mind all packaging work is not a oneman or twoman show, mostly all in the Qt/KDE Debian mantainers are involved somehow. Either by putting their name under a upload or by being available via IRC, mail and answering questions, making jokes or doing what so ever. Jonathan Riddell visualized the dependencies for KDE Pim 16.08 with graphviz. But KDE Pim is a fast moving target, and I wanted to make my own graphs and make them more useful for packaging.

The dependencies you see on this graph are created out of the Build dependencies within Debian for KDE Pim 17.08. I stripped out every dependency that isn't part of KDE Pim. In contrast to Jonathan, I made the arrows from dependency to package. So the starting point of the arrow is the dependency and it is pointing to the packages that can be built from it. The green colour shows you packages that have no dependency inside KDE Pim. The blue indicates packages with nothing depending on them. But to be honest, neither Jonathan's nor my graph tells me any more than they do you. They are simply too convoluted. The only thing these graphs make apparent is that packaging KDE Pim is a very complex task :D

But fortunately we can simplify the graphs. For packaging, I'm not interested in "every" dependency, but only in "new" ones. That means, if a <package> depends on a,b and c, and b depends on a, than I know: Okay, I need to package b and c before <package> and a before b. I would call a an implicit dependency of <package>. Here again in a dot style syntax:

a -> <package>
b -> <package>
c -> <package>
a -> b

can be simplified to:

b -> <package>
c -> <package>
a -> b

With this quite simple rule to strip all implicit dependencies out of the graph we end up with a more useful one:

(You can find the dot file and the code to create such a graph at pkg-kde.alioth.debian.org)

At least this is a lot easier to consume and create a package ordering from. But still it looks scary. So I came up with the idea to define tiers, influenced by the tier model in KDE Frameworks. I defined one tier as the maximum set of packages that are independent from each other and only depend on lower tiers:

(The dot file and the code to create such a graph you can find at pkg-kde.alioth.debian.org)

Additionally I only show the dependencies, from the last tier to the current one. So a dependency from tier 0 -> tier 1 but not from tier 0 -> tier 2. That's why it looks like nothing depends on kdav or ktnef. But the ellipse shape tells you, that in higher tiers something depends on them. The lightblue diamond shaped ones in contrast indicating, nothing depending on them anymore. So here you can see the "hotpath" for dependencies. This shows that the bottleneck is libkdepim->pimcommon. Interestingly this is also, more or less, the border between the former split of kdepimlibs and kdepim during KDE SC 4 times. I think this is a useful visualization of the dependencies and may be a starting point to define a goal, what the dependencies should look like.

You also may ask yourself why an application needs so much more tiers than complete KDE Frameworks? Well, the third tier of KDE Frameworks is more of a collection for leftovers that don't reach tier 1 or tier 2. See the definition of tier 3 is: "Tier 3 Frameworks can depend only on other Tier 3 Frameworks, Tier 2 Frameworks, Tier 1 Frameworks, Qt official frameworks, or other system libraries.". The relevant part is that a framework tier 3 can depend on other tier 3 frameworks. If you use my tier definition in contrast, then you end up with more than ten tiers for KDE Frameworks, too.

After building all of these nice graphs for Debian, I wanted to see if I could create such graphs for KDE Pim directly. As KDE is mostly using the kde-build-metadata.git for documenting dependencies I updated my scripts to create graphs from this data directly: (the code to build the graphs yourselves is available here: kde-dev-scripts.git/pim-build-deps-graphs.py)

In detail this graph looks different and not just because of the version difference (17.08 vs. master). I think we need to update the dependencies data. This also may explain why sometimes kdesrc-build don't manage it to compile complete KDE Pim in the first run.

Inital sync akonadi from commandline

2017-12-18T00:00:00Z

When you start with Kontact you have to wait until the first sync of your mails with the IMAP or Kolab server is done. This is very annoying, because the first impression is that kontact is slow. So why not start this first sync with a script, and then the data is already available when the user starts kontact the first time?

1. Setup akonadi & kontact

We need to add the required config files to a new user home. This is simply copying config files to the new user home. We just need to replace username, email address and the password. Okay, that sounds quite easy, doesn't it? Oh wait - the password must be stored inside KWallet. KWallet can be accessed from the command line with kwalletcli. Unfortunatelly we can only use kwallet files not encrypted with a password because there is no way to enter the password with kwalletcli. Maybe pam-kwallet would be a solution; for Plasma 5 it there is an offical part for this, kwallet-pam, but I haven't tested it yet.

As an alternative to copying files around, we could have used kiosk system from KDE. With that you are able to preseed the configuration files for an user and have additionally the possibility to roll out changes. F.ex. if the server addresses changes. But for a smaller setup this is kind of overkill.

2. Start needed services

For starting a sync, we first need Akonadi running and Akonadi depends on a running DBus and kwalletd. KWallet refuses to start without a running XServer and is not happy with just xvfb.

3. Triggering the sync via DBus

akonadi has a great Dbus interface so it is quite easy to trigger a sync and track the end of the sync:

import gobject
import dbus
from dbus.mainloop.glib import DBusGMainLoop

def status(status, msg):
    if status == 0:
        gobject.timeout_add(1, loop.quit)

DBusGMainLoop(set_as_default=True)
session_bus = dbus.SessionBus()

proxy = session_bus.get_object('org.freedesktop.Akonadi.Resource.akonadi_kolab_resource_0', "/")
proxy.connect_to_signal("status", status, dbus_interface="org.freedesktop.Akonadi.Agent.Status")
proxy.synchronize(dbus_interface='org.freedesktop.Akonadi.Resource')

loop = gobject.MainLoop()
loop.run()

The status function gets all updates, and status=0 indicates the end of a sync. Other than that is just getting the SessionBus and trigger the sychronize method and wait for till the loop ends.

4. Glue everything together

After having all parts in place, it can be glued into a nice script. As language I use python, together with some syntactic sugar it is quite small:

config.setupConfigDirs(home, fullName, email, name, uid, password)

with DBusServer():
        logging.info("set kwallet password")
        kwalletbinding.kwallet_put("imap", akonadi_kolab_resource_0rc", password)

        with akonadi.AkonadiServer(open("akonadi.log", "w"), open("akonadi.err", "w")):
            logging.info("trigger fullSync")
            akonadi.fullSync(akonadi_resource_name)

first create the config files. If they are in place we need a DBus Server. If it is not available it is started (and stoped after leaving the with statement). Now the passwort is inserted in kwallet and the akonadiserver is started. If akonadi is running the fullSync is triggered.

You can find the whole at (github:hefee/akonadi-initalsync](https://github.com/hefee/akonadi-initalsync)

5. Testing

After having a nice script, the last bit that we want to test it. To have a fully controlled environment we docker images for that. One image for the server and one with this script. As base we use a Ubuntu 12.04 and our obs builds for kontact.

Because we already started with docker images for other parts of the depolyment of kontact I added them to the known repository github:/cmollekopf/docker

ipython ./automatedupdate/build.py #build kolabclient/percise
python testenv.py start set1  #start the kolab server (set1)

start the sync:

% ipython automatedupdate/run.py
developer:/work$ cd akonadi-initalsync/
developer:/work/akonadi-initalsync$ ./test.sh
+ export QT_GRAPHICSSYSTEM=native
+ QT_GRAPHICSSYSTEM=native
+ export QT_X11_NO_MITSHM=1
+ QT_X11_NO_MITSHM=1
+ sudo setfacl -m user:developer:rw /dev/dri/card0
+ export KDE_DEBUG=1
+ KDE_DEBUG=1
+ USER=doe
+ PASSWORD=Welcome2KolabSystems
+ sleep 2
+ sudo /usr/sbin/mysqld
151215 14:17:25 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
151215 14:17:25 [Note] /usr/sbin/mysqld (mysqld 5.5.46-0ubuntu0.12.04.2) starting as process 16 ...
+ sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf
+ ./initalsync.py 'John Doe' doe@example.com doe Welcome2KolabSystems akonadi_kolab_resource_0
INFO:root:setup configs
INFO:DBusServer:starting dbus...
INFO:root:set kwallet password
INFO:Akonadi:starting akonadi ...
INFO:root:trigger fullSync
INFO:AkonadiSync:fullSync for akonadi_kolab_resource_0 started
INFO:AkonadiSync:fullSync for akonadi_kolab_resource_0 was successfull.
INFO:Akonadi:stopping akonadi ...
INFO:DBusServer:stopping dbus...

To be honest we need some more quirks, because we need to setup the X11 forward into docker. And in this case we also want to run one MySQL server for all users and not a MySQL server per user, that's why we also need to start mysql by hand and add a database, that can be used from akonadi. The real syncing begins with the line:

./initalsync.py 'John Doe' doe@example.com doe Welcome2KolabSystems akonadi_kolab_resource_0

Current status for mailrendering in kube & kmail

2016-07-14T00:00:00Z

In my last entry I introduced libotp. But this name has some problems, that people thought, that it is a library for one-time-passwords, so we renamed it to libmimetreeparser.

Over the the last months I cleanup and refactored the whole mimetreeparser to turn it into a self-contained library.

Usage Dependencies

As a gerneral rule we wanted to make sure, that we only have dependecies in mimetreeparser, where we we can easily tell, why we need them. We end up with:

KF5::Libkleo
KF5::Codecs
KF5::I18n
KF5::Mime

KF5::Libkleo is the dependecy we are not happy with, because it pulls in many widget related dependencies, that we want avoid. But there is light at the end of the tunnel and we will be hopefully switch in the next weeks to GpgME directly. GpgME is planning to have a Qt interface, that fulfills our need for decrypting and verifying mails. The source of the Qt interface of GpgME is libkleo, that's why the patch will be get quite small. At KDEPIM sprint in Toulouse in spring this year, I already give the Qt interface a try and made sure, that your tests are still passing.
KF5::Codecs to translate between different codes, that can occur in a mail
KF5::I18n for translations of error messages. If we want consistent translations of error messages we need to handle them in libmimetreeparser.
KF5::Mime because the input mail is a mimetree.

Rendering in Kube

In Kube we have decided to use QML to render mails for the user, that's made it easy to switch all html rendering specific parts away. So we end up with just triggering the ObjectTreeParser and create a model out of the resulting tree. The model is than the input for QML. QML now loads different code for different parts in the mail. For example for plain text it just shows the plain text, for html it loads this part in a WebEngine. But as a matter of fact, the interface we use is quite new and it is currently still under development (T2308). For sure there will be changes, until we are happy with it. I will describe the interface in detail if we are happy with it. Just as sidenote, we don't want a separate interface for kube and kdepim. The new interface should be suitable for all clients. To not break the clients constently, we keep the current interface and develop the new interface from scratch and than switch we are happy with the interface.

Rendering in Kmail

As before we use html as rendering output, but with the rise of libmimtreeparser, kmail uses also the messageparttree as input and translate this into html. So we also have here a clear seperation between the parsing step ( handled in libmimetreeparser) and the rendering step, that is happening in messageviewer. Kmail has additional support for different mime-types like iTip (invitations) ind vCard. The problem with these parts are, that they need to interact directly with Akonadi to load informations. So we can actually detect if a event is already known in Akonadi or if we have the vCard already saved, which than changes the visible representation of that part. This all works because libmimetreeparser has an interface to add additional mime-type handlers ( called BodyPartFormatters).

And additionally messageviewer now using grantlee for creating the html, that is very handy and makes it now easy to change the visual presentation of mails, just by changing the theme files. That should help a lot if we want to change the look-and-feel of email presentation to the user. And allow us additionally to think about different themes for emailpresentation. We also thought about the implications of the easy changeable themes and came up with the problem, that it shouldn't be that easy to change the theme files, because malicious users, could fake good cryptomails. That's why the theme files are shipped inside resource files.

Meanwhile I was implementing this, Laurent Montel added javascript/jQuery support to the messageviewer. So I sat down and created a example to switch the alterantivepart ( html and textpart, that can be switched) with jQuery. Okay we came to the conclusion that this is not a good idea (D1991). But maybe others came up with good ideas, where we can use the power of jQuery inside the messageviewer.

libotp - email rendering in kube

2016-02-05T00:00:00Z

The important part of a mailreader is rendering an email. Nobody likes to read the raw mime message. For kube we looked around what we should use for mail rendering. and came to the conclusion, that we will use parts of kdepim for that task. But the current situation was not the usable for us, because is was tangled together with other things of the mailviewer (like Akonadi, Widgetcode,...) , so we would end up depending on nearly everything in kdepim. What was a nogo for us. But after a week of untangeling the rendering part out of messageviewer, we end up with a nice library that does only mail rendering called libotp branch dev/libotp. But for the moment it is just a working name. Maybe someone come up with a better one?

Why a lib - email rendering is easy?

Well if you look from the outside, it really looks looks like an easy task to solve. But in detail the task is quite complicated. We have crypted and signed mail parts, html/not html mailparts, alternate mime structure, broken mail clients and so on. And than we also want user interaction, do we want to decrypt the mail by default? Do we want to verify mails by default? Do we allow external html links? Does the user user prefer html over non html? ...

In total you have to keep many things in the mind while rendering a mail. And we are not talking, that we also want a pluginable system, where we be able to create own rendering for special types of mails. All these thing a already solved by the messageviewer of kdepim: We have high integrated crypto support, support for many different mime types and it was already used for years, additionally the test couverage is quite high. Like you see in the image above we are already been able to decrypt and verify mails.

libotp

libotp is a library that renders emails to html. Maybe you ask, why the hell html? I hate html mails, too :D But html is easy to display and back in time there was no alternative, to show something that is that dynamic. Nowadays we have other solutions like QML, but still we have html message, that we wanna be able to display. Currently, we have no way, to try out QML rendering for mails, because the output of libotp is limited to html. I hopefully can also solve this to give libotp the ability to redner to different output formats, by splitting the monolithic task of render an email to html into a parse step, in which the structure of the email is translated into the visible parts (a signed part is followed by a encrypted one, that has as child a html part,...) and the pure rendering step.

If you follow the link libotp branch dev/libotp, you may wonder, if a fork of messagelib is happening. No the repo is created, to use libotp now in kube and I made many shortcuts and use ugly hacks to get it working. The plan is that libotp is part of the messagelib repo and currently I have already made it to push the first part of (polished) patches upstream. If everything went fine, I will have everything upstreamed by next week.

How to use it ?

At the moment it is still work in process, so it may change. Also if other step up and give input about the way they wanna use it. Let's have a look how kube it is using kube/framework/mail/maillistmodel.cpp

// mail -> kmime tree
const auto mailData = KMime::CRLFtoLF(file.readAll());
KMime::Message::Ptr msg(new KMime::Message);
msg->setContent(mailData);
msg->parse();

first step - load the mail into KMime to have a tree with the mime parts. file is a mail in mbox format located in a local file.

// render the mail
StringHtmlWriter htmlWriter;
QImage paintDevice;
CSSHelper cssHelper(&paintDevice);
MessageViewer::NodeHelper nodeHelper;
ObjectTreeSource source(&htmlWriter, &cssHelper);
MessageViewer::ObjectTreeParser otp(&source, &nodeHelper);

now initalize the ObjectTreeParser. Therefore we need

HtmlWriter, that gets html output while rendering and do what ever the user wants to do with the html output (in our case just save it for later use - see htmlWriter.html()).
CSSHelper creates the header of the html with css, you have the possibility to set color schemas and fonts that are used for html rendering.
NodeHelper is the place, where information are stored, that need to be stored for longer (pointers to crypted part, that are currently in asynchronously been decrypted, or extra mail parts, that are visible but only on mime part in the mail). NodeHelper also informs you if any async job has endend. At the moment, we don't use the async mode nor we have mail internal links working, that's why the NodeHelper is a local variable here.
ObjectTreeSource is the setting object, here you can store, if decryption is allowed, if you prefer html output, if you like emotionicons, ...

And last not least the ObjectTreeParser(Otp) itself. It's doing the real work of parsing and rendering the mail :)

htmlWriter.begin(QString());
otp.parseObjectTree(msg.data());
htmlWriter.end();

return htmlWriter.html();

After initializing the Otp we can render the mail. This is done with otp.parseObjectTree(msg.data());. Around that we need to tell htmlWriter that a html creation has begun and ended afterwards.

As you may noticed, except the ObjectTreeParser and the NodeHelper kube has overloads of the objects. This makes libotp highly configurable for others needs already.

next steps

After the week of hacking now the current task is to push things upstream, to not create a fork and focus on one solution to render mails for kmail and kube together. After upstreaming I will start to extract the parts of libotp out of messageviewer (currently it is only another cmake target and not really divided) and make messageviewer to depend on libotp. With that libotp is a independent library that is used by both projects and I can focus again to polish libotp and messageviewer.

tl;dr;

Here you see kube can now render your spam now nicely:

Like the spam says (and spam can't lie) - get the party started!

Kontact and GnuPG under Windows

2015-09-02T00:00:00Z

Kontact has, in contrast to Thunderbird, integrated crypto support (OpenPGP and S/MIME) out-of-the-box. That means on Linux you can simply start Kontact and read crypted mails (if you have already created keys). After you select your crypto keys, you can immediately start writing encrypted mails. With that great user experince I never needed to dig further in the crypto stack.

But on Windows there is no GnuPG installed as default, so I need to dig into the whole world of crypto layers, that are between Kontact and the actual part that does the de-/encryption.

Crypto Stack Kontact uses a number of libraries that the team has written around GPGME.

The lowest level one is gpgmepp which is an object oriented wrapper for gpgme. This lets us avoid having to write code in C for KMail. Than we have libkleo which is a library built on top of gpgmepp that KMail uses to trigger de-/encryption in the lower levels. GPGME is the only required dependency to compile Kontact with crypto support.

But this is not enough to send and receive encrypted mail with Kontact on Windows, as I mentioned earlier. There are still runtime dependencies that we need to have in place. Fortunatelly the runtime crypto stack is already packaged by the GPG4Win team. Simply installing is still not enough to have crypto support, though. With GPG4Win, it is possible to select OpenPGP keys, create and read encrypted mails, but unfortunatelly it doesn't work with S/MIME.

So I had to dig futher into how GnuPG is actually working.

OpenPGP is handled by the gpg binary and for S/MIME we have gpgsm. Both are directly called from GPGME, using libassuan. Both application than talk to gpg-agent, which is actually the only programm that interacts with the key data. Both application can be used from the commandline, so it was easy to verify, that they were working and that we have no problems with GnuPG setup.

So first we start by creating keys (gpg --gen-key and gpgsm --gen-key) and than further testing what works with GPG4Win and what does not. We found a bug in GnuPG in the used version, but this one was closed in a newer version. Still Kontact didn't want to communicate with GPG4Win. The reason was a wrong standard path, preventing gpgme from finding gpgsm. With that fixed, we now have a working crypto stack under windows.

But to be honest, there are more application involved in a working crypto stack. At first we need gpgconf and gpgme-w32-spawn to be available in the Kontact directory. gpgconf helps gpgme to find gpg and gpgsm and is responsible to modify the content of .gnupg in the user's home directoy. Additionally, it infoms you about changes in config files. gpgme-w32-spawn is responsible for creating the other needed processes.

For having a UI where you can enter ypur password you need pinentry. S/MIME needs another agent, that does the CRL / OCSP checks. This is done by dirmgnr. In GnuPG 2.1 dirmgnr is the only component that performs connections to the outside. So every request that requires the Internet is done via dirmgnr.

This is, in short, the crypto stack that needs to work together to give you working encrypted mail support.

We are happy, that we now have a fully working Kontact under windows (again!). There are rumours, that Kontact was working also before that under windows with crypto support, but unfortunatelly when we started the crypted part was not working.

This work has done in the kolabsys branch, which is based on KDE Libraries 4. The next steps are to merge changes over to make sure that the current master branch of Kontact, which uses KDE Frameworks 5, is also working.

Randa Coming up next week is the yearly Randa meeting where we will have the chance to sit together for a week and work on the future of Kontact. This meetings help tremendously in injecting momentum into the project, and we have a variety of topics to cover to direct the development for the time to come (and of course a lot of stuff to actively hack on). If you’d like to contribute to that you can help us with some funding. Much appreciated!