Inital sync akonadi from commandline

Dec 18, 2017 kolab kde kontact

When you start with Kontact you have to wait until the first sync of your mails with the IMAP or Kolab server is done. This is very annoying, because the first impression is that kontact is slow. So why not start this first sync with a script, and then the data is already available when the user starts kontact the first time?

1. Setup akonadi & kontact

We need to add the required config files to a new user home. This is simply copying config files to the new user home. We just need to replace username, email address and the password. Okay, that sounds quite easy, doesn't it? Oh wait - the password must be stored inside KWallet. KWallet can be accessed from the command line with kwalletcli. Unfortunatelly we can only use kwallet files not encrypted with a password because there is no way to enter the password with kwalletcli. Maybe pam-kwallet would be a solution; for Plasma 5 it there is an offical part for this, kwallet-pam, but I haven't tested it yet.

As an alternative to copying files around, we could have used kiosk system from KDE. With that you are able to preseed the configuration files for an user and have additionally the possibility to roll out changes. F.ex. if the server addresses changes. But for a smaller setup this is kind of overkill.

2. Start needed services

For starting a sync, we first need Akonadi running and Akonadi depends on a running DBus and kwalletd. KWallet refuses to start without a running XServer and is not happy with just xvfb.

3. Triggering the sync via DBus

akonadi has a great Dbus interface so it is quite easy to trigger a sync and track the end of the sync:

import gobject
import dbus
from dbus.mainloop.glib import DBusGMainLoop

def status(status, msg):
    if status == 0:
        gobject.timeout_add(1, loop.quit)

DBusGMainLoop(set_as_default=True)
session_bus = dbus.SessionBus()

proxy = session_bus.get_object('org.freedesktop.Akonadi.Resource.akonadi_kolab_resource_0', "/")
proxy.connect_to_signal("status", status, dbus_interface="org.freedesktop.Akonadi.Agent.Status")
proxy.synchronize(dbus_interface='org.freedesktop.Akonadi.Resource')

loop = gobject.MainLoop()
loop.run()

The status function gets all updates, and status=0 indicates the end of a sync. Other than that is just getting the SessionBus and trigger the sychronize method and wait for till the loop ends.

4. Glue everything together

After having all parts in place, it can be glued into a nice script. As language I use python, together with some syntactic sugar it is quite small:

config.setupConfigDirs(home, fullName, email, name, uid, password)

with DBusServer():
        logging.info("set kwallet password")
        kwalletbinding.kwallet_put("imap", akonadi_kolab_resource_0rc", password)

        with akonadi.AkonadiServer(open("akonadi.log", "w"), open("akonadi.err", "w")):
            logging.info("trigger fullSync")
            akonadi.fullSync(akonadi_resource_name)

first create the config files. If they are in place we need a DBus Server. If it is not available it is started (and stoped after leaving the with statement). Now the passwort is inserted in kwallet and the akonadiserver is started. If akonadi is running the fullSync is triggered.

You can find the whole at (github:hefee/akonadi-initalsync](https://github.com/hefee/akonadi-initalsync)

5. Testing

After having a nice script, the last bit that we want to test it. To have a fully controlled environment we docker images for that. One image for the server and one with this script. As base we use a Ubuntu 12.04 and our obs builds for kontact.

Because we already started with docker images for other parts of the depolyment of kontact I added them to the known repository github:/cmollekopf/docker

ipython ./automatedupdate/build.py #build kolabclient/percise
python testenv.py start set1  #start the kolab server (set1)

start the sync:

% ipython automatedupdate/run.py
developer:/work$ cd akonadi-initalsync/
developer:/work/akonadi-initalsync$ ./test.sh
+ export QT_GRAPHICSSYSTEM=native
+ QT_GRAPHICSSYSTEM=native
+ export QT_X11_NO_MITSHM=1
+ QT_X11_NO_MITSHM=1
+ sudo setfacl -m user:developer:rw /dev/dri/card0
+ export KDE_DEBUG=1
+ KDE_DEBUG=1
+ USER=doe
+ PASSWORD=Welcome2KolabSystems
+ sleep 2
+ sudo /usr/sbin/mysqld
151215 14:17:25 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
151215 14:17:25 [Note] /usr/sbin/mysqld (mysqld 5.5.46-0ubuntu0.12.04.2) starting as process 16 ...
+ sudo mysql --defaults-extra-file=/etc/mysql/debian.cnf
+ ./initalsync.py 'John Doe' doe@example.com doe Welcome2KolabSystems akonadi_kolab_resource_0
INFO:root:setup configs
INFO:DBusServer:starting dbus...
INFO:root:set kwallet password
INFO:Akonadi:starting akonadi ...
INFO:root:trigger fullSync
INFO:AkonadiSync:fullSync for akonadi_kolab_resource_0 started
INFO:AkonadiSync:fullSync for akonadi_kolab_resource_0 was successfull.
INFO:Akonadi:stopping akonadi ...
INFO:DBusServer:stopping dbus...

To be honest we need some more quirks, because we need to setup the X11 forward into docker. And in this case we also want to run one MySQL server for all users and not a MySQL server per user, that's why we also need to start mysql by hand and add a database, that can be used from akonadi. The real syncing begins with the line:

./initalsync.py 'John Doe' doe@example.com doe Welcome2KolabSystems akonadi_kolab_resource_0