OWA Sync V0.6

This package performs one-way synchronisation of data from a Microsoft Exchange Outlook Web Access server. This uses my OWA package.

The package copies all data from Notes, Contacts, Calendar and Mail folders on an OWA server. It converts the notes to vNote 1.1 files, converts the contacts to vCard 2.1 files, converts the appointments to vCalendar 2.0 files and stores the mail in an IMAP server. See below for information on how these files can then be used with KDE kontact and with the Nokia 770 Internet Tablet.

Note that the synchronisation is one way: changes to the Exchange data are copied to the local files but changes to the local files are ignored. No changes are made to the data on the Outlook Web Access system.

OWA replication is used so that the synchronisation normally only receives updates (changes, additions and deletions). However, the user can request a reset: all the files in the relevant directory are deleted and are all copied again from scratch. This is particularly useful if some local files have been deleted by mistake as they will not normally be renewed unless they change in Exchange.

Download

The owaSync package is available as the executable owaSync.kit starkit. For TCL programmers, the owaSync package is also available as owaSync.tcl.

Copyright (C) 2006, 2007 Graham R. Cobb. See the end of this page for my email contact details

OWA For TCL is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

OWA For TCL is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Installation

As with any starkit, on a Linux or Unix system the owaSync.kit file can be executed as long as tclkit is in the PATH. On a Windows system it will be necessary to run as tclkitsh.exe owaSync.kit ....

I recommend that the first command you issue is

owaSync.kit -version -show
This will show you the installed version number and the initial default settings, including the three folders which are created automatically (Notes, Calendar and Contacts). You can then do any -add or -delete operations you wish before trying the first update.

Note that OWA Sync is developed on Debian Linux (Etch). It has been used successfully on PPC Linux and on Windows and may work on other platforms if the necessary tclkit and TCL extensions are available.

Usage

Summary usage information can be displayed by invoking owaSync.kit with the -help option.

General Options

These options must normally be specified each time owaSync.kit is invoked (for any action).

In order to save specifying the options on every invokation, owaSync.kit looks for a configuration file. If it finds the file it executes it, using the tcl source command. The file should contain tcl set commands to set the global variables corresponding to the options. For example:

set owa_url https://somewhere.com/exchange/username/
set owa_username myuser
set owa_password mysecret
set isa_fba 0

Note that if any of these strings contain "\" (backslash) then it must be doubled. For example:

set owa_username mydomain\\myuser

The configuration file is looked for in three places:

The values in the configuration file can be over-riden by command line options.

IMAP Options

These options must be specified (or the corresponding variables set in the configuration file) if any Mail folders are to be synchronised.

Actions

If more than one action is specified they are invoked in a logical order (version, add, reset, change, update, delete, show). If no action is specified, -update is assumed.

If the database does not exist at all it will be created with three folders: Notes, Contacts and Calendar.

Todo, Known Bugs and Restrictions

Synchronising with other programs

I run this whole synchronisation process every night (using crontab on my Linux system). As well as synchronising Outlook with the local directory, I also automatically send the data to kontact and to my Nokia Internet Tablet.

For notes, contacts and appointments, owaSync is compatible with Opensync, which can be used for synchronisation with kdepim tools and many other devices. As at the date of writing (August 2007) I strongly recommend using at least version 0.22 of Opensync (and Multisync and the plug-ins).

Note: when setting up Opensync synchronisation as in the examples below, do not forget that owaSync only synchronises one way. If you make changes in kontact or GPE, Opensync will copy the changes to the OWA file directory However, the changes will not get sent to Exchange and will be overwritten if the entry changes in Exchange and is re-downloaded. Even worse, if you have entries in kontact/GPE which are not present in Exchange (entered by hand, for example) you must not use the owaSync -reset command as it deletes all files and Opensync wil then delete all the entries from kontact/GPE on its next synchronisation. If you do need to use -reset you should use msynctool --delgroup to delete the Opensync synchronisation group, and then recreate it again, before doing another synchronisation.

kontact (kdepim, kmail, etc.)

For mail, I run a dovecot IMAP server on my local system and synchronise to that. I have then set up kontact mail to access that IMAP server.

For the other folder types, to synchronise using Opensync you use the file-sync plugin and configure it to access the local directories created by owaSync. The following commands can be used to set up the synchronisation:

msynctool --addgroup sync-kontact
msynctool --addmember sync-kontact file-sync
msynctool --addmember sync-kontact kdepim-sync
msynctool --configure sync-kontact 1

The configuration for file-sync should be set up as:

<config><path>~/OWA-files</path><recursive>TRUE</recursive></config>

The actual synchronisation is done using:

msynctool --sync sync-kontact --conflict 1

For notes files, Opensync is the only way to synchronise with kontact. However, the calendar component of kontact can access the local directory of synchronised vCalendar files directly (without using Opensync) if you wish. All that is required is to add a new Calendar Resource of type "Calendar in Local Directory". It is best to make it read-only as the synchronisation is only one way and any changes will never get sent back to Exchange. If you do this you may want to add --filter-objtype event to the msynctool --sync command.

Similarly, the contacts component of kontact can access the local directory of vCard files: just add a new Contacts Resource of type Directory (and possibly use --filter-objtype contact).

Nokia Internet Tablet

I also have Notes, Contacts and Calendar synchronising with the Nokia Internet Tablet (using GPE and Opensync). This works with GPE Calendar and GPE Contacts.

For mail I just access the IMAP server I mentioned above.

The Nokia stores its notes as simple text files (actually it uses HTML but if the HTML is not exactly as it is expecting it ends up displaying the HTML instead of the text so it is best to use simple text). This requires converting the Notes files from vNote files to simple text files. The following tcl script does this:

#
# Convert the Notes from vNote format to simple files
#
proc cleanDirectory {dir} {
    foreach f [glob -nocomplain -directory $dir *] {
	if {[file type $f] eq "directory"} {cleanDirectory $f}
	file delete $f
    }
}

cleanDirectory .Notes-Nokia

set files [glob -directory Notes *]
foreach f $files {
    set in [open $f "r"]
    set out [open [file join .Notes-Nokia [file tail $f]] "w"]
    while {[gets $in line] >= 0} {
	if {[regexp {^BODY:(.*)$} $line notused content]} {
		set content [regsub -all {\\r} $content "\r\n"]
		set content [regsub -all {\\n} $content "\n"]
		set content [regsub -all {\\"} $content {"}]
		set content [regsub -all {\\;} $content ";"]
		set content [regsub -all {\\,} $content ","]
		puts $out $content
	    }
    }
    close $in
    close $out
}

I then use rsync to copy the files to the Internet Tablet using a command like:

rsync --recursive --delete --verbose ~/OWA-files/.Notes-Nokia/ rsync-user@nokia::user/MyDocs/.documents/Outlook/ --password-file ~/nokia.rsync.password

Alternatively scp could be used to copy the files.

For Calendar and Contacts I use Opensync to synchronise with GPE, using the GPE plugin and gpesyncd on the Nokia (see this Internet Tablet Talk thread to get gpesyncd). The synchronisation is set up in a similar way to the kontact case, above:

msynctool --addgroup sync-gpe
msynctool --addmember sync-gpe file-sync
msynctool --addmember sync-gpe gpe-sync
msynctool --configure sync-gpe 1
msynctool --configure sync-gpe 2

The configuration for file-sync should be set up as:

<config><path>~/OWA-files</path><recursive>TRUE</recursive></config>

As the gpesyncd daemon has no security we use SSH to run gpesyncd in interactive mode as we need it. I use SSH public key authentication so that no password is needed (password-based authentication may also work, I have not used it). So, the configuration for gpe-sync should be set up as:

<config><handheld_ip>nokia</handheld_ip><handheld_user>user</handheld_user><use_ssh>1</use_ssh></config>
where "nokia" is the host name of the Internet Tablet.

The actual synchronisation is done using:

msynctool --sync sync-gpe --conflict 1

Note: the current version of gpesyncd tends to crash if asked to read large GPE databases. If this is happening to you I recommend deleting the database files before starting the synchronisation. The following command is what I use:

ssh -l user nokia rm .gpe/calendar .gpe/contacts .gpe/todo

Of course, this causes all data in GPE to be lost and then recreated from the owaSync data. So it is only any use if you are not modifying or adding entries in GPE.

Changes

In version 0.1

Initial release.

In version 0.2

In version 0.3

In version 0.4

In version 0.5

In version 0.6

Internals (for Tcl programmers)

The owaSync package can be loaded in to any Tcl interpreter (either by loading the owaSync.tcl file and its prerequisite packages or by loading the owaSync.kit starkit, which includes the necessary packages).

Note that if the starkit is loaded into an existing interpreter (using source), the argument processing described in the Usage section is not performed. In this case, the following global variables must be set before doing the package require owaSync:

If Mail folders are to be synchronised, the following global variables must also be set:

The owaSync package is dependent on (at least) the following packages: owa, webdav, tdom. It is also dependent on tls if the OWA URL starts https:.

Loading the owaSync package opens the webdav connection to the server and restores the folder synchronisation data.

The folder synchronisation database can be manipulated using the DBList, DBAdd, DBDelete and DBReset procedures. Synchronisations are performed using the OWAUpdate procedure. Note that the database and synchronisation procedures automatically rewrite the synchronisation data (for all folders) when the update has finished successfully.

Debugging

Debug printing uses my dbgprint package. The undocumented -debug option sets the control variable described below. If you are interested in using the dbgprint package in your code, read the comments in the dbgprint.tcl file. Some of that information is included here to help anyone trying to use the -debug option.

The control value is parsed as follows:

Debugpoint name matching occurs as follows: If the control element begins with :: then it has to either match the whole debugpoint name or a prefix of the name ending with ::. If it does not beging with :: it is matched against rightmost element of the debugpoint name (after the last :: in the debugpoint name, if any). The name matching uses string match -nocase.

Some examples:

The most useful -debug values to use are probably:

Contacting me

Please contact me by email on owa-tcl AT cobb.uk.net.

If you use owaSync, please let me know of your experience. In return I will let you know of any updates.


This page has been accessed Access counter times.

Graham Cobb
Creative Commons License
This web page is licensed under a Creative Commons Attribution-ShareAlike 2.0 License.