Summary

Before installing the suite, the following systems should be in place.

• A server or cluster for SugarCRM deployments with the appropriate customizations

• A Zimbra 5.x server or cluster

• A dedicated Ubuntu 8.04 portal server or cluster with appropriate Hardware Requirements

Get the latest source from the sourceforge project page

Assumptions/Requirements

A dedicated Ubuntu 8.04 portal server or cluster should be setup with the appropriate Hardware Requirements

Goals

• A symlink to the source root should be installed at /opt/datasync-suite
• Dependencies and suite libraries should be installed successfully

Steps

Extract Source

Extract the source to a folder inside /opt and within the same directory, create a symlink to it called datasync-suite.

tar -xf <tarball name> /opt/
cd /opt
ln -s /opt/<source root> datasync-suite

Install Dependencies

To automatically install the Ubuntu packages for the below dependencies use:

 cd /opt/suite
 sudo apt-get install build-essential && sudo make debpackages

• Ubuntu 8.04
• build-essential
• slapd (openldap)
• libpcre++-dev
• zlib1g-dev
• libldap2-dev
• libsasl2-dev
• libssl-dev
• mysql-client
• python-dev

Install the suite

 ln -s /opt/suite datasync-suite
 cd /opt/datasync-suite
 make develop

Enter desired password for LDAP and MySQL when prompted.

NOTE: If a package fails to download, attempt make develop again. If problems persist you will have to manually obtain and install the package.

NOTE: The suite requires that app-armor be removed and does so automatically with the develop target

Install custom LDAP configuration

cd /etc/ldap
mv slapd.conf slapd.conf.bk
ln -s /opt/datasync-suite/ldap/slapd.conf slapd.conf

After installing the symlink, edit the slapd.conf and change the rootpw setting.

Verification

The shell commands:

ls /opt/datasync-suite/
ls /opt/suite/

should both print the same listing

The shell command:

python -V

should print something like Python 2.5.X (Where X is some minor version number)

Assumptions/Requirements

The source should be gathered and installed, including any pre-requisites.

Goals

• An admin should be able to start, stop and restart the portal, nginx and slapd daemons successfully.

• An initial organization and system admin user should be added to the suite with the suiteadmin instance enabled. The user should not be called 'admin'.

• The system admin should be able to add, modify and remove users from the suite. Modifying should include adding and removing extension instances like 'changepassword' and 'suiteadmin'.

Steps

Configure Portal

cd /opt/datasync-suite/etc
cp portal.yml.in portal.yml

Edit the ldap password in the portal.yml file to match what was used in the slapd.conf

Configure Nginx

cp nginx.conf.in nginx/nginx.conf

Customize the proxy configuration by editing the nginx.conf. Nginx is a light-weight web/proxy server which the suite uses to reverse proxy requests to each suite application to the appropriate server. For example a request for /sugar/myorg would either be proxied to an apache web root running on the suite server or an apache webroot running on a seperate server. The suite also uses nginx to proxy requests to the suite's backend CherryPy process and to serve static portal content, such as the html and JavaScript for the portal and each portal extension. The default configuration within the etc/nginx.conf file has some example proxies and re-writes setup. Refer to the nginx configuration documentation when customizing this to suite your environment. For now simply comment the Zimbra, Drupal, Sugar and Joomla sections out.

Initialize LDAP

/etc/init.d/slapd stop
slapadd -l /opt/datasync-suite/ldap/provider-base.ldif
chown -R dssuite:dssuite /opt/datasync-suite/ldap/db
/etc/init.d/slapd start

Add a System User

Use the user script to add a user with just the change password and suiteadmin extension instances.

NOTE: Use something other then 'admin' as the user name

Make the user a system admin by using an LDAP browser to change the user's userClass property to be 'system'

Verification

The shell command

bin/nginx start

should start nginx

The shell command

bin/nginx stop

should stop nginx

The shell command

bin/nginx restart

should restart nginx

The shell commands

/etc/inid.d/slapd {start|stop|restart}

should start, stop and restart (respectively) the LDAP service.

Test the user by logging into the suite. The change password and suite admin buttons should be shown.

Add, modify and remove a user to verify full admin permissions.

Assumptions

At this point the portal portion of the suite should be setup with a functioning system admin user and applications should be installed and setup with the necessary customizations. Though they should not be enabled within the suite.

Goals

• Zimbra and one SugarCRM deployment should be enabled as instances within the suite.
• The system admin should be able to add, modify and remove users from both the suiteadmin and cli tools. This includes adding and removing the above instances for a user.
• A test user should be added with SugarCRM and Zimbra.
• Email Archiving should work with the test user.

Steps

Configure Nginx Proxies

Nginx is a light-weight web/proxy server which the suite uses to reverse proxy requests to each suite application to the appropriate server. For example a request for /sugar/myorg would either be proxied to an apache web root running on the suite server or an apache webroot running on a seperate server. The suite also uses nginx to proxy requests to the suite's backend CherryPy process and to serve static portal content, such as the HTML and JavaScript for the portal and each portal extension. The default configuration within the etc/nginx.conf file has some example proxies and re-writes setup. Uncomment and customize the sugar and zimbra sections of the etc/nginx.conf file. Refer to the nginx configuration documentation for more detailed information.

Adding Extension Instances

When the suite was installed, extensions were added for each application. However, an extension must be paired with a deployment which requires a different set of configurations to work for that application deployment. An extension instance is the suite's name for an extension/configuration pairing. In general a suite install will have an extension instance per application deployment. So a multi-tenant application (e.g. Zimbra) would have one instance and each single tenant application deployment (e.g. SugarCRM) will also have an instance.

Add Zimbra Instance

A Zimbra instance needs to be added in order to pair a configuration with a Zimbra deployment. See configuration options for more detail.

Add SugarCRM Instance

A Sugar instance needs to be added in order to pair a configuration with a Sugar deployment. See configuration options for more detail.

Add a User to the Instances

At this point the suite is ready to accept users using the SuiteAdmin portal extension. Follow the instructions provided in the DataSync University. Be sure to use the system user set up previously. Use this tool to add a test user to the suite with the sugar, zimbrauser and emailarchiving extensions.

Verification

• Attempt to access the proxied SugarCRM installation (ex.  http://<yourdomain>/sugar/<sugar_install_root> )

• Attempt to access the proxied Zimbra frontend (ex.  http://<yourdomain>/zimbra/ )

• Attempt to access the proxied Zimbra administration backend (ex.  https://<yourdomain>:7071 )

• Log in as the test user
• Make sure all configured apps show up as tabs on the top of the page

• Try to archive an email from Zimbra to SugarCRM.

Assumptions

The suite should be setup with applications and a test user should be added.

Goals

• An admin should be able to stop, start or restart the mq, suite-services, nginx and zimbrawaitset daemons successfully.
• A sync group should be created with a Zimbra instance, SugarCRM instance and the test user as one of the Zimbra tenants.
• All contact synchronization features should work for the user.

Steps

Configure Databases

Since this is a new system, databases will need to be created for each of the database configuration settings. Each application extension involved in synchronization, (ie. SugarCRM, Zimbra) share the same database for both the listener and dataadapter service. The message queue and pipes filters both require there own databases. Currently MySQL is the only supported database system. Create a database for each of the following systems:

• SugarCRM
• Zimbra

• MessageQueue

• PipesFilters

Sync Groups

Sync groups are a way to tell the synchronization system what data types and instances should sync with each other. Add a sync group with the two instances we had added earlier with the test user as a Zimbra tenant.

./bin/syncgrpmgr create -n <sync_group_name> -i <zimbrainstancename>,<sugarinstancename> -d person -t <zimbrainstancename>:<testuser_email_address>

Message Users

A message user is a special LDAP system user used for the sole purpose of authenticating messages sent to through the synchronization system. It also uses a level of access control to restrict messages to particular application instances. To add a message user to the system and associate that user with an existing extension instance, use the bin/message-user script. Create a user for Pipes-filters, Sugar and Zimbra

./bin/message-user -u pipesfilters_messenger -p <password>
./bin/message-user -u zimbra_messenger -p <password> -i <zimbra_instancename>
./bin/message-user -u sugar_messenger -p <password> -i <sugar_instancename>

Create Configs From Default

cd /opt/datasync-suite/etc
cp suite-services.yml.in suite-services.yml
cp messagequeue.yml.in messagequeue.yml
cp zimbra-waitset.yml.in zimbra-waitset.yml

Message Queue Configuration

The message queue process queues and persists messages sent to it within a MySQL database. The database section of etc/messagequeue.yml, the message queue configuration file, should be updated for your environment.

Suite Services Configuration

The suite services process is a service container that manages the services for synchronization within the suite. The configuration found in etc/suite-services.yml allows an administrator to decide how many processes should be started for a particular service type and configuration information for each. At a minimum we will need to configure the pipes-filters process as well as one or more Zimbra and sugar data adapter processes.

Pipes Filters Configuration

The pipes filters process is responsible for taking changes aggregated by each application listener and pushing the changes to the appropriate application channels for processing. It determines what those application channels should be based on the sync groups defined in the synchronization database. Currently the pipes filters service configuration is just at the root of the suite-services.yml. The database, ldap, messagequeue, smtp and zimbra sections should be customized for your environment. The messagequeue username and password should be match the message_user added above.

Data Adapter Configuration

The data adapter services are responsible for pushing changes destined for either one multi-tenant application instance or many single tenant application instances. In a typical suite installation you will have at least one service configured to push updates into a multi-tenant zimbra instance and at least one or more services configured to push updates to several sugar instances.

The default suite-services.yml configuration has two services configured for the mythical example.org's SugarCRM and Zimbra applications. The messagequeue, smtp and ldap sections under each dataadapter service will need to be customized.

Zimbra Waitset Configuration

In order to listen to changes from Zimbra, two different services need to be run. A Zimbra waitset service that listens for changes to accounts and a Zimbra sync request service that sends the appropriate account changes to the record changes channel (which Pipes Filters pulls from). Our service container is capable of starting up the sync request service but unfortunately is not able to start up waitset services. They must be configured and run independently.

With the bin/zimbrawaitset daemon you can, by default, start the service to monitor a single mailbox (as defined in the etc/zimbrawaitset.yml) or you can use a different argument to run multiple daemon's each monitoring a seperate mailbox. You will need to configure and run at least one waitset service for each Zimbra mailbox involved in synchronization.

Edit the instance_name and host parameters within etc/zimbra-waitset.yml.

Architecture Overview

Be sure to review the Synchronization Diagram

Verification

Start Services

The following should be run with sudo.

./bin/mq start
./bin/suite-services start
./bin/zimbrawaitset start

Note: The Warning: Table '<tablename>' already exists messages are the result of overzealous reporting in MySQL and can be safely ignored.

Sync Group Added Correctly

./bin/syncgrpmgr list

should show your sync group followed by the available data types and member-tenants.

Synchronization Works

• Log in as your test user
• Manually test that contact synchronization features work

Configure Log Rotation

The etc/logging.conf file contains configuration for determining log behavior for the suite. By default the suite's logs are found in /opt/datasync-suite/logs. To customize the logging conf to better suite your purposes, ie. log rotation see python's logging documentation.

Set Log Level

Logging level should be set in these files:

etc/zimbra-waitset.yml
etc/logging.conf
etc/suite-services.yml

*NOTE: Logging levels set in the handler_* blocks represent the highest possible logging level for the appropriate area of the suite.

Possible Logging Levels

Listed from high to low:

• DEBUG
• INFO
• WARNING
• ERROR
• CRITICAL

Verification

Use the shell command:

./bin/datasyncsuite stop

to stop all processes.

Use the shell command:

./testing/bin/clearlogs.sh

to clear the log files.

Tail the logs and use the ./bin/datasyncsuite start command to start the processes, making sure that only log statements of the appropriate level are being reported.

Note: The Warning: Table '<tablename>' already exists messages are the result of overzealous reporting in mysql and can be safely ignored.