ShoreTel Database Replication and Manipulation of MAXDBQUERIES!
March 4th, 2010
It really doesn’t matter what VoIP system you installed they all generally have one architectural characteristic in common; the configuration database.  Depending on the system, you might find a database engine that ranges in complexity from an Access Database to a full blown SQL database.  The database will store configuration information, status information and often, call detail records that document phone system activities.   The characteristic of the database that is consistent across all architectures is the fact that there can only be one “read/write” copy of that database! Some phone systems distribute the database across multiple servers.  ShoreTel, for example, distributes components of the database to application servers and distributed voice mail servers that characterize the single image architecture in a multi-site environment.   We need to better how a change to the database effects the operation of the system, the bandwidth of the WAN links and the demand on the database engine.  First, what constitutes a change to the database?   Well clearly any configuration change that is made to the system.  For example, adding or deleting a User are clearly going to cause a database update!  Lets take a more subtle example, however. Lets consider what happens to a Agent in a Workgroup, located at a remote site, behind a distributed voice mail server.  Each change that Agent makes on their Call Manager represents a database change.   Logging into the system, and Logging out of the system are database changes.  How about, accepting a call being offered to the Agent by a Workgroup? Each of these Changes is communicated to the database.  The change is first made on the “read/write” database and then replicated to the remote database copy.   ShoreTel system processes use MS Distributed Component Object Model (DCOM) objects to share information from the configuration database among themselves and to write configuration  information to the database.  User configuration options are written to the database from Personal Call Manager, and the telephone interface.  Each ShoreTel service on a distributed server caches it own copy of the configuration database.  When a distributed server loses connection to the HQ server (read “read/write” database) any changes made are no longer received by the distributed server.   If a DVM restarts without a connection the HQ database, services are started but are not functional.  When the network connection is restored, the configuration is retrieved and again cached by each service as the services become functional. If there is a flap in the WAN we note that the DVM will in fact reload a copy of the database.  This movement of the database between the HQ server and the DVM servers clearly uses bandwidth and also makes additional demands on the database engine.   In ShoreTel the database engine, is now MySQL.   The question becomes how many simultaneous database access (read, modify, write) transactions can the MySQL server handle at one time?  What happens if the transaction can not be completed?  Does it queue and retry?   In a large system of say 700 workgroup agents at a site, is it possible to overload the MySQL database with state change requests?   If you dig down through the Server registry you will find the MAXDBQUERIES is set by default to 100.  It has been our experience that, defending on the size of the system, it is sometimes necessary to dial this number back to eliminate overloading the database.   This adjustment should be made only on the DVM’s in the configuration and not on the HQ server.   You will need to reboot your DVM servers after making this change.  You should also note the difference between HEX and Decimal! The following SILENT file clip demonstrates how to make this adjustment.