Can I text your Enterprise Contact Center?

Phone only ‘call centers’ have been rapidly replaced with ‘contact centers’ that can also handle email and chat communications.  Customers want more options for interacting with companies they buy products and services from.  Chat requires the customer to be at a computer and though email may be sent from a mobile phone, generally the sender is at a desktop.    In a wireless world in which every man, woman and child seems to wonder around with a ‘text’  or sms enabled device on their person, does it not make sense that they would want to text your call center?

Most call centers seem to be comfortable adding more and more incoming telephone lines, but never seem to add more agents?  We now queue up more clients to the same number of agents and expect that our customer satisfaction scores will increase with each new telephone line we add.   Chat and email increase the options that an agent can use to communicate with a customer, but only text offers location independent immediacy and the highest level of accuracy in CRM integrations.

A text will be read, on the average, within 10 seconds of its arrival.  It has a significantly higher read rate than email.  It is considered spam free, as you must opt in from you own mobile phone to receive future text messages.  As most folks under 30 do not even have a land line, using the CID of a SMS text will yield a much higher accuracy rate when doing screen pops from CRM integrations.    Self Service options for SMS are enormous and scheduling an agent call back could not be any easier!

What would you rather do: call into a contact center, listen to the obligatory menu of options, self navigate to the customer service group and then hear the first queue message: “the next available agent will be with you momentarily”; or send a SMS text message directly to the contact center group, by passing the automated attendant  and if you do not receive an immediate call back, receiving a confirmation text that an agent will call you at your mobile number in four minutes?

We have created a website to enable you to immediately setup a text based marketing campaign! You can create an account at our TEXT PORTAL  and select a phone number for your campaign and be in the digital marketing world in minutes.  We give you free SMS credits when you activate your account!  Interested in extending this capability to your Contact Center?  We can implement text functionality to your ShoreTel Contact Center or CISCO UCCX in a matter of hours!

Contact or send the word CALLME to 603-426-3253 for sample application!  If you would like to test T2E (Text to Email) text your email address to the same number and we will set you up. – DrVoIP

ShoreTel fail over options using Vmware – Part 1 Building a VMware Test Lab!

The most often asked question we hear among ShoreTel system administrators is how best to achieve “fail over”, assure high availability and assure business continuity? There is no simple answer to this question nor is there any one “best practice”. It is going to depend on any number of interrelated issues including budget, facilities, availability and down time goals. With unlimited funding there are many more options then there are if we have a very limited budget. Is our deployment located in a single facility or scattered across multiple sites? Do we have an onsite data center or a “cloud” or collocation facility. Can we tolerate any down time at all or are we looking for hot fail over with no service interruption?

Redundancy by itself is not sufficient to guarantee high availability or continued uninterrupted business operations. Two power supplies are always better than one (especially if they are plugged into separate electrical sources) and RAID disk arrays are more reliable than a single spinning hard disk and iSCSI may even be a better. Many system administrators have explored commercial options like Double Take with its active/active fail over strategy.

At the end of the day, our view is that multiple hosts across multiple locations in a virtual or cloud based deployment are you best options. We think VMware and Amazon are unstoppable solutions providing high availability and business continuity assurance that maximizes budget, simplifies administration and with the lowest risk. Though we will have much more to say about Amazon Web Services, especially how it can best interface with VMware, we are going to demonstrate several configuration built on the vSphere ecosystem.

In our opinion  your ShoreTel partner should have installed your deployment on VMware from the onset.  A single VMware ESXi hypervisor host running your ShoreTel HQ Server and a ShoreTel Distributed Voice Mail server with the DVM installed at the same site and at the same level as the HQ server, will provide a very effective fail over solution at the lowest possible cost.   VMware Essentials gets you ESXi and vCenter for $540, what is to think about? A single hardware host with two virtual Windows servers, a shared iSCSI data store and a copy of FreeSCO  will best any effort to run redundant HQ servers!   ShoreTel servers can fail up, so just put all of your HQ switches and Users on the DVM and if it fails, HQ will take over.  If HQ fails, no  real harm done.

Sounds like a lot of hardware complexity but we are going to demonstrate this on a lab system consisting of a single Windows Laptop! Through the miracle of VMware,  Openfiler and FreeSCO we are going to create this entire solution and use it to prove out several different “fail over” strategies that can be used to develop “high availability” options for your ShoreTel deployment.   Additionally if your are just learning VMware, this will be an excellent  “play pen” and “sand box”and learning environment,  well within the budget of any serious student of virtualization. If you can access a lap top the rest of the requirements can be obtained as open source “free ware” or evaluation software. So lets lose the excuses and get to work!

Building your “Sandbox”!

Our test environment will consist of 3 ESXi Hosts, an iSCSI data store, a CISCO compatible routers; two Windows servers; and  two XP or better Windows PC’s.  As this entire lab will be built out on a single device here is the “parts list”:

(a) Windows laptop – If you have several spare PC’s or servers that you can make use, of great but we can build out this entire test lab on a single laptop. The only requirement is that we need 16GB of RAM! as long as you can expand the memory to at least 16GB.

(b) Your first lesson in virtualization is to understand the difference between a type 1 and a type 2 hypervisor. VMware ESXi is a type 1 hypervisor and that means that it is installed on a bare metal host computer, typically an appropriately configured server.  VMware Workstation is a type 2 hypervisor meaning it is installed on top of an operating system, like Windows,  already installed on a bared metal hardware platform.  In this case, we have a laptop running Windows 8 and we are going to install VMware Workstation on top of Windows. You can download an evaluation copy of VMware Workstation from and when the 90 day  evaluation is up, if you can find $249, it is our recommendation that no VoIP Engineer or Field technician should be without this a lap top running product.

(c) ESXi is a type 1 hypervisor and the really good news is it that it is still available absolutely free of charge. This will be the core of our test lab and we will build out three hosts, all running in VMware Workstation on our laptop, to support our ShoreTel deployment.

(d) Building out a ShoreTel HQ server under ESXi as a single server is what most folks do.  If you are going for high availability, however, you need to consider the size of your data store.   Even if you are only restoring a “snap shot”, the size of your data store may be the limitation that determines down time.  Rather than store the application data on the Windows server used for your ShoreTel HQ erver, we recommend that you install an iSCSI data store on your LAN.  In this way , if you have to restore the server, you will already have the data store available (this is where AWS S3 comes into play, so see our previous blog regarding backup strategies).  You can download community edition of Openfiler  the iSCSI data store we are going to deploy in this lab from as we will be configuring our deployment based on the availability of network area storage.

(e) One of the “must have” software tools in our Engineering laptop tool kit is a three interface router named FreeSCO!  It is pronounced “Free CISCO”as a take off on our favorite company to hate.  For those of you who ever wanted to deploy a fully functioning CISCO router from a USB drive,  down load this now from or download the ova we created which is available in the member portal of the DrVoIP web site.

(f) Lastly, you will need Windows Server software, either Windows 2008 or 2012.   You can  download an evaluation copy from Microsoft at if you do not have a copy kicking around your lab.

(g) Lastly, ShoreTel has never asked for our opinion but they do not make evaluation software easily available to lab environments or to students who hope to be future ShoreTel VoIP engineers.  ShoreTel software can only be legally obtained by purchasing a system or through a support agreement from either a ShoreTel partner or directly from ShoreTel TAC.  If you are a partner or covered under a support agreement, you can down load all sofware from the ShoreTel site.  The iPBX software will run license free for 45 days.   Our lab is going to make use of the ShoreTel HQ server, the ShoreTel DVM Server and several virtual Shoregear voice gateways.

The DrVoIP video demonstrates how this lab is constructed and how the various components are installed and is part of the over all VMware training material available (or soon to be posted) on the DrVoIP website.   This lab will enable you to not only become very comfortable with VMware in general, but help you explore the various options for providing high availability and business continuity to your ShoreTel deployment.

ShoreTel VPN or MPLS? What works and saves money?

An IPsec Virtual Private Network or VPN, is sometimes used as a backup route for a Wide Area Network failure.  VPN’s are typically deployed as a “tunnel” through the Internet and as such are “point to point” solutions by definition.  Unfortunately that will not get the job done for a VoIP deployment!  If you have ever deployed ShoreTel over a VPN in a multi site network that has more than two sites,  you will note that it has problems.  The first problem you will note  is that the Switch Connectivity display within the ShoreTel ShorewareDirector management portal looks like a Christmas tree.  Normally in a finally tuned network you should see all green in the connectivity display.  In an IPsec VPN network, using a “hub and spoke” implementation or “point to point” links you will see lots of Red and Yellow boxes and switch connectivity will be inconclusive at best.

Next, you will undoubtedly experience instances of “one way audio”. Again, this is because an IPsec VPN is a “point to point” solution, when you really require a fully messed solution that can handle more than unicast packet transfers. Additionally, as IPsec applies encryption based on a “shared key” so the two end points must possess the key! IPsec does not support Multicast or Broadcast and this make it less then desirable for a VoIP deployment. Unicast is when you address the source and destination IP address to a specific target device.  Broadcast is used when you must sent to all network devices because you do not know the destination address. Multicast is used when you send to a group of devices that monitor a target IP address for network management and service subscriptions. Using an IPsec point to point VPN might get your phones to register and enable you to make phone calls, but you will be plagued by network connectivity issues that will make your VoIP deployment problematic. Your technical support center or help desk phones will be constantly ringing with unhappy users and incomplete phone calls.

You don’t have to be a Network guru to understand a “hub and spoke” topology. All communications between devices at different sites (i.e. spoke end points) must traverse the hub site if they are to communicate between each other. This might work for unicast communication, but it is inefficient and invites disaster. For two sites (i.e. spokes) to communicate the have to go through the hub, unpacking and repacking, encrypting and decrypting, sharing keys before resending packets to the ultimate destination. Assuming you are using this configuration only as a backup during a real WAN disaster, this might be acceptable temporarily. Using IPsec VPN “hub and spoke” topology in a ShoreTel VoIP deployment, it is not very useful. We have two issues: first, IPsec does not support anything other than Unicast communication; and secondly “hub and spoke” is unworkable because “spoke to spoke” communication is required.

How do we solve this? Fortunately there are two strategies that fit the bill perfectly. First, GRE or ‘generic routing encapsulation’ should be used to support broadcast and multicast communications, a core component of any network deployment, especially those of a VoIP variety. Secondly, DMVPN or “dynamic multipoint virtual private network’ technology should be implemented to assure “spoke to spoke” communications. DMVPN, which employs mGRE (muti-point GRE) and Dynamic Next Hop Router Resolution protocol (DNHRP) technologies make it possible to deploy a ShoreTel VoIP solution over the public internet and achieve MPLS like connectivity at a fraction of the cost.  Given sufficient bandwidth, this should be more than adequate.

What about encryption you might ask?   ShoreTel, CISCO and most VoIP solutions provide encryption at the network and transport level anyway, so this component may not be needed.  If you are also moving data over this mesh, then you can use DMVPN in conjunction with IPsec to assure confidentiality, integrity and authentication (i.e. CIA).  The issue is that a fully meshed communications network is absolutely obtainable with VPN technology, but you have to implement the correct protocol to achieve the desired results!

WAN configuration is an exact science as is ShoreTel and CISCO VoIP technology. If you are fortunate to have that level of expertise in one individual or one vendor, then you are moving in the right direction with your VoIP deployment. If you need help in the WAN aspect of VoIP, then you need to call on DrVoIP. We can make the network.

Is there a RAT Virus in your phone system?

If you have a device on your network that you do not have root privileges for, then your entire enterprise is at risk for a Cybercrime! Do you want to know what a Trojan horse might look like? It might very well look like a Linux appliance provided by an outside manufacturer, delivered and installed on your network. This might be a network camera, firewall, phone system or monitoring device. As network security professionals we would never allow any device to be connected to our network, in which we did not have root administrative authority. IT Directors who budget for network security, intrusion prevention and detection and apply best practice to the care and feeding of their enterprise networks seem to overlook this very large potential security vulnerability. Every day, new networking equipment, appliances and hosts are connected to your network and nobody every questions the fact that you do not have root authority?

Most of the younger folks carrying an Android device have “rooted” their phone, why? Yet you will allow your company to install equipment for which you do not have root authority? Makes no sense to us? The fact is that most modern VoIP phone systems like those from ShoreTel and CISCO are delivered with key components built on Linux like platforms. These devices are placed on the network inside the firewall and perimeter security devices yet the root privilege is not available to the system owner. A very curious practice, would you not agree? Even if you have no clue about network security and hacking, would you allow someone to come into your place of business and install a “box” for which you have not access rights?

Anyone with root access could easily put programs on that appliance that could act unnoticed by network security devices. No virus protection would take note and the device would have complete access to the entire network. A common and popular hack is the RAT, a Trojan horse that can easily be placed on an unsuspecting users phone, computer, or other network device. These RAT’s or “remote access terminals” can be remotely controlled to turn on you microphone, camera and would have full access to all files and network resources. They become remotely controlled “bots” or computer zombies. The good news is that most modern virus protection will find these RAT’s if they are installed on a host computer. What about that appliance you just added to your network, the one you do not have root access privileges? You would never even know that RAT was there and you do not even have access permission to check!

Business owners, regardless of their personal level of technical savvy, need to question every device installed on their enterprise network. Who owns the box and who administers the box? Do you have root administrative authority on every device in your network? If not, why not?

Don’t look now but you have been hacked!

Hackers at the Front Door?

Most every home and business office now has a firewall that separates your internal computer network from the wild west of the world wide internet. The good news is that firewalls have become increasingly more sophisticated and properly configured can do an excellent job in securing your internal computer network devices.  Modern firewalls now include intrusion detection and prevention, email spam filtering, website blocking and most are able to generate reports on who did what and when. They not only block evil doers from outside your network, but they police the users on the inside from accessing inappropriate resources on the outside internet. Employees can be blocked from visiting sites that can rob your business of valuable productivity time or violate some security compliance requirement.  Prime business hours is really not the time to update your Facebook page! Nor do we want our medical and financial service folks using an instant messaging service to chat with and outsider!

The Firewall is the electronic equivalent of the “front door” to your computer network and there is an endless parade of potential evil doers spray painting your doors and windows, relentlessly looking for a way in. A properly configured, managed, and regularly updated Firewall can be very effective in protecting your computer network, both in the office and at home. Behind the firewall, must desktop computers and office servers have local software based firewalls installed that also provide virus protection.  Hopefully if something does get past the firewall, the internal virus and desktop firewall solutions will provide an additional level of security.

What is a Firewall Anyway?

Firewalls are both reasonable and appropriate but here is the bad news. Most of the hacking you now hear and read about is not done by evil doers coming through your firewall! The real damage is done by those inside your network! Malicious users and dishonest employees will always a treat. There is always the treat of the unscrupulous employee swiping credit card data or passing security information for money. The real danger, however, is from users who are just ignorant of today highly sophisticated security vulnerabilities. The most honest employee can unwittingly become the source of a major security breach resulting in the loss of their own personnel data, or the personal and financial data of your customers.

Take your average laptop user as a perfect example. How many times have you gone down to Starbucks and setup shop?  Beautiful day, open air, sun and a high speed internet connection, wireless phone and it is business as usual! If I told you how easy it is to setup a “man in the middle” attack at Starbucks you would give up coffee for the rest of your life. You think you are on the Starbucks WiFi, but actually that kid in the back of the Starbucks with the Wireless Access Point attached to his USB connector, has spoofed you into thinking he is your door to the Internet. He has been monitoring every key stroke on you laptop since you logged in. In fact he now has your log in, password and most everything else on your computer.  Now when you head back to the office and plug in,  you just unleashed a bot on the company network and he will be back later tonight!

If laptops were not enough, everybody is now walking around with a Smartphone!  Did you know that your Smartphone keeps a list of all the WiFi networks you have used recently? Remember when you were down at Starbucks checking your email while waiting for that cup of coffee? Now everywhere you go your phone is sending out a beacon request that sounds like “Starbucks WiFi are you there?” hoping it will get a response and auto connect you to the internet. Remember that kid we were just talking about?  He decided to answer your beacon request with a “yeah here I am, hop on!” Just another “MITM” attack and what he can do to your Smartphone, especially those Androids makes your laptop look like Fort Knocks!

Sometimes for fun and entertainment, while sitting at a gate in an airport waiting room, I will net scan the WiFi to identify how many phones, computers and ipads are online and connected. Not saying that I would do this, but I think you could execute a Netbios attack in less the five minutes?  It is amazing how many people leave their printer an network sharing options on when they travel.  Even more people leave their “Network Neighborhood” settings  in the default configuration!  The drill is always the same:  map the network to see what hosts are connected; port scan for know vulnerabilities; out the exploit tool kit and the rest is actually getting relatively boring for the ethical hacker.  Now credit card thieves on the other hand…….

Chances are your Internet browser is worst enemy when it comes to securing your privacy.  Every website you visit, every email you send and every link you follow is being tracked by hundreds of companies. Don’t believe me?  If you are using Firefox, install an add in extension named DoNotTrackme and study what happens.  Assuming you are an average internet surfer, in less that 72 hours you will have a list of over 100 companies that have been tracking your every move on the internet!  These companies don’t work for the NSA,  but they do sell your “digital profile” to those willing to pay for the information.  Where has your GPS been? What sites did you visit, what movies did you watch, what products did you buy, what search terms did you select – all of this dutifully reported back by you and your unsuspecting employees.  Ever wonder if your competitors want to know what your viewing on line?

Voice Over IP phone systems offer an entirely new range of vulnerabilities waiting to be exploited by the unscrupulous evil doer!  We recently illustrated to a client Law Firm (as a paid intrusion detection and penetration testing consultant and with the clients permission) just how easy it is to covertly switch on a conference room based speakerphone and broadcast the entire conference to a remote observer over the internet! In fact, capturing voice packets for replay is the first trick script kiddies learn in hacking school!

VoIP, Bluetooth, WiFi, GPS, RFid, file and print sharing and even the “cloud” all add up to a list of vulnerabilities that can be easily exploited. What can you do? You need to educate yourself and develop your own “best practice” for safe computing.  You need to educate your employees and co-workers about the various vulnerabilities we all face every day as we become more “wired” and more Mobile.  Hire a competent Computer Network Security professional to do “penetration testing” on your corporate network and firewall.  It would be better to pay a professional to “hack” you, then pay to  fix it after you have been hacked!  Remember if we can touch your network, we will own your network!

(DrVoIP provides VoIP network readiness assessments and is a certified  Network Security consultancy.   If you contact we recommend that you use Ipredator to do so!)

Hacking ShoreTel with Wireshark or Trouble Shooting One way Audio.

My First Hack?

When I was a little kid, back when there was black and white TV sets and 33 RPM records, I was always amazed at the work of the telephone company repair man! At that time there was only one Phone Company. When they sent a repair man out your house he arrived in a drab olive trunk like those used by the Army. The telephone repair man had a belt of tools including a very Kool line mans “butt set” or handset and some really super hand held drills and other stuff.

I remember watching as he installed our new “touch tone” wall phone! Then I watched as he took the “butt set” from his tool belt and like all those spy movies, he clipped it across the copper wires, which I later learned were Tip and Ring, to test the circuit! I did not even have to ask, I could hear it. When he clipped across the wires he could hear the conversations that were being held on that circuit. How freaking Kool is that!

Now with IP or VoIP telephony, the butt set has gone away, but listening in on phone calls is still possible. Forget the NSA, is one of your employees copying and recording your conversations to a USB drive and posting it on Facebook? The fact of the matter it is easier than using that old “butt set” which required a physical presence and an ability to touch the circuit. With VoIP, you can “remote “in from anywhere on the planet, do a remote packet capture and leave little or no trace that you were even there. In fact, using some deep net technology like Tor, or stacking multiple virtual machines in an Amazon cloud, not even the NSA could trace your route!

Network engineers long ago figured out they could not see the packets that run around the local area network, let alone those that go off into the Internet. Tools were needed to capture the packets, slow them down and sequence them through a protocol analysis. One of the early on tools to do this, now named Wireshark, is the minimum daily adult requirement for network trouble shooting and must definitely for VoIP problem analysis. With this software tool, a network engineer can capture all the traffic moving over the wired or wireless network that interconnects your office to the World Wide Web, and save it for future analysis. The TCP/IP protocol, though a mystery to the uninitiated, is like a microscope to a network engineer or serious hacker.

It continues to amaze me that technologically I can position myself as a “man in the middle” and basically watch as you type your user name and password into your favorite website. Bored teenagers or “script kiddy’s” now do this for light entertainment. Again, forget the NSA, your teenager has more ability to track your internet activity and probably more reason to do so. Now apply this concept to your VoIP network, and you have much the same situation. It is very possible to gather up the packets on your local network, or in route to your SIP provider and reassemble them into complete phone calls.

Next to QOS issues, “one way” audio issues are among the most common of VoIP network issues. When trouble shooting these kinds of issues on ShoreTel deployments, we typically telnet into each phone in the conversation and ping our way from the phone, to the default gateway and back to the other end. Invariable we find a configuration error in a default gateway somewhere on the network. QOS issues are best solved with a protocol analysis and verification of call control signals.

This is where Wireshark comes in.

Version 14 of ShoreTel simplifies the use of Wireshark. As a Network Engineer you are aware that if you install Wireshark on the ShoreTel HQ server, you are only going to see unicast packets sent to the Server or multicast broadcasts set to all devices on the network. Wireshark will not see unicast packets sent to the other devices on the network unless you setup remote monitoring or port mirroring. With Version 14 of ShoreTel, you can setup remote monitoring from the HQ server and copy packets for analysis and assembly. Voice or RTP media between ShoreTel phones and ShoreTel Switches is encrypted while on the network. Media traffic between devices in not encrypted and can be captured and played back. MGCP, unlike SIP, treats RTP as UDP and you will need to modify Wireshark preferences to capture it as playable voice.

The accompanying video walks you through the process of capturing VoIP traffic, looking at both MGCP and SIP call control and how to assemble voice and media streams for playback.

Top 5 Trends Transforming your Contact Center!

The Contact Center is being transformed at a rate of change that is beyond the ability of current management strategies to identify and react.   Most contact centers are still using 1990 thinking in a 2020 world!  The adoption rate of Smartphones, customer satisfaction scores through social media,  wide availability of video options, and the mobility of customer demographics are terrorizing your call center and what are you doing about it?  Still routing phone calls based on Area Code?   Queueing Callers on more incoming telephone lines, while employing less customer service representatives?  Unless you are Google or the IRS, neither of which cares about customer service, you are about to become extinct!  Here are the top five Contact Center Killers of traditional business models!

(1) Scheduled Call Back  – The traditional strategy for customer retention has been to increase the size of the “catchers mitt” by adding more incoming telephone lines.    Nobody ever says lets increase the number of agents answering incoming calls, but they are always quick to add more incoming telephone lines!   All this does is increase customer frustration, pressure agents to short change the current customer interaction and drive abandoned calls through the statistical roof!   Do not even consider this option until you have explored all the other options listed below!

(2) Mobile Phones –  Without exception, unless your client demographic is that of the Jitterbug generation, your clients are mobile phone users!  This means they have advanced smartphone functionality, SMS or Text capability and they are web savvy!   Tap the functionality of these devices to increase customer satisfaction while reducing over all costs.   Text messages can be used to initiate the Call Back function in  your contact center!   Smart Aps can be created to help clients “self navigate” through your call tree, with the the push of a single button!   Get Smart Phone integration into your contact center yesterday!

(3) Video Support – High “touch” now means Video!   The traditional talk path is narrow, strangles information and is inappropriate for todays high speed, information rich customer contact strategies.  Video offers a deeper and richer personal experience.  When it comes to “show me”, “teach me” and “help me” scenarios, one call completion statistics escalate when video is part of the contact center arsenal of customer satisfaction tools.   Get your Frequently Asked Questions into video format, or risk being ignored by a generation that might be able to read, but find YouTube a faster route to problem resolution.

(4) Social Media – Twitter can do more to damage your reputation than a bad restaurant review on Yelp!  What social media monitoring tools are in your contact center arsenal?  What website integration options have you implemented?   Can your Customer Service Representatives  open a real time video conversation with someone who has hit your website, or just told all their FB friends what the current hold time is in your Contact Center?

(5) Home base agents – did you read (1) above?   The availability of hight speed network connectivity, now makes it possible to tap a labor pool that has nothing to do with driving distance to the office! Quality, trained and experienced Customer Service representatives are out there, living where they want to live and are available to the Call Center that has put distributed workforce connectivity solutions in place.  Down the hall, or across the country, you can provide the exact same supervision, monitoring, and training for a remote customer service representative that you provide for that boiler room Contact Center that you heat, air condition, power and remains your biggest disaster recovery and business continuity challenge!

At DrVoIP we create software integrations that enable solutions for these Contact Center terrorists.    No need to throw out your current ShoreTel ECC or CISCO UCCX, we can wrap these solutions around your existing facilities with rapid deployment prototype options that have high impact and low exploration costs.    Click or Call!

Deploying VoIP in the Cloud or rolling your own “hosted PBX” – Part 1 Server Deployement

The entire subject of Virtualization and all things “cloud” has become something that even none technical people talk about.    You might say it has gone “viral” and captured the interest of geeks, business people, professional technology managers and entrepreneurs.   Personally, I never did get the whole fascination with hardware.  In my mind hardware was just something we had to put up with to get to play with the software.   When you stop and think about it, aside from the IT folks, nobody wants a Windows 2012 Server!  What they want is a Website,  a CRM package, a blog or a phone system.    Having to deal with hardware was always a chore and it always seemed to me that whatever we had was obsolete within a year or so.   The software could be upgraded, but the hardware had to be “refreshed” an expression that generally means, purchase new stuff!

Virtualization made hardware a bit more interesting.  Now we could at least run a half dozen servers on one huge hardware platform.   Back up and Restore became almost fun!  Now you start adding virtualized appliances like phone systems, gateways and firewalls to the mix and software professionals get almost giddy!     I think VMware has caused more new business creations than any other single “stimulus” package.  Now, even a guy working out of his garage could compete with the big guys!  Capital requirements were significantly reduced and new cloud based business could launch at the drop of a hat and the signing of a sales agreement!   Internet bandwidth, access, creativity and an Amazon account and you were in the revenue production business!
Unless you are in the business of refreshing hardware, why would you want to bother with any of that hardware stuff?   How long does it take your IT team to spin up a new server?   Even if you are a one man show and you can control everything without benefit of a working committee, it takes time to setup a server!   Some organizations take weeks to provision a new server!  Now if you happen to have an Amazon account, even your plain vanilla book buying Amazon account, you could spin up a new Linux or Microsoft Server in about 15 minutes!   With your “Amazon machine instance” you get a security group (read firewall) for your public IP address, a DNS name and a local network all in less time than it takes to unbox and rack a new hardware based solution.The Amazon portal lets you change the configuration of your instance on the fly.  This means you can increase disk size, RAM, change bandwidth and update your firewall without a screw driver!  Think about it, fully operational on net with pubic IP access in less than 15 minutes.

Now that 3CX, ShoreTel, Mitel  and so many others offer Gateways that are “virtual” machines, you could actually spin up a “hosted PBX” in just a few hours!   We though we would try it just for kicks!  Log into AWS spin up a new Windows Sever and deploy ShoreTel or 3CX completely virtualized, including SIP trunks, Border Controllers and Remote phones both Hard and Soft.    Should be hilarious!   (Thanks to winter storms back east, we just brought up a  169 users system, across three states and had the client fully operational in 12 hours from the emergency phone call to the DrVoIP hot line).   This first video clip just deals with provisioning the server.  In subsequent versions we will bring up an entire phone system and you can watch over our shoulders!

Backing up your iPBX Call Center, what is your plan?

“The Check is in the Mail”;
“I gave at the office”;
“The software if fully tested and bug free”;
“We are working on the documentation”;
“Go ahead tell me, I promise I won’t get mad”;
“Yes we backup our data every day, off site”.

These are just a few of our favorite white lies.   That last one, however, is a real resume creation event.  Today I took my late model SL convertible in for a smog inspection.   You could have knocked me over when the technician informed me that the car failed the certification!  Just the day before I had the car into the dealership to get the “consumer electronics” battery changed and have the normal scheduled maintenance.   Apparently, at least here in the peoples republic of California, if you have On Board Diagnostics (OBD)  in you automobile, that data is submitted or evaluated as a part of your smog inspection.  Having just had the battery changed, my dealer had failed to backup or protect this data and as a result the OBD  had no history.  I failed the smog check , wasted several hours of my life and can now retest when I get 100 miles of driving history back into the on board automotive computer.   Clearly my dealer had no data plan or process in place to protect the data during routine maintenance.

Now imagine if that had been your VoIP phone system or call center?   Simple server upgrade? New Version of the iPBX being installed?   The question is, does your dealer have a process in place to protect your data during routine maintenance?  More importantly do you have a plan and process in place for backing up your iPBX configuration database, system prompts, voice messages, call detail records and even your maintenance history (e.g. Logs)?   Want to play, “bet your company’?   Chances are that you have this on your list of “things to do” but you just have not had the time to execute.   You may even be trusting that your dealer is taking care of this as part of that expensive maintenance contract you entered into.

If you are really feeling secure about your iPBX failover plan why not just pull the power plug and test things out?  There is nothing like a crashed phone system to bring out the facts about database backup, recovery and the  business continuity preparedness of you and your vendors!  The facts are that having an active emergency back up and restoration plan in place is absolutely essential in this day and age.   Cloud backup automation services abound and there is not acceptable reason for not having this process in place.  Just as important, is a restoration plan that is periodically exercised.  You can have all the data available on backup, but if you can not restore that data, it is useless to you and your business.   Yes, it may be just another “fire drill”, but it will save your company if you include this process in your maintenance activities on a regularly scheduled basis.

Very recently we have had an opportunity to experience cloud based, on demand, iPBX redundancy and disaster recovery strategies.  We have explored some of the current cloud options for both day to day redundant operations and disaster recovery.  In one case we were able to bring up a complete PBX in the cloud in a matter of minutes, on demand using predefined Amazon Machine Instances.  The results were remarkable, dramatic and redefined operational readiness. Hit the AskDrVoIP button for details!

Trouble shoot “one way media” with the ShoreTel “phonectl” command!

One of the most common make/break/fix support tickets that come into the TAC center, have to do with “one way media”. In this scenario, a ShoreTel VoIP phone user calls another phone user, or places an outside phone call and the called party can hear the user, but the user can not hear the called party. We typically refer to this condition as “one way media”. We have look at hundreds of these situations, and though some were more difficult to resolve than others, they are generally attributable to a configuration error that defines the default gateway or a missing route.

Conceptually, your IP phone sits in a specific network. For example, your IP phone might have an IP address of which is in the network. When that device setups up a phone conversation to another phone, media(read voice) flows between the two devices. It is important to know that the “call manager” that provides the MGCP call setup and tear down information is the ShoreTel switch that the calling phone registered with, but the actual media stream, is between the two end points only. You can use the phonectl command to see which Shoregear gateway is managing the phone.

Generally, we experience the condition known as “one way media” when a phone in one subnet calls a phone in another subnet. In a multi-site deployment your phone may be in the network, but the phone you are calling might be in the network. The ability of these two end devices to set up a media stream requires that there be some routing device in the network. This routing device may be an actual router, or it might be an Ethernet switch that has “L3” (read routing ) capability.

When a device on the network wants to exchange packets with a device on another network, it sends those packets to the “default gateway”. The default gateway is an interface on a device that knows how to “route” to the other networks. Each device knows about the devices on the network it is resident in. It also knows that if it needs to communicate with a device in another network, it needs to send that request to the default gateway. The default gateway, will then forward it on to the target device, or to its own default gateway, until it reaches a device that knows the target device.

There are a few questions you need to ask when troubleshooting one way media:

(a) Can I make a call between phones in the same network?

(b) Can I ping the ShoreTel HQ server:

(c) Can I ping the ip address of the device (phone or gateway) that reports the one way media;

There are a couple of ShoreTel related exe files that are useful in trouble shooting one way media. You are going to want to see the network from inside the network device, regardless if it is a switch or a phone. ShoreTel has a security shell that runs on phones and switches. You will need to disable this shell, to enable the ability to telnet into the switch or phone. First, you will need to enable telenet with the ShoreTel ipbxctl command. You will also use this command to telenet into a phone (see previous blog “how to telenet into a ShoreTel phone). You will then telenet into the phone and test for network connectivity by use of the PING utility.

Invariably one way media can be traced to a network configuration error. Either a device somewhere in the network has the wrong default gateway; or the default gateway does have route to the destination network. As an aside, there was a time in which the standard ShoreTel media stream, always used transport level port 5004. A one way media condition, generally across a WAN, might be the result of having port 5004 blocked in one direction on a firewall. From a QOS perspective, advantage to ShoreTel as we could not only prioritize Voice over Data at the IP level but also at the TCP or transport layer. With the move to SIP, the RPT media stream is moving on ports all over the map so this is no longer high on the check list.