Tuesday, April 14, 2009

Unified Communications - One way Audio

''This information applies to all Cisco Unified Communications System releases.''

One-way audio and no audio at all (no-way audio) are problems that are fairly common during a new IP telephony network installation. The majority of these problems are caused by misconfigurations. For one-way audio problems, always pay attention the direction in which the one-way audio is occurring. For no audio in either direction, the troubleshooting methodology is the same. You might need to repeat the procedure for each direction of audio, but more likely you will find the source of the problem when trying to troubleshoot one direction.

There are several steps you can take to troubleshoot a one-way/no-way audio problem that occurs as soon as the call connects:

:# Verify bidirectional IP connectivity.
:# Check Cisco IOS software gateway configurations.
:# Check for NAT or firewall restrictions.

Other steps are required when a call connects successfully and then suddenly becomes one-way or the audio disappears entirely.

For additional directions on troubleshooting one-way audio problems, refer to the [http://www.cisco.com/en/US/tech/tk652/tk698/technologies_tech_note09186a008009484b.shtml Troubleshooting One-Way Voice Issues Tech Note].


== Verify Bidirectional IP Connectivity ==
You should verify IP connectivity as the first step in troubleshooting a one-way or no-way audio problem because IP connectivity must be present for voice packets to be exchanged between two devices. A large number of one-way or no-way audio problems are caused by lack of IP connectivity. Check that:

:* If the two endpoints involved in the call are on different IP subnets, each endpoint has the correct default gateway and subnet mask settings
:* If one of the endpoints is a Unified IP phone, the DHCP scope has an incorrectly configured default gateway parameter.
:* If one of the endpoints is a Cisco IOS software gateway, the default route is correct. Also, ping the other endpoint from the gateway. If the ping is successful, you know that you have IP connectivity. If the ping is unsuccessful, perform a traceroute to determine where the problem lies.
::'''Note:''' Remember that signaling packet traffic is always between Unified Communications Manager and the endpoint, whereas the RTP voice packet traffic is directly between the endpoints. So just because the endpoints are registered to Unified Communications Manager and can set up a call through Unified Communications Manager does not mean that the endpoints have proper IP connectivity between them.

Another useful tool for troubleshooting such a problem is the help ('''i''' or '''?''') button on Cisco Unified IP phones. Press the button twice in quick succession during an active call. The display shows you receive and transmit statistics for the call. If you do not see the receive counter (RxCnt) incrementing, the packets are probably not arriving on that IP phone. If you go to the originating IP phone and the transmit count (TxCnt) is incrementing, the packets are probably being lost somewhere in the network.

If a ping or traceroute does not provide enough information about where the packets are being lost, you may need to connect a sniffer to the network and perform the following steps:

:# Connect the sniffer to the back of the originating IP phone and make verify that the phone is actually transmitting packets.
:# On the originating phone, verify that the IP address and MAC address information is correct.
:# If the network settings on the originating phone are correct, go to the terminating IP phone to verify that the packets are not arriving.
:# If the voice packets are not arriving at the terminating phone, move the sniffer from network hop to network hop to isolate where the packets are being dropped. A common reason for a problem such as this is a missing or improperly configured IP route.


== Check Cisco IOS Software Gateway Configurations ==
There are various reasons why you might encounter one-way audio on calls to a Cisco IOS software gateway. Most of these problems can be solved using simple configuration commands.

:# '''Check if IP routing is enabled on the gateway that you are using''' You do not need to be running a routing protocol such as RIP, EIGRP, or OSPF, but IP routing must not be disabled. Make sure that the no ip routing command is not in your configuration. If it is, be sure to eliminate it by configuring the ip routing command. You can also issue the show ip route command to see if IP routing is enabled. If IP routing is disabled, there are no routes listed in the output, and the list of routing protocols is not present.
:# '''Determine if the VoIP subsystem is enabled''' The VoIP subsystem in Cisco IOS software uses the IP routing code to aid in encapsulating and transmitting the VoIP packets, so the subsystem must be enabled to transmit and receive VoIP packets. It does not need the IP routing code to perform signaling such as H.323 or MGCP, so the signaling still works with IP routing disabled.
:# '''Check IP address configurations on gateway interfaces''' Another common occurrence of one-way audio appears on Cisco IOS software H.323 voice gateways that have more than one data interface, such as a gateway that has both an Ethernet connection to the LAN and a serial connection to the WAN. When an H.323 gateway is configured in Cisco Unified Communications Manager Administration, you configure a specific IP address. Cisco Unified Communications Manager always uses this IP address for all its signaling to the gateway; however, Cisco IOS software voice gateways by default use the IP address of the interface that is closest to the destination. This could be a problem if Unified Communications Manager is connected via one interface and the device to which the RTP audio stream is destined for is connected to a different interface. To force the voice gateway to always use the same IP address, configure the h323-gateway voip bind srcaddr ip-address command on the interface that you are using for signaling on the Cisco IOS software voice gateway. Make sure this is the same IP address configured in Cisco Unified Communications Manager Administration. Failure to do so could result in one-way audio when the gateway tries to use a different source interface than the one configured in Unified Communications Manager.
:# '''Configure voice rtp send-recv on the gateway''' Sometimes you have one-way audio problems only when calling specific numbers, such as 411 or 911 in the North American numbering plan (NANP) or after you transfer a call or put it on hold. If you are having these problems when going through a Cisco IOS software voice gateway, be sure that the voice rtp send-recv command is configured on the gateway. Numbers such as 411 and 911 sometimes do not send back answer supervision (that is, an ISDN connect message) when the remote end answers. As a result, the Cisco IOS software voice gateway does not cut through audio in both directions to prevent toll fraud. Configuring the voice rtp send-recv command forces the voice gateway to cut through audio in both directions immediately.
:# '''If you are using a Cisco AS5350 or AS5400 as a gateway, configure the no voice-fastpath enable command in global configuration mode''' When enabled, this command causes the voice gateway to cache the IP address and UDP port number information for the logical channel opened for a specific call and forwards the packets using the cached information. This helps marginally reduce CPU utilization in high-call-volume scenarios. Because of how Cisco Unified Communications Manager opens and closes logical channels to redirect RTP audio streams, such as in the case of a transfer or music on hold (MOH) server, the Cisco AS5350 and AS5400 cache the IP address information of the old IP address. Therefore, you end up with one-way audio when the call gets redirected to a new IP address because the voice gateway still uses the cached information instead of the newly negotiated information.


== Check for NAT or Firewall Restrictions ==
One common cause of one-way or no-way audio is when Network Address Translation (NAT), Port Address Translation (PAT), or firewalls exist between two endpoints. The SCCP protocol embeds IP addresses in the IP packet's payload to signal which IP address to send RTP packets to. If the device performing NAT or PAT is unaware of this fact, the embedded IP addresses are not translated. Therefore, one-way or no-way audio results.

Firewalls can also be a problem if they are unaware of the voice traffic passing through them. Firewalls often are configured to block all UDP traffic going through them. Because voice traffic is carried over UDP, it might be blocked while the signaling carried over TCP is passed. A sniffer is the best tool for debugging such a scenario. If both devices appear to be transmitting audio but the audio is not reaching the opposite side, take a sniffer trace at each hop along the way until you find the hop where the audio is not passing through. If the firewall is blocking UDP packets, you might need to open a hole in it to allow the voice traffic to pass through.


== Problems Occurring After the Call Connects Successfully ==
The scenarios discussed so far are cases in which you have one-way audio or no-way audio from the beginning of the call or after a hold/transfer. Occasionally, however, you might encounter scenarios in which a call is up and suddenly becomes one-way or audio disappears entirely. Network problems are largely to blame for failures of this sort. Ensure that network connectivity between the two endpoints still exists and that nothing on the network might be causing intermittent network connectivity. An example would be a flapping network connection (a network connection that constantly transitions between up and down states) or a routing protocol that cannot converge correctly. Again, a sniffer is the best tool for diagnosing this kind of problem. The best place to start is on the device that originates the RTP stream to ensure that the stream is still being generated when the loss of audio occurs. If you discover that the originating device stops sending packets for no reason, you might be dealing with a software or hardware problem on the originating device.

A common cause of such a failure is a Digital Signal Processor (DSP) crash. If the end device is a Cisco IOS software voice gateway, you see an error displayed on the console that looks similar to the following:

''%VTSP-3-DSP_TIMEOUT: DSP timeout on event 6: DSP ID=0x2312: DSP error stats''

This message is also sent to a Syslog server if the Cisco IOS software voice gateway is configured to send Syslog information to a Syslog server. On a Cisco VG200, 2600, or 3600, you can issue the following command to check the status of the DSPs: test dsprm slot # The show voice dsp command displays which port and time slot are allocated to each DSP. If the test dsprm slot # command detects a DSP that has crashed, you can compare this with the information obtained from a show call active voice command (or a show call history voice command if the call has been disconnected) to see if the time slot of the failed call is the same as the slot of the DSP that is no longer available. Unfortunately, the only way to recover from this condition is to reload the gateway.



''Have you had experience troubleshooting Cisco Unified Communications equipment? Can you expand on this topic? Please contribute to this wiki and share the tools and techniques you used by sending E-mail to [mailto:sysdocwiki@cisco.com sysdocwiki@cisco.com]. Guidelines for submitting content are available on the [[DocWiki:About|About DocWiki]] pages.''

Followers