["Small, James" <JSmall@xxxxxxxxxxxx>]

Roland,

What kind of problems?  Do the transfers abort?  Are they slow?

When dealing with a carrier, you need to be specific.  Remember that carriers deal with troubleshooting Internet traffic for a living so they are understandably skeptical if a non-carrier tells them there is something wrong with their network.  Not to say that there are never carrier issues - there are.  However, most of the time, the problem is not with the carrier/ISP.

Things you can do to show the customer ISP that you've done your homework:
1)  Perform download/speedtests throughout the day.  I like speakeasy.net/speedtest.  Keep track of the results.  Is the problem just during business hours or all day long?  What about on the weekends?
2)  If you look at all the customer equipment - are there any interface errors?  Look at the end user PC, all routers, switches, firewall, traffic shapers, in-line security equipment, and anything else that touches Internet traffic.  Do any of them have any interface errors?  Are all of them operating at the correct speed and duplex settings?  Are all of them running solid network drivers and solid code versions with no known bugs/issues?
3)  When you get the problems - can you demonstrate them from multiple sites on the Internet?  Are you sure it's not just one site or another customer?
4)  You can also run tools to monitor Internet usage - is the customer maxing out their Internet pipe?  MRTG (UNIX/Linux), PRTG (Windows) are great and free/cheap tools to monitor Interfaces.
5)  Ask your ISP how they do speed tests.  Many ISPs have their own internal speed test or will setup an iperf server to allow CPE testing.
6)  Look at utilities like pingplotter and NetFlow Analyzer to watch traffic over time.


Once you do all of this and document your findings - if you're still stumped, you can forward your findings off to the customer's ISP to show them what you've done.  Then ask for their help.  Tell them you've done everything you can think of and ask them what else you should try to isolate the problem.  Often times if you show an ISP that you've done your homework and made a reasonable effort to rule out any CPE issues, they will then take the time to seriously look at their equipment to see if anything is amiss.  They might also ask you to run some more tests - but as long as you work with them you should be able to get to the bottom of it.  Put on your patience hat though - troubleshooting Internet performance issues can be difficult and is often very time consuming.

--Jim
 
________________________________________

I have a couple of customers that have been complaining of issues on their circuits, an issue that causes them to have problems with large file transfers. The only noteworthy problems in their data streams seem to be TCP Dup Acks - I've seen as many as sixty, or over a hundred, in file transfers of 100 MB test files. However, as near as I can determine, these errors are being introduced in the Internet, outside of our network (the customers use VPNs over internet circuits with major carriers for these file transfers).

As I said, we've tested our own network thoroughly, but I'm at a loss as to where to go with this issue. Obviously, telling the customer, "It's not our fault" is unacceptable, as that doesn't move them any closer to error-free file transfers. On the other hand, I'm not sure where to tell the carriers' help desk technicians to look for the source of this issue. Has anyone seen this before on Internet circuits, and is there some way I can use Wireshark to help pinpoint the issue more specifically than telling the carrier, "It's in your cloud"?