Wireshark-users: Re: [Wireshark-users] Slow database access **RESOLVED**

From: John Hinckley <john@xxxxxxxxxxxxxxxx>
Date: Tue, 1 Dec 2009 14:28:25 -0800
It was, in fact, Opportunistic Locking that was causing the latency.  Odd because it didn't start happening until after I installed Symantec Antivirus on the server.  The clients were already running SAV but just pointing to a diff server so nothing really changed there.

At any rate, I disabled Opportunistic Locking on the clients (regedit), rebooted and the problem went away.  It was the "status_sharing_violation" in packet 2279 that tipped me off so another win for Wireshark!

Thanks for the information Martin, I would have missed it w/o your review of my capture!

Best Regards,
- John

On Tue, Dec 1, 2009 at 12:07 PM, John Hinckley <john@xxxxxxxxxxxxxxxx> wrote:
Martin,
 
Could the SMB errors be related to "opportunistic locking"?  If so, would there be a way to find that in a capture?
 
Thanks,
John

On Mon, Nov 23, 2009 at 8:21 PM, Martin Visser <martinvisser99@xxxxxxxxx> wrote:
I don't think your issue is network (IP layer or lower) related. The TCP checksum errors are purely due to the smart & fast TCP offloading done by your NIC. (In your Wirehark Preferences and the Protocol section turn off the checksum verification option for TCP). If you were getting real physical errors you would see TCP retransmissions of even TCP header issues, which aren't

It is difficult to determine the issue from what you have give, especicially not knowing the intricacies of your application.

Some things I have noted is that that there a few points at which your client requests SMB resources but does not obtain them causing a short timeouts. For instance the SMB requests responded to in  frames 2279, 2282 and 2286 result in a sharing violation. Between each seems to be an *exact* 1 second timeout at the server. Each query is asking in a different way (with different access requirements), I guess in order to probe for the right resource access. But this definitely would "lock" your client app while it gets the answer. But this possibly doesn't explain the 3 - 10 second lockups you are seeing.
.
The other thing I see (which has the "btrieve" TCP port 3351 ) is lots of small transactions. While I don't think it is broken, it is probably inefficient. You may find that part of it can be fine tuned (in the application)

You have provided a 2 minute long capture, and it is hard to determine exactly where the user level interactions occur - and where they match the network traffic. For this scenario, particular if the app isn't well understood, I use a bit of a logged time and motion study. To do this

1. Run wireshark in a capture mode - but only display "capture info dialog"
2. When the user starts a transaction (clicks on button, presses enter, etc) note the packet total/ seconds from the capture info dialog
3. When you see an appropriate response on the client screen (data displayed, etc), note the related packet number/time
4. Repeat as necessary with different scenarios. I try to vary the transactions that help characterise the applications. You might find some user level queries/transactions have quick responses, and others can generate a long response time. These will help show up whether there is a network bandwidth issue, or server related problems.
5. From what you have you can try and bracket what network activity matches different transactions. Looking at the IO graph on wireshark can also help visualise this.

If you want to do this and effectively annotate the capture again we could look at it some more.


Regards, Martin

MartinVisser99@xxxxxxxxx



On Tue, Nov 24, 2009 at 12:02 PM, John Hinckley <john@xxxxxxxxxxxxxxxx> wrote:
Thanks Tal and Joan!  I'm pretty sure it has something to do with the tcp checksum errors but I'm going to keep digging.

Regards,
John


On Mon, Nov 23, 2009 at 3:54 PM, j.snelders <j.snelders@xxxxxxxxxx> wrote:
Hi,
 
Hopefully this information is useful:
Slow Network Write Speeds via SMB & CIFS
<snip>
File Transfer Benchmarks:
Windows XP to Windows Server (SMB Writing): ~ 25 Mbps
Windows XP to Windows Server (SMB Reading): ~ 75 Mbps
Windows 7 to Windows Server (SMB2 Writing): ~ 90 Mbps
Windows 7 to Windows Server (SMB2 Reading): ~ 90 Mbps
<snip>

http://social.technet.microsoft.com/Forums/en/winserverfiles/thread/46898c7f-92e0-4c99-98d2-18a7458a7d2d

Best regards
Joan



From: Tal Bar-Or <tbaror@xxxxxxxxx>
On Mon, 23 Nov 2009 23:43:53 +0200 Tal Bar-Or wrote:
just from first look it seems that your DB server is having some kind file
system delay (smb) 4 sec approx, although i saw it also  from client side
in the system part.

On Mon, Nov 23, 2009 at 8:47 AM, John Hinckley <john@xxxxxxxxxxxxxxxx> wrote:
   I'm not sure whether it was a windows update or a symantec update but
something caused my customer's pervasive database to start dragging.  I took
a capture from a workstation client and it only seems slow when retrieving
the data but once the data is retrieved, manipulating seems normal.  There
are 3-10 second delays in accessing tables etc.

   The workstation IP: 192.168.0.13 (windows xp pro sp3)
   Database Server IP: 192.168.0.3 (windows server 2008 standard)
   Application Name: Agris (pervasive database)

   I've completely disabled symantec on both the client and the server and
it hasn't made a bit of difference.  I'm having a hard time sifting through
the capture I took.  All I did was have the client login and access a report
during the capture.  Maybe 20 seconds tops.

   I've attached the capture file; if anyone can see anything useful, I
would appreciate you sharing it.

   TIA,
   John





___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
            mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe


___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
            mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe


___________________________________________________________________________
Sent via:    Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
            mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe