John's Blog: Broken Transfers

June 8, 2012

Rumpus is a large, sophisticated piece of software, but despite the fact that its primary job is transferring files, the act of actually handling file uploads and downloads is almost trivial. Handling logins, managing user accounts, sending notices, displaying directory listings, etc. is where Rumpus does the heavy lifting, but once a file transfer has started and is in progress, the server is really doing little more than reading bytes from the network and saving them to disk (or vice-versa).

So, if users are able to log in, access directories and generally work within the system, but file transfers are unreliable, diagnosing problems usually means looking at the network, your Mac, your Internet connection, or the client's network. Here are some questions you can ask to help diagnose the problem.

Is there any reliable way to reproduce the failure?

It's really helpful when there is a consistent, reproducible way to cause the problem. It makes finding the culprit, and confirming that the problem is fixed, a lot easier.

Does the problem occur for a specific user?

When most of your clients are able to reliably transfer files to and from your server, but one client (or perhaps serveral clients that are all at the same site) has trouble, the likely culprit is the client's network.

If you run your Rumpus FTP or Web service on an alternate port (some port other than 80 for Web service, 21 for FTP), it's a good bet that the client's network is protected by a firewall that blocks access to services on non-standard ports. Unfortunately, the only way to correct this is to run your Rumpus service on a standard port or to have the client ask their network administrator to change the firewall rules to allow access to your service.

The problem could also be related to the user account. Try logging in using the problem user's name and password from your own computer. If you are able to successfully transfer files, then the account itself is OK, but if not, there may be some problem with the account setup in Rumpus. In particular, check to make sure the user's Home Folder is correct and points to a valid, reliable disk drive or volume.

Does the problem occur for a specific file (or type of file)?

In terms of transferring files, there's really no difference between different file types. Video, image, text, or any other file type are just a series of bytes to Rumpus. But there are a couple of things about a file that can cause problems.

Big files: If large files are the problem, and if file transfers tend to cut off around the same point every time you try to transfer them, then a good bet is that some router or firewall is blocking transfers over a certain size. This is particularly common with virus-scanning firewalls, which buffer files in transfer but can run out of resources and will end up disconnecting large data transfers. To diagnose a problem like this, turn off your firewall or set it to a minimal protection mode and see if that allows transfers to complete.

FTP transfers: The most common form of file corruption occurs when binary file types (basically any type other than plain text) is transferred via FTP in text mode. Always make sure to use "binary" mode when using an FTP client to transfer files. Even text files transfer fine in binary mode, so it's always a reliable choice. (The transfer mode is set in the FTP client, so how users select the mode varies from client to client.)

If you have a consistently reproducible file transfer problem that isn't explained above, send an e-mail to support@maxum.com and send details about how I can connect to your server and see the problem firsthand. Reproducible problems are usually pretty easy to get to the bottom of.

Does the problem seem to be random?

When there's no rhyme or reason, but transfers just generally seem unreliable, things get a bit trickier.

After a transfer failure, have the user log back in and try again. Does the second or third attempt succeed?

By simply trying again, you are really trying to see if the problem is reproducible after all. If the same person tries to send the same file and it fails repeatedly, look above for possible causes. But if the client tries again and succeeds, then the problem is truly intermittent.

Intermittent problems are almost always network related. A few years ago, I had very intermittent (maybe once a week) problems on my own network. For a few months, I'd simply re-transfer the file and ignore the problem, but I finally started comprehensively replacing individual components of my network. Finally, I found an Ethernet cable that, when removed from my network, allowed everything to function reliably.

Something as simple as a network cable would seem to be either functional or not, but that's not the case. Cables, routers, ethernet adapters and other devices can degrade and begin to cause intermittent failures.

Unfortunately, when a network device is causing intermittent problems, the only good way to diagnose the problem is to replace components. One good way to see if a problem is network related is to load a demo version of Rumpus onto a second server, then connect the test server to your network and temporarily move service to the alternate server. See our FAQs for details on moving your user accounts and other settings to a second server, and if possible, plug the test server into the router closest to your upstream Internet connection. If this test results in more reliable service, then you can re-introduce components of the old network, up to and including the server itself until the system becomes unreliable. If the test fails, contact your ISP, as the problem is more likely to lie upstream from your network.

Does the problem tend to occur during peak usage times, or is there any other common element to the problem?

Network interruptions during peak Internet usage times tend to indicate an overworked Internet connection. Keep in mind that Rumpus file transfers aren't the only tasks using bandwidth on your Internet connection. Make sure that you have a policy in place to limit recreational Internet usage (Internet radio, YouTube, etc.) during peak times. You might also consider a packet shaping, or "Quality of Service", router, which helps manage traffic on busy, bandwidth-limited connections. (Google "quality of service router" for options.) Finally, of course, contact your ISP to find out if they offer high-performance Internet options in your area.

As I mentioned at the start of this post, once a file transfer is underway, Rumpus is doing very little itself. When a connection fails, all Rumpus can see is that the connection has terminated unexpectedly. The server simply has no way of knowing whether the problem was a firewall interrupting the transfer, a failing network device, a service failure somewhere across the Internet or a problem on the client's network. As far as Rumpus is concerned, the termination could even be the end user quiting their Web browser or FTP client or purposely disconnecting the session.

So unfortunately, when transfers seem generally unreliable, there are a lot of possible causes and few concrete clues. Start by asking the questions posed above, and if problems persist, contact me at support@maxum.com. Here are a few more good questions to answer when you contact support, that might help us narrow in on the problem.

How often, in general, does the problem occur?

Is there anything at all that seems to cause the problem to occur more or less often?

When did the problem start?

Do your clients tend to connect via Web browser, FTP client, or both equally?

Does the problem occur in both FTP and Web transfers?

Does the problem affect file uploads, downloads, or both?

© Copyright 2018, Maxum Development Corp.