How does ASCII upload and download work in vsftpd? Doesn't it work backwards?

It might appear that the ascii_download_enable and ascii_upload_enable options of vsftpd work strangely, or even backwards. However this is a misunderstanding, they behave exactly as documented. There are however a couple of things to realize in order to see that this is true.

When transferring a file in ASCII mode, the file is always converted to a standard form (NVT-ASCII) for transmission. In this standard form, new lines are represented using <CR><LF>. It does not matter what operating systems the server and the client are running. Even if both are unix, the files are converted to <CR><LF> during transmission [1, chapter 3.1.1.1].

The ASCII mode is meant to work in the following way:

1. The sending side takes the file to be sent, converts it to the standard form with <CR><LF> line terminators and sends it over the network. E.g. a unix client converts an <LF> line-terminated file to a <CR><LF> line-terminated file; a windows client sends a <CR><LF> line terminated file as it is, because it is already in the standard form.
2. The receiving side converts the file in the standard form, which it receives from the network, to the native form of the platform it’s running on. E.g. a unix server converts the <CR><LF> line-terminated file to an <LF> line-terminated file.

So the transformation between <CR><LF> and <LF> line terminators happens both on client side and server side.

The ascii_download_enable and ascii_upload_enable options of vsftpd are not supposed to always enable ASCII mode. They only say whether a client’s request to enable ASCII mode should be respected. And the slight annoyance is that if these options are set to NO and the client requests ASCII mode, vsftpd does not send an error response. It pretends that it has enabled ASCII mode on its side (in the FTP communication it sends the following response: 200 Switching to ASCII mode.). So in this case, the client thinks the server has enabled ASCII mode, even though it has not. This behaviour is clearly documented in the default vsftpd.conf configuration file:

# By default the server will pretend to allow ASCII mode but in fact ignore
# the request. Turn on the below options to have the server actually do ASCII
# mangling on files when in ASCII mode.
# Beware that on some FTP servers, ASCII support allows a denial of service
# attack (DoS) via the command "SIZE /big/file" in ASCII mode. vsftpd
# predicted this attack and has always been safe, reporting the size of the
# raw file.
# ASCII mangling is a horrible feature of the protocol.
#ascii_upload_enable=YES
#ascii_download_enable=YES

We think the reason it behaves in this way might be that the FTP specification actually requires that an FTP server supports the ASCII mode [1, chapter 3.1.1.1]. So in order to have the ability to disable the "horrible" ASCII mode and still conform (sort of) to the specification, the vsftpd developer decided to go this route.

Also note that although the ASCII mode is supposed to be the default mode according to the RFC [1], some clients, such as ftp or lftp, will by default switch to binary mode (at least in some cases - ftp will switch to binary mode if the server side system is unix).

Let’s take a look at how the ASCII mode works case by case. (We are discussing only the case when the user requests ASCII mode. If the user doesn’t request ASCII mode, binary mode might be used depending on the client, and then the file would always be sent and received exactly as it is in such a case, without any transformation on client side or server side.) The server platform is always unix with vsftpd. Also, we are only discussing the upload of a file. Download works similarly.

If we set the vsftpd options ascii_download_enable and ascii_upload_enable to YES, we will see the following behavior (the escape sequence \n denotes <LF> and \r denotes <CR>).

Now if we set the vsftpd options to NO and the user requests ASCII mode, then the client will do the newline transformation on its side, but the server will not.

A problem can arise when the sending side is asked to send a file with non-native line terminators in ASCII mode. This can in some cases result in doubled <CR> characters in the uploaded file. The FTP implementations treat non-native line terminators differently. E.g. the ftp program on Linux, when asked to send a <CR><LF> line terminated file in ASCII mode, will ignore the fact, that the file already has <CR><LF> line terminators and rewrite the <LF> occurrences to <CR><LF>, which results in <CR><CR><LF>. On the other hand, when vsftpd is asked to send a <CR><LF> line-terminated file, it will notice that the file already has the standard line terminators and will leave them be. The solution to this problem might be to always store text files in native format, or to use binary mode instead of ASCII mode.

The following examples demonstrate this behaviour. The following happens when the vsftpd options are set to YES and the ftp client program is used:

That is, the ftp client program sends <CR><CR><LF> over the network and vsftpd on the server side transforms <CR><LF> to <LF>, which results in <CR><LF>.

If we set the vsftpd options to NO, vsftpd will not do the translation, thus leaving <CR><CR><LF> as it is:

[1] RFC 959