From f9eed596a3115e583a124ccf7f929573ee5a7da4 Mon Sep 17 00:00:00 2001 From: Jay Satiro Date: Sat, 9 Jul 2016 03:05:55 -0400 Subject: FAQ: Update FTP directory listing section for MLSD command Explain how some FTP servers support the machine readable listing format MLSD from RFC 3659 and compare it to LIST. Ref: https://github.com/curl/curl/issues/906 --- docs/FAQ | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/docs/FAQ b/docs/FAQ index 8ce521754..681ce2927 100644 --- a/docs/FAQ +++ b/docs/FAQ @@ -1402,17 +1402,27 @@ FAQ CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use to list the files. - The follow-up question that tend to follow the previous one, is how a - program is supposed to parse the directory listing. How does it know what's - a file and what's a dir and what's a symlink etc. The harsh reality is that - FTP provides no such fine and easy-to-parse output. The output format FTP - servers respond to LIST commands are entirely at the server's own liking and - the NLST output doesn't reveal any types and in many cases don't even - include all the directory entries. Also, both LIST and NLST tend to hide - unix-style hidden files (those that start with a dot) by default so you need - to do "LIST -a" or similar to see them. - - The application thus needs to parse the LIST output. One such existing + The follow-up question tends to be how is a program supposed to parse the + directory listing. How does it know what's a file and what's a dir and what's + a symlink etc. If the FTP server supports the MLSD command then it will + return data in a machine-readable format that can be parsed for type. The + types are specified by RFC3659 section 7.5.1. If MLSD is not supported then + you have to work with what you're given. The LIST output format is entirely + at the server's own liking and the NLST output doesn't reveal any types and + in many cases doesn't even include all the directory entries. Also, both LIST + and NLST tend to hide unix-style hidden files (those that start with a dot) + by default so you need to do "LIST -a" or similar to see them. + + Example - List only directories. + ftp.funet.fi supports MLSD and ftp.kernel.org does not: + + curl -s ftp.funet.fi/pub/ -X MLSD | \ + perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/' + + curl -s ftp.kernel.org/pub/linux/kernel/ | \ + perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/' + + If you need to parse LIST output in libcurl one such existing list parser is available at https://cr.yp.to/ftpparse.html Versions of libcurl since 7.21.0 also provide the ability to specify a wildcard to download multiple files from one FTP directory. -- cgit v1.2.3