From 7e42cb61f75890832792c082510ec610f4c32cbe Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 21 Jan 2005 09:32:32 +0000 Subject: FTP third transfer support overhaul. See CHANGES for details. --- CHANGES | 44 ++++++++++++++++++++++++++ RELEASE-NOTES | 9 ++++-- docs/curl.1 | 13 +++++++- docs/examples/ftp3rdparty.c | 47 ++++++++++++++------------- docs/libcurl/curl_easy_setopt.3 | 16 ++++++++-- include/curl/curl.h | 32 ++++++++++--------- lib/ftp.c | 58 +++++++++++++++++++--------------- lib/http.c | 5 ++- lib/sendf.c | 16 +++++++--- lib/sendf.h | 3 +- lib/transfer.c | 43 ++++++++++++------------- lib/url.c | 28 +++++------------ lib/urldata.h | 14 +++------ src/main.c | 70 +++++++++++++++++++++++++++++++++++++---- tests/data/Makefile.am | 2 +- tests/data/test230 | 51 ++++++++++++++++++++++++++++++ tests/data/test231 | 51 ++++++++++++++++++++++++++++++ tests/data/test232 | 51 ++++++++++++++++++++++++++++++ 18 files changed, 414 insertions(+), 139 deletions(-) create mode 100644 tests/data/test230 create mode 100644 tests/data/test231 create mode 100644 tests/data/test232 diff --git a/CHANGES b/CHANGES index 24db8ef7b..b5b5082a3 100644 --- a/CHANGES +++ b/CHANGES @@ -7,6 +7,50 @@ Changelog +Daniel (21 January 2005) +- Major FTP third party transfer overhaul. + + These four options are now obsolete: CURLOPT_SOURCE_HOST, + CURLOPT_SOURCE_PATH, CURLOPT_SOURCE_PORT (this option didn't work before) + and CURLOPT_PASV_HOST. + + These two options are added: CURLOPT_SOURCE_URL and CURLOPT_SOURCE_QUOTE. + + The target-side didn't use the proper path with RETR, and thus this only + worked correctly in the login path (i.e without doing any CWD). The source- + side still uses a wrong path, but the fix for this will need to wait. Verify + the flaw by using a source URL with included %XX-codes. + + Made CURLOPT_FTPPORT control weather the target operation should use PORT + (or not). The other side thus uses passive (PASV) mode. + + Updated the ftp3rdparty.c example source to use the updated options. + + Added support for a second FTP server in the test suite. Named... ftp2. + Added test cases 230, 231 and 232 as a few first basic tests of very simple + 3rd party transfers. + + Changed the debug output to include 'target' and 'source' when a 3rd party + is being made, to make it clearer what commands/responses came on what + connection. + + Added three new command line options: --3p-url, --3p-user and --3p-quote. + + Documented the command line options and the curl_easy_setopt options related + to third party transfers. + + (Temporarily) disabled the ability to re-use an existing connection for the + source connection. This is because it needs to force a new in case the + source and target is the same host, and the host name check is trickier now + when the source is identified with a full URL instead of a plain host name + like before. + + TODO (short-term) for 3rd party transfers: quote support. The options are + there, we need to add test cases to verify their functionality. + + TODO (long-term) for 3rd party transfers: IPv6 support (EPRT and EPSV etc) + and SSL/TSL support. + Daniel (20 January 2005) - Philippe Hameau found out that -Q "+[command]" didn't work, although some code was written for it. I fixed and added test case 227 to verify it. diff --git a/RELEASE-NOTES b/RELEASE-NOTES index 47b789a38..19e672318 100644 --- a/RELEASE-NOTES +++ b/RELEASE-NOTES @@ -1,8 +1,8 @@ -Curl and libcurl 7.12.4 +Curl and libcurl 7.13.0 Public curl release number: 85 Releases counted from the very beginning: 112 - Available command line options: 100 + Available command line options: 103 Available curl_easy_setopt() options: 123 Number of public functions in libcurl: 46 Amount of public web site mirrors: 14 @@ -10,6 +10,10 @@ Curl and libcurl 7.12.4 This release includes the following changes: + o added CURLOPT_SOURCE_URL and CURLOPT_SOURCE_QUOTE + o obsoleted CURLOPT_SOURCE_HOST, CURLOPT_SOURCE_PATH, CURLOPT_SOURCE_PORT + and CURLOPT_PASV_HOST + o added --3p-url, --3p-user and --3p-quote o -Q "+[command]" was added o src/getpass.c license issue sorted (code was rewritten) o curl -w now supports 'http_connect' for the proxy's response to CONNECT @@ -17,6 +21,7 @@ This release includes the following changes: This release includes the following bugfixes: + o FTP third party transfers was much improved o proxy environment variables are now ignored when built HTTP-disabled o CURLOPT_PROXY can now disable HTTP proxy even when built HTTP-disabled o "curl dictionary.com" no longer assumes DICT protocol diff --git a/docs/curl.1 b/docs/curl.1 index f9faadbda..3556d1ce6 100644 --- a/docs/curl.1 +++ b/docs/curl.1 @@ -21,7 +21,7 @@ .\" * $Id$ .\" ************************************************************************** .\" -.TH curl 1 "20 Jan 2005" "Curl 7.12.4" "Curl Manual" +.TH curl 1 "20 Jan 2005" "Curl 7.13.0" "Curl Manual" .SH NAME curl \- transfer a URL .SH SYNOPSIS @@ -1065,6 +1065,17 @@ Forces curl to use SSL version 2 when negotiating with a remote SSL server. .IP "-3/--sslv3" (HTTPS) Forces curl to use SSL version 3 when negotiating with a remote SSL server. +.IP "--3p-quote" +(FTP) Specify arbitrary commands to send to the source server. See the +\fI-Q/--quote\fP option for details. (Added in 7.13.0) +.IP "--3p-url" +(FTP) Activates a FTP 3rd party transfer. Specifies the source URL to get a +file from, while the "normal" URL will be used as target URL, the file that +will be written/created. + +Note that not all FTP server allow 3rd party transfers. (Added in 7.13.0) +.IP "--3p-user" +(FTP) Specify user:password for the source URL transfer. (Added in 7.13.0) .IP "-4/--ipv4" If libcurl is capable of resolving an address to multiple IP versions (which it is if it is ipv6-capable), this option tells libcurl to resolve names to diff --git a/docs/examples/ftp3rdparty.c b/docs/examples/ftp3rdparty.c index 31391ba96..b4756f580 100644 --- a/docs/examples/ftp3rdparty.c +++ b/docs/examples/ftp3rdparty.c @@ -1,8 +1,8 @@ /***************************************************************************** - * _ _ ____ _ - * Project ___| | | | _ \| | - * / __| | | | |_) | | - * | (__| |_| | _ <| |___ + * _ _ ____ _ + * Project ___| | | | _ \| | + * / __| | | | |_) | | + * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * * $Id$ @@ -16,22 +16,20 @@ /* * This is an example showing how to transfer a file between two remote hosts. + * 7.13.0 or later required. */ - - int main(void) { CURL *curl; CURLcode res; - char sourceFileName[] = "/tmp/file"; - char targetFileName[] = "/tmp/curlTargetTest.dat"; - char sourceHost[] = "source"; - char targetHost[] = "target"; + char source_url[] = "ftp://remotehost.com/path/to/source"; + char target_url[] = "ftp://aotherserver.com/path/to/dest"; + char sourceUserPass[] = "user:pass"; char targetUserPass[] = "user:pass"; char url[100]; - + struct curl_slist *source_pre_cmd = NULL; struct curl_slist *target_pre_cmd = NULL; struct curl_slist *source_post_cmd = NULL; @@ -39,24 +37,25 @@ int main(void) char cmd[] = "PWD"; /* just to test */ curl_global_init(CURL_GLOBAL_DEFAULT); - + curl = curl_easy_init(); if (curl) { - sprintf(url, "ftp://%s@%s/%s", targetUserPass, targetHost, targetFileName); - printf("%s\n", url); - curl_easy_setopt(curl, CURLOPT_URL, url); + /* The ordinary URL is the target when speaking 3rd party transfers */ + curl_easy_setopt(curl, CURLOPT_URL, target_url); - /* Set a proxy host */ - curl_easy_setopt(curl, CURLOPT_SOURCE_HOST, sourceHost); + /* Set a source URL */ + curl_easy_setopt(curl, CURLOPT_SOURCE_URL, source_url); - /* Set a proxy user and password */ - curl_easy_setopt(curl, CURLOPT_SOURCE_USERPWD, sourceUserPass); + /* Set target user and password */ + curl_easy_setopt(curl, CURLOPT_USERPWD, targetUserPass); - /* Set a proxy full file name */ - curl_easy_setopt(curl, CURLOPT_SOURCE_PATH, sourceFileName); + /* Set source user and password */ + curl_easy_setopt(curl, CURLOPT_SOURCE_USERPWD, sourceUserPass); - /* Set a proxy passive host */ - curl_easy_setopt(curl, CURLOPT_PASV_HOST, 0); /* optional */ +#if 0 + /* FTPPORT enables PORT on the target side, instead of PASV. */ + curl_easy_setopt(curl, CURLOPT_FTPPORT, ""); /* optional */ +#endif /* build a list of commands to pass to libcurl */ source_pre_cmd = curl_slist_append(source_pre_cmd, cmd); @@ -77,7 +76,7 @@ int main(void) target_post_cmd = curl_slist_append(target_post_cmd, cmd); /* Set a post-quote command */ curl_easy_setopt(curl, CURLOPT_POSTQUOTE, target_post_cmd); - + /* Switch on full protocol/debug output */ curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); diff --git a/docs/libcurl/curl_easy_setopt.3 b/docs/libcurl/curl_easy_setopt.3 index 7beecfd1e..776e3da90 100644 --- a/docs/libcurl/curl_easy_setopt.3 +++ b/docs/libcurl/curl_easy_setopt.3 @@ -5,7 +5,7 @@ .\" * | (__| |_| | _ <| |___ .\" * \___|\___/|_| \_\_____| .\" * -.\" * Copyright (C) 1998 - 2004, Daniel Stenberg, , et al. +.\" * Copyright (C) 1998 - 2005, Daniel Stenberg, , et al. .\" * .\" * This software is licensed as described in the file COPYING, which .\" * you should have received as part of this distribution. The terms @@ -21,7 +21,7 @@ .\" * $Id$ .\" ************************************************************************** .\" -.TH curl_easy_setopt 3 "29 Nov 2004" "libcurl 7.12.3" "libcurl Manual" +.TH curl_easy_setopt 3 "20 Jan 2005" "libcurl 7.12.4" "libcurl Manual" .SH NAME curl_easy_setopt - set options for a curl easy handle .SH SYNOPSIS @@ -751,6 +751,18 @@ Try "AUTH SSL" first, and only if that fails try "AUTH TLS" .IP CURLFTPAUTH_TLS Try "AUTH TLS" first, and only if that fails try "AUTH SSL" .RE +.IP CURLOPT_SOURCE_URL +When set, it enables a FTP third party transfer, using the set URL as source, +while \fICURLOPT_URL\fP is the target. +.IP CURLOPT_SOURCE_USERPWD +Set "username:password" to use for the source connection when doing FTP third +party transfers. +.IP CURLOPT_SOURCE_QUOTE +Exactly like \fICURLOPT_QUOTE\fP, but for the source host. +.IP CURLOPT_SOURCE_PREQUOTE +Exactly like \fICURLOPT_PREQUOTE\fP, but for the source host. +.IP CURLOPT_SOURCE_POSTQUOTE +Exactly like \fICURLOPT_POSTQUOTE\fP, but for the source host. .SH PROTOCOL OPTIONS .IP CURLOPT_TRANSFERTEXT A non-zero parameter tells the library to use ASCII mode for ftp transfers, diff --git a/include/curl/curl.h b/include/curl/curl.h index 2acf76f02..07c028b3b 100644 --- a/include/curl/curl.h +++ b/include/curl/curl.h @@ -712,7 +712,7 @@ typedef enum { CINIT(SSLENGINE_DEFAULT, LONG, 90), /* Non-zero value means to use the global dns cache */ - CINIT(DNS_USE_GLOBAL_CACHE, LONG, 91), /* To become OBSOLETE soon */ + CINIT(DNS_USE_GLOBAL_CACHE, LONG, 91), /* To becomeO BSOLETE soon */ /* DNS cache timeout */ CINIT(DNS_CACHE_TIMEOUT, LONG, 92), @@ -842,25 +842,15 @@ typedef enum { /* Enable/disable the TCP Nagle algorithm */ CINIT(TCP_NODELAY, LONG, 121), - /* When doing 3rd party transfer, set the source host name with this */ - CINIT(SOURCE_HOST, OBJECTPOINT, 122), + /* 122 OBSOLETE, used in 7.12.3. Gone in 7.13.0 */ /* When doing 3rd party transfer, set the source user and password with this */ CINIT(SOURCE_USERPWD, OBJECTPOINT, 123), - /* When doing 3rd party transfer, set the source file path with this */ - CINIT(SOURCE_PATH, OBJECTPOINT, 124), - - /* When doing 3rd party transfer, set the source server's port number - with this */ - CINIT(SOURCE_PORT, LONG, 125), - - /* When doing 3rd party transfer, decide which server that should get the - PASV command (and the other gets the PORT). - 0 (default) - The target host issues PASV. - 1 - The source host issues PASV */ - CINIT(PASV_HOST, LONG, 126), + /* 124 OBSOLETE, used in 7.12.3. Gone in 7.13.0 */ + /* 125 OBSOLETE, used in 7.12.3. Gone in 7.13.0 */ + /* 126 OBSOLETE, used in 7.12.3. Gone in 7.13.0 */ /* When doing 3rd party transfer, set the source pre-quote linked list of commands with this */ @@ -885,6 +875,13 @@ typedef enum { CINIT(IOCTLFUNCTION, FUNCTIONPOINT, 130), CINIT(IOCTLDATA, OBJECTPOINT, 131), + /* To make a 3rd party transfer, set the source URL with this */ + CINIT(SOURCE_URL, OBJECTPOINT, 132), + + /* When doing 3rd party transfer, set the source quote linked list of + commands with this */ + CINIT(SOURCE_QUOTE, OBJECTPOINT, 133), + CURLOPT_LASTENTRY /* the last unused */ } CURLoption; @@ -910,6 +907,11 @@ typedef enum { #define CURLOPT_PASSWDDATA -4 #define CURLOPT_CLOSEFUNCTION -5 +#define CURLOPT_SOURCE_HOST -6 +#define CURLOPT_SOURCE_PATH -7 +#define CURLOPT_SOURCE_PORT -8 +#define CURLOPT_PASV_HOST -9 + #else /* This is set if CURL_NO_OLDIES is defined at compile-time */ #undef CURLOPT_DNS_USE_GLOBAL_CACHE /* soon obsolete */ diff --git a/lib/ftp.c b/lib/ftp.c index ffec9c647..3f4b23fe7 100644 --- a/lib/ftp.c +++ b/lib/ftp.c @@ -368,7 +368,7 @@ CURLcode Curl_GetFTPResponse(ssize_t *nreadp, /* return number of bytes read */ /* output debug output if that is requested */ if(data->set.verbose) - Curl_debug(data, CURLINFO_HEADER_IN, line_start, perline, conn->host.dispname); + Curl_debug(data, CURLINFO_HEADER_IN, line_start, perline, conn); /* * We pass all response-lines to the callback function registered @@ -593,7 +593,7 @@ CURLcode Curl_ftp_connect(struct connectdata *conn) if(ftpcode == 530) { /* 530 User ... access denied (the server denies to log the specified user) */ - failf(data, "Access denied: %s", &buf[4]); + failf(data, "Access denied: %03d", ftpcode); return CURLE_FTP_ACCESS_DENIED; } else if(ftpcode == 331) { @@ -610,7 +610,7 @@ CURLcode Curl_ftp_connect(struct connectdata *conn) or 530 Sorry, the maximum number of allowed users are already connected */ - failf(data, "not logged in: %s", &buf[4]); + failf(data, "not logged in: %03d", ftpcode); return CURLE_FTP_USER_PASSWORD_INCORRECT; } else if(ftpcode/100 == 2) { @@ -1594,7 +1594,7 @@ CURLcode ftp_use_pasv(struct connectdata *conn, } if(!*str) { - failf(data, "Couldn't interpret this 227-reply: %s", buf); + failf(data, "Couldn't interpret the 227-reply"); return CURLE_FTP_WEIRD_227_FORMAT; } @@ -1828,7 +1828,7 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn) return result; if(ftpcode>=400) { - failf(data, "Failed FTP upload:%s", buf+3); + failf(data, "Failed FTP upload: %03d", ftpcode); /* oops, we never close the sockets! */ return CURLE_FTP_COULDNT_STOR_FILE; } @@ -2022,7 +2022,7 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn) return result; if(ftpcode != 350) { - failf(data, "Couldn't use REST: %s", buf+4); + failf(data, "Couldn't use REST: %03d", ftpcode); return CURLE_FTP_COULDNT_USE_REST; } } @@ -2136,7 +2136,7 @@ CURLcode Curl_ftp_nextconnect(struct connectdata *conn) ftp->no_transfer = TRUE; /* don't think we should download anything */ } else { - failf(data, "%s", buf+4); + failf(data, "RETR failed: %03d", ftpcode); return CURLE_FTP_COULDNT_RETR_FILE; } } @@ -2367,7 +2367,7 @@ CURLcode Curl_ftpsendf(struct connectdata *conn, break; if(conn->data->set.verbose) - Curl_debug(conn->data, CURLINFO_HEADER_OUT, sptr, bytes_written, conn->host.dispname); + Curl_debug(conn->data, CURLINFO_HEADER_OUT, sptr, bytes_written, conn); if(bytes_written != (ssize_t)write_len) { write_len -= bytes_written; @@ -2553,6 +2553,9 @@ static CURLcode ftp_3rdparty_pretransfer(struct connectdata *conn) struct SessionHandle *data = conn->data; struct connectdata *sec_conn = conn->sec_conn; + conn->xfertype = TARGET3RD; + sec_conn->xfertype = SOURCE3RD; + /* sets transfer type */ result = ftp_transfertype(conn, data->set.ftp_ascii); if (result) @@ -2596,7 +2599,7 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) struct connectdata *pasv_conn; struct connectdata *port_conn; - if (data->set.pasvHost == CURL_TARGET_PASV) { + if (data->set.ftpport == NULL) { pasv_conn = conn; port_conn = sec_conn; } @@ -2612,9 +2615,11 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) /* sets the passive mode */ FTPSENDF(pasv_conn, "%s", "PASV"); result = Curl_GetFTPResponse(&nread, pasv_conn, &ftpcode); - if (result) return result; + if (result) + return result; + if (ftpcode != 227) { - failf(data, "Odd return code after PASV:%s", buf + 3); + failf(data, "Odd return code after PASV: %03d", ftpcode); return CURLE_FTP_WEIRD_PASV_REPLY; } @@ -2626,7 +2631,7 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) } if (!*str) { - failf(pasv_conn->data, "Couldn't interpret this 227-reply: %s", buf); + failf(pasv_conn->data, "Couldn't interpret the 227-reply"); return CURLE_FTP_WEIRD_227_FORMAT; } @@ -2640,7 +2645,7 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) return result; if (ftpcode != 200) { - failf(data, "PORT command attempts failed:%s", buf + 3); + failf(data, "PORT command attempts failed: %03d", ftpcode); return CURLE_FTP_PORT_FAILED; } @@ -2648,41 +2653,44 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) stor_cmd = data->set.ftp_append?"APPE":"STOR"; /* transfers file between remote hosts */ - FTPSENDF(sec_conn, "RETR %s", data->set.source_path); + /* FIX: this should send a series of CWD commands and then RETR only the + ftp->file file. The conn->path "full path" is not unescaped. Test case + 230 tests this. */ + FTPSENDF(sec_conn, "RETR %s", sec_conn->path); - if(data->set.pasvHost == CURL_TARGET_PASV) { + if(!data->set.ftpport) { result = Curl_GetFTPResponse(&nread, sec_conn, &ftpcode); if (result) return result; - if (ftpcode != 150) { - failf(data, "Failed RETR: %s", buf + 4); + if((ftpcode != 150) && (ftpcode != 125)) { + failf(data, "Failed RETR: %03d", ftpcode); return CURLE_FTP_COULDNT_RETR_FILE; } - result = Curl_ftpsendf(conn, "%s %s", stor_cmd, conn->path); + result = Curl_ftpsendf(conn, "%s %s", stor_cmd, conn->proto.ftp->file); if(CURLE_OK == result) result = Curl_GetFTPResponse(&nread, conn, &ftpcode); if (result) return result; - if (ftpcode != 150) { - failf(data, "Failed FTP upload: %s", buf + 4); + if (ftpcode >= 400) { + failf(data, "Failed FTP upload: %03d", ftpcode); return CURLE_FTP_COULDNT_STOR_FILE; } } else { - result = Curl_ftpsendf(conn, "%s %s", stor_cmd, conn->path); + result = Curl_ftpsendf(conn, "%s %s", stor_cmd, conn->proto.ftp->file); if(CURLE_OK == result) result = Curl_GetFTPResponse(&nread, sec_conn, &ftpcode); if (result) return result; - if (ftpcode != 150) { - failf(data, "Failed FTP upload: %s", buf + 4); + if (ftpcode >= 400) { + failf(data, "Failed FTP upload: %03d", ftpcode); return CURLE_FTP_COULDNT_STOR_FILE; } @@ -2690,8 +2698,8 @@ static CURLcode ftp_3rdparty_transfer(struct connectdata *conn) if (result) return result; - if (ftpcode != 150) { - failf(data, "Failed FTP upload: %s", buf + 4); + if((ftpcode != 150) && (ftpcode != 125)) { + failf(data, "Failed FTP upload: %03d", ftpcode); return CURLE_FTP_COULDNT_STOR_FILE; } } diff --git a/lib/http.c b/lib/http.c index 8e4ee7d82..2618c325f 100644 --- a/lib/http.c +++ b/lib/http.c @@ -858,8 +858,7 @@ CURLcode add_buffer_send(send_buffer *in, if(conn->data->set.verbose) /* this data _may_ contain binary stuff */ - Curl_debug(conn->data, CURLINFO_HEADER_OUT, ptr, amount, - conn->host.dispname); + Curl_debug(conn->data, CURLINFO_HEADER_OUT, ptr, amount, conn); *bytes_written += amount; @@ -1140,7 +1139,7 @@ CURLcode Curl_ConnectHTTPProxyTunnel(struct connectdata *conn, /* output debug if that is requested */ if(data->set.verbose) Curl_debug(data, CURLINFO_HEADER_IN, line_start, perline, - conn->host.dispname); + conn); /* send the header to the callback */ writetype = CLIENTWRITE_HEADER; diff --git a/lib/sendf.c b/lib/sendf.c index 4094c3239..95a88a27c 100644 --- a/lib/sendf.c +++ b/lib/sendf.c @@ -204,8 +204,7 @@ CURLcode Curl_sendf(curl_socket_t sockfd, struct connectdata *conn, break; if(data->set.verbose) - Curl_debug(data, CURLINFO_DATA_OUT, sptr, bytes_written, - conn->host.dispname); + Curl_debug(data, CURLINFO_DATA_OUT, sptr, bytes_written, conn); if((size_t)bytes_written != write_len) { /* if not all was written at once, we must advance the pointer, decrease @@ -468,18 +467,22 @@ static int showit(struct SessionHandle *data, curl_infotype type, } int Curl_debug(struct SessionHandle *data, curl_infotype type, - char *ptr, size_t size, char *host) + char *ptr, size_t size, + struct connectdata *conn) { int rc; - if(data->set.printhost && host) { + if(data->set.printhost && conn && conn->host.dispname) { char buffer[160]; const char *t=NULL; + const char *w="Data"; switch (type) { case CURLINFO_HEADER_IN: + w = "Header"; case CURLINFO_DATA_IN: t = "from"; break; case CURLINFO_HEADER_OUT: + w = "Header"; case CURLINFO_DATA_OUT: t = "to"; break; @@ -488,7 +491,10 @@ int Curl_debug(struct SessionHandle *data, curl_infotype type, } if(t) { - snprintf(buffer, sizeof(buffer), "[Data %s %s]", t, host); + snprintf(buffer, sizeof(buffer), "[%s %s %s%s]", w, t, + conn->xfertype==NORMAL?"": + (conn->xfertype==SOURCE3RD?"source ":"target "), + conn->host.dispname); rc = showit(data, CURLINFO_TEXT, buffer, strlen(buffer)); if(rc) return rc; diff --git a/lib/sendf.h b/lib/sendf.h index 0cd342bd7..77ff52e3a 100644 --- a/lib/sendf.h +++ b/lib/sendf.h @@ -62,7 +62,8 @@ CURLcode Curl_write(struct connectdata *conn, /* the function used to output verbose information */ int Curl_debug(struct SessionHandle *handle, curl_infotype type, - char *data, size_t size, char *host); + char *data, size_t size, + struct connectdata *conn); #endif diff --git a/lib/transfer.c b/lib/transfer.c index 344413f0d..71338977d 100644 --- a/lib/transfer.c +++ b/lib/transfer.c @@ -935,7 +935,7 @@ CURLcode Curl_readwrite(struct connectdata *conn, if(data->set.verbose) Curl_debug(data, CURLINFO_HEADER_IN, - k->p, k->hbuflen, conn->host.dispname); + k->p, k->hbuflen, conn); result = Curl_client_write(data, writetype, k->p, k->hbuflen); if(result) @@ -1032,14 +1032,12 @@ CURLcode Curl_readwrite(struct connectdata *conn, if(data->set.verbose) { if(k->badheader) { Curl_debug(data, CURLINFO_DATA_IN, data->state.headerbuff, - k->hbuflen, conn->host.dispname); + k->hbuflen, conn); if(k->badheader == HEADER_PARTHEADER) - Curl_debug(data, CURLINFO_DATA_IN, k->str, nread, - conn->host.dispname); + Curl_debug(data, CURLINFO_DATA_IN, k->str, nread, conn); } else - Curl_debug(data, CURLINFO_DATA_IN, k->str, nread, - conn->host.dispname); + Curl_debug(data, CURLINFO_DATA_IN, k->str, nread, conn); } #ifndef CURL_DISABLE_HTTP @@ -1270,7 +1268,7 @@ CURLcode Curl_readwrite(struct connectdata *conn, if(data->set.verbose) /* show the data before we change the pointer upload_fromhere */ Curl_debug(data, CURLINFO_DATA_OUT, conn->upload_fromhere, - bytes_written, conn->host.dispname); + bytes_written, conn); if(conn->upload_present != bytes_written) { /* we only wrote a part of the buffer (if anything), deal with it! */ @@ -2094,7 +2092,7 @@ CURLcode Curl_perform(struct SessionHandle *data) res = Curl_connect_host(data, &conn); /* primary connection */ if(res == CURLE_OK) { - if (data->set.source_host) /* 3rd party transfer */ + if (data->set.source_url) /* 3rd party transfer */ res = Curl_pretransfersec(conn); else conn->sec_conn = NULL; @@ -2105,7 +2103,7 @@ CURLcode Curl_perform(struct SessionHandle *data) res = Curl_do(&conn); /* for non 3rd party transfer only */ - if(res == CURLE_OK && !data->set.source_host) { + if(res == CURLE_OK && !data->set.source_url) { res = Transfer(conn); /* now fetch that URL please */ if(res == CURLE_OK) { @@ -2236,28 +2234,26 @@ CURLcode Curl_pretransfersec(struct connectdata *conn) CURLcode status = CURLE_OK; struct SessionHandle *data = conn->data; struct connectdata *sec_conn = NULL; /* secondary connection */ - bool reuse_fresh_tmp = data->set.reuse_fresh; - - /* update data with source host options */ - char *url = aprintf( "%s://%s/", conn->protostr, data->set.source_host); - - if(!url) - return CURLE_OUT_OF_MEMORY; + bool backup_reuse_fresh = data->set.reuse_fresh; + char *backup_userpwd = data->set.userpwd; if(data->change.url_alloc) free(data->change.url); - data->change.url_alloc = TRUE; - data->change.url = url; - data->set.ftpport = data->set.source_port; - data->set.userpwd = data->set.source_userpwd; + data->change.url_alloc = FALSE; + data->change.url = data->set.source_url; + + /* We must never actually alter 'data->set' properties, so we restore the + backed up values afterwards! */ +#if 0 /* if both remote hosts are the same host - create new connection */ if (strequal(conn->host.dispname, data->set.source_host)) - /* NOTE: this is restored back to the original value after the connect is - done */ +#endif data->set.reuse_fresh = TRUE; + data->set.userpwd = data->set.source_userpwd; + /* secondary connection */ status = Curl_connect_host(data, &sec_conn); if(CURLE_OK == status) { @@ -2267,7 +2263,8 @@ CURLcode Curl_pretransfersec(struct connectdata *conn) conn->sec_conn = sec_conn; } - data->set.reuse_fresh = reuse_fresh_tmp; + data->set.reuse_fresh = backup_reuse_fresh; + data->set.userpwd = backup_userpwd; return status; } diff --git a/lib/url.c b/lib/url.c index df2011ec6..8b433f606 100644 --- a/lib/url.c +++ b/lib/url.c @@ -1360,19 +1360,12 @@ CURLcode Curl_setopt(struct SessionHandle *data, CURLoption option, ...) break; /*********** 3rd party transfer options ***********/ - case CURLOPT_SOURCE_HOST: + case CURLOPT_SOURCE_URL: /* - * Use SOURCE HOST + * SOURCE URL */ - data->set.source_host = va_arg(param, char *); - data->set.printhost = (data->set.source_host != NULL); - break; - - case CURLOPT_SOURCE_PORT: - /* - * Use SOURCE PORT - */ - data->set.source_port = va_arg(param, char *); + data->set.source_url = va_arg(param, char *); + data->set.printhost = (data->set.source_url != NULL); break; case CURLOPT_SOURCE_USERPWD: @@ -1382,18 +1375,11 @@ CURLcode Curl_setopt(struct SessionHandle *data, CURLoption option, ...) data->set.source_userpwd = va_arg(param, char *); break; - case CURLOPT_SOURCE_PATH: - /* - * Use SOURCE PATH - */ - data->set.source_path = va_arg(param, char *); - break; - - case CURLOPT_PASV_HOST: + case CURLOPT_SOURCE_QUOTE: /* - * Indicates whether source or target host is passive + * List of RAW FTP commands to use after a connect */ - data->set.pasvHost = va_arg(param, long)?CURL_SOURCE_PASV:CURL_TARGET_PASV; + data->set.source_quote = va_arg(param, struct curl_slist *); break; case CURLOPT_SOURCE_PREQUOTE: diff --git a/lib/urldata.h b/lib/urldata.h index 311149d01..0cabdf23c 100644 --- a/lib/urldata.h +++ b/lib/urldata.h @@ -188,12 +188,6 @@ typedef enum { NTLMSTATE_LAST } curlntlm; -/* for 3rd party transfers to decide which side that issues PASV */ -typedef enum { - CURL_TARGET_PASV, - CURL_SOURCE_PASV -} curl_pasv_side; - /* Struct used for NTLM challenge-response authentication */ struct ntlmdata { curlntlm state; @@ -614,6 +608,8 @@ struct connectdata { #endif struct connectdata *sec_conn; /* secondary connection for 3rd party transfer */ + + enum { NORMAL, SOURCE3RD, TARGET3RD } xfertype; }; /* The end of connectdata. */ @@ -869,6 +865,7 @@ struct UserDefined { struct curl_slist *quote; /* after connection is established */ struct curl_slist *postquote; /* after the transfer */ struct curl_slist *prequote; /* before the transfer, after type */ + struct curl_slist *source_quote; /* 3rd party quote */ struct curl_slist *source_prequote; /* in 3rd party transfer mode - before the transfer on source host */ struct curl_slist *source_postquote; /* in 3rd party transfer mode - after @@ -901,11 +898,8 @@ struct UserDefined { curl_off_t max_filesize; /* Maximum file size to download */ - char *source_host; /* for 3rd party transfer */ - char *source_port; /* for 3rd party transfer */ + char *source_url; /* for 3rd party transfer */ char *source_userpwd; /* for 3rd party transfer */ - char *source_path; /* for 3rd party transfer */ - curl_pasv_side pasvHost; /* for 3rd party transfer indicates passive host */ /* Here follows boolean settings that define how to behave during this session. They are STATIC, set by libcurl users or at least initially diff --git a/src/main.c b/src/main.c index bcf2cd2b4..708b8124d 100644 --- a/src/main.c +++ b/src/main.c @@ -400,13 +400,12 @@ static void help(void) " --socks Use SOCKS5 proxy on given host + port", " --stderr Where to redirect stderr. - means stdout", " -t/--telnet-option Set telnet option", - " --trace Dump a network/debug trace to the given file", + " --trace Write a debug trace to the given file", " --trace-ascii Like --trace but without the hex output", - " -T/--upload-file Transfer/upload to remote site", - " --url Another way to specify URL to work with", - " -u/--user Specify user and password to use", - " Overrides -n and --netrc-optional", - " -U/--proxy-user Specify Proxy authentication", + " -T/--upload-file Transfer to remote site", + " --url Spet URL to work with", + " -u/--user Set server user and password", + " -U/--proxy-user Set proxy user and password", " -v/--verbose Make the operation more talkative", " -V/--version Show version number and quit", #ifdef __DJGPP__ @@ -422,6 +421,9 @@ static void help(void) " -1/--tlsv1 Use TLSv1 (SSL)", " -2/--sslv2 Use SSLv2 (SSL)", " -3/--sslv3 Use SSLv3 (SSL)", + " --3p-quote like -Q for the source URL for 3rd party transfer (F)", + " --3p-url source URL to activate 3rd party transfer (F)", + " --3p-user user and password for source 3rd party transfer (F)", " -4/--ipv4 Resolve name to IPv4 address", " -6/--ipv6 Resolve name to IPv6 address", " -#/--progress-bar Display transfer progress as a progress bar", @@ -540,6 +542,13 @@ struct Configurable { long req_retry; /* number of retries */ long retry_delay; /* delay between retries (in seconds) */ long retry_maxtime; /* maximum time to keep retrying */ + + char *tp_url; /* third party URL */ + char *tp_user; /* third party userpwd */ + struct curl_slist *tp_quote; + struct curl_slist *tp_postquote; + struct curl_slist *tp_prequote; + }; /* global variable to hold info about libcurl */ @@ -1240,6 +1249,11 @@ static ParameterError getparameter(char *flag, /* f or -long-flag */ {"$g", "retry", TRUE}, {"$h", "retry-delay", TRUE}, {"$i", "retry-max-time", TRUE}, + + {"$j", "3p-url", TRUE}, + {"$k", "3p-user", TRUE}, + {"$l", "3p-quote", TRUE}, + {"0", "http1.0", FALSE}, {"1", "tlsv1", FALSE}, {"2", "sslv2", FALSE}, @@ -1586,6 +1600,36 @@ static ParameterError getparameter(char *flag, /* f or -long-flag */ case 'i': /* --retry-max-time */ if(str2num(&config->retry_maxtime, nextarg)) return PARAM_BAD_NUMERIC; + break; + + case 'j': /* --3p-url */ + GetStr(&config->tp_url, nextarg); + break; + case 'k': /* --3p-user */ + GetStr(&config->tp_user, nextarg); + break; + case 'l': /* --3p-quote */ + /* QUOTE commands to send to source FTP server */ + err = PARAM_OK; + switch(nextarg[0]) { + case '-': + /* prefixed with a dash makes it a POST TRANSFER one */ + nextarg++; + err = add2list(&config->tp_postquote, nextarg); + break; + case '+': + /* prefixed with a plus makes it a just-before-transfer one */ + nextarg++; + err = add2list(&config->tp_prequote, nextarg); + break; + default: + err = add2list(&config->tp_quote, nextarg); + break; + } + if(err) + return err; + + break; /* break */ } break; @@ -2782,10 +2826,17 @@ static void free_config_fields(struct Configurable *config) free(config->capath); if(config->cookiejar) free(config->cookiejar); + if(config->tp_url) + free(config->tp_url); + if(config->tp_user) + free(config->tp_user); curl_slist_free_all(config->quote); /* checks for config->quote == NULL */ curl_slist_free_all(config->prequote); curl_slist_free_all(config->postquote); + curl_slist_free_all(config->tp_quote); + curl_slist_free_all(config->tp_prequote); + curl_slist_free_all(config->tp_postquote); curl_slist_free_all(config->headers); } @@ -3604,6 +3655,13 @@ operate(struct Configurable *config, int argc, char *argv[]) curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); } + /* curl 7.12.4 */ + curl_easy_setopt(curl, CURLOPT_SOURCE_URL, config->tp_url); + curl_easy_setopt(curl, CURLOPT_SOURCE_USERPWD, config->tp_user); + curl_easy_setopt(curl, CURLOPT_SOURCE_PREQUOTE, config->tp_prequote); + curl_easy_setopt(curl, CURLOPT_SOURCE_POSTQUOTE, config->tp_postquote); + curl_easy_setopt(curl, CURLOPT_SOURCE_QUOTE, config->tp_quote); + retry_numretries = config->req_retry; retrystart = curlx_tvnow(); diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index 4f8b3d670..7fe4a1818 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -31,7 +31,7 @@ EXTRA_DIST = test1 test108 test117 test127 test20 test27 test34 test46 \ test517 test518 test210 test211 test212 test220 test221 test222 \ test223 test224 test206 test207 test208 test209 test213 test240 \ test241 test242 test519 test214 test215 test216 test217 test218 \ - test199 test225 test226 test227 + test199 test225 test226 test227 test230 test231 test232 # The following tests have been removed from the dist since they no longer # work. We need to fix the test suite's FTPS server first, then bring them diff --git a/tests/data/test230 b/tests/data/test230 new file mode 100644 index 000000000..8ae9edfa4 --- /dev/null +++ b/tests/data/test230 @@ -0,0 +1,51 @@ +# +# Server-side + + +contents of source file 230 + + + +# +# Client-side + + +ftp +ftp2 + + +FTP 3rd party transfer + + +ftp://%HOSTIP:%FTPPORT/dest/destpath/230 -u daniel:stenberg --3p-url ftp://%HOSTIP:%FTP2PORT/source/sourcepath/230 --3p-user daniel2:stenberg2 + + + +# +# Verify data after the test has been "shot" + +# target-side protocol check + +USER daniel +PASS stenberg +PWD +TYPE I +CWD dest +CWD destpath +PASV +STOR 230 +QUIT + + +s/^(PORT 127,0,0,1).*/$1/ + + +USER daniel2 +PASS stenberg2 +PWD +TYPE I +PORT 127,0,0,1 +RETR source/sourcepath/230 +QUIT + + diff --git a/tests/data/test231 b/tests/data/test231 new file mode 100644 index 000000000..10ab24c48 --- /dev/null +++ b/tests/data/test231 @@ -0,0 +1,51 @@ +# +# Server-side + + +contents of source file 231 + + + +# +# Client-side + + +ftp +ftp2 + + +FTP 3rd party transfer, make target use PORT + + +ftp://%HOSTIP:%FTPPORT/dest/destpath/231 -u daniel:stenberg --3p-url ftp://%HOSTIP:%FTP2PORT/source/sourcepath/231 --3p-user daniel2:stenberg2 -P - + + + +# +# Verify data after the test has been "shot" + +# target-side protocol check +> +s/^(PORT 127,0,0,1).*/$1/ + + +USER daniel +PASS stenberg +PWD +TYPE I +CWD dest +CWD destpath +PORT 127,0,0,1 +STOR 231 +QUIT + + +USER daniel2 +PASS stenberg2 +PWD +TYPE I +PASV +RETR source/sourcepath/231 +QUIT + + diff --git a/tests/data/test232 b/tests/data/test232 new file mode 100644 index 000000000..2c584efd3 --- /dev/null +++ b/tests/data/test232 @@ -0,0 +1,51 @@ +# +# Server-side + + +contents of source file 232 + + + +# +# Client-side + + +ftp +ftp2 + + +FTP 3rd party transfer, anonymous user + + +ftp://%HOSTIP:%FTPPORT/dest/destpath/232 --3p-url ftp://%HOSTIP:%FTP2PORT/source/sourcepath/232 + + + +# +# Verify data after the test has been "shot" + +# target-side protocol check + +USER anonymous +PASS curl_by_daniel@haxx.se +PWD +TYPE I +CWD dest +CWD destpath +PASV +STOR 232 +QUIT + + +s/^(PORT 127,0,0,1).*/$1/ + + +USER anonymous +PASS curl_by_daniel@haxx.se +PWD +TYPE I +PORT 127,0,0,1 +RETR source/sourcepath/232 +QUIT + + -- cgit v1.2.3