Folks,
I am working on the driver for the Newport XPS motor controllers. These controllers can execute Position-Velocity-Time (PVT) trajectory scans which are complex multi-axis coordinated motion along a user-defined path in 8-space. The trajectory
is defined in a text file, which the EPICS driver creates and then uploads to the controller. The older XPS-C8 and XPS-Q8 controllers support FTP for the upload. I have a small C implementation of FTP which the driver uses. This C code is standalone, it
only needs to call the standard socket functions.
The new XPS-D8 controller does not support FTP, it only supports SFTP/SCP. I need a solution to be able to do SFTP transfers from within my C++ EPICS driver. I want the solution to work on both Linux and Windows, and ideally on vxWorks
as well.
I have tried 3 solutions, and measured the time that each solution required to upload a 5KB file. The results surprised me, and I am curious if anyone can explain them.
The code for the 3 solutions is in the attached xpsSFTPUpload.cpp and the test program is testSFTPUpload.cpp.
Solution 1. Use libssh2. This is selected if CONFIG_SITE defines HAVE_LIBSSH2=YES.
Execution time = 326 ms
Solution 2. Use libcurl with the curl_easy API. This is selected if CONFIG_SITE defines HAVE_LIBCURL=YES.
Execution time = 323 ms
Solution 3. Run the “curl” program with the C system() call. The driver builds the text of the curl command and runs it with the system() call. This is selected if CONFIG_SITE does not define HAVE_LIBSSH2=YES or HAVE_LIBCURL=YES.
Execution time (Linux) = 126 ms
Execution time (Windows) = 270 ms
So on Linux running the “curl” command via a call to system() is actually almost 3 times faster than calling either of the libraries directly. This surprises me.
Note that I actually prefer to use Solution 3 because it is easy to implement on Windows. There are several sites that provide downloads of a pre-built “curl” executable. However, I have not been able to find any site that provides the
libssh2 or libcurl libraries in a form that can be called from Visual Studio. They are built with mingw and give errors when I try to link with Visual Studio. Note that because of the performance measurements above I am not motivated to pursue the library
solution either, since it seems to be significantly slower.
Ideally I would like this driver to work on vxWorks as well, but I don’t see any solution for doing sftp from within a C program on vxWorks?
Thanks,
Mark
|
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <string>
#include <xpsSFTPUpload.h>
// The following version requires libcurl.
#ifdef HAVE_LIBCURL
#include <sys/stat.h>
#include <curl/curl.h>
int xpsSFTPUpload(std::string IPAddress, std::string trajectoryDirectory, std::string fileName,
std::string userName, std::string password, bool verbose) {
int status = 0;
CURL *curl = curl_easy_init();
if (curl == 0) {
fprintf(stderr, "curl_easy_init failed\n");
return -1;
}
std::string url = "scp://" + IPAddress + "/" + trajectoryDirectory + "/" + fileName;
status |= curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
status |= curl_easy_setopt(curl, CURLOPT_SSH_AUTH_TYPES, CURLSSH_AUTH_PASSWORD);
std::string auth = userName + ":" + password;
status |= curl_easy_setopt(curl, CURLOPT_USERPWD, auth.c_str());
status |= curl_easy_setopt(curl, CURLOPT_UPLOAD, 1);
FILE *fd = fopen(fileName.c_str(), "rb"); /* open file to upload */
struct stat fileStat;
fstat(fileno(fd), &fileStat);
status |= curl_easy_setopt(curl, CURLOPT_INFILESIZE, (long)fileStat.st_size);
status |= curl_easy_setopt(curl, CURLOPT_READDATA, fd);
if (verbose) status |= curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
status |= curl_easy_perform(curl);
fclose(fd);
curl_easy_cleanup(curl);
if (status) {
fprintf(stderr, "One or more curl_easy functions failed\n");
return -1;
}
return 0;
}
#elif defined HAVE_LIBSSH2
#include <libssh2.h>
#include <libssh2_sftp.h>
#include <osiSock.h>
int xpsSFTPUpload(std::string IPAddress, std::string trajectoryDirectory, std::string fileName,
std::string userName, std::string password, bool verbose) {
struct sockaddr_in sockAddr;
SOCKET sockFD = 0;
LIBSSH2_SESSION *session = 0;
LIBSSH2_SFTP_HANDLE *sftp_handle = 0;
LIBSSH2_SFTP *sftp_session = 0;
const char *fingerprint;
int i;
char mem[1024*100];
FILE *local = 0;
size_t nread;
char *ptr;
int rc;
int status = -1;
std::string remoteFile = trajectoryDirectory + fileName;
rc = libssh2_init(0);
if (rc != 0) {
fprintf(stderr, "libssh2 initialization failed (%d)\n", rc);
return -1;
}
/*
* The application code is responsible for creating the socket
* and establishing the connection
*/
sockFD = epicsSocketCreate(AF_INET, SOCK_STREAM, 0);
sockAddr.sin_family = AF_INET;
sockAddr.sin_port = htons(22);
sockAddr.sin_addr.s_addr = inet_addr(IPAddress.c_str());
if (connect(sockFD, (struct sockaddr*)(&sockAddr),
sizeof(struct sockaddr_in)) != 0) {
fprintf(stderr, "failed to connect to %s\n", IPAddress.c_str());
goto done;
}
/* Create a session instance */
session = libssh2_session_init();
/* Since we have set non-blocking, tell libssh2 we are blocking */
libssh2_session_set_blocking(session, 1);
/* ... start it up. This will trade welcome banners, exchange keys,
* and setup crypto, compression, and MAC layers */
rc = libssh2_session_handshake(session, sockFD);
if (rc) {
fprintf(stderr, "Failure establishing SSH session: %d\n", rc);
goto done;
}
/* At this point we haven't yet authenticated. The first thing to do
* is check the hostkey's fingerprint against our known hosts Your app
* may have it hard coded, may go to a file, may present it to the
* user, that's your call
*/
fingerprint = libssh2_hostkey_hash(session, LIBSSH2_HOSTKEY_HASH_SHA1);
fprintf(stderr, "Fingerprint: ");
for (i = 0; i < 20; i++) {
fprintf(stderr, "%02X ", (unsigned char)fingerprint[i]);
}
fprintf(stderr, "\n");
/* Authenticate via password */
if (libssh2_userauth_password(session, userName.c_str(), password.c_str())) {
fprintf(stderr, "Authentication by password failed.\n");
goto done;
}
local = fopen(fileName.c_str(), "rb");
if (!local) {
fprintf(stderr, "Can't open local file %s\n", fileName.c_str());
goto done;
}
sftp_session = libssh2_sftp_init(session);
if(!sftp_session) {
fprintf(stderr, "Unable to init SFTP session\n");
goto done;
}
/* Request a file via SFTP */
sftp_handle = libssh2_sftp_open(sftp_session, remoteFile.c_str(),
LIBSSH2_FXF_WRITE|LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
LIBSSH2_SFTP_S_IRUSR|LIBSSH2_SFTP_S_IWUSR|
LIBSSH2_SFTP_S_IRGRP|LIBSSH2_SFTP_S_IROTH);
if(!sftp_handle) {
fprintf(stderr, "Unable to open remote file %s with SFTP\n", remoteFile.c_str());
goto done;
}
do {
nread = fread(mem, 1, sizeof(mem), local);
if(nread <= 0) {
/* end of file */
break;
}
ptr = mem;
do {
/* write data in a loop until we block */
rc = libssh2_sftp_write(sftp_handle, ptr, nread);
if(rc < 0)
break;
ptr += rc;
nread -= rc;
} while(nread);
} while(rc > 0);
status = 0;
done:
if (local) fclose(local);
if (sftp_handle) libssh2_sftp_close(sftp_handle);
if (sftp_session) libssh2_sftp_shutdown(sftp_session);
if (session) {
libssh2_session_disconnect(session, "Normal Shutdown");
libssh2_session_free(session);
}
if (sockFD) {
epicsSocketDestroy(sockFD);
}
return status;
}
#else
// If HAVE_LIBCURL is not YES then try to execute the curl system command
int xpsSFTPUpload(std::string IPAddress, std::string trajectoryDirectory, std::string fileName,
std::string userName, std::string password, bool verbose) {
std::string verboseOption = verbose ? " --verbose " : " --silent ";
std::string command = "curl -k -u " + userName + ":" + password + " -T " + fileName + verboseOption + " scp://" + IPAddress + trajectoryDirectory + "/";
if (verbose) printf("curl command=%s\n", command.c_str());
int status = system(command.c_str());
return status;
}
#endif
#include <stdio.h>
#include <asynDriver.h>
#include <xpsSFTPUpload.h>
int main(int argc, char *argv[])
{
bool sftpVerbose = true;
asynUser *pasynUser = pasynManager->createAsynUser(0, 0);
asynPrint(pasynUser, ASYN_TRACE_ERROR, "Calling xpsSFTPUpload\n");
int status = xpsSFTPUpload("164.54.160.71", "/Admin/Public/Trajectories", "TrajectoryScan.trj",
"Administrator", "Administrator", sftpVerbose);
asynPrint(pasynUser, ASYN_TRACE_ERROR, "xpsSFTPUpload returned status=%d\n", status);
return 0;
}
- Replies:
- RE: Uploading a file with SFTP from an EPICS driver Freddie Akeroyd - UKRI STFC via Tech-talk
- Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
- Navigate by Date:
- Prev:
EPICS Codeathon 2020 Johnson, Andrew N. via Tech-talk
- Next:
RE: Uploading a file with SFTP from an EPICS driver Freddie Akeroyd - UKRI STFC via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
<2019>
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
RE: EPICS Codeathon 2020 Rose, Austen (DLSLtd, RAL, LSCI) via Tech-talk
- Next:
RE: Uploading a file with SFTP from an EPICS driver Freddie Akeroyd - UKRI STFC via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
<2019>
2020
2021
2022
2023
2024
|