EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: Uploading a file with SFTP from an EPICS driver
From: Mark Rivers via Tech-talk <[email protected]>
To: "'J. Lewis Muir'" <[email protected]>
Cc: "[email protected]" <[email protected]>
Date: Fri, 15 Nov 2019 00:06:39 +0000
Hi Lewis,

Thanks for the suggestion.  Now things get REALLY weird.

I did what you said, and ran curl with --libcurl.  I timed the command:

corvette:motorNewport/newportApp/src>time curl --libcurl curl_gen.c -k -u Administrator:Administrator -T TrajectoryScan.trj --verbose scp://164.54.160.71/Admin/Public/Trajectories/
*   Trying 164.54.160.71...
* TCP_NODELAY set
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 164.54.160.71 (164.54.160.71) port 22 (#0)
* SSH MD5 fingerprint: 8de67ccf8c2f9b3b584c703b9fb749f6
* SSH authentication methods available: publickey,password,keyboard-interactive
* Using SSH private key file '/home/epics/.ssh/id_rsa'
* SSH public key authentication failed: Username/PublicKey combination invalid
* Initialized password authentication
* Authentication complete
* SSH CONNECT phase done
} [5149 bytes data]
* We are completely uploaded and fine
100  5149    0     0    0  5149      0  43268 --:--:-- --:--:-- --:--:-- 43268
100  5149    0     0    0  5149      0  43268 --:--:-- --:--:-- --:--:-- 43268
* Connection #0 to host 164.54.160.71 left intact
0.013u 0.006s 0:00.12 8.3%      0+0k 0+8io 0pf+0w

Note that the elapsed time was 0.12 seconds, which is consistent with what I reported previously for running the curl command with system().  This now also includes the time to create the C program curl_gen.c.

I then added curl_gen to my EPICS Makefile and ran the generated curl_gen.c (with no edits), and timed that command:

corvette:motorNewport/newportApp/src>time ../../../../bin/linux-x86_64/curl_gen < TrajectoryScan.trj                                                      * About to connect() to 164.54.160.71 port 22 (#0)
*   Trying 164.54.160.71...
* Connected to 164.54.160.71 (164.54.160.71) port 22 (#0)
* SSH MD5 fingerprint: 8de67ccf8c2f9b3b584c703b9fb749f6
* SSH authentication methods available: publickey,password,keyboard-interactive
* Using SSH private key file '/home/epics/.ssh/id_rsa'
* SSH public key authentication failed: Username/PublicKey combination invalid
* Initialized password authentication
* Authentication complete
* SSH CONNECT phase done
* We are completely uploaded and fine
* Connection #0 to host 164.54.160.71 left intact
0.014u 0.007s 0:00.32 3.1%      0+0k 0+0io 0pf+0w

Note that it takes 0.32 seconds, again with what I reported previously for using libcurl.  So the C code generated by the curl command takes almost 3 times longer to execute than the curl command itself, including generating the C code!

In this case I used stdin to send the file to the curl_gen program.  I also tried this code in curl_gen.c rather than using stdin:

  FILE *fd = fopen("TrajectoryScan.trj", "rb");
  struct stat fileStat;
  fstat(fileno(fd), &fileStat);
  curl_easy_setopt(hnd, CURLOPT_INFILESIZE, (long)fileStat.st_size);
  curl_easy_setopt(hnd, CURLOPT_READDATA, fd);

The performance was the same, 0.33 seconds, much slower than running the curl command itself!

Mark

-----Original Message-----
From: J. Lewis Muir <[email protected]> 
Sent: Thursday, November 14, 2019 10:06 AM
To: Mark Rivers <[email protected]>
Cc: [email protected]
Subject: Re: Uploading a file with SFTP from an EPICS driver

On 11/14, Mark Rivers via Tech-talk wrote:
> So on Linux running the "curl" command via a call to system() is actually almost 3 times faster than calling either of the libraries directly.  This surprises me.

That makes no sense.  There must be something different about how you're using libcurl vs. curl.  Have you tried curl's "--libcurl <file>"
option?  I would try that.  Basically, take the curl command line that works, and then add the "--libcurl <file>" option to it, and it will emit C source code for a program using libcurl that does the same thing.
Compare that with how you're using libcurl.

Lewis

Replies:
Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
References:
Uploading a file with SFTP from an EPICS driver Mark Rivers via Tech-talk
Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk

Navigate by Date:
Prev: Re: Uploading a file with SFTP from an EPICS driver Michael Davidsaver via Tech-talk
Next: Release of EPICS Qt 3.7.2 [SEC=UNOFFICIAL] STARRITT, Andrew via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: Uploading a file with SFTP from an EPICS driver Michael Davidsaver via Tech-talk
Next: Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  2021  2022  2023  2024 
ANJ, 15 Nov 2019 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·