Hi Lewis,
Thanks for the suggestion. Now things get REALLY weird.
I did what you said, and ran curl with --libcurl. I timed the command:
corvette:motorNewport/newportApp/src>time curl --libcurl curl_gen.c -k -u Administrator:Administrator -T TrajectoryScan.trj --verbose scp://164.54.160.71/Admin/Public/Trajectories/
* Trying 164.54.160.71...
* TCP_NODELAY set
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 164.54.160.71 (164.54.160.71) port 22 (#0)
* SSH MD5 fingerprint: 8de67ccf8c2f9b3b584c703b9fb749f6
* SSH authentication methods available: publickey,password,keyboard-interactive
* Using SSH private key file '/home/epics/.ssh/id_rsa'
* SSH public key authentication failed: Username/PublicKey combination invalid
* Initialized password authentication
* Authentication complete
* SSH CONNECT phase done
} [5149 bytes data]
* We are completely uploaded and fine
100 5149 0 0 0 5149 0 43268 --:--:-- --:--:-- --:--:-- 43268
100 5149 0 0 0 5149 0 43268 --:--:-- --:--:-- --:--:-- 43268
* Connection #0 to host 164.54.160.71 left intact
0.013u 0.006s 0:00.12 8.3% 0+0k 0+8io 0pf+0w
Note that the elapsed time was 0.12 seconds, which is consistent with what I reported previously for running the curl command with system(). This now also includes the time to create the C program curl_gen.c.
I then added curl_gen to my EPICS Makefile and ran the generated curl_gen.c (with no edits), and timed that command:
corvette:motorNewport/newportApp/src>time ../../../../bin/linux-x86_64/curl_gen < TrajectoryScan.trj * About to connect() to 164.54.160.71 port 22 (#0)
* Trying 164.54.160.71...
* Connected to 164.54.160.71 (164.54.160.71) port 22 (#0)
* SSH MD5 fingerprint: 8de67ccf8c2f9b3b584c703b9fb749f6
* SSH authentication methods available: publickey,password,keyboard-interactive
* Using SSH private key file '/home/epics/.ssh/id_rsa'
* SSH public key authentication failed: Username/PublicKey combination invalid
* Initialized password authentication
* Authentication complete
* SSH CONNECT phase done
* We are completely uploaded and fine
* Connection #0 to host 164.54.160.71 left intact
0.014u 0.007s 0:00.32 3.1% 0+0k 0+0io 0pf+0w
Note that it takes 0.32 seconds, again with what I reported previously for using libcurl. So the C code generated by the curl command takes almost 3 times longer to execute than the curl command itself, including generating the C code!
In this case I used stdin to send the file to the curl_gen program. I also tried this code in curl_gen.c rather than using stdin:
FILE *fd = fopen("TrajectoryScan.trj", "rb");
struct stat fileStat;
fstat(fileno(fd), &fileStat);
curl_easy_setopt(hnd, CURLOPT_INFILESIZE, (long)fileStat.st_size);
curl_easy_setopt(hnd, CURLOPT_READDATA, fd);
The performance was the same, 0.33 seconds, much slower than running the curl command itself!
Mark
-----Original Message-----
From: J. Lewis Muir <[email protected]>
Sent: Thursday, November 14, 2019 10:06 AM
To: Mark Rivers <[email protected]>
Cc: [email protected]
Subject: Re: Uploading a file with SFTP from an EPICS driver
On 11/14, Mark Rivers via Tech-talk wrote:
> So on Linux running the "curl" command via a call to system() is actually almost 3 times faster than calling either of the libraries directly. This surprises me.
That makes no sense. There must be something different about how you're using libcurl vs. curl. Have you tried curl's "--libcurl <file>"
option? I would try that. Basically, take the curl command line that works, and then add the "--libcurl <file>" option to it, and it will emit C source code for a program using libcurl that does the same thing.
Compare that with how you're using libcurl.
Lewis
- Replies:
- Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
- References:
- Uploading a file with SFTP from an EPICS driver Mark Rivers via Tech-talk
- Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
- Navigate by Date:
- Prev:
Re: Uploading a file with SFTP from an EPICS driver Michael Davidsaver via Tech-talk
- Next:
Release of EPICS Qt 3.7.2 [SEC=UNOFFICIAL] STARRITT, Andrew via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
<2019>
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Uploading a file with SFTP from an EPICS driver Michael Davidsaver via Tech-talk
- Next:
Re: Uploading a file with SFTP from an EPICS driver J. Lewis Muir via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
<2019>
2020
2021
2022
2023
2024
|