Thursday, September 6, 2018

How do you use the command line program cURL?

cURL is an opensource URL client. You can look up the manual for cURL with the following:
------------
C:\>curl --help
Usage: curl [options...]
     --abstract-unix-socket Connect via abstract Unix domain socket
     --anyauth       Pick any authentication method
 -a, --append        Append to target file when uploading
     --basic         Use HTTP Basic Authentication
     --cacert CA certificate to verify peer against
     --capath   CA directory to verify peer against
 -E, --cert Client certificate file and password
     --cert-status   Verify the status of the server certificate
     --cert-type Certificate file type (DER/PEM/ENG)
     --ciphers SSL ciphers to use
     --compressed    Request compressed response
 -K, --config Read config from a file
     --connect-timeout Maximum time allowed for connection
     --connect-to Connect to host
 -C, --continue-at Resumed transfer offset
 -b, --cookie Send cookies from string/file
 -c, --cookie-jar Write cookies to after operation
     --create-dirs   Create necessary local directory hierarchy
     --crlf          Convert LF to CRLF in upload
     --crlfile Get a CRL list in PEM format from the given file
 -d, --data    HTTP POST data
     --data-ascii HTTP POST ASCII data
     --data-binary HTTP POST binary data
     --data-raw HTTP POST data, '@' allowed
     --data-urlencode HTTP POST data url encoded
     --delegation GSS-API delegation permission
     --digest        Use HTTP Digest Authentication
 -q, --disable       Disable .curlrc
     --disable-eprt  Inhibit using EPRT or LPRT
     --disable-epsv  Inhibit using EPSV
     --dns-interface Interface to use for DNS requests
     --dns-ipv4-addr
IPv4 address to use for DNS requests

     --dns-ipv6-addr
IPv6 address to use for DNS requests

     --dns-servers DNS server addrs to use
 -D, --dump-header Write the received headers to
     --egd-file EGD socket path for random data
     --engine Crypto engine to use
     --expect100-timeout How long to wait for 100-continue
 -f, --fail          Fail silently (no output at all) on HTTP errors
     --fail-early    Fail on first transfer error, do not continue
     --false-start   Enable TLS False Start
 -F, --form Specify HTTP multipart POST data
     --form-string Specify HTTP multipart POST data
     --ftp-account Account data string
     --ftp-alternative-to-user String to replace USER [name]
     --ftp-create-dirs Create the remote dirs if not present
     --ftp-method Control CWD usage
     --ftp-pasv      Use PASV/EPSV instead of PORT
 -P, --ftp-port
Use PORT instead of PASV

     --ftp-pret      Send PRET before PASV
     --ftp-skip-pasv-ip Skip the IP address for PASV
     --ftp-ssl-ccc   Send CCC after authenticating
     --ftp-ssl-ccc-mode Set CCC mode
     --ftp-ssl-control Require SSL/TLS for FTP login, clear for transfer
 -G, --get           Put the post data in the URL and use GET
 -g, --globoff       Disable URL sequences and ranges using {} and []
 -I, --head          Show document info only
 -H, --header
Pass custom header(s) to server

 -h, --help          This help text
     --hostpubmd5 Acceptable MD5 hash of the host public key
 -0, --http1.0       Use HTTP 1.0
     --http1.1       Use HTTP 1.1
     --http2         Use HTTP 2
     --http2-prior-knowledge Use HTTP 2 without HTTP/1.1 Upgrade
     --ignore-content-length Ignore the size of the remote resource
 -i, --include       Include protocol response headers in the output
 -k, --insecure      Allow insecure server connections when using SSL
     --interface Use network INTERFACE (or address)
 -4, --ipv4          Resolve names to IPv4 addresses
 -6, --ipv6          Resolve names to IPv6 addresses
 -j, --junk-session-cookies Ignore session cookies read from file
     --keepalive-time Interval time for keepalive probes
     --key      Private key file name
     --key-type Private key file type (DER/PEM/ENG)
     --krb    Enable Kerberos with security
     --libcurl Dump libcurl equivalent code of this command line
     --limit-rate Limit transfer speed to RATE
 -l, --list-only     List only mode
     --local-port Force use of RANGE for local port numbers
 -L, --location      Follow redirects
     --location-trusted Like --location, and send auth to other hosts
     --login-options Server login options
     --mail-auth
Originator address of the original email

     --mail-from
Mail from this address

     --mail-rcpt
Mail from this address

 -M, --manual        Display the full manual
     --max-filesize Maximum file size to download
     --max-redirs Maximum number of redirects allowed
 -m, --max-time
     --metalink      Process given URLs as metalink XML file
     --negotiate     Use HTTP Negotiate (SPNEGO) authentication
 -n, --netrc         Must read .netrc for user name and password
     --netrc-file Specify FILE for netrc
     --netrc-optional Use either .netrc or URL
 -:, --next          Make next URL use its separate set of options
     --no-alpn       Disable the ALPN TLS extension
 -N, --no-buffer     Disable buffering of the output stream
     --no-keepalive  Disable TCP keepalive on the connection
     --no-npn        Disable the NPN TLS extension
     --no-sessionid  Disable SSL session-ID reusing
     --noproxy List of hosts which do not use proxy
     --ntlm          Use HTTP NTLM authentication
     --ntlm-wb       Use HTTP NTLM authentication with winbind
     --oauth2-bearer OAuth 2 Bearer Token
 -o, --output Write to file instead of stdout
     --pass Pass phrase for the private key
     --path-as-is    Do not squash .. sequences in URL path
     --pinnedpubkey FILE/HASHES Public key to verify peer against
     --post301       Do not switch to GET after following a 301
     --post302       Do not switch to GET after following a 302
     --post303       Do not switch to GET after following a 303
     --preproxy [protocol://]host[:port] Use this proxy first
 -#, --progress-bar  Display transfer progress as a bar
     --proto Enable/disable PROTOCOLS
     --proto-default Use PROTOCOL for any URL missing a scheme
     --proto-redir Enable/disable PROTOCOLS on redirect
 -x, --proxy [protocol://]host[:port] Use this proxy
     --proxy-anyauth Pick any proxy authentication method
     --proxy-basic   Use Basic authentication on the proxy
     --proxy-cacert CA certificate to verify peer against for proxy
     --proxy-capath CA directory to verify peer against for proxy
     --proxy-cert Set client certificate for proxy
     --proxy-cert-type Client certificate type for HTTS proxy
     --proxy-ciphers SSL ciphers to use for proxy
     --proxy-crlfile Set a CRL list for proxy
     --proxy-digest  Use Digest authentication on the proxy
     --proxy-header
Pass custom header(s) to proxy

     --proxy-insecure Do HTTPS proxy connections without verifying the proxy
     --proxy-key Private key for HTTPS proxy
     --proxy-key-type Private key file type for proxy
     --proxy-negotiate Use HTTP Negotiate (SPNEGO) authentication on the proxy
     --proxy-ntlm    Use NTLM authentication on the proxy
     --proxy-pass Pass phrase for the private key for HTTPS proxy
     --proxy-service-name SPNEGO proxy service name
     --proxy-ssl-allow-beast Allow security flaw for interop for HTTPS proxy
     --proxy-tlsauthtype TLS authentication type for HTTPS proxy
     --proxy-tlspassword TLS password for HTTPS proxy
     --proxy-tlsuser TLS username for HTTPS proxy
     --proxy-tlsv1   Use TLSv1 for HTTPS proxy
 -U, --proxy-user Proxy user and password
     --proxy1.0 Use HTTP/1.0 proxy on given port
 -p, --proxytunnel   Operate through a HTTP proxy tunnel (using CONNECT)
     --pubkey   SSH Public key file name
 -Q, --quote         Send command(s) to server before transfer
     --random-file File for reading random data from
 -r, --range Retrieve only the bytes within RANGE
     --raw           Do HTTP "raw"; no transfer decoding
 -e, --referer Referrer URL
 -J, --remote-header-name Use the header-provided filename
 -O, --remote-name   Write output to a file named as the remote file
     --remote-name-all Use the remote file name for all URLs
 -R, --remote-time   Set the remote file's time on the local output
 -X, --request Specify request command to use
     --request-target Specify the target for this request
     --resolve Resolve the host+port to this address
     --retry    Retry request if transient problems occur
     --retry-connrefused Retry on connection refused (use with --retry)
     --retry-delay Wait time between retries
     --retry-max-time Retry only within this period
     --sasl-ir       Enable initial response in SASL authentication
     --service-name SPNEGO service name
 -S, --show-error    Show error even when -s is used
 -s, --silent        Silent mode
     --socks4 SOCKS4 proxy on given host + port
     --socks4a SOCKS4a proxy on given host + port
     --socks5 SOCKS5 proxy on given host + port
     --socks5-basic  Enable username/password auth for SOCKS5 proxies
     --socks5-gssapi Enable GSS-API auth for SOCKS5 proxies
     --socks5-gssapi-nec Compatibility with NEC SOCKS5 server
     --socks5-gssapi-service SOCKS5 proxy service name for GSS-API
     --socks5-hostname SOCKS5 proxy, pass host name to proxy
 -Y, --speed-limit Stop transfers slower than this
 -y, --speed-time Trigger 'speed-limit' abort after this time
     --ssl           Try SSL/TLS
     --ssl-allow-beast Allow security flaw to improve interop
     --ssl-no-revoke Disable cert revocation checks (WinSSL)
     --ssl-reqd      Require SSL/TLS
 -2, --sslv2         Use SSLv2
 -3, --sslv3         Use SSLv3
     --stderr        Where to redirect stderr
     --suppress-connect-headers Suppress proxy CONNECT response headers
     --tcp-fastopen  Use TCP Fast Open
     --tcp-nodelay   Use the TCP_NODELAY option
 -t, --telnet-option Set telnet option
     --tftp-blksize Set TFTP BLKSIZE option
     --tftp-no-options Do not send any TFTP options
 -z, --time-cond
     --tls-max Use TLSv1.0 or greater
     --tlsauthtype TLS authentication type
     --tlspassword   TLS password
     --tlsuser TLS user name
 -1, --tlsv1         Use TLSv1.0 or greater
     --tlsv1.0       Use TLSv1.0
     --tlsv1.1       Use TLSv1.1
     --tlsv1.2       Use TLSv1.2
     --tlsv1.3       Use TLSv1.3
     --tr-encoding   Request compressed transfer encoding
     --trace   Write a debug trace to FILE
     --trace-ascii Like --trace, but without hex output
     --trace-time    Add time stamps to trace/verbose output
     --unix-socket Connect through this Unix domain socket
 -T, --upload-file Transfer local FILE to destination
     --url      URL to work with
 -B, --use-ascii     Use ASCII/text transfer
 -u, --user Server user and password
 -A, --user-agent Send User-Agent to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes

This is what I have on my IIS site, localhost


You can get a taste for cURL  using the GET command. The response you get is the HTML file for the localhost


Get.png

You can get the header information using the switch -i as shown (the image only shows the header but the above HTML is also in the response):


Getwithi.png

No comments: