xrdcp - Man Page

xrdcp [options] source destination

options: [--cksum args] [--debug lvl]
[--coerce] [--force] [--help] [--license]
[--nopbar] [--posc] [--proxy ipaddr:port]
[--recursive] [--retry time] [--server]
[--silent] [--sources num] [--streams num]
[--tpc [delegate] first|only] [--verbose]
[--version] [--xrate rate] [--zip file]
[--dynamic-src] [--infiles fn]
[--parallel n] [--allow-http] [--xattr]
[--notlsok] [--tlsnodata] [--tlsmetalink]
[--zip-mtln-cksum] [--rm-bad-cksum] [--continue]
[--xrate-threshold] [--retry-policy]

legacy options: [-adler] [-DSparm string] [-DIparm number]
[-md5] [-np] [-ODcgi] [-OScgi] [-x]

The xrdcp utility copies one or more files from one location to another. The data source and destination may be a local or remote file or directory.  Additionally, the data source may also reside on multiple servers.

-C | --cksum type[:value|print|source]

obtains the checksum of type (i.e. adler32, crc32, md5 or zcrc32) from the source, computes the checksum at the destination, and verifies that they are the same. If auto is chosen as the checksum type, xrdcp will try to automatically infer the right checksum type based on source/destination configuration, source file type (metalink, ZIP), and available checksum plug-ins. If a value is specified, it is used as the source checksum. When print is specified, the checksum at the destination is printed but is not verified.

-d | --debug lvl

debug level: 1 (low), 2 (medium), 3 (high)

-F | --coerce

ignores locking semantics on the destination file. This option may lead to file corruption if not properly used.

-f | --force

re-creates a file if it is already present.

-h | --help

displays usage information.

-H | --license

displays license terms and conditions.

-N | --nopbar

does not display the progress bar.

-P | --posc

requests POSC (persist-on-successful-close) processing to create a new file. Files are automatically deleted should they not be successfully closed.

-D | --proxy proxyaddr:proxyport

[NOT YET IMPLEMENTED]

use proxyaddr:proxyport as a SOCKS4 proxy. Only numerical addresses are supported.

-r | --recursive

recursively copy all files starting at the given source directory.

--retry

retry failed copy-jobs.

--server

runs as if in a server environment. Used only for server-side third party copy support.

-s | --silent

neither produces summary information nor displays the progress bar.

-y | --sources num

uses up to num sources to copy the file.

-S | --streams num

use num parallel data streams to do the transfer (the main stream is  in this case not used to carry out the transfer). The maximum value is 15. The default is 0 (i.e., use only the control stream).

--tpc [delegate] first|only

copies the file from remote server to remote server using third-party-copy protocol (i.e., data flows from server to server). The source and destination servers must support third party copies. Additional security restrictions may apply and may cause the copy to fail if they cannot be satisfied. Argument 'first' tries tpc and if it fails, does a normal copy; while 'only' fails the copy unless tpc succeeds. When 'delegate' is specified, the copy delegates the command issuer's credentials to the target server which uses those credentials to authenticate with the source server. Delegation is ignored if the target server is not configured to use delegated credentials. Currently, only gsi credentials can be delegated.

-v | --verbose

displays summary output.

-V | -version

displays version information and immediately exits.

-z | --zip file

copy given file from a ZIP archive (same as xrdcl.unzip opaque info).

-X | --xrate rate

limits the copy speed to the specified rate. The rate may be qualified with the letter k, m, or g to indicate kilo, mega, or giga bytes, respectively. The option only applies when the source or destination is local.

-X | --xrate-threshold rate

If the transfer rate drops below given threshold force the client to use different source or if no more sources are available fail the transfer.

-Z | --dynamic-src

file size may change during the copy

-I | --infiles fn

specifies the file that contains a list of input files

--parallel n

number of copy jobs to be run simultaneously

--allow-http

allow HTTP as source or destination protocol. Requires the XrdClHttp client plugin

--xattr

preserve extended attributes

--notlsok

If server is too old to support TLS encryption fallback to unencrypted communication.

--tlsnodata

In case of roots/xroots protocol, encrypt only the control stream and leave the data streams unencrypted.

--tlsmetalink

Treat all URLs in metalink as roots/xroots.

--zip-mtln-cksum

Use the checksum available in a metalink file even if a file is being extracted from a ZIP archive.

--rm-bad-cksum

Remove the target file if checksum verification failed (enables also POSC semantics).

--continue

Continue copying a file from the point where the previous copy was interrupted.

Legacy options are provided for backward compatibility. These are now deprecated and should be avoided.

-adler

equivalent to "--cksum adler32:source".

-DIpname numberval

set the internal parameter pname with the numeric value numberval.

-DSpname stringval

set the internal parameter pname with the string value stringval.

-md5

equivalent to "--cksum md5:source".

-np

equivalent to "--nopbar".

-ODcgi

add cgi information cgi to any destination xrootd URL. You should specify the opaque information directly on the destination URL.

-OScgi

add cgi information cgi to any source xrootd URL.

-x

equivalent to "--sources 12".

source

a dash (i.e. -) indicating stanard in, a local file, a local directory name suffixed by /, or an xrootd URL in the form of

xroot://[user@]host[:port]/absolutepath

The absolutepath can be a directory.

destination

a dash (i.e. -) indicating stanard out, a local file, a local directory name suffixed by /, or an xrootd URL in the form

xroot://[user@]host[:port]/absolutepath

The absolutepath can be a directory.

The following environment variables are supported. They apply to xrdfs and any other application using the libXrdCl library, unless specified otherwise. The text in the brackets is a name of the corresponding xrdcp commandline parameter.

XRD_LOGLEVEL

Determines the amount of diagnostics that should be printed. Valid values are: Dump, Debug, Info, Warning, and Error.

XRD_LOGFILE

If set, the diagnostics will be printed to the specified file instead of stderr.

XRD_LOGMASK

Determines which diagnostics topics should be printed at all levels. It's a "|" separated list of topics. The first element may be "All" in which case all the topics are enabled and the subsequent elements may turn them off, or "None" in which case all the topics are disabled and the subsequent flags may turn them on. If the topic name is prefixed with "^", then it means that the topic should be disabled. If the topic name is not prefixed, then it means that the topic should be enabled.

The log mask may as well be handled for each diagnostic level separately by setting one or more of the following variables: XRD_LOGMASK_ERROR, XRD_LOGMASK_WARNING, XRD_LOGMASK_INFO, XRD_LOGMASK_DEBUG, and XRD_LOGMASK_DUMP. The default for each level is "All", except for the Dump level, where the default is "All|^PollerMsg". This means that, at the Dump level, all the topics but "PollerMsg" are enabled.

Available topics: AppMsg, UtilityMsg, FileMsg, PollerMsg, PostMasterMsg, XRootDTransportMsg, TaskMgrMsg, XRootDMsg, FileSystemMsg, AsyncSockMsg, TlsMsg

XRD_TLSDBGLVL

Determine the debug level for the TLS component. Valid values are: OFF, CTX, SOK, SIO, ALL and OUT. The default value is OFF.

XRD_PARALLELEVTLOOP

The number of event loops (i.e. the number of threads handling requests). Default number is 10.

XRD_READRECOVERY

Determines if read recovery should be enabled or disabled (enabled by default).

XRD_WRITERECOVERY

Determines if write recovery should be enabled or disabled (enabled by default).

XRD_OPENRECOVERY

Determines if open recovery should be enabled or disabled for mutable (truncate or create) opens (enabled by default).

XRD_CONNECTIONWINDOW (-DIConnectionWindow)

A time window for the connection establishment. A connection failure is declared if the connection is not established within the time window. If a connection failure happens earlier then another connection attempt will only be made at the beginning of the next window.

XRD_CONNECTIONRETRY (-DIConnectionRetry)

Number of connection attempts that should be made (number of available connection windows) before declaring a permanent failure.

XRD_REQUESTTIMEOUT (-DIRequestTimeout)

Default value for the time after which an error is declared if it was impossible to get a response to a request.

XRD_STREAMTIMEOUT (-DIStreamTimeout)

Default value for the time after which a connection error is declared (and a recovery attempted) if there are unfulfilled requests and there is no socket activity or a registered wait timeout.

XRD_SUBSTREAMSPERCHANNEL (-DISubStreamsPerChannel)

Number of streams per session.

XRD_TIMEOUTRESOLUTION (-DITimeoutResolution)

Resolution for the timeout events. Ie. timeout events will be processed only every XRD_TIMEOUTRESOLUTION seconds.

XRD_STREAMERRORWINDOW (-DIStreamErrorWindow)

Time after which the permanent failure flags are cleared out and a new connection may be attempted if needed.

XRD_RUNFORKHANDLER (-DIRunForkHandler)

Determines whether the fork handlers should be enabled, making the API fork safe.

XRD_REDIRECTLIMIT (-DIRedirectLimit)

Maximum number of allowed redirections.

XRD_NOTAUTHORIZEDRETRYLIMIT (-dNotAuthorizedRetryLimit)

Maximum number of allowed retries at a meta-manager for not-authorized error.

XRD_POLLERPREFERENCE (-DSPollerPreference)

A comma separated list of poller implementations in order of preference. The default is: built-in.

XRD_CLIENTMONITOR (-DSClientMonitor)

Path to the client monitor library.

XRD_CLIENTMONITORPARAM (-DSClientMonitorParam)

Additional optional parameters that will be passed to the monitoring object on initialization.

XRD_WORKERTHREADS (-DIWorkerThreads)

Number of threads processing user callbacks.

XRD_CPPARALLELCHUNKS (-DICPParallelChunks)

Maximum number of asynchronous requests being processed by the xrdcp command per connected channel substream (adjusted in real-time).

XRD_CPCHUNKSIZE (-DICPChunkSize)

Size of a single data chunk handled by xrdcp.

XRD_NETWORKSTACK (-DSNetworkStack)

The network stack that the client should use to connect to the server. Possible values are:

IPAuto - automatically detect which IP stack to use

IPAll - use IPv6 stack (AF_INET6 sockets) and both IPv6 and IPv4 (mapped to IPv6) addresses

IPv6 - use only IPv6 stack and addresses

IPv4 - use only IPv4 stack (AF_INET sockets) and addresses

IPv4Mapped6 - use IPv6 stack and mapped IPv4 addresses

XRD_DATASERVERTTL (-DIDataServerTTL)

Time period after which an idle connection to a data server should be closed.

XRD_LOADBALANCERTTL (-DILoadBalancerTTL)

Time period after which an idle connection to a manager or a load balancer should be closed.

XRD_APPNAME (-DSAppName)

Override the application name reported to the server.

XRD_PLUGINCONFDIR

A custom location containing client plug-in config files.

XRD_PLUGIN

A default client plug-in to be used.

XRD_CPINITTIMEOUT (-DICPInitTimeout)

Maximum time allowed for the copy process to initialize, ie. open the source and destination files.

XRD_CPTPCTIMEOUT (-DICPTPCTimeout)

Maximum time allowed for a third-party copy operation to finish.

XRD_TCPKEEPALIVE (-DITCPKeepAlive)

Enable/Disable the TCP keep alive functionality

XRD_TCPKEEPALIVETIME (-DITCPKeepAliveTime)

Time between last data packet sent and the first keepalive probe (Linux only)

XRD_TCPKEEPALIVEINTERVAL (-DITCPKeepAliveInterval)

Interval between subsequent keepalive probes (Linux only)

XRD_TCPKEEPALIVEPROBES (-DITCPKeepProbes)

Number of unacknowledged probes before considering the connection dead (Linux only)

XRD_METALINKPROCESSING

Enable/Disable Metalink processing (enabled by default)

XRD_LOCALMETALINKFILE

Enable/Disable local Metalink file processing (by convention the following URL schema has to be used: root://localfile//path/filename.meta4) The 'localfile' semantic is now deprecated, use file://localhost/path/filename.meta4 instead!

XRD_GLFNREDIRECTOR

The redirector will be used as a last resort if the GLFN tag is specified in a Metalink file.

XRD_XCPBLOCKSIZE

Maximu size of a data block assigned to a single source in case of an extreme copy transfer.

XRD_NODELAY

Disables the Nagle algorithm if set to 1 (default), enables it if set to 0.

XRD_PREFERIPV4

If set the client tries first IPv4 address (turned off by default).

XRD_MAXMETALINKWAIT

The maximum time in seconds a clinet can be stalled by the server if a Metalink redirector is available (defaults to 60s).

XRD_PRESERVELOCATETRIED

If set to 1 XRootD client will preserve tried/triedrc cgi opaque info for kXR_locate request across redirects/retries, if set to 0 XRootD client will treat kXR_locate as any other passive request.

XRD_PRESERVEXATTRS

If set to 1 (default) xrdcp will preserve file extended attribues, if set to 0 file extended attributes won't be preserved.

XRD_NOTLSOK

If set to 1 and the server is to old to support TLS encryption, xrdcp will fallback to unencrypted transmission although roots/xroots protocol was used. By default set to 0.

XRD_TLSNODATA

If set to 1, in case of roots/xroots protocol only the control stream will be encrypted, data streams will be left unencrypted. By default set to 0.

XRD_TLSMETALINK

If set to 1 all URLs in metalink will be treated as roots/xroots. By default set to 0.

XRD_ZIPMTLNCKSUM

If set to 1, use the checksum available in a metalink file even if a file is being extracted from a ZIP archive. By default set to 0;

XRD_CPTIMEOUT

Timeout for a classical (not TPC) copy job. By default set to 0 (disabled);

XRD_CLCONFDIR

User defined directory with config files (*.conf).

XRD_CLCONFFILE

User defined config file.

XRD_CPRETRY

Maximum number of times to retry failed copy-jobs (default: 0).

XRD_CPRETRYPOLICY

Copy job retry policy, either force or retry (default: force).

XRD_CPUSEPGWRTRD

Enable in-fly error correction of corrupted pages (default: 1).

50  : generic error (e.g. config, internal, data, OS, command line option)

51  : socket related error

52  : postmaster related error

53  : XRootD related error

54  : redirection error

55  : query response was negative (this is not an error)

Documentation for all components associated with xrdcp can be found at http://xrootd.org/docs.html

Errors yield an error message and a non-zero exit status.

LGPL

The xrdcp command is supported by the xrootd collaboration. Contact information can be found at:

http://xrootd.org/contact.html

xrdfs(1).

The man page xrdcopy(1) is an alias of xrdcp(1).