======================================================================= Documentation on dCache and the code is available at http://www.dcache.org ======================================================================= Distribution (06/04/2004) ========================= A set of RPMS allowing to install a full blown dCache instance can be obtained from http://www.dcache.org/downloads/dcache-v1.2.2-1.tgz The set contains 4 RPMs and some basic installation and user instructions. Dependencies ------------ A recent version of JAVA is expected to be installed (both jre and jsdk are fine). To support GridFTP and SRM a host certificate is required. Globus may be required on the client (to obtain a proxy, gridftp client and server, etc.). System Monitoring ================= http://admin.node.org:2288 - allows monitoring of services only Admin Interface =============== The admin interface offers a very rich set of commands (to be described elsewhere) allowing to alter system configuration while the system is running and to solve eventual problems. Since the distribution comes with an initial password it is important to login right after the installation in order to customize it. Note: The following is supported in version 1.2.2-1 for the first time and will be supported in future releases. Assuming you are logged in on the admin node log in to the admin interface to set the password ssh -l admin -c blowfish -p 22223 localhost (passwd : dickerelch) (local) admin > cd acm (acm) admin > create user admin (acm) admin > set passwd (acm) admin > .. (local) admin > logoff From now on login as user admin will only be successful if newPasswd is presented. Note: When setting the password string in the shell one can disable the echo by typing "ctrl I" following "set password". Services and Clients ==================== Client components are installed under /opt/d-cache. The libraries (32 and 64-bit versions) can be found at /opt/d-cache/dcap/lib. libdcap.so and libpdcap.so are symbolic links pointing to the 64-bit version. Also the gsidcap tunnel lib (libgsiTunnel.so) is installed here. Besides the libraries header files (/opt/d-cache/dcap/include) and the dccp binary supporting file copy applications (/opt/d-cache/dcap/bin) are installed with the Client RPM). dCap ---- There is a library the application can be linked against in addition to a cp-like copy program that allows to copy entire files into and out of dCache, called dccp. More information on both the libraries (libdcap.so and libpdcap.so, with latter being a preload library) and dccp can be found at http://www.dcache.org/manuals/index.shtml Example: - dccp can be used the same way cp works, however this requires the pnfs filesystem to be mounted on the node where the client is running. dccp /path/to/source/file /pnfs/path/to/destination/file Mounting pnfs mount -o intr,rw,noac,hard :/pnfs / - dccp can be used with URL-style addressing (does not require the pnfs filesystem to be mounted) Note: since no specific authentication mechanism enforces security only reading from dCache is supported dccp dcap://node.domain.org/pnfs/domain.org/data/cms/cms001/a-1 \ /tmp/a gsiDcap ------- To overcome the security issues mentioned above there is a dCap door to dCache that uses the Grid Security Infrastructure (gsi). Users holding a valid proxy issue a command like dccp \ gsidcap://node.domain.org:22128/pnfs/domain.org/data/cms/cms001/a-1 \ /tmp/a Add the following path to your LD_LIBRARY_PATH /opt/d-cache/dcap/lib GridFTP ------- A gridftp server that is interoperable with the version distributed by Globus is running on port 2811. Any client that can be used with the Globus GridFTP server (e.g. globus-url-copy) can be used to store and retrieve files via the GridFTP server running as a frontend to dCache. Since more attributes are required for storage services authentication within GridFTP sessions is not handled via the grid-mapfile. Authentication information (DNs etc) is taken from the following file: /opt/d-cache/etc/dcache.kpwd. The file contains directions regarding the format. Examples: - Storing a file from a local filesystem in dCache globus-url-copy \ file:///tmp/a \ gsiftp://node.domain.org/pnfs/domain.org/data/cms/cms001/a-4 - Reading a file back to the local filesystem globus-url-copy \ gsiftp://node.domain.org/pnfs/domain.org/data/cms/cms001/a-4 \ file:///tmp/a - globus-url-copy supports third party transfers. Please note that this requires the client to be fully connected to the public net (not required for SRM based third party transfers). globus-url-copy \ gsiftp://node.domain.org/pnfs/domain.org/data/cms/cms001/a-4 gsiftp://node.cern.ch/pnfs/cern.ch/data/cms/a - The performance of transfers, especially over the WAN, can be largely improved by using parallel streams and by increasing the tcp block size. 10 streams (-p 10) and a block size of 1MB (-tcp-bs 1048576) fits most bandwidth*delay product values for traffic within Europe and between Europe and the US SRM --- A Storage Resource Manager, compliant with the V1 specifications is available as a plug-in to dCache. It is running on port 8443. It supports qeueing for get, put and copy requests. A checksum (Adler32) creation and verification mechanism has been implemented to protect against data corruption. Resiliency in terms of protection against network outages and other failures (hangs, disconnects etc.) has been added to the bare data movement that is currently based on GridFTP (note: other transfer protocls are foreseen as well and SRM supports protocol negotiation between srm servers) Using the srmcp client application requires to have the dCache client RPM installed. Running srmcp for the first time will create a configuration file in ~/.srmconfig of the users' home directory. If needed parameters can be modified as required. The following environment variables should be set before running srmcp (shown for /bin/csh) setenv PATH /usr/java/j2sdk1.4.2_01/bin\:$PATH\:/opt/d-cache/srm/bin setenv JAVA_HOME /usr/java/j2sdk1.4.2_01 setenv GLOBUS_LOCATION /opt/globus source $GLOBUS_LOCATION/etc/globus-user-env.csh setenv X509_USER_PROXY ~/k5-ca-proxy.pem setenv SRM_PATH /opt/d-cache/srm/ Examples: - Storing a file from a local filesystem on a remote SE srmcp file:////tmp/a \ srmcp srm://node.domain.org:8443/pnfs/domain.org/data/cms/a-4 - Reading a file from a remote SE to the local filesystem srmcp srm://node.domain.org:8443/pnfs/domain.org/data/cms/a-4 \ file:////tmp/a - srmcp supports true third party transfer, i.e. clients can be installed behind a firewall and do not require full connectivity to the public net (note: third party transfers with g-u-c require ful connectivity to both servers involved in the transfer). srmcp srm://node.domain.org:8443/pnfs/domain.org/data/cms/a-4 \ srm://node.cern.ch:8443/pnfs/cern.ch/cms/a - To increase the transfer efficiency srmcp allows to batch files. On the command line multiple source URLs going to a destination URL (directory) can be specified. A file specifying a source and destination pair can be used as well: -copyjobfile= where is the path to the text file containing the list of sources and destination each line has a format: Get the full list of srmcp features with "srmcp -help".