ATLAS Worker Node

Introduction

This page illustrates the software used to support Grid interactions in OSG of ATLAS pilots. There are 2 packages:

• wn-client maintained by OSG and installed by system administrators
• ATLAS-wn maintained by ATLAS software managers and installed by them

The remaining of this page contains installation instructions and package description These are not user packages. If you are looking for a client for interactive use, please install WlcgClient

Installation

The WorkerNode client is a package required by OSG and installed in $OSG_GRID as mandated by OSG.

WorkerNode client does not have a version by itself. Refer to the VDT version to see if you have the latest available (1.10.1y as of 75/2009 but check VDT News). Use the standard WorkerNode client provided by OSG and install it in $OSG_GRID as explained in the OSG twiki.

To update, System administrators should replace the existing one (after a backup), e.g.:

rm -rf $OSG_GRID
mkdir $OSG_GRID
cd $OSG_GRID
pacman -get OSG:wn-client

During the Pacman installation you'll be asked some questions: check WLCG-Client installation for suggestions, the answers are the same. Probably you will be installing wn-client as root on a host which already has grid certificates installed and maintained in /etc/grid-security/.

If you prefer not to risk to interfere with regular production jobs see the alternative (longer) installation below

This is all a system administrator is required to install. The ATLAS pilot will also use additional software but this one will be installed and managed by the ATLAS pilot or the ATLAS software administrator (Xin) in a subdirectory of $OSG_APP. Specifically the software will be installed in $OSG_APP/atlas_app/atlaswn/ and the package installed is ATLAS-wn ( pacman -get http://www.mwt2.org/caches/:ATLAS-wn), containing dq2-client and other ATLAS specific software auxiliary to the jobs. Anyway system administrators have not to worry about this (Just FYI).

More cautious update

If you prefer not to risk to interfere with regular production jobs you can pick an alternative install directory but in this case you'll have to communicate the new directory to whoever is testing the pilots (probably Paul and the panda shift team) and let them know that for the test jobs that directory (the one where you installed wn-client_dev) is the new OSG_GRID and they have to source the setup file in that directory.

ATLAS wn package

The package ATLAS-wn has been created in order to rationalize and group together all middleware packages used by ATLAS (and not provided by OSG). This will be installed by the ATLAS software manager on all sites running ATLAS jobs in a well known subpath of $OSG_APP (currently $OSG_APP/atlas_app/atlaswn/). This package is similar to OSG's WN-Client in the sense that it contains software that must be available to all ATLAS jobs (1. the path has to be mounted on all WNs and headnodes -for batch and fork jobs-; 2. the setup file must be sourced by the atlas pilot), but it contains software controlled and needed only by ATLAS, therefore not included in OSG's WN-Client.

At the moment it contains dq2-client. To install the package you can type:

pacman -get http://www.mwt2.org/caches/:ATLAS-wn

Update

As it is today ATLAS-wn is a package adding features used mainly in analysis jobs and the recommended setup is not to fail the pilot if the package is not there. This allows for updates while the site is online and running production jobs.

Known issues / Fixed issues

v0.14 of ATLAS-wn fixes the PATH removal of expat performed by ATLAS release. Due to the setup order coded in Pathena the LFC library provided by WN-client was broken by the ATLAS release setup. This was affecting jobs using TAGs or back-navigation and showing as a security error in LFC commands. ATLAS-OSG-compat included in ATLAS-wn 0.14 fixes this problem and is also a placeholder for future compatibility fixes.

Use

There is no interactive use. Pilots (or other programs) will source both wn-client and ATLAS-wn setup files to have all the required clients. Supposing that command_string contains the command you want to execute, in your program you may use something like:

setup_string = "source $OSG_GRID/setup.sh"
if os.path.exists("$OSG_APP/atlas_app/atlaswn/setup.sh"):
    setup_string += "; source $OSG_APP/atlas_app/atlaswn/setup.sh"
ecode, out = commands.getstatusoutput(setup_string+'; '+command_string)

wn-client is guaranteed to exist on every OSG CE, older installations may not have ATLAS-wn so it is safer to check for it. The fragment above is a simple example, in your code you may want to add at least some exception or signals handling (try... except) and you may want to check separately for the environment variables and substitute them in the strings (os.getenv).

32 or 64 bit

There may be subtle errors coming from mixing 32bit and 64bit software. A 64bit OS with compatibility libraries installed (and a 32bit OS) can run 32bit software without problems:

• python
• wn-client
• ATLAS releases

If a 64bit applications is run (e.g. python 64bit) it will require 64bit libraries. Some packages in our distribution have only 32bit libraries (e.g. the MySQL binding included in DQ2Clients are currently only 32bit). So whenever those are used all the application needs to be 32bit. Also if a cluster has shared installation directories and mixed Worker Nodes (32 and 64 bit OSes), then all the grid middleware should be 32bit in order to run on both. In both cases you'll have to force VDT against its default and perform a 32bit installation.

Normally VDT chooses the right package for your platform and you should let VDT do it. It will install 32bit packages on 32bit OS, 64bit packages on 64bit OS.

Full 32bit installation

If you have a mixed environment (some 32 and some 64 bit machines) it may be convenient.

• You need to install compatibility libraries, at least the ones suggested in https://twiki.cern.ch/twiki/bin/view/Atlas/RPMcompatSLC4
• Before any VDT installation set the environment variable: export VDT_PRETEND_32=1
• Then install the package with Pacman (e.g. wn-client)
• Most likely you need also Python 32 bit: you can install on your own or use pacman -get http://www.mwt2.org/caches/:Python32bit and install it on top of the wn-client Pacman cache (it could be a different one but you'd have to add it to the default environment). The Python delivered is Python 2.4 (C API 1012 required by _lfc.so) and it is compiled on a SL4.7.

Since the VDT installation includes Python C extensions (e.g. LFC client), a 32 bit installation requires a 32 bit Python.

Notes on CA Certificates Directories

If you choose not to install the CA certificates and CRL updates when you install [[https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/WorkerNodeClient][_wn-client_] you still need to remember to link the certificate directory. Supposing that the certificates are in /etc/grid-security/certificates (default system location):

ln -s /etc/grid-security/certificates $VDT_LOCATION/globus/TRUSTED_CA

.

Without this worker nodes that source the setup scripts won't have proper environment setup for grid use.

-- MarcoMambelli - 07 Oct 2008