Java CoG Kit Installation Guide

Gregor von Laszewski, Mike Hategan, and Deepti Kodeboyina

Software Version: 4_1_2
Manual version: 02/18/05
Url: http://www.cogkit.org/release/4_1_2/manual/install.pdf
Url: http://www.cogkit.org/release/4_1_2/manual/install/install.html
Last update: July 11, 2005

Contents

1 About this Document
 1.1 Reproduction
 1.2 Viewing
 1.3 Format
2 Registration
3 Introduction
 3.1 Concepts
 3.2 Bundles
4 Decision Guide
 4.1 CVS, Source, or binary?
 4.2 Full or partial?
5 Complete Binary Install
 5.1 Summary
6 Partial Binary Install
 6.1 Summary
7 Source Distribution Install
 7.1 Summary
8 CVS Release Distribution Install
9 Development Distribution
 9.1 CVS Navigation Table
10 Downloading the API Documentation
11 Webstart Component Install
 11.1 Summary
12 Configuration of the Java CoG Kit
Appendix
A Java CoG Kit Guides
B Java CoG Kit Guides Under Construction
C Available Downloads
D Availability of the Document
E Bugs
F Administrative Contact

 

1 About this Document

This document includes basic information about the different Java CoG Kit distributions in order to help you chose the one that is most suitable to your needs.

1.1 Reproduction

The material presented in this document can not be published, mirrored, electronically or otherwise reproduced without prior written consent. As you can link to this document, this should not pose much of a restriction.

1.2 Viewing

The best way to read this document is with Adobe Acrobat Reader. Please make sure you configure Adobe Acrobat Reader appropriately so you can follow hyperlinks. This is the case if you follow the default installation. Acrobat Reader is available at http://www.adobe.com/products/acrobat/readermain.html. Because the hyperlinks are not available in the printed form of this manual and we support saving our environment we strongly discourage printing this document.

We recommend that you save this manual locally on your machine and use Acrobat Reader. This has the advantage that you do not lose your anchor points while switching back and forth between different hyperlinks. An HTML version of this manual is planed, but not available yet.

1.3 Format

We have augmented the document with some comments at places where we found issues. Our intend is to address these issues in a future release. The comments are marked by the icon  to do  and the name of the person that will work on the removal of the issue.

2 Registration

Please be a team player and support us indirectly by registering with us or reporting your use of the Java CoG Kit. Although this software is free, we still need to justify to our funders the usefulness of the projects. If you want to help us with our efforts please take a few seconds to complete this information. We do not use this information for other purposes. If you have special needs or concerns please contact gregor@mcs.anl.gov. The registration form can filled out in a variety of formats. The online form can be found at

http://www.cogkit.org/register

This form is available also as ASCII text at

http://www.cogkit.org/register/form.txt

which you can FAX to

Gregor von Laszewski, Fax: 630 252 1997

3 Introduction

Originally, we distributed the Java CoG Kit as a single package called cog-jglobus. However, the distribution has grown and in order to prevent that a large number of packages need to be downloaded and installed on each client or server, we decided to divide the contents of the Java CoG Kit into separate modules and build a system that allows incremental deployment. Each part of the Java CoG Kit distribution is preceded by the keyword “cog” and its version number. Please note that the “.” between version numbers is replaced by an “_”. The base for the package names are

cog-4_1_2-<package>[-<version>].[binsrc].[tar.gz—zip]

where <package> is the name of the package and <version> is an optional internal version number of the package. Hence, our packages are distributed in tar.gz or zip format.

3.1 Concepts

To avoid large packages, we have bundled a number of correlated components in separate packages. The names of the packages correspond to the naming scheme mentioned earlier. To illustrate which components of the Java CoG Kit are associated with which packages we like that you spend a moment inspecting Figures 1 and 2.


PIC

Figure 1: View of the layered architecture of the Java CoG Kit.



PIC

Figure 2: Java CoG Kit packages and their relationship to the Java CoG Kit layerd architecture.


The Java CoG Kit layered architecture provides a framework for developing Grid clients and servers. Hence, you will be able to access classes and methods not only including a convenient API focusing on elementary Grid functionality, but also the ability to formulate workflows through XML specifications, command line tools, and graphical user interfaces. The Java CoG Kit layers are conceptually organized as follows:

Providers:
A Java CoG Kit provider presents a way of interfacing convenient Grid abstractions pioneered by the Java CoG Kit with existing Grid middleware or distributed systems software. We have developed prototype providers for GT2, various versions of GT3, SSH, your local machine, and Condor1 . A provider for GT4 is under development. While using the Java CoG Kit abstractions, the user need not to worry about the underlying Grid middleware changes (see also Basic Grid abstractions). As providers are dynamically loaded at runtime, the provider choice can be made by late binding.
Basic abstractions:
The Java CoG Kit basic abstraction layer is a convenient subset of Grid functionality that may be used by the Grid developer to hide different versions of Grid middleware. These abstractions include file transfer, file access functions, job submission, and authentication. This subset is chosen to fulfill the needs of many, but not all Grid users. As the abstractions contain only a subset, we encourage you to help us in identifying future useful subsets that meet your needs.
Data and task management abstractions:
Besides the basic abstractions, we have developed a number of more advanced abstractions that support more sophisticated Grid paradigms. The abstractions that we provide and will enhance in future versions of the Java CoG Kit deal with data and task management issues. As these paradigms are not yet part of the Globus Toolkit, they provide a significant enhancement to the available Grid tools and services. We believe that our more sophisticated abstractions will have to be included into future Grid middleware in order to increase the functionality of Grids, thus making them easier to use. Not only are many of these abstractions available as a convenient API, but they can also be accessed as simple XML descriptions. Our abstractions are based on Java interfaces making it possible to extend their functionality or provide alternative implementations.
Grid faces:
Based on the observation that visual components follow a particular pattern, we also have introduced an abstraction layer for visual components. To illustrate this concept, consider the common visual pattern of a Grid directory browser. Although the implementation of the directory browser between a stand alone Java Application and a portlet is different, many graphical features between the two are identical. Thus, each has a window in which the tree is displayed or a field in which the URL is entered. At this time, we have included in our release a Java CoG Kit desktop that includes a number of such abstractions available through a single integrated interface. Additionally, we have started to develop Grid portlets that are also distributed with the Open Grid Computing Environment (OGCE) at http://www.ogce.org.

Although Figure 1 and 2 contain some additional components, they are not yet part of the Java CoG Kit. However, in future versions we anticipate you include for example also providers contributed by the community, such as the Unicore provider.

3.2 Bundles

The distribution bundles as introduced in Figures 1 and 2 are available as .tar.gz or .zip, and as binary (bin) or source (src) distributions. The names of the bundles and their purpose are as follows:

cog-4_1_2-jglobus-1_2
 
Is the original Java CoG Kit, that is now also distributed with GT3 and GT4. It includes the pre-ws-GRAM APIs, the newest GridFTP APIs, and the GSI APIs.

Other CoG Kit bundles listed next are not yet distributed with GT3 and GT4.

However, you will see that the functionality of these bundles enhances the Grid experience significantly. Hence, it will be worthwhile for you to consider their benefits and use them appropriately.

cog-4_1_2-main
 
Contains the Grid abstractions, workflow, desktop, shell and other convenient features that enhance the Grid experience beyond the concepts that are available in the Globus Toolkit.
cog-4_1_2-provider-ssh
 
Contains a provider to interface with ssh servers.
cog-4_1_2-provider-gt3-common
 
Contains components that are common amongst all Java CoG Kit GT3 providers.
cog-4_1_2-provider-gt3_0_2
 
Contains a provider to interface with GT3.0.2 servers.
cog-4_1_2-provider-gt3_2_0
 
Contains a provider to interface with GT3.2.0 servers.
cog-4_1_2-provider-gt3_2_1
 
Contains a provider to interface with GT3.2.1 servers.
cog-4_1_2-provider-gt4_0
 
 to do  This is not yet distributed with this release. Contains a provider to interface to GT4 servers.
cog-4_1_2-provider-local
 
Contains a provider to interface to your local machine
cog-4_1_2-provider-webdav
 
Contains a provider to interface to a WebDAV server.

Often we are asked why there are so many different GT3 versions. The answer is simple because various GT3 versions distribute different jar files but use internally the same name space. In order to avoid side effects between these different implementations, we redistribute them as part of our distribution strategy. In the Java CoG Kit GT3 common library you find the parts that are common amongst the three GT3 providers. To find more out about the GT3 compatibility, please consult the documentation at http://www.globus.org.

In case you inspect Figure 2 carefully, you may notice that we distribute at present the GT2 providers directly within our main distribution. This is due to the fact that this particular provider is at present time so popular and we wanted to avoid distributing it as separate provider bundle.

4 Decision Guide

Now that you have an overview about what is available, you may ask which version is the right version for you?

We provide the following guide that will make your choice easier. Please note that we have two dimensions in order to assist you with the install decision. You need to decide if you would like an install from CVS, a source, or a binary install. You need also to decide if you like a full or a partial install.

4.1 CVS, Source, or binary?

4.2 Full or partial?

Once you have decided which version is most appropriate for you we recommend that you continue reading at the appropriate section. In case you have not yet made up your mind read the following sections carefully or consult your decision question while using our ticketing system at http://www.cogkit.org/bugzilla. Alternatively, you can send e-mal to gregor@mcs.anl.gov with the subject line “Question: CoG Kit, <question>, where question is a short summary of the question formulated in the e-mail body.

5 Complete Binary Install

After registering and downloading the complete package [tar.gz][zip], unzip or untar in a directory of your choosing. Proceed by setting up the CoG Kit with the CoG Kit setup component.

5.1 Summary

  1. register with the Java CoG Kit as described in Section 2.
  2. download the complete package [tar.gz][zip]
  3. unpack the package
  4. change to the package directory
  5. start the setup with ./bin/cog-setup

6 Partial Binary Install

A partial binary install is almost as easy to conduct. First, you have to be clear about which packages you need. You will always need the main package [tar.gz][zip]. However, you should only download and install providers that you need. After you have downloaded the needed packages you unpack them and start the setup as explained earlier. The list of packages currently available is as follows:

In any case, you will need:

Please add additional packages. In case you need a GT3 provider, you will also need the Common GT3 package:

6.1 Summary

  1. register with the Java CoG Kit as described in Section 2.
  2. download the main package [tar.gz][zip]
  3. download other desired providers and packages
  4. mkdir cogkit
  5. place the packages in ./cogkit
  6. cd cogkit
  7. unpack the packages
  8. cd cog
  9. start the setup with ./bin/cog-setup

7 Source Distribution Install

After downloading the Java CoG Kit source package [tar.gz][zip], unpack it in a directory, then run

ant dist

in the cog directory and execute the setup.

The jglobus sources are also available as [tar.gz] and [zip] files.

7.1 Summary

  1. register with the Java CoG Kit as described in Section 2.
  2. mkdir cogkit
  3. mv cog-4_1_2-src.tar.gz cog
  4. cd cogkit
  5. tar -xzvf cog-4_1_2-src.tar.gz
  6. cd cog
  7. ant dist
  8. cd dist/cog-4_1_2
  9. ./bin/cog-setup

8 CVS Release Distribution Install

In case you need the most up to date version of the Java CoG Kit, you can check this version out from the CVS. To do so you should follow these simple steps:

  1. register with the Java CoG Kit as described in Section 2.
  2. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit login
  3. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout -rP cog_4_1_2  src/cog
  4. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout -rP cog_4_1_2  src/jglobus

In case you are not interested in looking at the jglobus or the matlab interfaces, you do not have to check them out.

9 Development Distribution

In case you need the most up to date version of the Java CoG Kit, you can check this version out from the CVS. To do so you should follow these simple steps:

  1. register with the Java CoG Kit as described in Section 2.
  2. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit login
  3. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout src/cog
  4. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout src/jglobus
  5. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout src/matlab

In case you are not interested in looking at the jglobus or the matlab interfaces, you do not have to check them out.

9.1 CVS Navigation Table

To help the developers to inspect the code in the CVS directly, we refer to following table for hyperlinks to the appropriate contents.






Component API ChangesCVSBugzilla










cog-jglobus htmlhtml htmlhtml





Java CoG Kit 4htmlhtml htmlhtml





matlab html htmlhtml





10 Downloading the API Documentation

In case you like to download the API documentation we have provided an additional package for you:

However we simply recommend to use our online API documentation at

11 Webstart Component Install

Several applications from the Java CoG Kit 4_1_2  are available as Webstart applications as listed in Table 1. Additional experimental components are also available.


Table 1: A list of Java CoG Kit Webstart applications



 

PIC

Setup Wizard: allows easy set up of the Java CoG Kit through an intuitive graphical wizard [install]  to do 



PIC

Grid Proxy Init: allows to initialization of a Grid proxy through a visual component [install]  to do 



PIC

Job Submission Form: Form interface that helps the user to submit jobs to remote machine by filling out a form with the appropriate details, such as the remote machine name and the specification of the job to be executed on that machine [not yet available]



PIC

CoG Kit Desktop: allows users to interact with the Grid through a desktop. Dragging a job icon onto a machine icon will start the job on that machine. Users can monitor the jobs submitted to the machines and fetch the output generated [not yet available]  to do 



PIC

LDAP Browser/Editor: provides a user-friendly Windows Explorer-like interface to LDAP directories with tightly integrated browsing and editing capabilities. It is entirely written in Java with the help of the JFC (SwingSet) and JNDI class libraries. It connects to LDAP v2 and v3 servers. [install]



PIC

Graph Editor: allows the display and editing of hierarchical directed graphs. It supports dynamic properties, multiple targets such as Swing, PostScript, and HTML [install]  to do 



PIC

Workflow Visualizer: The workflow GUI is a graphical interface to the Java CoG Kit Karajan workflow engine. It can be used for features otherwise not available in the command line interface (such as setting breakpoints, interactive error handling, visualization, etc.) [install]  to do 




11.1 Summary

  1. register with the Java CoG Kit as described in Section 2.
  2. go to the Webstart page and install some of the programs

12 Configuration of the Java CoG Kit

Once you have downloaded the Java CoG Kit, you can proceed to set up the Java CoG Kit. The setup step is conducted with a simple wizard that guides you through the configuration process. In case you like to use a Grid using GSI security, you should obtain a certificate that allows you to get access to this Grid. We recommend that you place the certificate and the public key for the certificate authority in the .globus directory in your home directory. The keys for the certificate authorities should be placed in .globus/certificates. We assume that you have started a terminal window and have changed to the directory the contains the release 4_1_2.

Now you can start the setup routine by launching it from your command line as follows. For Linux and Unix say

./bin/cog-setup

and for Windows

.
bin
cog-setup.bat

Naturally, you can make this even easier if you include the location of the cog_4_1_2/bin directory into your path environment variable. Additionally, you should set the variable COG_HOME to the directory that contains your release. Next, call the command cog-setup directly without worrying about having to cd to the bin directory.

You will then be walked through a number of easy to follow steps.

References

[1]   G. von Laszewski, I. Foster, J. Gawor, and P. Lane, “A Java Commodity Grid Kit,” Concurrency and Computation: Practice and Experience, vol. 13, no. 8-9, pp. 643–662, 2001. [Online]. Available: http://www.mcs.anl.gov/~gregor/papers/vonLaszewski--cog-cpe-final.pdf

[2]   “Java CoG Kit Wiki,” 2004. [Online]. Available: http://www.cogkit.org/wiki

[3]   “Java CoG Kit Registration,” 2004. [Online]. Available: http://www.cogkit.org/register

Additional publications about the Java CoG Kit can be found as part of the vita of Gregor von Laszewski http://www-unix.mcs.anl.gov/~laszewsk/vita.pdf. Most documents are avalable online if you follow the links. In future we intend to provide this information without Gregors vita data.

If you need to cite the Java CoG Kit, please use [1].

Appendix

A Java CoG Kit Guides




Short Title Description Format



Guide A guide to help you finding out what guides have been written[PDF] [HTML]



Install A guide to the different ways of installing the Java CoG Kit [PDF] [HTML]



Commands A guide to the command line tools of the Java CoG Kit [PDF] [HTML]



Workflow A guide to the Gridant/Karajan Workflow [PDF] [HTML]



AbstractionsA guide to the Java CoG Kit abstractions API [PDF] [HTML]



JavaDoc The Java API documentation to the Java CoG Kit [HTML]



Coding A guide to the Coding rules for the Java CoG Kit [PDF] [HTML]



Overview A future guide that will be an overview to the Java CoG Kit [PDF] [HTML]



B Java CoG Kit Guides Under Construction

More guides are under development. The following guides are not yet completed, but are listed here to help us improving these guides. Please, explore them and send us e-mail about improvement suggestions. If you like to contribute a guide yourself, please contact gregor@mcs.anl.gov.




Short Title Description Format



MPI A preliminary guide to execute MPI programs on the TeraGrid and alike[PDF] [HTML]



Release ProcessA preliminary guide to document the release process [PDF] [HTML]



Guide A preliminary guide to document writing guides [PDF] [HTML]



Examples A preliminary guide to examples alike [PDF] [HTML]



C Available Downloads

First time users of the Java CoG Kit should read the “Guide to Installing the Java CoG Kit” [PDF] [HTML]. We hope that you will find this guide useful to decide which bundles you need. For the more experienced user, we provide the following table.

Binary Distributions
 
Source Distributions
 
API Documentation
 
Module List
 
CVS
 

D Availability of the Document

The newest version of this document can be downloaded by the developers from

  1. cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout manual/guide

It is not allowed to reproduce this document or the source. This documentation is copyrighted and is not distributed under the CoG Kit license.

E Bugs

This guide is constantly improved and your input is highly appreciated. Please report suggestion, errors, changes, and new sections or chapters through our Bugzilla system at

http://www-unix.globus.org/cog/contact/bugs/

F Administrative Contact

The Java CoG Kit project has been initiated and is managed by Gregor von Laszewski. To contact him, please use the information below.

Gregor von Laszewski
Argonne National Laboratory
Mathematics and Computer Science Division
9700 South Cass Avenue
Argonne, IL 60439
Phone:(630) 252 0472
Fax: (630) 252 1997
gregor@mcs.anl.gov

Index

Abstraction, 3
Administrative Contact, 14

Contact, 14

Documentation
    API, 10

Gridfaces, 3

Install
    binary
        full, 8
        partial, 8
    CVS
        development, 10
        release, 10
    source, 9

Providers, 3

Registration, 2

Task management, 3