# Chirp Protocol Version 2

## Introduction

Chirp is a simple and lightweight remote I/O protocol with multiple
implementations. It is used in several distributed computing systems where
users wish to access data over the wide area using a Unix filesystem like
protocol.

This document lays out the underlying protocol so as to promote
interoperability between implementations, and permit additional implementations
by other parties.

At the time of writing, there are four independent implementations used for
both research and production:

* A standalone C server and client maintained by Douglas Thain at the University of Notre Dame. 
* A Java client implementation by Justin Wozniak used by the GEMS project at the University of Notre Dame. 
* A C server and Java client embedded in the Condor distributed processing system at the University of Wisconsin. 
* A C client implementation that is part of the ROOT I/O library used in the high energy physics community. 

## Authentication Protocol

Chirp is carried over a stream protocol such as TCP or Unix domain sockets. A
server waits for a client to connect by the appropriate mechanism, and then
waits for the client to identify itself. There are currently two methods of
authentication: cookie authentication and negotiated authentication.

### Cookie Authentication

In certain situations, the client may be provided in advance with a "cookie"
that provides expedited access to a server with a fixed identity. This method
is typically used within Condor when communicating with the embedded Chirp
proxy.

In this situation, the client is provided with a file named `chirp.config`
which contains the host and port of the server, and a cookie string. The
client must connect to the server and immediately send `cookie`, a space, the
cookie string, and a newline. If the response from the server is `0\n` then
the cookie is accepted, and the client may proceed to the main Chirp protocol
described below. Any other response indicates the cookie was rejected, and the
client must abort.

### Negotiated Authentication

In the general case, the client must prove its identity to the server. There
are several possible methods of authentication, and both client and server may
only accept some subset of them. The client drives the negotiation process by
simply sending the requested authentication type as a plain string followed by
a newline. If the server does not support the method, it responds `no\n` and
waits for the client to request another. If it does support the method, it
responds `yes\n` and then performs that method. At the time of writing, the
supported authentication methods are `hostname`, `unix`, `kerberos`, and
`globus`.

In this example, a client first requests `kerberos` authentication, and then
`hostname`:

```
client: kerberos
server: no
client: unix
server: yes
```

#### Hostname Authentication

This method of authentication simply asks that the server identify by the
client by the name of the host it is calling from. The server responds by
performing a remote domain name lookup on the caller's IP address. If this
succeeds, the server responds `yes\n`, otherwise `no\n` and fails (see below).
If the client is authorized to connect, the server sends an additional `yes\n`
otherwise `no\n` and fails.

#### Unix Authentication

This method of authentication causes the server the challenge the client to
prove its identity by creating a file on the local file system. The server
transmits the name of a non-existent file in a writable directory. The client
must then attempt to create it. If successful, the client should respond
`yes\n` otherwise `no\n` and fails. The server then checks that the file was
actually created, and if satisfied responds `yes\n` otherwise `no\n` and
fails.

#### Kerberos Authentication

This method is performed by invoking the standard Kerberos 5 libraries to
authenticate in the usual manner, and is defined by that standard.

#### Globus Authentication

This method is performed by invoking the Secure Sockets Layer to authenticate
using Globus credentials and is defined by that standard.

### Completing Authentication

If the authentication method succeeds, then the server will send two lines to
the client, giving the successful authentication method and the client's
identity. The client may then proceed to the main protocol below. If the
method fails, then the server and client return to negotiating a method.

## Chirp Protocol

A request consists of an LF-terminated line possibly followed by binary data.
At a minimum, a Chirp server must accept requests of 1024 characters. It may
accept longer lines. If a line exceeds a server's internal maximum, it must
gracefully consume the excess characters and return a TOO_BIG error code,
defined below. Certain requests are immediately followed by binary data. The
number of bytes is dependent on the semantics of the request, also described
below.

A request is parsed into words separated by any amount of white space
consisting of spaces and tabs. The first word of the request is called the
"command" and the rest are called "arguments."

Words fall into two categories: strings and decimals. A string is any
arbitrary sequence of characters, with whitespaces and unprintables encoding
according to RFC 2396. (Very briefly, RFC 2396 says that whitespace must be
encoding using a percent sign and two hex digits: `ab cd` becomes `ab%20cd`.)
A decimal is any sequence of the digits 0-9, optionally prefixed by a single +
or -.

At the protocol level, words have no maximum size. They may stretch on to fill
any number of bytes. Of course, each implementation has limits on concepts
such a file name lengths and integer sizes. If a receiver cannot parse, store,
or execute a word contained in a request, it must gracefully consume the
excess characters and return a TOO_BIG error response, defined below.

A response consists of an LF-terminated line, bearing an ASCII integer. A
valid response may also contain whitespace and additional optional material
following the response. If the response is greater than or equal to zero, the
response indicates the operation succeeded. If negative, the operation failed,
and the exact value tells why:

| | |
|-|-|
-1   |  NOT_AUTHENTICATED  |  The client has not authenticated its identity.
-2   |  NOT_AUTHORIZED     |  The client is not authorized to perform that action.
-3   |  DOESNT_EXIST       |  There is no object by that name.
-4   |  ALREADY_EXISTS     |  There is already an object by that name.
-5   |  TOO_BIG            |  That request is too big to execute.
-6   |  NO_SPACE           |  There is not enough space to store that.
-7   |  NO_MEMORY          |  The server is out of memory.
-8   |  INVALID_REQUEST    |  The form of the request is invalid.
-9   |  TOO_MANY_OPEN      |  There are too many resources in use.
-10  |  BUSY               |  That object is in use by someone else.
-11  |  TRY_AGAIN          |  A temporary condition prevented the request.
-12  |  BAD_FD             |  The file descriptor requested is invalid.
-13  |  IS_DIR             |  A file-only operation was attempted on a directory.
-14  |  NOT_DIR            |  A directory operation was attempted on a file.
-15  |  NOT_EMPTY          |  A directory cannot be removed because it is not empty.
-16  |  CROSS_DEVICE_LINK  |  A hard link was attempted across devices.
-17  |  OFFLINE            |  The requested resource is temporarily not available.
-127 |  UNKNOWN            |  An unknown error occurred.


!!! note
    Negative values beyond -17 are reserved for future expansion. The receipt
    of such a values should be interpreted in the same way as UNKNOWN.

## Chirp Commands

Following are the available commands. Each argument to a command is specific
Here are the available commands that are standardized. Note that each
implementation may support some additional commands which are not (yet)
documented because they are still experimental. Each argument to a command is
specified with a type and a name.

***
```text
open (string:name) (string:flags) (decimal:mode)
```

Open the file named "name" "mode" is interpreted in the same
manner as a POSIX file mode. Note that the mode is printed in decimal
representation, although the UNIX custom is to use octal. For example, the
octal mode 0755, representing readable and executable by everyone would be
printed as 493. "flags" may contain any of these characters which affect the
nature of the call:

| | |
|-|-|
r | open for reading 
w | open for writing 
a | force all writes to append 
t | truncate before use 
c | create if it doesn't exist 
x | fail if 'c' is given and the file already exists

The open command returns an integer file description which may be used with
later calls referring to open files. The implementation may optionally return
file metadata in the same form as the stat command, immediately following the
result. A client implementation must be prepared for both forms of result.

On failure, returns a negative value in the response. Note that a file
descriptor returned by open only has meaning within the current connection
between the client and the server. If the connection is lost, the server will
implicitly close all open files. A file descriptor opened in one connection has
no meaning in another connection.

***
```text
close (decimal:fd)
```

Complete access to this file descriptor. If the connection between a client an server is lost, all files are implicitly closed.

***
```text
read (decimal:fd) (decimal:length)
```

Read up to __length__ bytes from the file descriptor __fd__. If successful, the
server may respond with any value between zero and __length__. Immediately
following the response will be binary data of exactly as many bytes indicated
in the response. If zero, the end of the file has been reached. If any other
value is given, no assumptions about the file size may be made.

***
```text
pread (decimal:fd) (decimal:length) (decimal:offset)
```

Read up to __length__ bytes from the file descriptor __fd__, starting at __offset__. Return value is identical to that of read.

***
```text
read (decimal:fd) (decimal:length) (decimal:offset) (decimal:stride_length) (decimal:stride_skip)
```

Read up to __length__ bytes from the file descriptor __fd__, starting at
__offset__, retrieving __stride_length__ bytes every __stride_skip__ bytes.
Return value is identical to that of read.

***
```text
write (decimal:fd) (decimal:length)
```

Write up to __length__ bytes to the file descriptor __fd.__ This request should
be immediately followed by __length__ binary bytes. If successful, may return
any value between 0 and __length,__ indicating the number of bytes actually
accepted for writing. An efficient server should avoid accepting fewer bytes
than requested, but the client must be prepared for this possibility.

***
```text
pwrite (decimal:fd) (decimal:length) (decimal:offset)
```

Write up to __length__ bytes to the file descriptor __fd__ at offset
__offset__. Return value is identical to that of write.

***
```text
swrite (decimal:fd) (decimal:length) (decimal:offset) (decimal:stride_length) (decimal:stride_skip)
```

Write up to __length__ bytes to the file descriptor __fd__, starting at
__offset__, writing __stride_length__ bytes every __stride_skip__ bytes. Return
value is identical to that of write.

***
```text
fstat (decimal:fd)
```

Get metadata.

***
```text
fsync (decimal:fd)
```

Block until all uncommitted changes to this file descriptor have been written
to stable storage.

***
```text
lseek (decimal:fd) (decimal:offset) (decimal:whence)
```

Move the current seek pointer of __fd__ by __offset__ bytes from the base given
by __whence.__ __whence__ may be: 

| | |
|-|-|
| 0 | from the beginning of the file 
| 1 | from the current seek position 
| 2 | from the end of the file.

Returns the current seek position.


***
```text
rename (string:oldpath) (string:newpath)
```

Rename the file __oldpath__ to be renamed to __newpath__.

***
```text
unlink (string:path)
```

Delete the file named __path__.

***
```text
rmdir (string:path)
```

Delete a directory by this name.

***
```text
rmall (string:path)
```

Recursively delete an entire directory.

***
```text
mkdir (string:name) (decimal:mode)
```

Create a new directory by this name. __mode__ is interpreted in the same manner
as a POSIX file mode. Note that mode is expressed in decimal rather than octal
form.

***
```text
fstat (decimal:fd)
```

Get metadata describing an open file. If the response line indicates success, it is followed by a second line of thirteen integer values, indicating the following values: 

| | |
|-|-|
| st_dev     |  Device number. 
| st_ino     |  Inode number 
| st_mode    |  Mode bits. 
| st_nlink   |  Number of hard links. 
| st_uid     |  User ID of the file's owner. 
| st_gid     |  Group ID of the file. 
| st_rdev    |  Device number, if this file represents a device. 
| st_size    |  Size of the file in bytes. 
| st_blksize |  Recommended transfer block size for accessing this file. 
| st_blocks  |  Number of physical blocks consumed by this file. 
| st_atime   |  Last time file was accessed in Unix time() format. 
| st_mtime   |  Last time file data was modified in Unix time() format. 
| st_ctime   |  Last time the inode was changed, in Unix time() format.

!!! note
    Note that not all fields may have meaning to all implementations. For
    example, in the standalone Chirp server, st_uid has no bearing on access
    control, and the user should employ setacl and getacl instead.


***
```text
fstatfs (string:path)
```

Get filesystem metadata for an open file. If the response line indicates
success, it is followed by a second line of seven decimals with the following
interpretation: 

| | |
|-|-|
| f_type     |  The integer type of the filesystem. 
| f_blocks   |  The total number of blocks in the filesystem. 
| f_bavail   |  The number of blocks available to an ordinary user. 
| f_bsize    |  The size in bytes of a block. 
| f_bfree    |  The number of blocks free. 
| f_files    |  The maximum number of files (inodes) on the filesystem. 
| f_ffree    |  The number of files (inodes) currently in use.

***
```text
fchown (decimal:fd) (decimal:uid) (decimal:gid)
```

Change the ownership of an open file to the UID and GID indicated.


***
```text
fchmod (decimal:fd) (decimal:mode)
```

Change the Unix mode bits on an open file.


***
```text
ftruncate (decimal:fd) (decimal:length)
```

Truncate an open file.


***
```text
getfile (string:path)
```

Retrieves an entire file efficiently. A positive response indicates the number
of bytes in the file, and is followed by exactly that many bytes of binary data
which are the file's contents.

***
```text
putfile (string:path) (decimal:mode) (decimal:length)
```

Stores an entire file efficiently. The client first sends the request line
indicating the file name, the desired mode bits, and the length in bytes. The
response indicates whether the client may proceed. If it indicates success, the
client must write exactly __length__ bytes of data, which are the file's
contents. If the response indicates failure, the client must not send any
additional data.

***
```text
getlongdir (string:path)
```

Lists a directory and all metadata. If the response indicates success, it will
be followed by a series of lines, alternating the name of a directory entry
with its metadata in the same form as returned by fstat. The end of the list is
indicated by a single blank line.


***
```text
getdir (string:path)
```

Lists a directory. If the response indicates success, it will be followed by a
series of lines indicating the name of each directory entry. The end of the
list is indicated by a single blank line.


***
```text
getacl (string:path)
```

Get an access control list. If the response indicates success, it will be
followed by a series of lines indicating each entry of the access control list.
The end of the list is indicated by a single blank line.


***
```text
setacl (string:path) (string:subject) (string:rights)
```

Modify an access control list. On an object identified by path, set rights
rights for a given subject. The string __-__ (a single hyphen) may be used to
indicate no rights.


***
```text
whoami
```

Get the user's current identity with respect to this server. If the response is
greater than zero, it is followed by exactly that many bytes in data,
containing the user's identity.


***
```text
whoareyou (string:rhost)
```

Get the server's current identity with respect to a remote host. If the
response is greater than zero, it is followed by exactly that many bytes in
data, containing the server's identity.

***
```text
link (string:oldpath) (string:newpath)
```

Create a hard link from newpath to oldpath.


***
```text
symlink (string:oldpath) (string:newpath)
```

Create a symlink from newpath to oldpath.


***
```text
readlink (string:path)
```

Read the contents of a symbolic link. If the response is greater than zero, it
is followed by exactly that many bytes in data, containing the contents of the
link.


***
```text
mkdir (string:path) (decimal:mode)
```

Create a new directory with the given Unix mode bits.


***
```text
stat (string:path)
```

Get file status. If the response indicates success, it is followed by a second
line in the same format as the command fstat. If the pathname represents a
symbolic link, this command examines the target of the link.


***
```text
lstat (string:path)
```

Get file status. If the response indicates success, it is followed by a second
line in the same format as the command fstat. If the pathname represents a
symbolic link, this command examines the link itself.


***
```text
statfs (string:path)
```

Get filesystem status. If the response indicates success, it is followed by a
second line in the same format as the command fstatfs.


***
```text
access (string:path) (decimal:mode)
```

Check access permissions. Returns success if the user may access the file
according to the specified mode. The __mode__ may be any of the values logical-or'd together: 

| | |
|-|-|
| 0   |  Test for existence of the file. (F_OK)  
| 1   |  Test if the file is executable. (X_OK)  
| 2   |  Test if the file is readable.   (W_OK)  
| 4   |  Test if the file is executable. (R_OK)  

***
```text
chmod (string:path) (decimal:mode)
```

Change the Unix mode bits on a given path. ([Note 1.](#note-1))


***
```text
chown (string:path) (decimal:uid) (decimal:gid)
```

Change the Unix UID or GID on a given path. If the path is a symbolic link,
change its target. ([Note 1.](#note-1))

***
```text
lchown (string:path) (decimal:uid) (decimal:gid)
```

Change the Unix UID or GID on a given path. If the path is a symbolic link,
change the link. ([Note 1.](#note-1))

***
```text
truncate (string:path) (decimal:length)
```

Truncate a file to a given number of bytes.


***
```text
utime (string:path) (decimal:actime) (decimal:mtime)
```

Change the access and modification times of a file.


***
```text
md5 (string:path)
```

Checksum a remote file using the MD5 message digest algorithm. If successful,
the response will be 16, and will be followed by 16 bytes of data representing
the checksum in binary form.


***
```text
thirdput (string:path) (string:remotehost) (string:remotepath)
```

Direct the server to transfer the path to a remote host and remote path. If the
indicated path is a directory, it will be transferred recursively, preserving
metadata such as access control lists.


***
```text
mkalloc (string:path) (decimal:size) (decimal:mode)
```

Create a new space allocation at the given path that can contain __size__ bytes of data and has initial mode of __mode__.

***
```text
lsalloc (string:path)
```

List the space allocation state on a directory. If the response indicates
success, it is followed by a second line which gives the path of the containing
allocation, the total size of the allocation, and the 


## Notes

### Note 1

The standalone Chirp server ignores the traditional Unix mode bits when
performing access control. Calls such as fchmod, chmod, chown, and fchown are
essentially ignored, and the caller should employ setacl and getacl instead.


**Last edited: August 2019**

