ChemGPS-NP Web - Web Services API (REST)

This page documents the implementation of the REST web service architecture for Batchelor based on information from these document:

The REST web services is presented as a tree of URI's. Each URI has one or more associated HTTP action(s) (standard GET, POST, PUT or DELETE). All GET requests are non-modifying. An URI (a node resource) can be changed by using PUT or POST (add or modified) or DELETE (remove).

The HTTP status code is always 200. The response is always wrapped inside result tags, where the state attribute is either "success" or "failed". The content (the embedded message) is either an list of links or an object, where the object is the requested data, an status message or an error object.

<tns:result state="failed" type="error">
  <error>
    <code>3</code>
    <message>No such method</message>
  </error>
</tns:result>

The response encoding is either XML or FOA selectable by appending encode={xml|foa} to the request string. The default response encoding is XML. The XML encoded response is formalized by the XML Schema for REST responses. The FOA specification describes the FOA encoded response format.

The output format from a GET request on an URI is either an list (of links) or data (possibly multiple objects). The format is selected in two ways: either append format={list|data} to the request URI or append the format to the URI path.

/queue/all?format=data      /* get all jobs */
/queue/all/data             /* alternative way */

An modifying HTTP action (PUT, POST, DELETE) will either return a status message or data depending on the URI. Heres an example response for dequeue (removing) a job:

<tns:result state="success" type="status">
  <status>Removed job 1355</status>
</tns:result>

An link has possible action attributes like get, put, post and delete. The action attribute value describes the object returned by taking this action.

<tns:result state="success" type="link">
  <link xlink:href="/queue" get="link" put="job" />
  <link xlink:href="/result" get="link" />
    ...
</tns:result>

The XML snippet above tells us that the /queue URI supports GET and PUT, whereas the /result URI only accepts GET.

Schematic overview of the resources with theirs accepted actions (HTTP request method) on the right.

Node:                    Action:       Description:
------                   --------      -------------

root/                    GET           (the ws/rest service root)
+-- queue/               GET,PUT,POST  (get sort and filter, enqueue with PUT/POST(1))
|      +-- all/          GET,DELETE    (get or delete all objects)
|            +-- xxx/    GET,DELETE    (get or delete single job)
|      +-- sort/         GET           (2)
|            +-- xxx/    GET           (various sort options)
|      +-- filter/       GET           (2)
|            +-- xxx/    GET,DELETE    (get or delete all jobs matching filter)
+-- result/              GET           (list all job directories)
|      +-- dir/          GET           (list result files)
|            +-- <file>  GET           (get content of result file)
+-- watch/               POST          (watch jobs)
|      +-- <job>         GET           (get info about job)
+-- errors/              GET           (list all error types)
|      +-- <error>       GET           (get error object)
+-- suspend/             GET           (list suspendable jobs)
|      +-- <job>         GET,POST      (get info or suspend the job)
+-- resume/              GET           (list resumable jobs)
       +-- <job>         GET,POST      (get info or resume the job)

(1) Note that POST for enqueue new jobs is not stricly RESTful, but we allows it because not all web servers supports HTTP PUT (Apache does).

(2) Both sort and filter URI accepts an additional companion request parameter, i.e. 'queue/sort/jobid/data?filter=error' gives all error state jobs filtered on the job ID.

All tasks except for starting new jobs (enqueue) should be fairly obvious, so I will only outline the details on using PUT/POST and the /queue resource to start new jobs.

Because the indata might be arbitrary large, the data has to be uploaded thru PUT or POST. Encoding the data in the URL is on a typical system limited to 32kB, not to mention it goes against REST principles also.

curl_setopt($curl, CURLOPT_PUT, 1);
curl_setopt($curl, CURLOPT_INFILE, fopen($options->file, "r"));
curl_setopt($curl, CURLOPT_INFILESIZE, filesize($options->file));

This is how to do such an POST using PHP's cURL library. The data must be posted in a multipart/form-data named 'file'.

$post = array(
    'file' => sprintf("@%s", $options->file)  // Notice the '@'!
);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $post);

You can download the client side Java (tm) library batchelor-java. Both SOAP and REST is supported by the library. However, the REST implementation contains more features, including using FOA for stream oriented downloads of result files. The library is fully documented using javadoc comments.

The web service utility (utils/ws.php) can be used to browse the REST service. Start with: 'php ws.php --type=rest --params='' and then append the relative URI path in the params option. The default output is XML, but FOA output can be requested by appending 'encode=foa' to the request parameters.

Browsing the root requesting FOA encoded output requires that the method argument (a virtual method) is explicit set:

bash$> php ws.php --type=rest --params='root?encode=foa'

Enqueue a new job with data.txt as indata is done by putting (PUT) the file on the queue URI:

bash$> php ws.php --type=rest --file=data.txt --put --params='queue'

Watch jobs for completion. This variant includes sending the timestamp using POST.

bash$> php ws.php --type=rest --post='stamp=1241490725' --params='watch'

Batchelor - batch job queue manager

ChemGPS-NP Web - Web Services API (REST)