ITQC/gls_docs
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Readme

Browse Source
This commit is contained in:
Fabian Märki
2016-04-15 15:06:20 +02:00
parent cfb327006f
commit ef9cad3b9f
1 changed files with 122 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files

241
Readme.md
View File

@ -102,86 +102,11 @@ curl -H "Content-Type: application/json" -X POST -d '{"regex": "AMPLT|PHASE"}' h
] ]
``` ```
<a name="query_range"/>
## Query Range
Queries are applied to a range. The following types of ranges ranges are supported.
### By Pulse-Id
```json
"range":{
"startPulseId":0,
"endPulseId":100
}
```
- **startPulseId**: The start pulse-id of the range request.
- **endPulseId**: The end pulse-id of the range request.
### By Date
```json
"range":{
"startDate":"2015-08-06T18:00:00.000",
"endDate":"2015-08-06T18:59:59.999",
}
```
- **startDate**: The start date of the time range (ISO8601 format (YYYY-MM-DDThh:mm:ss.sTZD e.g. 1997-07-16T19:20:30.123+02:00 (omitting +02:00 falls back to the local time zone)). Additionally, ISO8601 format up to nano seconds is also supported (e.g. 1997-07-16T19:20:30.123456789+02:00).
- **endDate**: The end date of the time range.
### By Time
```json
"range":{
"startSeconds":"0.0",
"endSeconds":"1.000999999"
}
```
- **startSeconds**: The start time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds.
- **endSeconds**: The end time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds.
<a name="query_channel_names"/>
## Query Channel Names
The simplest way to define channels is to use an array of channel name Strings.
```json
"channels":[
"Channel_02",
"Channel_04"
]
```
The query interface will automatically select the backend which contains the channel (e.g., *databuffer* for *Channel_02* and *archiverappliance* for *Channel_04*. In case name clashes exist, the query interface will use following order of priority: *databuffer* and *archiverappliance*.
It is also possible to explicitly define the backend to overcome name clashes.
```json
"channels":[
{
"name":"Channel_01",
"backend":"archiverappliance"
},
{
"name":"Channel_01",
"backend":"databuffer"
}
]
```
<a name="query_data"/> <a name="query_data"/>
## Query Data ## Query Data
### Query request endpoint ### Request
``` ```
GET http://<host>:<port>/query GET http://<host>:<port>/query
@ -201,21 +126,98 @@ The following attributes can be specified:
- **binSize**: Activates data binning. Specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies). - **binSize**: Activates data binning. Specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies).
- **aggregations**: Activates data aggregation. Array of requested aggregations (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Aggregation.java) for possible values). These values will be added to the *data* array response. - **aggregations**: Activates data aggregation. Array of requested aggregations (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Aggregation.java) for possible values). These values will be added to the *data* array response.
- **aggregationType**: Specifies the type of aggregation (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/AggregationType.java)). The default type is *value* aggregation (e.g., sum([1,2,3])=6). Alternatively, it is possible to define *index* aggregation for multiple arrays in combination with binning (e.g., sum([1,2,3], [3,2,1]) = [4,4,4]). - **aggregationType**: Specifies the type of aggregation (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/AggregationType.java)). The default type is *value* aggregation (e.g., sum([1,2,3])=6). Alternatively, it is possible to define *index* aggregation for multiple arrays in combination with binning (e.g., sum([1,2,3], [3,2,1]) = [4,4,4]).
- **compression**: Defines the compression algorithm to use, default value is **none**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Compression.java))
- **responseFormat**: Specifies the format the response of the requested data is in, either in JSON or CSV format, default value **JSON**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/ResponseFormat.java)) - **responseFormat**: Specifies the format the response of the requested data is in, either in JSON or CSV format, default value **JSON**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/ResponseFormat.java))
- **compression**: Defines the compression algorithm to use, default value is **none**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Compression.java))
### `compression`: compression of data can be enabled <a name="query_channel_names"/>
By default, no data is compressed when transferred from the server to the client. However, compression can be enabled by setting the `compression` attribute to a value other than `none`, i.e. to `gzip` or `deflate`. ### Define Channel Names
If compression is enabled, we have to tell `curl` that the data is compressed so that it is being decompressed automatically. `curl` decompresses the response when the `--compressed` parameter is set. The simplest way to define channels is to use an array of channel name Strings.
### `responseFormat`: data is in JSON by default ```json
"channels":[
"Channel_02",
"Channel_04"
]
```
The query interface will automatically select the backend which contains the channel (e.g., *databuffer* for *Channel_02* and *archiverappliance* for *Channel_04*). In case name clashes exist, the query interface will use following order of priority: *databuffer* and *archiverappliance*.
It is also possible to explicitly define the backend to overcome name clashes.
```json
"channels":[
{
"name":"Channel_01",
"backend":"archiverappliance"
},
{
"name":"Channel_01",
"backend":"databuffer"
}
]
```
<a name="query_range"/>
### Define Query Range
Queries are applied to a range. The following types of ranges are supported.
#### By Pulse-Id
```json
"range":{
"startPulseId":0,
"endPulseId":100
}
```
- **startPulseId**: The start pulse-id of the range request.
- **endPulseId**: The end pulse-id of the range request.
#### By Date
```json
"range":{
"startDate":"2015-08-06T18:00:00.000",
"endDate":"2015-08-06T18:59:59.999",
}
```
- **startDate**: The start date of the time range in the ISO8601 format (such as 1997-07-16T19:20:30.123+02:00 or 1997-07-16T19:20:30.123456789+02:00 (omitting +02:00 falls back to the local time zone)).
- **endDate**: The end date of the time range.
#### By Time
```json
"range":{
"startSeconds":"0.0",
"endSeconds":"1.000999999"
}
```
- **startSeconds**: The start time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds.
- **endSeconds**: The end time of the range in seconds.
### Define `responseFormat`
Responses can be formatted as CSV or JSON using the `responseFormat` field. The returned data is JSON-formatted by default. Responses can be formatted as CSV or JSON using the `responseFormat` field. The returned data is JSON-formatted by default.
CSV export does not support `index` and `extrema` aggregations. CSV export does not support `index` and `extrema` aggregations.
### Define `compression`
By default, data is not compressed when transferred from the server to the client. However, compression can be enabled by setting the `compression` attribute to a value other than `none`, i.e. to `gzip` or `deflate`.
If compression is enabled, you have to tell `curl` that the data is compressed so that it is being decompressed automatically. `curl` decompresses the response when the `--compressed` parameter is set.
### Example Queries ### Example Queries
The following examples build on waveform data (see below). They also work for scalars (consider it as a waveform of length = 1) and images (waveform of length = dimX * dimY). The following examples build on waveform data (see below). They also work for scalars (consider it as a waveform of length = 1) and images (waveform of length = dimX * dimY).
@ -270,7 +272,7 @@ The following examples build on waveform data (see below). They also work for sc
### Query Examples ### Query Examples
##### Query by Pulse-Id Range #### Query by Pulse-Id Range
```json ```json
{ {
@ -284,19 +286,19 @@ The following examples build on waveform data (see below). They also work for sc
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
See JSON representation of the data above. See JSON representation of the data above.
##### Query by Time Range #### Query by Time Range
```json ```json
{ {
@ -310,19 +312,19 @@ See JSON representation of the data above.
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
See JSON representation of the data above. See JSON representation of the data above.
##### Query by Date Range #### Query by Date Range
```json ```json
{ {
@ -338,19 +340,19 @@ See JSON representation of the data above.
Supported format is ISO8601 *YYYY-MM-DDThh:mm:ss.sTZD* (e.g. *1997-07-16T19:20:30.475+02:00*). Supported format is ISO8601 *YYYY-MM-DDThh:mm:ss.sTZD* (e.g. *1997-07-16T19:20:30.475+02:00*).
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"range":{"startDate":"1970-01-01T01:00:00.000","endDate":"1970-01-01T01:00:00.030"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"range":{"startDate":"1970-01-01T01:00:00.000","endDate":"1970-01-01T01:00:00.030"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
See JSON representation of the data above. See JSON representation of the data above.
##### Querying Archiver Appliance #### Querying Archiver Appliance
```json ```json
{ {
@ -374,17 +376,18 @@ See JSON representation of the data above.
Archiver Appliance supports queries by *time range* and *date range* only (as it has no notion about pulse-id). Archiver Appliance supports queries by *time range* and *date range* only (as it has no notion about pulse-id).
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":[{"name": "Channel_01","backend":"archiverappliance"}]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":[{"name": "Channel_01","backend":"archiverappliance"}]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
See JSON representation of the data above. See JSON representation of the data above.
##### Query using compression
#### Query using compression
```json ```json
{ {
@ -414,7 +417,7 @@ or `deflate` can be used too:
} }
``` ```
###### Command (gzip) ##### Command (gzip)
The `curl` command has a `--compressed` option to decompress data automatically. The `curl` command has a `--compressed` option to decompress data automatically.
@ -422,7 +425,7 @@ The `curl` command has a `--compressed` option to decompress data automatically.
curl --compressed -H "Content-Type: application/json" -X POST -d '{"compression":"gzip","range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl --compressed -H "Content-Type: application/json" -X POST -d '{"compression":"gzip","range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
##### Query setting CSV response format #### Query setting CSV response format
```json ```json
{ {
@ -449,13 +452,13 @@ curl --compressed -H "Content-Type: application/json" -X POST -d '{"compression"
It is possible to request the time in seconds (since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds - using fields *globalSeconds* and *iocSeconds*) or in milliseconds (since January 1, 1970 (the JAVA epoch) - using fields *globalMillis* and *iocMillis*) It is possible to request the time in seconds (since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds - using fields *globalSeconds* and *iocSeconds*) or in milliseconds (since January 1, 1970 (the JAVA epoch) - using fields *globalMillis* and *iocMillis*)
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"responseFormat":"csv","range":{"startPulseId":0,"endPulseId":4},"channels":["channel1","channel2"],"fields":["channel","pulseId","iocSeconds","globalSeconds","shape","eventCount","value"]}' http://data-api.psi.ch/sf/query curl -H "Content-Type: application/json" -X POST -d '{"responseFormat":"csv","range":{"startPulseId":0,"endPulseId":4},"channels":["channel1","channel2"],"fields":["channel","pulseId","iocSeconds","globalSeconds","shape","eventCount","value"]}' http://data-api.psi.ch/sf/query
``` ```
###### Response ##### Response
The response is in CSV. The response is in CSV.
@ -473,7 +476,7 @@ testChannel2;3;0.030000000;0.030000000;[1];1;3
testChannel2;4;0.040000000;0.040000000;[1];1;4 testChannel2;4;0.040000000;0.040000000;[1];1;4
``` ```
##### Querying for Specific Fields #### Querying for Specific Fields
Allows for server side optimizations since not all data needs to be retrieved. Allows for server side optimizations since not all data needs to be retrieved.
@ -490,13 +493,13 @@ Allows for server side optimizations since not all data needs to be retrieved.
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -524,7 +527,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","valu
] ]
``` ```
##### Data Ordering #### Data Ordering
```json ```json
{ {
@ -543,13 +546,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","valu
Use **none** in case ordering does not matter (allows for server side optimizations). Use **none** in case ordering does not matter (allows for server side optimizations).
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -579,7 +582,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields
``` ```
##### Value Aggregation #### Value Aggregation
```json ```json
{ {
@ -596,13 +599,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -652,7 +655,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
![Value Aggregation](doc/images/Value_Aggregation.png) ![Value Aggregation](doc/images/Value_Aggregation.png)
##### Value Aggregation with Binning (nrOfBins) #### Value Aggregation with Binning (nrOfBins)
```json ```json
{ {
@ -670,13 +673,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":2,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":2,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -709,7 +712,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
![Value Aggregation with Binning](doc/images/Value_Binning_NrOfBins.png) ![Value Aggregation with Binning](doc/images/Value_Binning_NrOfBins.png)
##### Value Aggregation with Binning (binSize) #### Value Aggregation with Binning (binSize)
**binSize** specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies). **binSize** specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies).
@ -729,13 +732,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"binSize":10,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["globalMillis","value"],"range":{"startSeconds":"0.0","endSeconds":"0.030000000"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"binSize":10,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["globalMillis","value"],"range":{"startSeconds":"0.0","endSeconds":"0.030000000"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -768,7 +771,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
![Value Aggregation with Binning](doc/images/Value_Binning_BinSize.png) ![Value Aggregation with Binning](doc/images/Value_Binning_BinSize.png)
##### Index Aggregation #### Index Aggregation
```json ```json
{ {
@ -786,13 +789,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregationType":"index","aggregations":["min","max","mean","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregationType":"index","aggregations":["min","max","mean","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [
@ -838,7 +841,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregation
![Index Aggregation with Binning](doc/images/Index_Binning.png) ![Index Aggregation with Binning](doc/images/Index_Binning.png)
##### Extrema Search #### Extrema Search
```json ```json
{ {
@ -855,13 +858,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregation
} }
``` ```
###### Command ##### Command
```bash ```bash
curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"extrema","aggregations":["min","max","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"extrema","aggregations":["min","max","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool
``` ```
###### Response ##### Response
```json ```json
[ [