679 lines
16 KiB
Markdown
679 lines
16 KiB
Markdown
#Overview
|
|
|
|
This project provides a REST interface to execute queries on the databuffer.
|
|
|
|
# Requirements
|
|
|
|
This project requires Java 8 or greater.
|
|
|
|
# Deployment
|
|
|
|
Use the instructions provided by [ch.psi.daq.install](https://github.psi.ch/projects/ST/repos/ch.psi.daq.install/browse#query_rest) to install the application on a server.
|
|
|
|
## Application Properties
|
|
|
|
Following files define and describe application properties:
|
|
|
|
- [Cassandra](https://github.psi.ch/projects/ST/repos/ch.psi.daq.cassandra/browse/src/main/resources/cassandra.properties) specific properties.
|
|
- [Query](https://github.psi.ch/projects/ST/repos/ch.psi.daq.dispatcher/browse/src/main/resources/query.properties) specific properties..
|
|
- [Query REST](https://github.psi.ch/projects/ST/repos/ch.psi.daq.queryrest/browse/src/main/resources/queryrest.properties) specific properties.
|
|
|
|
It is possible to overwrite properties by defining new values in `${HOME}/.config/daq/queryrest.properties`
|
|
|
|
<a name="rest_interface"/>
|
|
|
|
# REST Interface
|
|
|
|
<a name="query_channel_names"/>
|
|
|
|
## Query Channel Names
|
|
|
|
### Request
|
|
|
|
```
|
|
POST http://<host>:<port>/channels
|
|
```
|
|
|
|
#### Data
|
|
|
|
```json
|
|
{"regex": "TRFCA|TRFCB", "dbMode": "databuffer"}
|
|
```
|
|
|
|
##### Explanation
|
|
|
|
- **regex**: Reqular expression used to filter channel names (default: no filtering). Filtering is done using JAVA's [Pattern](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html), more precisely [Matcher.find()](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Matcher.html#find--)).
|
|
- **dbMode**: Defines the database to access (values: **databuffer**|archiverappliance)
|
|
|
|
|
|
### Example
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"regex": "TRFCA|TRFCB"}' http://sf-nube-14.psi.ch:8080/channels
|
|
```
|
|
|
|
<a name="query_data"/>
|
|
|
|
## Query Data
|
|
|
|
### Request
|
|
|
|
```
|
|
GET http://<host>:<port>/query
|
|
```
|
|
|
|
#### Data
|
|
|
|
A request is performed using JSON. The JSON query defines the channels to be queried, the range, and how the data should be aggregated (this is optional but highly recommended).
|
|
|
|
There exist following fields:
|
|
|
|
- **channels**: Array of channel names to be queried.
|
|
- **startPulseId** and **endPulseId** : A pulse-id range request with start and end pulse-id.
|
|
- **startMillis/[startNanos]** and **endMillis/[endNanos]**: A time range request with start and end milliseconds since January 1, 1970 (the UNIX/JAVA epoch) and optionally supplemented with the nanosecond offset to the milliseconds (range [0..999999]).
|
|
- **startDate/[startNanos]** and **endDate/[endNanos]**: A time range request with start and end date (format yyyy/MM/dd HH:mm:ss.SSS or dd.MM.yyyy HH:mm:ss.SSS) and optionally supplemented with the nanosecond offset to the milliseconds (range [0..999999]).
|
|
- **ordering**: The ordering of the data (see [here](https://github.psi.ch/projects/ST/repos/ch.psi.daq.common/browse/src/main/java/ch/psi/daq/common/ordering/Ordering.java) for possible values).
|
|
- **fields**: The requested fields (see [here](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/QueryField.java) for possible values).
|
|
- **nrOfBins**: Activates data binning. Specifies the number of bins the pulse/time range should be devided into.
|
|
- **binSize**: Activates data binning. Specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries.
|
|
- **aggregations**: Activates data aggregation. Array of requested aggregations (see [here](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/Aggregation.java) for possible values). These values will be added to the *data* array response.
|
|
- **aggregationType**: Specifies the type of aggregation (see [here](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/AggregationType.java)). The default type is *value* aggregation (e.g., sum([1,2,3])=6). Alternatively, it is possible to define *index* aggregation for multiple arrays in combination with binning (e.g., sum([1,2,3], [3,2,1]) = [4,4,4]).
|
|
- **aggregateChannels**: Specifies whether the data of the requested channels should be combined together using the defined aggregation (values: true|**false**)
|
|
- **dbMode**: Defines the database to access (values: **databuffer**|archiverappliance)
|
|
|
|
|
|
### Example
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"channels":["channel1","channel2"],"startPulseId":0,"endPulseId":4}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
### Response example
|
|
|
|
The response is in JSON.
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"channel1",
|
|
"data":[
|
|
{
|
|
"pulseId":0,
|
|
"iocMillis":0,
|
|
"iocNanos":0,
|
|
"globalMillis":0,
|
|
"globalNanos":0,
|
|
"value":0
|
|
},
|
|
{
|
|
"pulseId":2,
|
|
"iocMillis":2,
|
|
"iocNanos":2,
|
|
"globalMillis":2,
|
|
"globalNanos":2,
|
|
"value":2
|
|
},
|
|
{
|
|
"pulseId":4,
|
|
"iocMillis":4,
|
|
"iocNanos":4,
|
|
"globalMillis":4,
|
|
"globalNanos":4,
|
|
"value":4
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"channel":"channel2",
|
|
"data":[
|
|
{
|
|
"pulseId":1,
|
|
"iocMillis":1,
|
|
"iocNanos":1,
|
|
"globalMillis":1,
|
|
"globalNanos":1,
|
|
"value":1
|
|
},
|
|
{
|
|
"pulseId":3,
|
|
"iocMillis":3,
|
|
"iocNanos":3,
|
|
"globalMillis":3,
|
|
"globalNanos":3,
|
|
"value":3
|
|
}
|
|
]
|
|
}
|
|
]
|
|
```
|
|
|
|
### Example Queries
|
|
|
|
Following examples build on a waveform data (see below). They also work for scalars (consider it as a waveform of length = 1) and images (waveform of length = dimX * dimY).
|
|
|
|
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"iocMillis":0,
|
|
"iocNanos":0,
|
|
"pulseId":0,
|
|
"globalMillis":0,
|
|
"globalNanos":0,
|
|
"shape":[
|
|
4
|
|
],
|
|
"value":[1,2,3,4]
|
|
},
|
|
{
|
|
"iocMillis":10,
|
|
"iocNanos":0,
|
|
"pulseId":1,
|
|
"globalMillis":10,
|
|
"globalNanos":0,
|
|
"shape":[
|
|
4
|
|
],
|
|
"value":[2,3,4,5]
|
|
},
|
|
{
|
|
"iocMillis":20,
|
|
"iocNanos":0,
|
|
"pulseId":2,
|
|
"globalMillis":20,
|
|
"globalNanos":0,
|
|
"shape":[
|
|
4
|
|
],
|
|
"value":[3,4,5,6]
|
|
},
|
|
{
|
|
"iocMillis":30,
|
|
"iocNanos":0,
|
|
"pulseId":3,
|
|
"globalMillis":30,
|
|
"globalNanos":0,
|
|
"shape":[
|
|
4
|
|
],
|
|
"value":[4,5,6,7]
|
|
}
|
|
]
|
|
}
|
|
]
|
|
```
|
|
|
|
### Query Examples
|
|
|
|
##### Query by Pulse-Id Range
|
|
|
|
```json
|
|
{
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
See JSON representation of the data above.
|
|
|
|
|
|
|
|
##### Query by Time Range
|
|
|
|
```json
|
|
{
|
|
"startMillis":0,
|
|
"startNanos":0,
|
|
"endMillis":30,
|
|
"endNanos":999999,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"startMillis":0,"startNanos":0,"endMillis":30,"endNanos":999999,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
See JSON representation of the data above.
|
|
|
|
|
|
|
|
##### Query by Date Range
|
|
|
|
```json
|
|
{
|
|
"startDate":"1970/01/01 01:00:00.000",
|
|
"startNanos":0,
|
|
"endDate":"1970/01/01 01:00:00.030",
|
|
"endNanos":999999,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
Supported formats: *yyyy/MM/dd HH:mm:ss.SSS* and *dd.MM.yyyy HH:mm:ss.SSS*
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"startDate":"1970/01/01 01:00:00.000","startNanos":0,"endDate":"1970/01/01 01:00:00.030","endNanos":999999,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
See JSON representation of the data above.
|
|
|
|
|
|
|
|
##### Querying Archiver Appliance
|
|
|
|
```json
|
|
{
|
|
"dbmode":"archiverappliance",
|
|
"startMillis":0,
|
|
"startNanos":0,
|
|
"endMillis":30,
|
|
"endNanos":999999,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
Archiver Appliance supports queries by *time range* and *date range* only (as it has no notion about pulse-id).
|
|
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"dbmode":"archiverappliance","startMillis":0,"startNanos":0,"endMillis":30,"endNanos":999999,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
See JSON representation of the data above.
|
|
|
|
|
|
|
|
##### Querying for Specific Fields
|
|
|
|
Allows for server side optimizations since not all data needs to be retrieved.
|
|
|
|
```json
|
|
{
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"pulseId":0,
|
|
"value":[1,2,3,4]
|
|
},
|
|
{
|
|
"pulseId":1,
|
|
"value":[2,3,4,5]
|
|
},
|
|
{
|
|
"pulseId":2,
|
|
"value":[3,4,5,6]
|
|
},
|
|
{
|
|
"pulseId":3,
|
|
"value":[4,5,6,7]
|
|
}
|
|
]
|
|
}
|
|
]
|
|
```
|
|
|
|
##### Data Ordering
|
|
|
|
```json
|
|
{
|
|
"ordering":"desc",
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
Use **none** in case ordering does not matter (allows for server side optimizations).
|
|
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"pulseId":3,
|
|
"value":[4,5,6,7]
|
|
},
|
|
{
|
|
"pulseId":2,
|
|
"value":[3,4,5,6]
|
|
},
|
|
{
|
|
"pulseId":1,
|
|
"value":[2,3,4,5]
|
|
},
|
|
{
|
|
"pulseId":0,
|
|
"value":[1,2,3,4]
|
|
}
|
|
]
|
|
}
|
|
]
|
|
|
|
```
|
|
|
|
|
|
##### Value Aggregation
|
|
|
|
```json
|
|
{
|
|
"aggregationType":"value",
|
|
"aggregations":["min","max","mean"],
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"pulseId":0,
|
|
"value":{
|
|
"min":1.0,
|
|
"max":4.0,
|
|
"mean":2.5
|
|
}
|
|
},
|
|
{
|
|
"pulseId":1,
|
|
"value":{
|
|
"min":2.0,
|
|
"max":5.0,
|
|
"mean":3.5
|
|
}
|
|
},
|
|
{
|
|
"pulseId":2,
|
|
"value":{
|
|
"min":3.0,
|
|
"max":6.0,
|
|
"mean":4.5
|
|
}
|
|
},
|
|
{
|
|
"pulseId":3,
|
|
"value":{
|
|
"min":4.0,
|
|
"max":7.0,
|
|
"mean":5.5
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
|
|
```
|
|
|
|
Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/Aggregation.java).
|
|
|
|
|
|
|
|
|
|
##### Value Aggregation with Binning
|
|
|
|
```json
|
|
{
|
|
"nrOfBins":2,
|
|
"aggregationType":"value",
|
|
"aggregations":["min","max","mean"],
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":2,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"pulseId":0,
|
|
"value":{
|
|
"min":1.0,
|
|
"max":5.0,
|
|
"mean":3.0
|
|
}
|
|
},
|
|
{
|
|
"pulseId":2,
|
|
"value":{
|
|
"min":3.0,
|
|
"max":7.0,
|
|
"mean":5.0
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
|
|
```
|
|
|
|
Array value [aggregation](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/Aggregation.java) with additional binning.
|
|
|
|
|
|
|
|
##### Index Aggregation
|
|
|
|
```json
|
|
{
|
|
"nrOfBins":1,
|
|
"aggregationType":"index",
|
|
"aggregations":["min","max","mean","sum"],
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregationType":"index","aggregations":["min","max","mean","sum"],"fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":[
|
|
{
|
|
"pulseId":0,
|
|
"value":[
|
|
{
|
|
"min":1.0,
|
|
"max":4.0,
|
|
"mean":2.5,
|
|
"sum":10.0
|
|
},
|
|
{
|
|
"min":2.0,
|
|
"max":5.0,
|
|
"mean":3.5,
|
|
"sum":14.0
|
|
},
|
|
{
|
|
"min":3.0,
|
|
"max":6.0,
|
|
"mean":4.5,
|
|
"sum":18.0
|
|
},
|
|
{
|
|
"min":4.0,
|
|
"max":7.0,
|
|
"mean":5.5,
|
|
"sum":22.0
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
]
|
|
```
|
|
|
|
[Aggregation](https://github.psi.ch/projects/ST/repos/ch.psi.daq.query/browse/src/main/java/ch/psi/daq/query/model/Aggregation.java) of array indices with binning (several nrOfBins are also possible).
|
|
|
|
|
|
|
|
|
|
##### Extrema Search
|
|
|
|
```json
|
|
{
|
|
"aggregationType":"extrema",
|
|
"aggregations":["min","max","sum"],
|
|
"fields":["pulseId","value"]
|
|
"startPulseId":0,
|
|
"endPulseId":3,
|
|
"channels":[
|
|
"Channel_01"
|
|
]
|
|
}
|
|
```
|
|
|
|
###### Command
|
|
|
|
```bash
|
|
curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"extrema","aggregations":["min","max","sum"],"fields":["pulseId","value"],"startPulseId":0,"endPulseId":3,"channels":["Channel_01"]}' http://sf-nube-14.psi.ch:8080/query
|
|
```
|
|
|
|
###### Response
|
|
|
|
```json
|
|
[
|
|
{
|
|
"channel":"Channel_01",
|
|
"data":{
|
|
"minima":{
|
|
"min":{
|
|
"value":1.0,
|
|
"event":{
|
|
"pulseId":0,
|
|
"value":[1,2,3,4]
|
|
}
|
|
},
|
|
"sum":{
|
|
"value":10.0,
|
|
"event":{
|
|
"pulseId":0,
|
|
"value":[1,2,3,4]
|
|
}
|
|
}
|
|
},
|
|
"maxima":{
|
|
"max":{
|
|
"value":7.0,
|
|
"event":{
|
|
"pulseId":3,
|
|
"value":[4,5,6,7]
|
|
}
|
|
},
|
|
"sum":{
|
|
"value":22.0,
|
|
"event":{
|
|
"pulseId":3,
|
|
"value":[4,5,6,7]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
]
|
|
``` |