diff --git a/Readme.md b/Readme.md index ad8f7cd..f712a6b 100644 --- a/Readme.md +++ b/Readme.md @@ -102,86 +102,11 @@ curl -H "Content-Type: application/json" -X POST -d '{"regex": "AMPLT|PHASE"}' h ] ``` - - -## Query Range - -Queries are applied to a range. The following types of ranges ranges are supported. - -### By Pulse-Id - -```json -"range":{ - "startPulseId":0, - "endPulseId":100 -} -``` - -- **startPulseId**: The start pulse-id of the range request. -- **endPulseId**: The end pulse-id of the range request. - - -### By Date - -```json -"range":{ - "startDate":"2015-08-06T18:00:00.000", - "endDate":"2015-08-06T18:59:59.999", -} -``` - -- **startDate**: The start date of the time range (ISO8601 format (YYYY-MM-DDThh:mm:ss.sTZD e.g. 1997-07-16T19:20:30.123+02:00 (omitting +02:00 falls back to the local time zone)). Additionally, ISO8601 format up to nano seconds is also supported (e.g. 1997-07-16T19:20:30.123456789+02:00). -- **endDate**: The end date of the time range. - - -### By Time - -```json -"range":{ - "startSeconds":"0.0", - "endSeconds":"1.000999999" -} -``` - -- **startSeconds**: The start time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds. -- **endSeconds**: The end time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds. - - - -## Query Channel Names - -The simplest way to define channels is to use an array of channel name Strings. - -```json -"channels":[ - "Channel_02", - "Channel_04" -] -``` - -The query interface will automatically select the backend which contains the channel (e.g., *databuffer* for *Channel_02* and *archiverappliance* for *Channel_04*. In case name clashes exist, the query interface will use following order of priority: *databuffer* and *archiverappliance*. - -It is also possible to explicitly define the backend to overcome name clashes. - -```json -"channels":[ - { - "name":"Channel_01", - "backend":"archiverappliance" - }, - { - "name":"Channel_01", - "backend":"databuffer" - } -] -``` - - ## Query Data -### Query request endpoint +### Request ``` GET http://:/query @@ -201,21 +126,98 @@ The following attributes can be specified: - **binSize**: Activates data binning. Specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies). - **aggregations**: Activates data aggregation. Array of requested aggregations (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Aggregation.java) for possible values). These values will be added to the *data* array response. - **aggregationType**: Specifies the type of aggregation (see [here](https://git.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/AggregationType.java)). The default type is *value* aggregation (e.g., sum([1,2,3])=6). Alternatively, it is possible to define *index* aggregation for multiple arrays in combination with binning (e.g., sum([1,2,3], [3,2,1]) = [4,4,4]). -- **compression**: Defines the compression algorithm to use, default value is **none**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Compression.java)) - **responseFormat**: Specifies the format the response of the requested data is in, either in JSON or CSV format, default value **JSON**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/ResponseFormat.java)) +- **compression**: Defines the compression algorithm to use, default value is **none**, see all values [here](https://github.psi.ch/sf_daq/ch.psi.daq.query/blob/master/src/main/java/ch/psi/daq/query/model/Compression.java)) -### `compression`: compression of data can be enabled + -By default, no data is compressed when transferred from the server to the client. However, compression can be enabled by setting the `compression` attribute to a value other than `none`, i.e. to `gzip` or `deflate`. +### Define Channel Names -If compression is enabled, we have to tell `curl` that the data is compressed so that it is being decompressed automatically. `curl` decompresses the response when the `--compressed` parameter is set. +The simplest way to define channels is to use an array of channel name Strings. -### `responseFormat`: data is in JSON by default +```json +"channels":[ + "Channel_02", + "Channel_04" +] +``` + +The query interface will automatically select the backend which contains the channel (e.g., *databuffer* for *Channel_02* and *archiverappliance* for *Channel_04*). In case name clashes exist, the query interface will use following order of priority: *databuffer* and *archiverappliance*. + +It is also possible to explicitly define the backend to overcome name clashes. + +```json +"channels":[ + { + "name":"Channel_01", + "backend":"archiverappliance" + }, + { + "name":"Channel_01", + "backend":"databuffer" + } +] +``` + + + +### Define Query Range + +Queries are applied to a range. The following types of ranges are supported. + +#### By Pulse-Id + +```json +"range":{ + "startPulseId":0, + "endPulseId":100 +} +``` + +- **startPulseId**: The start pulse-id of the range request. +- **endPulseId**: The end pulse-id of the range request. + + +#### By Date + +```json +"range":{ + "startDate":"2015-08-06T18:00:00.000", + "endDate":"2015-08-06T18:59:59.999", +} +``` + +- **startDate**: The start date of the time range in the ISO8601 format (such as 1997-07-16T19:20:30.123+02:00 or 1997-07-16T19:20:30.123456789+02:00 (omitting +02:00 falls back to the local time zone)). +- **endDate**: The end date of the time range. + + +#### By Time + +```json +"range":{ + "startSeconds":"0.0", + "endSeconds":"1.000999999" +} +``` + +- **startSeconds**: The start time of the range in seconds since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds. +- **endSeconds**: The end time of the range in seconds. + + +### Define `responseFormat` Responses can be formatted as CSV or JSON using the `responseFormat` field. The returned data is JSON-formatted by default. CSV export does not support `index` and `extrema` aggregations. + +### Define `compression` + +By default, data is not compressed when transferred from the server to the client. However, compression can be enabled by setting the `compression` attribute to a value other than `none`, i.e. to `gzip` or `deflate`. + +If compression is enabled, you have to tell `curl` that the data is compressed so that it is being decompressed automatically. `curl` decompresses the response when the `--compressed` parameter is set. + + ### Example Queries The following examples build on waveform data (see below). They also work for scalars (consider it as a waveform of length = 1) and images (waveform of length = dimX * dimY). @@ -270,7 +272,7 @@ The following examples build on waveform data (see below). They also work for sc ### Query Examples -##### Query by Pulse-Id Range +#### Query by Pulse-Id Range ```json { @@ -284,19 +286,19 @@ The following examples build on waveform data (see below). They also work for sc } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response See JSON representation of the data above. -##### Query by Time Range +#### Query by Time Range ```json { @@ -310,19 +312,19 @@ See JSON representation of the data above. } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response See JSON representation of the data above. -##### Query by Date Range +#### Query by Date Range ```json { @@ -338,19 +340,19 @@ See JSON representation of the data above. Supported format is ISO8601 *YYYY-MM-DDThh:mm:ss.sTZD* (e.g. *1997-07-16T19:20:30.475+02:00*). -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"range":{"startDate":"1970-01-01T01:00:00.000","endDate":"1970-01-01T01:00:00.030"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response See JSON representation of the data above. -##### Querying Archiver Appliance +#### Querying Archiver Appliance ```json { @@ -374,17 +376,18 @@ See JSON representation of the data above. Archiver Appliance supports queries by *time range* and *date range* only (as it has no notion about pulse-id). -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"range":{"startSeconds":"0.0","endSeconds":"0.030999999"},"channels":[{"name": "Channel_01","backend":"archiverappliance"}]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response See JSON representation of the data above. -##### Query using compression + +#### Query using compression ```json { @@ -414,7 +417,7 @@ or `deflate` can be used too: } ``` -###### Command (gzip) +##### Command (gzip) The `curl` command has a `--compressed` option to decompress data automatically. @@ -422,7 +425,7 @@ The `curl` command has a `--compressed` option to decompress data automatically. curl --compressed -H "Content-Type: application/json" -X POST -d '{"compression":"gzip","range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -##### Query setting CSV response format +#### Query setting CSV response format ```json { @@ -449,13 +452,13 @@ curl --compressed -H "Content-Type: application/json" -X POST -d '{"compression" It is possible to request the time in seconds (since January 1, 1970 (the UNIX epoch) as a decimal value including fractional seconds - using fields *globalSeconds* and *iocSeconds*) or in milliseconds (since January 1, 1970 (the JAVA epoch) - using fields *globalMillis* and *iocMillis*) -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"responseFormat":"csv","range":{"startPulseId":0,"endPulseId":4},"channels":["channel1","channel2"],"fields":["channel","pulseId","iocSeconds","globalSeconds","shape","eventCount","value"]}' http://data-api.psi.ch/sf/query ``` -###### Response +##### Response The response is in CSV. @@ -473,7 +476,7 @@ testChannel2;3;0.030000000;0.030000000;[1];1;3 testChannel2;4;0.040000000;0.040000000;[1];1;4 ``` -##### Querying for Specific Fields +#### Querying for Specific Fields Allows for server side optimizations since not all data needs to be retrieved. @@ -490,13 +493,13 @@ Allows for server side optimizations since not all data needs to be retrieved. } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -524,7 +527,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","valu ] ``` -##### Data Ordering +#### Data Ordering ```json { @@ -543,13 +546,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","valu Use **none** in case ordering does not matter (allows for server side optimizations). -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -579,7 +582,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields ``` -##### Value Aggregation +#### Value Aggregation ```json { @@ -596,13 +599,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"ordering":"desc","fields } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -652,7 +655,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu ![Value Aggregation](doc/images/Value_Aggregation.png) -##### Value Aggregation with Binning (nrOfBins) +#### Value Aggregation with Binning (nrOfBins) ```json { @@ -670,13 +673,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":2,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -709,7 +712,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu ![Value Aggregation with Binning](doc/images/Value_Binning_NrOfBins.png) -##### Value Aggregation with Binning (binSize) +#### Value Aggregation with Binning (binSize) **binSize** specifies the number of pulses per bin for pulse-range queries or the number of milliseconds per bin for time-range queries (using number of pulses and number of milliseconds makes this binning strategy consistent between channel with different update frequencies). @@ -729,13 +732,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"binSize":10,"aggregationType":"value","aggregations":["min","max","mean"],"fields":["globalMillis","value"],"range":{"startSeconds":"0.0","endSeconds":"0.030000000"},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -768,7 +771,7 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu ![Value Aggregation with Binning](doc/images/Value_Binning_BinSize.png) -##### Index Aggregation +#### Index Aggregation ```json { @@ -786,13 +789,13 @@ Array value [aggregations](https://github.psi.ch/projects/ST/repos/ch.psi.daq.qu } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregationType":"index","aggregations":["min","max","mean","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [ @@ -838,7 +841,7 @@ curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregation ![Index Aggregation with Binning](doc/images/Index_Binning.png) -##### Extrema Search +#### Extrema Search ```json { @@ -855,13 +858,13 @@ curl -H "Content-Type: application/json" -X POST -d '{"nrOfBins":1,"aggregation } ``` -###### Command +##### Command ```bash curl -H "Content-Type: application/json" -X POST -d '{"aggregationType":"extrema","aggregations":["min","max","sum"],"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01"]}' http://data-api.psi.ch/sf/query | python -m json.tool ``` -###### Response +##### Response ```json [