ITQC/gls_docs
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docu of new functionality.

Browse Source
This commit is contained in:
Fabian Märki
2017-02-28 13:57:58 +01:00
parent 5008df8aa3
commit 5bb1c4a363
1 changed files with 277 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

278
Readme.md
View File

@ -151,7 +151,26 @@ A request is performed by sending a valid JSON object in the HTTP request body.
"response":{
"format":"json",
"compression":"none"
}
},
"mapping":{
"incomplete":"provide-as-is"
},
"valueTransformations":[
{
"pattern":"ImageChannel",
"backend":"sf-imagestorage",
"sequence":[
...
]
},
{
"pattern":"WaveformChannel",
"backend":"sf-databuffer",
"sequence":[
...
]
}
]
}
```
@ -163,6 +182,8 @@ A request is performed by sending a valid JSON object in the HTTP request body.
- **fields**: Array of requested fields (see [here](Readme.md#requested_fields)).
- **aggregation**: Setting this attribute activates data aggregation (see [here](Readme.md#data_aggregation) for its specification).
- **response**: Specifies the format of the response of the requested data (see [here](Readme.md#response_format)). If this value is not set it defaults to JSON.
- **mapping**: Activates a table like alignment of the response which allows a mapping of values belonging to the same pulse-id/global time (see [here](Readme.md#value_mapping)). Leave this parameter undefined for the default response alignment.
- **valueTransformations**: Provides the option to apply transformations to channel values (see [here](Readme.md#value_transformations)). Transformations are mapped to channels based on the provided pattern and backend.
<a name="define_channel_names"/>
@ -310,6 +331,134 @@ It is possible to specify the response format the queried data should have.
- **compression**: Responses can be compressed when transferred from the server (values: **none**|gzip). If compression is enabled, you have to tell `curl` that the data is compressed by defining the attribute `--compressed` so that it decompresses the data automatically.
<a name="value_mapping"/>
### Value Mapping
It is possible to map values based on their pulse-id/global time. Setting this option activates a table like alignment of the response which differs from the standard response format.
```json
"mapping":{
"incomplete":"provide-as-is"
}
```
- **incomplete**: Defines how incomplete mappings should be handled (e.g., when the values of two channels should be mapped but these channels have different frequencies or one was not available at the specified query range (values: **provide-as-is**|drop|fill-null). *provide-as-is* provides the data as recorded, *drop* discards incomplete mappings, and *fill-null* fills incomplete mappings with a *null* string (simplifies parsing).
<a name="value_transformations"/>
### Value Transformations
It is possible to apply transformations to values. These transformations are available as *transformedValue* in the response (the format/value of *transformedValue* can differ based on the transformation sequence).
```json
"valueTransformations":[
{
"pattern":"ChannelNamePattern",
"backend":"sf-databuffer",
"sequence":[
...
]
}
]
```
- **valueTransformations**: An array of different value transformations. Assigning transformations to queried channels is done using regular expressions applied to channel names (see: [Pattern](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html), more precisely using [Matcher.find()](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Matcher.html#find--)) whereas a longer match sequence is considered superior to a shorter match sequence.
- **pattern**: The regular expression applied to channel names (e.g., "" applies to all channel, '\^SINEG01' to all channels starting with SINEG01 like '**SINEG01**-RCIR-PUP10:SIG-AMPLT').
- **backend**: The backend (usually left undefined).
- **sequence**: A sequence of transformation operations (see [Value Transformation Operations](Readme.md#value_transformation_operations) for examples).
<a name="value_transformation_operations"/>
### Value Transformation Operations
#### Image Resize
Following transformation sequence resizes images by the factor of 8, sets the value range to [400...4000], uses the *temperature* color model, and provides the image as a base64 encoded string.
```json
"valueTransformations":[
{
"pattern":"Image",
"backend":"sf-imagestorage",
"sequence":[
{
"imageResize":{
"downScaleFactor":8,
"valueAggregation":"first-value"
},
"valueFunctions":[
{
"min":400.0,
"mapTo":400.0
},
{
"max":4000.0,
"mapTo":4000.0
}
],
"colorModel":{
"modelType":"temperature",
"nrOfColors":32,
"colors":[
"#3B4CC0",
"#B40426"
]
}
},
{
"imageFormat":"png",
"imageEncoder":"base64string"
}
]
}
]
```
- **imageResize**: The image resize operation.
- **downScaleFactor**: The image downscale factor (e.g. a factor of 8 reduces a square of 8x8 pixels into one pixel.
- **valueAggregation**: Defines how the pixel values should be aggregated (values: **first-value**|max-value|mean-value). *first-value* uses the first value of the reduction square (very fast but signals might get lost), *max-value* uses the max value of the reduction square, and *mean-value* the mean value of the reduction square.
- **valueFunctions**: Array of functions to be applied to pixel values.
- **min**: Map all values smaller than the defined values to the *mapTo* value. Allows to define a filter to eliminate noise (using "mapTo":0) or to set a lower pixel value bound (e.g. to focus on a specific pixel value range - need to be used together with a *max* value function). In case *mapTo* is not defined it defaults to *min*.
- **max**: Map all values bigger than the defined values to the *mapTo* value. In case *mapTo* is not defined it defaults to *max*.
- **colorModel**: The color model to be used for the image.
- **type**: The color model type (values: linear|**temperature**|gradient). Each type has a decent default color scheme (linear uses a grayscale, temperatur a blue to red, and gradient a rainbow color scheme).
- **nrOfColors**: The number of colors to map between the min. and max. pixel value (use a power of 2 like 16, 32, 64, 128, or 256). The more colors you use the more details will be visible but the larger the images will be).
- **colors**: The colors to use for the color model (defaults for *linear* ["#000000","#FFFFFF"] (grayscale), for *temperature* ["#3B4CC0","#B40426"], and for *gradient* ["#0000FF","#00FFFF","#00FF00","#FFFF00","#FF0000"] (rainbow: blue, cyan, green, yellow, red)).
- **imageFormat**: The output image format (values: **png**|jpg).
- **imageEncoder**: The encoding of the image (values: **base64string**|byte).
Applying an image resize transformation has the advantage that the computation is done on the backend resulting in much less data transfer.
#### Value Sampling
Following transformation sequence resizes images by the factor of 4, filters all values below 200 (considers values below 200 as noise), and provides the image in the raw format. The response has the format '"transformedValue":{"shape":[width,height],"value":[value1,value2,...]}'. This transformation could potentially also be applied to waveforms.
```json
"valueTransformation":{
"valueSampler":{
"downSampleFactor":4,
"valueAggregation":"first-value"
},
"valueFunctions":[
{
"min":200.0,
"mapTo":0.0
}
]
}
```
Applying a value sampling transformation has the advantage that the computation is done on the backend resulting in much less data transfer.
### Example Queries
The following examples build on waveform data (see below). They also work for scalars (consider it as a waveform of length = 1) and images (waveform of length = dimX * dimY).
@ -361,6 +510,50 @@ The following examples build on waveform data (see below). They also work for sc
"value":[4,5,6,7]
}
]
},
{
"channel":{
"backend": "sf-databuffer",
"name": "Channel_02"
},
"data":[
{
"iocSeconds":"0.000000000",
"pulseId":0,
"globalSeconds":"0.000000000",
"shape":[
4
],
"value":[1,2,3,4]
},
{
"iocSeconds":"0.010000000",
"pulseId":1,
"globalSeconds":"0.010000000",
"shape":[
4
],
"value":[2,3,4,5]
},
{
"iocSeconds":"0.020000000",
"pulseId":2,
"globalSeconds":"0.020000000",
"shape":[
4
],
"value":[3,4,5,6]
},
{
"iocSeconds":"0.030000000",
"pulseId":3,
"globalSeconds":"0.030000000",
"shape":[
4
],
"value":[4,5,6,7]
}
]
}
]
```
@ -950,4 +1143,87 @@ Illustration of array index aggregation with additional with binning (several nr
![Index Aggregation with Binning](doc/images/Index_Binning.png)
#### Query with Value Mapping
Maps values based on their pulse-id/global time. Please note that the response format differs in order to represent the value mapping. Lets assume for this query that *Channel_02* only provides events for even pulse-ids.
```json
{
"fields":["pulseId","value"],
"range":{
"startPulseId":0,
"endPulseId":3
},
"channels":[
"Channel_01",
"Channel_02"
],
"mapping":{
"incomplete":"provide-as-is"
}
}
```
##### Command
```bash
curl -H "Content-Type: application/json" -X POST -d '{"fields":["pulseId","value"],"range":{"startPulseId":0,"endPulseId":3},"channels":["Channel_01","Channel_02"],"mapping":{"incomplete":"provide-as-is"}}' https://data-api.psi.ch/sf/query | python -m json.tool
```
##### Response
With *provide-as-is*, the response will contain all available data. In case *drop* would be used, only events with even pulse-ids would be returned (assuming *Channel_02* only provides events for even pulse-ids).
```json
{
"data":[
[
{
"channel":"Channel_01",
"backend":"sf-databuffer",
"pulseId":0,
"value":0
},
{
"channel":"Channel_02",
"backend":"sf-databuffer",
"pulseId":0,
"value":0
}
],
[
{
"channel":"Channel_01",
"backend":"sf-databuffer",
"pulseId":1,
"value":1
}
],
[
{
"channel":"Channel_01",
"backend":"sf-databuffer",
"pulseId":2,
"value":2
},
{
"channel":"Channel_02",
"backend":"sf-databuffer",
"pulseId":2,
"value":2
}
],
[
{
"channel":"Channel_01",
"backend":"sf-databuffer",
"pulseId":3,
"value":3
}
]
]
}
```