From 05cd7edf715ac2b393db9428a79dc8a214222229 Mon Sep 17 00:00:00 2001 From: Andrew Johnson Date: Wed, 28 Dec 2022 15:52:30 -0600 Subject: [PATCH] Update channel filter documentation, adding $ syntax --- .../database/src/std/filters/filters.dbd.pod | 297 ++++++++++++++---- 1 file changed, 233 insertions(+), 64 deletions(-) diff --git a/modules/database/src/std/filters/filters.dbd.pod b/modules/database/src/std/filters/filters.dbd.pod index 7b3146609..7f72ebe6c 100644 --- a/modules/database/src/std/filters/filters.dbd.pod +++ b/modules/database/src/std/filters/filters.dbd.pod @@ -1,59 +1,214 @@ -=head1 Channel Filters +=title Field Modifiers and Channel Filters -Channel Filters can be applied to Channel Access channels by a client, using -a JSON Field Modifier to select the filter and any parameters. -The following filters are available in this release: +=head2 Contents =over -=item * L +=item * L -=item * L +=item * L -=item * L +=item * L -=item * L +=item * L -=item * L +=over -=item * L +=item * L + |/"Long String Field Modifier $"> + +=item * L] >>> + |/"Subarray Field Modifier [start:increment:end]"> =back -=head2 Using Filters +=item * L -Channel filters can be added to any Channel Access channel name. -There can be more than one filter applied to the same channel, in which case the -order that they are specified will control the order in which they are applied -to the resulting data-stream. -The filter specification must appear after the field name, or if the default -(VAL) field is used after a dot C<.> appended to the record name. -With the exception of the array short-hand which is described below, all filters -must appear inside a pair of braces C< {} > after the dot expressed as a JSON -(L) object, which allows filter -parameters to be included as needed. +=over -Each filter is given as a name/value pair. The filter name (given in parentheses -in the titles below) is a string, and must be enclosed inside double-quotes C<"> -characters as per the JSON specification. -Parameters to that filter are provided as the value part of the name/value pair, -and will normally appear as a child JSON object consisting of name/value pairs -inside a nested pair of braces C< {} >. +=item * L>> + |/"TimeStamp Filter ts"> -=head4 Example Filter +=item * L}} >>> + |/"Deadband Filter dbnd"> -Given a record called C the following would apply a filter C to -the VAL field of that record, giving the filter two numeric parameters named -C and C: +=item * L}} >>> + |/"Array Filter arr"> - test:channel.{"f":{"lo":0,"hi":10}} +=item * L}} >>> + |/"Synchronize Filter sync"> -Note that due to the required presence of the double-quote characters in the -JSON strings in the name string, it will usually be necessary to enclose a -filtered name within single-quotes C<< ' ... ' >> when typing it as an -argument to a Unix shell command. +=item * L}} >>> + |/"Decimation Filter dec"> -=head2 Filter Reference +=item * L}} >>> + |/"User Tag Filter utag"> + +=back + +=back + + +=head2 Introduction + +A Field Modifier is a string that is appended to the field name part of a +Channel Access or PV Access channel name of an IOC-based server. +The IOC currently recognizes 3 different kinds of field modifier, which are +described below. + +A Channel Filter is an IOC plugin that can be attached to an IOC process +variable channel using a field modifier, and can alter the data updates that are +served to the client that requested the filtering. +Clients that use a channel filter have no effect on other clients connected to +the same PV. +Filters can only modify or drop monitor events that the IOC posts; introducing +extra monitor events (in between the events posted by the IOC itself) is not +currently possible. + +Most Channel Filters are configured by a field modifier that uses JSON5 +syntax to select the type of filter and provide any parameters it accepts. + + +=head2 Using Field Modifiers and Channel Filters + +Modifiers can be added to any Channel Access or PV Access channel name. +There can be more than one modifier or filter applied to a channel. +Modifiers must appear immediately after the field name if one is included in the +channel name. +If no field is named because the default (VAL) field is the target, the +modifiers must come immediately after a dot C<.> appended to the record name. + +The order that modifiers and filters are specified controls the order in which +they are applied to the resulting data-stream. +If used, the Long String and Subarray field modifiers must appear first and in +that order, followed by any channel filters inside a pair of braces C< {} > that +form a JSON5 (L) map, which +allows filters and their parameters to be specified as needed. + +Each JSON5 filter is specified as a name:value pair. +The filter name is a map key that may be an unquoted identifier name (which all +of the filter names given below are), or a string enclosed inside either single- +or double-quote characters C<'> or C<"> as per the JSON5 specification. +Parameters to the filter are provided in the value part of the key:value pair +after the colon C<:> character, and will normally be an embedded JSON5 map +containing zero or more key/value pairs inside a nested pair of braces C< {} >. + +Unless included inside a quoted string, white space characters are ignored and +skipped over by the JSON5 parser between other token characters. +This includes horizontal and vertical tabs, line feed/form feed/carriage return, +space and the non-breaking space character C< \xA0 >. +Within a quoted string, line breaks may escaped with a backslash C<\> to be +omitted from the parsed string. + +An IOC Channel Access link can include filters in its channel name, but it is +important to not include any spaces at all in the filter specification. +If a filter name or parameter must contain a space it will be necessary to +express that space character as an escaped character C<\x20> or C<\u0020> inside +a quoted string, otherwise the space will mark the end of the channel name to +the link parsing code inside the IOC. + +=head4 Example Filters + +Given a record called C the following would all apply a channel +filter C to the VAL field of that record, giving the filter two numeric +parameters named C and C: + + test:channel.{f:{lo:0,hi:10}} + test:channel.{"f":{"lo":0, "hi":10}} + test:channel.{'f': {'lo':0, 'hi':10} } + +When typing a filtered channel name as an argument to a Unix shell command, if +quote characters are used for keys or values in JSON strings within the channel +name string it may be necessary to enclose the name within quotes C<'> or C<"> +or to use back-slash escapes before them. +Quotes may not be required when the Long String modifier C<$> is used at the end +of a field name with nothing following it, but will be necessary for a square +bracketted Subarray filter or if a dollar sign is followed by something else. + + Hal$ caget test:channel.{f:{lo:0,hi:10}} + ... + Hal$ caget 'test:channel.{"f":{"lo":0, "hi":10}}' + ... + Hal$ caget -S calc:record.CALC$ + ... + Hal$ caget -S 'test:channel.NAME$[0:4]' + test:channel.NAME$[0:4] test + +=head2 Field Modifier Reference + +The two built-in field modifiers use a simplified syntax following the record +field name. + +=head3 Long String Field Modifier C<$> + +Appending a dollar sign C<$> to the name of a C field causes the IOC +to change the representation of that field into an array of characters, which +allows strings longer than 40 character to be transported through Channel +Access. +Long strings are particularly useful for the CALC fields of a calculation or +calcout record, which can hold up to 80 characters, or the VAL fields of an lsi +(Long String Input) or lso (Long String Output) record which can be any length +as chosen by the database designer. + + Hal$ cainfo test:channel.NAME + test:channel.NAME + State: connected + Host: 10.234.56.78:5064 + Access: read, no write + Native data type: DBF_STRING + Request type: DBR_STRING + Element count: 1 + Hal$ cainfo test:channel.NAME$ + test:channel.NAME$ + State: connected + Host: 10.234.56.78:5064 + Access: read, no write + Native data type: DBF_CHAR + Request type: DBR_CHAR + Element count: 61 + +A CA client accessing a channel that uses the Long String field modifier will +have to be specifically configured to treat the data as a string instead of the +array of C that it looks like. +CA clients should not attempt to parse the channel name themselves to recognize +this field modifier in the name. +All long string values returned by the IOC should include a trailing zero byte +in their data as is standard for strings in the C language. +For the catools programs provided with Base, the flag C<-S> indicates that a +channel containing a character array should be treated as a long string. + + Hal$ caget test:channel.NAME + test:channel.NAME test:channel + Hal$ caget test:channel.NAME$ + test:channel.NAME$ 61 116 101 115 116 58 99 104 97 110 110 101 108 0 0 ... + Hal$ caget -S test:channel.NAME$ + test:channel.NAME$ test:channel + + +=head3 Subarray Field Modifier C<[start:increment:end]> + +This square-bracket field modifier syntax gets translated within the IOC into +calls to the L, see that section below for +details of this shorthand. + +The subarray field modifier syntax can immediately follow a Long String field +modifier, which permits fetching various kinds of substrings from the field. +This syntax cannot appear after a JSON filter specification though, the JSON +"arr" filter syntax must be used to apply an array filter after any other JSON +filter type. + + Hal$ caget -S 'test:channel.NAME$[0:4]' + test:channel.NAME$[0:4] test + Hal$ caget -S 'test:channel.NAME$[5:-1]' + test:channel.NAME$[5:-1] channel + + +=head2 JSON5 Channel Filters =cut @@ -61,11 +216,11 @@ registrar(tsInitialize) =head3 TimeStamp Filter C<"ts"> -This filter is used to set the timestamp of the value fetched through -the channel to the time the value was fetched (or an update was sent), -rather than the time the record last -processed which could have been days or even weeks ago for some records, or set -to the EPICS epoch if the record has never processed. +This filter replaces the timestamp in the data fetched through the channel with +the time the value was fetched (or an update was sent). The record's timestamp +indicates when the record last processed, which could have been days or even +weeks ago for some records, or set to the EPICS epoch if the record has never +processed. =head4 Parameters @@ -125,7 +280,7 @@ The default mode is C if no mode parameter is included. test:channel 2012-09-01 22:10:23.601023 5 test:channel 2012-09-01 22:10:24.601136 6 HIGH MINOR ^C - Hal$ camonitor 'test:channel.{"dbnd":{"abs":1.5}}' + Hal$ camonitor 'test:channel.{"dbnd":{"d":1.5}}' test:channel.{"dbnd":{"d":1.5}} 2012-09-01 22:11:49.613341 1 LOLO MAJOR test:channel.{"dbnd":{"d":1.5}} 2012-09-01 22:11:51.613615 3 LOW MINOR test:channel.{"dbnd":{"d":1.5}} 2012-09-01 22:11:53.613804 5 @@ -144,16 +299,21 @@ subarrays). =head4 Parameters -Note: Negative index numbers address from the end of the array, with C<-1> being the last element. - =over =item Square bracket notation C<[start:increment:end]> (shorthand) -The common square bracket notation which can be used in place of JSON. +This much shorter square bracket notation can be used in place of JSON. Any parameter may be omitted (keeping the colons) to use the default value. -If only one colon is included, this means C<[start:end]> with an increment of 1. -If only a single parameter is used C<[index]> the filter returns one element. +If only one colon is included, it means C<[start:end]> with an increment of 1. +If only a single parameter is given C<[index]> the filter returns one element. + +Index numbers for the start and end parameters must be integers, with the first +array element being found at index C<0>. +The value of an index may be negative, in which case the indexing is counted +backwards from the end of the array, with C<-1> being the last element. +If the start index selects an element that comes after the end index element, +the subarray returned will always be empty. =item Start index C<"s"> @@ -161,8 +321,8 @@ Index of the first original array element to retrieve. =item Increment C<"i"> -Index increment between retrieved elements of the original array; must be -a positive number. +The stride or increment to apply between elements of the original array to be +retrieved. This value must be a positive integer. =item End index C<"e"> @@ -176,9 +336,9 @@ C (first element), C (fetch all elements), C =head4 Example - Hal$ caget test:channel 'test:channel.{"arr":{"s":2,"i":2,"e":8}}' test:channel.[3:5] test:channel.[3:2:-3] + Hal$ caget test:channel 'test:channel.{"arr":{s:2,i:2,e:8}}' test:channel.[3:5] test:channel.[3:2:-3] test:channel 10 0 1 2 3 4 5 6 7 8 9 - test:channel.{"arr":{"s":2,"i":2,"e":8}} 4 2 4 6 8 + test:channel.{"arr":{s:2,i:2,e:8}} 4 2 4 6 8 test:channel.[3:5] 3 3 4 5 test:channel.[3:2:-3] 3 3 5 7 @@ -202,13 +362,14 @@ C. =item Mode+State Mode and state can be specified in one definition (shorthand). -The desired mode is given as parameter name (C<"before"> / C<"first"> / -C<"while"> / C<"last"> / C<"after"> / C<"unless">), with the state name -(enclosed in double quotes C<">) as value. +The desired mode is given as the parameter name (C<"before"> / C<"first"> / +C<"while"> / C<"last"> / C<"after"> / C<"unless"> which may be unquoted), with +the state name (enclosed in single or double quotes C<"> or C<'>) as the value. =item Mode C<"m"> -A single word from the list below, enclosed in double quotes C<">. +A single word from the list below, enclosed in single or double quotes C<'> or +C<">. This controls how the state value should affect the monitor stream. =over @@ -235,7 +396,7 @@ as the state is false. =item State C<"s"> -The name of a state variable, enclosed in double quotes C<">. +The name of a state variable, enclosed in single or double quotes C<"> or C<'>. =back @@ -245,7 +406,7 @@ Assuming there is a system state called "blue", that is being controlled by some other facility such as a timing system, updates could be restricted to periods only when "blue" is true by using - Hal$ camonitor 'test:channel' 'test:channel.{"sync":{"while":"blue"}}' + Hal$ camonitor 'test:channel' 'test:channel.{sync:{while:"blue"}}' ... =cut @@ -283,17 +444,18 @@ client connects. To sample a 60Hz channel at 1Hz, a 10Hz channel every 6 seconds or a 1Hz channel once every minute: - Hal$ camonitor 'test:channel' 'test:channel.{"dec":{"n":60}}' + Hal$ camonitor 'test:channel' 'test:channel.{dec:{n:60}}' ... =cut registrar(utagInitialize) -=head3 UTag Filter C<"utag"> +=head3 User Tag Filter C<"utag"> -This filter applies a test UTAG&M==V to the value taken from the UTAG record field -and drops those updates which evaluate as false. +This filter applies a test C< (UTAG & M) == V > to the value of the record's +UTAG field at the time each monitor event is posted, and drops all updates for +which this expression is false. =head4 Parameters @@ -301,12 +463,19 @@ and drops those updates which evaluate as false. =item Mask C<"M"> -Bit mask. +An integer to be used as a bit mask. =item Value C<"V"> -Required value. +The integer value to be matched after applying the mask to the UTAG value. =back +=head4 Example + +To read a channel only when the UTAG value is even (bit 0 is 0): + + Hal$ camonitor 'test:channel' 'test:channel.{utag:{m:1, v:0}}' + ... + =cut