pvDatabaseCPP

EPICS v4 Working Group, Working Draft, 11-Dec-2012

Latest version:
pvDatabaseCPP.html
This version:
pvDatabaseCPP20121211.html
Previous version:
pvDatabaseCPP_20121127.html
Editors:
Marty Kraimer, BNL

This product is made available subject to acceptance of the EPICS open source license.

Abstract

This document describes pvDatabaseCPP, which is a framework for implementing a network accessable database of smart memory resident records. Network access is via pvAccess. The data in each record is a top level PVStructure as defined by pvData. The framework includes a complete implementation of ChannelProvider as defined by pvAccess. The framework must be extended in order to create record instances. The minimum that an extenson must provide is a top level PVStructure and a process method but the framework provides for complex extensions.

EPICS version 4 is a set of related products in the EPICS V4 control system programming environment:

pvData
pvData (Process Variable Data) defines and implements an efficent way to store, access, and communicate memory resident structured data
pvAccess
pvAccess is a software library for high speed controls network communications, optimized for pvData
pvIOC
pvIOC is a software framework for building network accessable "smart" real time databases, suitable for interfacing devices in a distributed control system, that can exchange pvData over pvAccess.
pvService
A middle layer for implementing data services.

Each of these products has a Java and a C++ implementation.

Status of this Document

This is the 11-Dec-2012 version of the definition of pvDatabaseCPP.

This is the beginning of the implementation of pvDataBaseCPP. It describes the features that will be provided. The class definitions for PVRecord are implemented. The class definition for PVDatabase are defined but not implemented.

Table of Contents

Introduction

Overview

This document descibes a C++ implementation of some of the components in pvIOCJava. It extracts the core components required to create a network accessible database of smart memory resident records. pvDatabaseCPP does not and will not implement any of the specialized support that pvIOCJava provides. Instead other projects will implement the specialized support. It is expected that many services will be created that do not require the full features provided by pvIOCJava. In the future pvIOCJava should be split into multiple projects with one of them named pvDatabaseJava.

A brief description of a pvDatase is that it is a network accessible set of smart memory resident records. Each record has data composed of a top level PVStructure. Each record has a name which is the channelName for pvAccess. A local Channel Provider implements the complete ChannelProvider and Channel interfaces as defined by pvAccess. This local provider is accessed by the remote pvAccess server. A record is smart because code can be attached to a record.

This document describes components that provides the following features:

database
This encapsulates the concept of a database of memory resident smart records. The two main components are:
pvRecord
This encapsulates the concept of a smart record. It can be processed. Changes to field values can be trapped. A record can be locked.
pvDatabase
This is a database of pvRecords. Records can be added and removed from a database.
localChannelProvider
This is a complete implementation of ChannelProvider and Channel as defined by pvAccess. It is used by the server side of pvAccess to attach to pvRecords. This component also includes the monitor and pvCopy components from pvIOCJava

database does not itself implement pvRecord instances. Instead it provides a base classes that make it easy to create record instances. What does have to be implemented is a top level PVStructure and the following two methods:

process
This is what makes a record smart. What process does is up to the implementation except that it must decide if it's execution model is synchronous or asynchronous. Synchronous means that when process returns the processing is complete. Asynchronous means that when process returns the processing is not complete. Instead process invokes other threads that will complete the processing at a later time.
isSynchronous
Which execution model is being implemented.

Example PVRecord Extension

Directory example/record has an example PVRecord implementation. It implements a counter. The top level structure is:

structure
    long value

NOTE: The example compiles but does not build because nothing is implemented.

exampleRecord.h

This is the class description. The example extends PVRecord.

class ExampleRecord :
  public virtual PVRecord
{
public:
    POINTER_DEFINITIONS(ExampleRecord);
    static PVRecordPtr create(epics::pvData::String const & recordName);
    virtual ~ExampleRecord();
    virtual bool isSynchronous();
    virtual void process(
        epics::pvDatabase::RecordProcessRequesterPtr const &processRequester);
private:
    ExampleRecord(epics::pvData::String const & recordName,
        epics::pvData::PVStructurePtr const & pvStructure,
        epics::pvData::PVLongPtr const &pvValue);
    epics::pvData::PVLongPtr pvValue;
};

where

create
This is example specific. See the implemention for details.
~ExampleRecord
The destructor must be declared virtual.
isSynchronous
The implementation must say if process is synchronous or asynchronous.
process
The implementation.
ExampleRecord
For the example this is private.

exampleRecord.cpp

This is the class implementation.

ExampleRecord::~ExampleRecord(){}

PVRecordPtr ExampleRecord::create(String const & recordName)
{
    String properties;
    PVStructurePtr pvStructure = getStandardPVField()->scalar(pvLong,properties);
    PVLongPtr pvValue =  pvStructure->getLongField("value");
    PVRecordPtr pvRecord(new ExampleRecord(recordName,pvStructure,pvValue));
    return pvRecord;
}

ExampleRecord::ExampleRecord(
    String const & recordName,
    PVStructurePtr const & pvStructure,
    PVLongPtr const &pvValue)
: PVRecord(recordName,pvStructure),
  pvValue(pvValue)
{}

bool ExampleRecord::isSynchronous() {return true;}

void ExampleRecord::process(
    RecordProcessRequesterPtr const &processRequester,bool alreadyLocked)
{
    if(!alreadyLocked) lock();
    pvValue->put(pvValue->get() + 1);
    processRequester->recordProcessResult(Status::Ok);
    unlock();
    processRequester->recordProcessComplete();
    dequeueProcessRequest(processRequester);
}

where

create
Creates a PVStructure with a single subfield named value. It gets the interface to the value field. It then creates an ExampleRecord and returns it.
~ExampleRecord
Does not have to do anything because of shared pointers.
ExampleRecord
Calls the base class constructor and sets pvValue.
isSynchronous
The example is synchronous.
process
Gets the curent value, increments it, and puts the new value. It than calls two processRequester callbacks.

exampleRecordMain.cpp

This is a main for creating and running the example.

int main(int argc,char *argv[])
{
    String recordName("exampleRecord");
    PVRecordPtr pvRecord = ExampleRecord::create(recordName);
    PVDatabasePtr pvDatabase = PVDatabase::getMaster();
    pvDatabase->addRecord(pvRecord);
    cout << recordName << "\n";
    string str;
    while(true) {
        cout << "Type exit to stop: \n";
        getline(cin,str);
        if(str.compare("exit")==0) break;

    }
    return 0;
}

The main program creates an example record and adds it to the database. It then runs until the process is stopped by typing exit.

Until the process is stopped, pvAccess clients can put and get the value field. For example

pvget exampleRecord
pvput exampleRecord 5

Will both work.

Phased Development

This documentation describes the first phase of a phased implementation of pvDatabaseCPP:

pvRecord
Wrapper on PVStructure that implements methods required by Local Channel Provider.
pvDatabase
Database of PVRecords. Has methods find, add, and remove.
Local Channel Provider
These two features will be the first phase. But only synchronous record processing will be supported.

Future phases of pvDatabaseCPP should include:

Install
This provides on-line add and delete.
Field support
Add ability to optionally add support to fields. In addition some of the basic support defined in pvIOCJava will also be implemented.
XML parser
This provides the ability to create record instances without writing any code.

The completion of each phase provides useful features that can be used without waiting for the completion of later phases. The rest of this document discusses only the first phase.

Features Required for localChannelProvider

pvCopy
Creates a PVStructure that contains a copy of an arbitary subset of the fields of another top level PVStructure. It can copy data between the two and maintains a bitSet that show which fields are changed.
monitor
This provides the ability to monitor changes to fields of a record.
PVRecord and PVDatabase
Defined below.
local ChannelProvider
This is the pvAccess package in pvIOCJava. The localChannelProvider will access data from PVRecords. It will implement all channel methods except channelRPC.

Minumum Features Required for pvRecord

The first phase will only implement record processing, i. e. the process method has to do everything itself without any generic field support. This will be sufficient for starting to implement services. The following are the minimium features required

PVDatabase
This holds a set of PVRecords. It has methods to find, add, and remove records.
PVRecord
This, and a set of related interfaces, provide the following:
PVStructure
PVRecord is a wrapper on a top level pvStructure.
Record locking
A record can be locked and unlocked. A record must be locked whenever data in the pvStructure is accessed.
Trapping data changes
A client can request to be notified when data in the pvStructure is modified. It can do this on a field by field basis.

The following sections provide a first attempt to describe the classes required for the first phase.

The last section gives a brief overview of the features provided by pvIOCJava.

database

The classes in pvDatabase.h implement a database of memory resident smart records. It describes the following classes:

PVRecord
This provides the methods required by localChannelProvider to implement Channel.
PVRecordField
PVRecordStructure
These wrap PVField and PVStructure so that pvCopy and monitor can be implemented.
PVRecordClient
This is called by anything that acceses PVRecord.
PVListener
This is implemented by anything that wants to trap calls to the PVRecord::message.
RecordProcessRequester
This is implemented by anything that calls PVRecord::queueProcessRequest.
RecordPutRequester
This is implemented by anything that calls PVRecord::queuePutRequest.
PVDatabase
This is a database of PVRecords.

Each class is described in a separate subsection.

C++ namespace and typedefs

namespace epics { namespace pvDatabase {

class PVRecord;
typedef std::tr1::shared_ptr<PVRecord> PVRecordPtr;

class PVRecordField;
typedef std::tr1::shared_ptr<PVRecordField> PVRecordFieldPtr;
typedef std::vector<PVRecordFieldPtr> PVRecordFieldPtrArray;
typedef std::tr1::shared_ptr<PVRecordFieldPtrArray> PVRecordFieldPtrArrayPtr;

class PVRecordStructure;
typedef std::tr1::shared_ptr<PVRecordStructure> PVRecordStructurePtr;

class PVRecordClient;
typedef std::tr1::shared_ptr<PVRecordClient> PVRecordClientPtr;

class PVListener;
typedef std::tr1::shared_ptr<PVListener> PVListenerPtr;

class RecordProcessRequester;
typedef std::tr1::shared_ptr<RecordProcessRequester> RecordProcessRequesterPtr;

class RecordPutRequester;
typedef std::tr1::shared_ptr<RecordPutRequester> RecordPutRequesterPtr;

class PVDatabase;
typedef std::tr1::shared_ptr<PVDatabase> PVDatabasePtr;

class PVRecord

NOTES:

  • This section uses the name record instead of "an instance of PVRecord".
  • Most clients will access a record via the local channel provider, i. e. via pvAccess. Thus this section is mainly of interest to the local channel provider and record implementers.
PVRecord Methods 
class PVRecord
     public epics::pvData::Requester,
     public std::tr1::enable_shared_from_this<PVRecord>
{
public:
    POINTER_DEFINITIONS(PVRecord);
    PVRecord(
        epics::pvData::String const & recordName,
        epics::pvData::PVStructurePtr const & pvStructure);
    virtual ~PVRecord();
    virtual void process(
        RecordProcessRequesterPtr const &recordProcessRequester,
        bool alreadyLocked) = 0;
    virtual bool isSynchronous() = 0;
    virtual bool requestImmediatePut(epics::pvData::PVFieldPtr const &pvField);
    virtual void immediatePutDone();
    virtual void destroy();
    epics::pvData::String getRecordName();
    PVRecordStructurePtr getPVRecordStructure();
    PVRecordFieldPtr findPVRecordField(
        epics::pvData::PVFieldPtr const & pvField);
    bool addRequester(epics::pvData::RequesterPtr const & requester);
    bool removeRequester(epics::pvData::RequesterPtr const & requester);
    void lock();
    void unlock();
    bool tryLock();
    void lockOtherRecord(PVRecordPtr const & otherRecord);
    void addPVRecordClient(PVRecordClientPtr const & pvRecordClient);
    void removePVRecordClient(PVRecordClientPtr const & pvRecordClient);
    void detachClients();
    void beginGroupPut();
    void endGroupPut();
    void queueProcessRequest(
        RecordProcessRequesterPtr const &recordProcessRequester);
    void dequeueProcessRequest(
        RecordProcessRequesterPtr const &recordProcessRequester);
    void queuePutRequest(
        RecordPutRequesterPtr const &recordPutRequester);
    void putDone(
        RecordPutRequesterPtr const &recordPutRequester);
    virtual epics::pvData::String getRequesterName();
    void message(
        epics::pvData::String const & message,
        epics::pvData::MessageType messageType);
    void message(
        PVRecordFieldPtr const & pvRecordField,
        epics::pvData::String const & message,
        epics::pvData::MessageType messageType);
    void toString(epics::pvData::StringBuilder buf);
    void toString(epics::pvData::StringBuilder buf,int indentLevel);
    //init MUST be called after derived class is constructed
    void init();

};

The methods are:

PVRecord
The constructor. It requires a recordName and a top level PVStructure.
~PVRecord
The destructor which must be virtual. A derived class must also have a virtual destructor.
process
Pure virtual method.

Derived classes must implement this method.

A client must only call this method when RecordProcessRequester::becomeProcessor is called as a result of a queueProcessRequest. A client can either call lock before calling processs or let process lock the record. If a client wants to put data into the record it should lock, put, and then call process.

If the record is synchronous, process will return only when all processing is complete. If the record is asynchronous then process arranges for some other thread to do the processing and returns.

When processing is done the record calls two client callbacks:

RecordProcessRequester::recordProcessResult
This is called with the record still locked. The clients can get data from the record.
RecordProcessRequester::recordProcessComplete
This is called with the record unlocked. The client can no longer access the record.
isSynchronous
Pure virtual method. Derived classes must implement this method.
requestImmediatePut
This is a virtual method.

The purpose is to allow the implementation to provide fields that allow a client to abort process. For example a motor record might provide a field stop

The default always returns false.

A record implementation can override the default and return true. In it does requestImmediatePut it returns with the record locked.

The client can change the value of the associated field and then call immediatePutDone

immediatePutDone
This is a virtual method.

The default does nothing.

Must be called by client as a result of a call to requestImmediatePut that returns true.

destroy
This is a virtual method.

The default does nothing.

getRecordName
Return the recordName.
getPVRecordStructure
Get the top level PVStructure.
findPVRecordField
Given a PVFieldPtr return the PVRecordFieldPtr for the field.
addRequester
Add a requester to receive messages.
removeRequester
Remove a message requester.
lock
unlock
Lock and Unlock the record. Any code accessing the data in the record or calling other PVRecord methods must have the record locked.
tryLock
If true then just like lock. If falseclient can not access record. A client can try to simultaneously hold the lock for more than two records by calling this method. But must be willing to accept failure.
lockOtherRecord
A client that holds the lock for one record can lock one other record. A client must not call this if the client already has the lock for more then one record.
addPVRecordClient
Every client that accesses the record must call this so that the client can be notified when the record is deleted.
removePVRecordClient
Client is no longer accessing the record.
detachClients
Ask all clients to detach from the record
addListener
Add a PVListener. This must be called before calling pvRecordField.addListener.
removeListener
Removes a listener. The listener will also be removed from all fields to which it is attached.
beginGroupPut
Begin a group of puts. This results in all registered PVListeners being called
endGroupPut
End a group of puts. This results in all registered PVListeners being called.
queueProcessRequest
Queue a process request.
dequeueProcessRequest
This must be called by record implementation after it has completed a process request.
queuePutRequest
Queue a put request.

This is for code that wants to change data in a record without processing. If RecordPutRequester::requestResult is called with result true then the record is locked and the client can make changes. When done the client must call putDone

putDone
Called by RecordPutRequester after changing values in record. This method unlocks the record
getRequesterName
virtual method of Requester
message
Can be called by implementation code. The message will be sent to every requester.
init
This method must be called by derived class after class is completely constructed.

class PVRecordField

class PVRecordField {
     public virtual epics::pvData::PostHandler,
     public std::tr1::enable_shared_from_this<PVRecordField>
public:
    POINTER_DEFINITIONS(PVRecordField);
    PVRecordField(
        epics::pvData::PVFieldPtr const & pvField,
        PVRecordStructurePtr const &parent,
        PVRecordPtr const & pvRecord);
    virtual ~PVRecordField();
    PVRecordStructurePtr getParent();
    epics::pvData::PVFieldPtr getPVField();
    epics::pvData::String getFullFieldName();
    epics::pvData::String getFullName();
    PVRecordPtr getPVRecord();
    bool addListener(PVListenerPtr const & pvListener);
    virtual void removeListener(PVListenerPtr const & pvListener);
    virtual void postPut();
    virtual void message(
        epics::pvData::String const & message,
        epics::pvData::MessageType messageType);
};

When PVRecord is created it creates a PVRecordField for every field in the PVStructure that holds the data. It has the following methods:

PVRecordField
The constructor.
~PVRecordField
The destructor.
getParent
Get the parent PVRecordStructure for this field.
getPVField
Get the PVField associated with this PVRecordField.
getFullFieldName
This gets the full name of the field, i.e. field,field,..
getFullName
This gets recordName plus the full name of the field, i.e. recordName.field,field,..
getPVRecord
Returns the PVRecord to which this field belongs.
addListener
Add A PVListener to this field. Whenever this field or any subfield if this field is modified the listener will be notified. PVListener is described below. Before a listener can call addListener it must first call PVRecord.registerListener.
removeListener
Remove a PVListener.
postPut
This is called by the code that implements the data interface. It is called whenever the put method is called.
message
Called by implementation code. It calls PVRecord::message after prepending the full fieldname.

class PVRecordStructure

class PVRecordStructure : public PVRecordField {
public:
    POINTER_DEFINITIONS(PVRecordStructure);
    PVRecordStructure(
        epics::pvData::PVStructurePtr const & pvStructure,
        PVRecordFieldPtrArrayPtr const & pvRecordField);
    virtual ~PVRecordStructure();
    PVRecordFieldPtrArrayPtr getPVRecordFields();
    epics::pvData::PVStructurePtr getPVStructure();
    virtual void removeListener(PVListenerPtr const & pvListener);
    virtual void postPut();
};

When PVRecord is created it creates a PVRecordStructure for every structure field in the PVStructure that holds the data. It has the following methods:

PVRecordStructure
The constructor.
~PVRecordStructure
The destructor.
getPVRecordFields
Get the PVRecordField array for the subfields
getPVStructure
Get the PVStructure for this field.
removeListener
Remove a PVListener.
postPut
This is called by the code that implements the data interface. It is called whenever the put method is called.

class PVRecordClient

class PVRecordClient {
    POINTER_DEFINITIONS(PVRecordClient);
    virtual ~PVRecordClient();
    virtual void detach(PVRecordPtr const & pvRecord);
};

where

~PVRecordClient
The destructor.
detach
The record is being removed from the master database,

class PVListener

class PVListener {
    virtual public PVRecordClient
public:
    POINTER_DEFINITIONS(PVListener);
    virtual ~PVListener();
    virtual void dataPut(PVRecordFieldPtr const & pvRecordField) = 0;
    virtual void dataPut(
        PVRecordStructurePtr const &
        requested,PVRecordFieldPtr const & pvRecordField) = 0;
    virtual void beginGroupPut(PVRecordPtr const & pvRecord) = 0;
    virtual void endGroupPut(PVRecordPtr const & pvRecord) = 0;
};

where

~PVListener
The destructor.
dataPut(PVRecordFieldPtr const & pvRecordField)
pvField has been modified. This is called if the listener has called PVRecordField::addListener for pvRecordField.
dataPut( PVRecordStructurePtr const & requested,PVRecordFieldPtr const & pvRecordField)
pvField has been modified. Requested is the field to which the requester issued a pvField-&addListener. This is called if the listener has called PVRecordField-&addListener for requested.
beginGroupPut
A related set of changes is being started.
endGroupPut
A related set of changes is done.

class RecordProcessRequester

class RecordProcessRequester :
    virtual public PVRecordClient,
    virtual public epics::pvData::Requester
{
public:
    POINTER_DEFINITIONS(RecordProcessRequester);
    virtual ~RecordProcessRequester();
    virtual void recordDestroyed() = 0;
    virtual void becomeProcessor() = 0;
    virtual void recordProcessResult(epics::pvData::Status status) = 0;
    virtual void recordProcessComplete() = 0;
};

where

~RecordProcessRequester
The destructor.
recordDestroyed
Record is being destroyed.
becomeProcessor
Called as a result of queueRequeProcessst. The requester can the call process.
recordProcessResult
The results of record processing. This is called with the record locked so that the process requester can access data from the record.
recordProcessComplete
Processing is complete. This is called with the record unlocked. If the process requester called process with leaveActive true then the requester must call setInactive.

class RecordPutRequester

class RecordPutRequester :
    virtual public PVRecordClient
{
public:
    POINTER_DEFINITIONS(RecordPutRequester);
    virtual ~RecordPutRequester();
    virtual void requestResult(bool result) = 0;
};

where

~RecordPutRequester
The destructor.
requestResult
Result of a call to queuePutRequest. If requestResult is false then the caller can not access the record. If requestResult is true then the record is locked and the caller can get and put data in the record. When done the caller must call PVRecord::putDone, which will unlock the record.

class PVDatabase

class PVDatabase : virtual public epics::pvData::Requester {
public:
    POINTER_DEFINITIONS(PVDatabase);
    static PVDatabasePtr getMaster();
    virtual ~PVDatabase();
    PVRecordPtr findRecord(epics::pvData::String const& recordName);
    bool addRecord(PVRecordPtr const & record);
    bool removeRecord(PVRecordPtr const & record);
private:
    PVDatabase();
};

where

getMaster
Get the master database. This is the database that localChannelProvider access.
~PVDatabase
The destructor.
findRecord
Find a record. An empty pointer is returned if the record is not in the database.
addRecord
Add a record to the database. If the record already exists it is not modified and false is returned.
removeRecord
Remove a record from the database. If the record was not in the database false is returned.

Local Channel Provider

Not yet described.

A brief description is that it must implement the following components of pvIOCJava:

pvCopy
monitor
pvAccess
See the next section for a description

Summary of Packages in pvIOCJAVA

The following are the direct sub packages of pvIOCJava/src/org/epics/pvioc:

pvCopy
This provides a copy of an arbitrary subset of the fields in a PVRecord. It also provides the ability to detect and report changes to fields. It is required for pvAccess.
monitor
This provides the ability to monitor changes to a PVRecord. It is required for pvAccess monitors.
pvAccess
The local implementation of Channel Provider and Channel. It is accessed by the remote pvAccess server and can also be accessed by code in the same IOC.
database
This defines and implements PVRecord, PVDatabase , and PVListener. It supports the basic feature required the implement a local Channel Provider.
support
This provides the ability to optionally attach code to any field of a pvRecord. It and several sub packages provide a set of standard support modules.
install
This provides the ability to dynamically initialize and add new PVRecords. It also provides the ability to dynamicall delete PVRecords.
xml
This provides the ability to configure record instances without writing code.
util
This is misnamed since it is code related to scanning.
pdrv
This is portDriver, which is a proposed sucessor to the asynManager component of asynDriver.
swtshell
This is shell that is can either run under the same process as a JavaIOC or as a remote shell. It is like a version of probe but for pvData/pvAccess. Almost all of it's features work in either local or remote mode. With a little more work all or it's features could work remotely. This should be done and then only remote mode should be supported. It can then be rewritten in a completely different language and using a complely different GUI framework.
caV3
This has two components:
ClientFactory
This is a small wrapper on top of the caV3 client support implemented by pvAccess. It allows code in the pvIOC to access V3Records via pvAccess.
ServerFactory
This is a caV3 server that allows a caV3 client to access a PVRecord. The Java implementation uses CAJ, which does most of the work. For now it will not be discussed in this document.
v3a
I do not know what this is.

In addition there is one class file JavaIOC.java. This is starting a IOC instance. This is not required for pvIOCCPP which is either a main or runs as part of a V3 IOC.