epics/pvDatabase
0
0
Fork 0
Code Issues Pull Requests Projects Wiki Activity

changed monitor queue implementation

Browse Source
This commit is contained in:
Marty Kraimer
2013-11-13 09:56:18 -05:00
parent 3039e1cdb0
commit 3e1a405d01
5 changed files with 1854 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
documentation/pvDatabaseCPP.html
View File

@ -38,7 +38,7 @@
<h1>pvDatabaseCPP</h1>
<!-- Maturity: Working Draft or Request for Comments, or Recommendation, and date. -->
<h2 class="nocount">EPICS v4 Working Group, Working Draft, 12-Nov-2013</h2>
<h2 class="nocount">EPICS v4 Working Group, Working Draft, 13-Nov-2013</h2>
<dl>
<dt>Latest version:</dt>
<dd><a
@ -46,11 +46,11 @@
</dd>
<dt>This version:</dt>
<dd><a
href= "pvDatabaseCPP_20131112.html">pvDatabaseCPP20131112.html</a>
href= "pvDatabaseCPP_20131113.html">pvDatabaseCPP20131113.html</a>
</dd>
<dt>Previous version:</dt>
<dd><a
href= "pvDatabaseCPP_20130904.html">pvDatabaseCPP20130904.html</a>
href= "pvDatabaseCPP_20131112.html">pvDatabaseCPP20131112.html</a>
</dd>
<dt>Editors:</dt>
<dd>Marty Kraimer, BNL</dd>
@ -79,26 +79,12 @@ V4 control system programming environment:<br />
<h2 class="nocount">Status of this Document</h2>
<p>This is the 12-Nov-2013 version of of pvDatabaseCPP.</p>
<p>This is the 13-Nov-2013 version of of pvDatabaseCPP.</p>
</p>
<p><b>Problem</b> When arrayPerformance is run with a small array and queueSize 1 it fails.
For example:
<pre>
mrk&gt; bin/linux-x86_64/arrayPerformanceMain arrayPerformance 50 0.00 local 1 1 0.001
arrayPerformance arrayPerformance 50 0 local 1 1 0
...
first 2245711 last 2245711 sum 112285550 elements/sec 0.0452426million changed {1, 2} overrun {}
first 2246087 last 2246087 sum 112304350 elements/sec 0.0451597million changed {1, 2} overrun {}
arrayPerformance value 2246117 time 1 iterations/sec 336447 elements/sec 16.8223million
first 2246448 last 2246448 sum 112322400 elements/sec 0.0450084million changed {1, 2} overrun {}
first 2246827 last 2246827 sum 112341350 elements/sec 0.0446581million changed {1, 2} overrun {}
...
first 2282180 last 2282180 sum 114109000 elements/sec 0.0447461million changed {1, 2} overrun {}
first 2282540 last 2282540 sum 114127000 elements/sec 0.044695million changed {1, 2} overrun {}
Segmentation fault (core dumped)
</pre>
I do not know why and am stuck about what to do.
Looking at SingleElementQueue I do not see any problem.</p>
<p>The previous version reported a problem with a queueSize of 1.
Further thought showed that queueSize must be &gt;= 2.
This ensures that the locking between poll and dataChanged ensures that these two
methods always use separate instances of pvStructure, changeBitSet, and overrunBitSet.</p>
<p>All channel methods except channelRPC, which is implemented
by pvAccess, have been implemented.
@ -118,14 +104,6 @@ This project is ready for alpha users.
<dt>Monitor Algorithms</dt>
<dd>Monitor algorithms have not been implemented.
Thus all monitors are onPut.</dd>
<dt>Testing</dt>
<dd>Needs more testing.
Also none of the examples that use pvAccess can be run with gdb.
</dd>
<dt>High Performance Issues.</dt>
<dd>When arrayPerformance is run with size=5000 and delay really small or zero,
either arrayPerformance or longArrayMonitor will sometimes crash while running or at termination.
Also, in the same environment, longArrayMonitor frequently gets the array with size = 0</dd>
<dt>Create regression tests</dt>
<dd>Currently only examples exist and have been used for testing.</dd>
</dl>
@ -1614,7 +1592,7 @@ mrk&gt; pwd
mrk&gt; bin/linux-x86_64/arrayPerformanceMain -help
arrayPerformanceMain recordName size delay providerName nMonitor queueSize waitTime
default
arrayPerformance arrayPerformance 50000000 0.01 local 1 2 0.0
arrayPerformance arrayPerformance 10000000 0.0001 local 1 2 0.0
mrk&gt; bin/linux-x86_64/longArrayMonitorMain -help
longArrayMonitorMain channelName queueSize waitTime
default
@ -1624,19 +1602,19 @@ mrk&gt;
<h3>Example output</h3>
<pre>
mrk&gt; bin/linux-x86_64/arrayPerformanceMain
arrayPerformance arrayPerformance 50000000 0.01 local 1 2 0.0
arrayPerformance arrayPerformance 10000000 0.0001 local 1 2 0.0
...
first 5 last 5 sum 250000000 elements/sec 51.5164million changed {0} overrun {}
arrayPerformance value 7 time 1.08532 iterations/sec 6.44973 elements/sec 322.487million
first 6 last 6 sum 300000000 elements/sec 493.414million changed {1, 2} overrun {}
first 7 last 7 sum 350000000 elements/sec 172.566million changed {1, 2} overrun {}
first 8 last 8 sum 400000000 elements/sec 515.064million changed {1, 2} overrun {}
first 9 last 9 sum 450000000 elements/sec 515.359million changed {1, 2} overrun {}
arrayPerformance value 11 time 1.10601 iterations/sec 3.6166 elements/sec 180.83million
first 10 last 10 sum 500000000 elements/sec 80.5679million changed {1, 2} overrun {}
first 11 last 11 sum 550000000 elements/sec 499.455million changed {1, 2} overrun {}
first 12 last 12 sum 600000000 elements/sec 504.324million changed {1, 2} overrun {}
first 13 last 13 sum 650000000 elements/sec 101.517million changed {1, 2} overrun {}
first 526 last 526 sum 5260000000 elements/sec 499.337million changed {1, 2} overrun {}
first 527 last 527 sum 5270000000 elements/sec 568.729million changed {1, 2} overrun {}
first 528 last 528 sum 5280000000 elements/sec 505.281million changed {1, 2} overrun {}
first 529 last 529 sum 5290000000 elements/sec 506.33million changed {1, 2} overrun {}
first 530 last 530 sum 5300000000 elements/sec 574.022million changed {1, 2} overrun {}
arrayPerformance value 532 time 1.00839 iterations/sec 51.5672 elements/sec 515.672million
first 531 last 531 sum 5310000000 elements/sec 484.185million changed {1, 2} overrun {}
first 532 last 532 sum 5320000000 elements/sec 531.4million changed {1, 2} overrun {}
first 533 last 533 sum 5330000000 elements/sec 428.229million changed {1, 2} overrun {}
first 534 last 534 sum 5340000000 elements/sec 427.387million changed {1, 2} overrun {}
first 535 last 535 sum 5350000000 elements/sec 443.669million changed {1, 2} overrun {}
...
</pre>
<h3>arrayPerformance</h3>
@ -1726,17 +1704,13 @@ mrk&gt; pwd
/home/hg/pvDatabaseCPP-md
mrk&gt; bin/linux-x86_64/arrayPerformanceMain
</pre>
<p>This means that the array will hold 50 million elements.
<p>This means that the array will hold 10 million elements.
The delay will be a millisecond.
There will be a single monitor and it will connect directly
to the local channelProvider, i. e. it will not use any network
connection.</p>
<p>The report shows that arrayPerformance can perform about 4 iterations per second
and is putting about 200million elements per second.
Since each element is an int64 this means about 1.6gigaBytes per second.
</p>
<p>If queueSize is set to 1 thereport shows that arrayPerformance can perform about 10 iterations
per second and is putting about 500million elements per second.
<p>The report shows that arrayPerformance can perform about 50 iterations per second
and is putting about 500million elements per second.
Since each element is an int64 this means about 4gigaBytes per second.
</p>
<p>When no monitors are requested and a remote longArrayMonitorMain is run:<p>
@ -1745,11 +1719,9 @@ mr&gt; pwd
/home/hg/pvDatabaseCPP-md
mrk&gt; bin/linux-x86_64/longArrayMonitorMain
</pre>
<p>The performance drops to about 300million elements per second.
In addition the time between reports varies from just over 1 second to 1.3 seconds.
The reason is contention for transfering data between main memory and local caches.
<p>The performance drops to about 25 interations per second and 250 million elements per second.
The next section has an example that demonstrates what happens.
Note that if the array size is small enough to fix in the local cache then running longArrayMonitor
Note that if the array size is small enough to fit in the local cache then running longArrayMonitor
has almost no effect of arrayPerforance.
</p>
<h2>Vector Performance</h2>

1803
documentation/pvDatabaseCPP_20131113.html Normal file
View File

File diff suppressed because it is too large Load Diff

4
example/arrayPerformance/arrayPerformanceMain.cpp
View File

@ -40,8 +40,8 @@ int main(int argc,char *argv[])
bool result(false);
String recordName;
recordName = "arrayPerformance";
size_t size = 50000000;
double delay = .01;
size_t size = 10000000;
double delay = .0001;
String providerName("local");
size_t nMonitor = 1;
int queueSize = 2;

127
src/pvAccess/monitorFactory.cpp
View File

@ -37,8 +37,6 @@ static ConvertPtr convert = getConvert();
class ElementQueue;
typedef std::tr1::shared_ptr<ElementQueue> ElementQueuePtr;
class SingleElementQueue;
typedef std::tr1::shared_ptr<SingleElementQueue> SingleElementQueuePtr;
class MultipleElementQueue;
typedef std::tr1::shared_ptr<MultipleElementQueue> MultipleElementQueuePtr;
@ -64,31 +62,6 @@ protected:
}
};
class SingleElementQueue :
public ElementQueue
{
public:
POINTER_DEFINITIONS(SingleElementQueue);
virtual ~SingleElementQueue(){}
SingleElementQueue(
MonitorLocalPtr const &monitorLocal);
virtual void destroy(){}
virtual Status start();
virtual Status stop();
virtual bool dataChanged(bool firstMonitor);
virtual MonitorElementPtr poll();
virtual void release(MonitorElementPtr const &monitorElement);
private:
std::tr1::weak_ptr<MonitorLocal> monitorLocal;
PVStructurePtr pvCopyStructure;
MonitorElementPtr monitorElement;
bool gotMonitor;
BitSetPtr changedBitSet;
BitSetPtr overrunBitSet;
BitSetPtr dataChangedBitSet;
BitSetPtr dataOverrunBitSet;
};
typedef Queue<MonitorElement> MonitorElementQueue;
typedef std::tr1::shared_ptr<MonitorElementQueue> MonitorElementQueuePtr;
@ -312,25 +285,22 @@ bool MonitorLocal::init(PVStructurePtr const & pvRequest)
}
pvCopyMonitor = pvCopy->createPVCopyMonitor(getPtrSelf());
// MARTY MUST IMPLEMENT periodic
if(queueSize<2) {
queue = SingleElementQueuePtr(new SingleElementQueue(getPtrSelf()));
} else {
std::vector<MonitorElementPtr> monitorElementArray;
monitorElementArray.reserve(queueSize);
size_t nfields = 0;
for(size_t i=0; i<queueSize; i++) {
PVStructurePtr pvStructure = pvCopy->createPVStructure();
if(nfields==0) nfields = pvStructure->getNumberFields();
MonitorElementPtr monitorElement(
new MonitorElement(pvStructure));
monitorElementArray.push_back(monitorElement);
}
MonitorElementQueuePtr elementQueue(new MonitorElementQueue(monitorElementArray));
queue = MultipleElementQueuePtr(new MultipleElementQueue(
getPtrSelf(),
elementQueue,
nfields));
if(queueSize<2) queueSize = 2;
std::vector<MonitorElementPtr> monitorElementArray;
monitorElementArray.reserve(queueSize);
size_t nfields = 0;
for(size_t i=0; i<queueSize; i++) {
PVStructurePtr pvStructure = pvCopy->createPVStructure();
if(nfields==0) nfields = pvStructure->getNumberFields();
MonitorElementPtr monitorElement(
new MonitorElement(pvStructure));
monitorElementArray.push_back(monitorElement);
}
MonitorElementQueuePtr elementQueue(new MonitorElementQueue(monitorElementArray));
queue = MultipleElementQueuePtr(new MultipleElementQueue(
getPtrSelf(),
elementQueue,
nfields));
// MARTY MUST IMPLEMENT algorithm
monitorRequester->monitorConnect(
Status::Ok,
@ -402,73 +372,6 @@ MonitorAlgorithmCreatePtr MonitorFactory::getMonitorAlgorithmCreate(
return nullMonitorAlgorithmCreate;
}
SingleElementQueue::SingleElementQueue(
MonitorLocalPtr const &monitorLocal)
: monitorLocal(monitorLocal),
pvCopyStructure(monitorLocal->getPVCopy()->createPVStructure()),
monitorElement(new MonitorElement(pvCopyStructure)),
gotMonitor(false),
changedBitSet(new BitSet(pvCopyStructure->getNumberFields())),
overrunBitSet(new BitSet(pvCopyStructure->getNumberFields())),
dataChangedBitSet(new BitSet(pvCopyStructure->getNumberFields())),
dataOverrunBitSet(new BitSet(pvCopyStructure->getNumberFields()))
{
}
Status SingleElementQueue::start()
{
gotMonitor = true;
changedBitSet->clear();
overrunBitSet->clear();
dataChangedBitSet->clear();
dataOverrunBitSet->clear();
MonitorLocalPtr ml = monitorLocal.lock();
if(ml==NULL) return wasDestroyedStatus;
ml->getPVCopyMonitor()->startMonitoring(
changedBitSet,
overrunBitSet);
return Status::Ok;
}
Status SingleElementQueue::stop()
{
return Status::Ok;
}
bool SingleElementQueue::dataChanged(bool firstMonitor)
{
MonitorLocalPtr ml = monitorLocal.lock();
if(ml==NULL) return false;
if(firstMonitor) changedBitSet->set(0);
ml->getPVCopy()->updateCopyFromBitSet(pvCopyStructure,changedBitSet);
(*dataChangedBitSet) |= (*changedBitSet);
(*dataOverrunBitSet) |= (*overrunBitSet);
changedBitSet->clear();
overrunBitSet->clear();
gotMonitor = true;
return true;
}
MonitorElementPtr SingleElementQueue::poll()
{
MonitorLocalPtr ml = monitorLocal.lock();
if(ml==NULL) return NULLMonitorElement;
if(!gotMonitor) return NULLMonitorElement;
gotMonitor = false;
convert->copyStructure(pvCopyStructure,monitorElement->pvStructurePtr);
BitSetUtil::compress(dataChangedBitSet,pvCopyStructure);
BitSetUtil::compress(dataOverrunBitSet,pvCopyStructure);
(*monitorElement->changedBitSet) = (*dataChangedBitSet);
(*monitorElement->overrunBitSet) = (*dataOverrunBitSet);
dataChangedBitSet->clear();
dataOverrunBitSet->clear();
return monitorElement;
}
void SingleElementQueue::release(MonitorElementPtr const &monitorElement)
{
}
MultipleElementQueue::MultipleElementQueue(
MonitorLocalPtr const &monitorLocal,
MonitorElementQueuePtr const &queue,

9
src/pvAccess/pvCopy.cpp
View File

@ -182,7 +182,14 @@ void PVCopy::initCopy(
{
bitSet->clear();
bitSet->set(0);
updateCopyFromBitSet(copyPVStructure,bitSet);
pvRecord->lock();
try {
updateCopyFromBitSet(copyPVStructure,bitSet);
} catch(...) {
pvRecord->unlock();
throw;
}
pvRecord->unlock();
}
void PVCopy::updateCopySetBitSet(