epics/pvDatabase
0
0
Fork 0
Code Issues Pull Requests Projects Wiki Activity

added longArrayGet and longArrayPut; much more testing

Browse Source
This commit is contained in:
Marty Kraimer
2013-11-20 10:41:29 -05:00
parent 3e1a405d01
commit ce116eefb8
14 changed files with 3280 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
documentation/pvDatabaseCPP.html
+183 -50
View File
@@ -38,7 +38,7 @@
<h1>pvDatabaseCPP</h1>
<!-- Maturity: Working Draft or Request for Comments, or Recommendation, and date. -->
<h2 class="nocount">EPICS v4 Working Group, Working Draft, 13-Nov-2013</h2>
<h2 class="nocount">EPICS v4 Working Group, Working Draft, 20-Nov-2013</h2>
<dl>
<dt>Latest version:</dt>
<dd><a
@@ -46,7 +46,7 @@
</dd>
<dt>This version:</dt>
<dd><a
href= "pvDatabaseCPP_20131113.html">pvDatabaseCPP20131113.html</a>
href= "pvDatabaseCPP_20131120.html">pvDatabaseCPP20131120.html</a>
</dd>
<dt>Previous version:</dt>
<dd><a
@@ -79,17 +79,27 @@ V4 control system programming environment:<br />
<h2 class="nocount">Status of this Document</h2>
<p>This is the 13-Nov-2013 version of of pvDatabaseCPP.</p>
<p>This is the 20-Nov-2013 version of of pvDatabaseCPP.</p>
</p>
<p>The previous version reported a problem with a queueSize of 1.
Further thought showed that queueSize must be &gt;= 2.
This ensures that the locking between poll and dataChanged ensures that these two
methods always use separate instances of pvStructure, changeBitSet, and overrunBitSet.</p>
<p>All channel methods except channelRPC, which is implemented
by pvAccess, have been implemented.
This project is ready for alpha users.
</p>
<p>Since the last version the longArrayGet and longArrayPut examples were added.
More testing for monitor queues and memory leaks was done.
Everything looks good!!
But there are still two unresolved problems:
<dl>
<dt>memory leak</dt>
<dd>arrayPerformanceMain shows a slight memory leak at termination.</dd>
<dt>channel destroy and recreate</dt>
<dd>longArrayGet and longArrayPut fail if the channel is destroyed and
immediately recreated.
</dd>
</dl>
</p>
<p>Future enhancements in priority order:</p>
<dl>
<dt>Separate example that also has pvaSrv</dt>
@@ -1582,10 +1592,28 @@ where
<dd>This creates the example using the remote channel provider.</dd>
</dl>
<h2>Array Performance</h2>
<p>Two main programs are available for testing the performance of large
arrays: arrayPerformanceMain and longArrayMonitorMain.
Each has support for <b>-help</b>.</p>
<h2>Array Performance and Memory Example</h2>
<p>This section describes main programs that demonstrate performance
of large arrays and can also be used to check for memory leaks.
Checking for memory leaks can be accomplished by running the programs with valgrind
or some other memory check program.
</p>
<h3>Brief Summary</h3>
<p>The programs are:</p>
<dl>
<dt>arrayPerformanceMain</dt>
<dd>This is server and also a configurable number of longArrayMonitor clients.
The clients can use either the local or
remote providers. The moitor code is the same code that is used by longArrayMonitorMain.
</dd>
<dt>longArrayMonitorMain</dt>
<dd>Remote client that monitors the array served by arrayPerformanceMain.</dd>
<dt>longArrayGetMain</dt>
<dd>Remote client that uses channelGet to access the array served by arrayPerformanceMain.</dd>
<dt>longArrayPutMain</dt>
<dd>Remote client that uses channelPut to access the array served by arrayPerformanceMain.</dd>
<dl>
<p>Each has support for <b>-help</b>.</p>
<pre>
mrk&gt; pwd
/home/hg/pvDatabaseCPP-md
@@ -1593,33 +1621,97 @@ mrk&gt; bin/linux-x86_64/arrayPerformanceMain -help
arrayPerformanceMain recordName size delay providerName nMonitor queueSize waitTime
default
arrayPerformance arrayPerformance 10000000 0.0001 local 1 2 0.0
mrk&gt; bin/linux-x86_64/longArrayMonitorMain -help
longArrayMonitorMain channelName queueSize waitTime
default
longArrayMonitorMain arrayPerformance 2 0.0
mrk&gt; bin/linux-x86_64/longArrayGetMain -help
longArrayGetMain channelName iterBetweenCreateChannel iterBetweenCreateChannelGet delayTime
default
longArrayGetMain arrayPerformance 0 0 1
mrk&gt; bin/linux-x86_64/longArrayPutMain -help
longArrayPutMain channelName arraySize iterBetweenCreateChannel iterBetweenCreateChannelPut delayTime
default
longArrayPutMain arrayPerformance 10 0 0 1
mrk&gt;
</pre>
<h3>Example output</h3>
<p><b>Note:</b> These may fail if run on a platform that does not have sufficent memory,</p>
<p>To see an example just execute the following commands in four different terminal windows:</p>
<pre>
bin/linux/&lt;arch&gt;/arrayPerformanceMain
bin/linux/&lt;arch&gt;/longArrayMonitorMain
bin/linux/&lt;arch&gt;/longArrayGetMain
bin/linux/&lt;arch&gt;/longArrayPutMain
</pre>
<p>Each program generates a report every second when it has somthing to report.
Examples are:
<pre>
mrk&gt; bin/linux-x86_64/arrayPerformanceMain
arrayPerformance arrayPerformance 10000000 0.0001 local 1 2 0.0
arrayPerformance arrayPerformance 10000000 0.0001 local 1 2 0
...
first 526 last 526 sum 5260000000 elements/sec 499.337million changed {1, 2} overrun {}
first 527 last 527 sum 5270000000 elements/sec 568.729million changed {1, 2} overrun {}
first 528 last 528 sum 5280000000 elements/sec 505.281million changed {1, 2} overrun {}
first 529 last 529 sum 5290000000 elements/sec 506.33million changed {1, 2} overrun {}
first 530 last 530 sum 5300000000 elements/sec 574.022million changed {1, 2} overrun {}
arrayPerformance value 532 time 1.00839 iterations/sec 51.5672 elements/sec 515.672million
first 531 last 531 sum 5310000000 elements/sec 484.185million changed {1, 2} overrun {}
first 532 last 532 sum 5320000000 elements/sec 531.4million changed {1, 2} overrun {}
first 533 last 533 sum 5330000000 elements/sec 428.229million changed {1, 2} overrun {}
first 534 last 534 sum 5340000000 elements/sec 427.387million changed {1, 2} overrun {}
first 535 last 535 sum 5350000000 elements/sec 443.669million changed {1, 2} overrun {}
monitors/sec 66 first 131 last 131 changed {1, 2} overrun {} megaElements/sec 656.999
arrayPerformance value 132 time 1.00486 Iterations/sec 65.681 megaElements/sec 656.81
monitors/sec 66 first 197 last 197 changed {1, 2} overrun {} megaElements/sec 656.304
arrayPerformance value 198 time 1.00563 Iterations/sec 65.6307 megaElements/sec 656.307
monitors/sec 66 first 263 last 263 changed {1, 2} overrun {} megaElements/sec 654.824
...
</pre>
<pre>
mrk&gt; bin/linux-x86_64/longArrayMonitorMain
longArrayMonitorMain arrayPerformance 2 0
...
monitors/sec 6 first 2357 last 2357 changed {1, 2} overrun {} megaElements/sec 68.6406
monitors/sec 13 first 2385 last 2385 changed {1, 2} overrun {} megaElements/sec 118.72
monitors/sec 9 first 2418 last 2418 changed {1, 2} overrun {1, 2} megaElements/sec 85.0984
...
</pre>
<pre>
mrk&gt; bin/linux-x86_64/longArrayPutMain
longArrayPutMain arrayPerformance 10 0 0 1
...
put numChannelPut 0 time 1.00148 Elements/sec 79.8819
put numChannelPut 1 time 1.00176 Elements/sec 79.8598
...
</pre>
<pre>
mrk&gt; bin/linux-x86_64/longArrayGetMain
longArrayGetMain arrayPerformance 0 0 1
...
get kiloElements/sec 7384.61
get kiloElements/sec 8726.34
...
</pre>
<h3>arrayPerformance</h3>
<p>The arguments for arrayPerforamanceMain are:</p>
<dl>
<dt>recordName</dt>
<dd>The name for the arrayPerform record.</dd>
<dt>size</dt>
<dd>The size for the long array of the value field.</dd>
<dt>delay</dt>
<dd>The time in seconds to sleep after each iteration.</dd>
<dt>providerName</dt>
<dd>The name of the channel provider for the longArrayMonitors
created by the main program. This must be either <b>local</b>
or <b>pvAccess</b>.
</dd>
<dt>nMonitor</dt>
<dd>The number of longArrayMonitors to create.</dd>
<dt>queueSize</dt>
<dd>The queueSize for the element queue.
A value less than 1 will become 1.
</dd>
<dt>waitTime</dt>
<dd>The time that longArrayMonitor will sleep after poll returns a monitorElement.</dd>
</dl>
<p>
This creates a PVRecord that has the structure:.
arrayPerformance creates a PVRecord that has the structure:.
<pre>
recordName
long[] value
@@ -1631,7 +1723,7 @@ Thus it holds an array of 64 bit integers.</p>
until the record is destroyed executing the following algorithm:</p>
<dl>
<dt>report</dt>
<dd>At least once a second it produces a report.
<dd>Once a second it produces a report.
In the above example output each line starting with
<b>ArrayPerformance</b> is an arrayPerformance report.
</dd>
@@ -1655,40 +1747,66 @@ until the record is destroyed executing the following algorithm:</p>
<dt>delay</dt>
<dd>If delay is greater than zero epicsThreadSleep is called.</dd>
</dl>
<p>The arguments for arrayPerforamanceMain are:</p>
<dl>
<dt>recordName</dt>
<dd>The name for the arrayPerform record.</dd>
<dt>size</dt>
<dd>The size for the long array of the value field.</dd>
<dt>delay</dt>
<dd>The time in seconds to sleep after each iteration.</dd>
<dt>providerName</dt>
<dd>The name of the channel provider for the longArrayMonitors
created by the main program. This must be either <b>local</b>
or <b>pvAccess</b>.
</dd>
<dt>nMonitor</dt>
<dd>The number of longArrayMonitors to create.</dd>
<dt>queueSize</dt>
<dd>The queueSize for the element queue.
A value less than 1 will become 1.
</dd>
<dt>waitTime</dt>
<dd>The time that longArrayMonitor will sleep after poll returns a monitorElement.</dd>
</dl>
<h3>longArrayMonitor</h3>
<p>This is a pvAccess client that monitors an arrayPerformance record.
It generates a report for each monitor event.
In the example output shown above each line starting with <b>first</b>
is a report.</p>
It generates a report every second showing how many elements has received.
For every monitor it also checks that the number of alements is &gt;0 and the
the first element equals the last element. It reports an error if either
of these conditions is not true.</p>
<p>The arguments for longArrayMonitorMain are:</p>
<dl>
<dt>channelName</dt>
<dd>The name for the arrayPerform record.</dd>
<dt>queueSize</dt>
<dd>The queueSize. Note that any size &lt;2 is made 2.</dd>
<dt>waitTime</dt>
<dd>The time to wait after a poll request returns a monitorElement.
This can be used to force an overrun of the client even if there is no
overrun on the server.</dd>
</dl>
<h3>longArrayGet</h3>
<p>This is a pvAccess client that uses channelGet to access an arrayPerformance record.
Every second it produces a report.</p>
<p>The arguments for longArrayGetMain are:</p>
<dl>
<dt>channelName</dt>
<dd>The name for the arrayPerform record.</dd>
<dt>iterBetweenCreateChannel</dt>
<dd>The number of iterations between destroying and recreating the channel.
A value of 0 means never destroy and recreate.
</dd>
<dt>iterBetweenCreateChannelGet</dt>
<dd>The number of iterations between destroying and recreating the channelGet.
A value of 0 means never destroy and recreate.
</dd>
<dt>delayTime</dt>
<dd>The time to dalay between gets.</dd>
</dl>
<h3>longArrayPut</h3>
<p>This is a pvAccess client that uses channelPut to access an arrayPerformance record.
Every second it produces a report.</p>
<p>The arguments for longArrayPutMain are:</p>
<dl>
<dt>channelName</dt>
<dd>The name for the arrayPerform record.</dd>
<dt>arraySize</dt>
<dd>The capacity and length of the array to put to the server.</dd>
<dt>iterBetweenCreateChannel</dt>
<dd>The number of iterations between destroying and recreating the channel.
A value of 0 means never destroy and recreate.
</dd>
<dt>iterBetweenCreateChannelPut</dt>
<dd>The number of iterations between destroying and recreating the channelPut.
A value of 0 means never destroy and recreate.
</dd>
<dt>delayTime</dt>
<dd>The time to dalay between gets.</dd>
</dl>
<h3>Some results</h3>
<h4>array performance</h4>
<p>The results were from my laptop.
It has a 2.2Ghz intel core i7 with 4Gbytes of memory.
The operating system is linux fedora 16.</p>
@@ -1724,6 +1842,21 @@ The next section has an example that demonstrates what happens.
Note that if the array size is small enough to fit in the local cache then running longArrayMonitor
has almost no effect of arrayPerforance.
</p>
<h4>memory leaks</h4>
<p>Running longArrayMonitorMain, longArrayPutMain, and longArrayGetMain
under valgrind shows no memory leaks.</p>
<p>arrayPerformanceMain shows the following:</p>
<pre>
==9125== LEAK SUMMARY:
==9125== definitely lost: 0 bytes in 0 blocks
==9125== indirectly lost: 0 bytes in 0 blocks
==9125== possibly lost: 576 bytes in 2 blocks
</pre>
<p>The possibly leaked is either 1 or 2 blocks.
It seems to be the same if clients are connected.
</p>
<h4>iterBetweenCreateChannel</h4>
<p>If this is not zero then the attempt to destroy and recreate the channel fails.</p>
<h2>Vector Performance</h2>
<p>This example demonstrates how array size effects performance.
The example is run as:</p>