Added contrib directory
SVN revision: 780
This commit is contained in:
Executable
+85
@@ -0,0 +1,85 @@
|
||||
NAME
|
||||
elog2sql - copy an elog logbook to a MySQL database
|
||||
|
||||
SYNOPSIS
|
||||
parsecfg.pl - translate the elogd.cfg file to a MySQL database template
|
||||
|
||||
parselog.pl - translate a elog logbook to a MySQL insert template file
|
||||
|
||||
DESCRIPTION
|
||||
elog2sql was created to help translate logbooks created by the program
|
||||
"elog" (http://midas.psi.ch/elog/) from the native elog flat file format
|
||||
to a MySQL database. I found elog to be one of the easiest, yet
|
||||
feature-rich web-based programs for maintaining journals and logbooks;
|
||||
The program is fast, the administration and setup are easy and simple,
|
||||
and the features are extremly well suited for the application. However,
|
||||
I was bothered by all-in-one (web server and application) design of
|
||||
elog, rather then the typical use of Apache/PHP/MySQL platform for such
|
||||
an application. IMHO, having elog on such a platform will allow easier
|
||||
and simpler integration of new functions and extensions for elog. The
|
||||
long-term goal is to create a version of elog functionality based on the
|
||||
LAMP platform.
|
||||
|
||||
Therefore, I created a set of perl scripts that will allow the
|
||||
translation of elog logbooks into a MySQL database. The design and
|
||||
implementation of these scripts are a simple one, and allow the one-time
|
||||
copying of a set of logbooks.
|
||||
|
||||
The elog2sql toolkit consists of two scripts. The first script,
|
||||
parsecfg.pl, reads a elogd.cfg, and creates a sql file that will create
|
||||
a set of db tables corresponding to elog logbooks. The second script,
|
||||
parselog.pl, takes a set of elog logfiles, and creates a sql file that
|
||||
will enter the logbook data into the database. The result is a copy of
|
||||
the elog logbook that can used as desired inside the framework of MySQL.
|
||||
Attachments are handled by inserting an entry of the attachment name
|
||||
into an seperate attachment table. This allows multiple attachments per
|
||||
entry.
|
||||
|
||||
USAGE
|
||||
1) Create a MySQL database (example: elog)
|
||||
2) Create the sql database templates
|
||||
parsecfg.pl < elogd.cfg >elog.sql
|
||||
|
||||
3) Create the MySQL tables
|
||||
mysql -u user -p elog <elog.sql
|
||||
|
||||
4) Translate an elog logbook (example: journal)
|
||||
cat logbook/journal/*log | parselog.pl journal >journal.sql
|
||||
|
||||
5) Load the table into MySQL
|
||||
mysql -u user -p elog <journal.sql
|
||||
|
||||
6) Repeat steps 4 and 5 for each logbook directory.
|
||||
ISSUE and BUGS
|
||||
this is a quick and dirty programming job, so I expect this to be a
|
||||
brittle program. Use at your own risk, and check the resulting output
|
||||
before using mysql.
|
||||
|
||||
Elog is sloppy on logbook formats - Extra attributes could remain in a
|
||||
elog logbook file if the user created a logbook, and then deleted
|
||||
attributes later on. These extra attribute entries will break the sql
|
||||
file, and you will need to delete them from the logbook or from the sql
|
||||
file manually.
|
||||
|
||||
elog2sql only tranlates attributes to a fixed element datatype
|
||||
(varchar). It doesn't recognize the elog "options" command, and is
|
||||
braindead on things like "option boolean".
|
||||
|
||||
elog2sql was tested only on limited datasets (my logbooks), and as such,
|
||||
did not test all possible elogd.cfg configuration commands. Since the
|
||||
logbook format is not specified in the documentation, I had to work on
|
||||
the information in my logbooks. Any changes to this format by the Elog
|
||||
develops may break this code.
|
||||
|
||||
elog2sql was tested on elog version 2.3.8. YMMV on other versions.
|
||||
|
||||
LICENSE
|
||||
The author issues this code under the GPL. (See GPL.txt). Use this code
|
||||
at your own risk.
|
||||
|
||||
AUTHOR
|
||||
dave@davidfannin.com
|
||||
|
||||
VERSION
|
||||
Date: 2003-07-02 Version 0.99
|
||||
|
||||
Executable
+30
@@ -0,0 +1,30 @@
|
||||
# elog2sql
|
||||
# global configuration vars
|
||||
|
||||
|
||||
#
|
||||
# set the database table names to "$table_prefix<logbookname>"
|
||||
# This is used in case in only allowed one database
|
||||
# and want to avoid table naming conflicts.
|
||||
# normally, keep the default to ("")
|
||||
|
||||
$table_prefix = "" ;
|
||||
|
||||
|
||||
#
|
||||
# set the default element type for mysql
|
||||
# (each elogd attribute is set to this type)
|
||||
#
|
||||
|
||||
$default_element_type = "varchar(100)" ;
|
||||
|
||||
#
|
||||
# set the debug flat : set to 1 if you want to see debugging output.
|
||||
#
|
||||
$debug = 0 ;
|
||||
|
||||
|
||||
#
|
||||
# Do not change the variables after this line
|
||||
#
|
||||
$version = "0.99" ;
|
||||
Executable
+30
@@ -0,0 +1,30 @@
|
||||
elog2sql was created to help translate logbooks created by the program
|
||||
``elog'' from the native elog flat file format to a MySQL database. I had a
|
||||
need to have the elog data in a database, and it appears from the forum that
|
||||
several others had a similar need.
|
||||
|
||||
I created a set of perl scripts that will allow the translation of elog
|
||||
logbooks into a MySQL database. The design and implementation of these
|
||||
scripts are a simple one, and allow the one-time copying of a set of logbooks.
|
||||
|
||||
The elog2sql toolkit consists of two scripts. The first script, parsecfg.pl,
|
||||
reads a elogd.cfg, and creates a sql file that will create a set of db
|
||||
tables corresponding to elog logbooks. The second script, parselog.pl, takes
|
||||
a set of elog logfiles, and creates a sql file that will enter the logbook
|
||||
data into the database. The result is a copy of the elog logbook that can
|
||||
used as desired inside the framework of MySQL. Attachments are handled by
|
||||
inserting an entry of the attachment name into an seperate attachment table.
|
||||
This allows multiple attachments per entry.
|
||||
|
||||
You can download the elog2sql program archive at
|
||||
http://www.davidfannin.com/elog2sql/elog2sql.tar.gz . It contains the
|
||||
scripts and basic documentation. You can read the man page at
|
||||
http://www.davidfannin.com/elog2sql/index.html
|
||||
|
||||
|
||||
I have also uploaded a copy of the archive here.
|
||||
|
||||
|
||||
email me for questions or comments.
|
||||
|
||||
-- Fred Hooper <fhooper@sushisoft.com>
|
||||
Executable
+1
@@ -0,0 +1 @@
|
||||
javascript:var t=document.title;var u=document.location.href;var s=document.getSelection();var d=window.open('','d');with(d.document){write('<form method=\'POST\' action=\'http://ELOG_URL/LOGBOOK/?cmd=New\' enctype=\'multipart/form-data\'><input type=\'hidden\' name=\'attfile\' value=\'cmd\'><input type=\'hidden\' name=\cmd\' value=\'Update\'><input type=\'submit\' name=\'cmd\' value=\'Submit\'><input type=\'reset\' name=\'cmd\' value=\'Clear\'><br><br><input type=\'text\' name=\'Author\' value=\'YOUR_NAME\'><br><input type=\'text\' name=\'Email\' value=\'YOUR_EMAIL\'><br><input type=\'text\' name=\'Url\' value=\''+u+'\' size=80><br><input type=\'text\' name=\'Title\' value=\''+t+'\' size=80><br><br><textarea name=\'Text\' rows=30 cols=80 wrap>'+s+'</textarea><br><br><input type=\'hidden\' name=\'attfile\' value=\'cmd\'><input type=\'hidden\' name=\cmd\' value=\'Update\'><input type=\'submit\' name=\'cmd\' value=\'Submit\'><input type=\'reset\' name=\'cmd\' value=\'Clear\'></form>');void(close())}
|
||||
Executable
+77
@@ -0,0 +1,77 @@
|
||||
I have created a javascript to be used as a browser link that allows a one
|
||||
step cut and paste from a web browser into a elog logbook.
|
||||
|
||||
The intended application is allow a user to do a text selection in a web
|
||||
browser, then click on a bookmark that automagically pastes the selected
|
||||
text, the current browser page url, and the current browser page title into
|
||||
a pre-defined elog logbook. I do some research where I would like to save
|
||||
some text from a webpage, but also have a record of where the webpage came
|
||||
from. However, you should find that you can extend this script in a varity
|
||||
of ways for your own application.
|
||||
|
||||
The script is a simple one: it uses javascript in a saved bookmark to get
|
||||
your selected text, title, and url, and then creates a new browser window
|
||||
with a elog form, and print the document variables into the form, and then
|
||||
submits the form to elog. The key advantage to this approach is that you
|
||||
can use the "post" command, rather than "get", to submit to the text section
|
||||
of an elog logbook. The only way I found now to submit to elog via a
|
||||
bookmark is using the "get" command, and it doesn't allow entry of the
|
||||
"text" field, only attribute fields.
|
||||
|
||||
The second major advantage to using POST is that you can submit a much
|
||||
large quanity of information ; However, some checking on this leads me to
|
||||
believe that the limit is browser and server depended, so YMMV. However, a
|
||||
great discussion on the limits of browsers can be found here:
|
||||
http://www.squarefree.com/bookmarklets/browsers.html .
|
||||
|
||||
One of the major limits is that IE6.0 browsers have a maxium of 508 bytes
|
||||
per bookmark - This book runs over 800 bytes, so I suspect tha IE6+ will
|
||||
not allow it. I tested the link with Mozilla and Firebird 0.7.
|
||||
|
||||
This script will need to edited for you to use with your elog logbook.
|
||||
The script should be fairly self-explainitory, if you are used to html
|
||||
forms and have some exposure to javascript.
|
||||
|
||||
You will need to modifiy the following fields:
|
||||
|
||||
1) in form action = http://ELOG_URL/LOGBOOK/?cmd=New
|
||||
|
||||
change the link to point to your specific logbook to be used for entry.
|
||||
|
||||
2) the attribute fields need match up with the ones in your logbook.
|
||||
|
||||
The ones listed in the template are Author, Email, Title, and URL.
|
||||
|
||||
If you have fixed fields (like Author and Email), then you can
|
||||
predefine these fields as shown.
|
||||
|
||||
I have the page title used as the entry for Title, and the page url is
|
||||
use as the URL attribute.
|
||||
|
||||
Finally, I have the text selection used as the entry for the Text field.
|
||||
|
||||
You can add additional fields by creating a new <input ...> segment
|
||||
in the script. For those more clever than me, you can concatinate the
|
||||
title, url and selection to paste into the Text area as well.
|
||||
|
||||
3) once you have a edited version of the script (make sure you keep it as a
|
||||
single line), you can then create a new bookmark in your browser, and then
|
||||
paste the script into the properties->location field (for Mozilla/Firebird)
|
||||
or the properties->url field (IE). Give it a good name like "post to elog"
|
||||
|
||||
4) once saved, you can then go a web page, select some text, and then go to
|
||||
your bookmarks and click on the bookmark. It should then create a new
|
||||
window in elog with a completed logbook entry.
|
||||
|
||||
|
||||
some notes:
|
||||
|
||||
1) again, this may not work on IE6+ browsers due to M$ limitations.
|
||||
|
||||
2) You may have to be logged in already to elog for this work - I have not
|
||||
tested the interaction using a password protected elog
|
||||
|
||||
3) You can only post to a single elog logbook - You'll need to have
|
||||
multiple bookmarks for multiple logbooks.
|
||||
|
||||
-- Fred Hooper <fhooper@sushisoft.com>
|
||||
Executable
+133
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/perl -w
|
||||
|
||||
=head1 NAME
|
||||
|
||||
doelog - save a mime message to elog
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
doelog <mime-msg-file> <mime-msg-file> ...
|
||||
|
||||
someprocess | doelog -
|
||||
|
||||
=head1 DESCRIPTION
|
||||
|
||||
Takes one or more files from the command line that contain MIME
|
||||
messages, and explodes their contents out into /tmp. The parts
|
||||
are sent to elog as attachments.
|
||||
|
||||
Modified mimeexplode of the MIME::Tools in perl
|
||||
|
||||
|
||||
This was written as an example of the MIME:: modules in the
|
||||
MIME-parser package I wrote. It may prove useful as a quick-and-dirty
|
||||
way of splitting a MIME message if you need to decode something, and
|
||||
you don't have a MIME mail reader on hand.
|
||||
|
||||
=head1 COMMAND LINE OPTIONS
|
||||
|
||||
None yet.
|
||||
|
||||
=head1 AUTHOR
|
||||
|
||||
sak@essc.psu.edu
|
||||
|
||||
=cut
|
||||
|
||||
BEGIN { unshift @INC, ".." } # to test MIME:: stuff before installing it!
|
||||
|
||||
require 5.001;
|
||||
|
||||
use strict;
|
||||
use vars qw($Msgno $cmd);
|
||||
|
||||
use MIME::Parser;
|
||||
use Getopt::Std;
|
||||
|
||||
## these should be options too?
|
||||
## base elog cmd
|
||||
$cmd = "~/elog -h localhost -p 8080 ";
|
||||
|
||||
#------------------------------------------------------------
|
||||
# dump_entity - dump an entity's file info
|
||||
#------------------------------------------------------------
|
||||
sub dump_entity {
|
||||
my $ent = shift;
|
||||
my @parts = $ent->parts;
|
||||
my $file;
|
||||
|
||||
die "too many attachments\n" if ($#parts>10);
|
||||
|
||||
if (@parts) { # multipart...
|
||||
map { dump_entity($_) } @parts;
|
||||
}
|
||||
else { # single part...append to elog cmd
|
||||
$file = $ent->bodyhandle->path;
|
||||
$cmd .= "-f \"$file\" ";
|
||||
## print $cmd, "\n";
|
||||
## print " Part: ", $ent->bodyhandle->path,
|
||||
## " (", scalar($ent->head->mime_type), ")\n";
|
||||
}
|
||||
}
|
||||
|
||||
#------------------------------------------------------------
|
||||
# main
|
||||
#------------------------------------------------------------
|
||||
sub main {
|
||||
my $file;
|
||||
my $entity;
|
||||
my $subject;
|
||||
my $logbook;
|
||||
our($opt_l);
|
||||
|
||||
# Sanity:
|
||||
## (-w ".") or die "cwd not writable, you naughty boy...";
|
||||
|
||||
## check if user wants a particular logbook
|
||||
## fix to add host and port?
|
||||
getopts('l:');
|
||||
if($opt_l) { $logbook=$opt_l;} else {$logbook="emails";}
|
||||
$cmd .= "-l $logbook ";
|
||||
|
||||
# Go through messages:
|
||||
@ARGV or unshift @ARGV, "-";
|
||||
while (defined($file = shift @ARGV)) {
|
||||
|
||||
|
||||
# Create a new parser object:
|
||||
my $parser = new MIME::Parser;
|
||||
|
||||
# Optional: set up parameters that will affect how it extracts
|
||||
# documents from the input stream:
|
||||
$parser->output_under("/tmp");
|
||||
|
||||
# Parse an input stream:
|
||||
open FILE, $file or die "couldn't open $file";
|
||||
$entity = $parser->read(\*FILE) or
|
||||
print STDERR "Couldn't parse MIME in $file; continuing...\n";
|
||||
close FILE;
|
||||
|
||||
## get the subject, assumes all logbooks have a subject
|
||||
## attribute - not necessarily true. Mine do...
|
||||
chomp($subject = $entity->head->get('Subject', 0));
|
||||
$cmd .= "-a subject=\"$subject\" ";
|
||||
print $cmd, "\n";
|
||||
|
||||
# Congratulations: you now have a (possibly multipart) MIME entity!
|
||||
dump_entity($entity) if $entity;
|
||||
### $entity->dump_skeleton if $entity;
|
||||
### print $cmd, "\n";
|
||||
exec $cmd;
|
||||
}
|
||||
1;
|
||||
}
|
||||
|
||||
exit (&main ? 0 : -1);
|
||||
#------------------------------------------------------------
|
||||
1;
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Executable
+34
@@ -0,0 +1,34 @@
|
||||
Attached is a perl script to which you can pipe a message (or a
|
||||
single-message file) to submit that message to `elog'. The `elog'
|
||||
distribution includes two programs `elogd', which is the main daemon that is
|
||||
accessed via the browser, and `elog', which is a command-line interface to
|
||||
elogd.
|
||||
|
||||
The attached perl script `mailelog', will split a multipart MIME message
|
||||
into its components and submit each as an attachment to elog to create a new
|
||||
entry in a specified logbook. The attributes are the subject, from, and cc
|
||||
of the message.
|
||||
|
||||
Usage: mailelog [-|file] [-l logbook]
|
||||
|
||||
(if there are no arguments, read from stdin)
|
||||
(makes a command that looks like this:
|
||||
elog -p 8080 -h localhost -l emails -a subject=<subject> -a from=<from> -a
|
||||
cc=<cc> -f attachment-1 -f attachment-2 -f ...
|
||||
|
||||
attachment-1 is the body of the message and attachment-2... are the actual
|
||||
MIME attachments. Set the elogd configuration to display attachments, so
|
||||
that the message body is immediately visible.
|
||||
|
||||
Defaults: -h localhost -p 8080 -l emails
|
||||
|
||||
If no `-l logbook' flag is specified, then the entry is sent to the `emails'
|
||||
logbook, so make sure that logbook exists. Save this in, e.g,
|
||||
~/bin/mailelog, and make sure it is executable (`chmod +x mailelog') and on
|
||||
your path (bash: `export PATH=$HOME/bin:$PATH' or csh/tcsh: `setenv PATH
|
||||
$HOME/bin:$PATH')
|
||||
|
||||
Bugs: multi-message files don't work. can't add other attributes. if the
|
||||
logbook doesn't have attributes subject, from, cc, they are quietly lost.
|
||||
|
||||
-- Sridhar Anandakrishnan <sak@essc.psu.edu>
|
||||
Executable
+213
@@ -0,0 +1,213 @@
|
||||
#!/usr/bin/perl
|
||||
|
||||
=head1 NAME
|
||||
|
||||
parsecfg.pl - create a sql db template from a elog.cfg file
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
B<parsecfg.pl> I<stdin> I<stdout>
|
||||
|
||||
=head1 DESCRIPTION
|
||||
|
||||
reads the elogd.cfg (elog config file) on stdin and writes to stdout the
|
||||
mysql statements for creating of database template for that elog configuration.
|
||||
|
||||
A sql template is created for each logbook, with each attribute being assigned
|
||||
to a sql column. Default columns are automatically created for ID, Date, Reply_to, In_reply_to and the entry text ("Notes"). Additional columns are created, one for each attribute.
|
||||
|
||||
Attachments are handled by created a separate table "<$table_prefix>attachments", which can be indexed by the logbook_name and entry id.
|
||||
|
||||
=head1 OPTIONS
|
||||
|
||||
=over 4
|
||||
|
||||
=item Globals
|
||||
|
||||
See the "config.inc" file for global parameters. Currently, the default data type, and table_prefix are assignable options in this file.
|
||||
|
||||
=back
|
||||
|
||||
=head1 FILES
|
||||
|
||||
=over 4
|
||||
|
||||
=item I<elogd.cfg>
|
||||
|
||||
a well-formed elogd.cfg file
|
||||
|
||||
=item I<sql output>
|
||||
|
||||
A mysql output command file, suitable to be used as input into "mysql", as in,
|
||||
mysql -u <<user>> -p <<database>> << I<sql_output>
|
||||
|
||||
=back
|
||||
|
||||
=head1 BUGS
|
||||
|
||||
this is a quick and dirty programming job, so I expect this to be a brittle program. Use at your own risk, and check the resulting output before using mysql.
|
||||
|
||||
=head1 LICENSE
|
||||
|
||||
The author issues this code under the GPL. (See GPL.txt)
|
||||
|
||||
=head1 AUTHOR
|
||||
|
||||
dave@davidfannin.com
|
||||
|
||||
=cut
|
||||
|
||||
|
||||
require "config.inc" ;
|
||||
|
||||
# do not change these values, for development only
|
||||
|
||||
$opentable = 0 ;
|
||||
|
||||
|
||||
#
|
||||
# trims whitespace, and removes blanks between words
|
||||
#
|
||||
sub tokenize($) {
|
||||
my $t = shift ;
|
||||
$t =~ s/^\s+// ; # get rid of leading ws
|
||||
$t =~ s/\s+$// ; # get rid of trailing ws
|
||||
$t =~ s/\s+/_/g ; #replace spaces with _
|
||||
return $t ;
|
||||
}
|
||||
|
||||
|
||||
#
|
||||
# starts a new logbook
|
||||
#
|
||||
sub section($) {
|
||||
my $s = shift ;
|
||||
if ($opentable && ($current_section ne "global")) { emit_table() ; } ;
|
||||
|
||||
$debug && print "#section:$s\n";
|
||||
$s=tokenize($s) ;
|
||||
$current_section=$s ;
|
||||
$opentable=1 ;
|
||||
}
|
||||
|
||||
#
|
||||
# reads a logbook's comment line
|
||||
#
|
||||
sub comment($) {
|
||||
my $c = shift ;
|
||||
$debug && print "#$current_section:comment:$c\n";
|
||||
$comment{'$current_section'}=$c ;
|
||||
}
|
||||
|
||||
#
|
||||
# parses the attribute lines
|
||||
#
|
||||
sub attribute($) {
|
||||
my $a = shift ;
|
||||
@al=split(/,/,$a) ;
|
||||
foreach $av (@al) {
|
||||
$av=tokenize($av) ;
|
||||
if ($av eq "Id") { next ; }
|
||||
$debug && print "#$current_section:attribute:$av\n" ;
|
||||
push @attribute,$av ;
|
||||
}
|
||||
}
|
||||
|
||||
#
|
||||
# parses the option lines (currently not used)
|
||||
#
|
||||
sub option($$) {
|
||||
my $on = shift ;
|
||||
my $ov = shift ;
|
||||
@ol=split(/,/,$ov) ;
|
||||
foreach $olv (@ol) {
|
||||
$olv=tokenize($olv);
|
||||
$debug && print "#$current_section:options:$on:$olv\n";
|
||||
}
|
||||
}
|
||||
#
|
||||
# parses the moption lines (currently not used)
|
||||
#
|
||||
sub moption($$) {
|
||||
my $mon = shift ;
|
||||
my $mov = shift ;
|
||||
$debug && print "#$current_section:moptions:$mon:$mov\n";
|
||||
}
|
||||
#
|
||||
# parses the ioption lines (currently not used)
|
||||
#
|
||||
sub ioption($$) {
|
||||
my $ion = shift ;
|
||||
my $iov = shift ;
|
||||
$debug && print "#$current_section:ioptions:$ion :$iov\n";
|
||||
}
|
||||
#
|
||||
# parses the roption lines (currently not used)
|
||||
#
|
||||
sub roption($$) {
|
||||
my $ron = shift ;
|
||||
my $rov = shift ;
|
||||
$debug && print "#$current_section:roptions:$ron:$rov\n";
|
||||
}
|
||||
|
||||
|
||||
#
|
||||
# writes out the sql code for a logbook (table)
|
||||
#
|
||||
sub emit_table {
|
||||
|
||||
print "\n\nCREATE TABLE `$table_prefix$current_section` (\n" ;
|
||||
print " `id` int(10) NOT NULL auto_increment,\n" ;
|
||||
print " `Reply_to` int(10) default NULL ,\n" ;
|
||||
print " `In_reply_to` int(10) default NULL,\n" ;
|
||||
print " `Date` datetime NOT NULL,\n" ;
|
||||
while ($a=pop(@attribute)) {
|
||||
if ($a =~ /Date/) {
|
||||
$type="datetime" ;
|
||||
} else {
|
||||
$type=$default_element_type ;
|
||||
}
|
||||
print " `$a` $type default NULL,\n" ;
|
||||
}
|
||||
print " `Note` text NOT NULL,\n" ;
|
||||
print " UNIQUE INDEX id (`id`)\n" ;
|
||||
print ") TYPE=MyISAM;\n" ;
|
||||
}
|
||||
|
||||
#
|
||||
# writes out the sql code for an attachment table
|
||||
#
|
||||
sub emit_attachment {
|
||||
|
||||
print "\nCREATE TABLE `".$table_prefix."attachment` (\n";
|
||||
print " `id` int(10) NOT NULL auto_increment,\n";
|
||||
print " `logname` varchar(255) ,\n";
|
||||
print " `pid` int(10) ,\n";
|
||||
print " `filename` varchar(255) ,\n";
|
||||
print " UNIQUE INDEX id (`id`)\n";
|
||||
print ") TYPE=MyISAM\n;" ;
|
||||
|
||||
}
|
||||
|
||||
#
|
||||
# main()
|
||||
#
|
||||
|
||||
#
|
||||
# loop on input, parsing out the relavant sections
|
||||
#
|
||||
while(<>) {
|
||||
chomp ;
|
||||
/^\s*\[(\w+)\]\s*$/ && section($1) ;
|
||||
/^\s*(Comment)\s*=(.*)/ && comment($2);
|
||||
/^\s*(Attributes)\s*=(.*)/ && attribute($2);
|
||||
/^\s*(Options)\s*(\w+)\s*=(.*)/ && option($2,$3);
|
||||
/^\s*(MOptions)\s*(\w+)\s*=(.*)/ && moptions($2,$3);
|
||||
/^\s*(IOptions)\s*(\w+)\s*=(.*)/ && ioptions($2,$3);
|
||||
/^\s*(ROptions)\s*(\w+)\s*=(.*)/ && roptions($2,$3);
|
||||
}
|
||||
|
||||
#
|
||||
# close any open tables,
|
||||
if ($opentable && ($current_section ne "global")) { emit_table() ; } ;
|
||||
emit_attachment() ;
|
||||
Executable
+197
@@ -0,0 +1,197 @@
|
||||
#!/usr/bin/perl
|
||||
|
||||
=head1 NAME
|
||||
|
||||
parselog.pl - create a sql db input file from a elog logbook
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
B<parselog.pl> I<logbook-name> I<stdin> I<stdout>
|
||||
|
||||
=head1 DESCRIPTION
|
||||
|
||||
reads the *.log files (elog logbook) on stdin and writes to stdout the
|
||||
sql statements for inputing the logbook data into an MySQL database.
|
||||
|
||||
You will need to create the MySQL database first by using the parsecfg.pl command file.
|
||||
|
||||
Default entries are created by ID, Date, Reply_to, In_reply_to, and Notes (the text of the logbook entry). Other attributes are translated as needed.
|
||||
Note that any attribute name that contains "Date" is translated to the MySQL datetime variable type, and the script will attempt to convert the attribute value to the datetime format.
|
||||
|
||||
Attachments entries are handled by created a separate table "<$table_prefix>attachments", which can be indexed by the logbook_name and entry id.
|
||||
|
||||
=head1 OPTIONS
|
||||
|
||||
=over 4
|
||||
|
||||
=item I<logbook name>
|
||||
|
||||
Required Option: The name to use of the database table. Should be the same name as used by elog for a logbook, and created by parsecfg.pl.
|
||||
|
||||
=item Globals
|
||||
|
||||
See the "config.inc" file for global parameters. Currently, the default data type, and table_prefix are assignable options in this file.
|
||||
|
||||
=back
|
||||
|
||||
=head1 FILES
|
||||
|
||||
=over 4
|
||||
|
||||
=item I<*.log>
|
||||
|
||||
The log files in a elog logbook directory
|
||||
|
||||
=item I<sql output>
|
||||
|
||||
A mysql output command file, suitable to be used as input into "mysql", as in,
|
||||
mysql -u <<user>> -p <<database>> << I<sql_output>
|
||||
|
||||
=back
|
||||
|
||||
=head1 BUGS
|
||||
|
||||
this is a quick and dirty programming job, so I expect this to be a brittle program. Use at your own risk, and check the resulting output before using mysql.
|
||||
|
||||
Elog is sloppy on logbook formats - Extra attributes could remain in a elog logbook file if the user created a logbook, and then deleted attributes later on. These extra attribute entries will break the sql file, and you will need to delete them from the logbook or from the sql file manually.
|
||||
|
||||
Attribute names containing "Date" are assumed to contain datetime information and are automatically converted to MySQL datetime format. This may not be the correct action and will cause things to break.
|
||||
|
||||
=head1 LICENSE
|
||||
|
||||
The author issues this code under the GPL. (See GPL.txt)
|
||||
|
||||
=head1 AUTHOR
|
||||
|
||||
dave@davidfannin.com
|
||||
|
||||
=cut
|
||||
|
||||
use Time::ParseDate ;
|
||||
use Date::Format ;
|
||||
|
||||
require "config.inc" ;
|
||||
|
||||
|
||||
if ($#ARGV == 0) {
|
||||
$table_name=$ARGV[0] ;
|
||||
|
||||
} else {
|
||||
print "usage: parselog.pl <logbook-dbname>\n" ;
|
||||
print "you are missing the logbook name\n" ;
|
||||
exit(1) ;
|
||||
};
|
||||
|
||||
$openentry = 0 ;
|
||||
|
||||
sub tokenize($) {
|
||||
my $t = shift ;
|
||||
$t =~ s/^\s+// ; # get rid of leading ws
|
||||
$t =~ s/\s+$// ; # get rid of trailing ws
|
||||
$t =~ s/\s+/_/g ; #replace spaces with _
|
||||
return $t ;
|
||||
}
|
||||
|
||||
sub quote_sql($) {
|
||||
my $qstr = shift ;
|
||||
# escape special chars
|
||||
$qstr =~ s/(['"\\\0])/\\$1/g ;
|
||||
return "'".$qstr."'" ;
|
||||
}
|
||||
|
||||
|
||||
sub entry($) {
|
||||
my $e = shift ;
|
||||
if ($openentry) { emit_entry() ; } ;
|
||||
|
||||
$debug && print "#entry:$e\n";
|
||||
$e=tokenize($e) ;
|
||||
$current_entry=$e ;
|
||||
$openentry=1 ;
|
||||
}
|
||||
|
||||
sub attribute($$) {
|
||||
my $name = shift ;
|
||||
my $value = shift ;
|
||||
$name=tokenize($name) ;
|
||||
$debug && print "#attribute:$name:$value\n" ;
|
||||
if ( $name eq "Attachment" ) {
|
||||
if ( $value == "" ) { return ; }
|
||||
push @attachment, "$current_entry;$value" ;
|
||||
return ;
|
||||
};
|
||||
return if ( $name eq "Id" ) ;
|
||||
return if ( $name eq "Encoding" ) ;
|
||||
if ( $name =~ /Date/ ) {
|
||||
$value=time2str("%Y-%m-%d %X",parsedate($value)) ;
|
||||
}
|
||||
$attribute{$name}=$value ;
|
||||
}
|
||||
|
||||
sub encoding {
|
||||
|
||||
$encoding = "";
|
||||
<STDIN> ;
|
||||
while (<STDIN>) {
|
||||
/^\$\@MID\@\$: (\d+)$/ && return ;
|
||||
$encoding .= $_ ;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
sub option($$) {
|
||||
my $on = shift ;
|
||||
my $ov = shift ;
|
||||
@ol=split(/,/,$ov) ;
|
||||
foreach $olv (@ol) {
|
||||
$olv=tokenize($olv);
|
||||
$debug && print "#$current_section:options:$on:$olv\n";
|
||||
}
|
||||
}
|
||||
sub moption($$) {
|
||||
my $mon = shift ;
|
||||
my $mov = shift ;
|
||||
$debug && print "#$current_section:moptions:$mon:$mov\n";
|
||||
}
|
||||
sub ioption($$) {
|
||||
my $ion = shift ;
|
||||
my $iov = shift ;
|
||||
$debug && print "#$current_section:ioptions:$ion :$iov\n";
|
||||
}
|
||||
sub roption($$) {
|
||||
my $ron = shift ;
|
||||
my $rov = shift ;
|
||||
$debug && print "#$current_section:roptions:$ron:$rov\n";
|
||||
}
|
||||
|
||||
sub emit_entry {
|
||||
|
||||
print "\n\nINSERT INTO $table_prefix$table_name \n" ;
|
||||
print " SET id = ($current_entry),\n" ;
|
||||
foreach $name (keys %attribute) {
|
||||
print " $name = (".quote_sql($attribute{$name})."),\n" ;
|
||||
}
|
||||
print " Note = (".quote_sql($encoding).");\n";
|
||||
|
||||
}
|
||||
|
||||
sub emit_attachment {
|
||||
while($value = pop(@attachment)) {
|
||||
($id,$filename)= split(/;/,$value) ;
|
||||
print "\n\nINSERT INTO ".$table_prefix."attachment \n" ;
|
||||
print " SET logname = (\"$table_prefix$table_name\"),\n" ;
|
||||
print " pid = ($id),\n" ;
|
||||
print " filename = (\"$filename\");\n" ;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
while(<STDIN>) {
|
||||
chomp ;
|
||||
/^Encoding:/ && encoding() ;
|
||||
/^\$\@MID\@\$: (\d+)$/ && entry($1) ;
|
||||
/^([\w\s]+): (.*)$/ && attribute($1,$2) ;
|
||||
}
|
||||
if ($openentry) { emit_entry() ; } ;
|
||||
|
||||
emit_attachment() ;
|
||||
+32
-5
@@ -31,15 +31,42 @@ them directly<p>
|
||||
|
||||
<UL>
|
||||
|
||||
<LI><h2>Perl program to submit an email to elog</h2>
|
||||
<LI><h2>Perl script to submit an email to ELOG</h2>
|
||||
|
||||
<h3>by <a href="mailto:sak@essc.psu.edu">Sridhar Anandakrishnan</a></h3>
|
||||
|
||||
Perl program to split apart an email message and submit each of the mime attachments as an
|
||||
attachment to elog. The <i>-a</i> command line is fixed for now as "subject=subject of the email",
|
||||
so the logbook has to include that.<p>
|
||||
Perl program to split apart an email message and submit each of the mime
|
||||
attachments as an attachment to ELOG. The <i>-a</i> command line is fixed for
|
||||
now as "subject=subject of the email", so the logbook has to include that.<p>
|
||||
|
||||
<a href="doelog.txt">doelog.txt</a>
|
||||
<a href="../contrib/mailelog.txt">mailelog.txt</a>
|
||||
<a href="../contrib/mailelog.pl">mailelog.pl</a>
|
||||
<p>
|
||||
|
||||
<LI><h2>Perl scripts to convert an ELOG logbook to a MySQL database</h2>
|
||||
|
||||
<h3>by <a href="mailto:fhooper@sushisoft.com">Fred Hooper</a></h3>
|
||||
|
||||
Perl scripts to help translate logbooks created by ELOG from the native elog
|
||||
flat file format to a MySQL database. The elog2sql toolkit consists of two
|
||||
scripts. The first script, parsecfg.pl, reads an elogd.cfg, and creates a
|
||||
sqlfile that will create a set of db tables corresponding to ELOG logbooks.
|
||||
The second script, parselog.pl, takes a set of ELOG logfiles, and creates a
|
||||
sql file that will enter the logbook data into the database.<p>
|
||||
|
||||
<a href="../contrib/elog2sql.txt">elog2sql.txt</a>
|
||||
<a href="../contrib/elog2sql">elog2sql</a>
|
||||
<p>
|
||||
|
||||
<LI><h2>Javascript for Bookmark link for one-click submission to ELOG</h2>
|
||||
|
||||
<h3>by <a href="mailto:fhooper@sushisoft.com">Fred Hooper</a></h3>
|
||||
|
||||
Javascript to be used as a browser link that allows a one step copy and paste
|
||||
from a web browser into an ELOG logbook.<p>
|
||||
|
||||
<a href="../contrib/elogsubmit.txt">elogsubmit.txt</a>
|
||||
<a href="../contrib/elogsubmit.js">elogsubmit.js</a>
|
||||
<p>
|
||||
|
||||
</UL>
|
||||
|
||||
Reference in New Issue
Block a user