ipfprep

ipfprep
This program prepares incoming data to tweak it before IPFORMAT is run against
it.
Each line of data - or each record if it is not a line - is checked against a
series of tests to try and determine the type of line it is. In the parlance of
this program is tries to find a KEY. According to this key the record is either
thrown away or, more usually, labelled with a record type. This record type is
normally at the beginning of the line as that is the easiest for IPFORMAT to
catch and work on.
A series of parameter lines in the normal FIP syntax are used to describe the
patterns and tests. (see below for a fuller description of all the variables).
Firstly the file is described with the parameters :
filtyp - type of file
recsep - each record ends with ...
reclen - (optional fixed record length)
wild - if you are using WILD cards, what chr signifies
The next thing is a series of FMTs and KEYs which try to test the line eg :
fmt:BLANK s:0 b:0
key:BLANK tst:2="" delete
ie split the line into two fields - the first contains all the leading spaces
and the second is the rest of the line. If field 2 has nothing in it, it is a
blank line. The action is strip the line.
Normally if the result of the tests is good, you just want to label the line.
In this case, if fields 3, 6, 9 and 12 each have a single space and field 1 is
NOT a space, then we prefix "ZOG," to the begining of the line
fmt:ZOG b:1 b:9 s:1 b:1 b:9 s:1 b:1 b:9 s:1 b:1 b:9 s:1 b:0
key:ZOG chk:3 chk:6 chk:9 chk:12 tst:1#" " addkey:1:ZOG,
You can also create new records from the fields you have created :
fmt:STX b:40 s:0 n:0 s:0 b:0
key:STX chk:1 chk:3 tst:5="" delete newrec:STX,{3},{1}
This deletes the original record and adds a new one starting "STX," followed by
field 3 comma field 1.
Lastly it is good idea to have a catch all for the lines you missed or for the
majority of lines :
; setup the default keyname for text files
defkey: addkey:1:DAT,
This just prefixes four chrs "DAT," at position 1 of the line. Note there
should be at least one space/tab after 'defkey:' and any (or all) default
action - addkey/delete/newrec
Notes - the NAME of a 'fmt' must be unique but there can be several 'key's for
that fmt which may be testing different things - and creating newrecords.
; Headers Par Value - strip
fmt:HDR p:10 b:0
key:HDR tst:1="Par Value " delete newrec:PAR,next
key:HDR tst:1="Div NvG " delete newrec:DIV,next
key:HDR tst:1="Yld Notes " delete newrec:YLD,next
key:HDR tst:1="Chg on wk " delete newrec:CHG,next
key:HDR tst:1="Earn/Shr " delete newrec:ERN,next
Here the original record is replaced by one of five records with no extra data.
Each FMT is checked sequentially - ie top down in the parameter file. If a
single key matches, then the parameter file is checked for more key lines with
the same name. Processing stops on the next 'fmt' record. This means all the
keys for a fmt should follow immediately after as in th examples above.
Note that if you have a 'newrec' and do not want to delete the original, the
new one is inserted BEFORE the original.
FMT - Syntax : and a single line , field marked [] are optional
fmt:(name) (typ1):(len1)[:(startpos1):(startstring1):(endstr1)]
... (typn):(lenn)
where name is the name of the record format (case insensitive)
tabs or spaces are separators between fields
colons are separators between parms for the same field
length or size of field is 0 for variable size
type is a-alphabetic, u-uppercase, l-lowercase
n-number, p-printable, s-space(or tab, CR, NL or FF)
b-binary (ie anything), x-alphanumeric,
c-control (ie < 040 or >= 0177),
z-anpa hdr field (ie alnum plus non-quad/format punct.
t-punctuation
startposn is the starting posn in the record from posn1
startstring is the starting string for the field
endstring is the end string for the field
KEY - Syntax :
key:(name) chk:(fldx) tst:(fldx)=(string) tst:(fldn)#(string)
addkey:(posn):(len) delete: newrec:(syntax for newrec)
key:(name) same name as FMT previously specified
chk:3 chk field 3 is of the format is the right length
or if field 3 is varibale length, has 1 or more.
tst:7=ABC Does field 7 equal ABC
tst:9#" " Does field 9 NOT equal 4 spaces
Note that tst is case sensitive.
addkey:1:3 Add a key of the name to the record at 1
delete: Delete this record
newrec:PAG0003 Add a record before with data PAG0003 (recsep)
DO NOT ADD THE END OF LINE it is done automatically
RECKEY Check the type and size of the key
Syntax : reckey: : (length) : (type)
where length or size of key is 0 for no size check
type is a-alphabetic, u-uppercase, l-lowercase
n-number, p-printable, s-space(or tab, CR, NL or FF)
b-binary (ie anything), x-alphanumeric,
c-control (ie < 040 or >= 0177),
z-anpa hdr field (ie alnum plus non-quad/format punct.
t-punctuation
Input parameters are :
Optional :
Either
-1 : filename (and optional path) of a single file def: spool queue
The input file is NOT deleted in this case.
or
-i : input queue default: spool/fprep
-o : output queue default: spool/form
-f : file creep time for files default: 0
arriving across the network - normally make it 5 secs
-t : scan time of directory default: 5 secs
-l : do NOT log every incoming file default: log
-z : use the basename of the file for the parameter file if there is ..
.. no FP FIP header field default: use FP only
-Z : always use this parameter file default: use the FP field.
-v : display version number and exit.
Example :
;
; type of file is text
filtyp:t
; each record is separated by a CR CR NL
recsep:\r\r\n
; strip all cr nl combinations including blank lines
; this boils the 'recsep' into a single CR
stripeol:y
; There are no reclen and reckey commands - comment them out
;;reclen:121
;;reckey:3:u
; change the wild card chr to a dollar (normally it is a star '*')
wild:$
; blank lines - to be stripped - put first to catch all of them
fmt:BLANK s:0 b:0
key:BLANK tst:2="" delete
; text 8 col lines
fmt:TX8 s:1 p:38 p:1
key:TX8 tst:3="-" addkey:1:TX8
; format of the trailer records
; (spc) AJ (4 num) (2 spc) (10 prinatable repeated 4 times)
fmt:TRAILER s:1 u:2 n:4 s:2 p:28 p:28 p:28 p:28
; a trailer record needs to be deleted - and subrecords created from it
key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 delete
key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:5#" " newrec:E{5}
key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:6#" " newrec:E{6}
key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:7#" " newrec:E{7}
key:TRAILER chk:1 tst:2=AJ chk:3 chk:4 tst:8#" " newrec:E{8}
; Main Heading - check the syntax of fields 1 and 3 and make sure f2 = '1'
fmt:HEAD S:8 N:1 S:20 P:14 P:12:44 p:4:56
key:HEAD chk:1 tst:2=1 chk:3 addkey:1:HD1
key:HEAD chk:1 tst:2=1 chk:3 tst:4="CITROEN" newrec:PAG0003
key:HEAD chk:1 tst:2=1 chk:3 tst:4="PEUGEOT" newrec:PAG0024
key:HEAD chk:1 tst:2=1 chk:3 tst:4="RENAULT" newrec:PAG0104
key:HEAD chk:1 tst:2=1 chk:3 tst:4="TALBOT" newrec:PAG0105
key:HEAD chk:1 tst:2=1 chk:3 tst:4="FORD" newrec:PAG0106
key:HEAD chk:1 tst:2=1 chk:3 tst:4="MAZDA" newrec:PAG0107
key:HEAD chk:1 tst:2=1 chk:3 tst:4="LADA" newrec:PAG0108
key:HEAD chk:1 tst:2=1 chk:3 tst:4="ROVER" newrec:PAG0109
key:HEAD chk:1 tst:2=1 chk:3 tst:4="NISSAN" newrec:PAG0141
key:HEAD chk:1 tst:2=1 chk:3 tst:4="BMW" newrec:PAG0144
; 1st Col headers
fmt:COL n:0 s:39 p:10 p:10 p:10 p:10 p:10 p:10 p:10 p:10
key:COL tst:1#0 tst:3#" " addkey:1:COL
; 2nd Col headers
fmt:CL2 s:40 p:10 p:10 p:10 p:10 p:10 p:10 p:10 p:10
key:CL2 chk:1 tst:2#" " addkey:1:CL2
; Section Head
fmt:SECTION s:1 u:1 t:1 p:0
key:SECTION chk:1 chk:2 tst:3="-" addkey:1:SCT
; SubSection Head
fmt:SUBSECT s:1 u:1 n:0 t:1 p:0
key:SUBSECT chk:1 chk:2 chk:3 tst:4="-" addkey:1:SUB
; setup the default keyname for text files
defkey: addkey:1:TX8
(copyright) 2018 and previous years FingerPost Ltd.