ipchkmail

ipchkmail
This program spools spool/xsmtp for files in MIME format left by
Sendmail/sffmail.
If can be used to :
- check the Sender is valid for sending to that destination by checking
against a list of names in tables/mail/check.
- zap or reroute any mail without either a 'From' or a 'Reply-to' field.
- check and/or split off and/or convert attachments
Normal running is to force all attachments into files and route them accoring
to their type - xls, jpeg, xml etc
It uses a parameter file, by default tables/mail/CHKMAIL. The format of this
file is :
; comment line
strip-mime-header: Strip the Mime Header of incoming files. default: no
strip-mime-header:yes
strip-mime-header:all these two will always strip all mimes
strip-mime-header:no this will leave all mime lines
strip-mime-header:some this will check for 'ignore' see below
defdest: (default Fip Destination - DU FipHdr field) default: FIPMSG
chrset: (Source character set) default: email
ie this provides a different FipHdr SC field to SC:email
extra-fiphdr: (Extra Fip Hdr information to add to each file) def: none
no-from-message:
invalid-sender-message:
check-sender: if there is a file in tables/mail/check, make sure the
'From' or 'Reply-to' address is included.
fiphdr: (Map incoming mail field to a Fip Hdr field) default: none
fiphdr:DI Subject
fiphdr:AZ Message-Id
convert-encoded-words: (8bit/utf8/no) default: utf8
Mime fields such as Subject are often in a format called 'encoded words'
=?iso-8859-1?q?this=20is=20some=20text?=
The DATA of the encoded word will usually be converted to UTF8 or 8bit
ignore: (Mail header field to always ignore) default: none
Used with 'strip-mime-header:some' (see above) to filter out or delete
selected mime header lines.
ignore: Status
ignore: Apparently-To
ignore: X-Msmail-Priority
fiphdr-TO-field: (FipHdr)
Use this where you already have the real TO address in a fipHdr field. eg if
you are running sffmail in
/etc/aliases : donkeyrides: "|/fip/bin/sffmail -hJA:DKR"
then use fiphdr-TO-field:JA
to capture all DKR traffic no matter what is in the To field.
max-TO-length: Set the Maximum length of the 'To:' field to this.
default: 400 chrs.
Listservers often give tens/hundreds of names in the 'To:' field. This chops
to the first few.
max-single-fiphdr-size: (number > 400) default: 1023 chrs
This does NOT override the max-TO-length parameter.
This overrides the -H input switch
send-all-with-attachments: (DU)
queue-all-with-attachments: (folder name)
Fip Destination and folder name to send any file (complete, unaltered) with
one or more attachments.
If there is no "queue-..", the default output queue is used.
This is incompatible with decode, -a/-A and any other attachment processing.
decode:
unencode: Decode any base64 sections. default: no
decode: This will NOT decode attachments
decode:text This will NOT decode attachments
decode:all This will decode body AND attachments
A second parameter may be added 'utf8' which will decode the result to UTF8
eg decode:text,utf8
split-every-attachment: (yes/no)
Normally any attachment that is has a Content-Type of 'text' is left in the
main message.
Use this keyword to split ALL attachments and either leave in the folder (-a)
or sent to the destination (-A).
att-message: Message to replace an attachment with.
The attachment is called fipHdr DN.
att-message:<a href="readdisboyo.pl?namo=\\DN>Read this Attachment
\\DN</a><br>
default:** Attachment has been Stripped, pls contact System Managers for :
\\DN
alt-html-inline: (yes/no)
show the Alternate HTML version inline or NOT default: yes
add-fiphdr-on-atts:yes/no
add the whole fiphdr on each attachment (-A only)
att-data-only:
do NOT add any mimeheaders on the top of the attachment (-A only)
fiphdr-att-type: (XX)
fiphdr-att-chrset: (XX)
fiphdr-att-name: (XX)
fiphdr-att-bytes: (XX)
fiphdr-att-total: (XX)
fiphdr-att-level: (2 chr FipHdr code to replace PL in start/end-att-marker
fiphdr-att-inline: (2 chr FipHdr code to replace PI in start/end-att-marker
2 letter FipHdr fields for the Content-Type, Charset and Filename of any
attachments
plus level (0 is top, 1++ is embedded attachements)
plus total : for the Top, this = Number of attachments
for each attachment, its sequence number
plus bytes : no of bytes/size of file
no defaults, except for PL and PI
Note that total bytes for an attachment is only non-zero for END-att-marker
eg
; for each Att, save the type, chrset and name in temp FipHdr fields
fiphdr-att-type:D1
fiphdr-att-chrset:D2
fiphdr-att-name:D3
; D4 seqno
fiphdr-att-total:D4
; D5 total bytes of ATT or Top - only available for END-att-marker
fiphdr-att-bytes:D5
; for each Att, top and tail with a marker - if NOT inline
; PI says whether att is inline or not
option:VN PI,,,,NO
start-att-marker:\n
end-att-marker:\n\VN**FIP-ATT**|\PL|\PI|\D1|\D2|\D3|\D4|\D5|\$o
att-message:\VN\DN\$o\n
start-att-marker: (FipSeq)
Extra Data to insert into the main data file at the point where an
attachement starts
extra FipHdr fields at this point are PL : level, PI : inline YES or NO
Plus the 3 fiphdr-att-* fields if specified
option:VY PI,,,,YES
start-att-marker:\VYInLine attachment starts here ->\n\$o
no defaults
end-att-marker: (FipSeq)
Extra Data to insert into the main data file at the point where an
attachement ends
PL : level, PI : inline YES or NO or IGNORE (not inline and no file)
option:VN PI,,,,NO
end-att-marker:\VNEndOfStrippedAttachment\n\$o
no defaults
before: (FipSeq)
after: (FipSeq)
Extra Data to insert into the main data file at the top or tail
no defaults
preserve-existing-filename:
preserve-existing-source:
ignore-existing-filename:
ignore-existing-source:
These two will ignore either the FipHdr field SN or SU for name or source
from the raw file and add new ones.
The default is to preserve the source but ignore the filename!
process-if-no-check-file:yes/no
If you are using Check files and there is no check file for that address, the
default is to SEND. Use the following to NOT send:
process-if-no-check-file:no
newDA: FipHdr field to use in place of DA for the 'To:' field
If you are using smtpwire, DA will already be filled in correctly.
So map it to another fipHdr newDA:RA
newDI: FipHdr field to use in place of DI for the 'Subject:' field
If you are erworking the Subject line - forcing it to QuotePrintable for
example,
map it to another fipHdr newDI:RI
hdr-hash:(FipSeq)
replacement chr for a hash '#' in a Fiphdr field
default is '-'
script: (FipSeq script and Filename)
Script to run against the MAIN, top file (ie NOT the attachments)
E1 is the temp file name with FipHdr and Data
E2 WILL BE the output filename if needed
no default
eg script:/fip/local/list_attachments.php \E1
round-robin: (number) default: none
round-robin-fiphdr: (2 letter FipHdr field) default: none
Round-Robin the output files and add the RR number to the fipHdr.
Both parameters are required - the Number is the MAXimum.
eg to leave the output in folder1 to folder9
round-robin:9
round-robin-fiphdr:RR
and a suitable output folder might be -o avcheck\RR
Note that the round-robin number is NOT added automatically to any output
folder - ie you MUST specify a FipHdr as in /fip/spool/2xml\RR
-- Check files --
There needs to be a check file for EACH address you want checking. The filename
is just the NAME (not the Domain) forced uppercase :
eg if the TO address is billy@never.inhale.com
the check file is 'tables/mail/check/BILLY'
The entries in the check file are the sender's address. ie the FROM field.
The format of a check file is
; is a comment line
; there can be one or many entries, each on a separate line
; for a single sender
name@domain
billyGoat@bridge.to.far.com
; for a complete domain - prefix a '*'
*@completely_idiotic.com
For Example
;
; Control files to NationalDesk
;
; allow all internal users
*@tuebrook-bugle.com
; Star reporter
famousJock@ego.inflate.com
; the editor
S.Leaze@weasel.org
-- Blacklists --
A single BLACKLIST file can be added to ignore files FROM certain addresses
and/or domains.
It is the same syntax as the Check files (see above) but is in the main folder
tables/mail/BLACKLIST.
Note that you can block all domains for a name with a wild card too :
Undisclosed.Recipients@*
-- Conversions --
; What conversions do we want to do (double quotes to keep the spaces)
convert:binhex script:"/usr/forum/bin/alchemy \QN \QN.nohex -Q -o -e --."
-- FipHdr fields --
Note the following FipHdr fields are filled in for each incoming file (in
addition to any 'fiphdr' parameters)
SA - From address
DA - To address (can be changed using the 'newDA' parameter)
DI - Subject field (can be changed using the 'newDI' parameter)
If the contents of these fields are a different charset, it is ignoted and
converted to the Binary chrs
However the 'fiphdr:' fields are preserved without conversion.
Note that FipHdr fields are used for conversions:
QN - Full Path and Filename of the input file to be converted
QR - Path or queue for converted files and files to convert
which defaults to /fip/spool/epsf
SN - name of the input file to the program !
QI - name of the scrolled queue
Input parameters are (all optional) :
-a : Folder to put attachments we have stripped
default: att left in data
-A : Destination (FipHdr DU) to send all the
attachments we have stripped. default: att left in data
-d : done queue for incoming default: none
this will leave the raw, incoming file in this done folder.
-H : max size of any FipHdr field default :1023
-i : input queue to scan default: spool/xsmtp
-o : output queue for processed files default: spool/2go
This can be FipSeq - eg -o avcheck\RR
-l : do NOT log incoming files default: log
-w : file wait interval in seconds default: no wait
set this for nfs or slow systems eg -w 10
-z : name of parameter file if not default: tables/mail/CHKMAIL
the default.
-T : do NOT attempt to clean up the DU default: do
only letters, numbers '_' and '-' are allowed.
-v : print version no and exit
Version Control
;1w32 20jun03 added fiphdr-to-field, process-if-no-check-file
;b 26jul03 bugette - dbl ATT on attachment
;c-g 30jul03 bugette - decode NOT always working correctly plus newDA added
;h 31oct03 timings
;i-j 05dec03 added add-fiphdr-on-atts
;k-m 30mar04 bugette when hacker sends > 64000 on one line of header
;n-o 03sep04 added check_for_fiphdr for attachments too
;p-q 20sep04 speedy
;r 09feb05 Imail has invalid Content-type
;s-t 31oct05 bugette - need to be able to handle blank lines with spaces in
the mime header
;u-v1 21mar07 bugette with attachments and long TO: addresses
;w1-9 10dec08 attachments inside attachments - major mods ! added -1 for
single ;7 bugette without -a -A ;8-9 newDI added
;10-11 added unicode mapping ;12 preserve att fiphdr better
;13-18 put inline TEXT back in.. plus allow '+' in DU
;19-24 tuning strip-mime-hdr to properly zap all the mimes of multi-level
atts and hdr-hash added
;25 2jan2013 de-dup added ;26 1oct13 bugette in dedupbalance ;27 donque move
not link
;28 30nov14 trimmed extrafiphdr to 8k
;29 8jun15 added strparse to -o outque and round-robin
;30-31 29oct15 woops - allow multiline Subject lines ....
;32 1sep17 max size of a FipHdr field is now FIP_limitHdrFieldSize (HDR-STD)
;000z 21jan99 original version
;a/b/c/d 3jun99 small mods
;e 17feb00 allow fiphdr on incoming file.
;f 06mar00 maxTOlength of 400 chrs
;g/h 10mar00 added decode and better Dispo-inline handlng
;i 10apr00 mod for sending any file with an attachment to another DU
NOTE need to WINNT DU and DUallAtts **
;j/k 14apr00 woops make max len of a FipHdr field. (k) and filename
;m/n 09may00 small mods and bugette in do_decode
;o/p/q 05jun01 check_sender better; added Blacklist, strip leading NULS/SPCs
;r 03aug01 always check for HH fields..
;s 29oct01 WINNT filename needed better validation
;t 24feb02 added split-every-attachment
;u 04jun02 bugette in boundaries etc
;v/w/x 13jun02 if there is an SU already - use it
;y 03oct02 bugettes - where the data does not contain a single CR or NL
;z 23apr03 decode Headers too
(copyright) 2017 and previous years FingerPost Ltd.