]>
Spam reporting using IMAP: SREPResearch In Motion Limited1875 Buckhorn GateMississaugaL4W 5P1OntarioCanada+19056294746x15674+12892615950zordogh@blackberry.comThis document defines an IMAP extension which allows a client to report spam by reference and allows an IMAP server to perform any action on the reported messages, including leaving the action at the client's discretion.In addition, this document discusses how an IMAP server can tap into spam aggregator services, ultimately allowing the IMAP server to improve its decision-making process.In examples, "C:" and "S:" indicate lines sent by the client or the server, respectively.The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in .This specification follows the recommendations in .The Internet Message Access Protocol does not support reporting spam on its own.
There are a number of solutions available based on the multipart/report content type defined in and its revision, .
However, these solutions require including the message contents and hence, consume bandwidth to transmit the entire message.
In bandwidth-constrained environments - such as mobile networks - it is highly desirable to send only a minimum set of information - a reference - instead of the entire message.
Solutions that exist today employ manipulating proprietary flags in the IMAP storage to achieve the bare minimum, however more advanced solutions cannot be developed by using flags only; the IMAP server needs to be involved actively in the spam reporting process.Furthermore, it is highly desirable to permit individual server implementations to handle spam in any way these systems choose to: do nothing, flag, perform deletion or relocation, recommend deletion or relocation to the client, or, leave the decision at the client's discretion as a whole.
However, in order to make such a decision on the server side, a spam aggregator service, such as , needs to be involved in the decision-making process.This document specifies a new IMAP command, SREP, along with its syntax, which allows a client to inform the server that the user considered a message (or parts thereof) spam, or, that the user no longer considers a message (or parts thereof) spam.
Since all information about the message is readily available on the server, the command also allows the server to implement a more intelligent and accurate decision logic, which may be invoked when the spam is reported and the server can respond with its decision to the client.Additionally, this document contains example flows, illustrating various decisions that the server may choose to evaluate, including invoking an aggregator service, such as one based on .The SREP command allows:
reporting spam; i.e. set the spam condition, and,reporting that a message (that was reported spam earlier) is no longer spam; i.e. clear the spam condition.This document focuses only on the client-server interactions and the scope is limited to messages that either exist on the IMAP server, or, exist elsewhere and the IMAP server is configured to access them.
Consequently, deposit-time filtering, messages that have been deleted, and messages that exist in an external storage but are accessible only via an access protocol unknown to the IMAP server are out of scope.The SREP command follows the conventions of .Arguments:
directive; see OPTIONAL abuse type; see reference; see OPTIONAL list of part identifiers; see OPTIONAL request action; see Responses; see :
OPTIONAL OK response: RELOCATEOPTIONAL OK response: RELOCATEDOPTIONAL OK response: DELETEOPTIONAL OK response: DELETEDOPTIONAL OK response: KEYWORDResult:
OK - command completed successfullyNO - the server cannot access one or more messages (deleted or unauthorized)BAD - there was an error during processing the command (syntax or unsupported parameter)The formal syntax of the SREP command is defined in .The SREP command allows:
- reporting spam; i.e. set the spam condition, and,- reporting that a message (that was reported spam earlier) is no longer spam; i.e. clear the spam condition.The SREP command may be used with any IMAP4 server implementation that returns "SREP" as one of the supported capabilities in response to the CAPABILITY command.
If the server does not indicate support for the SREP capability, the client MUST NOT use the SREP command.The SREP command may result in ambiguity, therefore the client MUST NOT send any commands before the result of the SREP command has been received, see Section 5.5 in [IMAP].The command MAY be issued on one or more messages at a time, in the currently selected mailbox.The command MAY be extended in the future with new parameters (actions, directives, reference types, etc).
Servers MUST be able to recognize parameters unknown to them and respond with a BAD response in case they encounter such a parameter.The directive argument tells the server whether a message is being reported as a spam, or, it is being reported as no longer a spam. The SREP command MUST include the directive.
To report a spam, the directive MUST be SET.
To report that a message is no longer considered to be a spam, the directive MUST be CLEAR.
Extensions are permitted, as defined in .
The client may have additional information about the spam regarding the nature of the abuse.
When such information is available, the client SHOULD include the abuse type argument in the request.
When such information is not available, the client MUST omit the abuse type argument from the request.
When the directive argument is CLEAR, the client MUST omit the abuse type argument from the request.
This specification defines the following abuse types:
Phishing (forgery, link manipulation, etc.): an attempt to divulge information from the recipient by masquerading the sender and/or the content(s) of the message as a trustworthy form of communication.Malware (virus, spyware, etc.): a malicious piece of software code embedded or attached to the message specifically designed to disrupt normal operation, gather sensitive information, gain unauthorized access, and/or perform other abusive behavior upon execution.
Extensions are permitted, as defined in .
The reference argument consists of a reference type and a reference value.
In general, the reference type MUST indicate the format of the reference while the reference value MUST contain a value corresponding to the indicated reference format.
To use a unique identifier specified in , the reference type MUST be UID and the reference value MUST be a number expressing the unique identifier of the message.
To use a sequence set specified in , the reference type MUST be SEQ and the reference value MUST be sequence numbers corresponding to the specified message sequence number set.
To use an authorized URL specified in , the reference type MUST be URLAUTH and the reference value MUST be an URLAUTH-authorized URL, authorizing the entire message.
Extensions are permitted, as defined in .
When the reference identifies one and only one message, the list of part identifiers MAY be included to improve the accuracy of spam detection.
When the reference identifies more than one message, the list of part identifiers MUST be omitted.The list of part identifiers is a parenthesized list of part identifiers.
Part identifiers MAY identify header fields or bodies.
Header field identifiers MUST be prefixed with the word 'header' and the dot ('.') character MUST be used as the separator character.
Header fields MUST be identified by the name of the header field.Example:
The 'From' header field is identified as 'header.from'.Body identifiers MUST be prefixed with the word 'body' and the dot ('.') character MUST be used as the separator character.
Bodies MUST be identified by their positions within the message hierarcy, where the first position is 1 and the main level is 1.
To refer the entire body of a message (or all bodies of a multipart message), the separator character, the position MUST be omitted.Examples:
- The entire body of a message (or all bodies of a multipart message) is identified as 'body'.- Considering a simple multipart message, the part following the first boundary is identified as 'body.1'.- Considering a multipart message that includes an email attachment following the second boundary, and the email attachment containing text following the first boundary, the text within the email message is identified as 'body.2.1'.
The formal syntax of the part-id-list is defined in .
The request action argument explicitly tells the server what to do with the the message.
To request a specific action from the server explicitly, the SREP command MUST include the request action argument.
To not request a specific action, the SREP command MUST NOT include the request action argument; in this case, the server MUST decide the course of action.
The client MAY specify either one of the following actions:
- The KEYWORD the client requests that only keyword(s) should be added to the message. The server MUST add the appropriate keyword.- The RELOCATE the client requests that the message should be relocated. The server MUST relocate the message by copying the message to the destination mailbox removing the original as if another connected client requested this action using an IMAP MOVE command.- The DELETE the client requests that the message should be deleted. The server MUST delete the message as if another client performed this action.
The server MUST ignore the destination mailbox in case it is nil, or, the request action is KEYWORD or DELETE.
It is assumed that the server is pre-configured with the location where user's spam messages are stored. If the server is not configured with such information and the destination mailbox in a RELOCATE action is nil, or, destination mailbox in a RELOCATE action is otherwise inaccessible to the user (does not exist, insufficient permission, etc) the the server MUST reject the request (see BAD response in ).
NOTE: While the DELETE action does not seem appropriate in case the directive argument is CLEAR, it is permitted.
The formal syntax of the request action argument is defined in .
The SREP command MAY result in system flag changes, keyword changes, message relocation, message removal, or a combination of these.The result of the command MUST be either OK, NO or BAD:
- The OK result MUST be returned only in case the server parsed and completed the command successfully.- The NO result MUST be returned only in case the server parsed the command successfully, but there is a problem with the referenced message(s) that prevents the server from completing the requested actions, such as one or more messages do not exist on the server, one or more messages are not properly authorized by URLAUTH, etc.- The BAD result MUST be returned only in case the server cannot parse the command, or a configuration error is preventing the server from completing the requested actions.When the result is OK, the response to a SET directive MUST be either KEYWORD, RELOCATE, RELOCATED, DELETE, or DELETED.When the result is OK, the response to a CLEAR directive MUST be either KEYWORD, RELOCATE, or RELOCATED.The server responses are:
- The KEYWORD response occurs in case this specific action has been explicitly requested by the client, or, in case the server decided that only the keywords should be updated either because it does not wish to give any recommendation to the client (RELOCATE or DELETE), or, because it does not have sufficient information either internally, or, from the spam aggregator service that it is configured to use.- The RELOCATE response occurs in case the server decided that the message should be relocated, however leaves this action to the client. The client MAY decide what to do with the message.- The RELOCATED response occurs in case this specific action has been explicitly requested by the client, or, in case the server decided that the message should be relocated and it performed relocation of the message to the appropriate location before the response was sent. The server MUST relocate the message by copying the message to the appropriate location and removing the original as if another connected client requested this action using an IMAP MOVE command.- The DELETE response occurs in case the server decided that the message should be deleted, however leaves this action to the client. The client MAY decide what to do with the message.- The DELETED response occurs in case this specific action has been explicitly requested by the client, or, in case the server decided that the message should be deleted, and performed deletion of the message before the response was sent. The server MUST delete the message as if another client performed this action.
The KEYWORD, RELOCATE and DELETE responses MUST include the list of flags/keywords that have been added or removed.
Added keywords MUST be prefixed with a plus sign ('+'), while removed keywords MUST be prefixed with a minus sign ('-').
The RELOCATED and DELETED responses MUST NOT include keywords.
The formal syntax of the actions is defined in .
This document extends the formal syntax defined in using the Augmented Backus-Naur Form (ABNF) notation specified in .
Note: all string literals are case insensitive.
Report single message as spam; no identified parts; server only flags the message and hints that it should be moved:
C: Z020 SREP SET SEQ 10S: Z020 OK [RELOCATE +$OMAEVVM10-spam-user-identified] SREP Completed.Report single message as spam; header and body identified; server adds the appropriate flags and hints that it should be deleted:
C: Z040 SREP SET SEQ 9 (header.from body.2)S: Z040 OK [DELETE (+$OMAEVVM10-spam-user-identified-field.from +$OMAEVVM10-spam-user-identified-body.2)] SREP Completed.Report single message as spam; no identified parts; server moves the message (may clear/set flags too, but that is irrelevant because the client will need to reconcile anyway).
C: Z060 SREP SET SEQ 8S: Z060 OK [RELOCATED] SREP Completed.Report single message as spam; no identified parts; server deletes the message.
C: Z080 SREP SET SEQ 6S: Z080 OK [DELETED] SREP Completed.Report single message as spam; no identified parts; client requests explicitly to delete the message, server deletes the message as the client requested.
C: Z100 SREP SET SEQ 4 DO DELETE NILS: Z100 OK [DELETED] SREP Completed.Report single message as no longer spam; no identified parts; server only clears the appropriate flags from the message.
C: Z020 SREP CLEAR SEQ 10S: A020 OK [KEYWORD -$OMAEVVM10-spam-user-identified] SREP Completed.Report single message as no longer spam; earlier the header and body were identified as spam; the server clears the appropriate flags.
C: Z040 SREP CLEAR SEQ 9S: A040 OK [KEYWORD (-$OMAEVVM10-spam-user-identified-field.from -$OMAEVVM10-spam-user-identified-body.2)] SREP Completed.Report single message as no longer spam; no identified parts; server moves the message (may set/clear flags, too but that is irrelevant because the client will need to reconcile anyway).
C: Z060 SREP CLEAR SEQ 8S: A060 OK [RELOCATED] SREP Completed.The new command specified in this document allows a client to save significant bandwidth by sending only reference(s) instead of full-blown spam reports that most if the time include the entire message.
By doing so, the responsibility of recording metadata and handling the message designated as spam has been shifted to the server side.
It is not the purpose of IMAP servers to deal with various aspects of spam reporting, such as creating and storing metadata, making the metadata available when new messages arrive, etc.
IMAP servers should take advantage of an aggregator service and perform the exchanges related to spam reporting in the background, delegating the work to the aggregator service.
Spam aggregator services are expected to collect and store metadata in very large volumes, evaluate the stored metadata and support queries to decide whether an incoming message is a spam or not.
Open Mobile Alliance (OMA) specified the enabler release that supports deploying such aggregator services.
The flows in the following sub-sections illustrate informative examples for various scenarios that may be triggered by the SREP command defined in .
The user finds a voicemail that he/she deems to be spam.He invokes the appropriate functions on the client to report the message as spam.The client reports the spam using the appropriate command to the server: SREP SET SEQ 10The server prepares the message referenced by the command for inquiry.The server queries its internal database for precedence.The server gets a 'not found' response from the internal database.The server queries an external database for precedence, such as an aggregator service based on .The server gets a 'not found' response from the external database as well.No records of the message are found; the server prepares the message to be recorded for future reference.The server reports the message as spam to its internal database.The server gets an 'ok' response from the internal database.The server reports the message as spam to external database, such as an aggregator service based on .The server gets an 'ok' response from the external database.The server checks the user's preferences and finds no guidance about handing the spam, so... it checks the service provider policies - and yet again, finds no instructions.In the end, lacking any sort of guidance, the end the server stores a keyword for the message, and... informs that client about that using the appropriate response: OK [KEYWORD +$OMAEVVM10-spam-user-identified] Completed.The client updates its representation of the voicemail repository and... the message turns red on the user interface.The user cheers.| 3 | | |
| |--------->| | |
| | |---- | |
| | | | 4 | |
| | || |
| | | 6 | |
| | ||
| | | 8 |
| | || |
| | | 11 | |
| | ||
| | | 13 |
| | |The author acknowledges and appreciates the work and comments from Josh Soref, Gaelle Martin-Cocher, Suresh Chitturi, Clara Severino and Christophe Le Thierry D'Ennequin.
This document constitutes registration of the SREP capability in the imap4-capabilities registry.
SREP command directives are registered by publishing a standards track or IESG-approved experimental RFC.
The registry is currently located at:
http://www.iana.org/assignments/spam-directive-registry
SREP command abuse types are registered by publishing a standards track or IESG-approved experimental RFC.
The registry is currently located at:
http://www.iana.org/assignments/spam-abuse-type-registry
SREP command reference types are registered by publishing a standards track or IESG-approved experimental RFC.
The registry is currently located at:
http://www.iana.org/assignments/spam-reference-type-registry
All registries are case insensitive.
This document constitutes the following registrations with IANA:
Note to RFC Editor: replace "[this document]" with the RFC number before publication.When an aggregator service is actively involved in a deployment, the service provider MUST ensure that:
- a mutual trust relation is in place between the IMAP server and the aggregator service, and,- the aggregator service does not leak any information.See additional security considerations in and , respectively.
&rfc2119;
Augmented BNF for Syntax Specifications: ABNFBrandenburg InternetWorking675 Spruce Dr.SunnyvaleCA94086USdcrocker@bbiw.netTHUS plc.99 Berkeley StreetGlasgowG37HRUKdcrocker@bbiw.netINTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1Networks and Distributed Computing, University of Washington15th Avenue NESeattleWA4545USMRC@CAC.Washington.EDUInternet Message Access Protocol (IMAP) - MOVE ExtensionSchweppermannstr. 8MuenchenD-81671Germany+49 89 4502 9758arnt@gulbrandsen.priv.noOracle800 Royal OaksMonroviaCA91016-6347USned+ietf@mrochek.comInternet Message Access Protocol (IMAP) - URLAUTH ExtensionNetworks and Distributed Computing, University of Washington15th Avenue NESeattleWA4545USMRC@CAC.Washington.EDUDeprecating the "X-" Prefix and Similar Constructs in Application ProtocolsCisco Systems, Inc.1899 Wynkoop Street, Suite 600DenverCO80202USpsaintan@cisco.comBrandenburg InternetWorking675 Spruce Dr.SunnyvaleCA94086USdcrocker@bbiw.netRackspacemnot@mnot.netThe Multipart/Report Content Type for the Reporting of Mail System Administrative MessagesCloudmark128 King St., 2nd FloorSan FranciscoCA94107USmsk@cloudmark.comThe Multipart/Report Content Type for the Reporting of Mail System Administrative MessagesLucent Technologies7291 Williamson RdDallasTX75214USGregV@ieee.orgMobile Spam Reporting 1.0, OMA-ERP-SpamRep-V1_0Open Mobile Alliance