Syslog Sending Policy Messages
Huaweisymantec Inc.
Building 17
Zhongguancun Software Park
No.8 Dongbeiwang West Road
Beijing
100085
PRC
+86 010 58791314
washam.fan@huaweisymantec.com
This document defines special syslog messages called Sending
Policy messages
for indicating how syslog senders process syslog
messages before sending them.
The information Sending Policy messages convey is of interest to syslog
receivers and helpful for audit.
Generally, syslog senders would send all generated messages to targeted receivers. But there would be a different story in some circumstances. For some reason, syslog system may buffer messages before sending them further. When some important messages have been hold in sending queue for a long time, the senders would like to send those urgent messages timely even out of order they are in sending queue or they were generated. When transport senders are going to drain their sending queue, they would not be able to send all the messages timely. In this case, senders would filter out less important messages to reduce queue overload or store new coming messages persistently in disk for sending them later.
These scenarios may be not rare in real world. When congestion control is applied to syslog transport, some mechanisms will be employed to adjust sending rate based on feedback approach. When network congestion occurs, in order to reduce package loss in transit and improve network throughout, sending rate will slow down. Congestion control increase the latency of messages sending and the chance sending queue is full. Message burst due to malicious behaviors is likely to exhaust sending queue, too.
When senders process messages before sending them, something would have happend, e.g, network congestion occurs or intrusion led to log messages burst. So the event that senders process messages before sending them might be an alert to operators. Additionally, the information how senders process messages before sending them is of interest to receivers and helpful for audit, e.g, the information which messages are more important than others may be of value to receivers. For an instance, when receivers can identify the more important
messages from the sender's perspective, it could handle this event timely
when many messages near-simultaneously arrive. For another instance,
taking into account that the sender has dropped which messages during which period of time
may help us
do a more accurate audit. This
document defines special messages called Sending Policy messages to notify this kind of event and deliver
this information.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this
document are to be interpreted as described in RFC 2119 [RFC2119].
Message refer to syslog message whose on-the-wire format is
defined in section 6, [RFC5424].
Sender refer to transport sender which is defined in section 3,
[RFC5424].
Receiver refer to transport receiver which is defined in section
3, [RFC5424].
Sending queue is a conceptual buffer for buffering messages to
be sent in sender's side.
Receiving queue is a conceptual buffer for buffering messages to
be received and processed in receiver's side.
A Sending Policy contains an action senders take on messages before sending them and criteria which are used to evaluate the importance of messages. Actions should be taken based on criteria.
According to what kind of actions senders would take, Sending Policies can be probably divided into three category:
When latency occurs, senders would like to send the more
important messages timely even out of order the messages were generated or are in sending queue. How to evaluate the importance of messages
depend on what criteria senders adopts.The criteria are of interest to
the receiver, as the
receiver can also pick up important messages to be processed timely
when it
starves, i.e, when so many near-simultaneously messages arrive.
Priority Policy indicates messages would be sent based on level of importance even out of order they were generated or are in sending queue. Priority Policy also contains the criteria adopted to evaluate the importance of messages.
When the number of messages to be sent increases,
it is possible to drain the sending queue. Senders may simply dropping
messages to alleviate sending queue load.
Comparing to messages dropped by routers in transit, senders has more
sense
what messages they'd like to abondon whilst the others
preserved. We assume senders always drop less important messages.
This policy may do not affect receiver's behavior
very much. But when audit later, it would do more accurate analysis by
taking criteria which are used to evaluate importance of messages into
account.
Filtering Policy indicates some messages would be dropped based on their importance. Filtering Policy also contains the criteria adopted to evaluate the importance of messages.
As an alternative to Filtering Policy, Senders may want to
store the messages persistently for more reliability but less
timeliness. This policy may not affect receiver's bahavior very much. The event that this policy has been applied may be of
interest to receivers. It would be considered as an alert indicating congestion
occurs or messages burst.
Persistency Policy do not care about importance of messages, since no messages dropping and no messages sent out of order. With regard to Priority Policy and Filtering Policy, they both need to identify the more important messages to be processed before sending them. There are commonly 2 ways to evaluate importance of syslog messages:
Priority Value consists of two parts: Facility and Severity. Facility value takes value from 0 to 23. Severity value takes value from 0 to 6. In both cases, the lower the value is, the more important the message is.
Timestamp Value is not mandatory in syslog messages, but when it shows up, it indicates the time when message was generated. When we filter out messages, we can simply dropping new coming messages or let new coming messages override the old ones(e.g, circular queue). The latter takes newer message more valuable, but the former takes newer message less important. Priority Policy would not base on these criteria.
Sending Policy Messages are used to contain policies specified in section 4 and the event when specific policy was applied or revoked. These information are carried as Sending Policy Blocks in Sending Policy Messages actually.
There is a need
to distinguish the Sendig Policy Block itself from the
syslog message that is used to carry a Sending Policy Block. Sending Policy
Blocks MUST be encompassed within completely formed syslog
messages.
Syslog messages that contain a Sending Policy Block are also referred to
as Sending Policy messages.
A Sending Policy message is identified by the presence of an SD
ELEMENT with an SD-ID with the value "sending-policy". In addition, a Sending
Policy message MUST contain valid APP-NAME, PROCID, and
MSGID fields to be compliant with [RFC5424]. This specification
does
not mandate particular values for these fields; however, for
consistency, originators MUST use the same values for APP-NAME,
PROCID, and MSGID fields for every Sending Policy message that is
sent, whichever values are chosen. To allow for the possibility of
multiple originators per host, the combination of APP-NAME, PROCID,
and MSGID MUST be unique for each such originator. If an
originator
daemon is restarted, it MAY use a new PROCID for what is otherwise
the same originator but MUST continue to use the same APP-NAME and
MSGID. The Sending Policy Block is carried as
Structured Data
within the Sending Policy Block message, per the definitions that follow
in the subsection. The MSG part of a Sending
Policy message SHOULD be any readable text for explaining the Sending Policy Block.
The Sending
Policy Block MUST be encoded as an SD ELEMENT, as defined in
[RFC5424].
The SD-ID MUST have the value of "sending-policy".
The SDE contains the fields of the Sending Policy Block encoded as
SD Parameters, as specified in the following. The Sending Policy Block is
composed of the following fields. The value of each field MUST be
printable ASCII.
Field
SD-PARAM-NAME
Size in octets
Version
VER
2
Type
TYPE
1
TimeType
TT
1
TimeValue
TV
1-50
Criteria
CRI
1
Threshold
THRE
1-10
The fields MUST be provided in the order listed if they were present. Each SD
parameter
MUST occur once in the Sending Policy Block. Version is mandatory. A Sending Policy
Block is likely to be accordingly encoded as follows, where xxx denotes a
placeholder for the particular values:
[sending-policy VER="xxx" TYPE="xxx" TT="xxx" TV="xxx" CRI="xxx"
THRE="xxx"]
Values of the fields constitute SD parameter values and are hence
enclosed in quotes, per [RFC5424]. The fields are separated by
single spaces and are described in the subsequent subsections.
The Version field is a numeric value that has
a length of 2 octets, which may include leading zeroes. The two octets
contain a decimal character in the range of "0" to "9". The value
in this field specifies the version of this Block with "01" as the
value for the
protocol version that is described in this document.
This field indicates which type of policies specified in
section 4 the sender has adopted. The TYPE parameter may take any
value from 0-2 inclusive.
"0" -- Priority Policy has been adopted by the
sender.
"1" -- Filtering Policy has been adopted by the
sender.
"2" -- Persistency Policy has been adopted by the
sender.
The two fields show us when the sender starts to apply the
policy indicated in TYPE field or when the sender revokes it, that
is dependent on TimeType Value. TT parameter may take any value from
0-1 inclusive
"0" -- Subsequent TimeValue field represent start
time.
"1" -- Subsequent TimeValue field represent end time.
When Priority Policy was applied, if TT Value equals "0", TV
Value SHOULD be the TIMESTAMP Value of the first message out of order;
if TT Value equals "1", TV Value SHOULD be the TIMESTAMP Value of the
last message out of order.
When Filtering Policy was applied, if TT Value equals "0", TV
Value SHOULD be the TIMESTAMP Value of the first message dropped; if
TT Value equals "1", TV Value SHOULD be the TIMESTAMP Value of the
last message dropped.
When Persistency Policy was applied, if TT Value equals "0" or
"1", TV Value SHOULD be the system time when the Policy message was sent(I.e., TV Value equals to TIMESTAMP Value in message HEADER).
sending-policy SDE can convey only criteria for importance evaluation, in this case, no indicator for policy has been applied or revoked, so this two field can be absent.
TV parameter is a formalized timestamp derived from
[RFC3339] and follow the restrictions specified in [RFC5424](section
6.2.3).
Criteria field indicates the criteria which are used to
evaluate the importance of messages. Criteria parameter may take any
value from 0-2 inclusive.
"0" -- Evaluate importance of a message based on Severity Value which is a part of PRI Value contained in HEADER part.
"1" -- Evaluate importance of a message based on Facility Value which is a part of PRI Value contained in HEADER part.
"2" -- Evaluate importance of a message based on Timestamp Value contained in HEADR part.
When Criteria field takes value "2", Threshold Value indicates
whether earlier or older messages are considered more important. In
this case, Threshold Value can be intepreted in this way:
"0" -- the earlier messages were generated, more important
they were.
"1" -- the older messages were generated, more important
they were.
When Criteria field takes value "0" or "1", Threshold Value indicates the border of more important messages and less important messages.
E.g, when Criteria
parameter takes value "0", and, Threshold Value equals "3",
that means any messages bearing severity Value greater than 3 may be
dropped(when Filter Policy was adopted) or delayed their sending(when
Priority Policy was adopted).
When Persistency Policy was adopted, these two fields can be
absent.
Sending Policy messages can only convey criteria for importance evaluation, in this case, Version, CRI and THRE MUST be present. E.g., senders would send this messages following the session is established.
The following example shows the criteria are that messages bearing Severity Value which is less than or equals to 3 are more important.
[sending-policy VER="01" CRI="0" THRE="3"]
When Sending Policy messages are used as event notification, TT and TV MUST be present, CRI and THRE MUST be present unless criteria for importance evaluation have been delivered before. two Sending Policy messages are recommended to indicate event start and event end respectively. In this case, Sending Policy messages SHOULD have enough high priority to be sent timely and avoid being filtered out.
The following example shows Filtering Policy has been adopted by the sender and the first affected message(i.e. the first message dropped) was generated on Feb 24 2009 05:14:15am, 3 microseconds into the
next second( and the local time is -7 hours from UTC). The CRI and THRE are interpreted as last example
[sending-policy VER="01" TYPE="1" TT="0" TV="2009-02-24T05:14:15.000003-07:00" CRI="0" THRE="3"]
This specification shares the same security considerations addressed in section 8.5, [RFC5424]. Additionally, two issues should be taken care of.
When Priority Policy was applied, syslog messages may be sent out of order they were generated. The receivers should not reconstruct sequence of events based on the order they were received. In this case, TIMESTAMP may be used to do this work. Besides, when Priority Policy message is used as real-time event notification, syslog-sign may not work well in this case, since the urgent message sent out of order can not be verified until the corresponding Signature message has been interpreted. By the same token, receivers would not process messages out of order they were received.
When Filtering Policy was applied, syslog messages may be lost by senders. In this case, Sending Policy messages could tell what kind of messages was dropped. If syslog-sign is also applied, syslog-sign can tell how many and which messages was dropped. It is recommended that Signature Block messages have enough high priority to avoid being filtered out by senders easily.
If UDP-like congestion control is adopted for syslog transport, Sending Policy messages themselves would probably be lost in transit. Sending Policy messages should be sent several times in this case.
IANA is requested to register the SD-ID value "sending-policy" together with
the PARAM-NAME values specified in Section 5.1 in the registry for
SYSLOG structured data id values according to section 9 in
[RFC5424].
SD-ID
PARAM-NAME
sending-policy
VER
mandatory
TYPE
optional
TT
optional
TV
optional
CRI
optional
THRE
optional
[RFC5424] Gerhards, R., "The syslog Protocol", RFC 5424,
December 2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet:
Timestamps", RFC 3339, July 2002.