Protocols/OSCAR/Rate Limits

OSCAR Protocol

Introduction • Terms • Clients

Basic

Datatypes • FLAP • SNAC • TLV

UUIDs • Errors • Tool IDs

Host Interaction

Rate Limits • Migration • Messages

Other Services

ADMIN • ADVERT • ALERT

BART • BOS • BUCP • CHAT

CHAT_NAV

Tutorials

Sign On • BART • Rendezvous

ICBM • Locate • Buddies

Foodgroups

OSERVICE (0x0001)

LOCATE (0x0002)

BUDDY (0x0003)

ICBM (0x0004)

ADVERT (0x0005)

INVITE (0x0006)

ADMIN (0x0007)

POPUP (0x0008)

PD (0x0009)

USER_LOOKUP (0x000A)

STATS (0x000B)

TRANSLATE (0x000C)

CHAT_NAV (0x000D)

CHAT (0x000E)

ODIR (0x000F)

BART (0x0010)

FEEDBAG (0x0013)

ICQ (0x0015)

BUCP (0x0017)

ALERT (0x0018)

PLUGIN (0x0022)

UNNAMED_FG_24 (0x0024)

MDIR (0x0025)

ARS (0x044A)

To protect the server and other users from abusive clients, the server implements SNAC rate limiting. Rate limiting is done with a simple formula that calculates the average time between SNACs over the last few SNACs sent from the client to the server.

A client can optionally subscribe to notifications about its rates so that it can warn the user ahead of time or show UI about the rates. Even if the client does not subscribe, it will receive notifications when the rate limit has been reached and the server has started dropping SNACs. If the client continues to send SNACs, it will eventually be disconnected.

All SNACs are assigned to a Rate Class which controls the parameters to the rate limit forumla. Most SNACs are in the most lenient rate class by default, with SNACs like IM sending being in more strict classes. The rate formula is currentAvg = ((currentAvg * (windowSize -1)) + delta)/windowSize.

The currentAvg falling below certain thresholds causes the server to warn the client that it is about to be rate limited or disconnected. Once a client is rate limited, its average has to fall above the clear threshold before it can start sending SNACs again.

The actual parameters for the formula are not published in this document since they can change from time to time and are different depending on the current warning level and other things. A client can average around one IM every two seconds without being rate limited.

Client sends OSERVICE__RATE_PARAMS_QUERY

Server responds OSERVICE__RATE_PARAMS_REPLY

Client can send OSERVICE__RATE_ADD_PARAM_SUB to subscribe to specific rate changes.

Client can send OSERVICE__RATE_DEL_PARAM_SUB to unsubscribe to specific rate changes.

Server sends OSERVICE__RATE_PARAM_CHANGE when changes occur to any subscribed rate classes.

Datatype: Rate Parameters

Describes the rate parameters for a single rate class.

Name	Type	Notes
classId	uint16 (word)	Rate class being described
windowSize	uint32 (dword)	Number of events to count
clearThreshold	uint32 (dword)	Once rate limited the average has to reach this value in order to clear
alertThreshold	uint32 (dword)	Server will tell the client it is getting close to the limit
limitThreshold	uint32 (dword)	SNACs will be dropped below this value
disconnectThreshold	uint32 (dword)	Server will disconnect below this value
currentAverage	uint32 (dword)	Current value for the class; higher is better
maxAverage	uint32 (dword)	The maximum rate value; if the current value rises about this value it should be reset
lastArrivalDelta	uint32 (dword)	Last message was received this long ago
droppingSNACs	uint8 (byte)	Is the server dropping SNACs for this rate class

Datatype: Rate Class Members

Describes all the SNACs in a single rate class.

Name	Type	Notes
id	uint16 (word)	Rate class being described
numMembers	uint16 (word)	Number of SNACs in this rate class
snacIds	Array of SNAC__ID length numMembers	SNACs in this rate class

Class: OSERVICE__RATE_CODE

These are the codes used in OSERVICE__RATE_PARAM_CHANGE to describe the state of the rate class.

Name	Value	Notes
OSERVICE__RATE_CODE_CHANGE	0x01	Rate parameters have changed
OSERVICE__RATE_CODE_WARNING	0x02	Rate limit warning reached; if client does not slow down LIMIT state will be hit
OSERVICE__RATE_CODE_LIMIT	0x03	Rate limit reached; if client does not slow down client will be disconnected
OSERVICE__RATE_CODE_CLEAR	0x04	Rate limit cleared; client can send SNACs normally now

From Aleksandr Shutko: Rate-limits description

Rate-limits

Rate limits is a way to control client->server data flow. This is done by calculating rate level on every client snac. If client rate goes above alert server send warning, if it goes above limit server send warning and drop snacs from client, if it goes above disconnect level client disconnected from server and can't connect again for some time.      At some point in the logon sequence the client should send SNAC(01,06) which is the "rate request" packet. In reply, the server will send the "rate response" SNAC(01,07) which contain rate limit parameters formatted as follows:      First comes a word value telling you how many rate classes there are. Then for each class you get a structure like this:  xx xx   word   Rate class ID  xx xx xx xx   dword   Window size  xx xx xx xx   dword   Clear level  xx xx xx xx   dword   Alert level  xx xx xx xx   dword   Limit level  xx xx xx xx   dword   Disconnect level  xx xx xx xx   dword   Current level  xx xx xx xx   dword   Max level  xx xx xx xx   dword   Last time;  xx   byte   Current state; And after those you get another set of structures (one for each class) like this:  xx xx   word   rate group id  xx xx   word   count of pairs in group  xx xx xx xx   dword   family/subtype pair #1  ....   ....   ....  xx xx xx xx   dword   family/subtype pair #n      The rest of the structure is just count word pairs, a SNAC family and SNAC subtype for each SNAC that will use the rate information from this class.      If the number of classes received is zero you should not reply. If it is greater than zero, then you should reply with SNAC(01,08) - "rate acknowledge" which is just a list of words with the Class ID of each class you received.      Now for some more explanations about the protocol. For each rate class, ICQ can be in one of three states: "limited" (state 1), in which no data is sent; "alert" (state 2), which is when you're sending too fast, but you aren't yet limited; and "clear" (state 3), when everything is cool. Every time a packet is sent (assuming you aren't being "limited"), ICQ updates a "Last Time" value to keep track of the last sent time (as with the state value, there is a separate time for each rate class).      It also looks up the time since the previous packet was sent and uses that to keep a running average of time between packets (I refer to this as the rate level). This level is calculated using a window size which specifies how many of the previous times to take into account, as follows:      NewLevel = (Window - 1)/Window * OldLevel + 1/Window * CurrentTimeDiff      There is also a Maximum Level at which this value will be capped. This formula is for both server and client because they both should calculate current level. You can check if your calculations is ok - just send SNAC(01,06) in the middle of a session and compare server value from SNAC(01,07) with yours.      Once the new level has been calculated, ICQ updates your state as follows: if your level is less than the Limit Level your state will be set to 1 ("limited"); if it's greater than the Limit Level, but less than the Alert Level, your state will be set to 2 ("alert"); if it's greater than the Alert Level your state will be 3 ("clear"). If your state was already set to 1, the calculation is slightly different. It only compares your level against the Clear Level value - if it's greater than that, your state becomes 3 ("clear"), otherwise it remains at 1 ("limited"). And one more thing - icq often receive SNAC(01,07) with state=114 - this is ok, this mean that you not limited.      Incidentally, this calculation happens before your packet is actually sent, so if your state changes to "limited" at this point, the packet won't be sent. If your state is not "clear", ICQ will also start a timer for some time in the future (the actual duration is rather complicated so I won't go into that now) so that your state can be recalculated, giving you a chance to get back to "clear".      Now that you know the basic protocol, some of the SNAC parameters will make more sense. The Window Size is the Window mentioned in calculating the running average time between packets (the level). The Clear, Alert and Limit Levels are used in calculating the rate state. Disconnect level is the level at which the ICQ server will disconnect you so it doesn't used by the client. The Current level is what the level is initially set to you when SNAC 1/07 is received. The Max Level is what the level is capped at when calculating the running average.      The Last Time is a duration in milliseconds which is used to set the initial value of the last sent time (the time is set to the specified duration into the past, i.e. 1000 milliseconds means 1 second ago). The Start State specifies the initial rate state, and although the state is immediately recalculated, the current state does effect that calculation (as explained above).