Re: making psamp export congestion-aware

[Jennifer Rexford <jrex@research.att.com>]

Peter,

> We had to solve this same problem when developing sFlow (www.sflow.org). 
> We decided to use a time-based reservation scheme.

Great!  The past experience will be really useful here to make sure
we don't miss something important.

> The basic goal we had in designing the sampling agent was
> simplicity. A few simple mechanisms in the agent allows all the
> intelligence to be pushed to the collection application. In this case
> a timeout is set at the collector and the collector is responsible for
> managing the agent timeout, ensuring that the agent will stop
> transmitting if the collector fails.

Exactly.  Your points about reconfiguring the export address (to do
graceful failover of collectors) are important, too.

> As Jennifer pointed out, the only mechanisms needed in the agent to
> allow the collector to be congestion aware are: the ability to
> remotely set the sampling rate, ...

I would stress that we might want a separation between the *export
rate* (say, in bits/sec) and the *sampling rate* (say, in terms of a
selection filter and a packet sampling ratio).  In some cases, we
might need to make short term changes in the export rate (say, due to
transient congestion) without reconfiguring the sampling parameters.
Also, in some cases it may be hard to estimate how much data the
sampler might generate, which gives extra merit to having a separate
configuration of the (maximum) export rate and sampling parameters.
If the sampler (temporarily) generates data in excess of the export
rate, then packets can be dropped at the agent, and this can be
detected via missing sequence numbers (and perhaps other information
in the export stream about data dropped at the agent).

If the data volume is in excess of the configured export rate for 
a *sustained* period, of course the collector might choose to
reconfigure the sampling rate as well, rather than having the agent
discard so many records.

-- Jen

--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>