Subject: [domainrep] Possible changes to the standard (HTTP)

From: Steve Allam <[email-masked]>
Date: Mon, 18 Jul 2011 14:21:30 +0100

All,

The other interface we use to connect to our 'reputation engine' (the 
LogiQ product) uses an HTTP GET query.  A typical query would be:

http://logiq.server.com:8000/check_sender?i=193.4.5.2&s=[email-masked]&r=[email-masked]&d=inbound

and this can be extended using a variety of other parameters such as:

http://logiq.server.com:8000/check_sender?i=193.4.5.2&s=[email-masked]&r=[email-masked]&spf=pass&dkim=pass&v=spam&subject=test_message&d=inbound

If we try and encode that in the current proposed standard, we can get 
as far as perhaps:

http://logiq.server.com:8000/email/here.com/sender-rep

But then need to define the sender and other parameters.  This is my 
main comment - there needs to be a mechanism for people to add 
extensible parameters to the basic query.

When we decided on the query were were going to use, we took a number of 
factors into consideration:

1.  Where will the url parsing take place?
     - we decided to keep the url fixed for all queries, and pass the 
variable data in via parameters.  Mainly this was due to having many 
parameters to parse, and keeping the http engine for what it was good 
at, passing all the parameters to the application to handle.  We do 
however do some work in the HTTP server (nginx) based on the params - 
for instance, depending on whether we get d=inbound or d=outbound, we 
set different rate limiting values.  However, the basic premise was - 
keep the application separate from the web server - although clearly an 
application is quite capable of parsing a URL as well as a param stream.

2. Do we use a GET or a POST to provide params?
     - we decided that as we weren't sending extended parameter data, we 
could live without the extra overhead of a POST command - one of the 
purposes of our engine is to work as fast as possible after all.
     - parameter names were again simply made as small as possible to 
ensure the query fitted into as few packets as possible - a little thing 
perhaps....

3.  Responses....
     - LogiQ is capable of returning responses in three formats:
         - XML:  we wanted to use this as it is nicely extensible.  In 
practice, not many of our interfaces use it; partly because when people 
start using it they load up an XML parser and process the 
result....whoops...that's gone slow for some reason....
         - Text:   As the response line is usually a single line, its 
simple.....
         - HTTP response code - this is the fastest and used directly in 
out interface with the Cloudmark Gateway - as it can support direct HTTP 
get queries in its rules the LogiQ interface needs no extra modules.  In 
order to get the quickest integration, the gateway merely uses the 
response code to determine reputation, and can drill down into the 
response body if more information is needed.  I don't expect this to be 
considered for a standard however.  I can't remember the exact responses 
we use but its something like 201 for good reputation, 404 for unknown 
and 503 for a bad reputation.

I know perhaps I have rambled a bit here - but I hope it gives an idea 
of how we are using similar mechanisms for reputation query/response - 
please let me know if you have any questions.

Regards,

Steve
-- 
*
Steve Allam | * Chief Technology Officer *| TrustSphere *

3 Phillip Street, #13-03 Commerce Point, Singapore, 048693
Tel: +65 6536 5203 | Fax: +65 6536 5463
[email-masked] | www.trustsphere.com