Subject: [saag] Algorithms/modes requested by users/customers

From: Randall Atkinson <[email-masked]>
Date: Sat, 16 Feb 2008 15:42:57 -0800

Earlier, someone said:
% I think it would help enormously if we had some sort of
% cross IETF statement of the set of algorithms that are
% currently the consensus recommendations for support.

I will answer a slightly different question.  For the question:
     "What algorithms/modes are most paying customers asking for ?"
the answers turn out to be:

1) NIST FIPS-140 conforming algorithms/modes.
and
2) Suite-B conforming algorithms/modes.

Approximately speaking, (2) above is a subset of (1) above.

The IETF might make some different decision than those,
but equipment vendors will have to implement (1) or (2)
to please most commercial users (e.g. banks, insurance firms,
stock brokerages/markets, top international commercial
firms in other areas).  So whether or not these are specified
by IETF on the standards-track, there is interoperability value
in having open specifications (e.g. Informational RFC would
do quite nicely) for (1) and (2) for nearly any Internet-related
protocol using cryptography.

This seems to be driven externally by insurance firms that tell
their customers to only use equipment whose cryptographic
subsystems/modules have been (or are going to be) evaluated
formally under FIPS-140.

And I'll note that this are not really driven particularly by US firms.
European, Asia/Pacific, and Latin American firms are making the
exact same requests for FIPS-140 of their equipment vendors.

Yours,

Ran
[email-masked]

---

From: mcgrew <[email-masked]>
Date: Tue, 19 Feb 2008 11:27:32 -0800

Hi Ran,

Winston Churchill said that democracy is the worst form of government,
except for all of the others.  I think that the same is true for the
FIPS-140 cryptomodule validation process ;-)

I agree with you that there is good value in having open specs for FIPS-140
and Suite-B versions of Internet protocols (even though I also agree with
many of the criticisms of the validation process made on this thread).   I
expect that the choice of which algorithm(s) should be
mandatory-to-implement will continue to be made on the basis of technical
(well, mostly technical) discussions in the WGs.

Best,

David

On 2/16/08 3:42 PM, "Randall Atkinson" <[email-masked]> wrote:

> _______________________________________________
> saag mailing list
> [email-masked]
> http://mailman.mit.edu/mailman/listinfo/saag

---

From: [email-masked] (Peter Gutmann)
Date: Thu, 21 Feb 2008 00:36:39 +1300

mcgrew <[email-masked]> writes:

>Winston Churchill said that democracy is the worst form of government, except
>for all of the others.  I think that the same is true for the FIPS-140
>cryptomodule validation process ;-)

I think it's more a case of the Politician's Fallacy:

1. Something must be done.
2. This is something.
3. This must be done.

It'd be interesting to see a study of the effectiveness in terms of finding
security and interop problems of:

A. A FIPS 140 eval.

B. Running the code through Fortify/Coverity/whatever and completing a crypto
   exchange with a peer (TLS, S/MIME, PGP, whatever the underlying crypto is
   that's being used).

in particular in terms of return for effort-involved.

Peter.

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Wed, 20 Feb 2008 13:10:48 +0000

On Thu, 21 Feb 2008 00:36:39 +1300
[email-masked] (Peter Gutmann) wrote:

> >Winston Churchill said that democracy is the worst form of
> >government, except for all of the others.  I think that the same is
> >true for the FIPS-140 cryptomodule validation process ;-)
> 
> It'd be interesting to see a study of the effectiveness in terms of
> finding security and interop problems of:
> 
> A. A FIPS 140 eval.
> 
> B. Running the code through Fortify/Coverity/whatever and completing
> a crypto exchange with a peer (TLS, S/MIME, PGP, whatever the
> underlying crypto is that's being used).
> 

Right.  But here's the problem with this choice: FIPS-140 is mostly
about assurance of security, and not just correctness of the crypto.
Given the really bad mistakes we've all seen -- things that would be
caught by any decent outside evaluation -- what is the alternative?
What is the *assurance* a customer has that the product is adequately
secured?

		--Steve Bellovin, http://www.cs.columbia.edu/~smb

---

From: [email-masked] (Peter Gutmann)
Date: Thu, 21 Feb 2008 02:50:42 +1300

"Steven M. Bellovin" <[email-masked]> writes:

>FIPS-140 is mostly about assurance of security, and not just correctness of
>the crypto.

Is it about assurance of security, or assurance of production of paperwork
showing the certification conditions have been met?  There have been plenty of
security advisories issued for CC and FIPS-140 evaluated products (and even
more not publicised but silently fixed in a certification-voiding manner).

>Given the really bad mistakes we've all seen -- things that would be caught
>by any decent outside evaluation -- what is the alternative? What is the
>*assurance* a customer has that the product is adequately secured?

Politician's Fallacy again: Is FIPS 140 really the best way to spend your
money?  If FIPS 140 is the answer now, why wasn't the Orange Book the answer
then?  What about giving the money to (picking a random name) Cigital and
saying "make sure this code is OK"?  What about giving the money to Dan
Bernstein and saying "implement this and make it secure"?  What about having
the code written by Germans and pen-tested by the French? [0].  We have no
hard data either way (although I'd put my money on Cigital and Dan to produce
the more secure product :-).  But simply saying "We must use FIPS 140...
just... well, just because!" is hardly a scientific approach to solving the
problem.

Peter.

[0] A very much under-exploited strategy in security evaluation.

---

From: Randall Atkinson <[email-masked]>
Date: Wed, 20 Feb 2008 07:27:19 -0800

Earlier, Peter Gutmann wrote:
% Politician's Fallacy again: Is FIPS 140 really the best way to spend your
% money?

If someone has a better proposal, I am very sure that there is a large
audience that would love to hear it. (More on this at bottom)

% If FIPS 140 is the answer now, why wasn't the Orange Book
% the answer then?

You are comparing apples to oranges above.

FIPS-140 is only about assurance for cryptographic modules.
Orange Book (TCSEC) was only about operating system security.

The two address different issues.

% What about giving the money to (picking a random name) Cigital and
% saying "make sure this code is OK"?

One needs a process that is as consistent and reproducible as practical
-- no human process could ever be 100% consistent and reprodcible --
otherwise implementers will legitimately complain about a non-level
playing field.  Or were you proposing to setup a monopoly ?

FIPS-140 has multiple certification labs in multiple countries evaluating
products -- to avoid creating a monopoly.  This HAS driven the evaluation
costs downwards over time, and it permits implementers the choice
to trade more money for less evaluation time.

I don't think anyone has claimed FIPS-140 is perfect.  The claims (not by
me so much as by other folks on the SAAG list) have been that (1) FIPS-140
 is better than other extant security evaluations and that (2) so far
no serious alternative proposal that looks reasonably better has appeared.

If you think that FIPS-140-* is a target-rich environment, then please
try to seriously propose something better.  I understand NIST and its
partners are looking to evolve into FIPS 140-3 from FIPS 140-2.

Have you sent them any concrete suggestions for improvement ?
I know the folks at NIST are happy to listen to any serious inputs
or proposals.

Cheers,

Ran

---

From: "Santosh Chokhani" <[email-masked]>
Date: Wed, 20 Feb 2008 10:32:39 -0500

Orange Book was not about OS security alone.

It was successfully applied to network components and DBMS systems also.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Randall Atkinson
Sent: Wednesday, February 20, 2008 10:27 AM
To: Peter Gutmann
Cc: [email-masked]
Subject: Re: [saag] Algorithms/modes requested by users/customers

Earlier, Peter Gutmann wrote:
% Politician's Fallacy again: Is FIPS 140 really the best way to spend
your
% money?

FIPS-140 has multiple certification labs in multiple countries
evaluating
products -- to avoid creating a monopoly.  This HAS driven the
evaluation
costs downwards over time, and it permits implementers the choice
to trade more money for less evaluation time.

I don't think anyone has claimed FIPS-140 is perfect.  The claims (not
by
me so much as by other folks on the SAAG list) have been that (1)
FIPS-140
 is better than other extant security evaluations and that (2) so far
no serious alternative proposal that looks reasonably better has
appeared.

Cheers,

Ran

---

From: Randall Atkinson <[email-masked]>
Date: Wed, 20 Feb 2008 07:39:59 -0800

Earlier, Santosh Chokhani wrote:
% Orange Book was not about OS security alone.
%
% It was successfully applied to network components and DBMS systems also.

Santosh,

I was using terms precisely, not loosely.

The network evaluations were done against Red Book (TNI)
published as NCSC TG-005, and not against the Orange Book
proper.

DBMS evals were done against the Purple Book, published
as NCSC TG-021, and not against the Orange Book
proper.

(I have a full set of colouring books to hand from
a previous life.  :-)

Cheers,

Ran

---

From: "Santosh Chokhani" <[email-masked]>
Date: Wed, 20 Feb 2008 10:41:33 -0500

The other books were no more than the interpretations so that we can
reuse the standard.

Except for TNI Part 2, the source document and requirement for rainbow
series is TCSEC.

-----Original Message-----
From: Randall Atkinson [mailto:[email-masked]] 
Sent: Wednesday, February 20, 2008 10:40 AM
To: Santosh Chokhani
Cc: [email-masked]
Subject: RE: [saag] Algorithms/modes requested by users/customers

Earlier, Santosh Chokhani wrote:
% Orange Book was not about OS security alone.
%
% It was successfully applied to network components and DBMS systems
also.

Santosh,

Cheers,

Ran

---

From: Randall Atkinson <[email-masked]>
Date: Wed, 20 Feb 2008 07:45:13 -0800

Earlier Santosh wrote:
% The other books were no more than the interpretations so that we can
% reuse the standard.
%
% Except for TNI Part 2, the source document and requirement for rainbow
% series is TCSEC.

So then you do agree with me, Network evals and
DBMS evals used the other volumes that I cited,
because the Orange book didn't really cover either
networking or DBMS topics.

I was involved in formal evaluations for the US DoD,
while a DoD employee in a prior life, so I've got
specific first hand experience with this topic.

:-)

Cheers,

Ran

---

From: "Santosh Chokhani" <[email-masked]>
Date: Wed, 20 Feb 2008 10:49:32 -0500

Folks like me when we had issues, we focused on pragmatics and the TCSEC
and not the two interpretations you cite.

-----Original Message-----
From: Randall Atkinson [mailto:[email-masked]] 
Sent: Wednesday, February 20, 2008 10:45 AM
To: Santosh Chokhani
Cc: [email-masked]
Subject: RE: [saag] Algorithms/modes requested by users/customers

Earlier Santosh wrote:

:-)

Cheers,

Ran

---

From: Jon Callas <[email-masked]>
Date: Wed, 20 Feb 2008 08:54:00 -0800

> no serious alternative proposal that looks reasonably better has

I agree that both of these are true, but these true statements are not  
inconsistent with the complaints.

A system can be the best available, as well as there being no "serious  
alternative" and still be inadequate.

I think this is the true problem.

I hope that we're not saying that FIPS 140 is the best of all possible  
systems.

	Jon

---

From: Stephen Kent <[email-masked]>
Date: Wed, 20 Feb 2008 17:51:41 -0500

>security and interop problems of:

since I and others have pointed out several times that FIPS 140 eval 
has nothing to do with protocol interoperability, the reference to 
"interoo" above must be viewed purely within the context of crypto 
algorithms and modes thereof.

>in particular in terms of return for effort-involved.

FIPS 140 encompasses both hardware and software implementations of 
crypto modules. I see its greatest benefits in the context of the 
former. The process described above does not address hardware 
security module eval at all.

Steve

---

From: mcgrew <[email-masked]>
Date: Thu, 21 Feb 2008 07:00:32 -0800

Hi Peter,

> 1. Something must be done.

I like that.

> in particular in terms of return for effort-involved.

I share you interest in the automation of validation testing; the more that
can be automated, the better.  It would be great to see more work in this
area. 

David

---

From: [email-masked] (Peter Gutmann)
Date: Tue, 26 Feb 2008 04:44:00 +1300

>The two address different issues.

They certainly seem to be addressing them in very similar ways.

>Or were you proposing to setup a monopoly ?

Ten years ago the rule-of-thumb estimate for a FIPS 140 level 1 eval was $100K
and 6-12 months no matter who you went to.  In 2006 (last time I checked) the
rule-of-thumb estimate for a FIPS 140 level 1 eval was $100K and 6-12 months
no matter who you went to.  So it looks like we already have a well-
established oligopoly with FIPS 140 evals, the only difference is the
nomenclature.

>One needs a process that is as consistent and reproducible as practical

Why?  This is the ISO 9000-for-software fallacy, reproducibility is great when
you're cranking out lightbulbs, but not much use for software, which isn't a
mass production process but instead is based on the cloning of the result of a
one-off development effort which is the product of the creativity, skill, and
co-operation of developers and users.  I can easily get 100% consistency in
reproducing an executable on a CDROM, but I don't think that's what's meant
here.

>If you think that FIPS-140-* is a target-rich environment, then please try to
>seriously propose something better.  I understand NIST and its partners are
>looking to evolve into FIPS 140-3 from FIPS 140-2.

It depends on what you want from FIPS 140 (and I'm talking specifically FIPS
140 for software modules here, which is the form that 99.9% of users encounter
it in ).  Some people have said you get "assurance", but the assurance you're
getting here is a high level of assurance that the vendors were desperate
enough for government sales that they shelled out a large amount of money in
order to get a ticket to ride the gravy train.

The problem is that like "religion" or "morals", "assurance" is a loaded term
and can be interpreted to mean almost anything.  Without some basic definition
of what's meant by "assurance", it's not really possible to reason about this.
So here's an attempt to nail it down: For me (and I would guess most end
users) assurance is being able to sleep at night knowing there's little chance
of anyone compromising my system ("things that should happen do happen, things
that shouldn't happen don't happen").

Let's see what FIPS 140 gives you to help you sleep at night.  It tests some
crypto mechanisms, but ignores others.  If you're buying an IPsec product it
doesn't test all the mechanisms needed for IPsec.  If you're buying a TLS
product it doesn't test the mechanisms needed for TLS.  If you're buying...
well, you get the idea.

Compare this to the example I gave earlier of performing a TLS exchange with
Amazon.  This performs an in-depth test of all the crypto algorithms
(corresponding to the FIPS algorithm tests, including ones that FIPS ignores),
and the crypto mechanisms (many/most of which FIPS again ignores).  In other
words simply by connecting to Amazon using TLS and ordering a "Scrubs" DVD for
$19.95 I'm getting more comprehensive algorithm testing than I can for a five-
figure sum with the FIPS algorithm tests.

So what else do you get?  You get a bit of a code review and some checking
that you don't do stupid things with keys.  Only some of the code is reviewed
(for example that trivial buffer overflow in the client handshake that allows
a remote attacker to do whatever they want with your server, including the
FIPS 140-evaluated portion, isn't checked because it's outside the scope of
what FIPS 140 is interested in), and (from having gone through several of
these) the review itself seems to be more coding style nitpicks than looking
for potential exploits, and in any case you can often document away any
problems without having to fix them.

So the assurance a FIPS 140 eval gets you is a tautological "the product
vendor was able to pass a FIPS 140 eval".  In terms of being able to
demonstrate this type of assurance, I agree completely with you that there is
no better mechanism for it than FIPS 140.

Admittedly there is some value to FIPS 140: You know that a subset of the
crypto used is OK, that the code doesn't do anything stupid with its keys
(unless the vendor has documented away the stupid things that do happen, e.g.
CryptoAPI's plaintext private-key export, you ask it "Give me the user's
private key in plaintext form" and it says "Sure, here you go"), and that it's
had a bit of a code check (and obviously if you're doing hardware crypto you
get a lot more value than this, but the discussion here is about software
modules).

What you don't get from FIPS 140 is the stuff that customers really care
about: Assurance that your IPsec product can actually do IPsec, that your TLS
product can do TLS, or that the product really is secure(-ish).  Excusing this
by saying that a FIPS 140 crypto product eval doesn't have to check half your
crypto so this isn't a problem is like saying that your TSA passenger-
screening process is OK even though it doesn't bother checking men with beards
or anyone under 5'6" tall.

If I'm going to spend $100K to feel safe then I can really think of better
ways than FIPS 140 (or CC, or ITSEC, or ...).  How about the following
comparison: You get to spend the cost of a FIPS 140 eval in one of two ways:

$100K: FIPS 140

-- or --

$20K: Fortify (or whatever) scan.
$20K: Cigital (or whatever) audit/analysis (these are just representative
      figures/estimates since there don't seem to be any fixed rates for this
      sort of thing).
$5K: Grant to French university to pen-test code "written at a German
     university".
$5K: Grant to British university to pen-test code "written at a French
     university".
$5K: Set up a fake banking site running the code and bring it to the attention
     of the Russian Business Network.
$free: Leak "security code for Windows 7" to Slashdot.
$free: Leak "security code for BluRay++" on warez boards.
$free: Leak "Diebold voting machine security code" to Cryptome.
$45K: 50" plasma TV, home theatre system, and beer to kill time while you
      wait for the results to come in (and to emphasise how much cheaper
      this is than FIPS 140 :-).

If your life depended on the security of the product that went through those
two processes, which one would you trust?  FIPS 140 is just not cost-effective
in providing assurance - it costs too much, it takes too long, and it doesn't
evaluate the stuff that real users care about having evaluated.

>The claims (not by me so much as by other folks on the SAAG list) have been
>that (1) FIPS-140 is better than other extant security evaluations

Thus showing that the politician's fallacy is alive and well :-).

There was a long and extremely interesting discussion on the (somewhat
misnamed) open-source list recently, thread topic "How can high assurance
FLOSS development be encouraged?".  The archives are at
http://lists.csl.sri.com/mailman/listinfo/open-source, but you have to sign up
to the list in order to access them.

Peter.

---

From: "Santosh Chokhani" <[email-masked]>
Date: Mon, 25 Feb 2008 12:20:43 -0500

Peter,

You are wrong about FIPS 140-1 costs being 100K for Level 1.  It is more
like 30K.

In terms of what FIPS buys is that you ensure that the algorithm is
implemented correctly, keys will be generated in accordance with FIPS
(meaning that the seed feeding the PRNG will have requisite entropy and
PRNG will be FIPS approved).

You also get the assurance that the keys are being managed properly in
the crypto module.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Peter Gutmann
Sent: Monday, February 25, 2008 10:44 AM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [saag] Algorithms/modes requested by users/customers

>Or were you proposing to setup a monopoly ?

Ten years ago the rule-of-thumb estimate for a FIPS 140 level 1 eval was
$100K
and 6-12 months no matter who you went to.  In 2006 (last time I
checked) the
rule-of-thumb estimate for a FIPS 140 level 1 eval was $100K and 6-12
months
no matter who you went to.  So it looks like we already have a well-
established oligopoly with FIPS 140 evals, the only difference is the
nomenclature.

>One needs a process that is as consistent and reproducible as practical

Why?  This is the ISO 9000-for-software fallacy, reproducibility is
great when
you're cranking out lightbulbs, but not much use for software, which
isn't a
mass production process but instead is based on the cloning of the
result of a
one-off development effort which is the product of the creativity,
skill, and
co-operation of developers and users.  I can easily get 100% consistency
in
reproducing an executable on a CDROM, but I don't think that's what's
meant
here.

>If you think that FIPS-140-* is a target-rich environment, then please
try to
>looking to evolve into FIPS 140-3 from FIPS 140-2.

It depends on what you want from FIPS 140 (and I'm talking specifically
FIPS
140 for software modules here, which is the form that 99.9% of users
encounter
it in ).  Some people have said you get "assurance", but the assurance
you're
getting here is a high level of assurance that the vendors were
desperate
enough for government sales that they shelled out a large amount of
money in
order to get a ticket to ride the gravy train.

The problem is that like "religion" or "morals", "assurance" is a loaded
term
and can be interpreted to mean almost anything.  Without some basic
definition
of what's meant by "assurance", it's not really possible to reason about
this.
So here's an attempt to nail it down: For me (and I would guess most end
users) assurance is being able to sleep at night knowing there's little
chance
of anyone compromising my system ("things that should happen do happen,
things
that shouldn't happen don't happen").

Let's see what FIPS 140 gives you to help you sleep at night.  It tests
some
crypto mechanisms, but ignores others.  If you're buying an IPsec
product it
doesn't test all the mechanisms needed for IPsec.  If you're buying a
TLS
product it doesn't test the mechanisms needed for TLS.  If you're
buying...
well, you get the idea.

Compare this to the example I gave earlier of performing a TLS exchange
with
Amazon.  This performs an in-depth test of all the crypto algorithms
(corresponding to the FIPS algorithm tests, including ones that FIPS
ignores),
and the crypto mechanisms (many/most of which FIPS again ignores).  In
other
words simply by connecting to Amazon using TLS and ordering a "Scrubs"
DVD for
$19.95 I'm getting more comprehensive algorithm testing than I can for a
five-
figure sum with the FIPS algorithm tests.

So what else do you get?  You get a bit of a code review and some
checking
that you don't do stupid things with keys.  Only some of the code is
reviewed
(for example that trivial buffer overflow in the client handshake that
allows
a remote attacker to do whatever they want with your server, including
the
FIPS 140-evaluated portion, isn't checked because it's outside the scope
of
what FIPS 140 is interested in), and (from having gone through several
of
these) the review itself seems to be more coding style nitpicks than
looking
for potential exploits, and in any case you can often document away any
problems without having to fix them.

So the assurance a FIPS 140 eval gets you is a tautological "the product
vendor was able to pass a FIPS 140 eval".  In terms of being able to
demonstrate this type of assurance, I agree completely with you that
there is
no better mechanism for it than FIPS 140.

Admittedly there is some value to FIPS 140: You know that a subset of
the
crypto used is OK, that the code doesn't do anything stupid with its
keys
(unless the vendor has documented away the stupid things that do happen,
e.g.
CryptoAPI's plaintext private-key export, you ask it "Give me the user's
private key in plaintext form" and it says "Sure, here you go"), and
that it's
had a bit of a code check (and obviously if you're doing hardware crypto
you
get a lot more value than this, but the discussion here is about
software
modules).

What you don't get from FIPS 140 is the stuff that customers really care
about: Assurance that your IPsec product can actually do IPsec, that
your TLS
product can do TLS, or that the product really is secure(-ish).
Excusing this
by saying that a FIPS 140 crypto product eval doesn't have to check half
your
crypto so this isn't a problem is like saying that your TSA passenger-
screening process is OK even though it doesn't bother checking men with
beards
or anyone under 5'6" tall.

If I'm going to spend $100K to feel safe then I can really think of
better
ways than FIPS 140 (or CC, or ITSEC, or ...).  How about the following
comparison: You get to spend the cost of a FIPS 140 eval in one of two
ways:

$100K: FIPS 140

-- or --

$20K: Fortify (or whatever) scan.
$20K: Cigital (or whatever) audit/analysis (these are just
representative
      figures/estimates since there don't seem to be any fixed rates for
this
      sort of thing).
$5K: Grant to French university to pen-test code "written at a German
     university".
$5K: Grant to British university to pen-test code "written at a French
     university".
$5K: Set up a fake banking site running the code and bring it to the
attention
     of the Russian Business Network.
$free: Leak "security code for Windows 7" to Slashdot.
$free: Leak "security code for BluRay++" on warez boards.
$free: Leak "Diebold voting machine security code" to Cryptome.
$45K: 50" plasma TV, home theatre system, and beer to kill time while
you
      wait for the results to come in (and to emphasise how much cheaper
      this is than FIPS 140 :-).

If your life depended on the security of the product that went through
those
two processes, which one would you trust?  FIPS 140 is just not
cost-effective
in providing assurance - it costs too much, it takes too long, and it
doesn't
evaluate the stuff that real users care about having evaluated.

There was a long and extremely interesting discussion on the (somewhat
misnamed) open-source list recently, thread topic "How can high
assurance
FLOSS development be encouraged?".  The archives are at
http://lists.csl.sri.com/mailman/listinfo/open-source, but you have to
sign up
to the list in order to access them.

Peter.

---

From: [email-masked] (Peter Gutmann)
Date: Tue, 26 Feb 2008 19:33:57 +1300

"Santosh Chokhani" <[email-masked]> writes:

>You are wrong about FIPS 140-1 costs being 100K for Level 1.  It is more like
>30K.

The figures I've been given, from numerous vendors going through numerous labs
over a number of years, is that their all-up cost for a level 1 software eval
was around $100K (give or take a few tens of $K).  This isn't just the final
cheque they cut to get the coloured piece of paper, this is the all-up cost of
getting their product through a FIPS 140 eval.

I realise the following may be a bit unfair since you weren't intending to
provide a price quote :-), but I'm willing to put my money where my mouth is:
If Cygnacom can get me a FIPS 140 level 1 on my code for an all-up cost of
$30K I'll send you a cheque and CDROM of the source within 24 hours (I need to
get mgt.approval first).  Just let me know where to send it and who to make
the payment out to.

>implemented correctly,

That a *subset* of the algorithms used are impemented correctly, in other
words a subset of what you can get for $19.95 via a TLS connect to Amazon.
And the actual crypto mechanisms don't get tested at all.

>keys will be generated in accordance with FIPS (meaning that the seed feeding
>the PRNG will have requisite entropy and PRNG will be FIPS approved).

A nice circular definition: "A FIPS evaluation guarantees that keys will be
generated as required in order to pass a FIPS evaluation".

>You also get the assurance that the keys are being managed properly in the
>crypto module.

... unless the vendor has documented away the mismanagement, e.g. CryptoAPIs
plaintext private key export.

You're not making a very convincing argument here :-).

Peter.

---

From: "Santosh Chokhani" <[email-masked]>
Date: Tue, 26 Feb 2008 07:39:40 -0500

Peter,

I do not think this is a forum for negotiations.  But, we will be happy
to do FIPS testing for your product for Level 1 for quoted price.

As to algorithms, all FIPS approved algorithms need to be tested.

As to key generation there are standards that come out of NIST and ANSI
X9 that IETF also takes its cue from, and FIPS process ensures that the
keys are generated in accordance with those standards.

Have you yourself participated in a FIPS evaluation or have you looked
at the NIST FIPS 140-2 DTR and FIPS 140-2 IG (i.e. Implementation
Guidance) available on the Web?

-----Original Message-----
From: pgut001 [mailto:[email-masked]] 
Sent: Tuesday, February 26, 2008 1:34 AM
To: [email-masked]; [email-masked]; Santosh Chokhani
Cc: [email-masked]
Subject: RE: [saag] Algorithms/modes requested by users/customers

>You are wrong about FIPS 140-1 costs being 100K for Level 1.  It is
more like

The figures I've been given, from numerous vendors going through
numerous labs
over a number of years, is that their all-up cost for a level 1 software
eval
was around $100K (give or take a few tens of $K).  This isn't just the
final
cheque they cut to get the coloured piece of paper, this is the all-up
cost of
getting their product through a FIPS 140 eval.

I realise the following may be a bit unfair since you weren't intending
to
provide a price quote :-), but I'm willing to put my money where my
mouth is:
If Cygnacom can get me a FIPS 140 level 1 on my code for an all-up cost
of
$30K I'll send you a cheque and CDROM of the source within 24 hours (I
need to
get mgt.approval first).  Just let me know where to send it and who to
make
the payment out to.

>In terms of what FIPS buys is that you ensure that the algorithm is

That a *subset* of the algorithms used are impemented correctly, in
other
words a subset of what you can get for $19.95 via a TLS connect to
Amazon.
And the actual crypto mechanisms don't get tested at all.

>the PRNG will have requisite entropy and PRNG will be FIPS approved).

A nice circular definition: "A FIPS evaluation guarantees that keys will
be
generated as required in order to pass a FIPS evaluation".

the
>crypto module.

... unless the vendor has documented away the mismanagement, e.g.
CryptoAPIs
plaintext private key export.

Peter.

---

From: Stephen Kent <[email-masked]>
Date: Tue, 26 Feb 2008 21:09:51 -0500

At 4:44 AM +1300 2/26/08, Peter Gutmann wrote:
>...

BBN developed the first crypto module evaluated at level 3 under FIPS 
140-1. The fee we paid to the eval lab was about $40K, and the 
elapsed time was about 6 months.  That was more than  ten years ago, 
but we're also talking about a much more thorough eval. The cost to a 
developer is, of course, higher, when you add in the time required to 
prepare the necessary documentation. Also, if the developer is not 
experienced in secure development practices, the likelihood increases 
that the eval will not go smoothly, and thus the costs will be 
higher.  Maybe what you're reporting is the result of more 
developers, with less experience  secure crypto module development 
experience submitting products that are "not quite ready."

...

>order to get a ticket to ride the gravy train.

Sotware modules are easier for a developer to create and submit, and 
this too may cause more submissions that are less well thought out. 
If you're building hardware there may be more of an effort to "get it 
right" in the planning stages, because the costs of redesign are 
potentially higher.

>that shouldn't happen don't happen").

Assurance (in the security context) is not as squishy a term as your 
suggest. Security assurance is generally characterized as confidence 
that an implementation meet certain objective security-relevant 
criteria. These are not interoperability criteria. It is 
distinguished from security functionality, which refers to the set of 
functions (features) ascribed to a product (without confidence that 
these features operate as promised).

>product it doesn't test the mechanisms needed for TLS.  If you're buying...

For a protocol such as TLS or IPsec,  crypto module security 
assurance does not encompass most aspects of what a users sees as 
"correct" operation. It makes sense to look for both functionality 
testing and security assurance evaluation, if you care about both 
functionality and security.

>figure sum with the FIPS algorithm tests.

nonsense! what you get is confidence that the SSL/TLS implementation 
at Amazon and in your browser work compatibly, even if both happen to 
be "wrong."  In fact, the test you propose checks only a small subset 
of the SSL/TLS spec, as it verifies only that the RSA key transport 
method of key management works, none of the other key management 
methods defined by the RFCs. So, maybe what you meant to say was that 
the proposed test regimen verifies that a protocol and associated 
crypto work compatibility with an implementation of the most common 
subset of the crypto modes and protocol features deployed.  That's a 
useful finding, but let's not confuse it with protocol standard 
conformance or security assurance.

I won't bother to respond to the rest of your observations, since 
they repeat many of the themes I touched upon above.

It is silly to ask FIPS 140 to address protocol interop issues; that 
is simply not in scope for a generic crypto security eval criteria. 
So let's forgo all the arguments along those lines.

Vendor claims about security sometimes (often?) prove to be wrong. 
Thus is makes sense to seek external, independent assurances about 
security, if you care. The fact that FIPS 140 addresses only a subset 
of those claims is also not a basis for criticizing a generic crypto 
module evaluation criteria like FIPS 140. Common Criteria eval of 
products is more encompassing, but also MUCH more expensive.

So, IF one cares about crypto security, THEN FIPS 140 eval is the 
best option for now. That does not make it ideal, but it also does 
not make it as bad as some comments have suggested, and it is unfair 
to criticize it for not doing something that it has never promised to 
do.

Steve