Comments/suggestions on draft

["Wang,Nai-Pin K." <knwang@mitre.org>]

George,

First off, this is already a very good document, and I can appreciate
how much time and effort you already had to put into getting to this
stage.

I have reviewed up to and including section 2 of the draft and have some
initial comments/suggestions:

Abstract - 
You list some examples for devices that implement IP, like cable modems,
personal firewalls, hosts. I think the term "hosts" needs to be
clarified. Many
people will think of hosts as including desktop workstations. Desktops
should
not be held to the same standards as critical server-class machines, for
example.
Are we really just focusing on critical server-class machines here? If
so, we should probably replace "hosts" with "critical core servers".

Requirement 2.1.1, second paragraph in "Justification" - 
Replace "It applies in situations..." with "This requirement applies in
situations..." The "it" can be ambiguous.

Requirement 2.1.3 - 
1) In "Requirement", remove the extraneous "of" in the sentence.
2) Add to the justification "If the device is compromised, filters
may be modified by the attacker.

Requirement 2.1.4 - 
IMHO, the example given is not currently a best practice, which is what
section 2 is focused on. Unless I misunderstood the example (in which
case, it needs to be clarified), this requirement might fit better in
section 3 as a "non-standard" requirement.

Requirement 2.1.6 - 
This requirement specifies storing and retrieving system configs from a
remote server. However, there is no requirement for implementing
mechanisms to ensure integrity of the configuration files. I think there
needs to be a requirement along the following lines: Requirement: Device
MUST provide configuration file integrity ensuring mechanisms.
Justification: An attacker may modify configuration files upon
compromise of device. Examples: Integrity could be ensured through means
of calculating checksums of the configuration files. Blah, blah, blah.
Something like that.

Requirement 2.2.2 -
I agree with a lot of this requirement. However, there are some examples
given (all IP addresses and blocks, system names, domain names, banners)
that I wonder if we're overdoing this requirement. If I were wearing my
auditor hat, I would need
to know what your IP addresses are, along with your warning banners, for
example. If I were a system administrator, what's the point of me
looking at the configurations if I can't see information like IP
addresses and system names? I can see obscuring information like
passwords and shared secrets, though. Maybe I misunderstood the
requirement, but it just seemed overly done in the examples listed.

Requirement 2.3.1 -
Do we want to include ARP and RARP in the examples listing as well, or
is that implied? I just thought these were major enough protocols to get
mentioned.

Requirement 2.3.2, 2.3.3 -
Do these requirements include proprietery protocols? If so, how much
information does the vendor have to supply? As for the "Warning" field
in requirement 2.3.3...
from my experience dealing with vendors, this is going to be interpreted
by the vendors as having permission NOT to disclose information about
the protocols. EVERY vendor will opt for the case-by-case appeal. I
think we should either remove the warning statement, or remove the
requirement entirely. I don't know who would be dealing with the
case-by-case judgements, but I wouldn't want to be them.

Requirement 2.3.4 - 
What about disclosing which services get disabled upon installation of
the product? In addition, some firewall applications automatically make
modifications to the kernel. Should this be listed as well? Some devices
use stripped down versions of popular OSes as a foundation. Should the
vendor document how the OS was modified?

Requirement 2.3.8 -
We should remove the apology to computer scientists in the Warnings
field. This document is supposed to unify people (I know, I'm naive),
not insult and drive some away.

Requirement 2.3.9 - 
This requirement seems academic. I don't really know how to explain why,
but it just seems like something that's already understood, and doesn't
seem to fit with the rest of the requirements, which seems more applied.

Requirement 2.3.14 - 
Should the vendor also specify any changes that may have been made to
the OS? I'm thinking specifically of the stripped down Solaris 8 version
on the Cisco IDS (Netranger), which was missing enough critical
libraries that source compilation was not working. It would have been
nice to know in advance that core library functions were removed.

Requirement 2.6.5  -
Maybe I'm missing something, or are vendors really horrific enough to
call a device
"Layer 2" and not have it be able to filter based on MAC addresses? If
vendors are not this horrific, then do we need this requirement? If they
are this horrific, wow.


Here are some additional requirements I thought we might want to
include. This is just food for thought at this point. We can debate on
whether these should be requirements or not. While others have stated
that your draft is route-centric, they may find my requirements to be
NIDS and firewall-centric. But, that's what I have the most experience
with:

Architecture-related requirements:

1) Vendor appliances should be rack-mountable and not have unusually
restrictive power or cooling requirements.

2) Device should be able to operate with a multiprocessor architecture
with multi-threaded code

3) The product architecture should provide the flexibility to allow
either centralized or distributed management and control of devices.

4) Product components using public-key cryptography should securely
store their private keys.

5) Product components straddling a firewall must communicate using
protocols that can be securely proxied through the firewall.

6) Where possible, the system architecture should provide the option for
component redundancy to prevent single points of failure.

7) The device should provide a watchdog capability to automatically
re-start or reboot a “stuck” component.

8) The product should provide the ability to store the raw data stream
(e.g., network traffic, system logs, etc.) portion triggering an alarm
for later forensic analysis.

9) The vendor should implement mechanisms to support device
load-balancing capabilities.

10) Filters should be easy to apply. This means the administrator can
specify IP address blocks, port ranges, etc in one filter rule.

11) Device should support user account lockout upon x unsuccessful login
attempts.

Installation and configuration-related:

1) System installation documentation should include steps to verify
proper operation upon completion of installation.

System management-related:

1) The vendor should implement mechanisms for group device management
that improve management scalability.

2) The device should provide a robust command line interface (CLI) as
backup to the management interface.

3) The vendor should provide a straightforward mechanism for obtaining
system software upgrades.

4) The vendor should provide an automated mechanism for notifying
customers when software upgrades and/or signature updates are available.

5) The system software upgrade process should be simple and involve a
minimal amount of system down time.

6) The device should contain the instrumentation and health monitoring
to alert the administrator under the following scenarios:  1) Loss of
communications, 2) System overload resulting in packet loss, 3) Little
or no observed network traffic,4) software failures.

7) The device should have the capability to synchronize with a Stratum-1
NTP server.

8) The vendor must implement the capability to view device logs in
real-time for administrative purposes (remotely and/or through console
access)

9) If vendor allows for granularity of logging facilities (e.g.,
informational, critical, etc), the documentation must be provided
specifying log message types that are included in each granularity
category.

Notification-related:

1) The device should be capable of providing remote notification of
events via e-mail, page, fax, or SNMP traps.

Vendor support-related:

1) The product should be stable (i.e., relatively low number of bug-fix
releases).

2) The vendor should be willing to provide source code escrow

3) The vendor should be able to provide long-term maintenance agreements
with fixed pricing


OK, that's it for today.

Kathy