What's considered?

Products are rated in six categories:

• Features,
• Ease of use,
• Performance,
• Documentation,
• Support, and
• Value for money.

Each product is then given an Overall rating, which is determined from a combination of these categories.

Ratings are marked from one to five stars, which is referred to as the star rating:

* * * * * Outstanding in all respects - an "A" on the product's report card

* * * * Exceeds basic expectations and requirements - a "B" on the product's report card

* * * Meets the expectations of the review - a "C" on the product's report card

* * Failed to achieve some basic requirements - a "D" on the product's report card

* Seriously deficient - an "F" on the product's report card.

Testing criteria

We test against two sets of criteria. The first is a set of generalized points that are organized according to the six categories.

These are the criteria appearing in our generic (baseline) test matrix database. Under each category there is a list of generic criteria that the tester checks "yes" or "no" (in most cases), indicating the presence or absence of the criterion in the product under test.

For example, the first criterion under easeof use is "management interface intuitiveness." The tester would check "yes" or "no" to indicate whether or not the management interface displayed appropriately intuitive characteristics. Additionally, the tester has space in the database to include comments about the specific criteria.

Finally, each product group has individual tests designed specifically for the group, and each group has a unique test bed configuration designed to facilitate testing of the product types within that group.

The criteria for each major category are listed below as they appear in our generic test matrix.

Ease of use

• Management interface intuitiveness
• Task efficient
• Learning curve
• Integratable with other products
• Easy to work with
• Ease of fixing problems
• Defined set up procedure
• Intuitive installation of
• Errors encountered
• User friendly
• Automation/scripting
• Installation verification
• Stable installation
• Bundled hardware and software
• Installation log file
• Intuitive user interface
• User tutorials or wizards
• Predefined reports
• Graphical reports

Features

• Expected features?
• Missing features?
• Standout features?
• Features of little use?
• Function of advertised features?
• Integrated database
• Scripting language

Performance
(Criteria in this category are graded as "strong" or "weak")

• Typical condition performance
• Performance under attack
• Command response
• Event information

Documentation

• Understandable documentation
• Effective documentation
• Online manuals
• Installation documentation
• User oriented manuals
• Supplemental documentation

Support

• Telephone support?
• Web-based downloads?
• Online forum?

Value for money

• Purchase for your own company?
• Appropriately priced?
• Extra fees
• Updates included

The Overall rating

This page of our database automatically calculates the average rating from each of the scores awarded by the tester in the individual categories.

The reviewer also has the opportunity to change individual scores if he/she thinks it's appropriate. This causes a new calculation for reviewer rating and both the reviewer rating and the tester ratings now appear here in the database for future reference.

The tester provides a paragraph or more on key strengths and a paragraph or more on key weaknesses as observed during the testing the sleep. The reviewer then has the opportunity to write a paragraph or more with additional comments.

Back to Products Review Labs