Check that:
The protocol is stable over time.
A test often stresses the application for many hours. Tests based on the protocol must continue to run correctly without crashing the test, the load generator, or the controller.
Memory consumption does not increase over time for unknown reasons.
Memory usage is expected to increase in proportion to the number of Vusers running and may also increase for known reasons in your protocol, such as caching. With these exceptions, consumption should not increase.
Scalability is acceptable.
How many Vusers can be run on a load generator and controller with the expected configurations?
The protocol run is thread-safe if configured to allow running a virtual user as a thread.
Check that:
All the API functions and each function's arguments behave as designed.
If the run-time settings are changed dynamically during the run:
the new value is used by the Vuser in which they were changed
Scripts that emulate typical business processes run correctly.
Supported I18N functionality is performed correctly. See I18N (Internationalization)
Multi-protocol scripts with the SDK-based protocol run correctly. Test all combinations of protocols that you expect to be used with your protocol.
Script logging levels are implemented correctly. For example, when "Data returned by server" is selected, the protocol logs the server response to the replay log in a readable way.
Error and warning messages are logged appropriately.
The functionality tests pass on all supported operating systems.
Check that:
The virtual user creates an accurate simulation of the real client, running the same scenarios and sending the same messages, so that the transportation on the network is the same.
The transaction time is similar to the real client transaction time. See Accuracy.
Non-negligible process time on the client side during test run is reported as wasted time in the transaction.
Check that:
Transaction times are recorded. See Accuracy
Data points are recorded.