Below are just some personal notes on the topic: Software random testing techniques. Information is not complete, verified and systematised. Some references will be provided in future.
Software random testing techniques. I will try to give an overview and describe all new/well known software testing techniques with accents to topics you need to know about random software testing for interview. I will attempt to introduce guidelines for choosing correct techniques for the project. All definitions will be taken from accepted and identified sources.
Testing Techniques: Classification of random testing techniques with descriptions of what types of testing needs to be done. This classification will include descriptions of random testing techniques.
Specific random testing techniques are applicable for any single project. They will change from project to project. Usually required for the current project testing techniques are described in the test plan.
Remember that one of the main selection criterions of a test design technique that will be used for the current application is coverage result for the system under test. Different testing techniques can find different types of defects.
Example: You need to test new functionality of your application. For testing you randomly generate data for all existing and new fields in the application under test.
Example: You have a GUI interface with multiple tabs for entering information and only one save button. The system can work correctly if the order of entering of information is tab 1, tab 2 and tab 3, but may be will give error if the sequence is tab 2, tab1, tab3.
Example: You need to test new functionality of your application. For testing you randomly by member ID select members from existing system database and verify that new functionality working as expected.
This page presents a methodology and different techniques for random testing of software applications, and algorithm types that can be used for random testing tools very effectively. Measuring the efficiency and effectiveness of the applied random technique can be done my measurement of testing coverage from one side and failure detection effectiveness from other side.
Types of algorithms for random testing:
Random testing inputs can give surprisingly good results by using correct algorithm and if the distribution of inputs is spread out evenly over the input domain or by taking into account risk factors during distribution of inputs. Random testing is not only a useful testing technique in itself, but also can be used in many other testing techniques
Some authors are trying to prove that testing using randomised quasi-random sequences is often significantly more effective than random testing.
Adaptive Random Testing is a Black Box testing method based on the intuition that Random Testing failure-finding efficiency can be improved upon, in certain situations, by ensuring a more widespread and evenly distributed spread of test cases in the input domain.
Adaptive Random Testing (ART) denotes a family of testing algorithms that have a better performance compared to pure random testing with respect to the number of test cases necessary to detect the first failure.
When you are choosing the tool for generating random numbers, you need to decide which type of data you need (format), what range you need (minimum to maximum), and how they must be distributed. The distribution near boundaries may be the best way. There are two common types of distributions: uniform and normal. In a uniform distribution all numbers in the range are equally likely to be produced.
In some random generators a particular random number may be generated repeatedly for every new ran. If you don't want the same numbers to be repeated, put the values in the log and use logic - if x was previously chosen, do not use more. Remember to clear this log file periodically.
We need to reproduce errors in order to fix them. For this reason, log file must exist to store the seed value before using it. This will help to reproduce the error in the case if a computer or application fails dramatically. Code coverage tools can be used to verify quality of your random testing. Every random test generator can be expected to find a different set of bugs. Before using random test generator for your testing open log file and verify that data generated is suitable for your needs.
An experienced tester would consider write some tests cases that cover the following scenarios:Random testing became more and more popular and already a must for gaming and protocol testing. The power of the random testing comes from running large number of test cases with high failure detection effectiveness. Certainly we won't get perfect or optimal results with random testing, but we can get pretty good results with low cost. In some circumstances, random testing methods are more practical than any alternative. Coverage measures allowed us to understand the problems of selected algorithm for random testing, thus providing opportunities for future improvement.
On this site you can find information about software testing and automated software testing tools, recommendations for software testing training course, list of books about software testing with recommendations and books suggested for software testing certification, online software testing quiz, interview questions for software testers, software testing dictionary,samples of software testing documentation including test case template, jokes about software testing and testers, software testing techniques and more ...
random testing definition from @dictionary.com A black-box testing approach in which software is tested by choosing an arbitrary subset of all possible input values. Random testing helps to avoid the problem of only testing what you know will work.
Random-input testing. The processes of testing a program by randomly selecting a subset of all possible input values.[Glenford J. Myers, 2004]
Monkey Testing. Testers use the term monkey when referring to a fully automated testing tool. This tool doesn’t know how to use any application, so it performs mouse clicks on the screen or keystrokes on the keyboard randomly. The test monkey is technically known to conduct stochastic testing, which is in the category of black-box testing. There are different types of monkey testing.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]
Monkey Testing. (smart monkey testing) Input are generated from probability distributions that reflect actual expected usage statistics -- e.g., from user profiles. There are different levels of IQ in smart monkey testing. In the simplest, each input is considered independent of the other inputs. That is, a given test requires an input vector with five components. In low IQ testing, these would be generated independently. In high IQ monkey testing, the correlation (e.g., the covariance) between these input distribution is taken into account. In all branches of smart monkey testing, the input is considered as a single event.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]
Monkey Testing. (brilliant monkey testing) The inputs are created from a stochastic regular expression or stochastic finite-state machine model of user behavior. That is, not only are the values determined by probability distributions, but the sequence of values and the sequence of states in which the input provider goes is driven by specified probabilities.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]
Monkey Testing. (dumb-monkey testing) Inputs are generated from a uniform probability distribution without regard to the actual usage statistics.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ] Error Guessing: Another common approach to black-box validation. Black-box testing is when everything else other than the source code may be used for testing. This is the most common approach to testing. Error guessing is when random inputs or conditions are used for testing. Random in this case includes a value either produced by a computerized random number generator, or an ad hoc value or test conditions provided by engineer.
Shotgunning testing technique - testing technique that distributed test scripts randomly during testing in each test cycle.