Random Software Testing Techniques


This page is under construction.
Coming Soon.
Random Software testing techniques.


under construction



we are moving from geocities.com/xtremetesting

Below are just some personal notes on the topic: Software random testing techniques. Information is not complete, verified and systematised. Some references will be provided in future.


Software Random Testing Techniques

Software random testing techniques. I will try to give an overview and describe all new/well known software testing techniques with accents to topics you need to know about random software testing for interview. I will attempt to introduce guidelines for choosing correct techniques for the project. All definitions will be taken from accepted and identified sources.


Random testing was mentioned first time in the book of Glenford J. Myers in 1979

Testing Techniques: Classification of random testing techniques with descriptions of what types of testing needs to be done. This classification will include descriptions of random testing techniques.

Specific random testing techniques are applicable for any single project. They will change from project to project. Usually required for the current project testing techniques are described in the test plan.


Below are types of random software testing techniques that can be applied in software testing.
Questions to be answered regarding software testing techniques Scope: Which techniques are more valuable and not?
  • Which are easy to apply, which are not?
  • Which find genuine faults, which are not?
  • Manual (knowledge based tests) vs automatically generated tests?
  • Which are effective?
  • Knowledge acquisition; which are easy to comprehend, which are not.

    Remember that one of the main selection criterions of a test design technique that will be used for the current application is coverage result for the system under test. Different testing techniques can find different types of defects.



    Main types of random testing techniques:
    1. random input data generation

    Example: You need to test new functionality of your application. For testing you randomly generate data for all existing and new fields in the application under test.


    2. random sequence of data input (sometimes called as stochastic testing)

    Example: You have a GUI interface with multiple tabs for entering information and only one save button. The system can work correctly if the order of entering of information is tab 1, tab 2 and tab 3, but may be will give error if the sequence is tab 2, tab1, tab3.


    3. random data selection from existing database

    Example: You need to test new functionality of your application. For testing you randomly by member ID select members from existing system database and verify that new functionality working as expected.


    It is possible to combine all the above testing techniques.
    ************************************************************NOTES

    This page presents a methodology and different techniques for random testing of software applications, and algorithm types that can be used for random testing tools very effectively. Measuring the efficiency and effectiveness of the applied random technique can be done my measurement of testing coverage from one side and failure detection effectiveness from other side.

  • Measurement of the coverage that applies by different random techniques will provide opportunities for future improvement.
  • In some circumstances, random testing methods are more practical than any types of systematic structural types of testing.

    Types of algorithms for random testing:

    1. uniform random walks (where every trace has the same probability to occur),
    2. randomised quasi-random sequences,
    3. a path-oriented random test data generator,
    4. sequence of random test data that execute only a subset of paths in a program.

    Random testing inputs can give surprisingly good results by using correct algorithm and if the distribution of inputs is spread out evenly over the input domain or by taking into account risk factors during distribution of inputs. Random testing is not only a useful testing technique in itself, but also can be used in many other testing techniques

  • In general my experience shows that GUI-based applications to be less reliable and random testing has high failure detection effectiveness during GUI testing.
  • Adaptive Random Testing is a Black Box testing technique based on the assumption that Random Testing failure-finding efficiency can be improved by evenly distributing test cases in the input domain.
  • The computer can only generate pseudorandom numbers--numbers that are generated by a formula.
  • With random number generators (RNGs), there are no guarantees. We can only predict what will happen in practice.

    Some authors are trying to prove that testing using randomised quasi-random sequences is often significantly more effective than random testing.

    Adaptive Random Testing is a Black Box testing method based on the intuition that Random Testing failure-finding efficiency can be improved upon, in certain situations, by ensuring a more widespread and evenly distributed spread of test cases in the input domain.

    Adaptive Random Testing (ART) denotes a family of testing algorithms that have a better performance compared to pure random testing with respect to the number of test cases necessary to detect the first failure.



    Some tips for using random test generation tools:

    When you are choosing the tool for generating random numbers, you need to decide which type of data you need (format), what range you need (minimum to maximum), and how they must be distributed. The distribution near boundaries may be the best way. There are two common types of distributions: uniform and normal. In a uniform distribution all numbers in the range are equally likely to be produced.

    In some random generators a particular random number may be generated repeatedly for every new ran. If you don't want the same numbers to be repeated, put the values in the log and use logic - if x was previously chosen, do not use more. Remember to clear this log file periodically.

    We need to reproduce errors in order to fix them. For this reason, log file must exist to store the seed value before using it. This will help to reproduce the error in the case if a computer or application fails dramatically. Code coverage tools can be used to verify quality of your random testing. Every random test generator can be expected to find a different set of bugs. Before using random test generator for your testing open log file and verify that data generated is suitable for your needs.


    **************************************************************END NOTES
    ******************** Conclusion

    Random testing became more and more popular and already a must for gaming and protocol testing. The power of the random testing comes from running large number of test cases with high failure detection effectiveness. Certainly we won't get perfect or optimal results with random testing, but we can get pretty good results with low cost. In some circumstances, random testing methods are more practical than any alternative. Coverage measures allowed us to understand the problems of selected algorithm for random testing, thus providing opportunities for future improvement.



    On this site you can find information about software testing and automated software testing tools, recommendations for software testing training course, list of books about software testing with recommendations and books suggested for software testing certification, online software testing quiz, interview questions for software testers, software testing dictionary,samples of software testing documentation including test case template, jokes about software testing and testers, software testing techniques and more ...



    Dictionary:

    random testing definition from @dictionary.com A black-box testing approach in which software is tested by choosing an arbitrary subset of all possible input values. Random testing helps to avoid the problem of only testing what you know will work.

    Random-input testing. The processes of testing a program by randomly selecting a subset of all possible input values.[Glenford J. Myers, 2004]

    Monkey Testing. Testers use the term monkey when referring to a fully automated testing tool. This tool doesn’t know how to use any application, so it performs mouse clicks on the screen or keystrokes on the keyboard randomly. The test monkey is technically known to conduct stochastic testing, which is in the category of black-box testing. There are different types of monkey testing.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]

    Monkey Testing. (smart monkey testing) Input are generated from probability distributions that reflect actual expected usage statistics -- e.g., from user profiles. There are different levels of IQ in smart monkey testing. In the simplest, each input is considered independent of the other inputs. That is, a given test requires an input vector with five components. In low IQ testing, these would be generated independently. In high IQ monkey testing, the correlation (e.g., the covariance) between these input distribution is taken into account. In all branches of smart monkey testing, the input is considered as a single event.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]

    Monkey Testing. (brilliant monkey testing) The inputs are created from a stochastic regular expression or stochastic finite-state machine model of user behavior. That is, not only are the values determined by probability distributions, but the sequence of values and the sequence of states in which the input provider goes is driven by specified probabilities.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ]

    Monkey Testing. (dumb-monkey testing) Inputs are generated from a uniform probability distribution without regard to the actual usage statistics.[Visual Test 6 Bible by Thomas R. Arnold, 1998 ] Error Guessing: Another common approach to black-box validation. Black-box testing is when everything else other than the source code may be used for testing. This is the most common approach to testing. Error guessing is when random inputs or conditions are used for testing. Random in this case includes a value either produced by a computerized random number generator, or an ad hoc value or test conditions provided by engineer.

    Shotgunning testing technique - testing technique that distributed test scripts randomly during testing in each test cycle.



    Bibliography:
    1. Visual Test 6 Bible by Thomas R. Arnold, 1998
    2. http://en.wikipedia.org/wiki/Fuzz_testing
    3. Fuzz Testing of Application Reliability
    4. free software for random number generation and testing recommendations.
    5. Materials from the first international workshop on random testing -2006



    QA interview questions    Main Page

    Copyright © 2009 eXtremeSoftwareTesting.com  All Rights Reserved.