Test Hypothesis Using t-Test

 

Updated: March 2, 2017

Compares means from two columns using a t-test

Category: Statistical Functions

You can use the Test Hypothesis Using t-Test module to generate scores for three types of t-tests:

  • Single sample t-test
  • Paired t-test
  • Unpaired t-test

In general, a t-test helps you compare whether two groups have different means. For example, suppose you are evaluating trial data for patients who received Drug A vs. patients who received Drug B, and you need to compare a recovery rate metric for both groups. The null hypothesis would assume that the recovery rate is the same in both groups, and furthermore, that the values for the recovery rate have a normal distribution in both two groups.

By using Test Hypothesis Using t-Test and providing the columns that contain the recovery rates as input, you can get scores that indicate whether the difference is meaningful, which would signify that the null hypothesis should be rejected. The test takes into account factors such as how big the difference is between the values, the size of the sample (larger is better), and how big the standard deviation is (lower is better).
By reviewing the results of the Test Hypothesis Using t-Test module, you can determine whether the null hypothesis is TRUE or FALSE, and review the confidence (P) scores from the t-test.

How to Choose a T-Test

Choose a single sample t-test when these conditions apply:

  • You have a single sample of scores.

  • All scores are independent from each other.

  • The sampling distribution of xˉ is normal.

In general, the single sample t-test is used to compare an average value to a known number.

Choose a paired t-test when these conditions apply:

  • You have a matched pairs of scores. For example, you might have two different measures per person, or matched pairs of individuals (such as a husband and wife).

  • Each pair of scores is independent of every other pair.

  • The sampling distribution of dˉ is normal.

A paired t-test is useful when comparing related cases. By averaging the differences between the scores of the paired cases, you can determine whether the total difference is statistically significant.

Choose an unpaired t-test when these conditions apply:

  • You have two independent samples of scores. That is, there is no basis for pairing scores in sample 1 with those in sample 2.

  • All scores within a sample are independent of all other scores within that sample.

  • The sampling distribution of x1- x2 is normal.

  • Optionally, satisfy the requirement that the variance among the groups be roughly equal.

The columns that you are comparing must be in the same dataset. If you need to compare columns from different datasets, you can isolate each column to compare by using Select Columns in Dataset, and then merge them into one dataset by using Add Columns.

  1. Add the Test Hypothesis Using t-Test module to your experiment. You can find this module in the Statistical Functions group in the experiment items list in Azure Machine Learning Studio.

  2. Add the dataset that contains the column or columns that you want to analyze.

  3. Decide which kind of t-test is appropriate for your data. See How to Choose a T-Test.

  4. Additional parameters might be required, depending on the type of data you have and the type of t-test you chose.

    If you select the Single sample option -

    1. For Null hypothesized µ, type the value to use as the null-hypothesized mean for the sample. This specifies the expected mean value against which the sample mean will be tested.
    2. For Target column, use the Column Selector to choose a single numeric column for testing.
    3. Set Hypothesis type to choose a one-tail or two-tail test. The default is a two-tailed test. This is the most common type of test, in which the expected distribution is symmetric around zero. The One Tail GT option is for a one-tailed greater than test. This test gives more power to detect an effect in one direction, by not testing the effect in the other direction. The One Tail LT option gives a one-tailed less than test.
    4. For α, specify a confidence factor. If the value of P (the first output of the module) is lower than the confidence factor, the null hypothesis is rejected.

    If you select the PairedSamples option -

    1. For Null hypothesized µ, type a value that represents the sample difference between the pair of samples.
    2. For Target column, use the Column Selector to choose numeric columns for testing.
    3. Set Hypothesis type to choose a one-tail or two-tail test. The default is a two-tailed test.
    4. For α, specify a confidence factor. If the value of P (the first output of the module) is lower than the confidence factor, the null hypothesis is rejected.

    If you select the UnpairedSamples option -

    1. Deselect the option, Assume equal variance, if the samples are from different populations.
    2. For Null hypothesized µ1, type a value that represents the mean for the first column.
    3. For Null hypothesized µ2, type a value that represents the mean for the second column.
    4. For Target columns, use the Column Selector to choose the numeric columns for testing.
    5. Set Hypothesis type to choose a one-tail or two-tail test. The default is a two-tailed test. T
    6. For α, specify a confidence factor. If the value of P (the first output of the module) is lower than the confidence factor, the null hypothesis is rejected.
  5. Run the experiment.

Results

The output of the module is a dataset containing the t-test scores, and a transformation that you can optionally save to re-apply to this or another dataset using Apply Transformation.

No matter which type of t-test you choose, the module outputs two scores:

  • A probability score that indicates the confidence of the null hypothesis
  • A value that indicates whether the Null hypothesis should be rejected

The goal is to determine if we can reject the null hypothesis. Therefore, a score of 0 doesn’t mean we should accept it. It means that we don’t have enough data, and further investigation is needed.

The module automatically names the output columns according to the following conventions, depending on which type of t-test was selected, and whether the result was to reject or accept the null hypothesis.

Given input columns with names {0} and {1}, the module creates the following names:

ColumnsSingleSampleSetPairedSamplesUnpairedSamples
Output column PP_ss({0})P_ps({0}, {1})P_us({0}, {1})
Output column RejectH0RejectH0_ss({0})"RejectH0_ps({0}, {1})RejectH0_us({0}, {1})

This module computes and uses the sample standard deviation; therefore, the equation used contains (n-1) in the denominator.

Computing Scores for Single Sample Set

Given a single sample of scores, all independent of each other, and a normal distribution, the score is calculated as follows:

  1. Take the following input:

    • A single column of values from the dataset

    • The null hypothesis (H0) parameter μ0

    • The confidence score specified by α

  2. Extract the number of samples (n).

  3. Calculate the mean of the sample data.

  4. Calculate the standard deviation (s) of the sample data.

  5. Calculate t and degrees of freedom (df):

    formula for degrees of freedom

  6. Extract probability P from distribution table T by using t and df.

Computing Scores for Paired T-Test

Given a matched set of scores, with each pair independent of the other, and a normal distribution in each set, the score is calculated as follows:

  1. Take the following input:

    • Two columns of values from the dataset

    • The null hypothesis (H0) parameter d0

    • The confidence score specified by α

  2. Extract some number of sample pairs (n).

  3. Calculate the mean of differences for the sample data:

    formula for mean of differences

  4. Calculate the standard deviation of differences (sd).

  5. Calculate t and the degrees of freedom (df):

    formula for degrees of freedom

  6. Extract probability (P) from the distribution table (T) by using t and df.

Computing Scores for Unpaired T-Test

Given two independent samples of scores, with a normal distribution of values in each sample, the score is calculated as follows:

  1. Take the following input:

    • A dataset that contains two columns of doubles

    • The null hypothesis (H0) parameter (d0)

    • The confidence score specified by α

  2. Extract a number of samples in each group, n1 and n2.

  3. Calculate the means for each of the sample sets.

  4. Calculate the standard deviation for each group as s1 and s2.

  5. Calculate t and degrees of freedom (df):

    Optionally, satisfy the requirement that the variance among the groups be roughly equal, as follows:

    1. Calculate the pooled standard deviation first:

      formula for pooled standard distribution

    2. If there is no assumption about variance equality, calculate as follows:

      formula for pooled standard deviation

  6. Extract P from the distribution table (T) by using t and df.

Computing the Null Hypothesis

The probability of the null hypothesis, designated as P, is calculated as follows:

  • If P < α, set the Reject flag to True.

    If P ≥ α, set the Reject flag to False.

NameTypeDescription
DatasetData TableInput dataset
NameRangeTypeDefaultDescription
Hypothesis typeAnyHypothesisTwo-tailStudent's t-test null hypothesis type
Null hypothesized μAnyFloat0.0For the single sample t-test, the null-hypothesized mean for the sample

For the paired t-test, the sample difference
Target column(s)AnyColumnSelectionNoneTarget column(s) selection pattern
Assume equal variancesAnyBooleanTrueAssume variances of two samples are equal

Applies only to unpaired samples
Null hypothesized μ1AnyFloat0.0Null-hypothesized mean for first sample
α[0.0;1.0]Float0.95Confidence factor (if P is lower than the confidence factor, null hypothesis is rejected)
NameTypeDescription
PData TableA probability score that indicates the confidence of the null hypothesis
Reject H0Data TableValue that indicates whether the Null hypothesis should be rejected

For a complete list of error messages, see Module Error Codes.

ExceptionDescription
Error 0003Exception occurs if one or more of inputs are null or empty.
Error 0008Exception occurs if parameter is not in range.
Error 0017Exception occurs if one or more specified columns have a type that is unsupported by the current module.
Error 0020Exception occurs if the number of columns in some of the datasets passed to the module is too small.
Error 0021Exception occurs if the number of rows in some of the datasets passed to the module is too small.
Error 0031Exception occurs if the number of columns in column set is less than needed.
Error 0032Exception occurs if the argument is not a number.
Error 0033Exception occurs if the argument is infinity.

Statistical Functions
A-Z Module List

Show: