Testing – the Most Effective Tool for Database Marketing

Each marketing campaign plan should be tested on a smaller group of Customers, before being deployed. Based on the results of the test campaign, the marketing campaign can be reshaped to achieve better results.

Testing is essential in every campaign, given that the business environment changes dynamically. Nothing should be taken for granted. Testing can be used not only for the estimation of short term response rates and marketing campaign ROI, but also to monitor the medium term effect (e.g. one year) of a campaign.

The steps to implement a test campaign, are the following:

1. identify a group of customers, who are potential buyers of a product, e.g. 20.000 Business Customers to receive an offer developed for the Business segment.

2. set up a test and a control group (e.g. 2.000 Business Customers each). The test group gets the offer. The control group is treated as usual, but it does not get the offer or any other offer. Ideally, the test & control groups consist of the same type of Customers (Business Customers). The groups can be easily selected by sorting the Customer group and taking the 3rd out of each 10 for the test group and the 5th out of each 10 for the control group. The size of the control group has to be sufficient to provide statistically accurate results.

3. the test campaign is executed and results (orders made by the 2 groups) are captured.

4. the effectiveness of the test campaign is evaluated by the difference between orders by the 2 groups (test & control), in the given campaign period.

In order to exploit efficiently a control group, the following additional rules apply:

o no other offers should be made to the control group, during the ‘control’ period

o the control group should not be informed of the offer made to others – this may lead to negative reactions and introduce bias to the evaluation

The test group is expected to produce increased orders. The comparison to the control group can factor out any market or economy factors, affecting the result. This is why the set up of a control group is so important, in order to evaluate accurately the effect of the test campaign.

Increased ordering made by the test group (vis-à-vis the control group), can be identified in certain cases on non-promoted products also. This can be attributed to the campaign, since it may increase the visits (or interest for the Business and its products) of the test group.
The effect of the campaign can be extended beyond the campaign period. This is attributed to the fact that those Customers who accepted the campaign offering, have increased their recency.

The same approach to use a control group for evaluation purposes, should be applied to the roll out of the campaign as well.

In a test campaign, alternative offers may be tested, in order to figure out the most efficient approach, yielding higher ROI. In this case, only a single variable should be different between alternative offers (e.g. the level of discount), in order to produce valid results. Two test groups (or more according to the number of alternative offers) are needed, since each Customer should receive a single offer (during a promotion, a Customer should not be asked to choose between alternatives). Again a control group should be set up. After the test campaign is concluded, comparisons between test groups behavior as well as evaluation vis-a-vis the control group, should be carried out.

Testing comes at a cost. The Business has to assign resources to execute the test and capture & evaluate the results. Moreover it has to delay the campaign roll out, for one or more months. Since roll out of the campaign follows the test campaign evaluation, there is a delay in campaign execution.