Managing Poor Parameter Recovery

Under Construction

Poor parameter recovery can indicate places where the model is struggling to get a good read on measuring particular channels. This can be both an opportunity to improve the model as well as explain to your client where the difficult reads are, and strategize on how to improve the signal the MMM is getting.

Modeling strategies

1. Consolidate similar channels

2. Assess placement of saturation spikes

Saturation spikes reduce the channel saturation in the region of the spike, making spend more effective. Figuring out where to place the spikes to get good recovery may take some trial and error.

3. Make small channels non-moving

Reducing the number of parameters in the model can improve recovery. If you have many channels with small spend, you won't have enough data to inform how the beta changes over time. By setting them to non-moving, you estimate a single beta for the entire time series (ROI will still change based on saturation levels and contextual variables), which cuts down on the number of parameters the model has to estimate.

4. Adjust the spend levels for particular channels

Spend levels control how fast a channel will saturate. Lower spend levels will mean quicker saturation, with higher spend level resulting in less saturation. Adjusting these values may make the model easier to identify.

5. Increase the number of days of data

If you have less than 2 years of data, getting more may improve parameter recovery.

6. [Coming Soon] Group channels by betas, timeshift, or both

Grouping is a strategy that is similar to consolidating channels but less extreme. Instead of merging the channels the model will group their estimates hierarchically, so that they share information with each other. Both the betas and the timeshifts can be grouped.

7. [Coming Soon] Adjust kernel stiffness