I'm a huge fan of A/B testing, and use it regularly. (Even wrote an open source implementation with a backend running on AWS Lambda), but have a bit of a moral / public perception / potential backlash fear of running pricing A/B tests... How do you deal with this? (Or you don't?)
I am not sure what you mean by a moral issue. Money and goods exchange hands (or not) in a mutually agreed on transaction. I don't think there is an issue so long as there is no intentionally targeted discrimination.
If someone is unhappy with their purchase and lets me know, I nearly always give a refund (where possible; the App Store prevents this). Sometimes I've given my software away for free if someone asks nicely.
I have never had any backlash to running pricing A/B tests.
About once a year I will run prices at 0.5 and 1.5 the normal price for a week or two to make sure I am still on the right point of the demand curve. Sometimes it moves and I will stick with a new price.
> I don't think there is an issue so long as there is no intentionally targeted discrimination.
Is random discrimination fine? I'm genuinely asking. I don't know the answer, but somehow it still feels morally shaky to me or unfair.
Yes, we do give refunds or discounts retrospectively in many cases, and try to be nice and fair and open. But there's just something a bit awkward with randomly charging people more (or less) for the exact same product, at the exact same time. I think most people will feel manipulated if they discovered it, even if they eventually paid the lower price.
Given my approach; a single A/B test once a year, one week at 0.5 price and one week at 1.5 price, less than 2% got a half-off discount, and less than 2% paid the higher price (actually, since sales usually drop for the 1.5 price, it's likely closer to 1% of sales pay the higher price).
That would pollute the test, IMO. As things like JCPenny's failed pricing experiments show, price-consciousness and deal-consciousness are two different things, and can lead to some stupid behavior where customers only buy overpriced goods that are marked as being on sale, rather than buying the same thing at the same price without the sale tag.
