No walking allowed
How to perform price optimisation with constraints on conversion and retention without allowing for price walking – comparison of two approaches.
A revisit on price optimisation ethics
First, let’s briefly recap what price optimisation is (or what do we mean by that in the context of this article) – it is an adjustment of profit margin across an insurance portfolio, in a way that maximises overall profit within constraints determined by business considerations (legal, ethical, strategic goals). More on that, and technique of price optimisation can be found in an earlier post [link].
Price optimisation does come with some ethical challenges. Adjusting profit margin independently for various customers means that some of them will pay more than others for essentially the same product, which does raise some concerns. This has also been discussed in that earlier post, to recap:
The mechanism of price optimisation will identify clients who are more likely to accept higher price (and still purchase the product) and will offer them that higher price. Looking at it differently, it will identify clients who are more sensitive to price increases and offer them a lower price. That change in perspective seems to also change how ethical it appears, and it does seem to be not only a matter of perspective (how you look at it), but also of what it actually means in individual cases.
A less price-sensitive client is one that will more likely accept an offer, with less regard for price, a more sensitive client is one that will not decide so easily, will put more effort into searching for a better offer or perhaps choose to forgo the purchase altogether if the price is not good enough.
This can be caused by various factors, lower price sensitivity can be a result of trust (if clients believe that we give them a fair price, they will look no further), inability to search for alternatives (e.g. no access or ability to efficiently browse through offers on the Internet), or simply the ability to pay more without a second thought (client might simply accept paying more because they can easily afford it and can’t be bothered to put more effort into searching for a better price and risking an unknown provider, even if that could save them some cash). It would seem perfectly ethical to offer a lower price to those less affluent (who signal it by putting more effort into search for a good price) while charging more to those who are not much bothered by it, so that company still makes its desired profit. It wouldn’t seem ethical to increase prices to those who are simply more trusting or attached to a provider. This should be identified and avoided. One such controversial (and banned in some countries) practice is the so-called price walking.
What is price walking
By price walking we mean increasing the price of renewing customers versus new business. It is a fact that renewing customers are more likely to accept whatever price they are offered, and less likely to compare it against competition than new customers, a fact that is often exploited in both insurance and other businesses. This does raise eyebrows, it does seem unfair to treat your old customers worse than strangers, it is specifically banned in some insurance markets (particularly UK) and even if legal, it might be unwise to behave in ways considered unethical and take advantage of customers’ trust – it might raise profits in the short term, at a cost of negatively impacting company image and long term profits.
In this article we’ll describe a price optimisation process that avoids price walking, while still allowing for setting sales targets with a distinction between existing and new business.
Constrained optimisation with no walking allowed
The main tools used to control optimisation algorithms are constraints applied to individual prices and selected aggregate measures, most commonly average demand rate (i.e. conversion/retention probability). Unconstrained optimisation algorithm would find prices that maximise overall profit, which doesn’t consider the desired business volume, demand constraints are introduced to manage it. Demand behaves very differently in new and existing business, so it does make sense to model it separately, treat it as a different variable and apply controls to it separately even when not wanting to make a distinction between new and existing customer in the final price.
On the other hand, price optimisation algorithm, which decides on the final price offer cannot depend on the distinction between new and old business – so we want to make sure that optimisation algorithm doesn’t “know” whether any individual profile is a new or existing customer and sets a price that doesn’t depend on that distinction, while still allowing control over new business sales and renewals. Added difficulty comes from the fact that we cannot expect new and existing customers to have the same characteristics distribution, which has to be taken into account.
The following compares two distinct approaches to the problem: optimising a combined new business and renewal portfolio, or optimising them separately and then averaging the final price.
Let’s start with a
Separate optimisation approach
- Import new business and renewal datasets separately, build separate demand models.
- Optimise premiums with desired constraints on renewals and new business sales.
- Join (concatenate) new business and renewal datasets with their respective optimised prices. Find midpoint prices by building a regression model with target of the optimised premium weighted by probability of sale (be it renewal or conversion, at this point information whether it’s new or old business is lost).
What this essentially means in a simple example is that if consider new and existing client profiles with the same characteristics, and that client:
- if already with us, would get an optimised price of say 100 and accept the offer with probability of 60%, and there’s 50 people like that
- if not yet with us, would get an optimised price of say 90 and accept the offer with probability of 5%, and we expect 200 quotations from clients matching this profile.
Then their weighted average of optimised prices will be calculated as
This is how a simple pipeline realising that process (for simplicity we assume the same costs model for renewals and new business) would look like in Quantee.
Now let’s look at
Joint optimisation
- Merge (concatenate) new business and old business model training data, while keeping purchase information as separate variables, since they do mean something different, so keep two separate columns for retention/conversion flag, fill retention column with zero for conversion dataset and the other way around.
Both those datasets might have differing characteristics, distribution of customers’ profiles. That’s ok, joining them does not change it, only waters down the information. We still know how many and which customers renewed or converted, what we don’t know is who are those people that didn’t, were they a new or existing customer – and that’s fine, it is what we want, we want not to distinguish between new and existing customers as they ask for a price offer, but we want to still be aware of number of renewals and conversions.
- Train conversion and retention models on the joined dataset.
Note that conversion rate, calculated as a number of data rows that converted divided by the total number of rows, does not hold its meaning now. It’s no longer a probability that a quotation will convert, but a probability that this row represents new business times the probability that it will convert. That probability multiplied by the number of rows still means the number of converted policies. And the same applies for retention. - Optimise with constraints on total numbers of conversions and retentions – not on rate, or alternatively on rate but keeping in mind that it relates to “watered down” new and old business demand, as above e.g. for retention does not mean average probability of retaining a customer, but average probability of retaining that customer * [number of rows in renewal dataset]/[number of rows in new business dataset].
What this method allows is to produce an efficient frontier chart showing possible combinations of sales for new and existing business of the same product.
Setting all prices to the allowed minimum would maximise both conversion and retention, setting all prices to maximum would minimise both of them. In between extremes, because demand characteristics for new and existing business are different, one sector can be more price sensitive than another in renewals but comparatively less in new purchases – by increasing or decreasing prices for various client profiles independently we can to a degree move in all directions in the space of new and old business demand. We can decide independently on how many existing clients do we aim to retain and how many new clients to attract, and price optimisation algorithm will find a combination of prices that achieves it while maximising expected margin.
This is how a relevant simple pipeline would look in Quantee:
Comparison of the two
Assumptions:
- We perform optimisation in both ways described above on new business and renewal portfolio, with the same actuarial costs model, extra 10% added to new business as acquisition costs.
- New business dataset is 10 times bigger than renewals.
- Global constraints are set on 0.5 average renewal and 0.05 average conversion.
- Final prices are scored against new business and renewal portfolios separately, with demands calculated using separately trained models (as in first approach above) for better accuracy.
Results are as follows:
All models scored against conversion dataset:
All models scored against renewal dataset:
First column represents joint optimisation, second – model averaging separate optimisation, and last two – two results of optimisation fitted separately on conversion and retention portfolios.
As expected best results in both cases are given by optimisation fitted separately to that portfolio, and worst – by optimisation fitted to the other one, joined model that averages those two, falls in the middle, while joined optimisation in comparison to joined model, manages to hit demand goals more accurately in both cases, while giving up a bit of margin on conversion portfolio but achieving more in retention and in total.
Other than numbers:
The joined optimisation approach allows for easier scenario testing, and generating an efficient frontier chart, and it ensures that constraint values do not change significantly in “averaging” step, while returning optimised price, hence is expected to be more accurate and return higher total margin. It is a significantly more complex calculation though, more difficult to initially correctly set up, and particularly sensitive to inconsistencies between demand models.
On the other hand, averaging of separately optimised prices is less complex, easier to explain, and the final price calculation is performed by a single regression model, all the rest of the pipeline is only required to set a target for that regression. This means that the working price calculation is simple and fast. On the other hand, the same can be done at the end of joined optimisation pipeline (the final price set as a target for regression model, then used for quotations).
Considerations
While not price walking (not explicitly changing price depending on whether client is new or existing), this approach might still systematically decrease (or increase, dependent on selected constraints) profit margin to existing customers vs new – by increasing or decreasing it for those sections of market with particularly high or low proportions of existing customers, rather than for individual customers. This should allow for either focusing on keeping existing customer base or increasing presence in currently underrepresented market areas.
It should be noted that how demand models are built does influence final results, in particular whether price tests made on existing and new customers follow the same pattern. Otherwise, joining new and old business datasets would impact demand curve shapes (not only scale them) and would not accurately represent price elasticity. Results might be more accurate in joined optimisation approach with separate models for probability of client being new/existing, built on joined dataset, and without using offered price as a feature, and them purchasing policy (depending on price) built on separate datasets for renewals and conversions, rather than doing it in one step as above. Those models combined would give, possibly with greater accuracy, the same result – probability of client being new or old multiplied by probability of them accepting price offer.
