When to try new things.?

Have you ever been in this situation, deciding between trying a new Chinese restaurant, ‘Dragon House’ or sticking to your favourite Domino’s pizza?
.
.
.
.
Always going to your favourite one isn’t going to work. You’d never discover anything better if you only exploit your few favourite things. If you only ever explore, you’d find new things and never enjoy your favourite food. The best strategy to try is some combination of the two. But what combination?

There are two options, with a different probability of reward and a certain amount of effort, like money or time you sacrifice to pursue them. That is where John Gittens comes to rescue us.
Now an interesting thing that Gittens did was he assumed that a good meal tonight is worth more than a good meal tomorrow. And a good meal tomorrow is worth more than a good meal a year from now. This idea is called discounting, and it accounts for the uncertainty of the future.

Taking all of this into account. He came up with a formula which gives us a number called the Gittens index which looks like this;

To pick a restaurant, choose the option with the highest Gittens index.

Let’s say with Domino’s a total of eight times; six of those times, you had a good meal and two times you had a bad one. Then Gittens index for Dominos will be 0.84. Because you’ve never been to Drogon house, you had zero good and bad experiences. And when we calculate this, it gives the Gittens index of 0.87. ‘Dragon House’ is the highest, So tonight, you should explore.
You’d say that a place you have no information about has a 50/50 chance of being good or bad. And you’d be right. But the good index doesn’t capture the probability of a place being good or bad. It captures the total reward you’d receive from going there. Think about it. If you went to ‘Dragon house’ and there were flies in the soup, which counts as one bad meal and probably you won’t visit again, but if it ended up having the best noodles you’ve ever had, you have many more nights of deliciousness ahead. Here the Gittens index considers that the game is potentially finding the new favourite far exceeds the loss.

This Explore/ exploit trade-off applies to many more decisions than you think. Whether it is picking up a new stock/depositing your money in FD, meeting new people/hanging out with old friends. On a broader level, whether a company should invest its time and money into R&D to discover a new product or stick to improving the current business model. In the end,

Balancing both is crucial to get the most out of life; optimising your information to act and discover something new every once in a while is key to survival.!!

credits -
Algorithms to Live by (book)
Practical Calculation of Gittins Indices for Multi-armed Bandits (research paper)