How probable is your profit? (robustness?)

[PAULM03]

Following on from Foolsgold's thread about chi-square, and in particular a suggestion by sgt. sunshine I've been thinking about the best way to tell if your profit after a particular number of bets is down to luck or judgment. IMO the suggestion by SS in that thread that a good way to test would be to repeatedly simulate all your bets using fair odds, get the distribution of yields and then see where your yield lies on that distribution is a good one. So that's what I'm going to do (using Foolsgold's data if he doesn't mind:ok) Should have something up and running fairly quickly so watch this space (and thanks for the suggestion SS:))

[AJ]

Re: How probable is your profit? (robustness?) Go ahead and use my data, I'll be very interested in the results

[PAULM03]

Re: How probable is your profit? (robustness?) OK, got some figures. I thought of a more sensible (and much easier) way to do this. I simply ran a simulation of those bets 1000 times and recorded the number of winners each time, along with the mean, median and standard deviation. First off, the mean and median are very similar (243.74 and 244 respectively) indicating that the number of winning bets is normally distributed around the mean. The standard deviation for this particular record of bets is 9.73, so now you have the mean and SD you can calculate how far your number of winning bets is away from the mean in terms of the number of SDs. Subtract the mean number of winners from your actual number of winners and divide that by the SD. In this example: actual no. of winners = 280 mean = 243.74 SD = 9.73 So your number of winners = (280 - 243.74)/9.73 = 3.8 standard deviations from the mean. This is highly statisticaly significant as very nearly 100% of random runs (or luck:)) falls within 3 SDs, so a value of 3.8 means that it's as certain as statistically possible that your record so far is down to skill rather than luck. There is a caveat to this though, the mean is highly dependant on the overround, so it would be a good idea to find out what overround would make your results statistically insignificant. So using the fairly standard definition of significant as something that has less than a 5 percent probability of occuring by chance and assuming a normal distribution we're looking for a number of winners at least 1.645 SDs away from the mean. For foolsgold's data this occurs with an overround of just over 3% making it highly likely that even if the assumption about the overround of 12% is invalid, unless he's consistently placing bets at something close to fair odds his results ARE significant:)

[Sgt.Sunshine]

Re: How probable is your profit? (robustness?) Just to complicate matters, don't forget that the over-round isn't applied evenly between home, draw and away due to the fav-longshot bias. This needs to be taken into account, especially in this case as a lot of Foolsgold's bets are odds-on.