When can a system be considered to have "proven" itself ?

[Datapunter]

In another thread i made a comment that you cannot draw any meaningful conclusion about a betting system based on having watched football matches for only a few months. You need a substantial number of bets before you can say a system has "proven" itself and you can commit cash to it. Prompting these questions.

Datapunter,you appear to have answered a question I asked on another thread a few weeks ago re.when exactly a system is said to be "proven". Could you please confirm that its proven after 500 games? Is this something that is generally agreed upon?

If so,presumably that means that,at the present rate,we will have to wait til about 2016 to find out if this works before we all lump on!? I spose the bookie offering the best prices on the double that day will get a big surprise:)!! also,in november 2016,what roi will the bets have had to have made before its deemed proven? would 1% be enough or is there a minimum roi required after 500 games? ( Question relating to thread: http://forum.punterslounge.com/f21/safe-multibet-system-122797/ )

How many people in the history of PL have actually achieved this 500 game proof and do we know what happened afterwards ie did the prices move on the week after the system was proven? If not,then does anyone have an explanation why not?

So how many bets do you need before you can say "right, let's go for it." ? Nice enough subject to merit a thread of its own.

[Grex]

Re: When can a system be considered to have "proven" itself ? There are probably some very complex statistical probability theories that can give the answer to this question. I am not in this league, so I will give my opinion based on logic and common sense (assuming I have any :p). I don't think a system can ever be "proven". The best you can hope for is a high level of confidence in the system. For me, consistency is the most important issue. If a system is showing yields that go up and down like a fiddler's elbow, then it should be treated with caution. The system should also show this degree of consistency over at least two seasons (with any system, you can get the occasional "rogue" season where results go out-of-line). As for the minimum number of bets..... I don't think this is particularly important. Obviously a system cannot be "proven" based on good consistency over two seasons, but with only 20 bets. I would say that a figure of around 250-300 should give a reasonable level of confidence in any system.

[aderuk]

Re: When can a system be considered to have "proven" itself ? With all the back testing possible now through various sites maybe somebody from the SSB could maybe give an answer. Something from one of the profitable systems like ROI etc from say the first- 10 bets , 50 bets ,100 bets,200 bets ,500 bets etc and see what the stats come up with after each stage,if they are consistent at each stage then i suppose there is no real answer to how many bets are required.

[Datapunter]

Re: When can a system be considered to have "proven" itself ?

Datapunter,you appear to have answered a question I asked on another thread a few weeks ago re.when exactly a system is said to be "proven". Could you please confirm that its proven after 500 games? Is this something that is generally agreed upon?

The answer to the question: how many bets do you need before you can say a system has 'proven' itself is.... there is no answer. At least there is no specific answer. Ultimately it is a matter of choice and declaration. A system has 'proven' itself when you say it has. So how do you determine this for yourself ? One possibility is to say "never". This would be a matter of attitude and strategy rather than mathematics. You simply assume that any system can flip from profitable to losing at any given moment in time no matter how much history it has. After all: results from the past are no guarantee for the future. This simply results in very prudent and risk-averse money management. I.e. modest staking. In other words when you reach a point, by whatever method, that you are confident enough to place your first bet then that's it. Another possibility is to set a mathematical target. There is a mathematical thing called: Chi-squared test . What it allows you to do is compare a set of data against a reference set resulting is a number that represents: "Goodness of fit" . Hint: Google is your friend. In other words you compare the result of your betting against a sample of random bets. The number you get represents how confident you can be that your result is indeed due to something you are doing rather than just luck. The more bets you get the more accurate the number gets. At what point you declare to have reached sufficient "confidence" is ultimately a matter of choice. Regrettable due to work and studies i lack the time to go into this in detail. But i'm sure someone else will pop up to explore this option in more depth. Also do a search on PL as there have been a few threads exploring the subject. And then there is plain common sense. Which will obviously be different from person to person so i'll stick to my own view which is "several seasons". I tend to declare a system 'proven' if i have data that shows reasonable consistent profit over a period of several seasons. ( ! providing there are a substantial number of bets in a season ) Personally i've only ever looked at systems that produce a lot of bets and by that i mean at least several bets per week so the simple statement of 'several seasons' has always been good enough for me. Other people look at systems with less bets and there you need a longer period. So back to the original question of how many bets ? Whatever number your common sense tells you. Note here that "several seasons" does not nessesarily mean actual betting. Results based from back testing can also provide that info. In that case i personally look for several seasons back tested data and a period of maybe half a season of real life data, i.e. actual betting. This to confirm the odds i'm getting in real life match up to the odds used in back testing. Also note there can be huge differences between systems for football, tennis, horse racing, favourites or underdogs so you need to place things in the right context. In other words a discussion of the subject for a specific individual system is worth the effort.

Is this something that is generally agreed upon?

I don't think there is a general agreement on this subject. At best most people that apply common sense will arrive at a similar statement. But in the end you need to make your mind up for yourself.

[Datapunter]

Re: When can a system be considered to have "proven" itself ?

If so,presumably that means that,at the present rate,we will have to wait til about 2016 to find out if this works before we all lump on!? I spose the bookie offering the best prices on the double that day will get a big surprise:)!! also,in november 2016,what roi will the bets have had to have made before its deemed proven? would 1% be enough or is there a minimum roi required after 500 games? ( Question relating to thread: http://forum.punterslounge.com/f21/safe-multibet-system-122797/ )

The problem with that particular system is that the selection process is partially a matter of personal assesment. In other words different people will come up with similar but not the same selections. Also some of the criteria used only exist the moment itself. And then some of the criteria used is... ermmm... highly debatable. The result is that it simply cannot be back tested against sets of historic data. From the look of things there are only a few bets per week, say an average of 3, so a season might get just over 100 bets (individual ones). That's pretty thin considering the number of possible selections. So yes, my common sense tell's me several seasons which puts us into 2014/2015. Assuming it is actually possible to be consistent with that particular system from season to season (which i doubt). As to the second part of the question: No. There is no minimum yield. Again it's what you choose. The yield you get will mainly determine your staking strategy. If it is low then it won't take much to flip into a loss and prudent staking is needed. If it's on the high side you can stake a little more as your risk(s) are somewhat lower.

[Datapunter]

Re: When can a system be considered to have "proven" itself ?

How many people in the history of PL have actually achieved this 500 game proof and do we know what happened afterwards ie did the prices move on the week after the system was proven? If not,then does anyone have an explanation why not?

I can only refer you to a thread i ran some time ago: soccer-hot-favourites-system ( note i'm typing this post based on memory, forgive me if my memory is not entirely accurate ) At some point in that thread you'll find a pretty long post explaining why i thought the system would never fail. After all there where 10 years of back testing and something like 3 - 4 seasons of actual betting. And the reasoning behind "will never fail" was pretty solid. And yet, after all those seasons and a few thousand bets (including the back testing) the whole system fell flat on its face in just a few months. As far as i know it has never recovered ( then again i haven't examined any recent data ) It was an expensive lesson. If you go through the 1000+ posts it will be time well spent.

[v-zero]

Re: When can a system be considered to have "proven" itself ? A Chi-sqaured test (pronounced kai, by the way) is somewhat suited to the task, but not perfectly so. A more useful test is to put a bound on how likely it would be to observe your results from your set of bets under the assumption that the odds received are fair. For instance, in my close games thread there is around a 5% chance that my bets would have achieved that yield or better over that many bets by mere chance. The easiest way to do this with accuracy is to do something called a monte carlo simulation. The ROI and number of bets are hugely important to this calculation, as are the odds at which the bets are placed.

[Matthew]

Re: When can a system be considered to have "proven" itself ? The Archie score is a pretty good alternative to the Chi Square.

[UsManyDead]

Re: When can a system be considered to have "proven" itself ? Pretty interesting debate you have here. When a system can be considered to have "proven" itself? This is the million dollar question. If we could answer this with ease then we would be, at least rich, if not millionaires. You could actually profit at maximum from your system, cause as fast as you can get to this conclusion the sooner you can start betting and catch the good results (assuming your system has a + yield). So yeah I think it is pretty hard to say this. The only fact that could contribute to the answer would be: consistency, meaning you either have a constant yield or a constant S/R over a period of time. IMO this should be enough even for a single season, cause if your system has proven profitable enough (I don't know the avg yield but I would say something above 6% - 7%, should be good, but above 10% should suffice). Also an important factor should be the bad run that any system encounter: you must see how long is and how often it happens. But if you still end on profit than this should mean that your system is taking care of this and even bring you profit. Why could you actually believe that since this season was profit the next will be the same? After all past results are no guarantee for future results, or so is the saying. Well how did your system manage to give this results? Any method your system has used, a whole season of profit can't be called "the beginners luck". O the other hand you have to trust your system; after all you are the one who made it, you saw how it went, what are is weaknesses, it's strong points, you know everything about it. Analysing it's past performance carefully you should be able to say that you trust your system, this is the case if you had a profit at the end of the season - a number that would please you. So after a lot of talk, and probably a lot more that wasn't said, my own conclusion is that: YOU are the only one who could say when you are CONFIDENT enough to start betting on YOUR SYSTEM. If you think that someone else could say this for you, you are wrong. Also there wouldn't be possible to actually be a generalized idea of length/limit/yield/nr. bets that would be applicable to any system that is out there. This is impossible. So you are the last barrier that stops you from believing in your system; after all would you bet on something that you believe in or in something that someone else is telling you to believe on. I, for one, would go with the first option. One more thing I want to add: it is pretty interesting to say that after 300 or 500 bets the system should be declared a sure shot. Why is that? Isn't this the same as "results from the past are no guarantee for future results"? I think it is. You can have 3-4 seasons on row when your system brings profit and on the 5th, when you finally decide that it's TIME to bet, you get the bad one, the one that would put you in red. It could be the other way round, but that would be the best fact to happen. Anyway at the end you'll realize that you've lost 3-4 seasons of good results and profits. Anyway you take it, I believe that a system can't be used for more than 5-6 seasons without changing some things on it, as a lot of factors will differ. Please don't get me wrong, I agree with the saying "results from the past are no guarantee for future results", but if you believe it too much, then you'll never be able to move on. You actually need to believe in you, that you can stay concentrated, self-disciplined, self-confident and you are able to deal with the ambiguity. All these combined with a profitable system will get you further. L.E.: Following a system won't be like racing with a Lamborghini, where you get from 0 to 100 mph, in less than 5 seconds, but more likely riding in a roller coaster, where you go either up or down with fast speed. ;)

[plinge]

Re: When can a system be considered to have "proven" itself ?

The Archie score is a pretty good alternative to the Chi Square.

I'm no stats expert, but Archie is in fact a probablity value determined from a chi-square distribution table is it not? I have seen some debate about its merits in the past, which I believe actually relates to the discussion we had in the SSB thread about yields at various odds bands. My understanding is that if betting to take out a fixed return, a chi-square test would measure the precise difference between a set of results and that anticipated by the bookies' prices. It should also, therefore, be an excellent measure of a sample like V-Zero's "close matches" because of the intrinsically narrow odds range. On the other hand, I believe it is true that Archie won't take into consideration extra emphasis on some bets. Eg: for someone level staking, there is an inherent assumption that higher odds should return a higher yield. Or if someone is using a rated price/probablity, Kelly staking would advise a range of bet sizes depending on "overlay" or "advantage". (I think the reason that Kelly has a reputation as a high risk approach in the punting community is that it is based on KNOWING your EXACT advantage. The only gambling situation I'm of which comes close to this is card counting at blackjack, and even then it is a slightly different scenario to the assumptions of Kelly due to the possiblity of splits and doubles after a bet has been made. With sports or racing ratings there are, in reality, few if any whose precision will remain constant at any market odds.) I still believe Archie is a fine measure for level stakes or betting to rated prices, however, it should surely be useful to divide results into samples representing various odds bands and/or degrees of theoretical advantage and generate separate p values?

[plinge]

Re: When can a system be considered to have "proven" itself ?

A Chi-sqaured test (pronounced kai, by the way) is somewhat suited to the task, but not perfectly so. A more useful test is to put a bound on how likely it would be to observe your results from your set of bets under the assumption that the odds received are fair. For instance, in my close games thread there is around a 5% chance that my bets would have achieved that yield or better over that many bets by mere chance. The easiest way to do this with accuracy is to do something called a monte carlo simulation. The ROI and number of bets are hugely important to this calculation, as are the odds at which the bets are placed.

For those of us without a stats background, would you be good enough to clarify whether the "monte carlo simulation" generates a different result to that of the "Archie" probability value? And congrats on the results in your "close matches" thread. A 17% yield with your high volume of action and excellent strike rate is very impressive; and no surprise to see that it is already down to a 5% chance that you've simply been dancing with La Divina Fortuna.

[plinge]

Re: When can a system be considered to have "proven" itself ?

One more thing I want to add: it is pretty interesting to say that after 300 or 500 bets the system should be declared a sure shot. Why is that? Isn't this the same as "results from the past are no guarantee for future results"? I think it is. You can have 3-4 seasons on row when your system brings profit and on the 5th' date=' when you finally decide that it's TIME to bet, you get the bad one, the one that would put you in red. It could be the other way round, but that would be the best fact to happen. Anyway at the end you'll realize that you've lost 3-4 seasons of good results and profits. Anyway you take it, I believe that a system can't be used for more than 5-6 seasons without changing some things on it, as a lot of factors will differ.[/quote'] It helps to have at least a theory as to why a system has worked in the past and/or is showing current profits. Some are more resistant to change than others, and if they had enough statistical validity before souring, then yes it is likely due to altering circumstances. In any sporting field time brings changes of tactics, players and rules. I would have liked to have seen someone with a system based around certain player statistics maintain their strike rate on Australian rugby leage over the past ten years. A host of rule changes and changes to rule interpretations has combined with a new age of professionalism and competitiveness among the coaches, so that the game has altered dramatically on a season-by-season basis and very different demands are now made of players in various positions. On the other hand, some things have reamined constant, such as the most isolated teams having excellent home ground records. If a system had managed to work something stable like that into a profit, it might well keep on ticking over at a similar strike rate and yield. It is also important to be aware that market psychology changes over time, and this means that similar scenarios will not necessarily result in similar odds. A simple example would be a previously profitable tipping service or system becoming overexposed and driving down the prices of its selections. A more complex scenario would be what has occured in Australian horse racing over the past 20 years: where the biggest betting action has moved from the on-course ring to the off-course corporate bookmakers, and a new style of punter with a very different mind-set--and access to a banquet of internet and software based ratings and information--has emerged.

[v-zero]

Re: When can a system be considered to have "proven" itself ?

For those of us without a stats background, would you be good enough to clarify whether the "monte carlo simulation" generates a different result to that of the "Archie" probability value? And congrats on the results in your "close matches" thread. A 17% yield with your high volume of action and excellent strike rate is very impressive; and no surprise to see that it is already down to a 5% chance that you've simply been dancing with La Divina Fortuna.

It will not produce a result that is significantly different to an Archie value if the number of bets is large and the average strike rate is close to 0.5, but otherwise it will produce a result that is more specific. Moreover the simulation will not give an approximation, but will in fact render exactly the answer to your question.

[Junction 6]

Re: When can a system be considered to have "proven" itself ? I've been creating systems for quite awhile now and it's usually around the 300-500 mark do I find the vast majority of them I persevere with going pear-shaped. However, recently I've come up with a horse racing system that made a profit of £6,500 from 1400 bets (29.7% yield), betting 1% of the bank. The original bank was £200. Sounds pretty impressive but fast forward on to 2000 bets made, and the profit dips alarmingly to £1,267 with a 3.37% yield. :unsure It may have yielded a profit (exactly 100 days of betting) but it just goes to show what can happen even after a large number of bets get researched. Out of interest the same system made £904 profit (32.22% yield) after 1400 bets and £652 profit (16.32% yield) after 2000 bets using level stakes (£2).

[craigeecraigoo]

Re: When can a system be considered to have "proven" itself ? I've been using the system creator tool from football-bet-data that lets you filter your own selections based on their hstorical predictions. Seem to have devised a nice consistant system based around 2-2 draws and specific odds. I did post a rough cut system on here using that basic criteria but never had time to keep on top of the bets per day, but having used their tool to back test games it does produce a decent yield over 800 odd games - and has made me a steady profit since late January. Still need to work on a staking plan though - that is my next venture

[Peter May]

Re: When can a system be considered to have "proven" itself ? Most replies here are very helpful with this tricky problem. No matter what the results show and system will never guarantee future profits, however some will do better than others. Regarding the number of bets this depends on the variability of the return. So, for methods making long odds selections more bets are required before the method can be considered sound. Statistical tests can be used, such as those based on chi square and confidence intervals, however these should only be applied to "new" data, not the data used to develop and test the system. They can then be very helpful in determining whether to implement the method or not. I hope this helps.

[lunatism]

Re: When can a system be considered to have "proven" itself ? Woohoo, it's been ages since I wrote on here, but I feel like this is something that i can throw some knowledge and opinon at... launching a system The "launch" of any system into real world betting (as opposed to paper testing and backtesting) is a big step, after all - ANYTHING you backtest or have previously bet on could be part of a long series of favourable results that ultimately mean nothing if you looked back a little further. As Datapunter mentioned, any system, no matter what the reasoning behind the selections could simply fail to work, the problem is addressing the likelihood of that and countering it either with a strong staking system (as was hinted at) or by analysing exactly how your system is generating selections to begin with. variable examples For example, if the variables that you consider for a system are primarily based on stadium size and attendance (which can be extremely useful if you are trying to work out if teams can be intimidated by large crowds even if they are the favourites). Then what happens if there is a worldwide economic crisis which means that despite having large stadiums, clubs are finding that there are lower attendances and lots of empty seats? Are teams still going to be intimated by all these silent seats, you would imagine this could work one of two ways - the home side is motivated by lack of fans (believing that better performances will bring the fans back) and start steamrolling people at home or the away side consider this whole thing a joke, bring a huge amount of travelling fans, outsing the home fans and intimidate the home team (and the refs, who are oftentimes influenced by crowds even if we'd not like to admit it) into losing. variables affect variables Now that was just an example based on simple attendance/stadium capacity variables, but such analysis could be done on any variable that you consider. The best way to judge the longevity or future success of a system, is to consider which of the variables are most likely to be influenced by other external variables...eg - if you are looking at pass completion, you have to presume that eventually the opposition teams will learn how to counter this, if you are looking at away form to judge performance in home matches, then you have to consider that some teams are simply PANTS away from home and that will never change (see Fulham or Atlético Madrid (in the last few seasons at last)), if you are looking at the overall value of the teams, then you have to consider that certain managers or certain clubs always spend big, but is the financial spending reflected in league performances or tournament wins? (look at Man City - won nothing so far this season, same can be said for Málaga). longevity percentages The best thing to do with your systems, after doing Chi tests and other statistical analysis, is to perform a variable analysis. Is there any variable that will be effected more than others? What variables will not be affected much at all? If you were to wade through your variables and give each a mark out of 10 for consistency (0 being fickle, 10 being very unlikely to be influenced), you'd be able to draw up a percentage of longevity, this could stand side by side with your statistical analysis to give you a much clearer idea about the success/future success of your system. the future The good thing about such an analysis, is that unlike the statistical analysis, that only can give you an interpretation of what has happened already (which is obviously critical), but also what could happen in the future. Plus, it would also help to explain why one season you have a terrible patch of results, while the next (during the same part of the season), the results are phenomenal. If you knew why your system threw up some bloopers last season (eg. one of the variables was heavily affected by something), then you would feel more confident about continuing with it next season. I'm not sure about you guys, but when i'm backtesting, if I find long consistent periods of bad results, if I cannot find a good reason for the slumps, I will toss the system. Now obviously that is a personal choice, but for me to be successful from betting, i need to be able to rely on my systems, and any systems that throw up unpredictable and unexplainable periods of poor form are not good investments and can hardly be considered reliable. With variable analysis, you can quickly pinpoint exactly what has gone wrong and see whether it is a "reasonable slump" or not. Like many others, in the past, i have created systems that have blown through a few seasons and looked phenomenal, and then be ready to start betting for real, only to find that they've simply turned to crap and been left with egg on my face! These things are learning experiences though - and you can only get wiser from learning from our mistakes :p

[kosminus]

Re: When can a system be considered to have "proven" itself ? i think a system is proven not in terms of numbers of bets. it`s proven in months of profit and in roi .

[moggis]

Re: When can a system be considered to have "proven" itself ? Hi,I am the person who asked those questions and so thanks for the answers. I am absoluetly fascinated to learn that a mod on here stated the reasons why a particular system would never fail. I assume you must remember the reasons? Any chance of repeating them here-1000 posts is a lot to look through...though I do intend to have a read at some point. Thanks

[cachirro]

Re: When can a system be considered to have "proven" itself ?

i think a system is proven not in terms of numbers of bets. it`s proven in months of profit and in roi .

how do you calculate ROI? is it the same as YIELD? money you won / money you would lose? thanks

[Grex]

Re: When can a system be considered to have "proven" itself ? ROI = YIELD = TOTAL PROFIT (loss) / TOTAL STAKED (expressed as a percentage) :ok

[AFK7]

Re: When can a system be considered to have "proven" itself ? All i know is if i want to believe in my bets i run 300,000 through my computer model! 500 is nothing, i have had 29 straight odds-on losers!