[CSGibson]

I got this e-mail recently, and I thought I'd throw it out to the forums:

Chris,

Some time ago, you and I had a discussion about the effectiveness of aggressive, variable preempts. In as much as you were on the opposite side from me, I think you are in a good position to comment on a methodology of studying the effectiveness of preempts. What are your thoughts on the following possible study.

Look at Kit Woolsey and Fred Steward, a pretty strong nationally-rated pair who preempt as frequently as they can, on both good hands and bad hands, and who believe their overall results from undisciplined preempts are excellent. Consider a study of the results of their preempts in national KO's that examines only their results after they preempt at the 2 or 3 level against very good but not fabulous opponents [say round 32 and better of national KO's but omitting client pairs]. Compute all imps they win or lose against the result at the other table on their preempts. Weed nothing out even if there appears to be some kind of anomaly. Here are my questions:


1. How many matches would you want to examine before you would think the results were meaningful? How many preempts?


2. Would you want to find a way to factor in the effect of the preempts on other hands - that is on either Woolsey and Stewarts bidding on other hands or on the opponents bidding or play on other hands? If so, how would you do it?


3. To what extent if any would you factor in the opinion Kit has often expressed that these bids have been a significant winner over time?


4. What do you think of the methodology? Would you suggest any changes or some other approach?


5. What would you expect the results to show? How different would you think they would be if you considered the effect of the preempts against weaker national level pairs? What about against stronger ones?


6. Finally, what do you think are the biggest and most frequent gains Stewart and Woolsey realize on their preempts? What about the biggest and most frequent losses they suffer?


I am asking you these questions only for perspective. I don't plan to renew our discussion though if you like, I would be willing to share a summary of my results.


Thanks for your thoughts.

[CSGibson]

My own first thoughts are below:

1. How many matches would you want to examine before you would think the results were meaningful? How many preempts?

I imagine that I would want to catalogue at least 200 preempts, not only to make the results meaningful, but also to examine the circumstances around the preempt and cataloguing those, so we can tell in what circumstances their preempts were effective and in what circumstances they were not.

2. Would you want to find a way to factor in the effect of the preempts on other hands - that is on either Woolsey and Stewarts bidding on other hands or on the opponents bidding or play on other hands? If so, how would you do it?

I might catalogue Woolsey/Stewart's overall imp expectation per hand, then calculate the imp expectation on hands where they preempt, and imp expectation on hands where they don't preempt, and see how they vary. Speculation as to why they vary might be interesting. I might also break it up into hands they defended vs declared when they did and did not preempt.

3. To what extent if any would you factor in the opinion Kit has often expressed that these bids have been a significant winner over time?

I don't think I would give much credence to the view that Woolsey espouses, in lieu of actual data. Though if Woolsey actually said in what way he thinks these bids win, that might give additional criteria by which to evaluate the preempts.

4. What do you think of the methodology? Would you suggest any changes or some other approach?

As I mentioned earlier, I think the far more relevant criteria would be to catalogue the circumstances surrounding the preempt - are they playing against an ambiguous opening like a precision 1D/1C opening, for example, which is notoriously better to preempt against do to the less defined nature of the bid, or against Canape where the opponents may now be preempted out of showing their longest suit at a safe level; how far off is the preempt from a "normal" preempt, and in what way does it vary. I would also want a control pair, someone who is more mainstream in their preempts, but has experienced a similar level of success to Stewart-Woolsey. Perhaps Boyd-Robinson (I'm not sure of their preempting style, we didn't play against them when we beat that team)? I would also like to examine the volatility of the results - what I mean by that is the number of imps churned out per board when they preempt vs when they don't, regardless of whether they are positive or negative. Another consideration is whether the preempt was attempted at the other table. If it was, then why was the result different at the two tables?

5. What would you expect the results to show? How different would you think they would be if you considered the effect of the preempts against weaker national level pairs? What about against stronger ones?

I really have no idea what to expect. I suspect that the effect of preempts against weaker national players to be more effective, but I also expect that the effect of non-preempts also to be more effective against weaker national players. Perhaps that points to a match-by-match comparison of preempt expected value vs non-preempt expected value.

6. Finally, what do you think are the biggest and most frequent gains Stewart and Woolsey realize on their preempts? What about the biggest and most frequent losses they suffer?

I expect that Woolsey and Stewart realize the biggest and most frequent gains from exploiting their opponents when they are ripe for preempts, just like everyone else. I suppose they probably have frequent gains from inferences in the bidding and defense when they have not preempted, since they have eliminated quite a few more hands to play for. I expect their biggest and most frequent losses to occur when they have preempted and it is their hand, or they run into a stack behind them.

[helene_t]

Suppose the SD of the IMPs gained by a preempt is five and you are interested in the average IMPs gained per preempt. Say you study 100 preempts. The standard error on the estimated average gain is then 5/sqrt(100)=0.5. I think this is too large a standard error since the average gain or loss is probably small. 1000 preempts may be more reasonable. But on many hands, Kit's opponents at the other table also chose to preempt. To find 1000 hands on which one pair preempted while the other did not you might need some 20,000 hands, don't you think? If you want to do subgroup analysis you need more hands.

Also consider the gains and losses associated with the negative inference when partner doesn't preempt. When Kit passes in first seat and then enters the auction later, the inference is stronger than it would be for a pair with a more disciplined preempt style.

[akhare]

Quote

2. Would you want to find a way to factor in the effect of the preempts on other hands - that is on either Woolsey and Stewarts bidding on other hands or on the opponents bidding or play on other hands? If so, how would you do it?

Has anyone studied the results from hands on which one pair had the methods to make a classic preempt at one table and the other did not (for example: Fantunes)?

Granted, the parameters for the Woolsey-Stewart preempts might be looser, but such a study might provide some interesting insights.

[FM75]

Is this about Woolsey?
If you are interested only in how Woolsey - Stewart play, I think your approach is fatally flawed. The reason for this is simple. You are measuring the results of strategy A versus the results of strategy B. For strategy A - their bidding - you can collect as many examples as you want. Unfortunately you will have zero, or nearly zero, examples of the alternative strategy, played by them. So you will have no measurable difference between the two strategies, since the second is only hypothetical. (The question of Strategy A vs. B is important to W-S, but I would expect that they have not measured it themselves, and that their opinion is subject to the typical flaws of confirmation bias and other biases associated with not actually recording and measuring results.)

Is this about ideal preemptive bidding?
I presume that you really want to measure something equivalent to "what is the break-even point in scoring" for various levels of preempts, presuming good play by both pairs at the table. For this, you could try using a singular value decomposition of all played hands that you can collect - tens to hundreds of thousands - which will include hands that could have been preempted but weren't, assuming that your set will include all results of each board in competition. Of course, you will want to include position, vulnerability and the form of the game as variables.

An alternative would be to generate your own data, similar to the approach chosen by Bird, Anthias in their recent publications. Generate random hands and assume that double dummy play will mirror actual results. I would recommend automation of the reporting, however. Their second book had many errors. In addition to automation of the reporting, full disclosure of the methods used would allow for independent study of the problem to confirm your results, as well as allow generating additional or fewer bidding systems, or even weighting of the frequency of other bidding systems. The difficulty of this approach is that you need to design competitive bidding strategies and apply them to the hands. You also have to include the results were preempts were made but did not win. Here, the double dummy assumption could potentially be a better mirror of real play, because the information provided by the preemptive bidding will probably more closely match double dummy results.

[CSGibson]

 FM75, on 2014-May-07, 17:02, said:

Is this about Woolsey?

Is this about ideal preemptive bidding?

This is mostly about how to approach the problem that was given to me, and using data to formulate a strategy about what is most effective in preemptive bidding, and when.

[mikeh]

I defer to Helene in terms of the math, but if we are going to limit the analysis to imp contests against their peers or near-peers, we are likely needing at least 20 years of results, which would be valid only if their methods and style has remained virtually constant over that period.

In any event, what would it prove to show, for example, that on balance they won or lost a modest number of imps per pre-empt?

We could compare their pre-empt imp gain/loss to their gain/loss on non-pre-empting hands to see if their pre-empting is a strong part of their successes, but that may be missing what I suspect may be the point about wide range pre-empting being a long term winner: a point that I doubt can ever be proven with statistical methods.

Where I think an aggressive style may pay dividends is that it makes the pair difficult to play against: they create difficult situations for their opps. They may wear their opps down. Anyone who has ever watched the last stages of a major event will know that many of the players, including multiple world champions, simply don't play as well as we would expect. Fatigue can cause lapses of attention: to be fair, these players at their worst are still far better than most of us at our best, but the point remains valid. Being tough to play against is a weapon in that sort of environment. Anyone who has followed meckwell knows this: their activity level, back in the 1980s and 1990s, was seen as hyper-aggressive at the time, and they often had huge late-match sets. Part of that was pure talent but, imo, part of it was that the opps kept getting worn down by the constant need to make decisions, in bidding and on defence, that meckwell's teammates were not facing in the other room.

[nige1]

 CSGibson, on 2014-May-07, 14:35, said:

I got this e-mail recently, and I thought I'd throw it out to the forums:
Chris, Some time ago, you and I had a discussion about the effectiveness of aggressive, variable preempts. In as much as you were on the opposite side from me, I think you are in a good position to comment on a methodology of studying the effectiveness of preempts. What are your thoughts on the following possible study. Look at Kit Woolsey and Fred Steward, a pretty strong nationally-rated pair who preempt as frequently as they can, on both good hands and bad hands, and who believe their overall results from undisciplined preempts are excellent. Consider a study of the results of their preempts in national KO's that examines only their results after they preempt at the 2 or 3 level against very good but not fabulous opponents [say round 32 and better of national KO's but omitting client pairs]. Compute all imps they win or lose against the result at the other table on their preempts. Weed nothing out even if there appears to be some kind of anomaly. Here are my questions:
1. How many matches would you want to examine before you would think the results were meaningful? How many preempts?

As many pre-empts as you can conveniently garner.

 CSGibson, on 2014-May-07, 14:35, said:

2. Would you want to find a way to factor in the effect of the preempts on other hands - that is on either Woolsey and Stewarts bidding on other hands or on the opponents bidding or play on other hands? If so, how would you do it?

No. Presumably there is a beneficial effect on other bidding because so many possible hands are excluded.

 CSGibson, on 2014-May-07, 14:35, said:

3. To what extent if any would you factor in the opinion Kit has often expressed that these bids have been a significant winner over time?

Let the results speak for themselves.

 CSGibson, on 2014-May-07, 14:35, said:

4. What do you think of the methodology? Would you suggest any changes or some other approach?

Consider only high level tournaments but include all hands where they pre-empt -- whether or not they pre-empt at the other table. Also don't exclude pre-empts against clients -- because a successful team has to do well against weaker teams -- and anyway random bids should offer less advantage against poor players.

 CSGibson, on 2014-May-07, 14:35, said:

5. What would you expect the results to show? How different would you think they would be if you considered the effect of the preempts against weaker national level pairs? What about against stronger ones?

Guess random pre-empts would be more effective against stronger players. But suck it and see.

 CSGibson, on 2014-May-07, 14:35, said:

6. Finally, what do you think are the biggest and most frequent gains Stewart and Woolsey realize on their preempts? What about the biggest and most frequent losses they suffer?

Dunno

 CSGibson, on 2014-May-07, 14:35, said:

I am asking you these questions only for perspective. I don't plan to renew our discussion though if you like, I would be willing to share a summary of my results.

Yes please.

[Zelandakh]

For #1 I think 200 is way too low. Even 1000 may well be too few to obtain meaningful results. I would want try to include (#2, #4) at least the hands on which second hand passed and 4th hand later entered the bidding. The reason for this is that the pass is much better defined in this style. You also need to factor in hands on which they made a simple overcall and a classic style would have done something different. The preempt style and the overcall style are related. The methodology is a reasonable starting point but will probably not be able to give a definitive answer, nor factor in everything. As you have pointed out it may be that the style does better against, for example, a minor suit opening than a 1M opening. More data points would allow for the data to be split up while retaining a reasonable volume for each section.

Opinions without data (#3) are not very trustworthy. Far better would be to discuss with Kit the types of hands where he feels there are gains and see if this suggests collecting additional data. I would expect to see (#5) an overall gain for the wide-ranging style, with the difference being greater against better pairs since W-S are already plus on average. Against much weaker pairs it could well be minus since, although they will make more mistakes, there is probably a form of randomising effect going on. The gains (#6) come from times when the opponents misguess, from finding the odd sacrifice more, in having more information on defence and from getting in and out quickly on hands where we do not want to declare. There is also likely to be a gain on constructive overcall hands since the bottom end of this range will often preempt instead. The bad hands are where they guess to reach a contract they would not have gotten to without the interference and it makes, from the additional inferences in the play, from Advancer misguessing and, naturally, from the odd penalty.

[winkle]

 mikeh, on 2014-May-07, 17:43, said:

Where I think an aggressive style may pay dividends is that it makes the pair difficult to play against: they create difficult situations for their opps. They may wear their opps down. Anyone who has ever watched the last stages of a major event will know that many of the players, including multiple world champions, simply don't play as well as we would expect. Fatigue can cause lapses of attention: to be fair, these players at their worst are still far better than most of us at our best, but the point remains valid. Being tough to play against is a weapon in that sort of environment. Anyone who has followed meckwell knows this: their activity level, back in the 1980s and 1990s, was seen as hyper-aggressive at the time, and they often had huge late-match sets. Part of that was pure talent but, imo, part of it was that the opps kept getting worn down by the constant need to make decisions, in bidding and on defense, that meckwell's teammates were not facing in the other room.

Curiously, though Meckwell is aggressive in many ways I don't think aggressive preempts were ever a part of their style.

[mikeh]

 winkle, on 2014-May-08, 12:29, said:

Curiously, though Meckwell is aggressive in many ways I don't think aggressive preempts were ever a part of their style.

I wasn't referring to pre-empts as such...more to the general proposition that aggressive bidding, of all kinds, tends to impose unusual stress on the opps. For meckwell it might be always having to defend 23 hcp 3N contracts, with others, always having to wrestle with preempts

[Fluffy]

I disagree with finding only preempts in one table vs non preempts on the other. Their wide ranging preempts are suposed to leave them behind if partner is strong, but better placed if partner is fitless and broke.

[jogs]

I'm absolutely convinced against top level opponents undisciplined pre-empts is a winning strategy. Your partner doesn't know what you have. Therefore he can't tell opponents if they ask. Make it a guessing game. Confuses two of them and only one of you. I've had good results playing this style.

[CSGibson]

 jogs, on 2014-May-08, 16:28, said:

I'm absolutely convinced against top level opponents undisciplined pre-empts is a winning strategy. Your partner doesn't know what you have. Therefore he can't tell opponents if they ask. Make it a guessing game. Confuses two of them and only one of you. I've had good results playing this style.

Not the point of this thread. The point of this thread is how to measure the effects of preempts against top level competition.

[FM75]

 CSGibson, on 2014-May-08, 16:51, said:

Not the point of this thread. The point of this thread is how to measure the effects of preempts against top level competition.

If the point is to measure against top level opposition - and not a particular (top) pair's results against top level competition - then I recommend my second suggestion of simulation with DD results considering both vulnerability and form of competition. Top level pairs will most likely play closer to DD than random pairs.

[PhilKing]

As many of you know, I have a searchable database of a bit less than 20K hands form top-level play. I am pretty certain this is not remotely large enough to provide an answer to the questions posed. When I get it up to 100k (and with more functionality) I will start looking at such things, and see if the data pool is large enough. Any methodology that suggests the use of DD analysis is, in my opinion worse than useless - preempts are all about pressure, and even world-class pairs underperform in this area - no pair can reach the par contract against a preempt without the use of coughing and sneezing.

I agree with the answers that point out that looking at whether (say) Woolsey/Stewart show positive results with an aggressive strategy are almost irrelevant for several reasons.

[awm]

It seems to me that even a small study would be useful. If Woolsey-Stewart results on preempts are significantly positive (and better than their average) it seems like a strong signal.

Of course there will also be some effect on the hands where they don't preempt (hard to measure) and some noise (can be bounded statistically) so a near-zero result won't tell us much.

[bluecalm]

Maybe it would be useful to email Richard Pavlicek so he could run something along the lines of "preempt with 6carder at one table, pass at the other" on his database.
He did similar studies already, link to one: http://www.rpbridge.net/9x13.htm

[Zelandakh]

 bluecalm, on 2014-May-08, 21:03, said:

Maybe it would be useful to email Richard Pavlicek so he could run something along the lines of "preempt with 6carder at one table, pass at the other" on his database.

But it is not always a pass at the other table is it? Plenty of the hands suitable for an undisciplined preempt become simple overcalls in a disciplined style. You have to measure both the impact of the preempt versus the simple overcall on these hands and also on the other hands where the undisciplined pair overcall and thus have a better defined auction. Since this auction may well be constructive, this is not negligible. One of the key things I like about the style, particularly for I/A players, is that it can allow for a quicker differentiation between constructive and non-constructive auctions than a more traditional style.

[whereagles]

It might be possible to run a few sims on preempt style, but finding appropriate constraints on the hands and evaluating outcome might be hard.

Still, if done the right way, you could get the statistics you want in a few mins.