[Cthulhu D]

Any conventional bid has a cost - in oppotunity and memory strain mostly. How do people ascertain whether the things they play are worth the cost? I track whether conventions win/lose/change nothing in an attempt to see if I can improve the system and cut any pointless gadgets but most things are sufficently infrequent to make analysis hopeless.

How do other pairs decide what to keel and what to cut?

[Codo]

Just feelings. You see a gap. You try to fix the hole and hope that the downsides are small enough.

[han]

Whether the memory strain of a certain convention is a loss depends for a large part on (1) how well you remember system, (2) how much you play together, and (3) how much effort you spend studying your system. It seems to me that a serious partnership should be able to always know their system. However, Bocci-Madala in the semi-finals of the Bermuda Bowl didn't know what the constructive auction 1D - 1H - 1S - 2C meant. Madala tried to sign off in 2D, Bocci thought it was fourth suit forcing, the dust settled in 5D, doubled by de Wijs.

If such a strong and serious pair cannot remember such a basic agreement, perhaps most of us would do better to give up on Stayman.

[Cthulhu D]

That is pretty funny.

Ultimately I am not hugely worried about memory strain though. Opportunity is a larger factor. However, Even identifying whether basic things are good/bad is hard. For example currently I play 2C: weak diamonds or any strong. This combine should be-EV on both strong and weak hands (winning only on hands where our 2D replacement is opened) but yet it's hard to see it is -EV at all.

That's a first round treatment to, trying to work out whether 1M-3M as 8-10 4 card support is a winner is difficult- we're lucky if this comes up once a month, and we play a teams night, every tournament in town and on BBO. Getting a meaningful sample size together is going to take a long time

[han]

Cthulhu D, on 2012-June-11, 09:00, said:

This combine should be-EV on both strong and weak hands (winning only on hands where our 2D replacement is opened) but yet it's hard to see it is -EV at all.

It is not at all clear whether this is -EV on either. What defense are the opponents going to play? Given the frequency they will probably assume that your opening is weak with diamonds, so they will only interfere with decent hands. That's nice when you are strong. You lose a little though because partner doesn't know that you are strong, and some of your responses (for example 2NT and 3D) will be geared to the weak hands. When you are strong partner will be forced to bid 2D more often, which is a small loss.

If you compare the weak hands with opening a weak 2D then probably you are going to lose on them. Partner can preempt less well (because you might be strong), and the opponents have more room and time. The upside is that partner will declare diamonds more often, but this is a very small upside. However, this is not a fair comparison, because probably when you play this opening bid you use the 2D for something else. The comparison should therefore be between opening 2C on those hands, or either passing/bidding 3D on those hands. Given that both are still possible when you play this convention, it seems to me that playing this convention you will do better on those weak hands.

[han]

Cthulhu D, on 2012-June-11, 09:00, said:

Trying to work out whether 1M-3M as 8-10 4 card support is a winner is difficult- we're lucky if this comes up once a month, and we play a teams night, every tournament in town and on BBO. Getting a meaningful sample size together is going to take a long time

Yes, trying to judge the merits of such an isolated treatment is difficult. You basically cannot know for sure even when it has come up a fair number of times, since it is so hard to judge what would have happened had you not played this convention.

Some systems or treatments are easier to judge. For example, many pairs have some way to play mixed raises in competition. They come up after 1M openings and overcalls, and (depending on how or when you play them) both after the opponents bid or double. So, you quickly get a feeling for them.

For example, you will know:

- whether the convention was easy to remember and recognize by partner.
- Whether it was pleasant for you to have the convention available (a hand might have been uncomfortable had you not).
- Whether partner had a hard time making a decision after you used the convention.
- Whether the auction made it easier or more difficult for the opponents.

Other examples of treatments that come up often are 2/1 gameforcing, different kinds of checkback, relay systems, 1NT responses, etc. These auctions come up often, and you will quickly form an opinion regarding the above questions.

[Fluffy]

You think an agreement is going to show up, then you stop playing with your partner for whatever reason, then you play again after some time and the fun begins.

[ggwhiz]

One pair I know used to run off 100 hands within the parameters of a new treatment, bid them together and check against all 4 hands for effectiveness. You could rate them plus, minus, neutral based on an existing treatment. Each one would peek at lho's cards to make interference bids.

They did this every year a couple of weeks before our team trials as a system/memory refresher too.

Some have no real answer. For me after a 1nt opener, we decided that 3 level responses as natural and invitational were not effective unless so narrowly defined that they never happen. We've tried a bunch of other things and they never come up either.

[JLOGIC]

You must judge effectiveness in most cases on a theoretical/intellectual basis. Clarify what you are gaining and what you are losing (be objectively honest with yourself) before you even start playing a convention. If you can boil it down to I am gaining this but losing this, it is often obvious whether you should play a convention, you have played enough bridge to have formed views on what you think is important. Sometimes it is not clear, in which case I would talk to others for their opinion or if it is a somewhat known convention, I would see if most people I respect play it or not. After playing it for a while, you might change your view, but don't be results oriented. If it is not clear theoretically/intellectually if you should be playing something, and if it is also pretty split whether people you respect like to play it or not, then congrats, it probably does not matter whether you play it and is roughly neutral.

Also, if you were going to attempt to track conventions (which I agree with you is useless because of sample size issues, as well as what han said, it's hard to predict what would have happened had you not done something, it is very artificial and they will never make any mistakes nor will you) then don't forget about negative inferences etc.

At some point, you want your conventions to fit in to your system as a whole, rather than just be patchwork, but it depends on what kind of convention youre talking about. It is ok to play some conventions that make your system better as a whole but may not be that great individually. I would also encourage you to play conventions that fit into your overall style and philosophy of the game. I like strong club because I can open lighter and bash more, I like non forcing new suits to my overcalls because I like to overcall light and get my suits in early, my general style is about getting in early and at a low level until a fit is found, and my preferred conventions reflect that.

[mycroft]

If you do do that, you need to also run several hands with the "would have bid the call I can't make now" parameters and see how that ties into your -EV. Sometimes "there ain't no such hands", no problem. Sometimes there are...

[nigel_k]

What Justin said. You do it mostly through analysis, not by extensive testing, and many improvements are obvious.

If you do want to test effectiveness of a treatment, I would not do it based on records of what happened when it came up at the table because there are too many other variables involved (including the two variables that sit on your left and right). Instead, generate random hands and bid them. This has the added advantage of showing up any weaknesses before they hurt you at the table. You can even compare two treatments by bidding random hands twice, using each one, like a teams match.

[hrothgar]

The following is not particularly scientific and is only viable for conventions / treatments that occur fairly frequently.

Start by establishing your baseline: How well do you "normally" play.

Next, look at individuals sessions that depart from the norm.

For example, this session we did especially well.
This session we did very poorly.
(Throw out sessions that we obviously unrelated to bridge. For example, the one where you were drunk or sober as the case may be)

Finally try to determine whether your results were attributable to a specific convention or treatment.
For example, this session had an unusually high number of strong club opening AND our score was much lower than usual.
Conversely, this session had an unusually high number of 2D openings and out score was much better than usual.

[gnasher]

I tried keeping track of system gains and losses for a while. The main thing I noticed was that most of the losses (and some of the gains) came from forgetting the system.

There seemed to be lots of times when the auction felt more comfortable because we had good methods, but it didn't actually lead to a swing. For example, they'd get to the same contract at the other table but with less certainty, or they'd get to a slightly inferior contract that also made, or we'd gain but the proximate cause was something teammates did.

Having an easy auction is also a gain, because it lets you use your time and energy on other problems, but quantifying this gain is just as hard as quantifying the loss from having to remember the system. However, there's some correlation between having an easy auction and playing methods that you like, so maybe you should just play whatever methods you want to. That's what most of us end up doing anyway, regardless of how we rationalise it.

[Cthulhu D]

gnasher, on 2012-June-11, 16:16, said:

I tried keeping track of system gains and losses for a while. The main thing I noticed was that most of the losses (and some of the gains) came from forgetting the system.

There seemed to be lots of times when the auction felt more comfortable because we had good methods, but it didn't actually lead to a swing. For example, they'd get to the same contract at the other table but with less certainty, or they'd get to a slightly inferior contract that also made, or we'd gain but the proximate cause was something teammates did.

Having an easy auction is also a gain, because it lest you use your time and energy on other problems, but quantifying this gain is just as hard as quantifying the loss from having to remember the system. However, there's some correlation between having an easy auction and playing methods that you like, so maybe you should just play whatever methods you want to. That's what most of us end up doing anyway, regardless of how we rationalise it.

Yeah, the biggest (only?) differences I have noticed are:

A) Forgetting the system is a leading cause of swings out
B) The biggest source of swings in is opening something, anything, totally irrespective of what it is and why you opened it this week, when the other table didn't.
C) Playing highly frequent unusual stuff leads to occasional big wins as people are not on firm defensive ground.

Aside from a couple of opening pre-empts - e.g. Ekrens 2H - which benefit very strongly from B and C, I honestly cannot see a difference with most stuff.

Ultimately I'm leaning towards your conclusion, I just wanted to see if anyone did anything better.

hrothgar said:

Start by establishing your baseline: How well do you "normally" play.

This is part of the problem I have with my current analysis attempts. We started playing only 9 months ago and have had a gigantic leap forward in results over that time. While the system has taken a leap forward over the same period (We've gone from playing 'standard' to playing 2/1 game forcing, transfer Walsh, 14-16NT and friends), it's really because our bidding judgement and card play have taken a quantum leap forward.