Using hierarchical clustering to generate a bidding system
#21
Posted 2023-April-28, 02:41
Any thoughts about what to do about this? Allowing lower level bids to be defined as a merger of two subtrees rather than a single subtree would solve it, but that would make the definitions of the lower level bids even more complex. It already takes about 50 boolean operations or so to define the 1♣ opening.
Csaba suggested using penalized regression to force it to simplify the lower level openings. That would require a custom agglomeration criterion for the hierachical clustering. I don't think that is supported by any R package, but maybe there's something like that in one of the machine learning packages for Python? Custom agglomeration would be good to have for auction termination also.
#22
Posted 2023-April-28, 13:23
1♠-2♣ (balanced or clubs, GF)
What is a bit weird is that it doesn't ask explicitly about major length at all although I suppose that could partially be an artefact of the way the tree is presented which may be biased by the minor suits coming first in the variable listings. By the way, if I remove the slam bonus, as I did for the opening scheme (where I thought it was a bit pointless to cluster opening hands on slam potential, and slams were rare so they created some annoying statistical flukes), it does represent spade length and heart shortness explicitly in the tree.
I suppose I could put a bit more info into the training data now that I have fewer hands to begin with. A binary (many/few) vasted points in a suit opposite a singleton would just be managable in this context.
#24
Posted 2023-April-28, 15:09
I agree, when both opponents have passed and you're in a GF, great time to pull out all the stops. But twice last night they interfered second round in our GF auction (once they were insane (and partner didn't lead the suit. Guess he's learned already?) and the other time it wasn't the right thing to do (and partner did lead the suit)). So, can't guarantee anything there either.
I love the idea, but I also have Chthonic's "1♣ shows these 1,487,826,470 hands..." running through my brain :-)
#25
Posted 2023-April-28, 15:12
But there's something wrong with the labeling since 3H is defined while 2S is not which isn't true.
#26
Posted 2023-April-28, 15:44
- 4(+)♣, 16(+) hcp and 2(+)♦, or
- 3(-)♣ (so 2-3), 1(-)♦ and 15(-) hcp, or
- 3(-)♣ (so 2-3), 2-3♦ and 20(+) hcp.
#27
Posted 2023-April-30, 13:28
helene_t, on 2023-April-27, 13:00, said:
Pass: Less than 14 points, club length
1c: 8-17 points
1d: 10+ points, spade length
1h: 14-21 points, diamond length
1s: 14-17 points, short clubs
1N: 18-21 points, both majors
2c: 20+ points, short clubs
2d: 22+ points, short clubs
2h: 22+ points, [4261]
2s: 22+ points, [2461]
2N: 22+ points, [3361]
It is a bit weird with all those bids showing club shortage, while the strong hands with long clubs being scatered randomly. Hopefully a system optimized to auctions where opps are going to interfere will be more inteligeble as you then can't rely on plenty of bidding space to resolve your shape later.
I must admit my enthusiasm for this approach is cooled a bit. In the old thread, since I optimized the openings to partner giving a punt immediately, the openings tended to be quite descriptive, but here it's like each opening bid is just a random pile of stuff that will be resolved later.
Could you give a list of responses to your one-level openings?
#28
Posted 2023-April-30, 13:42
helene_t, on 2023-April-21, 03:13, said:
What could easily be done is replacing HCPs and suit length with some estimate of the hand's playing strength in each of the denominations, that wouldn't necesarilly be higher dimensional than what I do now.
What I could also do is to switch to a different model once a GF has been established and a fit found. The we would have more parameters but fewer hands possible.
Clustering responder's hands should depend on what is known about opener's hand. Opposite a standardish 1♠ opener it makes a great difference whether responder has 2 or 3 spades; if he has 3 or more spades it's not that important whether he has 3 or 6 diamonds.
#29
Posted 2023-April-30, 14:14
dokoko, on 2023-April-30, 13:42, said:
This may be a bit confusing, but there are two stages of clustering.
The first clustering (which was the one you responded to) is the a priori reduction of the number of hand types to 1860, which is about the maximum I can manage with my laptop. This step I didn't find so interesting so I just did something simple.
The second clustering, which is the core of the idea (the hierarchical clustering) is based on how important it is for partner to distinguish two hands. Indeed, if partner has shown 5 spades and you have three it doesn't matter much what you have in the other suits (except that shortness can be useful).
#30
Posted 2023-April-30, 22:53
helene_t, on 2023-April-27, 13:00, said:
Pass: Less than 14 points, club length
1c: 8-17 points
1d: 10+ points, spade length
1h: 14-21 points, diamond length
1s: 14-17 points, short clubs
1N: 18-21 points, both majors
2c: 20+ points, short clubs
2d: 22+ points, short clubs
2h: 22+ points, [4261]
2s: 22+ points, [2461]
2N: 22+ points, [3361]
It is a bit weird with all those bids showing club shortage, while the strong hands with long clubs being scatered randomly. Hopefully a system optimized to auctions where opps are going to interfere will be more inteligeble as you then can't rely on plenty of bidding space to resolve your shape later.
I must admit my enthusiasm for this approach is cooled a bit. In the old thread, since I optimized the openings to partner giving a punt immediately, the openings tended to be quite descriptive, but here it's like each opening bid is just a random pile of stuff that will be resolved later.
A caveat before you read this. I understand practically nothing of programming or machine learning. I had a thought that I want to share (in the unlikely event it helps).
Say we load a million bridge deals where South is dealer and we include the raw scores that can be achieved for each deal (using some DD solver).
If the first step (or node?) is to get your program to bunch all South hands into "neutral", "constructive" and "obstructive" before the first pass of your program, will it help with the development of the hierarchy?
#31
Posted 2023-May-01, 06:42
helene_t, on 2023-April-27, 13:00, said:
1c: 8-17 points
1d: 10+ points, spade length
1h: 14-21 points, diamond length
1s: 14-17 points, short clubs
1N: 18-21 points, both majors
2c: 20+ points, short clubs
2d: 22+ points, short clubs
2h: 22+ points, [4261]
2s: 22+ points, [2461]
2N: 22+ points, [3361]
What are you opening with:-
All balanced hands
0-7 any
15+ with club length
0-9 without club length
10-13 without club or spade length
22+ with both minors
All hands with heart length
Presumably the undisclosed 1♣ will cover some of these but it is in compatible with most of them. It does look a lot like your iterative process got stuck in a local maximum rather than having a broad enough basis to find an optimal maximum. This is a common issue for such schemes.
#32
Posted 2023-May-05, 20:15
helene_t, on 2023-April-22, 05:16, said:
I don't think such a system has ever been used by humans at all. There was one reasonably playable AI system published a few years ago but the problem is that even if such a system would be technically great, it might be impossible to memorize, and it may be illegal.
thx - only useful and meaningful to another similar robot
I am generally very interested in clustering and thematic analysis of text, and dimension reduction, and finding meaningful descriptors for things
I am finding reading about relay fascinating from a hypothetical standpoint but it would overload my small brain capacity
My IT is probably even more rusty. I was hoping I may learn something
#33
Posted 2023-May-06, 03:16
thepossum, on 2023-May-05, 20:15, said:
In a similar vein I got a lot of wide-eyed looks when I played Precision for a while. We played a really simple version, but our opponents would expect some top of the line airtight structure. I think the common factor is that they're outside the comfort zone for a lot of players, nothing more.
#34
Posted 2023-May-06, 04:46
DavidKok, on 2023-May-06, 03:16, said:
My experience of shape-showing relays was a significant advantage over natural but a level of commitment to memorisation above what most club players are willing to do, particularly when it comes to the scans above the actual shape relays.
#35
Posted 2023-May-06, 05:00
I do like scans and denial cue bids, but they come with a host of other questions (if you have shown your shape in standard, why not play them there? Plenty of XYZ sequences have complete shape and strength descriptions, as do certain 2/1 and reverse auctions. Denial bids or control asking bids can also be more descriptive after preemptive or jump auctions. Also this is primarily relevant for slam territory, and slam simply isn't on very often).
Edit: I got a bit curious again, so I pulled up the last 96 boards on vugraph by Woolsey - Bramley (since I think they play KK Relay over 1C). They had 2 relay auctions, both of which ended in 4M making 11 tricks (identical to the other table). Incidentally they also missed a grand over a 1♣-1♦ start that was found at the other table after a 1♣-1♠ start - presumably an upgrade in a strong club system?
#36
Posted 2023-May-12, 05:00
DavidKok, on 2023-May-06, 05:00, said:
I recall an interview with Simon de Wijs shortly after he and Bauke Muller made the switch from stone-age Dutch Acol to symmetric relay Precision.
He thought it technically didn't matter so much but the advantage was that is is so much easier, you just do the relay auction like a headless chicken (as they say in Dutch) and the only times you have to make judgement is when you decide on the final contract, and when you decide whether to go go past the game level.
I volunteered for a while in the Looier Bridge Club in Amsterdam where they teach a relay system to beginners. Now, this wasn't one of the good relay systems used by de Wijs/Muller and similar, it was just something weird which the owner of the club had made up. But it got beginners to the level where they could reach sensible contracts most of the time much faster than those beginners who were taught natural methods. The downside was that it sucked.
Anyway, I considered using a relay system for the exercise I started with in this thread. The main reason for not doing it is that it wouldn't be very original. There are lots of people who have toyed with relay systems, and the only thing I could add to that would be to make it truly fibonacci while maximizing some other benchmarks such as getting relayer to declare as often as possible, at the expense of making it much more difficult to memorize than normal SR. I just didn't find that very interesting.
#37
Posted 2023-May-12, 05:02
shyams, on 2023-April-30, 22:53, said:
Yes I think something like that would be a good next step for the project. So far it is strictly constructive, but at some point it needs to take tactical aspects into account also.
#38
Posted 2023-May-12, 07:01
helene_t, on 2023-May-12, 05:00, said:
He thought it technically didn't matter so much but the advantage was that is is so much easier, you just do the relay auction like a headless chicken (as they say in Dutch) and the only times you have to make judgement is when you decide on the final contract, and when you decide whether to go go past the game level.
#39
Posted 2023-May-12, 07:22
DavidKok, on 2023-May-12, 07:01, said:
Yes but it makes perfect sense to me (although I have very little experience with serious relay systems so I just take Simon's words for it).
As you said earlier, slam auction just don't come up that often. If you read books on the 2/1 system by for example Lawrence, an enormous number of pages are devoted to GF auctions where you may or may not belong in slam. Lawrence says himself that for tactical considerations and for partscores and invitational sequences, 2/1 is no better than old-fashioned 4cM, 2/1 pairs have to live from the slam auctions. So you end up devoting a ridicolously large part of your system discussions to slam auctions. You see that in the GIB system also, where GIB has no clue how forcing a 2/1 response is when system if off, i.e. by a passed hand or when the 2/1 response is a freebid. The system may be technically OK but the prioritisation of the attention of the system development is wrong.
Muller/de Wijs have a single paradigm for uncontested and mildly contested slam auctions, it may be highly artificial but it is simple because it is only one paradigm, one player just describes his hand, and the other just makes relays until decision time. So they have more time to discuss partscore hands, invitational sequences and contested auctions.
But OK, that wasn't really Simon's point IIRC. It was more about not having to spend mental energy on bidding so that you can be more alert during the card play.
#40
Posted 2023-May-12, 08:46
"Most people" do not understand relays - frankly, even getting some to deal with Kickback responses or 1♠-2NT; 3♣ "any minimum, then 3♦ says 'reply the same steps as if you had extras last round'" is very difficult. And while I have never played a full relay system, I am quite certain that the judgement required when you know "3361, any 9+, if I ask any more we've lost 3NT" is very different - but probably no more difficult - than that which we all use after a similar amount of information passing in approach forcing.
And of course, the only experience "most people" - even "most people who declaim about relay systems" have with understanding relay systems is playing against it, when the information comes one hand at a time, without the coherent skeleton.
So, of course they won't be "easier to use". For most people.
Add to that that there are just people for whom lots of memorization, especially with sensible guiding principles that can be used to regenerate the path at the table if necessary, is low-energy. I am one of them, at least I was 15 years ago. Those that aren't - especially those that will concentrate so hard on not forgetting the system that they don't have energy left to follow suit - are probably not suited to relays.
I haven't played a full relay system, but I have 15 years of experience with Precision, half of that being full asking bid Precision. I can totally understand Tarzan here, because I frequently said the main advantage of Precision is "the system thinks for me, at least one round farther than standard." Because it's automated that much further, there are so many auctions that are mindless. Yes, of course, the reason it's mindless is that a huge amount of memorization and practise was put in beforehand; and yes, of course, artificial forgets were much harder to recover from than "when you're just bidding suits", but that really didn't happen often. And eventually, you *save* thinking energy for when it is needed.
And I don't think you should really equate "it technically didn't matter so much" with "less effective". It sounds an awful lot more like "just as effective, in a 'swings and roundabouts' fashion" to me.