Stockfish's time issue: What to do
Posted: Sun Oct 11, 2015 3:59 am
In this thread, I invite all those who like to share their view in the chat, to express their opinions in a more permanent medium. Feel free to recycle your best arguments from the chat into this thread.
Not long after the first loss on time, I became convinced that the only correct interpretation of the rules was that no Stockfish patch or version reversion could be allowed. Now, I am not that sure. The section says
Critical Engine Bugs
In the case of a serious, play-limiting bug (like crashing or interface communication problems) not discovered during the pre-Season testing, the engine can be updated once per Stage to fix this/these bug/bugs only. If this update still doesn't fix the problem(s) or if there is no update available, the engine might have the number of cores reduced, have the hash size reduced or have the tablebase access disabled - these changes will remain for the rest of the Stage.
I just noted that "crashing or interface communication problems" are merely examples in the wording, they are no definition, and thus "play-limiting bugs" may not be limited to those and could, viewed in isolation, include a bug leading to a time loss, at least when this almost surely has happened due to the latest version update. The other time losses in stage 2 and 1 are not of the same kind, as far as I know: The engines with these losses have had these issues previously, and there were no replacement ready as in SF's case (i.e. the version in stage 2), unless possibly some much older, weaker, version.
This said, even if there is no alternative, I still find it a bit unfair and far from satisfying to allow a correcting patch for Stockfish. After all, it was a gamble, as Martin rightly pointed out in is video, to allow a not very much tested SF version with the Lazy SMP algorithm participate. Somehow, it would be right for Stockfish to learn the lesson that the version participating in TCEC should be much more thoroughly tested. Still, it might be enough of a punishment to let the current results stand, while allowing just a correcting patch or even the version from stage 2. For entertainment purposes, the former alternative seems best, and is certainly my preference. I would have taken another view if the tournament was shorter, but since a non-intervention could mean weeks and months of unnecessary complications, a judgemental intervention should be considered valid.
One note of a more principal legal-philosophical character: A set of rules, like the rules of a game, a tournament, or those governing a country in a constitution, often fail to take into account every scenario that will occur. This is only natural, and a result of human imperfection. In this case, one could argue that Martin's interpretation of the above-quoted section of the rules in previous stages has set a precedent for this scenario, notwithstanding the distinctive characteristics of this particular case. The element of this scenario which TCEC's rules fail to take into account, is, though, that a continued stage 3 would be a huge threat to TCEC itself. Stage 3 would become very arbitrary, and I believe that both spectators and engine authors would agree that a continuation for the sake of the letter of the rules would defeat TCEC's main purpose: entertainment. Such a threat could warrant the use of what in constitutional law is known as constitutional necessity: In a situation of an emergency, branches of government might sidestep the constitution in doing an action, including the action of amending the constitution. To give an example Martin will be familiar with: In 1814, when Norway had got its own constitution, but before it entered the personal union with Sweden the same year, it saw the need to have a constiutional amendment, for instance allowing a Swedish king. In order to do this by the written procedure of the constitution, there would have to be an election beforehand, which there was no time for and would in any case have been unconstiutional itself. Therefore, the only solution was to sidestep the constitution and allow the parliament to amend it without a prior election - since any other alternative would clearly have defeated the purpose of the constitution itself, the independence of the Norwegian people.
Being a constitutionalist myself, the use of this measure is not something I take lightly, but I acknowledge that it is necessary in extreme situations. In TCEC's case, I think there is a strong argument at least for some action to be taken, even if it would imply side-stepping the rules.
Not long after the first loss on time, I became convinced that the only correct interpretation of the rules was that no Stockfish patch or version reversion could be allowed. Now, I am not that sure. The section says
Critical Engine Bugs
In the case of a serious, play-limiting bug (like crashing or interface communication problems) not discovered during the pre-Season testing, the engine can be updated once per Stage to fix this/these bug/bugs only. If this update still doesn't fix the problem(s) or if there is no update available, the engine might have the number of cores reduced, have the hash size reduced or have the tablebase access disabled - these changes will remain for the rest of the Stage.
I just noted that "crashing or interface communication problems" are merely examples in the wording, they are no definition, and thus "play-limiting bugs" may not be limited to those and could, viewed in isolation, include a bug leading to a time loss, at least when this almost surely has happened due to the latest version update. The other time losses in stage 2 and 1 are not of the same kind, as far as I know: The engines with these losses have had these issues previously, and there were no replacement ready as in SF's case (i.e. the version in stage 2), unless possibly some much older, weaker, version.
This said, even if there is no alternative, I still find it a bit unfair and far from satisfying to allow a correcting patch for Stockfish. After all, it was a gamble, as Martin rightly pointed out in is video, to allow a not very much tested SF version with the Lazy SMP algorithm participate. Somehow, it would be right for Stockfish to learn the lesson that the version participating in TCEC should be much more thoroughly tested. Still, it might be enough of a punishment to let the current results stand, while allowing just a correcting patch or even the version from stage 2. For entertainment purposes, the former alternative seems best, and is certainly my preference. I would have taken another view if the tournament was shorter, but since a non-intervention could mean weeks and months of unnecessary complications, a judgemental intervention should be considered valid.
One note of a more principal legal-philosophical character: A set of rules, like the rules of a game, a tournament, or those governing a country in a constitution, often fail to take into account every scenario that will occur. This is only natural, and a result of human imperfection. In this case, one could argue that Martin's interpretation of the above-quoted section of the rules in previous stages has set a precedent for this scenario, notwithstanding the distinctive characteristics of this particular case. The element of this scenario which TCEC's rules fail to take into account, is, though, that a continued stage 3 would be a huge threat to TCEC itself. Stage 3 would become very arbitrary, and I believe that both spectators and engine authors would agree that a continuation for the sake of the letter of the rules would defeat TCEC's main purpose: entertainment. Such a threat could warrant the use of what in constitutional law is known as constitutional necessity: In a situation of an emergency, branches of government might sidestep the constitution in doing an action, including the action of amending the constitution. To give an example Martin will be familiar with: In 1814, when Norway had got its own constitution, but before it entered the personal union with Sweden the same year, it saw the need to have a constiutional amendment, for instance allowing a Swedish king. In order to do this by the written procedure of the constitution, there would have to be an election beforehand, which there was no time for and would in any case have been unconstiutional itself. Therefore, the only solution was to sidestep the constitution and allow the parliament to amend it without a prior election - since any other alternative would clearly have defeated the purpose of the constitution itself, the independence of the Norwegian people.
Being a constitutionalist myself, the use of this measure is not something I take lightly, but I acknowledge that it is necessary in extreme situations. In TCEC's case, I think there is a strong argument at least for some action to be taken, even if it would imply side-stepping the rules.