Lazy eval on pv nodes

[benstoker]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

[Sentinel]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

You can't gain more than 2-3% of speed by applying lazy eval on PV nodes. An you can loose much more than 2-3 elo on that. Bad idea in general.

[hyatt]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

I do it everywhere, always have, and don't see any reason to not do it on PV nodes only...

People find the ugliest solutions to programming problems. Don't do exact hashes on PV nodes. Don't hash mate scores, convert them into bounds or don't store them at all. Etc. Find the easy way out by just not doing something, rather than figuring out how to do it and make it work...

[hyatt]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

You can't gain more than 2-3% of speed by applying lazy eval on PV nodes. An you can loose much more than 2-3 elo on that. Bad idea in general.

Based on what? I tested this nonsensical idea a while back. Doing it everywhere is better than in selected places, if it is done right.

[Richard Vida]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

You can't gain more than 2-3% of speed by applying lazy eval on PV nodes. An you can loose much more than 2-3 elo on that. Bad idea in general.

If you do it right, and it indeed does help in non-pv nodes, I see no reason why it would hurt along the PV (but maybe it is engine-specific).

Would you please elaborate more on theory why it would made a difference doing it only in non-pv nodes vs doing it everywhere? Did you run any tests?

[Sentinel]

You can't gain more than 2-3% of speed by applying lazy eval on PV nodes. An you can loose much more than 2-3 elo on that. Bad idea in general.

If you do it right, and it indeed does help in non-pv nodes, I see no reason why it would hurt along the PV (but maybe it is engine-specific).

Would you please elaborate more on theory why it would made a difference doing it only in non-pv nodes vs doing it everywhere? Did you run any tests?

If you do it right, indeed. The problem is, chances are quite high you won't do it right. It's all about the margin for lazy eval. I don't know how much experience you have with tuning it but what I can claim is that it's certainly not an easy task. The dependence (of strength on the margin size) is neither simple nor monotonic. The curve has a lot of local maxima and finding the best one is quite difficult.
The thing is if you setup your margin too low, it will hurt you much more in PV nodes than in non-PV. On the other hand the total benefit in speed is 2-3% (at least what I've tested) which is in the best case 2 Elo and practically unmeasurable even with Bob's cluster.
So, correctly speaking you can't know if it helps or hurts, but you can know if you don't do it right that it will hurt.

[hyatt]

You can't gain more than 2-3% of speed by applying lazy eval on PV nodes. An you can loose much more than 2-3 elo on that. Bad idea in general.

If you do it right, and it indeed does help in non-pv nodes, I see no reason why it would hurt along the PV (but maybe it is engine-specific).

Would you please elaborate more on theory why it would made a difference doing it only in non-pv nodes vs doing it everywhere? Did you run any tests?

If you do it right, indeed. The problem is, chances are quite high you won't do it right. It's all about the margin for lazy eval. I don't know how much experience you have with tuning it but what I can claim is that it's certainly not an easy task. The dependence (of strength on the margin size) is neither simple nor monotonic. The curve has a lot of local maxima and finding the best one is quite difficult.
The thing is if you setup your margin too low, it will hurt you much more in PV nodes than in non-PV. On the other hand the total benefit in speed is 2-3% (at least what I've tested) which is in the best case 2 Elo and practically unmeasurable even with Bob's cluster.
So, correctly speaking you can't know if it helps or hurts, but you can know if you don't do it right that it will hurt.

I found it easy. Run enough games to measure Elo accurately, and then tune the value up and down to reach the max point... I didn't find "lots of local maxima" points. Just one. I don't see why there would be many. If you are too conservative, you don't get the speed gain and slowing down too much hurts. If you are too optimistic your eval becomes more unstable and it hurts.

[marcelk]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

Since evals in pv nodes are typically within the search window, a lazy eval there does the same as a full eval.
Lazy makes sense when you are (far) outside the window. I doubt you can measure a difference.
Besides, pv nodes are extremely rare.

[Sentinel]

I found it easy. Run enough games to measure Elo accurately, and then tune the value up and down to reach the max point... I didn't find "lots of local maxima" points. Just one. I don't see why there would be many. If you are too conservative, you don't get the speed gain and slowing down too much hurts. If you are too optimistic your eval becomes more unstable and it hurts.

So how much is enough games that you ran when you've concluded that doing lazy eval everywhere is better than just doing it in non-pv nodes???
Could you tell us that please?

[hyatt]

Elsewhere I read that Houdini improves upon ippo* in part by implementing lazy eval on pv nodes rather than just on cut and all. It doesn't seem that that would be very difficult to implement in Ivanhoe's pv_node.c.

Do other engines typically do lazy on pv nodes? Why would that or how could that speed up search? Would this be beneficial for all engines or is it something that could hurt some engines and help others because it's nonlinear? If you put lazy on pv nodes do you need to change ordering?

Just trying to understand these things ...

Since evals in pv nodes are typically within the search window, a lazy eval there does the same as a full eval.
Lazy makes sense when you are (far) outside the window. I doubt you can measure a difference.
Besides, pv nodes are extremely rare.

Not even close. I use an aspiration window, so that beta - alpha is < 1.00 unless I see a fail high or a fail low. The only time your statement would be correct in Crafty would be if the first move failed low several times to drop the aspiration alpha to -inf, and then another move failed high repeatedly to raise aspiration beta to +inf. Normally they are "close" and lazy eval pays off big.

They are not as rare as you think. Remember, for CUT nodes, 90% of the time we get 'em right, but 10% we don't. That 10% introduces a lot of hidden PV nodes.