I personally prefer it this way, because the cursor is returned to the panel to make further adjustments, without needing to mouse over back to it. It’s like a slingshot, snapping you back to where you started.
As people tend to look at the counter, rather than the cursor, it makes sense that the cursor returns to your FOV after using it.
Maybe this could be a made user preference instead.