By changing “Space between items” do you enter a numeric value or “auto” (in advanced layout settings it’s “Spacing mode › Space between”)?
“Auto” is what you need.
This can also be achieved like this: wrap the center part in a frame and set the “Fill container” horizontally, and then change the constraints on the objects inside that frame.