On Drag Interaction Simply not working

Im just trying to make a simple Interaction where a user can select a certain time by dragging on two sliders.
Im using a component with variants for all relevant states and the on drag interaction to switch between them. I dont want to implement simple scrolling, because I want the interaction to be snappy and to know what state the component currently is in.
But it simply isnt working. While the interaction does show up (I see the blue border marking a possible interaction) dragging on it does nothing. I tried dragging into all directions, I tried checking the prototyping, I even tried slightly different ways of implementing it, nothing helped.
This is even more nonsensical because the very first On Drag Change To Interaction for the component works flawlessly, but after that its stuck and will no longer receive the On Drag input.
Im clueless as to why this is what it is.
This what the component actually looks like (its fairley simple), there is another row of dials both above and below the cut off point for the sliders, that currently arent visible because they are being clipped by the frame the sliders are held in. I added those just for smart animate.
The Sliders are just text instances in a group selection.

Actually I don’t know how your variants are arranged, but:

Or in other words, the content inside your variant must exceed the height of that variant.

Also, in order not to produce so many variants, you can use variables. See an example here: Drag direction up and down using variables and conditionals - #4 by tank666.

The dragable groups do extend beyond the frame that is outer boundary of the component variants. (I deactivated No Clipping on the Frame for the second image)
I also copied the first variant to create the later ones, so if it works on the first one, it should work on the later ones. All I changed between variants was the text.
I cant explain to myself why its being so weird.

As far as I can see from your screenshots, the dragged object does not change its coordinates in each variant.

This helped me figure out the problem, thanks.
Sadly the slider feels weird to use anyway, because you have to drag individually for each step of the slider, until you reached the desired slider state.
I might have to find a entirely different way to implement the selection.