How to change text fill via plugin API

Hello world, I am working on a Figma plugin where one of its features is changing fill of selected elements into color variable if the said fill is found within the variables. Everything else is straight forward but when it comes to text, since different characters within one text element could have different colors, I tried to access them using get StyledTextSegments like so:

const segments = element.getStyledTextSegments(['fills'])

The issue I am currently facing is that fill within textSegments is readonly, So the following code does not work; specifically the part of assigning segment.fills new value:

segments.forEach((segment) => {
    const fillsCopy = [...segment.fills]

    if (fillsCopy[index].type === 'SOLID') {
        fillsCopy[index] = figma.variables.setBoundVariableForPaint(
            fillsCopy[index],
            'color',
            matchResult.variable
          )

          segment.fills = fillsCopy
}

Is there another way I can make this work?