Why do I have these orange lines appear on the border?

See screenshot… I feel like I accidentally hit some command that caused it to happen.

Do you mean guides? You can hide the guides and rulers (Main menuViewRulers or the Shift + R shortcut) or remove the guides by dragging them onto the rulers or using plugins to remove guides.