Considering the previous scene he was mackin all over Cats daughter, and he had JUST called her Cat and even if you have only been half-ass paying attention to the story like I do you KNOW that Littlefinger is utterly obsessed with the Lady Katelyn Stark. It's a defining motivation for him in a couple of his appearances, and he goes into some detail about how much he wanted to bone her and never did. I think it would have been perfectly fine to say "Only Cat". Plus the line would have sounded better finishing with, "Only Cat" rather than "Your Sister".
"All my life I have loved only one woman. My entire life. Only one. Only Cat."
-----------------------------------------------------------------Your Sister."
That first is just a stronger spoken line. I wonder if it was an accidental adlib and they just didn't fix it, because whatever reason.