He killed those Kingsguard with his seven vs their three and almost died himself. Out of the 7, only himself and Howland Reed survived.
On that note, in books 2, one of Bran's chapters, Bran recalls Ned mentioning that Arthur Dayne would have killed him if not for Howland Reed. Cant say much more w/out spoilers. I'm not sure if it's mentioned if Ned had Ice in that battle or not, would be interesting to know.
Either way, I think its subtly hinted at in the books that Ned is a better tactician than fighter, his older bro was the beast. Ned is no pushover, but I imagine Jaime would think highly of Ned's skills considering he played a part in killing 3 of the kingdoms top alltime fighters. Nobody really knows how good a fighter Ned is because in the books we never see it first hand, while theres numerous references to Jaime's skill. One thought though, is if Ned uses Ice, and Jaime tries to block a strike, theres a very good chance of Ice breaking his sword and ruining Jaime's day.
Considering Jaime has a hard-on for Arthur Dayne, Im pretty sure he has respect for the skills of a man who took part in killing him. I can say for almost a certainty that Ned isint on Jaime's list. Jaime says theres 3 fighters in the kingdoms who can defeat him, so I would assume he means three alive people, and isint talking about legendary dead swordsman, otherwise there would be way more than 3 people on that list.
My 3 fighters who I think Jaime thinks are > than him:
1. Barristan. No question about it. Cant even be debated. Pure skills as a swordsman, more experience than anybody. Won numerous tourneys, melees, and countless heroic deeds and slaying of badasses, even at old age.
2. Gregor - Jaime states himself that hes inhumanely strong, and his sword has a much longer reach than Jaime's. Crazy amounts of battlefield expz, and Tyrion mentions he has great natural instincts and is a natural fighter. I would put Sandor in here too, and may give him the edge over Gregor, just because he is a badass, gives zero fucks, and he's a clever fighter. I forget where, but there's a part in the book where during a Sandor fight, it mentions somewhere of the lines of him being freakishly fast. He also holds off Gregor at the tourney. Sandor > Gregs IMO.
3. Garlan Tyrell - My wildcard. Its stated that he always practices against multiple opponents at once to replicate the battle environment. This guy's fo reals. Loras, while being a homo-prettyboy, is said to be an exceptional swordsman, but isin't respected by Jaime at all. That being said, its stated a few times in the books that Garlan is a far better fighter than his gay brother.
I think the two Jaime was talking about were Barristan and the Cleganes. Theres other fighters you could put on the list, but Jaime doesnt really 'know' about them, so to speak. Personally, on a good day, I think Brienne, The Halfhand, Mance, Thoros, and many others would have a fair chance of besting JL. Then again, I think Jaime's boast is only in the show, so it would have to be limited to characters on TV.