The VO is confirmed to be Maz Kanata, by Disney themselves.
You dont need to physically hold the saber, since we all saw Vader Saber throw in RoTJ but I think it takes a lot of concentration to focus on a small point (the button) and move the saber at the same time, thats why a throw is OK (since your not using the force to throw it) but floating sabers is probably too hard.
As for Yoda in ESB, I think he struggled a bit with that Xwing cause he was probably already dyeing and his power had faded a lot by that time. He was essentially a 95 year old man by the time of ESB.