You can leave the starting zone right away if you really want. Most of the quests there don't give that much exp and it's easier to just grind some monsters in most cases. The fastest method would probably be grinding monsters while completing the quests, and always remember to group up for things like Sarkoth. Some people are leveling to 5-10 through exploration and few fedex quests. It's mostly on the Alliance side, but you could technically level to 5 through exploration and it wouldn't be too hard. It would be a lot quicker to just do the starting area, so you should probably just do it. I'm personally skipping most of the annoying quests like the scorpid tail collection quest. Most of them don't have chains, and if they do they're short ones. You can get the same amount of exp by killing a few extra monsters.
Also don't forget you can go to other starting areas. From levels 1-12, Silverpine has around 15 collection quests, Mulgore has about 13 and Durotar wins with only 9, so I'll be questing in Durotar.