it's quite clearly NOT a single universe. we've seen clips from an unaired episode of winter soldier (bucky with long hair) but in the captain carter episode, bucky never falls off the train. even if we can't take that into consideration since it hasn't aired yet, the collector has captain america's shield, NOT captain carter's shield. captain america's shield has the star on it, captain carter's shield has the union jack. and thirdly, at the end of THIS episode, we clearly see a frozen captain america shield, which didn't happen in the first episode. like gavinmad pointed out you can't have the same shield be in 3 different places.
you also have the fact that in the first episode, captain carter comes back during the beginning of the avengers movie, but hawkeye is there with him and that obviously couldn't have happened if he died during the first thor movie.
in the beginning of each episode, the watcher narrates what happened in the MCU, and then he shows us what happened in a different universe. there should never be the assumption that he is only comparing 2 different universes.