I would not expect to see any more infinity stones until perhaps gotg2 or infinity war part 1 (which I read somewhere is the thanos quest storyline)
Basically, why would 3 stones be on earth?
The space gem, the tesseract, has been chilling on earth since viking age left here by Odin for safe keeping. It went back to asgard basically same time the mind gem inadvertently got left here by loki and taken by shield/hydra.
Odin sent the reality gem to the collector for safe keeping after the tesseract returned to asgard.
Granted Odin doesn't know the location of every infinity stone or anything, but it's somewhat of a ridiculous premise that 3 stones, half of the infinity gauntlet, has just been chilling here on earth in someone's attic somewhere.