The gravitational pull of the comet is larger than any other force acting on the rocks. At one point it may have been floating nearby like a small satellite, but given enough time they get closer until its just sitting there. And if it got close to something with enough pull to move the small rocks, it'd likely move the whole comet, and/or not have enough time to separate the small rock given the speed of the passing.
- 2