You might try plugging the wire from the front USB ports into a different USB connector on your motherboard. Most new-ish motherboards are going to have 2,3,4+ places that you can plug in USB ports. Also pay attention to if the ports are USB 2.0 vs 3.0 on the motherboard, and what the ports on the front of the case support(my case for instance has 4 USB 2.0 ports on the front, and 2 USB 3.0, plug them in to the wrong places on the motherboard internally and you're going to get all sorts of inconsistent performance).
Also, depending on your motherboard it could have varying amounts of USB controllers, if you have multiple USB devices you want to try to spread out the devices across as many separate controllers as possible. The more devices on 1 controller, the slower it's going to be.
And then just generally speaking, the ports on the back almost always perform a little better as they are connected/soldered straight to the motherboard, rather than being connected via a long wire like the front ports. That wire does cause latency, but it shouldnt be nearly as drastic as what you are experiencing.