If the header on the drivers side is touching the steering shaft. As the motor torques it will raise on the drivers side forcing the shaft to pivot away from the motor forcing the steering shaft to the left. When you let off the throtle the motor would return to a nuetral position, pulling the shaft down forcing a right pull. A broken motor mount could also factor in to this scenario.
Kind of at the outer limits of options but the OP is looking for suggestions.
LS3/M6 K&N, VMAX throttle body, RX Catch Can, skip shift eliminator, DT shorties and Solo HFC, DRL harness and 5K LED DRL, ZL1 rear sway, 1LE front sway