Mine is white and I love it! Considered black but was worried about swirls too. Downsides with white are paint chips show really well on the front bumper since the plastic underneath is black. I have a clear-bra and have been told the edges of the film will show dirt much easier on white too. That being said, I have no regrets getting white.
I also have a 2006 Ram that's pretty close in color to GMs ashen grey and it doesn't show swirls at all. A friend at work has an AG SS, 2010 I think, and his looks pretty decent too. Have you seen them swirl badly or just assuming they'll show since its a darker color?
In the end, one color will speak to you more than the other and you'll convince yourself you made the right choice. Go with whichever was your first choice and don't over-analyze... that's my opinion.