Thats been proven to be a wives tale. Guy with a high power microscope did a multi-year test with ammo in a tumbler, and if you didn't know what was control, and what had been tumbled for over a year, you wouldn't have been able to tell the difference.
Moisture is more likely to cause your powder to not ignite properly than any "breakdown" of powder.