Helmet manufacturers and the Snell Foundation suggest replacement after about 5 years.
Impact protection comes from the outer shell, the styrene inner shell, and the comfort lining. The styrene inner liner crushes as it absorbs energy. The comfort liner compresses until it bottoms out against the styrene. Glues, resins and other materials used in helmet production can affect liner materials, and general liner deterioration and helmet degradation can result from hair oils, body fluids, cosmetics, and normal "wear and tear."
From past experience there will probably be a noticeable improvement in the protective characteristic of helmets over a five year period due to advances in materials, design and the standards. Therefore, regardless of wear, you will probably want to replace your helmet every five years.