Keeping more email than is actually needed is a habit that most people fall in to at some stage. But make no mistake, it is a problem. And it's not one to be ignored.
The main issue with keeping large volumes of email is the affect it has on your mail server and therefore the knock-on effect it has on Outlook (assuming that's what you're using!).
With all of the additional load for your hardware to cope with, at best it'll become sluggish and unresponsive - at worst, it'll just grind to a halt altogether.