If I erroneously made excess contributions to my Traditional or Roth IRA, when can I remove them without incurring a penalty?

By the due date of your tax return for that year -- usually April 15.