Not necessarily. If you have not filed and do not have any taxable income (e.g. under the threshold of the Foreign Income Exclusion for all tax years), then neither penalties nor interest will be applied. If you do owe tax, then you may be eligible for the Streamlined Foreign Offshore Procedures, under which you would be able to avoid non filing and/or late filing penalties. You would then only have to file for three years (but you would still owe interest on any unpaid tax).