Committee chairman, Edward Leigh MP said: “The last thing anyone starting a new business wants to do is spend a lot of time talking to the tax man. But new businesses are frustrated by having to register separately with HMRC for each and every tax they have to pay.
“This is because the department is hampered in having a separate computer system for each tax. It must push towards a system where a new business registers just once for all the taxes it must pay."
He went on to say: "It is also ridiculous that owners of new businesses are badgered by different parts of HMRC all wanting the same bits of information. Providing each business with a unique tax reference number would be an important step towards the department being able to have an overview, for each business, of all of its tax affairs."
Around 700,000 new businesses start up each year and many have to register with HMRC for different taxes often at different times.
There are around 1.2 million registrations each year.
New businesses incur late filing penalties of over £8 million on Income Tax Self Assessment and Corporation Tax.
HMRC spends more than £10 million a year on staff engaged in registering new businesses, and a further £23 million in providing help through its website, printed guidance, telephone helplines, workshops and seminars.
The committee found that newly registered businesses are a diverse group and attitudes to compliance also vary.
But those who get their tax right the first time are more likely to continue to comply as they grow and their tax obligations increase.
Compliance can be more onerous for new and smaller businesses which have most to gain from assistance and simplified requirements.