That's what Quintilian implicitly said in his Institutio Oratoria (in the 1st century CE), and there's no real reason to doubt him in this case: the fact that the earliest attested plural form (in Plautus' Poenulus, almost three centuries earlier) is avo rather than avēte conforms to it being a Punic loan, and the Punic certainly started with ḥ (/ħ/ rather than /h/, but /h/ is the closest approximation Latin had). It's actually found written with the initial h in Latin in a number of cases, including at Pompeii, outside the House of the Faun:

Note that the e was originally short, as well: havĕ, not havē. (Though even if it were havē originally, it could still end up as havĕ due to iambic shortening.)
As for why it wasn't always spelled with the h, interference from avē, the 2sg. imperative of aveō 'I desire', is presumably to blame for that—Punic bilingualism was never widespread in Rome, so it's to be expected a folk etymology would develop. Semantically, "desire!" makes a lot less sense as a greeting than "live!", though, which is what the Punic actually meant.