I disagree with this. Our defense and special teams will be heavier lifts to rebuild, but it's not like those were traditional 4 year guys. The talent we're losing we gained from the portal originally, meaning we can certainly gain that talent again.
We're returning our starting RB (I know Kellom is gone, but he's not an irreplaceable talent by any means), the core of our WR room (Norton is gone, but Polk and others are back), and most importantly our starting QB. Look at the Portal - not many teams are returning their promising starting QB.
I don't think the Portal is affecting our momentum one bit - yet. Should the above change and we see late additions to the Portal from our roster, or should we fail to replenish with talent, then I might change my mind. As it currently stands, though, we're in a good spot I think.