Having started out studying AI in the 1980s, I have to agree with your reservations in that regard.

However, the argument here does not support the implied conclusion that the fact we feel ourselves to be real proves that we are.

Cogito ergo sum.

Given that my experience of something is itself real … that what I experience is as real as ‘real’ is ever going to be for me … that I’ll never know any different … Epistemology and Ontology are bunk — we cannot know whether we are a simulation or not.

Quite apart from the fact that, philosophically, until another position is proffered that is more logically unassailable, I have no choice but to be a solipsist … as a psychologist, I know that, if Descartes was wrong and it is all externally real … really real, so to speak … then, by the time I become consciously aware of anything, it has been so mediated by my brain that I’ll never know to what extent my neurocognitive model tallies with the external stimulus — and Descartes was, therefore, right even though he was wrong.

Furthermore, I have in my life experienced things that I knew not to be real even as I experienced them. I still really experienced them, however.

So …

My feelings … or even perceptions … on the matter are irrelevant: it is not a tractable problem (C.f. Gödel’s ‘Incompleteness Theorem’); in order to determine that it be the case, we would need to be able to exist outside the simulation itself … at which point we cease to be the very people making the enquiry and stop making it — thus no knowledge from outside the simulation may be returned into it and we cannot, therefore, know that we aren’t.