Indeed it is.That's a poor argument.
Especially when you recall the scene in Kingdom Hearts II Final Mix, where Xemnas accesses the computer and proceeds to make his way into the laboratory underneath the castle.
No one in the games has ever been made aware of that.