The increasingly complex nature of cyber intrusions into United States critical infrastructure and enterprise network systems requires independent defense systems that can be modified in real time without human intervention. The paper provides a systematic exploration of the field of secure multi-agent artificial intelligence systems of autonomous cyber defense, with particular focus on the combination of large language models and multi-agent reinforcement learning. It compares nine architectural paradigms in accuracy of detection, efficiency of operations, resistance to adversaries, scalability of coordination, scalability to domain, transferability to simulations and explainability. It is demonstrated that large language model-orchestrated multi-agent reinforcement learning architectures are more effective .Semantic reasoning and causal understanding are improved by the incorporation of fine-tuned large language models, whereas the agreement between agents is improved by transformer-based coordination. Altogether, this convergence constitutes a milestone in autonomous cyber defense of the critical infrastructure protection.