The advent of Open Radio Access Networks (O-RAN) has brought about significant changes in the telecommunications industry by promoting interoperability, vendor diversity, and rapid innovation.
Testing O-RAN components against industry specifications poses challenges due to the disaggregated architecture, leading to manual and error-prone processes within existing testing frameworks.
To address these challenges, AI5GTest is introduced as an AI-driven testing framework that automates the validation of O-RAN components, improving consistency and scalability.
AI5GTest utilizes a cooperative Large Language Models (LLM) framework, comprising Gen-LLM, Val-LLM, and Debug-LLM, to automate test case generation, compliance validation, and root cause analysis.
Gen-LLM generates procedural flows for test cases based on 3GPP and O-RAN specifications, while Val-LLM cross-references signaling messages to ensure compliance and detect deviations.
In case of anomalies, Debug-LLM conducts root cause analysis to pinpoint the cause of failure, enhancing transparency and trustworthiness in the testing process.
AI5GTest incorporates a human-in-the-loop mechanism where relevant official specifications are presented to the tester for approval, ensuring transparency and accuracy in validation.
Evaluation of AI5GTest using test cases from O-RAN TIFG and WG5-IOT test specifications shows a significant reduction in test execution time compared to manual methods, while maintaining high validation accuracy.