A new evaluation framework for assessing large language models' (LLMs') contextual understanding in question answering (QA) tasks. The framework, based on knowledge graphs, measures semantic and structural similarity to evaluate LLMs' ability to reason over context. Results show significant improvements over baseline models.