Quantum circuit optimization is improving the efficiency of quantum algorithms while addressing hardware resource limitations. Traditional optimization methods often rely on heuristic-based techniques, which can be inefficient for large-scale circuits.

In this study, we propose a reinforcement learning (RL)-based framework for optimizing quantum circuits. Our approach considers the universal gate set, including CNOT and single-qubit rotation gates, aiming to minimize both the number of CNOT gates and the overall circuit depth. By mapping quantum circuits into grid and cube representations, we enable an RL agent to learn efficient transformations. Specifically, we leverage Proximal Policy Optimization (PPO) and model-based RL techniques, incorporating a Convolutional Neural Network (CNN) to extract structural patterns from matrix representations.

Furthermore, we extend our method to quantum circuits with a larger number of qubits, ensuring its adaptability across diverse hardware architectures. Our approach is particularly well-suited for Noisy Intermediate-Scale Quantum (NISQ) devices, effectively addressing hardware constraints while enhancing computational efficiency.