An interior graphic design method based on the graphic generation network and the dual-branch Transformer is proposed aimed at the problems of insufficient layout rationality and difficulty in coordinating and optimizing style and function in the automation of interior graphic design, especially the low manual operation efficiency of computer-aided design (CAD) systems and the poor engineering adaptability of existing automation methods. The topological structure of functional areas and spatial connections is constructed through the graph generation network. The layout is incrementally generated by combining the improved breadth-first search algorithm, and the semantic prediction network is introduced to achieve the collaborative optimization of geometry and semantics. The dual-branch Transformer processes geometric topology and functional semantics respectively, optimizes detail design by using the cross-modal attention mechanism, and dynamically adjusts the feature fusion weights. Experiments show that this method achieves an average intersection and union ratio of 84.03% and a pixel error of 4.84% in layout generation quality, with a processing speed of 0.09 s per scheme, meeting the real-time interaction requirements of CAD tools. The generated graphic design scheme achieved a Peak Signal-to-Noise Ratio of 34.17 dB and a structural similarity of 0.91 in visual quality evaluation, showing a high degree of consistency with the professional design scheme, indicating that the generated scheme has high clarity and structural rationality. Compared with the existing methods, this method demonstrates significant advantages in terms of generation efficiency, layout rationality, and design diversity. Compared with the real label drawings, the generated results are close to the actual design requirements in terms of space utilization and layout consistency. This research, through the dual-branch collaborative modeling of geometry and semantics, has significantly enhanced the automation level of interior graphic design and the practical value of generating solutions in CAD integrated scenarios.