Conceptual knowledge allows us to comprehend the multisensory stimulation impinging on our senses. Its representation in the anterior temporal lobe is a subject of considerable debate, with the "enigmatic" temporal pole (TP) being at the center of that debate. The controversial models of the organization of knowledge representation in TP range from unilateral to fully unified bilateral representational systems. To address the multitude of mutually exclusive options, we developed a novel cross-modal approach in a multifactorial brain imaging study of the blind, manipulating the modality (verbal vs pictorial) of both the reception source (reading text/verbal vs images/pictorial) and the expression (writing text/verbal vs drawing/pictorial) of conceptual knowledge. Furthermore, we also varied the level of familiarity. This study is the first to investigate the functional organization of (amodal) conceptual knowledge in TP in the blind, as well as, the first study of drawing based on the conceptual knowledge from memory of sentences delivered through Braille reading. Through this paradigm, we were able to functionally identify two novel subdivisions of the temporal pole - the TPa, at the apex, and the TPdm - dorso-medially. Their response characteristics revealed a complex interplay of non-visual specializations within the temporal pole, with a diversity of excitatory/inhibitory inversions as a function of hemisphere, task-domain and familiarity, which motivate an expanded neurocognitive analysis of conceptual knowledge. The interplay of inter-hemispheric specializations found here accounts for the variety of seemingly conflicting models in previous research for conceptual knowledge representation, reconciling them through the set of factors we have investigated: the two main knowledge domains (verbal and pictorial/sensory-motor) and the two main knowledge processing modes (receptive and expressive), including the level of familiarity as a modifier. Furthermore, the interplay of these factors allowed us to also reveal for the first time a system of complementary symmetries, asymmetries and unexpected anti-symmetries in the TP organization. Thus, taken together these results constitute a unifying explanation of the conflicting models in previous research on conceptual knowledge representation.