The rhizosphere microbiome can strongly promote plant growth and health by increasing nutrient availability, enhancing plant stress tolerance, and improving disease resistance. It has become an important pathway to support the development of green agriculture to fully exploit the plant-beneficial functions of the rhizosphere microbiome. Therefore, there is a great need to systematically investigate the assembly processes and functional and regulation mechanisms of the rhizosphere microbiome, to enhance the ecosystem service functionality and promote the productivity, quality, and nutrient use efficiency of crops. Plants have the capability of recruiting specific functional microbes that are advantageous for their growth under diverse environmental conditions. As such, a fundamental correlation presents between the functional requirements of plants to adapt to environmental stresses and the functional features of the rhizosphere microbiome. We defined this trait, of which the rhizosphere microbiome-derived specific functions compensate the functional requirements of the host plant, as the “functional compensatory assembly” of the rhizosphere microbiome. In this review, we introduced this concept in four stages: (1) the development and current status of rhizosphere microbiome assembly concept, (2) the structural features and impacting factors of rhizosphere microbiome, (3) the functional characteristics of rhizosphere microbiome and their mechanisms in promoting plant growth, and (4) the concept and intension of functional compensatory assembly of rhizosphere microbiome. Firstly, three models of “two-step selection”, “multistep selection”, and “amplification selection” have been proposed to describe the compositional assembly process of the rhizosphere microbiome. These models demonstrate that the rhizosphere microbiome assembly process is a selective enrichment process of the soil microbiome under rhizodeposition. Secondly, it has been established that the composition of the rhizosphere microbiome is primarily influenced by soil properties and plant genotypes. Presently, there is a growing interest in identifying and clarifying the crucial host genes that can regulate the colonization of specific microbial taxa through microbiome genome-wide association studies (mGWAS) between the host plant’s genetic and the rhizosphere microbiome. Thirdly, the interaction between the rhizosphere functional microbes and the host plant under various environmental conditions has been extensively researched. Briefly, plant recruits specific functional microbes in the rhizosphere by releasing specific exudates, while enriched functional microbes can also promote the plants’ resistance to environmental stress through diverse approaches. Finally, the concept of the functional compensation assembly of rhizosphere microbiome was introduced. We elaborated on the content of functional compensation assembly, covering its condition and object, process and mechanism, regulation, and application strategy. To summarize, we highlighted the potential for enhancing resource utilization efficiency and promoting crop growth and health by increasing the functional compensation ability of the rhizosphere microbiome. However, although our fundament research achievement is growing exponentially, there is much to study before fully exploiting the plant beneficial functions of the rhizosphere microbiome. We encourage exploration of the mechanisms of functional compensation and exploit strategies of rhizosphere microbiome, improve the theoretical framework of the functional compensation assembly, and incorporate it into the soil quality assessment and diagnosis system. This review could present a theoretical basis to enhance the efficiency of resource utilization and crop productivity, hence providing new insights for promoting the green transition of agriculture.