The balance system is complex and involves the brain analyzing and interpreting information from three major systems: the inner ear sensors for balance (vestibular system), the visual system, and the sensation that goes from the skin, muscles and joints (proprioceptive system).
Any sensation of dizziness and imbalance may result when one or more of these four parts are not functioning well. Therefore, the perceived asymmetries you experience may stem from the vestibular organs, from the proprioceptive system and/or from the brain’s processing of the their information. You did not mention visual issues, but treating these symptoms often involve also addressing the processing of visual information, with and without head movements.
You may wish to pursue assessments to clarify whether you do have asymmetries in your balance system, namely a vestibular assessment (for the inner ear sensors and their connections with eyes and brain) and a physiotherapy assessment, for the proprioceptive and musculo-skeletal systems. Proper treatment/rehabilitation can then be tailored to your needs.