Jones will officially commence his duties from the 1st of January the organisation said on its website.
Jones resigned as Australia's head coach at the end of October in the wake of the Wallabies' worst Rugby World Cup campaign, where losses to Fiji and Wales saw the 1991 and 1999 champions exit at the pool stage for the first time.
Jones, whose mother and wife are Japanese, had denied media reports during the tournament that he had been interviewed for the Japan job a couple of days before his team's final World Cup warm-up.
Jones will be a replacement for his successor Jamie Joseph, who led Japan's Brave Blossoms to the World Cup quarter-finals on home soil in 2019 and a pool-stage exit in France.
After being sacked by England last December, Jones returned home in January for his second stint as Australia's coach after leading the Wallabies to the 2003 World Cup final in the first.
Jones immediately said Australia would claim a third World Cup triumph in France, an assertion that looked laughable after the Wallabies lost all five of their test matches in the run-up to the tournament.
He left the Wallabies with two wins, over Georgia and Portugal, and seven losses from nine tests in 2023 and ranked ninth in the world.
Jones remains highly regarded in Japan, however, having led the Brave Blossoms to a stunning upset of South Africa at the 2015 World Cup.
