The Court of Arbitration for Sport (CAS) has upheld Hiromasa Fujimori's two-year doping ban, meaning the Japanese swimmer will miss his home Olympic Games.
Fujimori tested positive for the stimulant methylephedrine during the International Swimming Federation (FINA) World Short Course Championships held in Hangzhou in China.
In October of last year, the the FINA Doping Panel handed him a two-year suspension, starting from the beginning of 2019.
Fujimori appealed to CAS, arguing for the ban to be reduced.
He claimed eating contaminated rice balls the day before the test could be the reason for the stimulant being present in his sample.
"The CAS arbitrator in charge of this matter concluded that Hiromasa Fujimori had failed to establish a plausible source to the presence of the prohibited substance in his sample and, accordingly, could not benefit from any reduction in the sanction imposed in the Challenged Decision", read a CAS statement.
"Consequently, the Sole Arbitrator dismissed the appeal and confirmed the Challenged Decision."
At the Championships where he returned the positive test, Fujimori had finished third in both the 100 metres and 200m individual medleys.
The Japanese Anti-Doping Authority had initially given Fujimori a four-month suspension, before FINA distributed a more severe punishment.
He cannot return to competition until 2021, meaning he will miss Tokyo 2020.