A government statement says 21 died of suffocation after a stampede while two others were killed as they tried to escape.
Some local media have disputed the account, citing unnamed witnesses who say prisoners were shot by the wardens.
There has been an unprecedented wave of protests in Ethiopia in recent months.
The identity of the prisoners has not been made public.
Sustained gunfire could be heard coming from Qilinto prison, on the outskirts of the capital Addis Ababa, after the fire broke out on Saturday, local media reported.
TV footage and photos posted on social media showed plumes of smoke rising from the prison compound.
Reports that the fire was started deliberately as part of an attempted jailbreak have not been independently verified.
There have been numerous protests in the Oromia region by members of the country’s largest ethnic group since November 2015.Qilinto is a remand prison, where people can be held for three years or more as they await trial.
The prison is divided in four zones made up of brick walls and tin-roofed cells.
The prison hosts around 3,000 inmates at a time who are held in cells measuring 24m by 12m. Each cell holds between 90 and 130 inmates.
It is a highly secured prison with surveillance cameras installed on many corners.
All types of prisoners are held there but it is where political prisoners including bloggers, journalists and activists are usually sent.
Political prisoners usually mix with other criminals but they are usually locked up in a designated “Kitat Bet” (punishment house) or “dark house” if they complain about mistreatment.
Inmates can be exposed to communicable diseases due to overcrowding and get poor medical attention.
Due to the bad quality of food provided by the prison administration, prisoners mainly depend on food brought by their families.
Many Oromo activists are being held at the Qilinto facility, according to pro-opposition media.
New York-based Human Rights Watch says that more than 400 people have been killed in clashes with the security forces in Oromia, although the government disputes this figure.
Ethiopian Prime Minister Hailemariam Desalegn has blamed “anti-peace forces” for the violence.