Several other people were reportedly also injured in the attack, a day after protests in Turkey over Russia’s military intervention in Syria.
According to Russian TV, the ambassador had been attending an exhibition called “Russia as seen by Turks”.
Police later “neutralised” the gunman, reports say, without giving details.
Karlov was rushed to hospital, reports said, but his death was later confirmed by the Russian foreign ministry.
“Terrorism will not pass! We will fight it resolutely,” said ministry spokeswoman Maria Zakharova.
“The memory of this outstanding Russian diplomat, a man who did so much to counter terrorism… will remain in our hearts forever,” she said.
US state department spokesman John Kirby said US officials were aware of reports about the attack on the ambassador.
“We condemn this act of violence, whatever its source,” he said. “Our thoughts and prayers are with him and his family.”
Video of the event shows Karlov making a speech when gunshots ring out.
The camera pulls back to show a smartly dressed gunman, wearing a suit and tie, waving a pistol and shouting.
He can be heard yelling “Don’t forget about Aleppo, don’t forget about Syria” and uses the Arabic phrase “Allahu Akbar” (God is great).
While there were protests in recent days about the situation in Aleppo, on a political level the Turkish and Russian governments have been co-operating in the ceasefire operation, the BBC’s Turkey correspondent, Mark Lowen, reports.
It is too early to know the motives of the attacker, who has not yet been named. Some reports say he entered the art gallery using a police ID card, our correspondent adds. (BBC)