The entirety of the film takes place on the planet Arkanar. No Renaissance has occurred to allow the natives to progress beyond the same stage as the Middle Ages in Europe. What brief glimpses of science or culture that arise are ruthlessly suppressed by the proto-fascist police-state government. What we see of Arkanar is unrelieved bedraggled squalor: The people are filthy; they dress in absurdly shabby clothes and live in shacks; the streets are permanently muddy and the sky is permanently dreary.
About thirty Earth scientists are sent to the planet incognito, in order to observe the evolution of the society. They are barred from influencing the local civilization or interfering with the city's natural course. One of the Earthmen has been living on the planet for twenty years, in a large house filled with his multitudes of slaves and servants. Disguised as a Noble Don named Rumata of Estor, he has become identified among some of the townspeople as the son of Goran (a local pagan god). In duels, Rumata does not kill his adversaries, but only cuts off their ears, as it is forbidden for him to kill. At night, he dreams of murder. He is tasked with saving the intellectuals of this society - bookworms and sages - who are constantly persecuted by the so-called Gray troops under the leadership of Don Reba, who through usurpation and oppression has become the all-powerful ruler.
From Muga, his head slave, Rumata learns of a certain tobacco-grower from Tobacco Street who is supposedly one of the "clever" inhabitants of Arkanar. Rumata hides Arkanarian scientists on rocky islets among the swamps that surround the city. From time to time, other Earthmen also gather in these swamps, progressively more alienated from each other, and constantly inebriated (as is Rumata). A doctor named Budakh, who was supposed to be taken to shelter, has disappeared along with his escort. Desiring to find out Budakh's fate, Rumata arranges an appointment with the king and Don Reba, but fails to learn anything about Budakh. In the saloons, Rumata meets his friend Pampa, a washed-up local baron. After a drunken night, Rumata is suddenly arrested in the palace by Grays and taken to Don Reba for interrogation. Reba informs him that he has become the Master of the Order, a militant religious sect from the other end of the country, and tries to find out if Rumata really is a god or not. Rumata manages to convince Reba of his divine powers, and he is released after being given a collection of security-clearance bracelets.
The black-robed monks of the Order invade the city during the night and seize power in Arkanar. The king and his family are killed and the courtiers are publicly executed. In the morning, Rumata goes to the torture chambers of the Tower of Joy and rescues Budakh and Baron Pampa, who also happened to end up there. While trying to escape from the city, Pampa is killed with arrows. Afterward, Rumata talks with Budakh, trying to figure out how the scientist would advise a god to rectify the state of affairs in the world of Arkanar. Budakh's answer does not satisfy the Earthman: The weak will replace the strong, but this will not stop the constant struggle for power. Rumata and Budakh return home. Rumata's castle has survived an attack during the night, although two servants were killed, including Uno, a young troublemaker and Rumata's favorite. Defeated and tired, Rumata meets Arata the Hunchback, a feared revolutionary figure, who tries to persuade him to lead a slave uprising with his Earth technology. Rumata refuses; time will pass, exploitation and slavery will recur in a never-ending cycle. Having given Arata a protective bracelet of the Order, Rumata orders the servants to drive him away with sticks. The next day, Ari, Rumata's lover on Arkanar, is killed by an arrow shot through the back of her head with a crossbow. Right then, soldiers of the Order burst into Rumata's house in search of heretics, guided by a certain crippled monk, by whose description it is easy to guess that it is Arata. Furious at the murder of Ari, Rumata declares to their leader, a former university student named Arima, that he will kill them all, and soon makes good on his threat. He kills Arima first. After disemboweling Arima's body, revealing his still-beating heart, Rumata sets off for the city and massacres everyone in his way.
A group of Earthmen, including the Elder Don Condor, make their way through the ruins of Arkanar. They discover mountains of corpses, including those of Arata and Don Reba. From the conversation among the Earthmen, it becomes clear that Ari died from an arrow shot by Arata with the aim of setting Rumata off on the monks of the Order, but also being killed himself. Finally, Don Condor and Pashka discover Rumata himself, sitting alone in a puddle in his pajamas. He refuses their offer to return him to Earth. “A god can get tired too,” he says to Condor. Don Condor silently accepts Rumata's decision. Finally, Rumata notes the historical sequence with a grave warning: "Where Grays triumph, Blacks always come in the end." He advises Don Condor what to write in the report on his actions: "Tell him that it is hard to be a god." With this, Don Condor and Pashka leave. Exhausted, Rumata falls asleep.
During the winter, two of the bookworms hiding in the swamps quarrel and kill each other. Rumata has stayed behind in the land of Arkanar, becoming a part of their world. He is older, graying, sports a new pair of glasses, and sets off into the snowy distance with his various slaves and horsemen. Meanwhile, a little girl, and her father walk through the snow. The girl asks her father what he thinks of Rumata's saxophone playing, which they hear as they leave. The father isn't sure what he thinks. "It makes my tummy hurt," the little girl responds. The screen fades from the white landscape to black.