Mueller was one of the chief architects of the programme to exterminate Jews in Europe.
The Bild newspaper says it has seen documents confirming his death.
The president of the Central Council of Jews in Germany, Dieter Graumann, said he was shocked by the revelation.
"That such a brutal Nazi sadist is buried in a Jewish cemetery of all places is a tasteless monstrosity. The memory of victims is being heavily trampled on," he told Bild.
It quotes Prof Johannes Tuchel, an historian, who says he has discovered archive evidence showing that Mueller died at the end of the war.
"Mueller never survived the war. His body was buried in 1945 in a mass grave in the Jewish cemetery," he said.
Correspondents say any efforts to find the body of Heinrich Mueller would be very difficult since the exact location of the grave is unknown.
There has always been an air of mystery surrounding the fate of Heinrich Mueller, a participant at the Wannsee Conference in January 1942 where the elimination of Jews - the "Final Solution" - was determined.
He was recorded as being present in Hitler's bunker on the day he committed suicide. But thereafter his whereabouts remained unknown.
In 1949 Germany's intelligence agency (BND) reported seeing Mueller in Czechoslovakia. But there seems to be no evidence that he was pursued.
Prof Tuchel, who is also the head of the German Resistance Memorial in Berlin, said that that sighting was incorrect. He said documents unearthed indicate that Mueller was first buried near an airport in Berlin in the last days of the war.
He said that "a body in a general's uniform with Mueller's identification documents" was then disinterred and reburied in one of three mass graves in the Jewish cemetery and certified by a local registrar of deaths in 1945.
Later theories suggested Mueller was buried in the Neukoelln district of Berlin. And in 1963 a gravedigger, Walter Lueders, told police that he had buried Mueller personally, according to the German online website, The Local.