Oskar Maria Graf was born on July 22, 1894 in Berg on Lake Starnberg in Bavaria, Germany. He was the youngest son of the eight surviving children of baker Max Graf and Therese Heimrath. Graf's father was successful as a baker, providing a comfortable home and surroundings for the Graf family. Oskar entered elementary school in Berg in 1900, however shortly after his father's death in 1905, he began an apprenticeship in the family bakery under his older brother Max. Although Oskar endured nearly five years under the strict supervision of his brother, in 1911, to escape the beatings of his brother, he ran away to Munich where he joined a Bohemian group. He made connections with the anarchist circle "Die Tat" and in 1912 tramped to the Tessin region in southern Switzerland to join an anarchist colony. He became disillusioned with the colony and returned to Munich. During this time, supporting himself with odd jobs, he began to write poems and short stories. It wasn't until 1914, however, that he was able to get several of his poems published in the expressionist periodical Die Aktion (Berlin).

Graf was drafted into the German army in 1915 to serve on the Eastern Front. Graf went on a hunger strike, feigning insanity until he was finally committed to a mental institution. By 1916, he was both released from the mental institution and discharged from military service. Graf returned to Munich where he worked in a factory and wrote reviews for the Munich newspaper München-Augsburg Abendzeitung. In 1918, Graf published his first significant work, Wir sind Revolutionäre, married his first wife, Karoline Bretting, with whom he had a daughter, his only child, Annemarie, a year later. In 1918, he also met Mirjam Sachs, the cousin of poetess Nelly Sachs, who would later become his second wife. Later in the same year, he took part in anti-war demonstrations, and in 1919 participated in a revolution which resulted in the short-lived Soviet Bavarian Republic, and, after its defeat, was imprisoned for his participation.

In 1920 Graf became the dramatic producer of the Socialist worker's theater "Die neue Bühne" in Munich, and in 1922, published both Frühzeit and Zur freundlichen Erinnerung, with his Bayrisches Lesebücherl following in 1924. It wasn't until 1927, however, that Graf gained international recognition with the publication Wir sind Gefangene. Graf continued to write, publishing Das bayrische Dekameron in 1928 and Kalender-Geschichten in 1929, followed by of Bolwieser: Roman eines Ehemanns in 1931 and Notizbuch des Provinzschriftstellers Oskar Maria Graf in 1932.

In the early months of 1933 Graf was on a lecture tour in Austria. It was during this time, shortly after Hitler's rise to power, that the Nazis ordered the first of the book burnings. With the exception of Prisoner's All, Graf's books were recommended rather than burned by the Nazi regime. This prompted Graf to write his famous "Verbrennt mich!" ("Burn me too"), perhaps one of the most famous anti-Nazi statements, which was published in the Wiener Arbeiter-Zeitung on May 22, 1933. Graf remained in Austria in exile, where he lived until 1934, working as co-editor of the German expatriate journal Neue deutsche Blätter. In 1934, after having participated in a political uprising of Austrian workers against the Engelbert Dollfuss regime, Graf escaped to (Brno) Czechoslovakia, where he remained until 1938. He left for a short period to attend the First Congress of Socialist Writers in Moscow but returned to Brno where he continued to write. Graf's novel Anton Sittinger was published in 1937 by the German exile publishing company Malik in London.

In 1938, Graf left Europe without his wife and child and fled to New York City. Mirjam Sachs followed Graf to New York, however it wasn't until 1944 that Karoline agreed to a divorce and the couple finally married. Graf's only daughter, Annemarie, remained in Germany to be raised by his mother, and eventually joined a German Youth organization under Hitler called the Bund Deutscher Mädchen (BDM). Graf, who was vehemently opposed to Hitler, was never close to his daughter and there are no letters in the collection between Graf and his daughter. While in New York, Graf made speeches before German-American groups and wrote regularly for the New York City German-language newspaper Aufbau, edited by his wife's brother, Manfred George. He also became president of the German-American Writers' Association during this period.

For a short time Graf lived in the artist colony Yaddo, near Saratoga Springs in upstate New York, where he completed work on the German version of his novel The Life of My Mother (Das Leben meiner Mutter), which portrayed life in Germany from the Bismarck era to the time of Hitler. He followed this with the publication of Unruhe um einen Friedfertigen, the story of a Jewish cobbler killed by the Nazis after having lived for many years fully integrated into the life of a Bavarian village.

Graf was unable to return to Germany immediately following the war. Not only had the Hitler regime revoked his German citizenship, but he was also unwilling to sign American citizenship papers because it would require him to bear arms. Finally in 1958, after the removal of the clause on bearing arms, Graf became an American citizen, and with his American passport, he was able to revisit Germany, and did so four times before his death.

Graf's second wife, Mirjam, died in 1959 after a long battle with breast cancer. He never wished to return permanently to Germany, claiming he did not like the "New Germany" and having become extremely comfortable in his exile home of New York. He never mastered the English language, primarily because as a writer he wanted his German to remain "pure". He established a regular German "Stammtisch" called "Die blaue Donau" in New York, and in 1962, married fellow exile Gisela Blauner, who had earned her Ph.D. in jurisprudence in Germany before being exiled. In 1964 Graf became corresponding member of the German Akademie der Künste (Academy for Arts) in Berlin.

Oskar Maria Graf died on June 25, 1967 in New York City of complications arising from pneumonia. His ashes were buried a year later in Munich, June 28, 1968. Graf was survived by his third wife, Gisela, who assisted in publishing new editions of his works and organized exhibitions devoted to displaying his life and works. His daughter, Annemarie Koch (1919-2008), and granddaughter Ricarda Glas (born 1943) continued to reside in Germany.