John H. Herz was born as Hans Herz in Düsseldorf, Germany to Carl and Elizabeth Aschaffenburg Herz on September 23, 1908. Herz received his grammar school and high school education in Düsseldorf and graduated from the Städtisches Realgymnasium zu Düsseldorf in 1927. Herz continued his studies at the Universities at Freiburg im Breisgau, Heidelberg, Berlin and Cologne, and graduated with a degree in public law from the University of Cologne in 1931 under the direction of Hans Kelsen. From 1931 to 1933, Herz served in the courts of Düsseldorf as Gerichtsreferendar (clerk), until the Nazi racist laws forced his dismissal in 1933. Even after his dismissal in 1933, Herz continued to work as an attorney's assistant to a private firm in Düsseldorf until 1935, when he was able to immigrate to Switzerland under a student visa. For the next three years, Herz worked as a translator for the International Labour Office in Geneva, while at the same time attending the Institut Universitaire de Hautes Études Internationales. Herz received a "diplôme" from the institute in 1938 with a thesis entitled The National Socialist Doctrine of International Law and published in the same year under the pseudonym Eduard Bristler.

In 1938, Herz emigrated to the U.S. and began his U.S. career as John H. Herz at the Institute of Advanced Studies in Princeton, New Jersey, where he remained for the next three years. In 1941, he married Anne Klein and accepted a position as an Instructor in Government at Howard University in Washington, D.C., where he remained until 1952. During his years in Washington, Herz became an American citizen in 1944 and also served as a political analyst in the Central European Section of the Office of Strategic Services and for the U.S. Intelligence and Research Office of the Department of State, as well as continuing to write and publish in the fields of political science and international politics. In 1951, Herz received the Woodrow Wilson Foundation Book Award for his publication, Political Realism and Political Idealism, and in 1952, with co-editor Gwendolen Carter, published the first edition of the textbook, Major Foreign Powers, which was reprinted 5 additional times (1952, 1957, 1962, 1967 and 1972).

In 1952, John Herz accepted a position as Associate Professor in the Department of Political Science at the City College of the City University of New York, where he remained until his retirement at the rank of Full Professor in 1977. During his years at City College, Herz served as Chairman for the Masters Program in International Relations (1953-1977) and also accepted Visiting Professorships at Columbia University, the Graduate Faculty of the New School for Social Research, the Fletcher School of Law and Diplomacy of Tufts University, the Freie Universität Berlin and the University of Marburg. In addition to his ambitious teaching career, Herz continued his association with Gwendolen Carter and together they published three additional works, which became standard textbooks in U.S. universities: Government and Politics in the Twentieth Century (1961), Documents of Major Foreign Powers (1968) and German Constitutional Documents Since 1871 (1970).

After his retirement in 1977 with the title of Professor Emeritus, Herz became a member of the Center for European Studies at the Graduate Center of the City University of New York and continued to publish books, articles and lecture in the area of political theory and government. In 1984, Herz completed his autobiography, which was published in German as Vom Überleben. Wie ein Weltbild entstand.

John Herz died on December 26, 2005 in Scarsdale, New York.