James Madison Broom (1776 – January 15, 1850) was an American lawyer and politician from Wilmington, in New Castle County, Delaware. He was a member of the Federalist Party, who served as a U. S. Representative from Delaware.

Broom was born in Wilmington, Delaware, the son of Delaware politician Jacob Broom and Rachel Pierce. James was born near Wilmington, Delaware. His first wife's name was Ann who died August 9, 1808, and he may have remarried Mary Lowber. By his first wife he has two children, Elizabeth and Jacob, a Congressman from Pennsylvania. James Broom graduated from Princeton College in 1794. He then studied law, was admitted to the Delaware Bar in 1801 and practiced law in New Castle, Wilmington, and Baltimore, Maryland.

Broom was elected as a Federalist in the 9th and 10th Congresses, serving from March 4, 1805, until his resignation 1807, before the assembling of the 10th Congress. He moved to Philadelphia, Pennsylvania in 1819 and resumed the practice of law. Later he was a member of the Pennsylvania House of Representatives in 1824.

He died in Philadelphia on January 15, 1850, and was buried in St. Mary’s Churchyard in Hamilton Village, now a part of Philadelphia, Pennsylvania.

Elections were held the first Tuesday of October. U.S. Representatives took office March 4 and have a two-year term.