On October 11, 1926, Weiss was crossing State Street with four of his men to enter his headquarters, when two gunmen suddenly sprayed them with buckshot and submachine gun bullets from a nearby second-story window. Hymie and one of his men were instantly killed, the others wounded. Bullets from the fusillade sprayed and slightly damaged the cornerstone of the Holy Name Cathedral, Chicago directly across the street (the bullet wholes aren't visible today, but you can see there's a new layer of cement above it). The 28-year old gang boss was said to have left a fortune of $1.8 million dollars. Following his death, Weiss was buried at Mount Carmel Cemetery in the Chicago suburb of Hillside, Illinois. Vincent Drucci took over as head of the North Siders.
Weiss was said to have suffered from arterial cancer, which often gave him blinding headaches, dizziness, and fainting spells. Because of his illness, Hymie often said "I don't expect to live long, but I expect to live long enough.", which may explain his fearlessness while fighting Al Capone. It is also believed that Hymie Weiss was the first to use the infamous words 'Lets take him for a ride' when they were planning a hit on somebody.