The shooter was one of the two killed, according to investigators. They identified him as Jeffrey Johnson, 56, who was laid off last year and apparently killed a former co-worker near the building, shooting the 41-year-old victim in the head.
Clad in a business suit and carrying a briefcase, police say the gunman apparently had a longstanding dispute with the victim over allegations of harassment in the work place.
A construction worker followed the gunman after the initial shots and alerted officers. The suspect then fired on police and they returned fire, killing him, said Police Commissioner Ray Kelly.
Two police officers are being treated at a nearby hospital, though all injuries are not considered life threatening, said Mayor Michael Bloomberg.
Some of the victims may have been hit by shots fired by police officers, Bloomberg said.
Johnson was a Manhattan resident and had worked as a designer of women's accessories at Hazan Imports.
He used a .45-caliber semi-automatic handgun and he was also carrying additional ammunition in his brief case, police said.
Witnesses said police shot the man at least three times.
"It's just a crazy scene here," said Rebecca Fox, who works across the street from the famous skyscraper. She said she had been getting coffee with her headphones on when she saw people running.
Fox said she saw a woman who had been shot near a local brewery, as well as blood splattered on the sidewalk.
One witness -- 22-year-old Max Kaplan -- said he heard at least nine shots and saw ambulances race to the scene.
"We're all very shaken up at the office," he said.
Aaron Herman, a CNN iReporter, painted a portrait of confusion.
"It was a little chaotic, police had barricaded the area and I saw one woman who was a victim, I think she had been grazed," he said. "Some said they heard around three 'pops' and ran into nearby local stores to be safe."
Local and federal authorities who converged on the building around 9 a.m. closed several streets around 5th Avenue and 34th Street, snarling traffic in the heart of Manhattan.
Shortly after the incident, Manhattan's Bellevue Medical Center reported that it was treating six victims suffering from gunshot wounds, who were at the Empire State Building Friday. All the injuries were not life threatening, according to hospital spokeswoman Alice Ramos.
The Empire State Building is one of the most famous skyscrapers in the world and one of New York City's best-known tourist attractions.
Each year, about 4 million people visit the building's two observation decks. At more than 1,453 feet tall, the landmark building reaches more than a quarter-mile into the sky.
The area typically has a large security presence.
"There's always a focus and concentration on the building," said retired police officer Lou Palumbo
"That building gets special attention."
The Empire State Building Company said in a statement Friday that "the building is fully operational at this time," and that police are investigating the incident.
CNN's Rose Arce, Poppy Harlow, Mark Norman, Rob Frehse, Leora Kapelus and Pauline Kim contributed to this report