Police have identified Tyrone Howard, 30, as the suspect in custody. Howard has multiple prior arrests, they said. Howard had also been wanted in connection with a Sept. 1 shooting in Manhattan, said James O'Neill, the NYPD's chief of department. Investigators suspected Howard had shot at a member of the East Army gang, but he wasn't arrested because he skipped out on court dates and police couldn't track him down at his home, O'Neill said.
On Tuesday night, Holder and his partner responded to a report of shots being fired at around 8:30 p.m. near a public housing development at 102nd Street. They saw people shooting at each other, and asked for assistance. According to the NYPD, many officers responded.
Police said Howard fled toward the FDR Drive promenade where he stole a bike and headed north. Holder and his partner caught up to the suspect on a pedestrian overpass that spans the FDR at 120th Street. Police said Howard fired and Holder was struck in the front of the head. Holder's partner returned fire, police said. Howard fled down a river path near the highway, tossing his gun and ammunition along the way, according to police.
PHOTOS: NYPD officer shot, killed in East Harlem
During a search, police apprehended Howard, who had been wounded, at East 125th Street. The suspect was taken to New York Hospital-Cornell Medical Center and has been released into police custody.
A gun and a bicycle were recovered at the bridge, police said. Divers found a loaded magazine from a semiautomatic pistol in the Harlem River. Ballistics were recovered on the FDR Drive.
The three other men are in custody in connection with the initial shots that were fired, preceding the officer's shooting. Those shots were fired from up to three separate weapons, investigators said.
Holder, 33, was assigned to PSA 5, which polices the city's public housing developments. He was a five-year veteran and native of Guyana.
VIDEO: Officer killed was member of NYPD for 5 years
"Our hearts are heavy and we offer our thoughts and prayers to his family who are experiencing unimaginable pain as we saw when we met with them earlier this evening," Mayor Bill de Blasio said.
"Say a prayer that this family will get through this emptiness that they will always have," said Pat Lynch, Patrolmen's Benevolent Association. "New York City police officers everyday go out and carry themselves like superheroes but the reality is when we're attacked we bleed, when we bleed we die and when we die we cry."
VIDEO: NYPD names Tyrone Howard as suspect in shooting death of officer
In honor of Holder, all flags in New York state and in New Jersey were lowered to half-staff immediately until the day of Holder's interment.
Nationwide, 100 officers have died in the line of duty so far in 2015. NYPD Officer Brian Moore, 25, was killed on May 9 during a patrol. A suspect was charged with murder. And at the end of 2014, on Dec. 20, officers Wenjian Liu, 32, and Rafael Ramos, 40 were ambushed and killed by a man who said he wanted to kill some cops in Brooklyn. The suspect killed himself in a nearby subway station.
Dozens of Holder's fellow officers stood outside the hospital early Wednesday and saluted as the ambulance carrying their fallen colleague left. Afterward, many embraced one another.
"Tonight, he did what every other officer in the NYPD does when the call comes - he ran toward danger," Bratton said. "It was the last time he will respond to that call."
Click here for full coverage on the fatal shooting of NYPD Officer Randolph Holder.