It happened at about 11 a.m. Sunday on Madison Avenue between 38th and 39th Streets.
Officers who responded to the high-rise building found that a crane's payload heading to the top of the building had broken free. It fell to the sidewalk, shearing the side of the building along the way.
The building is wider at the base, and narrows at the higher floors.
None of the injuries are life-threatening.
The crane is now leaning against the high-rise building. There is visible damage to the top portion of the building.
A city official tells ABC News the Buildings Department is trying to piece together what happened, what the crane was doing and whether all permits were in place and all rules followed.
The Fire Department and Buildings Department are now examining surrounding buildings to determine if any are structurally compromised.
The Associated Press contributed to this report.