The men climbed the crane sometime before 1:30 a.m. on East Santa Clara Street near San Jose City Hall.
Nearby streets had to be shut down while police worked to bring the men down.
One of the men was taken to Santa Clara County Jail on an outstanding arrest warrant.