The New York Police Department had released a photo of Rahami, 28, and said they wanted to question him about a Saturday night explosion that wounded 29 people in Manhattan's Chelsea neighbourhood and for a blast earlier that day in Seaside Park, New Jersey, authorities said.
The FBI released a wanted poster for Ahmad Khan Rahami, 28, a naturalized USA citizen of Afghan descent.
"We want to get this guy in for questioning", de Blasio said on CNN.
Bollwage said two men called police and reported seeing wires and a pipe coming out of a package after finding it around 8:30 p.m. Sunday. Physical evidence links Rahami to explosive devices in NY and New Jersey, the source told NBC News.
The explosion Saturday night injured 29 people in Manhattan's Chelsea neighborhood.
Multiple senior law enforcement officials told NBC News that the suspicious device discovered Sunday night in Elizabeth looked similar in appearance to the one that exploded Saturday morning in Seaside Park, N.J.
Local resident Romel Johnson, 46, said he was near the scene and heard about 10 to 12 gunshots in Linden, followed by the arrival of more police cars and an ambulance.
The announcement came hours after a backpack that appeared to contain pipe bombs exploded as a police robot examined it near a New Jersey train station. The bag had wires and a pipe protruding from it.
The home was roughly one mile from the site where investigators found five suspicious devices, one of which exploded, outside a train station. There were no injuries or damages, and the Semper Five race was canceled. It wasn't immediately clear if the apartment had any connection to the blast.
Ahmad Khan Rahami in a photo released by the Federal Bureau of Investigation. "I don't believe the City of Elizabeth at this stage - was a target", he said, according to WUNF.
New Jersey Transit trains resumed service along the Northeast Corridor and North Jersey Coast Line, but there are delays. Amtrak trains heading to NY were held in Trenton early Monday.
Train passengers reported being stuck on Amtrak and NJ Transit trains for hours Sunday night, while some trains moved in reverse to let passengers off at other stations. Amtrak said 2,400 passengers were affected.
Later the police released the name of an Afghanistan-born American citizen, Ahmad Khan Rahami, they are seeking for questioning in connection with both the NY and New Jersey incidents.
"I don't have the expertise to determine the blast function of that, but based on the loudness, I think people could have been severely hurt or injured if they were in the vicinity when that went off".
The attacks came as world leaders prepared to gather at the United Nations in NY for the annual General Assembly this week, just days after the 15th anniversary of the September 11 attacks.
"A bomb going off is generically a terrorist activity", said Cuomo, who ordered 1,000 New York State Police and National Guard members deployed across the city.