They put a stamp on my silence," he said in an indirect reference to the Election Commission which had stopped him from holding a rally in a Beniyabagh, a Muslim dominated area, on security grounds.
Modi also promised to clean up the holy city.
"By 2019, when we celebrate the 150th anniversary of Mahatma Gandhi, we should have cleaned up the country," Modi said on the banks of the river.
BJP became the first non-Congress party since Independence in 1947 to win a comfortable majority on its own in the 545-member Lok Sabha.