After BJP lost the elections in 2004, the party made him the leader of opposition in Rajya Sabha. He continued in that position till 2009.
In 2009, he was elected to 15th Lok Sabha from Darjeeling Lok Sabha constituency.
Jaswant Singh's relationship with his party soured for the first time when he praised Mohammad Ali Jinnah, the founder of Pakistan, in his book titled “Jinnah - India, Partition, and Independence.”
In that book, Jaswant had praised Jinnah and had put the blame for partition majorly on Congress party. Jaswant's relations with BJP deteriorated so much that the party expelled him on 19 August 2009.
Jaswant was re-inducted into BJP in June 2010 and veteran BJP leader L K Advani played an important role in his comeback to the party.