Gandhi was taken away from the dharna site in Bhatta Parsaul Village in a grey Tata Safari to an undisclosed location.
The Amethi MP smiled to the large battery of media before getting into the vehicle along with AICC General Secretary Digvijay Singh and Firozabad MP Raj Babbar.
"We will continue to support the farmers agitation," Singh said as he was pushed into the vehicle.
Congress promptly attacked Mayawati for Gandhi's arrest with AICC General Secretary Janardan Dwivedi saying UP has witnessed "the cruellest rule of several imperialistic regimes".
"UP government's action in arresting Rahul Gandhi proves that Mayawati is digging her own grave," Dwivedi said.
"If any sense of justice is left in the government, judicial inquiry is the minimum that the state government should order immediately," he said.
Sneaking into Batta Parsaul village, the epicentre of the agitation in Western Uttar Pradesh, in the wee hours riding pillion on a motorcyle, Gandhi sat with the agitating farmers throughout the day in scorching heat to emphathise with their cause and women who lost their men.
"I feel ashamed to call myself Indian after seeing what has happened here. The (state) government here has unleashed atrocities on its own people," he told the farmers.
Even as he sat on a dharna, a war of words erupted, with BJP terming it as a "drama" and wanting to know why the UPA government was still not acting on his earlier promise of bringing Land Acquisition Bill in Parliament.
The Congress, on its part, said it was keen to get on early passage of the bill in Parliament.
Gandhi said he associated with them as they were demanding their "right" and there was "nothing wrong" in it.
"I want to tell you that I am with you till your demands are met. Till your work is not done, the Congress party will not desert you," he said to a thunderous applause from the gathering which shouted slogans hailing him.
He said an official had told him that the agitation was like "naxalism" but he had found that the farmers were peacefully making their demands.
Gandhi demanded a judicial inquiry into the exchange of fire on Saturday last between farmers and police in which two farmers and two policemen were killed.
Earlier, after reaching the village at 4 AM along with party colleague Digvijay Singh, he heard the farmers and had tea with them.