The UN’s refugee agency, UNHCR, said on Monday it began relocating the more than 2,000 refugees fleeing Rwanda since the end of March.
They have been camped out across the border of northern Burundi.
The agency does not provide aid to the refugees at borders, for security reasons, Catherine-Lune Grayson, the head of UNHCR public relations in the Burundian capital, Bujumbura, told IRIN Monday.
"We encourage people to move inland," she said.
She added that UNHCR was coordinating with NGOs to help the refugees as soon as they reached transit centres farther inland.
The officer in charge of refugees in Burundi's Ministry of Interior, Didace Nzikoruriho, said the refugees were only being moved to temporary camps at Songore in the northeastern province of Muyinga, and at Mishisha in Cankuzo Province.
"The government and its partners are yet to decide on more long-term places to relocate them," he added.
The refugees have not been receiving assistance from UNHCR, while at the border, said Felix Niragira, the governor of Ngozi Province, which has been host to 1,152 refugees, including 480 children, at Marangara Commune.
Other refugees are near the Rwandan border in the commune of Nyamurenza, Ngozi Province. Some 939 people have sought refuge at a school in the rural community of Rushubije in Ntega Commune, Kirundo Province.
There are conflicting reports as to why the refugees fled Rwanda. According to many reports, they fear prosecution by Rwanda's traditional justice system known as "Gacaca".
Other sources report that the refugees are fleeing attacks by Rwanda troops. Some refugees reportedly said fellow villagers in Rwanda were being taken away at night and that they had not returned.