Overseas visits from the UK dropped 4% in the three months to January 2012, compared to the previous year.
Latest ONS travel and tourism figures showed spending by UK residents abroad has also fallen, by 2%.
Over a 12-month period, visits abroad by UK residents have risen by 1%, but spending has fallen by 2%.
Meanwhile, international visits to the UK fell 1% in the three-month period, with a 5% drop in spending.
In the past year, however, things are looking more positive, with visits to the UK up by 4% and earnings up by 6%.
Mike Saul, head of hospitality and leisure at Barclays Corporate said: "In this Olympic and Jubilee year, the UK continues to attract increasing numbers of tourists, with visitors to the UK up 4% in the past 12 months.
"As enthusiasm starts to build ahead of this summer’s festivities, early indications from our clients in the hotel industry suggest that bookings are looking strong from international and domestic travellers alike.
"With unemployment still high, however, outbound travel figures are stagnating year on year. All eyes are now on next week’s Budget and just what it has in store for the consumer purse."
by Bev Fearis
