It was also found that crowded tourist attractions are one of the most common places where people lose the document.
Paul Birkhead of Halifax Travel Insurance said that people should take several photocopies of their passport with them on a break abroad just in case they do lose their passport.
He explained: "This can help to speed the process up and avoid delays on the return leg of your trip, which is not only costly, but can cause serious hassle - especially if you're due back at work.
"Of course if you have a decent travel insurance policy then it is likely that much of the expense will be covered by your insurer."
Previous research by Halifax found that Britons only spend around seven hours away from their hotel when on a break abroad.
Click here to run an energy price comparison, and see if you could be paying less for your gas and electricity.