Seven other people remain in hospital after being rescued from the one-year-old Bourbon Dolphin, which is still connected to the Rosebank oilfield by its anchor.
"After an intensive search we must now accept that despite tremendous efforts from all rescue units involved it is extremely unlikely that the five missing crew will be found alive and our sympathies are with the families of the crew at this time," said Neville Davis of Shetland Coastguard.
"We would like to thank all the rescue units and vessels involved in this operation who have made every effort possible and thankfully did bring seven survivors to safety."
A 15-year-old work experience boy and his father are thought to be among the missing.
As part of rescue efforts Royal Navy divers entered the stricken vessel three times on Friday in the hope that the missing crew members had found an air pocket in its hull.
The Bourbon Dolphin capsized approximately 75 nautical miles north-west of the Shetland Islands on Thursday night.
A 194ft anchor-handling tug supply (AHTS) vessel, it is owned by the France-based Bourbon company.
It is described on the firm's website as being "a compact and reliable AHTS for anchor handling, tug and supply services designed to meet the future demands of offshore industry".
The website adds that the vessel "surpasses industry standards with regards to cargo capacities and performance".
It is not yet known why the boat capsized.
In a statement, Bourbon Offshore Norway, which manages the Dolphin vessel, said it would send a plane to Lerwick, Shetland, carrying officials, police and relatives to the site.
A church in Fosnavag, Norway, has been set up as a makeshift information centre for relatives.
Click here to run an energy price comparison, and see if you could be paying less for your gas and electricity.