The Customs House has been home to restaurants and nightclubs but has stood empty for years.