Michael Sherwood, a vice-chairman at the bank, said the prospect of a British withdrawal was a real worry given Prime Minister David Cameron's plans to hold a referendum on the subject if re-elected in 2015.
"I wouldn't say we are relaxed, we regard it as a very big tail-risk to our business," Sherwood, whose investment bank bases around 6,000 of its 7,000-strong European workforce in London, told the Evening Standard newspaper.
Talk of Britain turning its back on its biggest trading partner intensified in January when Cameron said he would seek a new deal with the EU and then hold a referendum before the end of 2017 to decide if Britain left the 28-nation bloc.
His promise of a plebiscite upset many European allies and prompted warnings from some business leaders that Britain's economy would suffer if it were outside the bloc.
The move went some way to placating rebellious eurosceptics in his ruling Conservative Party, who felt threatened by the rise of the UK Independence Party, which wants to leave the EU.
But Goldman's Sherwood predicted European banks would quickly relocate if Britain did sever its EU ties.
"Forget what we would do. Every European firm would be gone in very short order," he was quoted as saying.
In the event of a British withdrawal, Goldman would have to decide how to divide its staff between London and the rest of Europe, he added.
"Our people want to be in London. It works for us. We would hate to pull it apart," he said.
His colleague Richard Gnodde, who is his co-chief executive at Goldman Sachs International, said London's financial district, known as the City, would continue to operate for decades to come regardless of the outcome of any referendum.
"There was a City before the EU was ever thought of. In or out, there will be a City for the next 100 years," he was reported as saying. "The question is: What kind of City will it be?"
