The actor-and-director was shooting scenes for his new movie `The Town` - about a group of bankrobbers in Boston - when he forgot he was still in costume and walked into a financial depository, terrifying the customers as he was holding a gun.
Ben - whose character dressed as a skeleton for the movie raids - said: "One day we were shooting near a bank and they wanted to keep the bank open for the day so we tried to accommodate them as best we could.
"And we had this scene where I have my mask on and a machine gun and Jeremy Renner is the same and we`re dragging Rebecca Hall into the van.
"I realised that I should start from inside the bank to give myself a place to cut so I went into the bank. I walked in wearing a skeleton mask, holding a machine gun and I have never seen true terror until that moment. Including from the security guard."
However, the 38-year-old actor admitted that he only made things worse when he tried to calm one customer down.
He explained to US TV talk show host Jay Leno: "One lady just screamed so I took my mask off and went `It`s OK, it`s just me` and that made her scream even louder!"
Ben also revealed that hiring criminals to take part in the movie in order to make it more realistic proved difficult on set.
He explained: "We had a lot of felons in the movie which caused problems when we were shooting scenes because they`re supposed to hold guns and stuff but they`re not allowed to.
"So we had to go to the federal judge and their probation officers. It turns out they take those laws very seriously."