The `True Blood` star - the son of actor Stellan Skarsgard and his first wife My - started his career aged just seven but called time on his screen roles when he was 13 when he started getting media attention, because he "just wanted to be a kid".
Speaking on US TV show `Live with Regis and Kelly`, he said: "My father is an actor and I kind of grew up backstage, but I never wanted to be an actor when I was a kid.
"I did something and when I was 13 it got some attention and I didn`t like being famous. I didn`t want to be recognised, I was insecure, I didn`t like the attention, I didn`t want people knowing who I was; so to hear people on TV, radio talking about who I was and what I like was kind of confusing."
The Swedish actor then joined the Swedish military aged 19, before studying at university in Leeds, North England, before deciding to give acting another try.
Of his decision to return to acting, he added: "I don`t know, I just missed it. I was 20 years old, I`d lived in Leeds and I didn`t know what I wanted to do with my life, and thinking about reasons for quitting, it was nothing to do with being onstage or in front of the camera.
"It was everything the commotion around me, so I thought I should at least give it a last try before I decide this is not what I want to do, so I don`t end up 55 and bitter because I never tried."
Alexander, 34, then enrolled in a theatre course in New York in 1997 and started on the path to stardom.