THAT's the dress they called you an ita for Jess?! I love it and I'm not even really into lolita ^^
I like looking at Lolita pics, some of them, but I wouldn't wear it myself. I like some parts of the style, like the wide skirts with the petticoats. You'll need a lot of self confidence to wear this style though, because as you said, most people don't 'get' it. I saw a lolita girl in the city a few weeks ago and she was badmouthed constantly, out loud, by both kids and adults. You need to be quite strong-headed to be able to throw your nose in the air and just walk on and be proud.
The only other place where I've seen lolita and other striking fashion styles, is Dolliverse, the Dutch BJD convention. On that day people seem to go 'all out' to express themselves and indeed, I almost felt uncomfortable in my regular skirt and casual blazer XD