It does it in all of the browsers that I tested in. That particular shift is caused by your vertical scroll bar appearing on your long pages. Essentially, your long content filled pages end up about 20 pixels narrower than the short ones. That 20 pixels is then taken out of the margin:auto equation which moves the content slightly.
It's normal and expected. There are ways to fix it, requiring some changes in how you're coding the page. The vast majority of centered sites on the internet will shift that way when moving from a page without a vertical scroll bar to a page that needs one. Generally speaking nobody will ever notice it except those of us with Designer OCD.