I think that Backlinks do help, but my website had a PR of 3 7 days in from its purchase, and I didnt work on it for months.

But what really sucks about PR's is that they dont affect what position you come in when they search for your website. I tried searching for my website (http://www.xboxgrid.com) on google, so I typed in Xbox Grid, and it came up some 70 odd pages in. But on MSN and Alexa, it came up as #1 (or at least my forums did).. So PR doesnt nessisarily affect your whole site.

But to answer your question, no PR isnt only affected by Backlinking.