You may need to try several differing brands of 'poster tack' before you find one with the stickiness (or lack of it) that you like. I have had some blue blue tack (don't know the brand) that is horribly sticky, especially after being left for a bit on the model surface...it's quite a chore to get all of it removed sometimes, especially on matte surfaces.
The most recent variety I've tried was the one Mike mentioned above, from Hobby Lobby. It has a "medium" stick to it, and what residue it leaves behind is fairly easy to remove. Another white poster tack that I picked up from (I think) Office Depot a year or two ago has just about ZERO adhesion to it. I now use it more as a medium to plug openings with, as it does what you described (i.e. sticks to fingers but nothing else haha).