All things considered, the .308 is difficult to beat. It is not as good a varmint cartridge as the .22s, 24s or 25s, but it is at the top of the heap in long range accuracy. It is less expensive than the big 30s (the .338s are so limited in usefullness they can be discounted) to load and shoot, dies are readily availabel as is good brass. The latter is the downfall of many like the .25-06, high quality brass is essential for long range accuracy. The .25-06 is accurate to a point, but not in the same class at long range as the .308 is. This is the same fault with the .30-06, it cannot compete with the .308 at long range - this has been proven for decades in competition. The reason is inconsistent ignition and a high ES. 110-to 130-grain varmint bullets at 3000-3200 fps is a decent varmint load, and the various 155 to 200-grain MK bullets are proven to excel at 600-1000 yards.
The factory .22s do okay at 500 yards, but they cannot reach 1000 yards accurately- they will require a special fast-twist barrel and high-BC bullets, ditto the 6mms; the 6.5s are very good, but a .260 Remington is plenty for 500, based on actual results in competition. The .338 Lapua has a much better press than it does performance, and who wants to put up with the cost and recoil - it stops being fun after awhile.
Good luck whatever you choose, just remember the old Texas rule - if it isn't fun, then you've wasted your money.
