People ask me this all the time and I got tired of writing it over and over again in every email I get about the topic, so here is how to measure a setup and what to expect whether you are building a guitar or just setting it up.
Rule 1. If the frets are not level you are wasting your time. You cannot do a successful setup on a guitar that has unlevel frets.
Generally I like low action but with an acceptable level of buzzing. And ‘acceptable level’ is determined by measurement, sound and feel on a given guitar. I have professional players for customers who don’t like super low action, in fact most of them don’t. Higher action tends to provide better tone.
Always start with the relief. I adjust for very little relief on a good neck, about .008 at the 7th on a Strat type neck. I’ve got to where I just eyeball it holding down the low e at the first fret and the 22nd or last fret. If you are not ready to eyeball then put a capo on the first fret and hold down at the last fret and measure with a feeler gauge. This is a good reason to have the truss rod nut in the headstock of your neck otherwise you will be reluctant to adjust it and your guitar will never be setup properly. This is critical and must be done first with the guitar tuned. I tune to standard when doing this. It also has a significant effect on the action of the guitar. The truss rod nut should be turned in no more than 180 degree increments, then check and repeat until good. If you are afraid of breaking it then perhaps you should not do it to begin with. It’s very simple but it requires some mechanical aptitude to understand when you are exceeding the force needed. Not everyone has these mechanical abilities. I have smart, successful friends who don’t know which end of a wrench to use. It’s just not their thing.
Mahogany necks move more than maple necks so one day you may set the relief and the next day your guitar is buzzing and the guilty factor is the relief has changed because the neck has moved. Keep it in mind. Mahogany is my favorite neck material but it requires more truss rod tweaking than maple necks.
As a general rule of thumb for the average player on an average guitar with good fret level and a 10-16 compound radius, 2/32″ on the low e at the twelfth. 3/64″ at the high e. On a very good guitar 3/64″ on low e and 1/32″ to 3/64″ on high e. The latter is the exception and not many guitars will do it without a lot of tweaking and it’s wasted time for that amount. Also around this level and lower it becomes difficult to get under the strings to bend, the tone drops away and it’s in the area of damping and rattling because of the string excursion. The arc of the strings starts to damp out on the frets and buzzing is noticeable.
On guitars with those vintage 7.25 radius necks the action can get really high if the player wants to play lead and whole step bend past the twelfth on the e and b. Those things are rhythm guitars or chicken pickers but many play blues and such on them and they are going to require higher action than a flatter radius or they will fret out on the bends. You may have to go to 5/64″ on the high e to get the bend you want at the 15th with no fret out. You just have to play it to see and adjust from there.
The rule is, the smaller the fretboard radius, the higher the strings have to be to prevent fret out on bends.
Most everyone complains about low e buzzing. Strat style guitars come in all qualities and some have to go as high as 5/64″ on the low e for acceptable buzzing. It’s just geometry when analyzed. But it can be complex. I’ve had cheapo guitars in the shop that play as good or better than ‘expensive’ name brand guitars and I’ve had junk guitars that play like junk. Old strings buzz less. You may restring your guitar and notice increased buzzing. Play it for a while and it will settle down as the strings get worn in and their excursion decreases. Of course at some point you have to replace the dead strings and the cycle starts over again. Life is a compromise.
If you follow these measurements you should be able to get your guitar playing well. And remember that the measurements are not hard rules. They will get you very close, then tweak until you get it where you want it.