Got it!
It really is an old rectory, I checked on old OS maps, and Rectory Lane led to it from next to the school.
In recent years it has traded as “Henry’s Table”, “Chimneys” (very appropriate) and now back to “The Old Rectory”.
Used the old A228 a lot in the 70s and early 80s, worked in Snodland with the company fuel account in Larkfield. I then lived in Larkfield for about 3yrs, doubled my money on the house and moved to Grafty Green for the next 32yrs. Work and leisure meant that I drove these roads a lot over the years, we even had a firm’s Xmas “Do” at Chimneys.
Carry on JW😀