Bruton seems, at first, a typical, quaint English town, with its scattering of antiques shops, posh school playing fields, and narrow streets of old stone cottages. It is vaguely familiar to me, as the nearest station to the old site of the Sunrise music festival (past winner of a Guardian ethical travel award), but I never thought it special.

But there is an unexpected undercurrent in this area of Somerset, of creativity and a hippyish sub-culture, thanks to the proximity to "Glasto" and the other festivals. The big news this year is that the Hauser and Wirth gallery, which has branches in Zurich, London and New York, is to open another, in Bruton, next summer.

Thank goodness there is somewhere fitting to stay. At The Chapel – initially a restaurant, bar, artisan bakery and wine shop, which opened in 2008 – now has eight guest rooms in its super-stylish, white, minimalist space, hung with art lent by the gallery. At The Chapel will also operate the gallery's restaurant, At The Farm, when it opens.

Pushing open the doors, I can practically hear heralds singing hallelujah. The main part of the old listed chapel building, with its high ceilinged white space, is beautifully preserved, but also transformed – into a buzzing restaurant where a happy congregation of families and friends is munching away. We're bustled straight through to a table for lunch as our bags are whisked to our room.

"You couldn't get decent wine, coffee or bread when I moved here," explains Catherine Butler the owner, who seems to have single-handedly fixed the problem. From the bakery's wood-fired oven come lovely pizzas with interesting toppings, such as taleggio, field mushrooms and thyme.

My room, on the top floor with sloped ceilings and exposed rafters, is stylish and white, with art books and a Louise Bourgeois. It's a shame the bathroom is in the middle of the room in its own box, breaking up a great long space, but it's luxurious with a huge rain shower.

We zip out for a muddy walk to an old dovecote on a hill, then for a run up Lusty Hill past Sexey's school and out over the farmland. Lots of old landed families live in the surrounding countryside, says Catherine later, but there's a growing population of bohemian intellectuals too, and she swiftly name-drops a list: Sam Taylor-Wood, Don McCullin, Kevin McCloud, Anna Friel, Phoebe Philo, Alice Temperley, Pearl Lowe. The film director Julien Temple sometimes hosts informal film screenings in At The Chapel's downstairs lounge (there are poetry, music, literary and lecture nights too).

At dinner the menu is the same as lunch so, pizza ticked off, I go for a good pork terrine with an overly chunky piccalilli, flavour-filled seabass with potatoes and fennel – then panna cotta with a pomegranate sauce which isn't. It's fresh, simple fare, not perfect, but right in the light, airy room. In the morning, a bag of warm croissants awaits, hanging on our door handle, and we are also able to make fresh coffee in the room. There's more breakfast to be had downstairs off a menu. , – though more dishes can be ordered downstairs.

I like that this corner of the countryside has a bohemian edge, but there's hint of a celebby, slightly-smug Notting Hill set thing lurking here, I fear. No matter, most guests would be oblivious and just enjoy a laidback country weekend in style, all the better next year when the gallery opens.

• This article was amended on 16 April 2013. The original omitted mention of the Hauser & Wirth gallery in Zurich and the gallery's restaurant at Bruton will be called At the Farm, not At the Farmhouse, as we had it. This has been corrected. It was further amended on 19 April 2013 because it suggested the Bruton gallery would be the fourth branch of Hauser & Wirth. To clarify: it will be the fourth city or town to have a branch, but will be the sixth branch, as there are two in London, two in New York, and one in Zurich.