I think Batumi, on Georgia's Black Sea coast, is a slightly magical place and I'm sad when, after spending a day there, it's time to leave to head inland.

Just outside town, we come across two young cyclists having a breakfast picnic at the side of the road. They are Radek and Monica, both 26, Poles, who were hitchhiking in Iran when, on a whim, they bought a pair of bikes, cycled to Armenia, then here and will head to Turkey before going home; they have about 500km of their 2,000km odyssey left to go. "We've been camping. Usually, we go to a village and ask to pitch our tent in somebody's garden. We prefer to be near people. We heard there were wolves in the mountains."

And how have the Georgians taken to cycle touring?

"They're fine. To be honest, the Iranians were more puzzled."

Given the general state of Georgian driving, I think they're a little bit crazy, but you can catch up with their progress (in Polish) on their blog, monikaradek.blogspot.com.

The Greater Caucasus in Georgia are justly famous for their fantastic hiking routes. But I want to check out something less well-known: the Borjomi-Kharagauli National Park, set in 85,000 hectares among the valleys and mountains of the Likhi range that runs down the middle of the country.

As we drive, the mountains rise around us until we seem, inexplicably, to be in the middle of the Alps. At an appropriately alpine-style chalet for hikers in the village of Marelisi just north of the park, I meet Jaba, an earnest, young ranger who, I am horrified to discover, wants Nini and me to ride horses up into the hills with him. Fortunately, my smallish steed, Mimino (hawk) seems relaxed about having a huge sack of potatoes perched on his back and ambles up and down the two-mile track at his own relaxed pace (including a couple of, for me, rather fraught river crossings) ignoring any attempts to gee him on. "That isn't a horse!" shrieks Nini behind me. "It's a trollybus!"

After our ride, back at the lodge we bump into a German couple - Christopher, 36, and Simone, 34 - who have been visiting a friend in Tbilisi and have come on their own to the park for a few days' walking. They originally planned to visit last August but, four days before their flight, the war with Russia broke out and their friend temporarily fled to Romania. Did that not give them second thoughts about coming? "This time, we were warned about the opposition demos [in Tbilisi]. The official government advice was: go only if you have good reason," says Christopher. "But," adds Simone, "their advice is too cautious."

And how have they found travelling on their own?

"Basically, it's a great place," says Christopher, but not speaking Russian or Georgian (most Georgians are bilingual) has been a hurdle. However, he says: "After we got on the train here, we discovered it wasn't due to stop at our station. The conductor stopped it just for us - not something that would ever happen in central Europe."

Ultimately, though, he believes Georgia's big problem is that Europeans can find the same experiences closer to home. "There is no reason for westerners to come here. We wouldn't have come if it wasn't for our friend."

Later, I talk to to the park's senior administrator, Levan, a Georgian English literature graduate, and Byron aficionado. He tells me, as we walk by a river through a gladed wood on one of the park's nine trails (the longest takes about five days and goes right over the mountains), that it had 4,000 visitors last year. At first I assume he has made a mistake with his numbers. Surely 40,000, or even 400,000? But, no, for now it remains an undiscovered and tranquil gem.

Next door to the lodge, lives a rural Georgian extended family, who are keen to show us their way of life. After dinner, we head over to meet the matriarch of the household.

Tsitsino, 72, lives with her husband, Varga, daughter-in-law, grandson and his wife and great-grandchildren. We stay for a few glasses of cha-cha, a home-made local spirit made out of grape skins. (I notice, with mild alarm, that the cha-cha is coming out of an old gin bottle but it goes down easily enough.) What, I ask her, is life like for her these days? "Before the revolution, we used to keep the light off because of robbers. Life is better. I used to get 28 lari (about £9) a month pension. Now it's 85 lari."

I ask Tsitsino, who has lived through the most turbulent period in history - through world wars, revolutions, and the collapse of a global superpower - what has really changed the most in, say, the last 20 years?

"What's really changed? My husband and I can't do it any more!"

The next morning, Varga sets up the still in his yard, as chickens scurry around our feet, and shows me how they make the cha-cha. I stick my finger under the flow of warm liquid coming out of the still; it reminds me of saki. "If you did this in Britain, you'd go to prison," I tell him. He shakes his head in disbelief and pours me a glass.

After another cha-cha or two for the road, we're off again, heading to the country's south-eastern border with Azerbaijan, and to what many consider to be Georgia's greatest tourist attraction: the complex of monasteries known as David-Gareja, founded in the desert in the 6th century by St David. The rolling hills give way to semi-desert and soon it's like driving on the moon. There are virtually no signposts and the roads themselves are in terrible shape; often blocked by herds of sheep tended by Azerbaijani shepherds who have wandered over the border. (For the record, I can't imagine doing the journey, a comfortable day-trip from Tbilisi, without hiring a driver and a 4x4).

The Lavra, the picturesque cave city, and working monastery, where David first set up shop is right next to the car park and easy to reach but after looking round the complex, Nini and I have to toil over a hill in the midday heat to the historically important abandoned monasteries hewn from caves in the hillside. It's a very strenuous half-hour slog ("Too many cigarettes," gasps Nini stopping for a rest).

The top of the hill demarcates Georgia's southern border, and we actually cross a few yards into Azerbaijan to get to the caves. But it's worth every gasping breath: at the top we have the staggering views over the Azerbaijan plains to ourselves; with only a lonely vulture floating above us on the thermals for company.

• Tomorrow Maxton heads to the wine-growing region.

• Maxton Walker flew with bmi from London Heathrow to Tbilisi, Georgia. Return flights are from £430.