Behind the endless t-shirt shops, souvenir stands and occasional Starbucks you can find a hidden treasure - the old Key West...

While thousands of visitors flock to Key West each year in search of the fabled land of pirates, smugglers and spongers at the end of the U.S. 1, finding the real Key West can be a bit of an undertaking.

Over the last three decades the last vestiges of old Key West have virtually disappeared from Duval Street, the main drag through Old Town. The places Jimmy Buffett used to frequent in the early 70s are all but gone, replaced by endless t-shirt shops, souvenir stands and occasional Starbucks.

But venture a few blocks off Duval and you can still find those hidden treasures, places that few cruise ship tourists know of and which still capture the quirky history of this once remote part of the world.

Fort East Martello Museum and Gardens

If you haven't been to Key West before, you may want to start your visit at the Fort East Martello Museum and Gardens, near the entrance to the airport. This Civil War fort became obsolete before it was completed, but inside, you'll find plenty of local history and lore.

The various eras of Key West are chronicled here, from the boom years of shrimping and sponging to the arrival of the overseas railroad. You can also learn about some of the real characters that have called Key West home over the years. This includes Robert, the doll that is said to come alive now and then and the reclusive Carl von Cosel, who picked up his bride from the cemetery, spiriting her away in the "dead" of night.

Captain Tony's Saloon

If you still have a thirst for local lore, head to Captain Tony's Saloon. Jimmy Buffett, then an unknown singer-songwriter, started on his road to fame and fortune playing on the stage at the far end of the room. Earnest Hemingway was a regular too, enjoying a cocktail every afternoon back when the bar was known as Sloppy Joe's. Other famous visitors included Presidents John F. Kennedy and Harry Truman, Tennessee Williams, Truman Capote and Robert Frost.

The history of the building is really the history of Key West itself. Built on the original city dump, the bottles tucked into the walls near the pool tables were dug up from the floor. An old well was also uncovered, which contained the body of a woman. No one knows how she got there. The tree on the far side of the bar is the town's original hanging tree. In the 1800s, 17 convicted pirates were hung there. The entire town would make a day of it, heading to the hanging tree with their picnics. Until his passing in 2008, Captain Tony would sit on a stool by the door of the gift shop and tell tall tales of his days as a sea captain, bar owner and one-time mayor of Key West.

The Green Parrot

The oldest bar in town is The Green Parrot, which opened in 1890. It has changed little over the years and often has the best bands in town. Another authentic establishment is the Schooner Wharf at the Historic Maritime Seaport. The original bar was built on a real schooner, the Diamente, but it has since moved ashore and continues to be a real slice of the original Key West where you'll find sea captains, old salts and landlubbers alike.

B.O's Fish Wagon

Not far from the waterfront is another piece of authentic Key West, B.O's Fish Wagon. If you time your visit right, you may just find owner Buddy Owen serving up what's fresh today. Don't be too put off by the building, which looks as if it's ready to fall down. It is made from the odds and ends of the island, a throwback to Key West's eateries before the arrival of the Hard Rock Cafe and other chains. The fresh fish sandwiches are served on equally fresh Cuban bread and B.O.'s also has some of the best conch fritters on the island.

Blue Heaven

Another Key West original is Blue Heaven, made famous by Jimmy Buffett's ode to his favorite restaurant, My Blue Heaven. Tucked under a tropical canopy of trees, the restaurant not only has excellent food, but great ambience. The famed Key West chickens run free here and you'll enjoy the decor, service and the history of one of the most authentic Key West experiences in town.

The Custom House

Once you've had your fill of food and drink, head up to the Custom House near Mallory Square. You can't miss the building. Its red brick facade - adhering to the government's one-size-fits-all design standards of the time - looks a bit out of place in the tropics, though the building was considered the "Jewel of Key West" when it was constructed in 1891. Notice the chimneys along the roofline for the many needless fireplaces incorporated into the building's design.

Key West Historical Graveyard

No visit to Key West is complete without a stop at the Key West Historical Graveyard, which is located on Solares Hill in the northeast corner of the Old Town area. It's estimated that 100,000 people make the cemetery their home these days; three times the number of people who reside in Key West. Plan to spend a couple of hours walking among the graves, which include the gravesites of those who died in the explosion of the U.S.S. Maine in Cuba (Remember the Maine!) and many humorous gravestones, including one that says, "I told you I was sick." The graveyard is open during daylight hours only so plan accordingly and there is a map at the entrance.

Fort Zachary Taylor

If you are a Civil War buff, be sure to visit Fort Zachary Taylor, which is a bit of a walk but well worth it. The fort is currently undergoing restoration but features several of the cannon used to protect the strategic waterways surrounding Key West. Some of these cannon were so powerful that residents had to be warned to leave their windows open on the day of test firings as the concussion would blow them out across the island otherwise.

The fort also boasts the best beach in Key West, a windswept, shaded stretch of sand with great swimming and lounging potential. It is just beyond the fort and has a snack bar, picnic tables and showers.

Sunset at Mallory Square

At the end of the day, swing over to Mallory Square for the traditional sunset celebration. Here you can enjoy music, street entertainment and a cold drink while you watch another fun day in true Key West come to a close.