Taroko gorge is excellent, as is the east coast. We got the train to Hualien and hired a car.Didn't go properly into the 3,000 m mountains or Sun Moon Lake, but if I had a bicycle, some hiking boots and a few weeks to spare, it would come near the top of my east Asia outdoor wish list.

I rented a motorbike and rode up the Taroko Gorge Road, went for a hike, came back. Spectacular. (Wasn’t easy renting the motorbike as they didn’t want to rent to foreigners, but the guy running my guest house helped me.)

Taroko is great. I stayed in Hualien and explored Taroko for a couple days. I didn't have my own transport but there are several shuttle buses that travel through the area, taking people around for free including between Hualien and Taroko. I'd get off the shuttle, hike for several hours to some other spot that had a shuttle stop. Don't know about the interior of the island, but the coast of Taiwan is easy to travel around using trains. Tickets are easy to purchase as well. I didn't do this, but a few friends loved taking several days to cycle the east coast of the island.

I got to Taiwan by taking the ferry from Xiamen on the mainland to Kinmen Island, which is Taiwanese controlled but just off the mainland. I spent a couple days cycling around there. On its own, it's pretty boring and suburban but there are some rural bits to explore and you come across old remnants of the heavy militarization of the island, as well as seeing a lot of current Taiwanese soldiers hanging around. Flights to Taipei from the island are frequently, the airport is almost like a bus or train station, you show up and see the schedule of flights and can pretty much get a ticket for the next one to Taipei or a few other Taiwanese cities. A lot of people on the ferry from Xiamen did just that, with air tickets being sold at the ferry terminal when you get off.