Import predators and parasites: you can purchase and release predatory insects like ladybugs, or parasitic wasps. This isn't a great solution, because it tends to only work in the short term. (E.g. the released ladybugs quickly decimate your aphid population and then move on because they don't have any further food supply.) It is also only applicable as a control against certain pest populations (e.g. ladybugs for aphids).

If you can find them, plant cultivars that are not susceptible to insect damage. You may be able to find cultivars (either advertised or through your own experience) that are somehow not as attractive to pests as others. It doesn't have to be absolute protection, but in combination with other techniques here it may give you some advantage. Or even plant crops that are not preferred. (E.g. I forget which are preferred so this example may be backwards, but I remember reading that squash beetles prefer, say, winter squash to zucchini. So instead of planning to harvest winter squash, plant more zucchini and use the winter squash as a trap crop.)

Maintain plant health! Healthy plants are less susceptible to damage. Insects will attack weak plants. Sometimes you will notice that one plant in a population is weak and infested with insects, but the others are not infested. When you see this, pull out the weak plant (along with the insects), and destroy it!

Longer term, take steps to reduce their population in your garden from year to year. This doesn't work with all insects. It also won't work as well if you're close to neighbors with infestations that will spread to your garden. But some insects leave eggs in crop debris that will hatch the following season. By cleaning up crop debris from the garden and composting properly (so the pile heats up and destroys the eggs), or disposing of the debris, you can reduce the population that will be around the following season.

If you remember that you're trying to control the insect population instead of eradicate it, you can decide what is an acceptable population level and then take action when it reaches that level. E.g. when cucumber beetles reach a certain level in your zucchini trap crop, you can go out and vacuum the pests out of the zucchini to reduce their population and thereby reduce the risk of them spreading to your cucumbers.

Some insects will have specific, more effective controls that you can apply in addition to the generic strategies above.

"SE Gardening" contains quite a bit of information on this subject, below I've tried to gather up some of that information and post it here in one place for easy reference. All links are to "SE Gardening" posts unless noted otherwise.

Improve your soil, keep your soil in good health

I'm a huge! believer in the benefits (almost magical properties) of compost...

Soap spray is the usual recommendation given for "organic" insecticide. Organic is a relative term of course, but it will generally breakdown quicker than most other insecticides.

Hot pepper (I assume this is a capsaicin spray or powder?) is not going to work with insects (or birds), but should be effective against mammals. Wild pepper plants evolved their high capsaicin content to discourage mammals from eating their fruit. However it was okay for birds to eat the fruit as they did a better job of moving the seed (and I think the seeds are more likely to survive the bird's digestive system - but I could be mis-remembering that).

Are there specific insects that are posing a problem? You could try to encourage natural predators for these insects (ie. larger insectivorous insects, and birds). I've seen things like veg being grown in tropical conditions (Costa Rica) and I don't recall insects being a problem. However, there were so many insects, birds, and bats that were would be plenty of insectivores to keep them in check. Of course with such bio-diversity, it also means you really want to be very restrained when using the more deadly insecticides and herbicides!