Weights only help increase strength when you can increase the amount of weight. "progressive overload" stimulates hypertrophy. Meaning that challenging your muscles to their limit, then progressing past that, makes muscles grow. So working with a single pair of lightweight dumbells isn't going to help muscle development past a very basic level. After that you're basically just burning calories by lifting them.

You can do body weight exercises for strength if you don't have weights.

Like for pushups, start doing them with hands against a wall. See how many you can do. If it's more than maybe 20 at once you can probably try with your hands lower, you might have to get creative with what you lean on but eventually you can use a bed or table, then the floor on knees. Keep a log, to keep track of how many you do so that next time you can shoot for more. I use the 'simple workout log' app. Do three sets, with a rest in between each of perhaps one to two minutes. Once you can do 3 sets of more than ten each, time to go to something lower. Eventually you will be on the ground doing knee pushups, and eventually real pushups.

There are lots of good bodyweight exercises for strength and ways to help progress. Like for pullups there are many methods to get to the point of being able to do one real pullup.

Squats and lunges are good with just bodyweight. For an added challenge and greater muscle development, try one legged pistols. You can find things around the house to hold for added weight in squats.