They do look like aphids. Are there a lot? You may want to try a food-safe insecticide. Here's an article that might help: https://extension.sdstate.edu/are-your-pepper-plants-covered-aphids

Cayenne Diane Thanks for your reply and the link, I greatly appreciate.
Unfortunately, there is a lot. I will have to look for insecticide. :-(

I feel your pain. I have many plants and have dealt with my share of pests. Planting garlic, chives, leeks, catnip, fennel, dill and cilantro will help repel aphids. Marigolds are also known to drive away a number of unwanted pests. Sometimes having companion plants around peppers might help keep those pests away naturally (for future planting).