Those yellow bugs are aphids which are sucking the juices of the milkweed.
Small aphid populations are not serious; they may cause a little distortion of the leaves or buds, but they rarely harm the plant. This is especially true since beneficial insects will be attracted by the aphids and will control them. The most familiar of these predators is the lady bug.
Many insects are dependent on milkweed. It is the sole host for monarch butterflies, but milkweed also provides food and habitat for milkweed bugs, milkweed beetles, and a host of other native creatures. These insects will not destroy their own food supply, but insecticides will kill them all, destroying the important biodiversity and food web that the milkweed supports.
Please avoid using pesticides as they will kill not only the aphids but all the predators that are controlling the aphids naturally. Learn to tolerate some aphids or knock them off the plant with a stream of water if you must.