When floating left or right, do the margins of the floated elements get removed?

Yes and no. Margins and padding still apply if you set them.

Let's say your parent container is 1000px large, and you have two images you want to float left to be side by side.

If both images are 500px, they'll fit into the container, side by side when floated left, no margins/padding set. Easy two layout.

Let's say you have 3 images and want a triple columned layout, no margin/padding set, you would divide 1000px by 3, and you would end up finding that the 3 images need to be set to 333.333px to properly display in a 3 column line.

Now getting to margins/padding and going back to two column layout, you know that both objects are going to take up 500px max, because you have a 1000px container. You need to subtract how much padding/margin you're adding from 500px. If everything adds up correctly, it should all fit side by side into that 1000px container.

tl;dr there shouldn't be any margins regardless of if you float the object until you set a margin onto the object. The reason they join side by side in the parent container is because they're floated in the same container (assuming they're within the width of the parent container)