If a person is in a minor collision, say the speeds involved are around 30mph, the person wearing their seatbelt will remain firmly in their seat and completely in control of their vehicle. Whereas the person not wearing their seatbelt would get tossed around inside their vehicle, likely losing control in the process, and could potentially harm more people as a result.
So yes, wearing your seatbelt affects more than just yourself in a collision. And as such, the law requiring it is justified.