Sperm are neither male nor female, but they do determine the sex of the baby. Each of the millions of sperm ejaculated, have either an X or a Y chromosome, and there are about the same number of each. The female’s egg has only an X chromosome, and if the Y sperm gets there first, there will be a boy baby.
Similarly if the egg is fertised by an X sperm cell, the two XX’s will result in a girl.