The process of sexual differentiation is central for reproduction of almost all metazoan, and therefore, for maintenance of practically all multicellular organisms. In sex development, we can distinguish two different processes, sex determination, that is the developmental decision that directs the undifferentiated embryo into a sexually dimorphic individual. In mammals, sex determination equals gonadal development. The second process known as sex differentiation takes place once the sex determination decision has been made through factors produced by the gonads that determine the development of the phenotypic sex. Most of the knowledge on the factors involved in sexual development came from animal models and from studies of cases in whom the genetic or the gonadal sex does not match the phenotypical sex, that is, patients affected by disorders of sex development (DSDs). Generally speaking, factors influencing sex determination are transcriptional regulators, whereas factors important for sex differentiation are secreted hormones and their receptors. This review focuses on these factors and whenever possible, references regarding the 'prismatic' clinical cases are given.