For men. 

For men, the biggest factor for a receding hairline is just genetics. It's a process called androgenetic alopecia (commonly referred to as male-pattern baldness), and it happens like this: "High levels of androgen in the body produce a hormone called Dihydrotestosterone, which binds to certain proteins in the body," says Ziering. "It's not necessarily the amount of testosterone or DHT present that ultimately causes baldness; it can be more about the sensitivity or reactivity of the hair follicles, which is genetic. This in turn reduces follicle size, which results in miniaturization, delayed growth, and thinning of the hair. Eventually, the follicle will cease to function and produce new hair growth." 

For what it's worth: Women can experience androgenetic alopecia—or female-pattern baldness—as well. It just looks a little bit different on the scalp (more on that later). Men and women can also suffer from thyroid issues or mineral deficiencies, which can contribute to noticeable hair loss.

For women. 

While women can totally experience hereditary and age-related hair loss, there are some other factors to keep in mind. Usually, "Causes in receding hairlines for women are frontal fibrosing alopecia or traction alopecia," Ziering notes. "Fibrosing alopecia is an autoimmune condition that has the body mistakenly attack the follicles, which results in follicular damage or scarring. It is not very common, and it mostly affects postmenopausal women." The theory here is that postmenopausal women have a low-estrogen environment around the hair follicles, which is thought to trigger the fibrosing alopecia process. But again, it's not so common. 

Traction alopecia, however, is much more commonplace—it's a medical condition that results from long-term damage on the hair. It usually stems from consistent, too-tight hairstyles like "braids, weaves, buns or tight ponytails, especially on chemically treated hair," says Ziering.

Or, women may also experience postpartum hair loss, which can cause the hairline to look a bit meager. It does eventually grow back (usually in six to 12 months), but the wait time can understandably feel a little frustrating.