I had a problem after changing my firewall policy to drop all ipv6 traffic. After forgetting about it and restarting my computer a couple of days later, I discovered that Xorg wouldn't startup anymore. Sometimes it actually would, after waiting multiple minutes.
The problem with timeouts is that they don't generate errors as long as it isn't timed out yet. So it was actually quite hard to debug, especially because I'm not using ipv6.
After searching for a couple of hours I found a post with someting about network issues which reminded me about the firewall change. Next I added a rule to both ipv4 and ipv6 to accept local traffic, which of course instantly fixed the problem!