Why does no one drop into my public Payday 2 game?

Update 8/21/15
Due to the number of views on this question, I thought I should update about the situation, which is that Xbox One players still cannot join each other's games if they are not specifically invited to the lobby.
This is a well-known, ongoing, and much lamented issue with XB1 Payday 2. Since the original post, the behavior of the broken matchmaking has changed with subsequent patches, but the issue is not fixed. Check out the Overkill Twitter feed for more information.
=====================
Original Post
I'm playing on Xbox One, with the Payday 2: Crimewave Edition. My NAT ports are open, i have Fios boosted speed, everything technical is fine according to the XB1 network and multiplayer tests. Other games work perfectly fine.
In payday though, i play a lot with one other person. One of us always goes to crime net, selects an "untaken" job (no gamertag above it), and then invites the other person to the game. Its the easiest way to join each other in payday.
The game we create is always set to public, with drop-in allowed, no reputation limit, and playing with AI. We didnt mind playing a game or two alone at first, but after a dozen games in a row, we noticed no one was dropping into our game.
We have never had another person jump into a game we have created, since the midnight game launch last Tuesday. It doesnt matter who creates the game, if it's just one person or the two of us, any map, any difficulty. No one drops into the game.
Am i doing something wrong? We can separately join a hosted game from crime net perfectly fine, and invite the other person if there's room in the lobby (and if no one else jumps in first), but thats a really annoying procedure.
I want to make a public game, invite a friend, and then have strangers fill the empty slots. Anyone know how?
Note: we had this game on 360 and did NOT have this issue
Pictures about "Why does no one drop into my public Payday 2 game?"



Can you play Payday 2 with strangers?
The game is meant to be played online, so you will find lobbies with strangers and go from there. However your experience will vary. I personally find most matches fun even though I will not play again with the people in the lobby.How many lobby players are in Payday 2?
increase lobby limit to 5 or 6. We are a group of five friends, but there isn't a lot of game that can have more than 5 player that interest us.Why is my Payday 2 not working?
Run Basic Troubleshooting Restart Steam and your PC. Run Payday 2 as administrator (and also tried this from your installation folder) Verify the integrity of game cache (Library > Right-click on Payday 2 > Properties > Local files > Verify) etc.payday 2 glitch fix xbox/ps: if you can't join anyone/no one can join you this is how to fix it.
Sources: Stack Exchange - This article follows the attribution requirements of Stack Exchange and is licensed under CC BY-SA 3.0.
Images: Meruyert Gonullu, Zeeshaan Shabbir, Rahul Pandit, Raphael Brasileiro