How do I turn the auto-kick feature on and off in my team?

Unfortunately, this option is currently unavailable. That said, you can always allow the team members that were auto-kicked back on to your team.