A primary school teacher has said that he has “turned down Love Island twice” due to the fact that he cares too much about his students and teaching.