I switched to a Kyocera KY-42C specifically because you can sideload Whatsapp which I sadly need. The software is a bit crappy but enough if you just want to phone and rarely SMS, Whatsapp or e-mail.

Don't forget about the other stuff usually handled by smartphones, like music (my old Sansa Clip+ still works) or GPS (the one in my car will do).