We had them in our yard growing up, I recall regularly playing with the berries for the exact same reason. Funny enough my dad did warn me not to eat it, but based on this post eating the berry itself would have been one of the few ways it’s not toxic. Had no idea about the rest of the plant being so toxic until today.