Lustig found that the children in his study experienced anxiety and irritability during the first five days of eliminating sugar and caffeine, though it eventually subsided. One of the biggest concerns is the amount of added sugars in our diets, which are often hidden in foods.