Most of my students are very apprehensive about their ability in the course I teach, remedial math. Typically they have struggled with math for the majority of their lives. I've found that the best way to motivate them is to sincerely compliment them on their work when it's done correctly. For me, smiling and making eye contact while telling them how well they are doing goes a long way. If they make a mistake I gently try to show or explain where they went wrong and don't make a big fuss over the error.