Get a leather needle for handsewing, it has 3 sides on the needle tip instead of a rounded one, this makes it easier to punch through the neoprene. Also get a rubber cap or something like that to protect your thumb, else you will have a sore thumb in no time.

Good luck with the hand sewing.