I also had this problem. It may have something to do with SKMapView instances sharing some common data (at least I got this problem only when I previously had created another map instance with some annotations).
I'm not sure that it's gonna help but try to clear all annotations for any newly created map before starting to use it.