How strict should the filter be?

Move the threshold and watch how many invalid candidates get rejected versus how many valid ones get thrown away.

Threshold0.50
precision-
recall-
F1-

Raise the threshold: fewer false positives, more missed positives. There is no free lunch.