<ul data-eligibleForWebStory="true">Accurate uncertainty estimation is crucial for open-set recognition scenarios.The proposed HolUE method addresses uncertainty through a Bayesian probabilistic model.HolUE considers two sources of ambiguity: gallery uncertainty from overlapping classes and embedding uncertainty.Challenging datasets like IJB-C and VoxBlink were used to test HolUE, showing improved recognition error identification.Existing uncertainty estimation methods based solely on sample quality are outperformed by HolUE.HolUE introduces a holistic uncertainty estimation approach for open-set recognition.The method is designed to handle situations where a probe sample may belong to an unknown identity.Probabilistic embeddings play a role in determining sample quality for uncertainty estimation in open-set recognition.A new open-set recognition protocol for identification of whales and dolphins was introduced alongside HolUE.Bayesian probabilistic modeling forms the basis of the HolUE method for uncertainty estimation.The low variance of probabilistic embeddings may not always indicate low identification error probability in open-set recognition.HolUE performs well in scenarios where embeddings are close to multiple classes, leading to high uncertainty despite high sample quality.IJB-C and VoxBlink datasets were utilized to assess the effectiveness of HolUE.HolUE demonstrates superior recognition error identification compared to competing uncertainty estimation methods.Open-set recognition systems face challenges due to ambiguous gallery classes and embedding uncertainties.HolUE offers an improved approach for handling uncertainty in open-set recognition tasks.