A framework is proposed for robust online decision making, allowing multivalued models.The framework introduces convex sets of probability distributions for decision outcomes.Nature can choose distributions from the set in an arbitrary and non-oblivious manner.The framework demonstrates improved regret bounds in robust linear bandits and tabular robust online reinforcement learning.