The performance of algorithmic decision rules is largely dependent on the quality of training datasets available to them.Biases in these datasets can raise economic and ethical concerns due to the resulting algorithms' disparate treatment of different groups.The paper proposes algorithms for sequentially debiasing the training dataset through adaptive and bounded exploration.The algorithms aim to mitigate the impacts of data biases and achieve more accurate and fairer decisions.