This paper discusses the problem of social harms in multi-agent reinforcement learning.It proposes market-based mechanisms to measure and control the cost of social harms.The setup captures a wide range of scenarios and allows for different learning strategies.It provides practical applications, such as the Paperclips problem and pollution control.