RelChaNet is a supervised feature selection algorithm that utilizes neuron pruning and regrowth in a dense neural network's input layer.For pruning, a relative change metric is used to measure the impact a feature has on the network.In addition, an extension is proposed to dynamically adapt the size of the input layer during runtime.Experimental results on 13 datasets demonstrate that RelChaNet outperforms existing methods, with a 2% increase in accuracy on the MNIST dataset.