<ul><li>Concise 1-layer transformers have the ability to evaluate arbitrary functions under certain input representations.</li><li>However, they are incapable of performing this task when the function's inputs and values are assigned to different input positions.</li><li>Concise 2-layer transformers can successfully evaluate functions even with challenging input representations.</li><li>Experimental findings suggest a correlation between what concise transformers can compute and what can be effectively learned.</li></ul>

Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)

Discover more