Scaling Through Added Devices
Gaurav Sen describes proportional device increases and linear scaling, with total batch size rising alongside the training cluster. Codebasics places mixture of experts after transformer components, as a scaling step rather than a separate invention. The written definition agrees: multiple expert networks divide a problem space into homogeneous regions. The practical result is capacity growth through routing and hardware, with the batch acting as the visible lever.






