The musical arrangements and the service of the Levites during the Second Temple period were not a new invention. Instead, they represented a faithful return to a much older, sacred tradition. The customs, the division of shifts, and the specific appointments made during Nehemiah's time were executed precisely according to the original regulations set long before.
The foundations of this musical service were established deep in the past, specifically during the era of David and Asaph. In those ancient days, the exact roster of singers and the methods of their service were firmly decided. Asaph and his descendants were permanently established as the leaders of these choirs [מלבי״ם]. Centuries later, the leaders of the Second Temple simply restored this original format, assigning new singers to follow the exact same order that had been practiced generations earlier [אבן עזרא, ביאור שטיינזלץ].
The influence of David and Asaph extended far beyond administrative organization, as they also shaped the actual spiritual and musical content of the service. During that early founding period, they composed and arranged the specific songs, hymns, and expressions of thanksgiving dedicated to God. Generations later, the Levites continued to sing these exact same compositions, preserving the ancient tradition in its entirety [מצודת דוד, מלבי״ם].