Peel off one spin at a time
Start with the amplitudes as a two-row matrix: the first spin labels the rows and the other spins label the columns. Decompose it. Keep the left factor, a two by two matrix, as the tensor of the first spin. Multiply the singular values into the right factor, then reshape that factor so the second spin joins the rows.
Decompose again, keep the new left factor as the tensor of the second spin, and carry the rest forward. After the last spin, every amplitude reads as a row vector times a chain of matrices times a column vector, one matrix per spin, picked by whether that spin is up or down. Schollwöck builds the decomposition this way in Section 4.1.3 of his review.
Why the matrices stay orthonormal
Each kept left factor has orthonormal columns, because it came out of a singular value decomposition. Summed over the two values of its spin, the tensor times its own conjugate transpose gives the identity. Schollwöck calls such a tensor left-normalized.
This matters in practice. With every tensor to the left of a cut left-normalized, the columns of that cut are an orthonormal basis of the left part. The singular values at that cut are then the Schmidt coefficients of the full state, so the entanglement and the truncation error at any cut can be read off locally, without touching the rest of the chain.
Worked example
Count the numbers for twelve spins
Without truncation, the number of Schmidt terms at a cut cannot exceed the number of patterns on the smaller side. For twelve spins the link sizes can run 2, 4, 8, 16, 32, 64 and back down. Count the entries and an exact matrix product state of a general state stores 10,920 numbers, more than twice the original list. The format alone saves nothing. The saving comes from cutting the links.
Now cap every link at four, the compression Lab default. The link sizes become 1, 2, 4, then eight links of 4, then 4, 2, 1 at the far end. Each tensor holds two matrices, one per spin value, of size left link by right link. Adding them gives 296 numbers in place of 4,096. That is the stored count the Lab reports for the critical chain.
Vidal's form keeps the weights in view
Vidal wrote the same object differently in 2004. Between every pair of site tensors he stores the vector of Schmidt coefficients of that cut. A state of a chain of spins then needs a number of parameters that grows like the chain length times the square of the largest link size, instead of exponentially.
His form makes truncation transparent: the coefficients at every cut sit in the open, so cutting a link down means dropping its smallest coefficients. Chapter six uses this form to evolve a state in time.