This paper presents a generalization on the tensor spectral $p$-norm and nuclear $p$-norm via arbitrary tensor partitions, a much richer concept than block tensors. We show that the spectral $p$-norm and the nuclear $p$-norm of a tensor can be lower and upper bounded by manipulating the spectral $p$-norms and the nuclear $p$-norms of subtensors in any arbitrary partition of the tensor for $1\le p\le \infty$. Hence, it generalizes and answers affirmatively to the conjecture proposed by Li (SIAM J. Matrix Anal. Appl., 37:1440--1452, 2016) for a tensor partition and $p=2$. We study the relation of the norm of a tensor, the norms of matrix unfoldings of the tensor, and the bounds via the norms of matrix slices of the tensor. Various bounds of tensor norms in the literature are implied by our results. We derive general polynomial-time approximation bounds of the tensor spectral $p$-norm and nuclear $p$-norm.As a practical application, we propose a new tractable surrogate of the tensor nuclear norm via partitions and adopt it in low-rank tensor completion. Numerical experiments show good performance of the new surrogate.