A quantitative analysis of white matter fibers is based on different physical features (shape, scale, orientation and position) of the fibers, depending on the specific application. Due to the different properties of these features, one usually designs different metrics and spaces to treat them individually. We propose a comprehensive Riemannian framework that allows for a joint analysis of these features in a consistent manner. For each feature combination, we provide a formula for the distance, i.e. quantification of differences between fibers and a formula for geodesics, i.e. optimal deformations of fibers into each other. We illustrate this framework in the context of clustering fiber tracts from the corpus callosum and study the results from different combinations of features.