A moment-based approach is proposed for texture analysis of medical images. The neighborhood of texture pixel is calculated by moments for texture feature extraction. After verified on Brodatz textures, the moment-based texture analysis method is applied on CT liver scan classification and prostate ultrasound segmentation. A support vector machine and a multi-channel active contour model are used in our application. The result shows that our method works well but still have some limitations.